When Visual Privacy Protection Meets Multimodal Large Language Models

Pareto Optimality. We first introduce the definitions for Pareto dominance and Pareto optimality (Miettinen, 1999; Steuer & Choo, 1983) below:

In other words, if a solution is Pareto optimal, it indicates that there are no other solutions that can improve one objective without hurting the other one, i.e., this solution is an optimal trade-off between the two objectives.

Tchebecheff norm. The Tchebycheff norm is a scalarization technique in that minimizes the maximum weighted deviation from an ideal point across objectives:where \(\mathop {\sum }_{i} \lambda _i = 1\), \(i\in \{1,2\}\), \(z_i^*\) is the ideal value of the objective, and \((z_i^*-\varepsilon )\) is designed to be the unachievable ideal point with \(\varepsilon \) being a small positive scalar to ensure numerical stability and prevents the optimization from getting stuck when the values of the objectives approach the ideal values. Here, \((L_i(\theta )-(z_i^*-\varepsilon ))\) measures the distance between the current objective and the ideal value, and Eq. 1 essentially returns the objective that is more distant from its ideal point at the current step.

SPSA. Simultaneous Perturbation Stochastic Approximation (SPSA) (Spall, 1992) is a gradient-free optimization method for black-box optimization scenarios where gradients are unavailable. Specifically, to optimize parameters \(\theta \in \mathbb {R}^p\) according to the feedback of the black-box function f, SPSA estimates the gradient \(\hat{g}_k\) at the k-th optimization step as:where \(c_k>0\) is a positive scalar, and \(\Delta _k \in \mathbb {R}^{p}\) is the random perturbation vector with each of its elements sampled from Bernoulli \(\pm 1\) distribution.

In light of the widespread privacy concerns in the usage of MLLM services, we aim to investigate how to protect users’ privacy information (e.g., face, skin color, etc.) when using these MLLM services. Notably, we address the practical scenario where the MLLM is a “black box” with no access to its internal information (e.g., model structure and parameters). Our goal is to “erase” the privacy information in the user visual data before uploading it to the MLLM service, with minimum degradation to its performance.

To achieve this, we construct the framework with an anonymizer \(f_A\), a privacy reviewer \(f_P\), and a frozen black-box MLLM \(f_{MLLM}\), as illustrated in Fig. 2. The anonymizer transforms the input visual data X into anonymized data \(X_A = f_A(X)\), which is then evaluated to obtain two objectives: (1) the utility objective \(\mathcal {L}_{MLLM}\) from the black-box MLLM (with smaller values indicating better MLLM performance on the utilization task), and (2) the privacy objective \(\mathcal {L}_{Privacy}\) from the privacy reviewer (with smaller values indicating less privacy leakage). We learn an optimal anonymizer \(f_A\) to achieve the following two criteria: the privacy information in the visual data is maximally removed, and the performance of the MLLM is minimally affected. Thus, the overall goal of our problem is:where \(\theta \in \mathbb {R}^{p_A}\) is the learnable parameters in the anonymizer \(f_A\), and \(p_A\) is the number of the learnable parameters. The MLLM is kept frozen. However, achieving this goal can be challenging. First, the anonymizer is required to achieve visual privacy protection while keeping the anonymized data still directly understandable to the frozen MLLM. Yet, generally, stronger privacy-preserving techniques tend to degrade the quality and richness of the original data, which can significantly reduce the MLLM’s utility. Moreover, optimizing with black-box MLLM can be difficult to converge, adding to the challenge of achieving our goal. Below, we first discuss how to effectively find a better trade-off between privacy and MLLM’s utility, and then investigate how to achieve stable and efficient optimization with the black-box MLLM.

In our problem, we aim to optimize the anonymizer \(f_A\) with parameters \(\theta \) to protect visual privacy while keeping good performance of the black-box MLLM. To effectively address this challenging problem, inspired by Pareto optimality (Miettinen, 1999; Steuer & Choo, 1983), we propose to formulate our goal as finding a Pareto optimal solution with the two objectives (i.e., \(L_{Privacy}\) and \(L_{MLLM}\) in Eq. 3). Under this formulation, a Pareto-optimal solution represents a principled trade-off between privacy preservation and MLLM utility, where neither objective can be improved without sacrificing the other. Naively, to find Pareto optimal solutions, we can formulate the overall objective as a weighted sum of the two objectives:where \(\lambda _1 + \lambda _2=1\), and \(\lambda _1\) and \(\lambda _2\) are the weights. While this simplifies the trade-off between these two goals to a single objective, we empirically observe that naively applying the weighted sum objective leads to inferior performance in solving our problem (see results in Tab. 5). This could be due to the unknown characteristics of the black-box MLLM. Specifically, the weighted sum objective is shown to be effective with the assumption that all objectives and the trade-off surface are convex (Geoffrion, 1968). Hence, it struggles to find optimal solutions when the problem space is non-convex, limiting its effectiveness in capturing the optimal trade-off in such a complex scenario as our problem (see Supplementary for more elaboration). As we regard MLLM as a black-box function, there is no guarantee of the convexity of the corresponding objective \(L_{MLLM}\) and the trade-off surface. Moreover, in our problem, we obtain two types of gradients for optimizing the two objectives \(L_{Privacy}\) and \(L_{MLLM}\): (1) for \(L_{Privacy}\), we can obtain the computed gradient from the white-box privacy reviewer, but (2) for \(L_{MLLM}\), we can only derive the estimated gradient from the black-box MLLM. Thus, the two types of gradients exhibit very different characteristics. Such a gap between the two “unbalanced” gradients further adds uncertainty and challenges to the optimization process of our problem. Therefore, applying the weighted sum objective does not effectively find the optimal solution in our task.

Considering this, in our framework, we aim to specifically enable the overall objective to find a Pareto optimal solution with our complex and possibly non-convex problem. Instead of simply linearly combining the two objectives, we propose to incorporate \(L_{MLLM}\) and \(L_{Privacy}\) with the help of the Tchebycheff norm:where \(L_{T_i} \in \{L_{MLLM}, L_{Privacy}\}\). Minimizing this term means that, at each step, we focus on minimizing the objective that is less satisfied (i.e., more distant from its ideal point). This allows the optimization process to alternately optimize the two objectives, gradually pushing the two objectives to reach their minimum in the learning process. It also breaks the limitation of linearly combining the two objectives through a weighted sum, and enable searching for solutions in non-convex objective spaces.

Moreover, to further ensure the Pareto optimality of our solution, we add an augmentation term to formulate the overall objective as augmented Tchebycheff (AT) objective:where \(\sigma \) is a sufficiently small positive scalar. Combining this augmentation term essentially ensures that the solution of Eq. 6 is Pareto optimal as analyzed below:

Note that while we adopt the normalization convention of the weights (\(\lambda _1 + \lambda _2=1\)), Theorem 1 does not require the weights to be normalized. We provide elaboration of the proof of this theorem in Supplementary. Theorem 1 theoretically motivates the use of the AT objective that optimizing the objective in Eq. 6 leads to a Pareto optimal solution of the problem defined in Eq. 3. In our privacy-preserving problem, a Pareto optimal solution represents the best achievable balance between privacy and utility: we cannot improve privacy protection without degrading MLLM performance, and vice versa. This is crucial because any non-Pareto optimal solution would be wasteful: it would unnecessarily sacrifice privacy or utility when both could be improved simultaneously. By focusing on Pareto optimal solutions, the formulation discourages suboptimal solutions where privacy and utility are unnecessarily sacrificed.

To this end, by designing our overall objective as Eq. 6, we can train the anonymizer to achieve an optimal trade-off between the privacy and the black-box MLLM’s utility. Nevertheless, this optimization process can still be challenging due to the difficulty of optimization with the black-box MLLM. Thus, we investigate tackling this challenge in the following.

In this section, we first discuss traditional zeroth-order optimization and analyze its challenges specifically in the context of black-box MLLM. Then, to address these challenges, we propose the critical-history enhanced optimization scheme.

Zeroth-order optimization. To achieve optimization with the black-box function, we first turn to the SPSA method (Spall, 1992), as it can achieve gradient estimation with only two forward evaluations of the black-box function at each step. Specifically, in our framework, to optimize the anonymizer \(f_A\) with learnable parameters \(\theta \) according to the feedback of the black-box MLLM \(L_{MLLM}\), SPSA estimates the gradient \(\hat{g}_k\) at the k-th optimization step as:where \(c_k>0\) is a positive scalar and \(\Delta _k \in \mathbb {R}^{p_A}\). Essentially, at each optimization step, SPSA randomly perturbs the parameters and evaluates how the random changes impact the performance of the black-box MLLM. Based on the feedback (performance) of the black-box MLLM, it then approximates the optimization direction for adjusting the parameters.

Bild vergrößern

Though SPSA can allow gradient estimation, we empirically find that SPSA optimization can be unstable and slow in convergence, as shown in Fig. 3. This can be because, at each optimization step, SPSA assumes that the target gradient of the estimation is a truly arbitrary vector and estimates the gradient by randomly perturbing the parameters. Thus, it inevitably introduces randomness in the search directions and can cause random errors, making the estimation particularly difficult for the high-dimensional problem in our task. Though it is theoretically proved that the random errors can eventually average out after a number of steps and SPSA can converge (Spall, 1992), we find that using SPSA in our problem can even be hard to reach convergence for GPT-4V within two weeks, which can lead to high costs for API calls. This necessitates the design of a more effective scheme to handle optimization with black-box MLLMs in our problem, as follows.

Critical-history enhanced optimization. Motivated by the challenges in optimization with the black-box MLLM, we aim to reduce the random errors in the gradient estimation, to enable faster convergence. Particularly, we find that although we cannot access the internal information of the black-box MLLM, the target gradient of the estimation is not necessarily a totally arbitrary vector. In fact, at each optimization step, the gradient estimation can be seen as an exploration of the landscape of the black-box MLLM objective. Thus, throughout the optimization process, we can actually accumulate knowledge about the black-box MLLM from previous explorations. This accumulated knowledge can provide valuable prior information, which can be used to mitigate the randomness in estimating the target gradient.

To achieve this, as the gradient’s contribution to the optimization is dependent on the gradient norm (Paul et al., 2021; Mcrae et al., 2022), we leverage the L2 norm as the indicator to determine the importance of the estimated gradients. Also, to make sure that the selected gradients are effective, we apply a decay factor to reduce their impact over time. More concretely, we keep a gradient collection set \(C_g\) with size m. At each step k, we obtain the estimated gradient \(\hat{g}_k\) via Eq. 7. Then, we compute the importance score for the current gradient \(\hat{g}_k\), and update the importance scores for each of the previous gradients \(\hat{g}_{t_i}\) in \(C_g\) with a decay factor \(d\in (0,1]\):In this way, we obtain importance scores of all previous gradients in \(C_g\): \(S = [s_{t_1}, s_{t_2},..., s_{t_m}]\). If the importance score of the current gradient \(s_{k}\) is greater than the smallest importance score in S, i.e., \(s_k > s_{t_n}, s_{t_n} = \min S\), we remove the history gradient \(\hat{g}_{t_n}\) and add the current gradient \(\hat{g}_k\) to the collection set \(C_g\). By doing so, we keep a set of critical gradients that contain important and timely information.

Optimization with history gradient. With the collected critical history gradients as prior information, we propose to guide the optimization with the black-box MLLM by improving the gradient direction and adjusting the step size. In particular, at step k, with the collected m critical history gradients in \(C_g\), we can improve gradient directions and compute the enhanced estimated gradient \(\tilde{g}_k\) as:where \(\gamma \in (0,1)\) controls the impact of the history gradients. This can help to obtain more accurate optimization directions: the enhanced estimated gradient \(\tilde{g}_k\) is estimated with the information in m critical history gradients in addition to the current step, rather than relying solely on the current step, leading to a more stable optimization.

Moreover, besides improving the gradient directions, we also adaptively adjust the step size according to the optimization status. Specifically, we propose to assess the optimization status at each step and adjust the step size accordingly. To evaluate this status, we compare the directions between the estimated gradient \(\hat{g}_k\) and the enhanced estimated gradient \(\tilde{g}_k\). Denoting the angle between \(\hat{g}_k\) and \(\tilde{g}_k\) as \(\varphi _k\), we can compute the status indicator \(I_k\) at the k-th step as:where \(b\in (0,2)\) is the factor controlling the magnitude of \(I_k\), and \(I_k\) is within the range of \((-\frac{b}{2}, \frac{b}{2})\). Larger \(I_k\) indicates that \(\hat{g}_k\) and \(\tilde{g}_k\) are more aligned, i.e., the current gradient estimation is more consistent with the critical gradients in previous steps. Conversely, smaller \(I_k\) suggests that the directions of \(\hat{g}_k\) and \(\tilde{g}_k\) diverge, and the optimization direction in the current step may be noisier and deviate from the correct path. In this way, we can use \(I_k\) to reflect the current optimization status. Then, we adaptively adjust the original step size \(a_k\) with the status indicator \(I_k\) as \(a_k' = (1 + I_k) a_k\). By doing so, we can reduce the impact of random errors in noisy estimations, which can further stabilize and improve the overall optimization.

Overall, in the critical-history enhanced optimization, we collect critical history gradients from previous optimization steps, and use them as prior information to improve gradient directions and adjust step sizes. These designs all facilitate to reduce the random errors in the gradient estimation, and achieve a more stable and efficient optimization with the black-box MLLM.

During training, we update the parameters \(\theta \) in the anonymizer \(f_A\) such that we can achieve an optimal trade-off between the utilization performance of MLLM and privacy preservation. Specifically, we formulate this optimization problem as seeking a Pareto optimal solution using the overall objective \(L_{AT}\) in Eq. 6. In each iteration k, we obtain the gradient \(\hat{\nabla }L_{AT}\) with objective \(L_{AT}\):where \(i_{max}=\mathop {\arg \max }_{i}\ \{\lambda _i \big (L_{T_i}(\theta ) - (z_i^* - \varepsilon )\big ) \}\), \(g = \lambda _1(1+I_k)\tilde{g}_k + \lambda _2 h_k\), and \(h_k\) is the gradient computed using the privacy preservation objective \(L_{Privacy}\). Then, we update the parameter \(\theta \) accordingly:We show the detailed algorithm to obtain the estimated gradient \(\tilde{g}_k\) and status indicator \(I_k\) in Supplementary. During testing, we first process the input visual data using the anonymizer \(f_A\) and generate the anonymized data, which is then passed to the MLLM to perform the utilization task. We also evaluate the performance of privacy protection using the privacy reviewer.

First, following the well-established privacy-preserving action recognition benchmark (Dave et al., 2022; Wu et al., 2020; Peng et al., 2023), we evaluate our framework with two settings. (1) We conduct experiments with same-dataset evaluation (HMDB51-VISPR) (Wu et al., 2020; Dave et al., 2022), where Kuehne et al. (2011) is a common video action recognition datasets, and Orekondy et al. (2017) is a privacy attribute dataset. During training, following (Wu et al., 2020; Dave et al., 2022), the utility objective and privacy objective are obtained on HMDB51 and VISPR training sets respectively, and during testing, the action recognition performance and privacy performance are evaluated using PA-HMDB (Wu et al., 2020), which contains both action class and privacy attribute annotations. (2) We also follow the cross-dataset training and evaluation (UCF101-VISPR) (Dave et al., 2022), where  Soomro et al. (2012) is a common action recognition dataset. During training, following (Wu et al., 2020; Dave et al., 2022), the objectives of utility and privacy are obtained on UCF101 and VISPR training sets respectively, and during testing, the action recognition performance is evaluated using UCF101 test set, and the privacy performance is evaluated using VISPR test set (Wu et al., 2020; Dave et al., 2022).

Moreover, we also evaluate our framework with the VQA dataset OK-VQA (Marino et al., 2019), which has been commonly used to evaluate MLLMs (Huang & Zhang, 2024; Liu et al., 2023a; Chen et al., 2023). We also follow the two evaluation settings (i.e., same-dataset and cross-dataset) (Dave et al., 2022) to train and evaluate our framework on VQA task. For all experiments, we follow the definition of privacy (i.e., privacy attributes) in previous works (Dave et al., 2022; Peng et al., 2023) (see more details for the benchmarks in Supplementary).

Following (Wu et al., 2018, 2020; Peng et al., 2023), we evaluate privacy preservation performance using privacy attribute classifier (i.e., the privacy reviewer) with class-wise mean average precision (cMAP). We report the Top-1 accuracy for action recognition following (Wu et al., 2018, 2020; Peng et al., 2023), and evaluate VQA performance using VQA accuracy score following (Antol et al., 2015; Marino et al., 2019) (see Supplementary for more details about the metrics).

We construct the anonymizer \(f_A\) based on Ronneberger et al. (2015), which has shown its effectiveness in previous privacy-preserving works (Wu et al., 2020; Dave et al., 2022; Peng et al., 2023), and follow (Wu et al., 2020; Dave et al., 2022) to adopt ResNet-50 (He et al., 2016) as the privacy reviewer \(f_P\). We follow the training scheme in Wu et al. (2020, 2018); Peng et al. (2023) for \(f_P\). To evaluate our framework, we conduct the following two groups of experiments: (1) As it can be costly and inconvenient to conduct experiments on commercial cloud MLLMs such as GPT-4V, we first adopt popular MLLMs Video-LLaVA Lin et al. (2023) (for video action recognition) and Liu et al. (2023a) (for image VQA). We deploy Video-LLaVA and LLaVA locally and regard them as black-box models to conduct comprehensive experiments. (2) Then, to further explore the real-world scenarios, we also evaluate our framework with commercial cloud MLLM service GPT-4V (Achiam et al., 2023). For all experiments, we follow standard SPSA (Spall, 1992) to assign \(a_k\) and \(c_k\). Our main experiments are conducted on an RTX 3090 GPU. We show the details regarding \(L_{MLLM}\), \(L_{Privacy}\), \(f_A\), \(f_P\), and more details about the implementation and hyperparameters in Supplementary.

Following (Wu et al., 2020; Dave et al., 2022), we evaluate our framework and compare it with Downsample and Obfuscation-based methods (Dave et al., 2022) that use heuristic rules to protect privacy information, and previous privacy-preserving methods (Wu et al., 2020; Dave et al., 2022; Kumawat & Nagahara, 2022; Peng et al., 2023; Li et al., 2023; Ilic et al., 2024). Specifically, to enable (Wu et al., 2020; Dave et al., 2022; Kumawat & Nagahara, 2022; Peng et al., 2023; Li et al., 2023) to optimize with the black-box MLLM, we follow (Liu et al., 2016) and consider the action recognition model in the each corresponding framework as the surrogate model for the black-box MLLM. The surrogate model is trained on action recognition task with cross-entropy loss. More discussion about the methods and implementation details are in Supplementary. We report the results using Video-LLaVA and GPT-4V in Tab. 1 and Tab. 2 respectively. We observe that the compared methods typically fail to achieve a robust privacy-utility balance in the black-box MLLM setting. Specifically, methods such as obfuscation-based baselines tend to result in substantial drops in MLLM performance. This can be because these methods suppress privacy attributes by degrading visual content, which can also remove task-relevant information and thus degrade MLLM performance. On the other hand, methods that require white-box access to the utility model often struggle to achieve good privacy-utility trade-off, which is possibly because these methods fail to learn meaningful anonymization strategy tailored to the black-box MLLM, as they do not directly support optimization with black-box MLLM. In contrast, our method achieves top performance on the utilization task while having better privacy preservation performance. Overall, our proposed framework achieves the best trade-off between privacy and utility in experiments with both Video-LLaVA and GPT-4V.

To better evaluate our framework, we also conduct experiments with the VQA task, which is commonly used to evaluate the performance of MLLMs (Liu et al., 2023a; Chen et al., 2023). Specifically, we adopt the subcategories of OK-VQA with images that are most critical to privacy leakage, i.e., People and Everyday Life and Sports and Recreation. We annotate the VQA testing images with the same privacy attributes following (Dave et al., 2022) and evaluate our framework following cross-dataset training and same-dataset evaluation setting (Dave et al., 2022; Wu et al., 2020). Following (Dave et al., 2022; Wu et al., 2020), the objectives of utility and privacy are obtained with OK-VQA and VISPR training samples respectively, and the performance of VQA and privacy is evaluated using the annotated OK-VQA testing samples. Also, following the cross-dataset training and evaluation setting (Dave et al., 2022; Wu et al., 2020), we evaluate our framework using VISPR testing samples. We compare our framework with methods introduced in Sec. 4.3. Specifically, for Wu et al. (2020); Dave et al. (2022); Kumawat and Nagahara (2022); Peng et al. (2023); Li et al. (2023), we adopt VQA model (Gardères et al., 2020) pre-trained on OK-VQA as the surrogate model. More details about the annotation and comparison methods are in Supplementary. We report our results using LLaVA and GPT-4V in Tab. 3 and Tab. 4 respectively. As shown, our method can achieve the best trade-off between the VQA accuracy and privacy protection, showing its effectiveness.

We evaluate the designs of our method on UCF101-VISPR following (Dave et al., 2022; Wu et al., 2018). More ablation studies, visualizations, and further analysis are provided in Supplementary.

Impact of the augmented Tchebycheff objective \(L_{AT}\). In our framework, we design the overall objective as augmented Tchebycheff to achieve optimal trade-off between privacy and utility. We evaluate this design and compare with the following variants: w/ weighted sum variante that only uses Eq. 4 as the objective, and w/ Tchebycheff norm variant that only uses Eq. 5 as the objective. We report the best result for the variants. As shown in Tab. 5, using weighted sum objective is not effective to solve our problem, and using augmented Tchebycheff objective can achieve the best trade-off, showing the efficacy of our design.

Comparisons of zeroth-order optimization methods. We propose the critical-history enhanced optimization to improve the optimization with the black-box MLLM. We compare this design with other zeroth-order optimization methods. We first compare with the original SPSA and its recent variant SPSA-GC (Oh et al., 2023). As shown in Fig. 4, our critical-history enhanced optimization achieves much faster and more stable optimization, which is more suitable for our problem. We also compare other common methods for optimization with the black-box function, including evolution strategies (ES) Hansen et al. (2003), and reinforcement learning (RL) Watkins and Dayan (1992), and report the number of MLLM calls required per sample during training. As shown in Tab. 6, our proposed method achieves the best results while significantly reducing the number of API calls.

Bild vergrößern

Impact of the critical-history enhanced optimization. To evaluate the design of the critical-history enhanced optimization, we conduct experiments with the following variants: (1) w/o history gradient: we discard history gradients and do not adjust gradient direction and step size. (2) w/o critical gradient collection: we use the latest m history gradients without selection to adjust gradient directions and step sizes. (3) w/o gradient direction adjustment: we do not adjust gradient directions and only adjust step sizes using the critical gradients. (4) w/o step size adjustment: we do not adjust step sizes and only adjust gradient directions using the critical gradients. As shown in Tab. 7, our proposed method show the best performances on both action recognition and privacy protection, demonstrating the effectiveness of our design.

Qualitative results. We show visualization results of the video frames before and after our anonymizer \(f_A\) for the action recognition task in Fig. 5. We also provide the visualization results for the VQA task in Fig. 6. We show more anonymized images generated by our method on VISPR testing set in Fig. 7, which contains annotations of privacy information such as nudity and credit card. As shown, our anonymizer can effectively remove the privacy information within the visual data, while the MLLM can still give correct predictions with the anonymized data.

Bild vergrößern

Bild vergrößern

Bild vergrößern

Discussion on scenarios with constrained budget on API usage. Our framework is designed with API efficiency in mind from the outset. In particular, the proposed critical-history enhanced optimization substantially reduces the number of MLLM calls required for convergence, while consistently achieving better privacy–utility trade-offs than existing zeroth-order alternatives (Tab. 6). Beyond this inherent efficiency, the framework also naturally supports additional strategies to further adapt to budget-constrained scenarios. Specifically, in low-budget scenario, increasing the size of the history gradient collection set m can allow greater reuse of informative past gradients, thereby reducing the need for new MLLM queries. Second, we can employ adaptive scheduling to allocate the API budget non-uniformly across samples, prioritizing those with higher uncertainty or slower convergence rates, thereby avoiding unnecessary queries on already well-optimized samples. Finally, early stopping criteria based on the stabilization of the optimization trajectory can be applied to terminate the training once marginal performance gains diminish. With these strategies, our method can further reduce API calls in constrained budget scenario, while still achieving robust performance.