Spaces:
Running
Running
Update index.html
Browse files- index.html +204 -188
index.html
CHANGED
|
@@ -3,11 +3,10 @@
|
|
| 3 |
<head>
|
| 4 |
<meta charset="utf-8">
|
| 5 |
<meta name="description"
|
| 6 |
-
content="Demo Page of
|
| 7 |
-
<meta name="keywords" content="
|
| 8 |
<meta name="viewport" content="width=device-width, initial-scale=1">
|
| 9 |
-
<title>
|
| 10 |
-
Adversarial Perturbation using Generative Models</title>
|
| 11 |
|
| 12 |
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
|
| 13 |
rel="stylesheet">
|
|
@@ -82,8 +81,7 @@
|
|
| 82 |
<div class="container is-max-desktop">
|
| 83 |
<div class="columns is-centered">
|
| 84 |
<div class="column has-text-centered">
|
| 85 |
-
<h1 class="title is-1 publication-title">
|
| 86 |
-
Adversarial Perturbation using Generative Models</h1>
|
| 87 |
<div class="is-size-5 publication-authors">
|
| 88 |
<span class="author-block">
|
| 89 |
<a href="#" target="_blank">ZAITANG LI</a><sup>1</sup>,</span>
|
|
@@ -176,7 +174,7 @@
|
|
| 176 |
<h2 class="title is-3">Abstract</h2>
|
| 177 |
<div class="content has-text-justified">
|
| 178 |
<p>
|
| 179 |
-
|
| 180 |
</p>
|
| 181 |
</div>
|
| 182 |
|
|
@@ -199,12 +197,8 @@
|
|
| 199 |
<h2 class="title is-3">Method Overview of GREAT Score</h2>
|
| 200 |
<div class="columns is-centered">
|
| 201 |
<div class="column container-centered">
|
| 202 |
-
<img src="./static/images/
|
| 203 |
-
<p><strong>
|
| 204 |
-
(1) Data Generation: We use a generative model to create synthetic samples.
|
| 205 |
-
(2) Local Robustness Evaluation: For each generated sample, we calculate a local robustness score based on the classifier's confidence.
|
| 206 |
-
(3) Global Robustness Estimation: We aggregate the local scores to estimate the overall robustness of the classifier.
|
| 207 |
-
This method provides a certified lower bound on the true global robustness without requiring access to the original dataset or exhaustive adversarial attacks.</p>
|
| 208 |
</div>
|
| 209 |
</div>
|
| 210 |
</div>
|
|
@@ -216,14 +210,13 @@
|
|
| 216 |
|
| 217 |
<!-- Robustness Certificate Definition -->
|
| 218 |
<section class="section">
|
| 219 |
-
|
| 220 |
<div class="container is-max-desktop">
|
| 221 |
-
<h2 class="title is-3">
|
| 222 |
|
| 223 |
<div class="columns is-centered">
|
| 224 |
<div class="column container formula">
|
| 225 |
<p>
|
| 226 |
-
|
| 227 |
</p>
|
| 228 |
</div>
|
| 229 |
</div>
|
|
@@ -232,28 +225,25 @@
|
|
| 232 |
<div class="column container-centered">
|
| 233 |
<div id="adaptive-loss-formula" class="container">
|
| 234 |
<div id="adaptive-loss-formula-list" class="row align-items-center formula-list">
|
| 235 |
-
<a href=".
|
| 236 |
-
<a href=".
|
| 237 |
-
<a href=".local-robustness-score">Local Robustness Score</a>
|
| 238 |
<div style="clear: both"></div>
|
| 239 |
</div>
|
| 240 |
<div class="row align-items-center adaptive-loss-formula-content">
|
| 241 |
-
<span class="formula
|
| 242 |
-
$$
|
| 243 |
-
\displaystyle
|
| 244 |
-
\Omega(f) = \mathbb{E}_{x\sim P}[\Delta_{min}(x)]= \int_{x \sim P} \Delta_{\min}(x) p(x)dx
|
| 245 |
-
$$
|
| 246 |
-
</span>
|
| 247 |
-
<span class="formula global-robustness-estimate formula-content" style="display: none;">
|
| 248 |
$$
|
| 249 |
-
\
|
| 250 |
-
|
|
|
|
|
|
|
| 251 |
$$
|
| 252 |
</span>
|
| 253 |
-
<span class="formula
|
| 254 |
$$
|
| 255 |
-
\
|
| 256 |
-
|
|
|
|
|
|
|
| 257 |
$$
|
| 258 |
</span>
|
| 259 |
</div>
|
|
@@ -263,227 +253,226 @@
|
|
| 263 |
|
| 264 |
<div class="columns is-centered">
|
| 265 |
<div class="column container adaptive-loss-formula-content">
|
| 266 |
-
<p class="formula
|
| 267 |
-
where
|
| 268 |
-
</p>
|
| 269 |
-
<p class="formula global-robustness-estimate formula-content" style="display: none">
|
| 270 |
-
where g(x) is a local robustness statistic, and this estimate is used when the exact probability density function of P and local minimal perturbations are unknown.
|
| 271 |
</p>
|
| 272 |
-
<p class="formula
|
| 273 |
-
where
|
| 274 |
</p>
|
| 275 |
</div>
|
| 276 |
</div>
|
| 277 |
-
|
| 278 |
-
|
| 279 |
-
|
| 280 |
-
</section>
|
| 281 |
|
| 282 |
|
| 283 |
|
| 284 |
<!-- Results -->
|
| 285 |
<section class="section">
|
| 286 |
<div class="container is-max-desktop">
|
| 287 |
-
<h2 class="title is-3">
|
| 288 |
<div class="columns is-centered">
|
| 289 |
<div class="column container-centered">
|
| 290 |
<table class="tg" border="1" style="width:100%;">
|
| 291 |
-
<caption><strong>Table 1.</strong>
|
| 292 |
<thead>
|
| 293 |
<tr>
|
| 294 |
-
<th class="tg-amwm"
|
| 295 |
-
<th class="tg-baqh">
|
| 296 |
-
<th class="tg-baqh">
|
| 297 |
-
<th class="tg-baqh">
|
| 298 |
-
<th class="tg-baqh">Calibrated GREAT Score</th>
|
| 299 |
-
<th class="tg-baqh">CW Distortion</th>
|
| 300 |
-
</tr>
|
| 301 |
-
</thead>
|
| 302 |
-
<tbody>
|
| 303 |
-
<tr>
|
| 304 |
-
<td class="tg-baqh">Rebuffi_extra</td>
|
| 305 |
-
<td class="tg-baqh">82.32</td>
|
| 306 |
-
<td class="tg-baqh">87.20</td>
|
| 307 |
-
<td class="tg-baqh">0.507</td>
|
| 308 |
-
<td class="tg-baqh">1.216</td>
|
| 309 |
-
<td class="tg-baqh">1.859</td>
|
| 310 |
-
</tr>
|
| 311 |
-
<tr>
|
| 312 |
-
<td class="tg-baqh">Gowal_extra</td>
|
| 313 |
-
<td class="tg-baqh">80.53</td>
|
| 314 |
-
<td class="tg-baqh">85.60</td>
|
| 315 |
-
<td class="tg-baqh">0.534</td>
|
| 316 |
-
<td class="tg-baqh">1.213</td>
|
| 317 |
-
<td class="tg-baqh">1.324</td>
|
| 318 |
-
</tr>
|
| 319 |
-
<tr>
|
| 320 |
-
<td class="tg-baqh">Rebuffi_70_ddpm</td>
|
| 321 |
-
<td class="tg-baqh">80.42</td>
|
| 322 |
-
<td class="tg-baqh">90.60</td>
|
| 323 |
-
<td class="tg-baqh">0.451</td>
|
| 324 |
-
<td class="tg-baqh">1.208</td>
|
| 325 |
-
<td class="tg-baqh">1.943</td>
|
| 326 |
-
</tr>
|
| 327 |
-
<tr>
|
| 328 |
-
<td class="tg-baqh">Rebuffi_28_ddpm</td>
|
| 329 |
-
<td class="tg-baqh">78.80</td>
|
| 330 |
-
<td class="tg-baqh">90.00</td>
|
| 331 |
-
<td class="tg-baqh">0.424</td>
|
| 332 |
-
<td class="tg-baqh">1.214</td>
|
| 333 |
-
<td class="tg-baqh">1.796</td>
|
| 334 |
-
</tr>
|
| 335 |
-
<tr>
|
| 336 |
-
<td class="tg-baqh">Augustin_WRN_extra</td>
|
| 337 |
-
<td class="tg-baqh">78.79</td>
|
| 338 |
-
<td class="tg-baqh">86.20</td>
|
| 339 |
-
<td class="tg-baqh">0.525</td>
|
| 340 |
-
<td class="tg-baqh">1.206</td>
|
| 341 |
-
<td class="tg-baqh">1.340</td>
|
| 342 |
</tr>
|
| 343 |
<tr>
|
| 344 |
-
<
|
| 345 |
-
<
|
| 346 |
-
<
|
| 347 |
-
<
|
| 348 |
-
<
|
| 349 |
-
<
|
| 350 |
-
|
| 351 |
-
<tr>
|
| 352 |
-
<td class="tg-baqh">Augustin_WRN</td>
|
| 353 |
-
<td class="tg-baqh">76.25</td>
|
| 354 |
-
<td class="tg-baqh">86.40</td>
|
| 355 |
-
<td class="tg-baqh">0.583</td>
|
| 356 |
-
<td class="tg-baqh">1.206</td>
|
| 357 |
-
<td class="tg-baqh">1.332</td>
|
| 358 |
-
</tr>
|
| 359 |
-
<tr>
|
| 360 |
-
<td class="tg-baqh">Rade</td>
|
| 361 |
-
<td class="tg-baqh">76.15</td>
|
| 362 |
-
<td class="tg-baqh">86.60</td>
|
| 363 |
-
<td class="tg-baqh">0.413</td>
|
| 364 |
-
<td class="tg-baqh">1.200</td>
|
| 365 |
-
<td class="tg-baqh">1.486</td>
|
| 366 |
</tr>
|
|
|
|
|
|
|
| 367 |
<tr>
|
| 368 |
-
<td class="tg-baqh">
|
| 369 |
-
<td class="tg-baqh">
|
| 370 |
-
<td class="tg-baqh">
|
| 371 |
-
<td class="tg-baqh">0.
|
| 372 |
-
<td class="tg-baqh">
|
| 373 |
-
<td class="tg-baqh">
|
|
|
|
| 374 |
</tr>
|
| 375 |
<tr>
|
| 376 |
-
<td class="tg-baqh">
|
| 377 |
-
<td class="tg-baqh">
|
| 378 |
-
<td class="tg-baqh">
|
| 379 |
-
<td class="tg-baqh">0.
|
| 380 |
-
<td class="tg-baqh">
|
| 381 |
-
<td class="tg-baqh">
|
|
|
|
| 382 |
</tr>
|
| 383 |
<tr>
|
| 384 |
-
<td class="tg-baqh">
|
| 385 |
-
<td class="tg-baqh">
|
| 386 |
-
<td class="tg-baqh">
|
| 387 |
-
<td class="tg-baqh">0.
|
| 388 |
-
<td class="tg-baqh">
|
| 389 |
-
<td class="tg-baqh">
|
|
|
|
| 390 |
</tr>
|
| 391 |
<tr>
|
| 392 |
-
<td class="tg-baqh">
|
| 393 |
-
<td class="tg-baqh">
|
| 394 |
-
<td class="tg-baqh">
|
| 395 |
-
<td class="tg-baqh">0.
|
| 396 |
-
<td class="tg-baqh">
|
| 397 |
-
<td class="tg-baqh">
|
|
|
|
| 398 |
</tr>
|
| 399 |
<tr>
|
| 400 |
-
<td class="tg-baqh">
|
| 401 |
-
<td class="tg-baqh">
|
| 402 |
-
<td class="tg-baqh">
|
| 403 |
-
<td class="tg-baqh">0.
|
| 404 |
-
<td class="tg-baqh">
|
| 405 |
-
<td class="tg-baqh">
|
|
|
|
| 406 |
</tr>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 407 |
<tr>
|
| 408 |
-
<
|
| 409 |
-
<
|
| 410 |
-
<
|
| 411 |
-
<td class="tg-baqh">0.160</td>
|
| 412 |
-
<td class="tg-baqh">1.020</td>
|
| 413 |
-
<td class="tg-baqh">1.084</td>
|
| 414 |
</tr>
|
|
|
|
|
|
|
| 415 |
<tr>
|
| 416 |
-
<td class="tg-baqh">
|
| 417 |
-
<td class="tg-baqh">
|
| 418 |
-
<td class="tg-baqh">
|
| 419 |
-
<td class="tg-baqh">0.152</td>
|
| 420 |
-
<td class="tg-baqh">1.040</td>
|
| 421 |
-
<td class="tg-baqh">1.097</td>
|
| 422 |
</tr>
|
| 423 |
<tr>
|
| 424 |
-
<td class="tg-baqh">
|
| 425 |
-
<td class="tg-baqh">
|
| 426 |
-
<td class="tg-baqh">
|
| 427 |
-
<td class="tg-baqh">0.275</td>
|
| 428 |
-
<td class="tg-baqh">1.101</td>
|
| 429 |
-
<td class="tg-baqh">1.165</td>
|
| 430 |
</tr>
|
| 431 |
<tr>
|
| 432 |
-
<td class="tg-baqh">
|
| 433 |
-
<td class="tg-baqh">
|
| 434 |
-
<td class="tg-baqh">
|
| 435 |
-
<td class="tg-baqh">0.112</td>
|
| 436 |
-
<td class="tg-baqh">0.909</td>
|
| 437 |
-
<td class="tg-baqh">1.095</td>
|
| 438 |
</tr>
|
| 439 |
</tbody>
|
| 440 |
-
|
| 441 |
</div>
|
| 442 |
</div>
|
| 443 |
</div>
|
| 444 |
</section>
|
| 445 |
-
<!-- Results -->
|
| 446 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 447 |
<!-- Model Ranking Comparison Section -->
|
| 448 |
<section class="section">
|
| 449 |
<div class="container is-max-desktop">
|
| 450 |
-
<h2 class="title is-3">Model
|
| 451 |
<div class="columns is-centered">
|
| 452 |
<div class="column is-full-width">
|
| 453 |
<div class="content has-text-justified">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 454 |
<table class="table is-bordered is-striped is-narrow is-hoverable is-fullwidth">
|
| 455 |
-
<caption><strong>Table
|
| 456 |
<thead>
|
| 457 |
<tr>
|
| 458 |
<th></th>
|
| 459 |
-
<th>
|
| 460 |
-
<th>
|
|
|
|
|
|
|
|
|
|
| 461 |
</tr>
|
| 462 |
</thead>
|
| 463 |
<tbody>
|
| 464 |
<tr>
|
| 465 |
-
<td>
|
| 466 |
-
<td>
|
| 467 |
-
<td>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 468 |
</tr>
|
| 469 |
<tr>
|
| 470 |
-
<td>
|
| 471 |
-
<td>
|
| 472 |
-
<td>
|
|
|
|
|
|
|
|
|
|
| 473 |
</tr>
|
| 474 |
<tr>
|
| 475 |
-
<td>
|
| 476 |
-
<td>
|
| 477 |
-
<td>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 478 |
</tr>
|
| 479 |
</tbody>
|
| 480 |
</table>
|
| 481 |
-
|
| 482 |
<p>
|
| 483 |
-
|
| 484 |
-
|
| 485 |
-
|
| 486 |
-
|
|
|
|
| 487 |
</p>
|
| 488 |
</div>
|
| 489 |
</div>
|
|
@@ -513,6 +502,33 @@
|
|
| 513 |
</section>
|
| 514 |
<!-- GREAT Score vs CW Attack Comparison Section -->
|
| 515 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 516 |
|
| 517 |
|
| 518 |
|
|
|
|
| 3 |
<head>
|
| 4 |
<meta charset="utf-8">
|
| 5 |
<meta name="description"
|
| 6 |
+
content="Demo Page of Retention Score AAAI 2025.">
|
| 7 |
+
<meta name="keywords" content="Retention Score, Adversarial robustness, Generative models, Vision-Language Models">
|
| 8 |
<meta name="viewport" content="width=device-width, initial-scale=1">
|
| 9 |
+
<title>Retention Score: Quantifying Jailbreak Risks for Vision Language Models</title>
|
|
|
|
| 10 |
|
| 11 |
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
|
| 12 |
rel="stylesheet">
|
|
|
|
| 81 |
<div class="container is-max-desktop">
|
| 82 |
<div class="columns is-centered">
|
| 83 |
<div class="column has-text-centered">
|
| 84 |
+
<h1 class="title is-1 publication-title">Retention Score: Quantifying Jailbreak Risks for Vision Language Models</h1>
|
|
|
|
| 85 |
<div class="is-size-5 publication-authors">
|
| 86 |
<span class="author-block">
|
| 87 |
<a href="#" target="_blank">ZAITANG LI</a><sup>1</sup>,</span>
|
|
|
|
| 174 |
<h2 class="title is-3">Abstract</h2>
|
| 175 |
<div class="content has-text-justified">
|
| 176 |
<p>
|
| 177 |
+
The emergence of Vision-Language Models (VLMs) is significant advancement in integrating computer vision with Large Language Models (LLMs) to enhance multi-modal machine learning capabilities. However, this progress has made VLMs vulnerable to advanced adversarial attacks, raising concerns about reliability. Objective of this paper is to assess resilience of VLMs against jailbreak attacks that can compromise model safety compliance and result in harmful outputs. To evaluate VLM's ability to maintain robustness against adversarial input perturbations, we propose novel metric called \textbf{Retention Score}. Retention Score is multi-modal evaluation metric that includes Retention-I and Retention-T scores for quantifying jailbreak risks in visual and textual components of VLMs. Our process involves generating synthetic image-text pairs using conditional diffusion model. These pairs are then predicted for toxicity score by VLM alongside toxicity judgment classifier. By calculating margin in toxicity scores, we can quantify robustness of VLM in attack-agnostic manner. Our work has four main contributions. First, we prove that Retention Score can serve as certified robustness metric. Second, we demonstrate that most VLMs with visual components are less robust against jailbreak attacks than corresponding plain VLMs. Additionally, we evaluate black-box VLM APIs and find that security settings in Google Gemini significantly affect score and robustness. Moreover, robustness of GPT4V is similar to medium settings of Gemini. Finally, our approach offers time-efficient alternative to existing adversarial attack methods and provides consistent model robustness rankings when evaluated on VLMs including MiniGPT-4, InstructBLIP, and LLaVA.
|
| 178 |
</p>
|
| 179 |
</div>
|
| 180 |
|
|
|
|
| 197 |
<h2 class="title is-3">Method Overview of GREAT Score</h2>
|
| 198 |
<div class="columns is-centered">
|
| 199 |
<div class="column container-centered">
|
| 200 |
+
<img src="./static/images/method.png" alt="Method Overview of Retention Score"/>
|
| 201 |
+
<p><strong>Flow chart of calculating Retention-Image and Retention-Text scores for VLMs. Given some evaluation samples, we first use diffusion generators to create semantically similar synthetic samples. Then, we pass the generated samples into a VLM to get responses and further use a toxicity judgment model (e.g., Perspective API \textsuperscript{1} or an LLM like Llama-70B (Touvron et al. 2023)) for toxicity level predictions. Finally, we use these statistics to compute the Retention Score as detailed in Section 3.2.</p>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
</div>
|
| 203 |
</div>
|
| 204 |
</div>
|
|
|
|
| 210 |
|
| 211 |
<!-- Robustness Certificate Definition -->
|
| 212 |
<section class="section">
|
|
|
|
| 213 |
<div class="container is-max-desktop">
|
| 214 |
+
<h2 class="title is-3">Establishing the Retention Score Framework</h2>
|
| 215 |
|
| 216 |
<div class="columns is-centered">
|
| 217 |
<div class="column container formula">
|
| 218 |
<p>
|
| 219 |
+
Revisiting concepts introduced earlier, minimal perturbations for Image-Text pair in context of VLMs were established. We proposed that greater values of minimal perturbations correlate with enhanced local robustness of model M for pair (I, T). Consequently, estimating lower bounds for these minimal perturbations provides measure of VLMs' robustness. To quantify robustness, we introduce Retention Score, which aims to provide assessment of VLM resilience against input perturbations. Higher Retention Scores signify model's inherent robustness, indicative of safeguards against adversarial toxicity manipulation. Retention Score is multimodal measure capable of assessing conditional robustness of VLMs across visual, textual domains, further divided into Retention-Image (Retention-I) and Retention-Text (Retention-T) scores.
|
| 220 |
</p>
|
| 221 |
</div>
|
| 222 |
</div>
|
|
|
|
| 225 |
<div class="column container-centered">
|
| 226 |
<div id="adaptive-loss-formula" class="container">
|
| 227 |
<div id="adaptive-loss-formula-list" class="row align-items-center formula-list">
|
| 228 |
+
<a href=".retention-image" class="selected">Retention-Image Score</a>
|
| 229 |
+
<a href=".retention-text">Retention-Text Score</a>
|
|
|
|
| 230 |
<div style="clear: both"></div>
|
| 231 |
</div>
|
| 232 |
<div class="row align-items-center adaptive-loss-formula-content">
|
| 233 |
+
<span class="formula retention-image formula-content">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 234 |
$$
|
| 235 |
+
\begin{align}
|
| 236 |
+
g_I(M,G_I(z|I), T) &= \sqrt{\frac{\pi}{2}} \cdot \{ M_{nt}(G_I(z|I), T) - M_t(G_I(z|I), T) \}^{+} \\
|
| 237 |
+
R_I(M, I, \mathbb{X}) &= \frac{1}{m \cdot n} \sum_{j=1}^m \sum_{i=1}^n g_I(M,G_I(z_i|I), T_j))
|
| 238 |
+
\end{align}
|
| 239 |
$$
|
| 240 |
</span>
|
| 241 |
+
<span class="formula retention-text formula-content" style="display: none;">
|
| 242 |
$$
|
| 243 |
+
\begin{align}
|
| 244 |
+
g_T(M,I, s(G_T(z|T))) &= \sqrt{\frac{\pi}{2}} \cdot \{ M_{nt}(I, \psi(s(G_T(z|T)))) - M_t(I, \psi(s(G_T(z|T)))) \}^{+} \\
|
| 245 |
+
R_T(M, I,\mathbb{X}) &= \frac{1}{m \cdot n} \sum_{j=1}^m \sum_{i=1}^n g_T(M,I, \psi(s(G_T(z_i|T_j))))
|
| 246 |
+
\end{align}
|
| 247 |
$$
|
| 248 |
</span>
|
| 249 |
</div>
|
|
|
|
| 253 |
|
| 254 |
<div class="columns is-centered">
|
| 255 |
<div class="column container adaptive-loss-formula-content">
|
| 256 |
+
<p class="formula retention-image formula-content">
|
| 257 |
+
where G_I(z|I) is a continuous diffusion-based image generation model that synthesizes semantically similar images to I, given a zero-mean isotropic Gaussian-distributed input z. The local score function g_I evaluates the non-toxicity of the generated image associated with the given prompt T.
|
|
|
|
|
|
|
|
|
|
| 258 |
</p>
|
| 259 |
+
<p class="formula retention-text formula-content" style="display: none;">
|
| 260 |
+
where G_T(z|T) refers to a text generator founded on paraphrasing diffusion techniques, s and ψ represent semantic encoder and decoder that translate discrete textual information into continuous vectorial representation and vice versa.
|
| 261 |
</p>
|
| 262 |
</div>
|
| 263 |
</div>
|
| 264 |
+
</div>
|
| 265 |
+
</section>
|
|
|
|
|
|
|
| 266 |
|
| 267 |
|
| 268 |
|
| 269 |
<!-- Results -->
|
| 270 |
<section class="section">
|
| 271 |
<div class="container is-max-desktop">
|
| 272 |
+
<h2 class="title is-3">Retention Image Score Results</h2>
|
| 273 |
<div class="columns is-centered">
|
| 274 |
<div class="column container-centered">
|
| 275 |
<table class="tg" border="1" style="width:100%;">
|
| 276 |
+
<caption><strong>Table 1.</strong> Jailbreak risk evaluation of VLMs to image attacks. This table presents a comparison among three VLMs — MiniGPT-4, LLaVA, and InstructBLIP — with regards to their Retention Scores (Retention-I), and Attack Success Rates (ASR, calculated as the percentage of outputs displaying toxic attributes).</caption>
|
| 277 |
<thead>
|
| 278 |
<tr>
|
| 279 |
+
<th class="tg-amwm"></th>
|
| 280 |
+
<th class="tg-baqh" colspan="2">MiniGPT-4</th>
|
| 281 |
+
<th class="tg-baqh" colspan="2">LLaVA</th>
|
| 282 |
+
<th class="tg-baqh" colspan="2">InstructBLIP</th>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 283 |
</tr>
|
| 284 |
<tr>
|
| 285 |
+
<th class="tg-amwm"></th>
|
| 286 |
+
<th class="tg-baqh">Retention-I</th>
|
| 287 |
+
<th class="tg-baqh">ASR (%)</th>
|
| 288 |
+
<th class="tg-baqh">Retention-I</th>
|
| 289 |
+
<th class="tg-baqh">ASR (%)</th>
|
| 290 |
+
<th class="tg-baqh">Retention-I</th>
|
| 291 |
+
<th class="tg-baqh">ASR (%)</th>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 292 |
</tr>
|
| 293 |
+
</thead>
|
| 294 |
+
<tbody>
|
| 295 |
<tr>
|
| 296 |
+
<td class="tg-baqh">Young</td>
|
| 297 |
+
<td class="tg-baqh">0.6121</td>
|
| 298 |
+
<td class="tg-baqh">40.93</td>
|
| 299 |
+
<td class="tg-baqh">0.2866</td>
|
| 300 |
+
<td class="tg-baqh">58.86</td>
|
| 301 |
+
<td class="tg-baqh">0.5043</td>
|
| 302 |
+
<td class="tg-baqh">49.72</td>
|
| 303 |
</tr>
|
| 304 |
<tr>
|
| 305 |
+
<td class="tg-baqh">Old</td>
|
| 306 |
+
<td class="tg-baqh">0.5917</td>
|
| 307 |
+
<td class="tg-baqh">43.27</td>
|
| 308 |
+
<td class="tg-baqh">0.2636</td>
|
| 309 |
+
<td class="tg-baqh">64.71</td>
|
| 310 |
+
<td class="tg-baqh">0.5650</td>
|
| 311 |
+
<td class="tg-baqh">47.76</td>
|
| 312 |
</tr>
|
| 313 |
<tr>
|
| 314 |
+
<td class="tg-baqh">Woman</td>
|
| 315 |
+
<td class="tg-baqh">0.5621</td>
|
| 316 |
+
<td class="tg-baqh">42.12</td>
|
| 317 |
+
<td class="tg-baqh">0.2261</td>
|
| 318 |
+
<td class="tg-baqh">57.70</td>
|
| 319 |
+
<td class="tg-baqh">0.4861</td>
|
| 320 |
+
<td class="tg-baqh">52.00</td>
|
| 321 |
</tr>
|
| 322 |
<tr>
|
| 323 |
+
<td class="tg-baqh">Man</td>
|
| 324 |
+
<td class="tg-baqh">0.5438</td>
|
| 325 |
+
<td class="tg-baqh">42.63</td>
|
| 326 |
+
<td class="tg-baqh">0.1971</td>
|
| 327 |
+
<td class="tg-baqh">52.16</td>
|
| 328 |
+
<td class="tg-baqh">0.4966</td>
|
| 329 |
+
<td class="tg-baqh">50.36</td>
|
| 330 |
</tr>
|
| 331 |
<tr>
|
| 332 |
+
<td class="tg-baqh">Average</td>
|
| 333 |
+
<td class="tg-baqh">0.5774</td>
|
| 334 |
+
<td class="tg-baqh">42.49</td>
|
| 335 |
+
<td class="tg-baqh">0.2434</td>
|
| 336 |
+
<td class="tg-baqh">58.36</td>
|
| 337 |
+
<td class="tg-baqh">0.5130</td>
|
| 338 |
+
<td class="tg-baqh">49.96</td>
|
| 339 |
</tr>
|
| 340 |
+
</tbody>
|
| 341 |
+
</table>
|
| 342 |
+
</div>
|
| 343 |
+
</div>
|
| 344 |
+
</div>
|
| 345 |
+
</section>
|
| 346 |
+
<!-- Results -->
|
| 347 |
+
|
| 348 |
+
<section class="section">
|
| 349 |
+
<div class="container is-max-desktop">
|
| 350 |
+
<h2 class="title is-3">Retention Text Score Results</h2>
|
| 351 |
+
<div class="columns is-centered">
|
| 352 |
+
<div class="column container-centered">
|
| 353 |
+
<table class="tg" border="1" style="width:100%;">
|
| 354 |
+
<caption><strong>Table 2.</strong> Jailbreak risk evaluation of VLMs to text attacks. This table presents a comparison among three VLMs — MiniGPT-4, LLaVA, and InstructBLIP — with regards to their Retention Scores (Retention-T), Attack Success Rates.</caption>
|
| 355 |
+
<thead>
|
| 356 |
<tr>
|
| 357 |
+
<th class="tg-amwm">VLM</th>
|
| 358 |
+
<th class="tg-baqh">Retention-T</th>
|
| 359 |
+
<th class="tg-baqh">Attack Success Rate</th>
|
|
|
|
|
|
|
|
|
|
| 360 |
</tr>
|
| 361 |
+
</thead>
|
| 362 |
+
<tbody>
|
| 363 |
<tr>
|
| 364 |
+
<td class="tg-baqh">MiniGPT-4</td>
|
| 365 |
+
<td class="tg-baqh">0.2073</td>
|
| 366 |
+
<td class="tg-baqh">46.1%</td>
|
|
|
|
|
|
|
|
|
|
| 367 |
</tr>
|
| 368 |
<tr>
|
| 369 |
+
<td class="tg-baqh">LLaVA</td>
|
| 370 |
+
<td class="tg-baqh">0.342</td>
|
| 371 |
+
<td class="tg-baqh">9.4%</td>
|
|
|
|
|
|
|
|
|
|
| 372 |
</tr>
|
| 373 |
<tr>
|
| 374 |
+
<td class="tg-baqh">InstructBLIP</td>
|
| 375 |
+
<td class="tg-baqh">0.164</td>
|
| 376 |
+
<td class="tg-baqh">84.5%</td>
|
|
|
|
|
|
|
|
|
|
| 377 |
</tr>
|
| 378 |
</tbody>
|
| 379 |
+
</table>
|
| 380 |
</div>
|
| 381 |
</div>
|
| 382 |
</div>
|
| 383 |
</section>
|
|
|
|
| 384 |
|
| 385 |
+
|
| 386 |
+
|
| 387 |
+
|
| 388 |
+
|
| 389 |
+
|
| 390 |
+
|
| 391 |
+
|
| 392 |
+
|
| 393 |
+
|
| 394 |
+
|
| 395 |
<!-- Model Ranking Comparison Section -->
|
| 396 |
<section class="section">
|
| 397 |
<div class="container is-max-desktop">
|
| 398 |
+
<h2 class="title is-3">API Model Analysis</h2>
|
| 399 |
<div class="columns is-centered">
|
| 400 |
<div class="column is-full-width">
|
| 401 |
<div class="content has-text-justified">
|
| 402 |
+
<p>
|
| 403 |
+
Assessing the robustness of black-box VLMs is of paramount importance, particularly since these models are commonly deployed as APIs, restricting users and auditors to inferential interactions. This constraint not only makes adversarial attacks challenging but also underscores the necessity for robust evaluation methods that do not depend on internal model access. In this context, our research deploys the Retention-I score to examine the resilience of APIs against synthetically produced facial images with concealed attributes, which are typically employed in model inferences.
|
| 404 |
+
</p>
|
| 405 |
+
|
| 406 |
+
<p>
|
| 407 |
+
Our evaluation methodology was applied to two prominent online vision language APIs: GPT-4V and Gemini Pro Vision. Noteworthy is that for Gemini Pro Vision, the API provides settings to adjust the model's threshold for blocking harmful content, with options ranging from blocking none to most (none, few, some, and most). We tested this feature by running identical prompts and images across these settings, leading to an evaluation of five model configurations.
|
| 408 |
+
</p>
|
| 409 |
+
|
| 410 |
+
<p>
|
| 411 |
+
The assessment centered around the Retention-I score, using a balanced set of synthetic faces that included young, old, male, and female groups. These images were generated using the state-of-the-art Stable Diffusion model, with each group contributing 100 images. A unique aspect of Google's Gemini is its error messaging system, which, in lieu of producing toxic outputs, provides rationales for prompt blocking. In our study, such blocks were interpreted as a zero toxicity score, aligning with the model's safeguarding strategy.
|
| 412 |
+
</p>
|
| 413 |
+
|
| 414 |
<table class="table is-bordered is-striped is-narrow is-hoverable is-fullwidth">
|
| 415 |
+
<caption><strong>Table 3.</strong> Retention-I analysis of VLM APIs. Each group consists of 100 images with 20 continuation prompts.</caption>
|
| 416 |
<thead>
|
| 417 |
<tr>
|
| 418 |
<th></th>
|
| 419 |
+
<th>Young</th>
|
| 420 |
+
<th>Old</th>
|
| 421 |
+
<th>Woman</th>
|
| 422 |
+
<th>Man</th>
|
| 423 |
+
<th>Average</th>
|
| 424 |
</tr>
|
| 425 |
</thead>
|
| 426 |
<tbody>
|
| 427 |
<tr>
|
| 428 |
+
<td>GPT-4v</td>
|
| 429 |
+
<td>1.2043</td>
|
| 430 |
+
<td>1.2077</td>
|
| 431 |
+
<td>1.2067</td>
|
| 432 |
+
<td>1.2052</td>
|
| 433 |
+
<td>1.2059</td>
|
| 434 |
+
</tr>
|
| 435 |
+
<tr>
|
| 436 |
+
<td>Gemini-None</td>
|
| 437 |
+
<td>0.3025</td>
|
| 438 |
+
<td>0.2432</td>
|
| 439 |
+
<td>0.2300</td>
|
| 440 |
+
<td>0.2126</td>
|
| 441 |
+
<td>0.2471</td>
|
| 442 |
</tr>
|
| 443 |
<tr>
|
| 444 |
+
<td>Gemini-Few</td>
|
| 445 |
+
<td>1.1955</td>
|
| 446 |
+
<td>1.1806</td>
|
| 447 |
+
<td>1.1972</td>
|
| 448 |
+
<td>1.1987</td>
|
| 449 |
+
<td>1.1930</td>
|
| 450 |
</tr>
|
| 451 |
<tr>
|
| 452 |
+
<td>Gemini-Some</td>
|
| 453 |
+
<td>1.2322</td>
|
| 454 |
+
<td>1.2486</td>
|
| 455 |
+
<td>1.2325</td>
|
| 456 |
+
<td>1.2382</td>
|
| 457 |
+
<td>1.2379</td>
|
| 458 |
+
</tr>
|
| 459 |
+
<tr>
|
| 460 |
+
<td>Gemini-Most</td>
|
| 461 |
+
<td>1.2449</td>
|
| 462 |
+
<td>1.2494</td>
|
| 463 |
+
<td>1.2388</td>
|
| 464 |
+
<td>1.2479</td>
|
| 465 |
+
<td>1.2453</td>
|
| 466 |
</tr>
|
| 467 |
</tbody>
|
| 468 |
</table>
|
| 469 |
+
|
| 470 |
<p>
|
| 471 |
+
Our results in Table 3 reveal intriguing variations across different APIs. For instance, Gemini-None exhibited notable performance contrasts when comparing Old versus Young cohorts. Other models showcased more uniform robustness levels across demographic groups. Also, Our analysis positions the robustness of GPT-4V somewhere between the some and most safety settings of Gemini. This correlation not only validates the efficacy of Gemini's protective configurations but also underscores the impact of safety thresholds on toxicity recognition, as quantified by our scoring method.
|
| 472 |
+
</p>
|
| 473 |
+
|
| 474 |
+
<p>
|
| 475 |
+
This robustness evaluation illustrates that Retention-I is a pivotal tool for analyzing group-level resilience in models with restricted access, enabling discreet and efficacious scrutiny of their defenses.
|
| 476 |
</p>
|
| 477 |
</div>
|
| 478 |
</div>
|
|
|
|
| 502 |
</section>
|
| 503 |
<!-- GREAT Score vs CW Attack Comparison Section -->
|
| 504 |
|
| 505 |
+
<!-- Run-time Analysis Section -->
|
| 506 |
+
<section class="section">
|
| 507 |
+
<div class="container is-max-desktop">
|
| 508 |
+
<h2 class="title is-3">Run-time Analysis</h2>
|
| 509 |
+
<div class="columns is-centered">
|
| 510 |
+
<div class="column container-centered">
|
| 511 |
+
<div>
|
| 512 |
+
<img src="./static/images/plot1.png"
|
| 513 |
+
class="method_overview"
|
| 514 |
+
alt="Run-time improvement comparison"/>
|
| 515 |
+
<p>
|
| 516 |
+
<strong>Figure 4.</strong> Run-time improvement (Retention Score over Visual and Text attacks).
|
| 517 |
+
</p>
|
| 518 |
+
<div class="content has-text-justified">
|
| 519 |
+
<p>
|
| 520 |
+
Figure 4 compares the run-time efficiency of Retention Score over adversarial attacks in [1] and [2].
|
| 521 |
+
We show the improvement ratio of their average per-sample run-time (wall clock time of Retention Score/Adversarial Attack is reported in Appendix)
|
| 522 |
+
and observe around 2-30 times improvement, validating the computational efficiency of Retention Score.
|
| 523 |
+
</p>
|
| 524 |
+
</div>
|
| 525 |
+
</div>
|
| 526 |
+
</div>
|
| 527 |
+
</div>
|
| 528 |
+
</div>
|
| 529 |
+
</section>
|
| 530 |
+
<!-- Run-time Analysis Section -->
|
| 531 |
+
|
| 532 |
|
| 533 |
|
| 534 |
|