shubhrapandit commited on
Commit
020912a
·
verified ·
1 Parent(s): c07b3ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -442,21 +442,21 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
442
  <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w8a8</td>
443
  <td>1.84</td>
444
  <td>1.2</td>
445
- <td>293</td>
446
  <td>4.0</td>
447
- <td>1021</td>
448
  <td>4.6</td>
449
- <td>1135</td>
450
  </tr>
451
  <tr>
452
  <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
453
  <td>2.73</td>
454
  <td>2.4</td>
455
- <td>314</td>
456
  <td>12.8</td>
457
- <td>1591</td>
458
  <td>16.0</td>
459
- <td>2019</td>
460
  </tr>
461
  <tr>
462
  <th rowspan="3" valign="top">H100x4</td>
@@ -473,21 +473,21 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
473
  <td>neuralmagic/Qwen2-VL-72B-Instruct-FP8-Dynamic</td>
474
  <td>1.70</td>
475
  <td>1.6</td>
476
- <td>236</td>
477
  <td>4.4</td>
478
- <td>623</td>
479
  <td>4.8</td>
480
- <td>669</td>
481
  </tr>
482
  <tr>
483
  <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
484
  <td>2.35</td>
485
  <td>5.2</td>
486
- <td>350</td>
487
  <td>13.2</td>
488
- <td>910</td>
489
  <td>14.4</td>
490
- <td>994</td>
491
  </tr>
492
  </tbody>
493
  </table>
 
442
  <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w8a8</td>
443
  <td>1.84</td>
444
  <td>1.2</td>
445
+ <td>586</td>
446
  <td>4.0</td>
447
+ <td>2042</td>
448
  <td>4.6</td>
449
+ <td>2270</td>
450
  </tr>
451
  <tr>
452
  <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
453
  <td>2.73</td>
454
  <td>2.4</td>
455
+ <td>1256</td>
456
  <td>12.8</td>
457
+ <td>6364</td>
458
  <td>16.0</td>
459
+ <td>8076</td>
460
  </tr>
461
  <tr>
462
  <th rowspan="3" valign="top">H100x4</td>
 
473
  <td>neuralmagic/Qwen2-VL-72B-Instruct-FP8-Dynamic</td>
474
  <td>1.70</td>
475
  <td>1.6</td>
476
+ <td>457</td>
477
  <td>4.4</td>
478
+ <td>1207</td>
479
  <td>4.8</td>
480
+ <td>1296</td>
481
  </tr>
482
  <tr>
483
  <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
484
  <td>2.35</td>
485
  <td>5.2</td>
486
+ <td>1400</td>
487
  <td>13.2</td>
488
+ <td>3640</td>
489
  <td>14.4</td>
490
+ <td>3976</td>
491
  </tr>
492
  </tbody>
493
  </table>