shubhrapandit commited on
Commit
2ea9797
·
verified ·
1 Parent(s): ea298ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -466,21 +466,21 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
466
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w8a8</td>
467
  <td>1.80</td>
468
  <td>1.2</td>
469
- <td>289</td>
470
  <td>4.0</td>
471
- <td>1020</td>
472
  <td>4.6</td>
473
- <td>1133</td>
474
  </tr>
475
  <tr>
476
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
477
  <td>2.75</td>
478
  <td>2.8</td>
479
- <td>341</td>
480
  <td>12.8</td>
481
- <td>1588</td>
482
  <td>16.4</td>
483
- <td>2037</td>
484
  </tr>
485
  <tr>
486
  <th rowspan="3" valign="top">H100x4</th>
@@ -497,21 +497,21 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
497
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-FP8-Dynamic</td>
498
  <td>1.73</td>
499
  <td>1.8</td>
500
- <td>247</td>
501
  <td>4.4</td>
502
- <td>621</td>
503
  <td>4.8</td>
504
- <td>669</td>
505
  </tr>
506
  <tr>
507
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
508
  <td>8.27</td>
509
  <td>13.2</td>
510
- <td>913</td>
511
  <td>13.2</td>
512
- <td>913</td>
513
  <td>99.2</td>
514
- <td>6777</td>
515
  </tr>
516
  </tbody>
517
  </table>
 
466
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w8a8</td>
467
  <td>1.80</td>
468
  <td>1.2</td>
469
+ <td>578</td>
470
  <td>4.0</td>
471
+ <td>2040</td>
472
  <td>4.6</td>
473
+ <td>2266</td>
474
  </tr>
475
  <tr>
476
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
477
  <td>2.75</td>
478
  <td>2.8</td>
479
+ <td>1364</td>
480
  <td>12.8</td>
481
+ <td>6352</td>
482
  <td>16.4</td>
483
+ <td>8148</td>
484
  </tr>
485
  <tr>
486
  <th rowspan="3" valign="top">H100x4</th>
 
497
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-FP8-Dynamic</td>
498
  <td>1.73</td>
499
  <td>1.8</td>
500
+ <td>479</td>
501
  <td>4.4</td>
502
+ <td>1203</td>
503
  <td>4.8</td>
504
+ <td>1296</td>
505
  </tr>
506
  <tr>
507
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
508
  <td>8.27</td>
509
  <td>13.2</td>
510
+ <td>3652</td>
511
  <td>13.2</td>
512
+ <td>3652</td>
513
  <td>99.2</td>
514
+ <td>27108</td>
515
  </tr>
516
  </tbody>
517
  </table>