Update README.md

README.md

@@ -10,6 +10,77 @@ library_name: transformers
 ## Overview
 This repository contains a compressed version of the Meta Llama-3-8B-Instruct model, utilizing the Palu framework for KV-Cache compression. Palu reduces the hidden dimensions of the KV-Cache through low-rank decomposition, significantly reducing the model's memory footprint while maintaining or enhancing performance.
 
+
+## Evaluation Results
+
+Here's a Markdown file to include the results of your comparisons:
+
+---
+
+# Meta Llama-3-8B-Instruct: Palu Compression Results
+
+## Perplexity (PPL)
+
+| Model                                  | PPL             |
+|----------------------------------------|-----------------|
+| **meta-llama-3-8b-instruct-palu**       | **8.8309**      |
+| **meta-llama-3-8b-instruct (Base)**     | **8.2845**      |
+
+## Zero-shot Evaluation
+
+### meta-llama-3-8b-instruct-palu
+
+|    Tasks        | Version | Filter | n-shot | Metric  | Value  | Stderr  |
+|-----------------|---------|--------|--------|---------|--------|---------|
+| winogrande      |      1  | none   |    0   | acc     | 0.7277 | ±0.0125 |
+| arc_challenge   |      1  | none   |    0   | acc     | 0.4949 | ±0.0146 |
+|                 |         |        |    0   | acc_norm| 0.5427 | ±0.0146 |
+| arc_easy        |      1  | none   |    0   | acc     | 0.7942 | ±0.0083 |
+|                 |         |        |    0   | acc_norm| 0.7551 | ±0.0088 |
+| piqa            |      1  | none   |    0   | acc     | 0.7655 | ±0.0099 |
+|                 |         |        |    0   | acc_norm| 0.7644 | ±0.0099 |
+| hellaswag       |      1  | none   |    0   | acc     | 0.5664 | ±0.0049 |
+|                 |         |        |    0   | acc_norm| 0.7511 | ±0.0043 |
+| openbookqa      |      1  | none   |    0   | acc     | 0.3360 | ±0.0211 |
+|                 |         |        |    0   | acc_norm| 0.4380 | ±0.0222 |
+
+### meta-llama-3-8b-instruct (Base)
+
+|    Tasks        | Version | Filter | n-shot | Metric  | Value  | Stderr  |
+|-----------------|---------|--------|--------|---------|--------|---------|
+| winogrande      |      1  | none   |    0   | acc     | 0.7206 | ±0.0126 |
+| arc_challenge   |      1  | none   |    0   | acc     | 0.5299 | ±0.0146 |
+|                 |         |        |    0   | acc_norm| 0.5683 | ±0.0145 |
+| arc_easy        |      1  | none   |    0   | acc     | 0.8161 | ±0.0079 |
+|                 |         |        |    0   | acc_norm| 0.7976 | ±0.0082 |
+| piqa            |      1  | none   |    0   | acc     | 0.7867 | ±0.0096 |
+|                 |         |        |    0   | acc_norm| 0.7856 | ±0.0096 |
+| hellaswag       |      1  | none   |    0   | acc     | 0.5769 | ±0.0049 |
+|                 |         |        |    0   | acc_norm| 0.7581 | ±0.0043 |
+| openbookqa      |      1  | none   |    0   | acc     | 0.3420 | ±0.0212 |
+|                 |         |        |    0   | acc_norm| 0.4320 | ±0.0222 |
+
+## Long-Bench Evaluation
+
+### triviaqa
+
+| Model                                  | Score  |
+|----------------------------------------|--------|
+| **meta-llama-3-8b-instruct-palu**       | 89.45  |
+| **meta-llama-3-8b-instruct (Base)**     | 90.56  |
+
+### qasper
+
+| Model                                  | Score  |
+|----------------------------------------|--------|
+| **meta-llama-3-8b-instruct-palu**       | 34.92  |
+| **meta-llama-3-8b-instruct (Base)**     | 31.74  |
+
+---
+
+This Markdown file should effectively summarize and present the results of your model comparison.
+
+
 ## Key Features
 - **Model**: Meta Llama-3-8B-Instruct
 - **Compression Framework**: Palu