Update README.md

README.md

@@ -3,40 +3,22 @@ license: apache-2.0
 library_name: transformers
 ---
 
-Benchmarks for this model:
+# Qwerky-QwQ-32B
 
-|    Tasks     |Version|Filter|n-shot|  Metric  |   |Value |   |Stderr|
-|--------------|------:|------|-----:|----------|---|-----:|---|-----:|
-|arc_challenge |      1|none  |     0|acc       |↑  |0.5239|±  |0.0146|
-|              |       |none  |     0|acc_norm  |↑  |0.5640|±  |0.0145|
-|arc_easy      |      1|none  |     0|acc       |↑  |0.8060|±  |0.0081|
-|              |       |none  |     0|acc_norm  |↑  |0.7837|±  |0.0084|
-|hellaswag     |      1|none  |     0|acc       |↑  |0.6398|±  |0.0048|
-|              |       |none  |     0|acc_norm  |↑  |0.8303|±  |0.0037|
-|lambada_openai|      1|none  |     0|acc       |↑  |0.6621|±  |0.0066|
-|              |       |none  |     0|perplexity|↓  |4.0357|±  |0.0917|
-|piqa          |      1|none  |     0|acc       |↑  |0.8036|±  |0.0093|
-|              |       |none  |     0|acc_norm  |↑  |0.8134|±  |0.0091|
-|sciq          |      1|none  |     0|acc       |↑  |0.9630|±  |0.0060|
-|              |       |none  |     0|acc_norm  |↑  |0.9440|±  |0.0073|
-|winogrande    |      1|none  |     0|acc       |↑  |0.7324|±  |0.0124|
-|mmlu          |      2|none  |      |acc       |↑  |0.7431|±  |0.0034|
+The following is a model converted from Qwen 32B QWQ, to the RWKV based architecture.
+For existing details of the process from our previous release, find it [here]: https://huggingface.co/recursal/QRWKV6-32B-Instruct-Preview-v0.1
 
-Benchmarks for base Qwen/QwQ-32B model:
+Benchmarks for Qwerky-QwQ-32B and the Qwerky-72B models
 
-|    Tasks     |Version|Filter|n-shot|  Metric  |   |Value |   |Stderr|
-|--------------|------:|------|-----:|----------|---|-----:|---|-----:|
-|arc_challenge |      1|none  |     0|acc       |↑  |0.5367|±  |0.0146|
-|              |       |none  |     0|acc_norm  |↑  |0.5563|±  |0.0145|
-|arc_easy      |      1|none  |     0|acc       |↑  |0.8102|±  |0.0080|
-|              |       |none  |     0|acc_norm  |↑  |0.7866|±  |0.0084|
-|hellaswag     |      1|none  |     0|acc       |↑  |0.6516|±  |0.0048|
-|              |       |none  |     0|acc_norm  |↑  |0.8407|±  |0.0037|
-|lambada_openai|      1|none  |     0|acc       |↑  |0.6683|±  |0.0066|
-|              |       |none  |     0|perplexity|↓  |3.8310|±  |0.0893|
-|piqa          |      1|none  |     0|acc       |↑  |0.7976|±  |0.0094|
-|              |       |none  |     0|acc_norm  |↑  |0.8118|±  |0.0091|
-|sciq          |      1|none  |     0|acc       |↑  |0.9630|±  |0.0060|
-|              |       |none  |     0|acc_norm  |↑  |0.9490|±  |0.0070|
-|winogrande    |      1|none  |     0|acc       |↑  |0.7048|±  |0.0128|
-|mmlu          |      2|none  |      |acc       |↑  |0.7985|±  |0.0032|
+| Tasks | Metric | Qwerky-QwQ-32B | Qwen/QwQ-32B | Qwerky-72B | Qwen2.5-72B-Instruct |
+|:---:|:---:|:---:|:---:|:---:|:---:|
+| arc_challenge | acc_norm | **0.5640** | 0.5563 | **0.6382** | 0.6323 |
+| arc_easy | acc_norm | 0.7837 | **0.7866** | **0.8443** | 0.8329 |
+| hellaswag | acc_norm | 0.8303 | **0.8407** | 0.8573 | **0.8736** |
+| lambada_openai | acc | 0.6621 | **0.6683** | **0.7539** | 0.7506 |
+| piqa | acc | **0.8036** | 0.7976 | 0.8248 | **0.8357** |
+| sciq | acc | **0.9630** | **0.9630** | 0.9670 | **0.9740** |
+| winogrande | acc | **0.7324** | 0.7048 | **0.7956** | 0.7632 |
+| mmlu | acc | 0.7431 | **0.7985** | 0.7746 | **0.8338** |
+
+> All benchmark's besides MMLU are 0 n-shot, and is version 1, MMLU is version 2