Commit
•
4b0f6b1
1
Parent(s):
f02d105
Update README.md
Browse files
README.md
CHANGED
|
@@ -107,26 +107,7 @@ GGUF Available [Here](https://huggingface.co/arcee-ai/Virtuoso-Small-GGUF)
|
|
| 107 |
|
| 108 |
# Virtuoso-Small
|
| 109 |
|
| 110 |
-
Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
## Performance Benchmarks
|
| 114 |
-
|
| 115 |
-
| **Groups** | **Metric** | ↑ | **Value** | ± | **Stderr** |
|
| 116 |
-
|---------------------------|--------------------------|---|----------:|----|-----------:|
|
| 117 |
-
| **Leaderboard** | **Accuracy** | ↑ | 0.5194 | ± | 0.0046 |
|
| 118 |
-
| | Normalized Accuracy | ↑ | 0.5814 | ± | 0.0051 |
|
| 119 |
-
| | Exact Match | ↑ | 0.3006 | ± | 0.0117 |
|
| 120 |
-
| | Instruction-Level Loose Accuracy | ↑ | 0.8489 | ± | N/A |
|
| 121 |
-
| | Instruction-Level Strict Accuracy | ↑ | 0.8249 | ± | N/A |
|
| 122 |
-
| | Prompt-Level Loose Accuracy | ↑ | 0.7856 | ± | 0.0177 |
|
| 123 |
-
| | Prompt-Level Strict Accuracy | ↑ | 0.7523 | ± | 0.0186 |
|
| 124 |
-
| **Leaderboard-BBH** | Normalized Accuracy | ↑ | 0.6516 | ± | 0.0058 |
|
| 125 |
-
| **Leaderboard-GPQA** | Normalized Accuracy | ↑ | 0.3389 | ± | 0.0137 |
|
| 126 |
-
| **Leaderboard-Math-Hard** | Exact Match | ↑ | 0.3006 | ± | 0.0117 |
|
| 127 |
-
| **Leaderboard-MuSR** | Normalized Accuracy | ↑ | 0.4286 | ± | 0.0175 |
|
| 128 |
-
|
| 129 |
-
---
|
| 130 |
|
| 131 |
## Key Features
|
| 132 |
|
|
|
|
| 107 |
|
| 108 |
# Virtuoso-Small
|
| 109 |
|
| 110 |
+
Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-Medium and Virtuoso-Large, offer even greater capabilities and are available via API at [models.arcee.ai](https://models.arcee.ai).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
|
| 112 |
## Key Features
|
| 113 |
|