picocreator
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,22 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
+
|
5 |
+
![An eagle soaring above a transformer robot](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bbd31a7-21b4-4ff6-b43f-8735d1decf25_2048x1652.png)
|
6 |
+
|
7 |
+
# Eagle 7B - in short
|
8 |
+
|
9 |
+
Eagle 7B is a 7.52B parameter model that:
|
10 |
+
|
11 |
+
- Built on the RWKV-v5 architecture
|
12 |
+
(a linear transformer with 10-100x+ lower inference cost)
|
13 |
+
- Ranks as the world’s greenest 7B model (per token)
|
14 |
+
- Trained on 1.1 Trillion Tokens across 100+ languages
|
15 |
+
- Outperforms all 7B class models in multi-lingual benchmarks
|
16 |
+
- Approaches Falcon (1.5T), LLaMA2 (2T), Mistral (>2T?) level of performance in English evals
|
17 |
+
- Trade blows with MPT-7B (1T) in English evals
|
18 |
+
- All while being an “Attention-Free Transformer”
|
19 |
+
- Is a foundation model, with a very small instruct tune - further fine-tuning is required for various use cases!
|
20 |
+
|
21 |
+
Find out more at our model announcment: https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers
|
22 |
+
Or our wiki: https://wiki.rwkv.com
|