Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,106 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
# Mixtral-8x22b
|
| 6 |
New MoE model by MistralAI
|
| 7 |
|
| 8 |
-
Model
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
pipeline_tag: text-generation
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
- fr
|
| 7 |
+
- es
|
| 8 |
+
library_name: transformers
|
| 9 |
+
tags:
|
| 10 |
+
- moe
|
| 11 |
+
- text-generation-inference
|
| 12 |
---
|
| 13 |
# Mixtral-8x22b
|
| 14 |
New MoE model by MistralAI
|
| 15 |
|
| 16 |
+
## Model Details:
|
| 17 |
+
- 65k context window
|
| 18 |
+
- 48 attention heads
|
| 19 |
+
- 56 layers
|
| 20 |
+
- 8 experts
|
| 21 |
|
| 22 |
+
The model is split into 27 parts, from the original torrent.
|
| 23 |
+
|
| 24 |
+
Magnet link and checksum: [https://twitter.com/mistralai/status/1777869263778291896](https://twitter.com/mistralai/status/1777869263778291896)
|
| 25 |
+
|
| 26 |
+
## How to use:
|
| 27 |
+
Run git clone:
|
| 28 |
+
```
|
| 29 |
+
git clone https://huggingface.co/leafspark/mixtral-8x22b
|
| 30 |
+
cd mixtral-8x22b
|
| 31 |
+
```
|
| 32 |
+
Make sure you have Python 2 or 3 installed (HuggingFace libraries not required):
|
| 33 |
+
```
|
| 34 |
+
python merge.py
|
| 35 |
+
```
|
| 36 |
+
This should take approximately 2 hours, you will be left with a 274GB file.
|
| 37 |
+
|
| 38 |
+
Check the MD5 hash of consolidated.safetensors:
|
| 39 |
+
```
|
| 40 |
+
3816cd2c4f827b4b868bc6481d5d3ba2
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
That's it! Now you have the complete torrent download on your computer.
|
| 44 |
+
|
| 45 |
+
## Credit to:
|
| 46 |
+
|
| 47 |
+
βββββ
|
| 48 |
+
ββββββββββββββββββ
|
| 49 |
+
βββββββββββββββββββββββββββββββ
|
| 50 |
+
ββββββββββββββββββββββββββββββββββ
|
| 51 |
+
ββββββββββββββ ββββββββββββββββββββββββββββββββββ
|
| 52 |
+
βββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββ
|
| 53 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 54 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 55 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 56 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 57 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 58 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 59 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 60 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 61 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 62 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 63 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 64 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 65 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββοΏ½οΏ½οΏ½βββββββββββββββββββββββ
|
| 66 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 67 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 68 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 69 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 70 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 71 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 72 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 73 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 74 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 75 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 76 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 77 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 78 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 79 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 80 |
+
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 81 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ βββββββ
|
| 82 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ βββ
|
| 83 |
+
ββββββββββββββββββββββββββββββββ ββββββββββββββββββ
|
| 84 |
+
ββββββββββββββββββββββββββββ
|
| 85 |
+
βββββββββββββββββ
|
| 86 |
+
βββββ
|
| 87 |
+
|
| 88 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 89 |
+
|
| 90 |
+
β β
|
| 91 |
+
β Β·Β· Released by the Mistral AI team Β·Β· β
|
| 92 |
+
|
| 93 |
+
β Albert Jiang, Alexandre Sablayrolles, Alexis Tacnet, Antoine Roux, β
|
| 94 |
+
β Arthur Mensch, Audrey Herblin-Stoop, Baptiste Bout, Baudouin de Monicault, β
|
| 95 |
+
β Blanche Savary, Bam4d, Caroline Feldman, Devendra Singh Chaplot, β
|
| 96 |
+
β Diego de las Casas, Eleonore Arcelin, Emma Bou Hanna, Etienne Metzger, β
|
| 97 |
+
β Gianna Lengyel, Guillaume Bour, Guillaume Lample, Harizo Rajaona, β
|
| 98 |
+
β Jean-Malo Delignon, Jia Li, Justus Murke, Louis Martin, Louis Ternon β
|
| 99 |
+
β Lucile Saulnier, LΓ©lio Renard Lavaud, Margaret Jennings, Marie Pellat, β
|
| 100 |
+
β Marie Torelli, Marie-Anne Lachaux, Nicolas Schuhl, Patrick von Platen, β
|
| 101 |
+
β Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, β
|
| 102 |
+
β Teven Le Scao, Thibaut Lavril, TimothΓ©e Lacroix, ThΓ©ophile Gervet, β
|
| 103 |
+
β Thomas Wang, Valera Nemychnikova, William El Sayed, William Marshall β
|
| 104 |
+
β β
|
| 105 |
+
|
| 106 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|