|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
Contributors: Nicolas Mejia Petit |
|
|
|
 |
|
|
|
# Mistral 29b: A New Base Model |
|
|
|
The objective of this model is to serve as a new fully open source base model with 29.2 billion parameters. |
|
|
|
This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune. |
|
|
|
## Model Creation |
|
|
|
The model was created by stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model. |
|
|
|
## Useful Resources |
|
|
|
- Model Merging (MergeKit) (https://github.com/cg123/mergekit) |
|
|
|
## Source Models |
|
|
|
- Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b) |
|
- Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) |
|
- MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B) |
|
- speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0) |
|
|
|
|
