Update README.md
Browse files
README.md
CHANGED
|
@@ -5,4 +5,10 @@ datasets:
|
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
pipeline_tag: text-generation
|
| 8 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
pipeline_tag: text-generation
|
| 8 |
+
---
|
| 9 |
+
## Model description
|
| 10 |
+
This is a Vicuna-like model with only 68M parameters, which is fine-tuned from [LLaMA-68m](https://huggingface.co/JackFram/llama-68m) on [ShareGPT](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered) data.
|
| 11 |
+
|
| 12 |
+
The training setup follows the [Vicuna suite](https://github.com/lm-sys/FastChat).
|
| 13 |
+
|
| 14 |
+
The model is mainly developed as a base Small Speculative Model. As a comparison, it can be better aligned to the Vicuna models than LLaMA-68m with little loss of alignment to the LLaMA models.
|