Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,6 @@ This is a direct extraction of the 8 experts from [Mixtral-8x7b-Instruct-v0.1](h
|
|
| 10 |
It is 2 experts per token. Performance is identical to instruct, if not a little better. Evals will come, It is more malleable to training. This is our first experiment with expert extraction and modification, more to come. Enjoy.
|
| 11 |
|
| 12 |
|
| 13 |
-
```markdown
|
| 14 |
## Instruction Format
|
| 15 |
To leverage instruction fine-tuning, your prompts should be enclosed with `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin-of-sentence id, while subsequent instructions should not. Assistant generation will conclude with an end-of-sentence token id.
|
| 16 |
|
|
|
|
| 10 |
It is 2 experts per token. Performance is identical to instruct, if not a little better. Evals will come, It is more malleable to training. This is our first experiment with expert extraction and modification, more to come. Enjoy.
|
| 11 |
|
| 12 |
|
|
|
|
| 13 |
## Instruction Format
|
| 14 |
To leverage instruction fine-tuning, your prompts should be enclosed with `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin-of-sentence id, while subsequent instructions should not. Assistant generation will conclude with an end-of-sentence token id.
|
| 15 |
|