File size: 2,889 Bytes
32b0592 436f600 6869e38 436f600 6869e38 e8a8916 7447e5d a358e0a 6988a4a a358e0a 6f7a305 7447e5d 6f7a305 79b5ea4 6f7a305 79b5ea4 6f7a305 79b5ea4 6f7a305 21da201 79b5ea4 21da201 4650536 6988a4a 4650536 6988a4a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
license: cc-by-nc-4.0
language:
- en
---
### Outdated! Move to [Lyra-v4](https://huggingface.co/Sao10K/MN-12B-Lyra-v4)
---
### Outdated
---
Mistral-NeMo-12B-Lyra-v4, layered over [Lyra-v3](https://huggingface.co/Sao10K/MN-12B-Lyra-v3), which was built on top of [Lyra-v2a2](https://huggingface.co/Sao10K/MN-12B-Lyra-v2a2), which itself was built upon [Lyra-v2a1](https://huggingface.co/Sao10K/MN-12B-Lyra-v2a1).
# Model Versioning
```
Lyra-v1 [Merge of Custom Roleplay & Instruct Trains, on Different Formats]
|
| [Additional SFT on 10% of Previous Data, Mixed]
v
Lyra-v2a1
|
| [Low Rank SFT Step + Tokenizer Diddling]
v
Lyra-v2a2
|
| [RL Step Performed on Multiturn Sets, Magpie-style Responses by Lyra-v2a2 for Rejected Data]
v
Lyra-v3
|
| [Backmerge to v2a1 + LoRA Extraction + Low Rank SFT Step for Coherency]
v
Lyra-v4
```
# This uses ChatML, or any of its variants which were included in previous versions.
```
<|im_start|>system
This is the system prompt.<|im_end|>
<|im_start|>user
Instructions placed here.<|im_end|>
<|im_start|>assistant
The model's response will be here.<|im_end|>
--------------------------------------------------
[INST]system
This is another system prompt.[/INST]
[INST]user
Your instructions placed here.[/INST]
[INST]assistant
The model's response will be here.[/INST]
```
# Recommended Samplers:
```
Temperature: 0.6 - 1 # I recommend lowering it a litte for v4?
min_p: 0.1 - 0.2 # Crucial for NeMo
```
# Recommended Stopping Strings:
```
<|im_end|>
</s>
[/INST]
```
*Introduces run-off generations at times, as seen in v2a2. It's layered on top of older models, so eh, makes sense. Easy to cut out though.*
# Notes
\- Some people have been having issues with run-on generations for Lyra-v3. Kind of weird, when I never had issues.
<br>\- Anyway, make sure not to skip special tokens, or ban EOS tokens. I think this is the main problem that happens when v3 was to be quanted. The special tokens map config is fucked in v3, Quantizing tools likely spazzed out seeing it. I blame llamafactory for it. It ran fine unquantised.
<br>\- I like long generations, though I can control it easily to create short ones. If you're struggling, prompt better. Fix your system prompts, use an Author's Note, use a prefill. They are there for a reason.
<br>\- Lyra passes my internal benchmark suite, hence why I'm releasing it. Do I like it? Yes? it's out. that's it. They are my models for my personal enjoyment first.
<br>\- Issues like roleplay format are what I consider worthless, as it follows few-shot examples fine. This is not a priority for me to 'fix', as I see no isses with it. Same with excessive generations. Its easy to cut out.
<br>\- If you don't like it, just try another model? Plenty of other choices. Ymmv, I like it. |