Update README.md
Browse files
README.md
CHANGED
@@ -5,14 +5,18 @@ language:
|
|
5 |
|
6 |
**Mistral Nemo 12B Lyra v2 - Stage 1**
|
7 |
|
8 |
-
This is Stage 1 of Lyra v2. Only SFT and Merge has been performed at this stage.
|
9 |
|
10 |
This is still raw, there may be issues.
|
11 |
|
12 |
Kinda wordy and verbose like Claude is, but that is fixable in stage 2, hopefully.
|
13 |
|
|
|
|
|
14 |
---
|
15 |
|
16 |
-
\- `[/INST]` may leak, let's say that is unintended. Just simply add it to your custom stopping strings.
|
|
|
|
|
17 |
|
18 |
-
\-
|
|
|
5 |
|
6 |
**Mistral Nemo 12B Lyra v2 - Stage 1**
|
7 |
|
8 |
+
This is Stage 1 of Lyra v2. Only SFT and 2-Step Merge has been performed at this stage.
|
9 |
|
10 |
This is still raw, there may be issues.
|
11 |
|
12 |
Kinda wordy and verbose like Claude is, but that is fixable in stage 2, hopefully.
|
13 |
|
14 |
+
Stage 2 Plans - RL currently TBD
|
15 |
+
|
16 |
---
|
17 |
|
18 |
+
\- `[/INST]` may leak, let's say that is an unintended side-effect. Just simply add it to your custom stopping strings. I tried adding it as an EOS in model config but that ruined the model somehow, so I didn't.
|
19 |
+
|
20 |
+
\- min_p value of 0.1 is a sane value to use. Temp? 0.7-1.2, up to you.
|
21 |
|
22 |
+
\- Prompt Format: ChatML
|