Update README.md
Browse files
README.md
CHANGED
@@ -22,15 +22,14 @@ widget:
|
|
22 |
inference:
|
23 |
parameters:
|
24 |
max_new_tokens: 64
|
25 |
-
|
|
|
26 |
---
|
27 |
|
28 |
# Locutusque's TinyMistral-248M trained on OpenAssistant TOP-1 Conversation Threads
|
29 |
|
30 |
-
- Base model: [Locutusque/TinyMistral-248M](https://huggingface.co/Locutusque/TinyMistral-248M
|
31 |
- Dataset: [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25)
|
32 |
-
- Availability in other ML formats:
|
33 |
-
- GGUF: [afrideva/TinyMistral-248M-SFT-v3-GGUF](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF)
|
34 |
|
35 |
## Recommended Prompt Format
|
36 |
|
@@ -39,34 +38,3 @@ inference:
|
|
39 |
{message}<|im_end|>
|
40 |
<|im_start|>assistant
|
41 |
```
|
42 |
-
|
43 |
-
## How it was trained
|
44 |
-
|
45 |
-
```ipython
|
46 |
-
%pip install autotrain-advanced
|
47 |
-
|
48 |
-
!autotrain setup
|
49 |
-
|
50 |
-
!autotrain llm \
|
51 |
-
--train \
|
52 |
-
--trainer "sft" \
|
53 |
-
--model './TinyMistral-248M/' \
|
54 |
-
--model_max_length 4096 \
|
55 |
-
--block-size 1024 \
|
56 |
-
--project-name 'trained-model' \
|
57 |
-
--data-path "OpenAssistant/oasst_top1_2023-08-25" \
|
58 |
-
--train_split "train" \
|
59 |
-
--valid_split "test" \
|
60 |
-
--text-column "text" \
|
61 |
-
--lr 1e-5 \
|
62 |
-
--train_batch_size 2 \
|
63 |
-
--epochs 5 \
|
64 |
-
--evaluation_strategy "steps" \
|
65 |
-
--save-strategy "steps" \
|
66 |
-
--save-total-limit 2 \
|
67 |
-
--warmup-ratio 0.05 \
|
68 |
-
--weight-decay 0.0 \
|
69 |
-
--gradient-accumulation 8 \
|
70 |
-
--logging-steps 10 \
|
71 |
-
--scheduler "constant"
|
72 |
-
```
|
|
|
22 |
inference:
|
23 |
parameters:
|
24 |
max_new_tokens: 64
|
25 |
+
penalty_alpha: 0.5
|
26 |
+
top_k: 12
|
27 |
---
|
28 |
|
29 |
# Locutusque's TinyMistral-248M trained on OpenAssistant TOP-1 Conversation Threads
|
30 |
|
31 |
+
- Base model: [Locutusque/TinyMistral-248M](https://huggingface.co/Locutusque/TinyMistral-248M)
|
32 |
- Dataset: [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25)
|
|
|
|
|
33 |
|
34 |
## Recommended Prompt Format
|
35 |
|
|
|
38 |
{message}<|im_end|>
|
39 |
<|im_start|>assistant
|
40 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|