Felladrin commited on
Commit
a2f772f
·
1 Parent(s): 144dc13

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -35
README.md CHANGED
@@ -22,15 +22,14 @@ widget:
22
  inference:
23
  parameters:
24
  max_new_tokens: 64
25
- repetition_penalty: 1.18
 
26
  ---
27
 
28
  # Locutusque's TinyMistral-248M trained on OpenAssistant TOP-1 Conversation Threads
29
 
30
- - Base model: [Locutusque/TinyMistral-248M](https://huggingface.co/Locutusque/TinyMistral-248M/blob/90b89d18fdf27937dc04ab8a9b543c5af2991c7f/README.md)
31
  - Dataset: [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25)
32
- - Availability in other ML formats:
33
- - GGUF: [afrideva/TinyMistral-248M-SFT-v3-GGUF](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF)
34
 
35
  ## Recommended Prompt Format
36
 
@@ -39,34 +38,3 @@ inference:
39
  {message}<|im_end|>
40
  <|im_start|>assistant
41
  ```
42
-
43
- ## How it was trained
44
-
45
- ```ipython
46
- %pip install autotrain-advanced
47
-
48
- !autotrain setup
49
-
50
- !autotrain llm \
51
- --train \
52
- --trainer "sft" \
53
- --model './TinyMistral-248M/' \
54
- --model_max_length 4096 \
55
- --block-size 1024 \
56
- --project-name 'trained-model' \
57
- --data-path "OpenAssistant/oasst_top1_2023-08-25" \
58
- --train_split "train" \
59
- --valid_split "test" \
60
- --text-column "text" \
61
- --lr 1e-5 \
62
- --train_batch_size 2 \
63
- --epochs 5 \
64
- --evaluation_strategy "steps" \
65
- --save-strategy "steps" \
66
- --save-total-limit 2 \
67
- --warmup-ratio 0.05 \
68
- --weight-decay 0.0 \
69
- --gradient-accumulation 8 \
70
- --logging-steps 10 \
71
- --scheduler "constant"
72
- ```
 
22
  inference:
23
  parameters:
24
  max_new_tokens: 64
25
+ penalty_alpha: 0.5
26
+ top_k: 12
27
  ---
28
 
29
  # Locutusque's TinyMistral-248M trained on OpenAssistant TOP-1 Conversation Threads
30
 
31
+ - Base model: [Locutusque/TinyMistral-248M](https://huggingface.co/Locutusque/TinyMistral-248M)
32
  - Dataset: [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25)
 
 
33
 
34
  ## Recommended Prompt Format
35
 
 
38
  {message}<|im_end|>
39
  <|im_start|>assistant
40
  ```