LoneStriker
commited on
Commit
•
8169190
1
Parent(s):
fd64287
Added note on repetition penalty setting
Browse files
README.md
CHANGED
@@ -9,6 +9,8 @@ datasets:
|
|
9 |
# Fine-tune of Y-34B with Spicyboros-3.1
|
10 |
One epoch of fine tuning with @jondurbin's SpicyBoros-3.1 dataset. 4.65bpw should fit on a single 3090/4090, 5.0bpw, 6.0bpw, and 8.0bpw will require more than one GPU 24 GB VRAM GPU.
|
11 |
|
|
|
|
|
12 |
# Original Yi-34B Model Card Below
|
13 |
<div align="center">
|
14 |
|
@@ -54,4 +56,4 @@ Although we use data compliance checking algorithms during the training process
|
|
54 |
## License
|
55 |
|
56 |
The Yi series model must be adhere to the [Model License Agreement](https://huggingface.co/01-ai/Yi-34B/blob/main/LICENSE).
|
57 |
-
For any questions related to licensing and copyright, please contact us ([[email protected]](mailto:[email protected])).
|
|
|
9 |
# Fine-tune of Y-34B with Spicyboros-3.1
|
10 |
One epoch of fine tuning with @jondurbin's SpicyBoros-3.1 dataset. 4.65bpw should fit on a single 3090/4090, 5.0bpw, 6.0bpw, and 8.0bpw will require more than one GPU 24 GB VRAM GPU.
|
11 |
|
12 |
+
**Please note:** you may have to turn down repetition penalty to 1.0. The model seems to get into "thesaurus" mode sometimes without this change.
|
13 |
+
|
14 |
# Original Yi-34B Model Card Below
|
15 |
<div align="center">
|
16 |
|
|
|
56 |
## License
|
57 |
|
58 |
The Yi series model must be adhere to the [Model License Agreement](https://huggingface.co/01-ai/Yi-34B/blob/main/LICENSE).
|
59 |
+
For any questions related to licensing and copyright, please contact us ([[email protected]](mailto:[email protected])).
|