Commit
·
eee093d
1
Parent(s):
46e1228
clarify benefit of prompt loss mask
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ tags: [llama, jupyter, colab, function calling, QLoRa, fine-tuning]
|
|
11 |
# *QLoRa Training Notebook for Small Datasets*
|
12 |
|
13 |
Fine-tune HuggingFace language models in Google Colab.
|
14 |
-
- Includes a prompt loss-mask for improved performance
|
15 |
- Includes a stop token after responses - allowing the model to provide a short reponse (e.g. a function call) and then stop.
|
16 |
- Allows for fine-tuning 7B and 13B models with the free plan.
|
17 |
- Allows for fine-tuning 70B models with Google Colab Pro.
|
|
|
11 |
# *QLoRa Training Notebook for Small Datasets*
|
12 |
|
13 |
Fine-tune HuggingFace language models in Google Colab.
|
14 |
+
- Includes a prompt loss-mask for improved performance when structured responses are required.
|
15 |
- Includes a stop token after responses - allowing the model to provide a short reponse (e.g. a function call) and then stop.
|
16 |
- Allows for fine-tuning 7B and 13B models with the free plan.
|
17 |
- Allows for fine-tuning 70B models with Google Colab Pro.
|