RonanMcGovern commited on
Commit
eee093d
·
1 Parent(s): 46e1228

clarify benefit of prompt loss mask

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -11,7 +11,7 @@ tags: [llama, jupyter, colab, function calling, QLoRa, fine-tuning]
11
  # *QLoRa Training Notebook for Small Datasets*
12
 
13
  Fine-tune HuggingFace language models in Google Colab.
14
- - Includes a prompt loss-mask for improved performance on small datasets.
15
  - Includes a stop token after responses - allowing the model to provide a short reponse (e.g. a function call) and then stop.
16
  - Allows for fine-tuning 7B and 13B models with the free plan.
17
  - Allows for fine-tuning 70B models with Google Colab Pro.
 
11
  # *QLoRa Training Notebook for Small Datasets*
12
 
13
  Fine-tune HuggingFace language models in Google Colab.
14
+ - Includes a prompt loss-mask for improved performance when structured responses are required.
15
  - Includes a stop token after responses - allowing the model to provide a short reponse (e.g. a function call) and then stop.
16
  - Allows for fine-tuning 7B and 13B models with the free plan.
17
  - Allows for fine-tuning 70B models with Google Colab Pro.