Update README.md

README.md

@@ -12,6 +12,8 @@ Differences in the qlora scripts:
 
 __I think there's a bug in gradient accumulation, so if you try this, maybe set gradient accumulation steps to 1__
 
+__5 epochs seemed to achieve the best results, but YMMV__
+
 Full example of tuning (used for airoboros-mpt-30b-gpt4-1.4):
 
 ```
@@ -23,11 +25,11 @@ export WANDB_PROJECT=airoboros-mpt-30b-gpt4-1.4
 python qlora.py \
     --model_name_or_path ./mpt-30b \
     --output_dir ./$WANDB_PROJECT-checkpoints \
-    --num_train_epochs 3 \
+    --num_train_epochs 5 \
     --logging_steps 1 \
     --save_strategy steps \
     --data_seed 11422 \
-    --save_steps 75 \
+    --save_steps 100 \
     --save_total_limit 3 \
     --evaluation_strategy "no" \
     --eval_dataset_size 2 \