Hugging Face's logo

Utshav
/
c4ai-command-r-v01-SFT

PEFT
Safetensors
trl
sft
Generated from Trainer
Model card Files Community
c4ai-command-r-v01-SFT / README.md
Utshav's picture
Utshav
Utshav/command-4-v01-Finetuned
69bbb84 verified
preview code
|
raw
history blame contribute delete
4.07 kB
metadata
license: cc-by-nc-4.0
library_name: peft
tags:
  - trl
  - sft
  - generated_from_trainer
base_model: CohereForAI/c4ai-command-r-v01-4bit
datasets:
  - generator
model-index:
  - name: c4ai-command-r-v01-SFT
    results: []

c4ai-command-r-v01-SFT

This model is a fine-tuned version of CohereForAI/c4ai-command-r-v01-4bit on the generator dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6709

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2.5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 0.03
  • training_steps: 500

Training results

Training Loss Epoch Step Validation Loss
1.5906 0.2326 10 1.5675
1.5542 0.4651 20 1.4713
1.3934 0.6977 30 1.3839
1.346 0.9302 40 1.2949
1.2605 1.1628 50 1.2154
1.1887 1.3953 60 1.1361
1.0784 1.6279 70 1.0520
1.0086 1.8605 80 0.9770
0.9247 2.0930 90 0.9134
0.8671 2.3256 100 0.8640
0.8391 2.5581 110 0.8284
0.7849 2.7907 120 0.8026
0.7458 3.0233 130 0.7818
0.7377 3.2558 140 0.7684
0.7112 3.4884 150 0.7544
0.7054 3.7209 160 0.7430
0.7029 3.9535 170 0.7331
0.657 4.1860 180 0.7263
0.675 4.4186 190 0.7189
0.6695 4.6512 200 0.7117
0.6418 4.8837 210 0.7058
0.6348 5.1163 220 0.7028
0.6414 5.3488 230 0.6981
0.612 5.5814 240 0.6951
0.6114 5.8140 250 0.6909
0.6149 6.0465 260 0.6876
0.5978 6.2791 270 0.6884
0.5955 6.5116 280 0.6839
0.6112 6.7442 290 0.6802
0.5841 6.9767 300 0.6794
0.5746 7.2093 310 0.6801
0.5849 7.4419 320 0.6773
0.5863 7.6744 330 0.6760
0.5738 7.9070 340 0.6748
0.5696 8.1395 350 0.6739
0.5632 8.3721 360 0.6747
0.5732 8.6047 370 0.6737
0.5512 8.8372 380 0.6725
0.5761 9.0698 390 0.6716
0.5471 9.3023 400 0.6727
0.5579 9.5349 410 0.6724
0.573 9.7674 420 0.6714
0.5459 10.0 430 0.6708
0.5677 10.2326 440 0.6710
0.5453 10.4651 450 0.6708
0.5638 10.6977 460 0.6708
0.5473 10.9302 470 0.6709
0.5553 11.1628 480 0.6709
0.5535 11.3953 490 0.6708
0.5409 11.6279 500 0.6709

Framework versions

  • PEFT 0.10.1.dev0
  • Transformers 4.40.0.dev0
  • Pytorch 2.1.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.19.0