Hugging Face's logo

sizhkhy
/
akoul_whitehorseliquidity_25c

PEFT
Safetensors
llama-factory
lora
unsloth
Generated from Trainer
Model card Files Community
akoul_whitehorseliquidity_25c / README.md
sizhkhy's picture
sizhkhy
Upload folder using huggingface_hub
5f29636 verified
preview code
|
raw
history blame contribute delete
3.65 kB
metadata
library_name: peft
license: other
base_model: unsloth/Llama-3.2-3B-Instruct
tags:
  - llama-factory
  - lora
  - unsloth
  - generated_from_trainer
model-index:
  - name: llm3br256
    results: []

llm3br256

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct on the akoul_whitehorseliquidity_25c dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0008

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 32
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 25.0

Training results

Training Loss Epoch Step Validation Loss
0.0078 0.0808 25 0.0079
0.0119 0.1616 50 0.0051
0.0036 0.2424 75 0.0032
0.004 0.3232 100 0.0025
0.0019 0.4040 125 0.0020
0.0021 0.4848 150 0.0018
0.0016 0.5657 175 0.0016
0.0013 0.6465 200 0.0015
0.0017 0.7273 225 0.0015
0.0015 0.8081 250 0.0014
0.0023 0.8889 275 0.0013
0.0012 0.9697 300 0.0013
0.0011 1.0505 325 0.0013
0.0011 1.1313 350 0.0013
0.0009 1.2121 375 0.0012
0.0015 1.2929 400 0.0011
0.0025 1.3737 425 0.0011
0.0016 1.4545 450 0.0011
0.001 1.5354 475 0.0011
0.0007 1.6162 500 0.0011
0.0008 1.6970 525 0.0011
0.001 1.7778 550 0.0010
0.0007 1.8586 575 0.0010
0.0013 1.9394 600 0.0009
0.0007 2.0202 625 0.0010
0.0006 2.1010 650 0.0009
0.0007 2.1818 675 0.0009
0.001 2.2626 700 0.0009
0.0015 2.3434 725 0.0009
0.0012 2.4242 750 0.0010
0.0012 2.5051 775 0.0009
0.0015 2.5859 800 0.0010
0.0011 2.6667 825 0.0009
0.0007 2.7475 850 0.0009
0.0009 2.8283 875 0.0009
0.0009 2.9091 900 0.0008
0.001 2.9899 925 0.0009
0.0006 3.0707 950 0.0009
0.0006 3.1515 975 0.0009
0.0007 3.2323 1000 0.0009
0.0004 3.3131 1025 0.0009

Framework versions

  • PEFT 0.12.0
  • Transformers 4.46.1
  • Pytorch 2.4.0+cu121
  • Datasets 3.1.0
  • Tokenizers 0.20.3