Hugging Face's logo

krishnapal2308
/
mistral-instruct-7b-finetuned-sroie

Text Generation
PEFT
TensorBoard
Safetensors
trl
sft
conversational
Model card Files Metrics Community
mistral-instruct-7b-finetuned-sroie / README.md
krishnapal2308's picture
krishnapal2308
Update README.md
78e8a51 verified
preview code
|
raw
history blame
1.88 kB
metadata
base_model: mistralai/Mistral-7B-Instruct-v0.1
datasets:
  - krishnapal2308/SROIE
library_name: peft
license: apache-2.0
tags:
  - trl
  - sft
model-index:
  - name: mistral-instruct-7b-finetuned-sroie
    results: []
pipeline_tag: text-generation

Visualize in Weights & Biases

mistral-instruct-7b-finetuned-sroie

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.1 on the generator dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7668

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_steps: 50
  • training_steps: 120
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.4458 0.4848 20 1.3313
1.1796 0.9697 40 1.1143
1.0555 1.4545 60 0.9733
0.9514 1.9394 80 0.8829
0.8599 2.4242 100 0.8150
0.857 2.9091 120 0.7668

Framework versions

  • PEFT 0.12.0
  • Transformers 4.42.4
  • Pytorch 2.3.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1