Experiment-2

This model is a fine-tuned version of NousResearch/Llama-2-7b-hf on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6750
• Accuracy: 0.596
• Precision: 0.5869
• Recall: 0.6263
• F1: 0.6060

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
No log	0.9874	54	0.6970	0.532	0.5313	0.4785	0.5035
No log	1.9931	109	0.6923	0.508	0.5103	0.1989	0.2863
0.694	2.9989	164	0.6888	0.5413	0.5303	0.6586	0.5875
0.694	3.9863	218	0.6926	0.5187	0.6279	0.0726	0.1301
0.694	4.992	273	0.6778	0.5947	0.6269	0.4516	0.525
0.6841	5.9977	328	0.6738	0.5827	0.5582	0.7608	0.6439
0.6841	6.9851	382	0.6701	0.5893	0.6301	0.4167	0.5016
0.6841	7.9909	437	0.6717	0.6013	0.5835	0.6855	0.6304
0.6699	8.9966	492	0.6768	0.5787	0.5553	0.7554	0.6401
0.6699	9.8743	540	0.6750	0.596	0.5869	0.6263	0.6060

Framework versions

• PEFT 0.14.0
• Transformers 4.46.3
• Pytorch 2.3.1.post300
• Datasets 3.2.0
• Tokenizers 0.20.3