mlm-code-roman-finetuned-final

This model is a fine-tuned version of rohanrajpal/bert-base-codemixed-uncased-sentiment on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.2295
• Model Preparation Time: 0.0031

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time
4.8669	1.0	1834	4.6996	0.0031
4.2832	2.0	3668	4.1303	0.0031
3.9681	3.0	5502	3.9828	0.0031
3.8238	4.0	7336	nan	0.0031
3.8079	5.0	9170	3.6848	0.0031
3.5443	6.0	11004	nan	0.0031
3.4345	7.0	12838	3.6615	0.0031
3.3339	8.0	14672	nan	0.0031
3.2448	9.0	16506	3.5461	0.0031
3.1693	10.0	18340	3.4980	0.0031
3.2229	11.0	20174	nan	0.0031
3.0313	12.0	22008	3.4317	0.0031
2.9187	13.0	23842	3.4374	0.0031
2.7454	14.0	25676	3.3087	0.0031
2.8174	15.0	27510	nan	0.0031
2.6992	16.0	29344	3.2808	0.0031
2.7873	17.0	31178	nan	0.0031
2.7171	18.0	33012	3.3550	0.0031
2.6304	19.0	34846	3.1810	0.0031
2.5512	20.0	36680	3.4333	0.0031
2.4428	21.0	38514	3.2822	0.0031
2.4032	22.0	40348	3.2255	0.0031
2.3325	23.0	42182	nan	0.0031
2.3242	24.0	44016	nan	0.0031
2.2707	25.0	45850	3.2783	0.0031
2.2549	26.0	47684	3.1391	0.0031
2.1746	27.0	49518	3.2615	0.0031
2.1261	28.0	51352	3.3844	0.0031
2.131	29.0	53186	nan	0.0031
2.0258	30.0	55020	nan	0.0031
2.0013	31.0	56854	3.2128	0.0031
1.9835	32.0	58688	3.2159	0.0031
1.9533	33.0	60522	3.1775	0.0031
1.8876	34.0	62356	nan	0.0031
1.8624	35.0	64190	nan	0.0031
1.8181	36.0	66024	nan	0.0031
1.8068	37.0	67858	3.2109	0.0031
1.7557	38.0	69692	3.0247	0.0031
1.6394	39.0	71526	nan	0.0031
1.7325	40.0	73360	3.1127	0.0031
1.7365	41.0	75194	nan	0.0031
1.6807	42.0	77028	3.0874	0.0031
1.5981	43.0	78862	3.0828	0.0031
1.6304	44.0	80696	3.1521	0.0031
1.568	45.0	82530	3.2340	0.0031
1.5382	46.0	84364	nan	0.0031
1.387	47.0	86198	3.2172	0.0031
1.5521	48.0	88032	3.1709	0.0031
1.4808	49.0	89866	nan	0.0031
1.4629	50.0	91700	nan	0.0031

Framework versions

• Transformers 4.46.1
• Pytorch 2.2.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.1