UIT-roberta-base-finetuned

This model is a fine-tuned version of FacebookAI/roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4545
• F1: 0.7327
• Roc Auc: 0.7990
• Accuracy: 0.4675

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 100
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
0.4684	1.0	139	0.4204	0.5966	0.7123	0.3736
0.4025	2.0	278	0.3654	0.6855	0.7572	0.4170
0.294	3.0	417	0.3667	0.6847	0.7556	0.4422
0.1918	4.0	556	0.3991	0.6904	0.7583	0.4368
0.1612	5.0	695	0.3956	0.7051	0.7696	0.4549
0.1231	6.0	834	0.4398	0.6869	0.7518	0.4278
0.0931	7.0	973	0.4745	0.7143	0.7814	0.4477
0.0628	8.0	1112	0.4545	0.7327	0.7990	0.4675
0.0494	9.0	1251	0.4986	0.6965	0.7653	0.4567
0.0549	10.0	1390	0.5357	0.7155	0.7819	0.4422
0.0307	11.0	1529	0.5767	0.7245	0.7862	0.4585
0.0216	12.0	1668	0.5965	0.7129	0.7772	0.4621
0.0164	13.0	1807	0.6145	0.7206	0.7814	0.4477
0.0137	14.0	1946	0.6378	0.7236	0.7904	0.4495
0.0102	15.0	2085	0.6641	0.7133	0.7814	0.4458
0.0084	16.0	2224	0.6668	0.7157	0.7832	0.4440
0.0063	17.0	2363	0.6920	0.7171	0.7842	0.4531
0.0098	18.0	2502	0.6977	0.6989	0.7698	0.4350
0.0039	19.0	2641	0.7074	0.7087	0.7782	0.4495
0.0041	20.0	2780	0.7103	0.7240	0.7889	0.4621
0.0031	21.0	2919	0.7077	0.7299	0.7941	0.4657
0.003	22.0	3058	0.7196	0.7196	0.7865	0.4531
0.0032	23.0	3197	0.7227	0.7220	0.7885	0.4639
0.0027	24.0	3336	0.7324	0.7194	0.7853	0.4567
0.0027	25.0	3475	0.7337	0.7212	0.7864	0.4549
0.0029	26.0	3614	0.7363	0.7229	0.7875	0.4585
0.0026	27.0	3753	0.7365	0.7224	0.7880	0.4567
0.0028	28.0	3892	0.7374	0.7212	0.7865	0.4585
0.0027	29.0	4031	0.7375	0.7216	0.7867	0.4567
0.0024	30.0	4170	0.7378	0.7216	0.7867	0.4567

Framework versions

• Transformers 4.48.1
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.21.0