arabert_baseline_style_task7_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4980
• Qwk: 0.5556
• Mse: 0.4980

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.5251	0.0993	1.5251
No log	0.6667	4	0.8432	0.4389	0.8432
No log	1.0	6	1.0961	0.4310	1.0961
No log	1.3333	8	1.4643	0.4246	1.4643
No log	1.6667	10	0.7875	0.4595	0.7875
No log	2.0	12	0.5137	0.6346	0.5137
No log	2.3333	14	0.6935	0.5893	0.6935
No log	2.6667	16	0.6820	0.5547	0.6820
No log	3.0	18	0.8761	0.4868	0.8761
No log	3.3333	20	1.0195	0.4553	1.0195
No log	3.6667	22	0.7568	0.4924	0.7568
No log	4.0	24	0.5131	0.5471	0.5131
No log	4.3333	26	0.4657	0.5627	0.4657
No log	4.6667	28	0.4764	0.5471	0.4764
No log	5.0	30	0.5337	0.4983	0.5337
No log	5.3333	32	0.6627	0.4983	0.6627
No log	5.6667	34	0.6382	0.4983	0.6382
No log	6.0	36	0.5386	0.4983	0.5386
No log	6.3333	38	0.4986	0.5398	0.4986
No log	6.6667	40	0.4838	0.5471	0.4838
No log	7.0	42	0.4885	0.5471	0.4885
No log	7.3333	44	0.5346	0.5398	0.5346
No log	7.6667	46	0.6171	0.5090	0.6171
No log	8.0	48	0.6649	0.5090	0.6649
No log	8.3333	50	0.6483	0.5090	0.6483
No log	8.6667	52	0.5915	0.5090	0.5915
No log	9.0	54	0.5598	0.4983	0.5598
No log	9.3333	56	0.5257	0.5556	0.5257
No log	9.6667	58	0.5073	0.5556	0.5073
No log	10.0	60	0.4980	0.5556	0.4980

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1