arabert_baseline_development_task6_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4958
• Qwk: 0.5513
• Mse: 0.4958

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.5	2	0.8754	0.1390	0.8754
No log	1.0	4	0.7157	0.5039	0.7157
No log	1.5	6	0.6589	0.3713	0.6589
No log	2.0	8	0.8933	0.2857	0.8933
No log	2.5	10	0.4415	0.5679	0.4415
No log	3.0	12	0.7544	0.5532	0.7544
No log	3.5	14	0.6791	0.4380	0.6791
No log	4.0	16	0.5398	0.4762	0.5398
No log	4.5	18	0.5209	0.5096	0.5209
No log	5.0	20	0.5172	0.5395	0.5172
No log	5.5	22	0.6042	0.4085	0.6042
No log	6.0	24	0.6539	0.5205	0.6539
No log	6.5	26	0.6064	0.4901	0.6064
No log	7.0	28	0.5186	0.4762	0.5186
No log	7.5	30	0.4734	0.5395	0.4734
No log	8.0	32	0.4610	0.5395	0.4610
No log	8.5	34	0.4597	0.5395	0.4597
No log	9.0	36	0.4714	0.5395	0.4714
No log	9.5	38	0.4877	0.5513	0.4877
No log	10.0	40	0.4958	0.5513	0.4958

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1