arabert_baseline_style_task7_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3701
• Qwk: 0.7283
• Mse: 0.3603

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	0.7720	0.3388	0.7758
No log	0.6667	4	0.5488	0.6361	0.5756
No log	1.0	6	0.6199	0.6083	0.6502
No log	1.3333	8	0.5891	0.3780	0.5962
No log	1.6667	10	0.5013	0.5556	0.5111
No log	2.0	12	0.4841	0.7640	0.4991
No log	2.3333	14	0.4386	0.784	0.4514
No log	2.6667	16	0.4343	0.6667	0.4393
No log	3.0	18	0.4876	0.6206	0.4868
No log	3.3333	20	0.4572	0.6294	0.4528
No log	3.6667	22	0.3651	0.6769	0.3604
No log	4.0	24	0.3743	0.6434	0.3662
No log	4.3333	26	0.4705	0.6261	0.4557
No log	4.6667	28	0.5940	0.5033	0.5754
No log	5.0	30	0.5564	0.5650	0.5374
No log	5.3333	32	0.4207	0.6607	0.4059
No log	5.6667	34	0.3696	0.6637	0.3577
No log	6.0	36	0.3816	0.6607	0.3687
No log	6.3333	38	0.4231	0.6261	0.4080
No log	6.6667	40	0.4191	0.6261	0.4044
No log	7.0	42	0.4158	0.6261	0.4016
No log	7.3333	44	0.4009	0.6261	0.3875
No log	7.6667	46	0.3811	0.6474	0.3694
No log	8.0	48	0.3746	0.6810	0.3639
No log	8.3333	50	0.3601	0.7283	0.3514
No log	8.6667	52	0.3557	0.7283	0.3474
No log	9.0	54	0.3591	0.7283	0.3506
No log	9.3333	56	0.3648	0.7283	0.3558
No log	9.6667	58	0.3690	0.7283	0.3593
No log	10.0	60	0.3701	0.7283	0.3603

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1