arabert_baseline_vocabulary_task1_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6498
• Qwk: 0.6216
• Mse: 0.6722

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	4.6252	-0.0554	4.6618
No log	0.6667	4	1.9114	0.1179	1.9451
No log	1.0	6	0.9690	0.1617	1.0110
No log	1.3333	8	0.8689	0.3771	0.9143
No log	1.6667	10	0.7767	0.3636	0.8305
No log	2.0	12	0.7835	0.3652	0.8429
No log	2.3333	14	0.7429	0.5097	0.7956
No log	2.6667	16	0.7735	0.4543	0.8152
No log	3.0	18	0.8347	0.4231	0.8732
No log	3.3333	20	0.7428	0.4755	0.7774
No log	3.6667	22	0.7764	0.5241	0.8145
No log	4.0	24	0.7551	0.5241	0.7880
No log	4.3333	26	0.7760	0.4815	0.8052
No log	4.6667	28	0.8464	0.4103	0.8794
No log	5.0	30	0.7218	0.5290	0.7519
No log	5.3333	32	0.7204	0.5241	0.7555
No log	5.6667	34	0.7510	0.5634	0.7913
No log	6.0	36	0.6866	0.5634	0.7239
No log	6.3333	38	0.6265	0.5592	0.6598
No log	6.6667	40	0.6769	0.5913	0.7100
No log	7.0	42	0.6960	0.5913	0.7277
No log	7.3333	44	0.6638	0.5913	0.6914
No log	7.6667	46	0.6521	0.5987	0.6776
No log	8.0	48	0.6739	0.6369	0.6999
No log	8.3333	50	0.6666	0.6369	0.6916
No log	8.6667	52	0.6558	0.6655	0.6793
No log	9.0	54	0.6498	0.6216	0.6729
No log	9.3333	56	0.6487	0.6216	0.6712
No log	9.6667	58	0.6495	0.6216	0.6718
No log	10.0	60	0.6498	0.6216	0.6722

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1