arabert_baseline_mechanics_task7_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3972
• Qwk: 0.6491
• Mse: 0.3972

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.3247	0.0816	1.3247
No log	0.6667	4	0.6683	0.4335	0.6683
No log	1.0	6	0.5559	0.4980	0.5559
No log	1.3333	8	0.6243	0.4454	0.6243
No log	1.6667	10	0.4715	0.6195	0.4715
No log	2.0	12	0.5116	0.6491	0.5116
No log	2.3333	14	0.6116	0.5702	0.6116
No log	2.6667	16	0.6154	0.5631	0.6154
No log	3.0	18	0.6351	0.4762	0.6351
No log	3.3333	20	0.4519	0.5248	0.4519
No log	3.6667	22	0.2987	0.6600	0.2987
No log	4.0	24	0.2664	0.6715	0.2664
No log	4.3333	26	0.3049	0.6491	0.3049
No log	4.6667	28	0.5042	0.5601	0.5042
No log	5.0	30	0.7298	0.4401	0.7298
No log	5.3333	32	0.7112	0.4401	0.7112
No log	5.6667	34	0.4277	0.6780	0.4277
No log	6.0	36	0.2880	0.7239	0.2880
No log	6.3333	38	0.2886	0.7239	0.2886
No log	6.6667	40	0.3219	0.7239	0.3219
No log	7.0	42	0.4290	0.6329	0.4290
No log	7.3333	44	0.5828	0.5497	0.5828
No log	7.6667	46	0.6367	0.5129	0.6367
No log	8.0	48	0.5900	0.5497	0.5900
No log	8.3333	50	0.4796	0.5955	0.4796
No log	8.6667	52	0.4272	0.6491	0.4272
No log	9.0	54	0.4126	0.6491	0.4126
No log	9.3333	56	0.4004	0.6491	0.4004
No log	9.6667	58	0.3997	0.6491	0.3997
No log	10.0	60	0.3972	0.6491	0.3972

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1