ArabicNewSplits7_FineTuningAraBERT_run1_AugV5_k2_task5_organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5317
• Qwk: 0.6147
• Mse: 0.5317
• Rmse: 0.7292

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.25	2	4.1135	0.0130	4.1135	2.0282
No log	0.5	4	2.2846	0.1047	2.2846	1.5115
No log	0.75	6	1.3250	-0.0030	1.3250	1.1511
No log	1.0	8	1.0522	0.1837	1.0522	1.0258
No log	1.25	10	1.0442	0.1360	1.0442	1.0219
No log	1.5	12	1.0736	0.1545	1.0736	1.0362
No log	1.75	14	1.1683	0.1454	1.1683	1.0809
No log	2.0	16	1.3520	-0.0320	1.3520	1.1628
No log	2.25	18	1.4032	0.0	1.4032	1.1846
No log	2.5	20	1.0958	0.2125	1.0958	1.0468
No log	2.75	22	0.8511	0.3476	0.8511	0.9226
No log	3.0	24	0.8826	0.3432	0.8826	0.9395
No log	3.25	26	0.8140	0.3922	0.8140	0.9022
No log	3.5	28	0.8160	0.3353	0.8160	0.9033
No log	3.75	30	0.7580	0.4409	0.7580	0.8706
No log	4.0	32	0.7589	0.4980	0.7589	0.8712
No log	4.25	34	0.7522	0.4613	0.7522	0.8673
No log	4.5	36	0.7229	0.5879	0.7229	0.8502
No log	4.75	38	0.7666	0.5674	0.7666	0.8755
No log	5.0	40	0.6977	0.5763	0.6977	0.8353
No log	5.25	42	0.6845	0.6187	0.6845	0.8273
No log	5.5	44	0.7104	0.6144	0.7104	0.8428
No log	5.75	46	0.7129	0.6052	0.7129	0.8443
No log	6.0	48	0.6838	0.5816	0.6838	0.8269
No log	6.25	50	0.6372	0.6186	0.6372	0.7983
No log	6.5	52	0.6234	0.6310	0.6234	0.7895
No log	6.75	54	0.6459	0.6215	0.6459	0.8037
No log	7.0	56	0.6598	0.6206	0.6598	0.8123
No log	7.25	58	0.6649	0.6365	0.6649	0.8154
No log	7.5	60	0.6388	0.6011	0.6388	0.7993
No log	7.75	62	0.6563	0.6564	0.6563	0.8102
No log	8.0	64	0.6015	0.6555	0.6015	0.7755
No log	8.25	66	0.6032	0.6593	0.6032	0.7766
No log	8.5	68	0.7643	0.5833	0.7643	0.8742
No log	8.75	70	0.8268	0.6312	0.8268	0.9093
No log	9.0	72	0.7127	0.6193	0.7127	0.8442
No log	9.25	74	0.6450	0.6272	0.6450	0.8031
No log	9.5	76	0.6335	0.6598	0.6335	0.7959
No log	9.75	78	0.6357	0.6570	0.6357	0.7973
No log	10.0	80	0.7572	0.6304	0.7572	0.8702
No log	10.25	82	0.9212	0.5504	0.9212	0.9598
No log	10.5	84	0.7210	0.6831	0.7210	0.8491
No log	10.75	86	0.6260	0.6988	0.6260	0.7912
No log	11.0	88	0.6695	0.6259	0.6695	0.8182
No log	11.25	90	0.6167	0.6681	0.6167	0.7853
No log	11.5	92	0.7750	0.6312	0.7750	0.8803
No log	11.75	94	0.8945	0.4995	0.8945	0.9458
No log	12.0	96	0.7172	0.6082	0.7172	0.8469
No log	12.25	98	0.6394	0.6424	0.6394	0.7996
No log	12.5	100	0.6456	0.6066	0.6456	0.8035
No log	12.75	102	0.6289	0.6649	0.6289	0.7930
No log	13.0	104	0.6887	0.6062	0.6887	0.8299
No log	13.25	106	0.6350	0.6426	0.6350	0.7969
No log	13.5	108	0.6074	0.6649	0.6074	0.7794
No log	13.75	110	0.6064	0.6349	0.6064	0.7787
No log	14.0	112	0.6658	0.6184	0.6658	0.8159
No log	14.25	114	0.6647	0.5653	0.6647	0.8153
No log	14.5	116	0.5837	0.7089	0.5837	0.7640
No log	14.75	118	0.5879	0.7239	0.5879	0.7667
No log	15.0	120	0.6079	0.6677	0.6079	0.7797
No log	15.25	122	0.5616	0.7303	0.5616	0.7494
No log	15.5	124	0.5601	0.6805	0.5601	0.7484
No log	15.75	126	0.5788	0.6460	0.5788	0.7608
No log	16.0	128	0.6200	0.6460	0.6200	0.7874
No log	16.25	130	0.6486	0.5635	0.6486	0.8053
No log	16.5	132	0.6611	0.5794	0.6611	0.8131
No log	16.75	134	0.6112	0.6460	0.6112	0.7818
No log	17.0	136	0.5856	0.6788	0.5856	0.7653
No log	17.25	138	0.5840	0.6924	0.5840	0.7642
No log	17.5	140	0.5983	0.6491	0.5983	0.7735
No log	17.75	142	0.6557	0.6811	0.6557	0.8098
No log	18.0	144	0.6277	0.6254	0.6277	0.7923
No log	18.25	146	0.6655	0.6012	0.6655	0.8158
No log	18.5	148	0.6444	0.6177	0.6444	0.8028
No log	18.75	150	0.6370	0.5714	0.6370	0.7981
No log	19.0	152	0.6244	0.5274	0.6244	0.7902
No log	19.25	154	0.6407	0.6138	0.6407	0.8004
No log	19.5	156	0.6243	0.6256	0.6243	0.7901
No log	19.75	158	0.6335	0.6444	0.6335	0.7959
No log	20.0	160	0.7065	0.6745	0.7064	0.8405
No log	20.25	162	0.6679	0.6570	0.6679	0.8172
No log	20.5	164	0.5981	0.6452	0.5981	0.7733
No log	20.75	166	0.6486	0.6282	0.6486	0.8053
No log	21.0	168	0.6551	0.6464	0.6551	0.8094
No log	21.25	170	0.6059	0.6333	0.6059	0.7784
No log	21.5	172	0.6247	0.6526	0.6247	0.7904
No log	21.75	174	0.6673	0.6154	0.6673	0.8169
No log	22.0	176	0.6872	0.5826	0.6872	0.8290
No log	22.25	178	0.6612	0.6032	0.6612	0.8132
No log	22.5	180	0.6207	0.6509	0.6207	0.7878
No log	22.75	182	0.6530	0.6218	0.6530	0.8081
No log	23.0	184	0.6115	0.6177	0.6115	0.7820
No log	23.25	186	0.6004	0.6426	0.6004	0.7749
No log	23.5	188	0.6531	0.5873	0.6531	0.8081
No log	23.75	190	0.6373	0.6194	0.6373	0.7983
No log	24.0	192	0.5724	0.6636	0.5724	0.7566
No log	24.25	194	0.5672	0.6509	0.5672	0.7531
No log	24.5	196	0.6090	0.6664	0.6090	0.7804
No log	24.75	198	0.6033	0.6536	0.6033	0.7767
No log	25.0	200	0.6090	0.6452	0.6090	0.7804
No log	25.25	202	0.6431	0.5966	0.6431	0.8019
No log	25.5	204	0.6304	0.6269	0.6304	0.7940
No log	25.75	206	0.6107	0.6546	0.6107	0.7815
No log	26.0	208	0.6722	0.6199	0.6722	0.8199
No log	26.25	210	0.7143	0.6101	0.7143	0.8452
No log	26.5	212	0.6679	0.6255	0.6679	0.8173
No log	26.75	214	0.6080	0.6164	0.6080	0.7798
No log	27.0	216	0.6258	0.6174	0.6258	0.7911
No log	27.25	218	0.6103	0.6426	0.6103	0.7812
No log	27.5	220	0.5868	0.6729	0.5868	0.7660
No log	27.75	222	0.5818	0.6788	0.5818	0.7628
No log	28.0	224	0.5923	0.6602	0.5923	0.7696
No log	28.25	226	0.5878	0.6681	0.5878	0.7667
No log	28.5	228	0.5893	0.6858	0.5893	0.7677
No log	28.75	230	0.6067	0.5770	0.6067	0.7789
No log	29.0	232	0.6073	0.5326	0.6073	0.7793
No log	29.25	234	0.5736	0.7201	0.5736	0.7574
No log	29.5	236	0.5514	0.7041	0.5514	0.7426
No log	29.75	238	0.5635	0.6623	0.5635	0.7507
No log	30.0	240	0.5790	0.6575	0.5790	0.7609
No log	30.25	242	0.5537	0.6548	0.5537	0.7441
No log	30.5	244	0.5359	0.7176	0.5359	0.7320
No log	30.75	246	0.5394	0.6958	0.5394	0.7345
No log	31.0	248	0.5418	0.6958	0.5418	0.7361
No log	31.25	250	0.5430	0.6850	0.5430	0.7369
No log	31.5	252	0.5406	0.6978	0.5406	0.7352
No log	31.75	254	0.5605	0.6519	0.5605	0.7487
No log	32.0	256	0.5790	0.6157	0.5790	0.7609
No log	32.25	258	0.5576	0.6519	0.5576	0.7467
No log	32.5	260	0.5577	0.6846	0.5577	0.7468
No log	32.75	262	0.5565	0.6748	0.5565	0.7460
No log	33.0	264	0.5525	0.7103	0.5525	0.7433
No log	33.25	266	0.5601	0.6602	0.5601	0.7484
No log	33.5	268	0.5645	0.6602	0.5645	0.7513
No log	33.75	270	0.5592	0.6927	0.5592	0.7478
No log	34.0	272	0.5666	0.6927	0.5666	0.7527
No log	34.25	274	0.5898	0.6215	0.5898	0.7680
No log	34.5	276	0.5792	0.6402	0.5792	0.7610
No log	34.75	278	0.5682	0.6725	0.5682	0.7538
No log	35.0	280	0.5221	0.7041	0.5221	0.7226
No log	35.25	282	0.5134	0.7049	0.5134	0.7165
No log	35.5	284	0.5208	0.6890	0.5208	0.7217
No log	35.75	286	0.5115	0.6778	0.5115	0.7152
No log	36.0	288	0.5052	0.7151	0.5052	0.7108
No log	36.25	290	0.5098	0.6846	0.5098	0.7140
No log	36.5	292	0.5100	0.6888	0.5100	0.7141
No log	36.75	294	0.5124	0.6890	0.5124	0.7158
No log	37.0	296	0.5300	0.6890	0.5300	0.7280
No log	37.25	298	0.5347	0.6890	0.5347	0.7312
No log	37.5	300	0.5183	0.6890	0.5183	0.7199
No log	37.75	302	0.5096	0.6995	0.5096	0.7139
No log	38.0	304	0.5142	0.6909	0.5142	0.7171
No log	38.25	306	0.5141	0.6995	0.5141	0.7170
No log	38.5	308	0.5188	0.7253	0.5188	0.7203
No log	38.75	310	0.5237	0.6778	0.5237	0.7237
No log	39.0	312	0.5306	0.6778	0.5306	0.7284
No log	39.25	314	0.5303	0.6890	0.5303	0.7282
No log	39.5	316	0.5255	0.6778	0.5255	0.7249
No log	39.75	318	0.5207	0.6737	0.5207	0.7216
No log	40.0	320	0.5222	0.7253	0.5222	0.7226
No log	40.25	322	0.5317	0.7285	0.5317	0.7292
No log	40.5	324	0.5282	0.7285	0.5282	0.7268
No log	40.75	326	0.5220	0.7305	0.5220	0.7225
No log	41.0	328	0.5317	0.7001	0.5317	0.7292
No log	41.25	330	0.5409	0.6854	0.5409	0.7355
No log	41.5	332	0.5497	0.6570	0.5497	0.7414
No log	41.75	334	0.5504	0.6570	0.5504	0.7419
No log	42.0	336	0.5425	0.6553	0.5425	0.7365
No log	42.25	338	0.5370	0.6709	0.5370	0.7328
No log	42.5	340	0.5302	0.6581	0.5302	0.7282
No log	42.75	342	0.5239	0.6627	0.5239	0.7238
No log	43.0	344	0.5251	0.6882	0.5251	0.7247
No log	43.25	346	0.5372	0.6716	0.5372	0.7329
No log	43.5	348	0.5602	0.7222	0.5602	0.7484
No log	43.75	350	0.5829	0.7056	0.5829	0.7635
No log	44.0	352	0.5963	0.7056	0.5963	0.7722
No log	44.25	354	0.6157	0.7009	0.6157	0.7847
No log	44.5	356	0.6063	0.7009	0.6063	0.7787
No log	44.75	358	0.5930	0.7009	0.5930	0.7701
No log	45.0	360	0.5849	0.7009	0.5849	0.7648
No log	45.25	362	0.6042	0.6883	0.6042	0.7773
No log	45.5	364	0.5784	0.7056	0.5784	0.7605
No log	45.75	366	0.5755	0.7056	0.5755	0.7586
No log	46.0	368	0.5578	0.7222	0.5578	0.7468
No log	46.25	370	0.5287	0.7326	0.5287	0.7271
No log	46.5	372	0.5301	0.7001	0.5301	0.7281
No log	46.75	374	0.5325	0.7001	0.5325	0.7297
No log	47.0	376	0.5141	0.6882	0.5141	0.7170
No log	47.25	378	0.5072	0.6770	0.5072	0.7122
No log	47.5	380	0.5083	0.6888	0.5083	0.7130
No log	47.75	382	0.5033	0.7041	0.5033	0.7094
No log	48.0	384	0.5033	0.7201	0.5033	0.7094
No log	48.25	386	0.5027	0.7201	0.5027	0.7090
No log	48.5	388	0.5036	0.7201	0.5036	0.7097
No log	48.75	390	0.5034	0.7201	0.5034	0.7095
No log	49.0	392	0.5051	0.6940	0.5051	0.7107
No log	49.25	394	0.5041	0.6853	0.5041	0.7100
No log	49.5	396	0.5214	0.6501	0.5214	0.7221
No log	49.75	398	0.5286	0.6562	0.5286	0.7270
No log	50.0	400	0.5117	0.6536	0.5117	0.7154
No log	50.25	402	0.5030	0.7201	0.5030	0.7092
No log	50.5	404	0.5055	0.7033	0.5055	0.7110
No log	50.75	406	0.5073	0.7033	0.5073	0.7123
No log	51.0	408	0.5108	0.7033	0.5108	0.7147
No log	51.25	410	0.5133	0.7139	0.5133	0.7165
No log	51.5	412	0.5129	0.7352	0.5129	0.7162
No log	51.75	414	0.5283	0.6886	0.5283	0.7268
No log	52.0	416	0.5759	0.6272	0.5759	0.7589
No log	52.25	418	0.6138	0.6592	0.6138	0.7835
No log	52.5	420	0.5981	0.6420	0.5981	0.7734
No log	52.75	422	0.5469	0.6758	0.5469	0.7395
No log	53.0	424	0.5278	0.6553	0.5278	0.7265
No log	53.25	426	0.5438	0.6598	0.5438	0.7374
No log	53.5	428	0.5809	0.7179	0.5809	0.7622
No log	53.75	430	0.6145	0.6878	0.6145	0.7839
No log	54.0	432	0.5943	0.7056	0.5943	0.7709
No log	54.25	434	0.5704	0.6830	0.5704	0.7553
No log	54.5	436	0.5655	0.6822	0.5655	0.7520
No log	54.75	438	0.5528	0.6882	0.5528	0.7435
No log	55.0	440	0.5456	0.6911	0.5456	0.7386
No log	55.25	442	0.5454	0.6681	0.5454	0.7385
No log	55.5	444	0.5402	0.6681	0.5402	0.7350
No log	55.75	446	0.5333	0.7151	0.5333	0.7303
No log	56.0	448	0.5300	0.6838	0.5300	0.7280
No log	56.25	450	0.5254	0.6838	0.5254	0.7249
No log	56.5	452	0.5218	0.6896	0.5218	0.7224
No log	56.75	454	0.5201	0.6896	0.5201	0.7212
No log	57.0	456	0.5192	0.6896	0.5192	0.7206
No log	57.25	458	0.5188	0.6896	0.5188	0.7203
No log	57.5	460	0.5220	0.6896	0.5220	0.7225
No log	57.75	462	0.5194	0.6896	0.5194	0.7207
No log	58.0	464	0.5205	0.6940	0.5205	0.7214
No log	58.25	466	0.5208	0.6940	0.5208	0.7216
No log	58.5	468	0.5246	0.6433	0.5246	0.7243
No log	58.75	470	0.5319	0.6398	0.5319	0.7293
No log	59.0	472	0.5265	0.6333	0.5265	0.7256
No log	59.25	474	0.5263	0.6510	0.5263	0.7255
No log	59.5	476	0.5161	0.6510	0.5161	0.7184
No log	59.75	478	0.5054	0.6813	0.5054	0.7109
No log	60.0	480	0.5045	0.6813	0.5045	0.7103
No log	60.25	482	0.5075	0.6813	0.5075	0.7124
No log	60.5	484	0.5164	0.6969	0.5164	0.7186
No log	60.75	486	0.5160	0.6969	0.5160	0.7183
No log	61.0	488	0.5136	0.6969	0.5136	0.7166
No log	61.25	490	0.5125	0.6969	0.5125	0.7159
No log	61.5	492	0.5148	0.6491	0.5148	0.7175
No log	61.75	494	0.5109	0.6737	0.5109	0.7148
No log	62.0	496	0.5056	0.6882	0.5056	0.7110
No log	62.25	498	0.5056	0.7201	0.5056	0.7110
0.2005	62.5	500	0.5092	0.7201	0.5092	0.7136
0.2005	62.75	502	0.5162	0.7089	0.5162	0.7185
0.2005	63.0	504	0.5177	0.6911	0.5177	0.7195
0.2005	63.25	506	0.5297	0.6256	0.5297	0.7278
0.2005	63.5	508	0.5340	0.6147	0.5340	0.7308
0.2005	63.75	510	0.5317	0.6147	0.5317	0.7292

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1