xlm-roberta_AGRO

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1553
• Exact Match: 28.8571
• F1 Score: 57.3777

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 3407
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1 Score
5.8392	0.0053	1	5.8105	0.0	8.3419
5.8343	0.0107	2	5.7849	0.0	8.4316
5.7993	0.0160	3	5.7441	0.0	8.4504
5.7869	0.0214	4	5.6912	0.0	8.5621
5.7257	0.0267	5	5.5845	0.0	8.7289
5.6119	0.0321	6	5.3128	0.0	9.0780
5.4565	0.0374	7	5.0020	0.0	9.9612
5.3734	0.0428	8	4.8134	0.2256	11.2348
5.276	0.0481	9	4.6671	0.3008	14.1031
5.1939	0.0535	10	4.5397	0.6767	18.7252
5.1236	0.0588	11	4.4828	1.5789	24.3832
4.9665	0.0641	12	4.4752	2.8571	30.7203
4.9217	0.0695	13	4.4690	3.8346	35.5977
4.7769	0.0748	14	4.4478	4.6617	40.6799
4.7217	0.0802	15	4.4240	6.5414	45.6464
4.6351	0.0855	16	4.4011	7.7444	48.5045
4.6804	0.0909	17	4.3807	8.5714	50.6061
4.5868	0.0962	18	4.3575	9.9248	51.9678
4.6173	0.1016	19	4.3315	11.4286	53.0364
4.5892	0.1069	20	4.3008	12.9323	54.0593
4.5701	0.1123	21	4.2633	14.5113	54.4900
4.6351	0.1176	22	4.2220	16.8421	55.2744
4.429	0.1230	23	4.1739	17.7444	55.4881
4.4307	0.1283	24	4.1211	18.7970	55.8574
4.329	0.1336	25	4.0618	19.4737	55.9693
4.3469	0.1390	26	3.9998	20.8271	56.2430
4.2975	0.1443	27	3.9353	21.9549	56.6353
4.3071	0.1497	28	3.8643	23.6842	57.2411
4.1047	0.1550	29	3.7867	24.2857	57.1472
4.1373	0.1604	30	3.7061	25.2632	57.1898
4.1239	0.1657	31	3.6258	26.4662	57.5706
3.959	0.1711	32	3.5479	27.5188	57.7931
3.7877	0.1764	33	3.4758	28.0451	57.7765
3.8374	0.1818	34	3.4128	29.0226	58.1740
3.8019	0.1871	35	3.3584	30.1504	58.6550
3.7692	0.1924	36	3.3101	30.8271	58.9151
3.6371	0.1978	37	3.2669	32.0301	59.2735
3.8148	0.2031	38	3.2291	32.1805	59.2572
3.7487	0.2085	39	3.1963	32.5564	59.2920
3.6244	0.2138	40	3.1672	32.1053	59.4804
3.6659	0.2192	41	3.1432	31.2030	59.2762
3.4609	0.2245	42	3.1235	30.0752	59.1610
3.5882	0.2299	43	3.1077	29.5489	58.9967

Framework versions

• Transformers 4.46.3
• Pytorch 2.4.0
• Datasets 3.1.0
• Tokenizers 0.20.3