twitter-roberta-base-sentiment-latest_12112024T120259

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-sentiment-latest on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4912
• F1: 0.8803
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 600
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Rate
No log	0.9942	86	1.7812	0.1230	0.0000
No log	2.0	173	1.6480	0.3596	0.0000
No log	2.9942	259	1.3462	0.4748	0.0000
No log	4.0	346	1.1145	0.5480	0.0000
No log	4.9942	432	0.9935	0.5877	0.0000
1.3977	6.0	519	0.9023	0.6390	0.0000
1.3977	6.9942	605	0.8538	0.6798	1e-05
1.3977	8.0	692	0.7381	0.7388	1e-05
1.3977	8.9942	778	0.6558	0.7696	0.0000
1.3977	10.0	865	0.6072	0.7963	0.0000
1.3977	10.9942	951	0.5771	0.8139	0.0000
0.6178	12.0	1038	0.5692	0.8270	0.0000
0.6178	12.9942	1124	0.5208	0.8513	0.0000
0.6178	14.0	1211	0.5416	0.8487	0.0000
0.6178	14.9942	1297	0.5073	0.8655	0.0000
0.6178	16.0	1384	0.5052	0.8740	0.0000
0.6178	16.9942	1470	0.4912	0.8803	6e-06
0.2205	18.0	1557	0.5557	0.8700	0.0000
0.2205	18.9942	1643	0.5021	0.8845	0.0000
0.2205	20.0	1730	0.5382	0.8837	0.0000
0.2205	20.9942	1816	0.6147	0.8730	0.0000
0.2205	22.0	1903	0.5978	0.8762	0.0000
0.2205	22.9942	1989	0.6037	0.8756	0.0000
0.0833	24.0	2076	0.6226	0.8755	0.0000
0.0833	24.9942	2162	0.6136	0.8777	0.0000
0.0833	26.0	2249	0.5938	0.8815	7e-07
0.0833	26.9942	2335	0.6318	0.8766	4e-07
0.0833	28.0	2422	0.6302	0.8783	2e-07
0.0462	28.9942	2508	0.6325	0.8777	0.0
0.0462	29.8266	2580	0.6322	0.8777	0.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.19.1