roberta-base-clickbait-task1-20-epoch-post_title

This model is a fine-tuned version of FacebookAI/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.5980
  • Accuracy: 0.705

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 1.0 200 0.7472 0.7
No log 2.0 400 0.7596 0.685
0.7443 3.0 600 0.7719 0.7275
0.7443 4.0 800 0.9436 0.7125
0.31 5.0 1000 1.0868 0.71
0.31 6.0 1200 1.5539 0.715
0.31 7.0 1400 1.9772 0.67
0.0987 8.0 1600 1.9892 0.7
0.0987 9.0 1800 2.1162 0.7
0.0445 10.0 2000 2.1434 0.7225
0.0445 11.0 2200 2.2596 0.7075
0.0445 12.0 2400 2.3839 0.705
0.0181 13.0 2600 2.4400 0.7025
0.0181 14.0 2800 2.4389 0.7075
0.0122 15.0 3000 2.5178 0.71
0.0122 16.0 3200 2.5055 0.7075
0.0122 17.0 3400 2.5805 0.7
0.0074 18.0 3600 2.5856 0.7075
0.0074 19.0 3800 2.5949 0.705
0.0052 20.0 4000 2.5980 0.705

Framework versions

  • Transformers 4.44.0.dev0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
15
Safetensors
Model size
125M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for abdulmanaam/roberta-base-clickbait-task1-20-epoch-post_title

Finetuned
(1370)
this model