pretrain_model

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6196
  • Precision: 0.6607
  • Recall: 0.6589
  • F1: 0.6598
  • Accuracy: 0.6575

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 2

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
0.6965 0.1377 500 0.6910 0.526 1.0 0.6894 0.526
0.6963 0.2755 1000 0.6921 0.526 1.0 0.6894 0.526
0.6957 0.4132 1500 0.6666 0.6154 0.7300 0.6678 0.618
0.6914 0.5510 2000 0.6834 0.7069 0.4677 0.5629 0.618
0.6768 0.6887 2500 0.6838 0.6412 0.6388 0.64 0.622
0.6786 0.8264 3000 0.6539 0.7273 0.4259 0.5372 0.614
0.663 0.9642 3500 0.6743 0.6560 0.5437 0.5946 0.61
0.6564 1.1019 4000 0.6381 0.6763 0.6198 0.6468 0.644
0.6468 1.2397 4500 0.6010 0.6613 0.7871 0.7188 0.676
0.6275 1.3774 5000 0.6103 0.7246 0.5703 0.6383 0.66
0.6275 1.5152 5500 0.6018 0.7311 0.5894 0.6526 0.67
0.6141 1.6529 6000 0.5947 0.7269 0.6578 0.6906 0.69
0.617 1.7906 6500 0.5872 0.7165 0.6920 0.7041 0.694
0.6059 1.9284 7000 0.5816 0.7227 0.7034 0.7129 0.702

Framework versions

  • Transformers 4.46.3
  • Pytorch 2.5.1+cu121
  • Datasets 3.1.0
  • Tokenizers 0.20.3
Downloads last month
4
Safetensors
Model size
125M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for charisgao/pretrain_model

Finetuned
(1696)
this model