Edit model card

arabert_baseline_style_task7_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4980
  • Qwk: 0.5556
  • Mse: 0.4980

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Qwk Mse
No log 0.3333 2 1.5251 0.0993 1.5251
No log 0.6667 4 0.8432 0.4389 0.8432
No log 1.0 6 1.0961 0.4310 1.0961
No log 1.3333 8 1.4643 0.4246 1.4643
No log 1.6667 10 0.7875 0.4595 0.7875
No log 2.0 12 0.5137 0.6346 0.5137
No log 2.3333 14 0.6935 0.5893 0.6935
No log 2.6667 16 0.6820 0.5547 0.6820
No log 3.0 18 0.8761 0.4868 0.8761
No log 3.3333 20 1.0195 0.4553 1.0195
No log 3.6667 22 0.7568 0.4924 0.7568
No log 4.0 24 0.5131 0.5471 0.5131
No log 4.3333 26 0.4657 0.5627 0.4657
No log 4.6667 28 0.4764 0.5471 0.4764
No log 5.0 30 0.5337 0.4983 0.5337
No log 5.3333 32 0.6627 0.4983 0.6627
No log 5.6667 34 0.6382 0.4983 0.6382
No log 6.0 36 0.5386 0.4983 0.5386
No log 6.3333 38 0.4986 0.5398 0.4986
No log 6.6667 40 0.4838 0.5471 0.4838
No log 7.0 42 0.4885 0.5471 0.4885
No log 7.3333 44 0.5346 0.5398 0.5346
No log 7.6667 46 0.6171 0.5090 0.6171
No log 8.0 48 0.6649 0.5090 0.6649
No log 8.3333 50 0.6483 0.5090 0.6483
No log 8.6667 52 0.5915 0.5090 0.5915
No log 9.0 54 0.5598 0.4983 0.5598
No log 9.3333 56 0.5257 0.5556 0.5257
No log 9.6667 58 0.5073 0.5556 0.5073
No log 10.0 60 0.4980 0.5556 0.4980

Framework versions

  • Transformers 4.44.0
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
0
Safetensors
Model size
135M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for salbatarni/arabert_baseline_style_task7_fold0

Finetuned
(424)
this model