Edit model card

arabert_baseline_style_task1_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5100
  • Qwk: 0.6698
  • Mse: 0.5053

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Qwk Mse
No log 0.3333 2 5.4651 -0.0419 5.4241
No log 0.6667 4 2.3074 0.1693 2.2743
No log 1.0 6 1.2115 0.1561 1.2001
No log 1.3333 8 0.9959 0.4358 0.9923
No log 1.6667 10 0.7290 0.4921 0.7317
No log 2.0 12 0.7145 0.5587 0.7193
No log 2.3333 14 0.7329 0.5435 0.7351
No log 2.6667 16 0.7694 0.5219 0.7660
No log 3.0 18 0.9233 0.4358 0.9210
No log 3.3333 20 0.8596 0.4516 0.8545
No log 3.6667 22 0.7445 0.5743 0.7322
No log 4.0 24 0.7760 0.4773 0.7614
No log 4.3333 26 0.6783 0.5743 0.6684
No log 4.6667 28 0.7836 0.5152 0.7834
No log 5.0 30 0.7387 0.5188 0.7401
No log 5.3333 32 0.5679 0.5743 0.5656
No log 5.6667 34 0.5183 0.5743 0.5136
No log 6.0 36 0.5055 0.5743 0.5017
No log 6.3333 38 0.5518 0.5188 0.5511
No log 6.6667 40 0.6558 0.5188 0.6585
No log 7.0 42 0.6773 0.6025 0.6812
No log 7.3333 44 0.6216 0.6025 0.6242
No log 7.6667 46 0.5382 0.6698 0.5375
No log 8.0 48 0.5083 0.6698 0.5056
No log 8.3333 50 0.5045 0.6698 0.5004
No log 8.6667 52 0.5059 0.7151 0.5009
No log 9.0 54 0.5070 0.7151 0.5018
No log 9.3333 56 0.5069 0.7151 0.5016
No log 9.6667 58 0.5086 0.7151 0.5037
No log 10.0 60 0.5100 0.6698 0.5053

Framework versions

  • Transformers 4.44.0
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
0
Safetensors
Model size
135M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for salbatarni/arabert_baseline_style_task1_fold0

Finetuned
(424)
this model