Edit model card

arabert_baseline_relevance_task2_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.1850
  • Qwk: 0.0597
  • Mse: 0.1935

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Qwk Mse
No log 0.3333 2 0.9871 0.0205 0.9763
No log 0.6667 4 0.1822 0.1049 0.1846
No log 1.0 6 0.2694 -0.0176 0.2773
No log 1.3333 8 0.2612 -0.0791 0.2700
No log 1.6667 10 0.2291 0.0187 0.2405
No log 2.0 12 0.1605 0.0755 0.1675
No log 2.3333 14 0.1556 0.1049 0.1617
No log 2.6667 16 0.1455 0.1370 0.1500
No log 3.0 18 0.1386 0.0 0.1414
No log 3.3333 20 0.1423 0.0 0.1448
No log 3.6667 22 0.1393 0.0 0.1429
No log 4.0 24 0.1807 0.4815 0.1875
No log 4.3333 26 0.2622 0.0 0.2718
No log 4.6667 28 0.2372 0.0 0.2476
No log 5.0 30 0.1765 -0.1290 0.1845
No log 5.3333 32 0.1633 0.0483 0.1688
No log 5.6667 34 0.1551 0.0755 0.1613
No log 6.0 36 0.1689 0.1674 0.1757
No log 6.3333 38 0.1736 0.2687 0.1798
No log 6.6667 40 0.1678 0.2150 0.1732
No log 7.0 42 0.1581 0.1250 0.1631
No log 7.3333 44 0.1540 0.0526 0.1591
No log 7.6667 46 0.1536 0.0526 0.1590
No log 8.0 48 0.1603 0.1250 0.1666
No log 8.3333 50 0.1716 0.2150 0.1790
No log 8.6667 52 0.1797 0.0597 0.1878
No log 9.0 54 0.1828 0.0597 0.1911
No log 9.3333 56 0.1841 0.0597 0.1925
No log 9.6667 58 0.1849 0.0597 0.1933
No log 10.0 60 0.1850 0.0597 0.1935

Framework versions

  • Transformers 4.44.0
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
0
Safetensors
Model size
135M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for salbatarni/arabert_baseline_relevance_task2_fold0

Finetuned
(423)
this model