Edit model card

xlm-roberta-xl-final-lora152520

This model is a fine-tuned version of facebook/xlm-roberta-xl on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.5503
  • Precision: 0.9267
  • Recall: 0.9291
  • F1: 0.9279
  • Accuracy: 0.9386

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • distributed_type: multi-GPU
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 40
  • num_epochs: 40
  • mixed_precision_training: Native AMP
  • label_smoothing_factor: 0.2

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
2.6885 1.0 250 1.9699 0.7944 0.8343 0.8139 0.8442
1.7948 2.0 500 1.6854 0.8702 0.8791 0.8746 0.8949
1.6148 3.0 750 1.6185 0.8827 0.8998 0.8911 0.9096
1.5365 4.0 1000 1.5710 0.9031 0.9054 0.9043 0.9195
1.4852 5.0 1250 1.5524 0.9124 0.9129 0.9126 0.9255
1.4538 6.0 1500 1.5431 0.9112 0.9176 0.9144 0.9272
1.4306 7.0 1750 1.5390 0.9145 0.9221 0.9183 0.9297
1.4132 8.0 2000 1.5358 0.9191 0.9219 0.9205 0.9321
1.4004 9.0 2250 1.5365 0.9174 0.9262 0.9218 0.9337
1.3883 10.0 2500 1.5407 0.9176 0.9263 0.9220 0.9332
1.3803 11.0 2750 1.5326 0.9218 0.9278 0.9248 0.9358
1.3727 12.0 3000 1.5353 0.9187 0.9245 0.9216 0.9329
1.3674 13.0 3250 1.5392 0.9202 0.9254 0.9228 0.9350
1.3609 14.0 3500 1.5384 0.9220 0.9259 0.9239 0.9347
1.3572 15.0 3750 1.5382 0.9201 0.9240 0.9220 0.9334
1.3522 16.0 4000 1.5410 0.9197 0.9270 0.9233 0.9342
1.3502 17.0 4250 1.5449 0.9245 0.9268 0.9256 0.9355
1.3456 18.0 4500 1.5439 0.9233 0.9278 0.9256 0.9360
1.3423 19.0 4750 1.5435 0.9259 0.9248 0.9253 0.9346
1.3419 20.0 5000 1.5432 0.9270 0.9282 0.9276 0.9371
1.3397 21.0 5250 1.5398 0.9250 0.9284 0.9267 0.9369
1.3377 22.0 5500 1.5411 0.9253 0.9270 0.9262 0.9358
1.3351 23.0 5750 1.5471 0.9274 0.9284 0.9279 0.9374
1.3369 24.0 6000 1.5542 0.9214 0.9240 0.9227 0.9339
1.3348 25.0 6250 1.5479 0.9268 0.9288 0.9278 0.9374
1.334 26.0 6500 1.5492 0.9268 0.9294 0.9281 0.9384
1.3334 27.0 6750 1.5471 0.9299 0.9287 0.9293 0.9377
1.3327 28.0 7000 1.5438 0.9291 0.9309 0.9300 0.9394
1.3314 29.0 7250 1.5445 0.9304 0.9315 0.9310 0.9403
1.3318 30.0 7500 1.5456 0.9291 0.9310 0.9301 0.9399
1.3312 31.0 7750 1.5474 0.9278 0.9295 0.9287 0.9386
1.3304 32.0 8000 1.5489 0.9273 0.9302 0.9288 0.9388
1.3298 33.0 8250 1.5469 0.9286 0.9299 0.9293 0.9388
1.3295 34.0 8500 1.5474 0.9291 0.9312 0.9302 0.9398
1.3288 35.0 8750 1.5518 0.9280 0.9300 0.9290 0.9386
1.3292 36.0 9000 1.5484 0.9271 0.9308 0.9289 0.9388
1.3287 37.0 9250 1.5487 0.9278 0.9297 0.9287 0.9382
1.328 38.0 9500 1.5492 0.9290 0.9305 0.9298 0.9394
1.3281 39.0 9750 1.5496 0.9278 0.9293 0.9285 0.9387
1.3285 40.0 10000 1.5503 0.9267 0.9291 0.9279 0.9386

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.0.1+cu118
  • Datasets 2.15.0
  • Tokenizers 0.15.0
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .

Model tree for jamesngai/xlm-roberta-xl-final-lora152520

Finetuned
(6)
this model