---
license: mit
base_model: facebook/xlm-roberta-xl
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: xlm-roberta-xl-lora4
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xlm-roberta-xl-lora4

This model is a fine-tuned version of [facebook/xlm-roberta-xl](https://huggingface.co/facebook/xlm-roberta-xl) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2790
- Precision: 0.9287
- Recall: 0.9301
- F1: 0.9294
- Accuracy: 0.9392

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- total_train_batch_size: 64
- total_eval_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 63
- num_epochs: 50
- label_smoothing_factor: 0.15

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 3.6153        | 1.0   | 63   | 2.8304          | 0.5449    | 0.6685 | 0.6004 | 0.6459   |
| 2.4657        | 2.0   | 126  | 2.1379          | 0.7373    | 0.8004 | 0.7676 | 0.8065   |
| 1.9364        | 3.0   | 189  | 1.7624          | 0.8330    | 0.8617 | 0.8472 | 0.8740   |
| 1.6441        | 4.0   | 252  | 1.5723          | 0.8663    | 0.8816 | 0.8739 | 0.8949   |
| 1.494         | 5.0   | 315  | 1.4820          | 0.8779    | 0.8884 | 0.8831 | 0.9020   |
| 1.3987        | 6.0   | 378  | 1.4190          | 0.8961    | 0.9012 | 0.8987 | 0.9135   |
| 1.3388        | 7.0   | 441  | 1.3814          | 0.9023    | 0.9073 | 0.9048 | 0.9187   |
| 1.2947        | 8.0   | 504  | 1.3609          | 0.8976    | 0.9082 | 0.9029 | 0.9200   |
| 1.2585        | 9.0   | 567  | 1.3415          | 0.8965    | 0.9113 | 0.9038 | 0.9203   |
| 1.2317        | 10.0  | 630  | 1.3246          | 0.9095    | 0.9095 | 0.9095 | 0.9246   |
| 1.2081        | 11.0  | 693  | 1.3111          | 0.9095    | 0.9143 | 0.9119 | 0.9268   |
| 1.1869        | 12.0  | 756  | 1.3005          | 0.9161    | 0.9194 | 0.9177 | 0.9305   |
| 1.1711        | 13.0  | 819  | 1.3085          | 0.9069    | 0.9169 | 0.9119 | 0.9265   |
| 1.1557        | 14.0  | 882  | 1.2989          | 0.9191    | 0.9204 | 0.9198 | 0.9309   |
| 1.1486        | 15.0  | 945  | 1.2962          | 0.9166    | 0.9185 | 0.9176 | 0.9295   |
| 1.1392        | 16.0  | 1008 | 1.2796          | 0.9202    | 0.9228 | 0.9215 | 0.9348   |
| 1.127         | 17.0  | 1071 | 1.2830          | 0.9200    | 0.9229 | 0.9214 | 0.9341   |
| 1.1224        | 18.0  | 1134 | 1.2814          | 0.9184    | 0.9248 | 0.9216 | 0.9336   |
| 1.1146        | 19.0  | 1197 | 1.2775          | 0.9206    | 0.9260 | 0.9233 | 0.9356   |
| 1.1081        | 20.0  | 1260 | 1.2798          | 0.9251    | 0.9263 | 0.9257 | 0.9358   |
| 1.1006        | 21.0  | 1323 | 1.2756          | 0.9220    | 0.9257 | 0.9238 | 0.9364   |
| 1.0972        | 22.0  | 1386 | 1.2755          | 0.9176    | 0.9258 | 0.9217 | 0.9357   |
| 1.0926        | 23.0  | 1449 | 1.2795          | 0.9217    | 0.9267 | 0.9242 | 0.9366   |
| 1.0898        | 24.0  | 1512 | 1.2830          | 0.9213    | 0.9260 | 0.9236 | 0.9348   |
| 1.0847        | 25.0  | 1575 | 1.2749          | 0.9234    | 0.9275 | 0.9255 | 0.9377   |
| 1.0818        | 26.0  | 1638 | 1.2806          | 0.9245    | 0.9270 | 0.9257 | 0.9368   |
| 1.0796        | 27.0  | 1701 | 1.2760          | 0.9243    | 0.9283 | 0.9263 | 0.9372   |
| 1.0753        | 28.0  | 1764 | 1.2776          | 0.9220    | 0.9264 | 0.9242 | 0.9364   |
| 1.072         | 29.0  | 1827 | 1.2755          | 0.9265    | 0.9288 | 0.9276 | 0.9388   |
| 1.0686        | 30.0  | 1890 | 1.2752          | 0.9240    | 0.9246 | 0.9243 | 0.9365   |
| 1.0676        | 31.0  | 1953 | 1.2755          | 0.9271    | 0.9293 | 0.9282 | 0.9386   |
| 1.0663        | 32.0  | 2016 | 1.2771          | 0.9261    | 0.9282 | 0.9272 | 0.9383   |
| 1.0646        | 33.0  | 2079 | 1.2774          | 0.9235    | 0.9283 | 0.9259 | 0.9370   |
| 1.0641        | 34.0  | 2142 | 1.2710          | 0.9274    | 0.9313 | 0.9294 | 0.9398   |
| 1.0648        | 35.0  | 2205 | 1.2759          | 0.9259    | 0.9284 | 0.9271 | 0.9387   |
| 1.0623        | 36.0  | 2268 | 1.2741          | 0.9260    | 0.9294 | 0.9277 | 0.9383   |
| 1.06          | 37.0  | 2331 | 1.2747          | 0.9243    | 0.9293 | 0.9268 | 0.9377   |
| 1.0592        | 38.0  | 2394 | 1.2757          | 0.9262    | 0.9293 | 0.9278 | 0.9389   |
| 1.0581        | 39.0  | 2457 | 1.2794          | 0.9251    | 0.9294 | 0.9273 | 0.9379   |
| 1.0574        | 40.0  | 2520 | 1.2765          | 0.9295    | 0.9298 | 0.9296 | 0.9400   |
| 1.0569        | 41.0  | 2583 | 1.2798          | 0.9253    | 0.9281 | 0.9267 | 0.9381   |
| 1.0557        | 42.0  | 2646 | 1.2813          | 0.9282    | 0.9294 | 0.9288 | 0.9391   |
| 1.0562        | 43.0  | 2709 | 1.2792          | 0.9253    | 0.9261 | 0.9257 | 0.9366   |
| 1.056         | 44.0  | 2772 | 1.2797          | 0.9266    | 0.9293 | 0.9280 | 0.9386   |
| 1.0545        | 45.0  | 2835 | 1.2800          | 0.9265    | 0.9284 | 0.9274 | 0.9382   |
| 1.0546        | 46.0  | 2898 | 1.2788          | 0.9284    | 0.9299 | 0.9292 | 0.9394   |
| 1.0544        | 47.0  | 2961 | 1.2794          | 0.9280    | 0.9292 | 0.9286 | 0.9386   |
| 1.0539        | 48.0  | 3024 | 1.2785          | 0.9285    | 0.9299 | 0.9292 | 0.9393   |
| 1.054         | 49.0  | 3087 | 1.2791          | 0.9284    | 0.9294 | 0.9289 | 0.9390   |
| 1.0538        | 50.0  | 3150 | 1.2790          | 0.9287    | 0.9301 | 0.9294 | 0.9392   |


### Framework versions

- Transformers 4.31.0
- Pytorch 2.1.0
- Datasets 2.14.5
- Tokenizers 0.13.3