Hugging Face's logo

jamesngai
/
xlm-roberta-xl-lora4

Generated from Trainer
Model card Files Community
jamesngai commited on
Commit
875377f
•
1 Parent(s): 9f1b142

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +115 -3
README.md CHANGED
@@ -1,9 +1,121 @@
1
  ---
2
- library_name: peft
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  ## Training procedure
5
 
6
- ### Framework versions
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
- - PEFT 0.5.0
 
 
 
 
1
  ---
2
+ license: mit
3
+ base_model: facebook/xlm-roberta-xl
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - precision
8
+ - recall
9
+ - f1
10
+ - accuracy
11
+ model-index:
12
+ - name: xlm-roberta-xl-lora4
13
+ results: []
14
  ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # xlm-roberta-xl-lora4
20
+
21
+ This model is a fine-tuned version of [facebook/xlm-roberta-xl](https://huggingface.co/facebook/xlm-roberta-xl) on an unknown dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 1.2790
24
+ - Precision: 0.9287
25
+ - Recall: 0.9301
26
+ - F1: 0.9294
27
+ - Accuracy: 0.9392
28
+
29
+ ## Model description
30
+
31
+ More information needed
32
+
33
+ ## Intended uses & limitations
34
+
35
+ More information needed
36
+
37
+ ## Training and evaluation data
38
+
39
+ More information needed
40
+
41
  ## Training procedure
42
 
43
+ ### Training hyperparameters
44
+
45
+ The following hyperparameters were used during training:
46
+ - learning_rate: 0.0001
47
+ - train_batch_size: 8
48
+ - eval_batch_size: 8
49
+ - seed: 42
50
+ - distributed_type: multi-GPU
51
+ - num_devices: 8
52
+ - total_train_batch_size: 64
53
+ - total_eval_batch_size: 64
54
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
55
+ - lr_scheduler_type: linear
56
+ - lr_scheduler_warmup_steps: 63
57
+ - num_epochs: 50
58
+ - label_smoothing_factor: 0.15
59
 
60
+ ### Training results
61
+
62
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
63
+ |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
64
+ | 3.6153 | 1.0 | 63 | 2.8304 | 0.5449 | 0.6685 | 0.6004 | 0.6459 |
65
+ | 2.4657 | 2.0 | 126 | 2.1379 | 0.7373 | 0.8004 | 0.7676 | 0.8065 |
66
+ | 1.9364 | 3.0 | 189 | 1.7624 | 0.8330 | 0.8617 | 0.8472 | 0.8740 |
67
+ | 1.6441 | 4.0 | 252 | 1.5723 | 0.8663 | 0.8816 | 0.8739 | 0.8949 |
68
+ | 1.494 | 5.0 | 315 | 1.4820 | 0.8779 | 0.8884 | 0.8831 | 0.9020 |
69
+ | 1.3987 | 6.0 | 378 | 1.4190 | 0.8961 | 0.9012 | 0.8987 | 0.9135 |
70
+ | 1.3388 | 7.0 | 441 | 1.3814 | 0.9023 | 0.9073 | 0.9048 | 0.9187 |
71
+ | 1.2947 | 8.0 | 504 | 1.3609 | 0.8976 | 0.9082 | 0.9029 | 0.9200 |
72
+ | 1.2585 | 9.0 | 567 | 1.3415 | 0.8965 | 0.9113 | 0.9038 | 0.9203 |
73
+ | 1.2317 | 10.0 | 630 | 1.3246 | 0.9095 | 0.9095 | 0.9095 | 0.9246 |
74
+ | 1.2081 | 11.0 | 693 | 1.3111 | 0.9095 | 0.9143 | 0.9119 | 0.9268 |
75
+ | 1.1869 | 12.0 | 756 | 1.3005 | 0.9161 | 0.9194 | 0.9177 | 0.9305 |
76
+ | 1.1711 | 13.0 | 819 | 1.3085 | 0.9069 | 0.9169 | 0.9119 | 0.9265 |
77
+ | 1.1557 | 14.0 | 882 | 1.2989 | 0.9191 | 0.9204 | 0.9198 | 0.9309 |
78
+ | 1.1486 | 15.0 | 945 | 1.2962 | 0.9166 | 0.9185 | 0.9176 | 0.9295 |
79
+ | 1.1392 | 16.0 | 1008 | 1.2796 | 0.9202 | 0.9228 | 0.9215 | 0.9348 |
80
+ | 1.127 | 17.0 | 1071 | 1.2830 | 0.9200 | 0.9229 | 0.9214 | 0.9341 |
81
+ | 1.1224 | 18.0 | 1134 | 1.2814 | 0.9184 | 0.9248 | 0.9216 | 0.9336 |
82
+ | 1.1146 | 19.0 | 1197 | 1.2775 | 0.9206 | 0.9260 | 0.9233 | 0.9356 |
83
+ | 1.1081 | 20.0 | 1260 | 1.2798 | 0.9251 | 0.9263 | 0.9257 | 0.9358 |
84
+ | 1.1006 | 21.0 | 1323 | 1.2756 | 0.9220 | 0.9257 | 0.9238 | 0.9364 |
85
+ | 1.0972 | 22.0 | 1386 | 1.2755 | 0.9176 | 0.9258 | 0.9217 | 0.9357 |
86
+ | 1.0926 | 23.0 | 1449 | 1.2795 | 0.9217 | 0.9267 | 0.9242 | 0.9366 |
87
+ | 1.0898 | 24.0 | 1512 | 1.2830 | 0.9213 | 0.9260 | 0.9236 | 0.9348 |
88
+ | 1.0847 | 25.0 | 1575 | 1.2749 | 0.9234 | 0.9275 | 0.9255 | 0.9377 |
89
+ | 1.0818 | 26.0 | 1638 | 1.2806 | 0.9245 | 0.9270 | 0.9257 | 0.9368 |
90
+ | 1.0796 | 27.0 | 1701 | 1.2760 | 0.9243 | 0.9283 | 0.9263 | 0.9372 |
91
+ | 1.0753 | 28.0 | 1764 | 1.2776 | 0.9220 | 0.9264 | 0.9242 | 0.9364 |
92
+ | 1.072 | 29.0 | 1827 | 1.2755 | 0.9265 | 0.9288 | 0.9276 | 0.9388 |
93
+ | 1.0686 | 30.0 | 1890 | 1.2752 | 0.9240 | 0.9246 | 0.9243 | 0.9365 |
94
+ | 1.0676 | 31.0 | 1953 | 1.2755 | 0.9271 | 0.9293 | 0.9282 | 0.9386 |
95
+ | 1.0663 | 32.0 | 2016 | 1.2771 | 0.9261 | 0.9282 | 0.9272 | 0.9383 |
96
+ | 1.0646 | 33.0 | 2079 | 1.2774 | 0.9235 | 0.9283 | 0.9259 | 0.9370 |
97
+ | 1.0641 | 34.0 | 2142 | 1.2710 | 0.9274 | 0.9313 | 0.9294 | 0.9398 |
98
+ | 1.0648 | 35.0 | 2205 | 1.2759 | 0.9259 | 0.9284 | 0.9271 | 0.9387 |
99
+ | 1.0623 | 36.0 | 2268 | 1.2741 | 0.9260 | 0.9294 | 0.9277 | 0.9383 |
100
+ | 1.06 | 37.0 | 2331 | 1.2747 | 0.9243 | 0.9293 | 0.9268 | 0.9377 |
101
+ | 1.0592 | 38.0 | 2394 | 1.2757 | 0.9262 | 0.9293 | 0.9278 | 0.9389 |
102
+ | 1.0581 | 39.0 | 2457 | 1.2794 | 0.9251 | 0.9294 | 0.9273 | 0.9379 |
103
+ | 1.0574 | 40.0 | 2520 | 1.2765 | 0.9295 | 0.9298 | 0.9296 | 0.9400 |
104
+ | 1.0569 | 41.0 | 2583 | 1.2798 | 0.9253 | 0.9281 | 0.9267 | 0.9381 |
105
+ | 1.0557 | 42.0 | 2646 | 1.2813 | 0.9282 | 0.9294 | 0.9288 | 0.9391 |
106
+ | 1.0562 | 43.0 | 2709 | 1.2792 | 0.9253 | 0.9261 | 0.9257 | 0.9366 |
107
+ | 1.056 | 44.0 | 2772 | 1.2797 | 0.9266 | 0.9293 | 0.9280 | 0.9386 |
108
+ | 1.0545 | 45.0 | 2835 | 1.2800 | 0.9265 | 0.9284 | 0.9274 | 0.9382 |
109
+ | 1.0546 | 46.0 | 2898 | 1.2788 | 0.9284 | 0.9299 | 0.9292 | 0.9394 |
110
+ | 1.0544 | 47.0 | 2961 | 1.2794 | 0.9280 | 0.9292 | 0.9286 | 0.9386 |
111
+ | 1.0539 | 48.0 | 3024 | 1.2785 | 0.9285 | 0.9299 | 0.9292 | 0.9393 |
112
+ | 1.054 | 49.0 | 3087 | 1.2791 | 0.9284 | 0.9294 | 0.9289 | 0.9390 |
113
+ | 1.0538 | 50.0 | 3150 | 1.2790 | 0.9287 | 0.9301 | 0.9294 | 0.9392 |
114
+
115
+
116
+ ### Framework versions
117
 
118
+ - Transformers 4.31.0
119
+ - Pytorch 2.1.0
120
+ - Datasets 2.14.5
121
+ - Tokenizers 0.13.3