Hugging Face's logo

haryoaw
/
scenario-NON-KD-SCR-COPY-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual

Text Classification
Transformers
PyTorch
Safetensors
deberta-v2
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
haryoaw commited on
Commit
a88c593
•
1 Parent(s): bc8f2a6

Initial Commit

Browse files
Files changed (5) hide show
  1. README.md +64 -65
  2. config.json +1 -1
  3. eval_results_cardiff.json +1 -1
  4. pytorch_model.bin +2 -2
  5. training_args.bin +2 -2
README.md CHANGED
@@ -1,20 +1,19 @@
1
  ---
 
2
  base_model: microsoft/mdeberta-v3-base
 
 
3
  datasets:
4
  - tweet_sentiment_multilingual
5
- library_name: transformers
6
- license: mit
7
  metrics:
8
  - accuracy
9
  - f1
10
- tags:
11
- - generated_from_trainer
12
  model-index:
13
  - name: scenario-NON-KD-SCR-COPY-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual
14
  results:
15
  - task:
16
- type: text-classification
17
  name: Text Classification
 
18
  dataset:
19
  name: tweet_sentiment_multilingual
20
  type: tweet_sentiment_multilingual
@@ -22,12 +21,12 @@ model-index:
22
  split: validation
23
  args: all
24
  metrics:
25
- - type: accuracy
26
- value: 0.4965277777777778
27
- name: Accuracy
28
- - type: f1
29
- value: 0.49536382507680443
30
- name: F1
31
  ---
32
 
33
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -37,9 +36,9 @@ should probably proofread and complete it, then remove this comment. -->
37
 
38
  This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the tweet_sentiment_multilingual dataset.
39
  It achieves the following results on the evaluation set:
40
- - Loss: 6.4409
41
- - Accuracy: 0.4965
42
- - F1: 0.4954
43
 
44
  ## Model description
45
 
@@ -61,66 +60,66 @@ The following hyperparameters were used during training:
61
  - learning_rate: 5e-05
62
  - train_batch_size: 32
63
  - eval_batch_size: 32
64
- - seed: 44
65
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
66
  - lr_scheduler_type: linear
67
  - num_epochs: 50
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
72
- |:-------------:|:-------:|:-----:|:---------------:|:--------:|:------:|
73
- | 1.0485 | 1.0870 | 500 | 1.0669 | 0.5108 | 0.5023 |
74
- | 0.751 | 2.1739 | 1000 | 1.4193 | 0.5100 | 0.5095 |
75
- | 0.4107 | 3.2609 | 1500 | 1.6735 | 0.5185 | 0.5189 |
76
- | 0.2216 | 4.3478 | 2000 | 2.0242 | 0.5185 | 0.5115 |
77
- | 0.1366 | 5.4348 | 2500 | 2.2238 | 0.5081 | 0.5068 |
78
- | 0.0976 | 6.5217 | 3000 | 2.7800 | 0.5062 | 0.5068 |
79
- | 0.073 | 7.6087 | 3500 | 2.7186 | 0.5085 | 0.5062 |
80
- | 0.0566 | 8.6957 | 4000 | 3.2897 | 0.4985 | 0.4971 |
81
- | 0.0506 | 9.7826 | 4500 | 3.2041 | 0.5093 | 0.5093 |
82
- | 0.0398 | 10.8696 | 5000 | 3.7832 | 0.5100 | 0.5050 |
83
- | 0.0283 | 11.9565 | 5500 | 3.5997 | 0.5050 | 0.5052 |
84
- | 0.0209 | 13.0435 | 6000 | 4.1365 | 0.5050 | 0.5037 |
85
- | 0.0226 | 14.1304 | 6500 | 4.0530 | 0.5104 | 0.5074 |
86
- | 0.0238 | 15.2174 | 7000 | 3.9582 | 0.5066 | 0.5073 |
87
- | 0.0166 | 16.3043 | 7500 | 4.0643 | 0.5050 | 0.5052 |
88
- | 0.0187 | 17.3913 | 8000 | 3.6702 | 0.5089 | 0.5077 |
89
- | 0.014 | 18.4783 | 8500 | 4.4844 | 0.5039 | 0.5013 |
90
- | 0.0083 | 19.5652 | 9000 | 4.5825 | 0.5073 | 0.5077 |
91
- | 0.0079 | 20.6522 | 9500 | 5.0765 | 0.5035 | 0.5026 |
92
- | 0.0108 | 21.7391 | 10000 | 4.8560 | 0.5123 | 0.5109 |
93
- | 0.0085 | 22.8261 | 10500 | 5.0581 | 0.5058 | 0.5052 |
94
- | 0.008 | 23.9130 | 11000 | 5.2723 | 0.5039 | 0.5014 |
95
- | 0.0063 | 25.0 | 11500 | 4.7063 | 0.5162 | 0.5162 |
96
- | 0.008 | 26.0870 | 12000 | 4.8183 | 0.5066 | 0.5056 |
97
- | 0.0043 | 27.1739 | 12500 | 5.2933 | 0.5050 | 0.5039 |
98
- | 0.003 | 28.2609 | 13000 | 5.6372 | 0.5058 | 0.5042 |
99
- | 0.006 | 29.3478 | 13500 | 4.9934 | 0.5069 | 0.5060 |
100
- | 0.0033 | 30.4348 | 14000 | 5.2603 | 0.5058 | 0.5029 |
101
- | 0.0037 | 31.5217 | 14500 | 5.5455 | 0.5019 | 0.5022 |
102
- | 0.0045 | 32.6087 | 15000 | 5.2618 | 0.5054 | 0.5000 |
103
- | 0.0034 | 33.6957 | 15500 | 5.2386 | 0.5031 | 0.5033 |
104
- | 0.0008 | 34.7826 | 16000 | 5.7685 | 0.5027 | 0.5026 |
105
- | 0.0007 | 35.8696 | 16500 | 5.9128 | 0.5062 | 0.5033 |
106
- | 0.002 | 36.9565 | 17000 | 5.9399 | 0.4969 | 0.4969 |
107
- | 0.001 | 38.0435 | 17500 | 6.0256 | 0.5066 | 0.5049 |
108
- | 0.0007 | 39.1304 | 18000 | 6.0998 | 0.5100 | 0.5088 |
109
- | 0.0003 | 40.2174 | 18500 | 6.3641 | 0.5008 | 0.5009 |
110
- | 0.0013 | 41.3043 | 19000 | 6.1494 | 0.5035 | 0.5036 |
111
- | 0.0005 | 42.3913 | 19500 | 6.0497 | 0.5104 | 0.5084 |
112
- | 0.0008 | 43.4783 | 20000 | 6.2485 | 0.5019 | 0.5017 |
113
- | 0.0005 | 44.5652 | 20500 | 6.2400 | 0.5004 | 0.5001 |
114
- | 0.0003 | 45.6522 | 21000 | 6.2941 | 0.5019 | 0.5013 |
115
- | 0.0009 | 46.7391 | 21500 | 6.4453 | 0.4969 | 0.4959 |
116
- | 0.0011 | 47.8261 | 22000 | 6.3999 | 0.4981 | 0.4968 |
117
- | 0.0006 | 48.9130 | 22500 | 6.4380 | 0.4954 | 0.4942 |
118
- | 0.0007 | 50.0 | 23000 | 6.4409 | 0.4965 | 0.4954 |
119
 
120
 
121
  ### Framework versions
122
 
123
- - Transformers 4.44.2
124
  - Pytorch 2.1.1+cu121
125
  - Datasets 2.14.5
126
- - Tokenizers 0.19.1
 
1
  ---
2
+ license: mit
3
  base_model: microsoft/mdeberta-v3-base
4
+ tags:
5
+ - generated_from_trainer
6
  datasets:
7
  - tweet_sentiment_multilingual
 
 
8
  metrics:
9
  - accuracy
10
  - f1
 
 
11
  model-index:
12
  - name: scenario-NON-KD-SCR-COPY-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual
13
  results:
14
  - task:
 
15
  name: Text Classification
16
+ type: text-classification
17
  dataset:
18
  name: tweet_sentiment_multilingual
19
  type: tweet_sentiment_multilingual
 
21
  split: validation
22
  args: all
23
  metrics:
24
+ - name: Accuracy
25
+ type: accuracy
26
+ value: 0.4957561728395062
27
+ - name: F1
28
+ type: f1
29
+ value: 0.4963309489168229
30
  ---
31
 
32
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
36
 
37
  This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the tweet_sentiment_multilingual dataset.
38
  It achieves the following results on the evaluation set:
39
+ - Loss: 6.7830
40
+ - Accuracy: 0.4958
41
+ - F1: 0.4963
42
 
43
  ## Model description
44
 
 
60
  - learning_rate: 5e-05
61
  - train_batch_size: 32
62
  - eval_batch_size: 32
63
+ - seed: 66
64
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
65
  - lr_scheduler_type: linear
66
  - num_epochs: 50
67
 
68
  ### Training results
69
 
70
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
71
+ |:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
72
+ | 1.0563 | 1.09 | 500 | 1.0531 | 0.4950 | 0.4889 |
73
+ | 0.7646 | 2.17 | 1000 | 1.2908 | 0.5131 | 0.5122 |
74
+ | 0.4473 | 3.26 | 1500 | 1.6933 | 0.5158 | 0.5165 |
75
+ | 0.2465 | 4.35 | 2000 | 2.2069 | 0.5135 | 0.5126 |
76
+ | 0.1511 | 5.43 | 2500 | 2.5482 | 0.5081 | 0.5082 |
77
+ | 0.121 | 6.52 | 3000 | 2.8075 | 0.5123 | 0.5107 |
78
+ | 0.0834 | 7.61 | 3500 | 3.1416 | 0.5085 | 0.5082 |
79
+ | 0.061 | 8.7 | 4000 | 2.8363 | 0.5093 | 0.5083 |
80
+ | 0.0496 | 9.78 | 4500 | 3.3908 | 0.5174 | 0.5162 |
81
+ | 0.0394 | 10.87 | 5000 | 3.6362 | 0.5123 | 0.5126 |
82
+ | 0.0305 | 11.96 | 5500 | 4.0351 | 0.5035 | 0.5047 |
83
+ | 0.0283 | 13.04 | 6000 | 4.0528 | 0.5031 | 0.5042 |
84
+ | 0.0184 | 14.13 | 6500 | 4.2723 | 0.5039 | 0.5045 |
85
+ | 0.0217 | 15.22 | 7000 | 4.2612 | 0.4981 | 0.4977 |
86
+ | 0.0193 | 16.3 | 7500 | 4.3257 | 0.4907 | 0.4915 |
87
+ | 0.0196 | 17.39 | 8000 | 4.6089 | 0.4904 | 0.4906 |
88
+ | 0.0154 | 18.48 | 8500 | 4.6472 | 0.4927 | 0.4935 |
89
+ | 0.014 | 19.57 | 9000 | 4.4510 | 0.4981 | 0.4982 |
90
+ | 0.0177 | 20.65 | 9500 | 4.2732 | 0.4907 | 0.4911 |
91
+ | 0.0114 | 21.74 | 10000 | 4.5261 | 0.4931 | 0.4921 |
92
+ | 0.0099 | 22.83 | 10500 | 4.9751 | 0.4888 | 0.4901 |
93
+ | 0.0073 | 23.91 | 11000 | 4.4316 | 0.4927 | 0.4923 |
94
+ | 0.0081 | 25.0 | 11500 | 4.8393 | 0.4942 | 0.4940 |
95
+ | 0.0039 | 26.09 | 12000 | 5.2291 | 0.4988 | 0.4958 |
96
+ | 0.0052 | 27.17 | 12500 | 5.1648 | 0.4931 | 0.4942 |
97
+ | 0.0065 | 28.26 | 13000 | 5.1350 | 0.4919 | 0.4924 |
98
+ | 0.0042 | 29.35 | 13500 | 5.2707 | 0.4988 | 0.4971 |
99
+ | 0.0033 | 30.43 | 14000 | 5.2902 | 0.4896 | 0.4911 |
100
+ | 0.0041 | 31.52 | 14500 | 5.3182 | 0.4958 | 0.4971 |
101
+ | 0.002 | 32.61 | 15000 | 5.4473 | 0.4961 | 0.4968 |
102
+ | 0.001 | 33.7 | 15500 | 5.7540 | 0.4942 | 0.4952 |
103
+ | 0.0016 | 34.78 | 16000 | 5.8709 | 0.4958 | 0.4929 |
104
+ | 0.001 | 35.87 | 16500 | 6.1489 | 0.4938 | 0.4936 |
105
+ | 0.0012 | 36.96 | 17000 | 6.4545 | 0.4942 | 0.4942 |
106
+ | 0.0011 | 38.04 | 17500 | 6.4864 | 0.4946 | 0.4936 |
107
+ | 0.0024 | 39.13 | 18000 | 6.2903 | 0.5012 | 0.4998 |
108
+ | 0.001 | 40.22 | 18500 | 6.2566 | 0.4954 | 0.4950 |
109
+ | 0.0002 | 41.3 | 19000 | 6.3660 | 0.4954 | 0.4955 |
110
+ | 0.001 | 42.39 | 19500 | 6.4778 | 0.4954 | 0.4923 |
111
+ | 0.0001 | 43.48 | 20000 | 6.5401 | 0.4985 | 0.4981 |
112
+ | 0.0002 | 44.57 | 20500 | 6.6695 | 0.5 | 0.4992 |
113
+ | 0.0 | 45.65 | 21000 | 6.7149 | 0.5012 | 0.5004 |
114
+ | 0.0001 | 46.74 | 21500 | 6.7514 | 0.5015 | 0.5011 |
115
+ | 0.0008 | 47.83 | 22000 | 6.7485 | 0.4958 | 0.4964 |
116
+ | 0.0001 | 48.91 | 22500 | 6.7745 | 0.4961 | 0.4968 |
117
+ | 0.0001 | 50.0 | 23000 | 6.7830 | 0.4958 | 0.4963 |
118
 
119
 
120
  ### Framework versions
121
 
122
+ - Transformers 4.33.3
123
  - Pytorch 2.1.1+cu121
124
  - Datasets 2.14.5
125
+ - Tokenizers 0.13.3
config.json CHANGED
@@ -39,7 +39,7 @@
39
  "relative_attention": true,
40
  "share_att_key": true,
41
  "torch_dtype": "float32",
42
- "transformers_version": "4.44.2",
43
  "type_vocab_size": 0,
44
  "vocab_size": 251000
45
  }
 
39
  "relative_attention": true,
40
  "share_att_key": true,
41
  "torch_dtype": "float32",
42
+ "transformers_version": "4.33.3",
43
  "type_vocab_size": 0,
44
  "vocab_size": 251000
45
  }
eval_results_cardiff.json CHANGED
@@ -1 +1 @@
1
- {"arabic": {"f1": 0.47947772610260553, "accuracy": 0.4781609195402299, "confusion_matrix": [[128, 107, 55], [87, 154, 49], [44, 112, 134]]}, "english": {"f1": 0.445334621975829, "accuracy": 0.4494252873563218, "confusion_matrix": [[156, 76, 58], [116, 91, 83], [70, 76, 144]]}, "french": {"f1": 0.5464246834532506, "accuracy": 0.5482758620689655, "confusion_matrix": [[166, 76, 48], [44, 185, 61], [53, 111, 126]]}, "german": {"f1": 0.5362629677705776, "accuracy": 0.539080459770115, "confusion_matrix": [[125, 85, 80], [59, 167, 64], [41, 72, 177]]}, "hindi": {"f1": 0.42370528574630323, "accuracy": 0.4264367816091954, "confusion_matrix": [[139, 68, 83], [88, 94, 108], [73, 79, 138]]}, "italian": {"f1": 0.48027685815233223, "accuracy": 0.49310344827586206, "confusion_matrix": [[85, 119, 86], [27, 193, 70], [44, 95, 151]]}, "portuguese": {"f1": 0.573249888651446, "accuracy": 0.5747126436781609, "confusion_matrix": [[159, 77, 54], [72, 145, 73], [32, 62, 196]]}, "spanish": {"f1": 0.5039814966278203, "accuracy": 0.503448275862069, "confusion_matrix": [[141, 100, 49], [83, 127, 80], [39, 81, 170]]}, "all": {"f1": 0.5019772050336919, "accuracy": 0.5015804597701149, "confusion_matrix": [[1099, 708, 513], [576, 1156, 588], [396, 688, 1236]]}}
 
1
+ {"arabic": {"f1": 0.47089949037256823, "accuracy": 0.4689655172413793, "confusion_matrix": [[133, 120, 37], [97, 153, 40], [71, 97, 122]]}, "english": {"f1": 0.46593250950336423, "accuracy": 0.46781609195402296, "confusion_matrix": [[153, 84, 53], [110, 105, 75], [82, 59, 149]]}, "french": {"f1": 0.534964240110648, "accuracy": 0.535632183908046, "confusion_matrix": [[164, 71, 55], [42, 170, 78], [67, 91, 132]]}, "german": {"f1": 0.5509262447152427, "accuracy": 0.5517241379310345, "confusion_matrix": [[143, 79, 68], [60, 173, 57], [57, 69, 164]]}, "hindi": {"f1": 0.44474479625697233, "accuracy": 0.44482758620689655, "confusion_matrix": [[135, 79, 76], [77, 127, 86], [84, 81, 125]]}, "italian": {"f1": 0.4734988640051931, "accuracy": 0.49195402298850577, "confusion_matrix": [[72, 131, 87], [11, 194, 85], [32, 96, 162]]}, "portuguese": {"f1": 0.5575518820795515, "accuracy": 0.5597701149425287, "confusion_matrix": [[170, 71, 49], [82, 131, 77], [45, 59, 186]]}, "spanish": {"f1": 0.5131583962747177, "accuracy": 0.5126436781609195, "confusion_matrix": [[147, 92, 51], [81, 138, 71], [57, 72, 161]]}, "all": {"f1": 0.5044375419579221, "accuracy": 0.5041666666666667, "confusion_matrix": [[1117, 727, 476], [560, 1191, 569], [495, 624, 1201]]}}
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:307f60615c3fec1907c8e173c4dcf69b6300fa4d284bad1ed6302a50c7ddb627
3
- size 942111086
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9680fd0665f2031db41c7f1d19be5b8fdc1bbd78377279b01f46491da5375543
3
+ size 945174322
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:20a6aa6aac39b04c9c386a5259edba6ee2d9f85f160266110308de366eba5b12
3
- size 5368
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:534b6d504b1d4129997135ebf6daae33d3a21de1f665c5be67a95eff367d11ec
3
+ size 4664