Hugging Face's logo

simecek
/
DNADebertaK6f

Fill-Mask
Transformers
PyTorch
deberta
Generated from Trainer
Inference Endpoints
Model card Files Community
simecek commited on
Commit
f95ec6e
•
1 Parent(s): b1c460e

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +175 -0
README.md ADDED
@@ -0,0 +1,175 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: DNADebertaK6f
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # DNADebertaK6f
13
+
14
+ This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 1.3707
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 5e-05
36
+ - train_batch_size: 64
37
+ - eval_batch_size: 64
38
+ - seed: 42
39
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
+ - lr_scheduler_type: linear
41
+ - num_epochs: 150
42
+ - mixed_precision_training: Native AMP
43
+
44
+ ### Training results
45
+
46
+ | Training Loss | Epoch | Step | Validation Loss |
47
+ |:-------------:|:------:|:--------:|:---------------:|
48
+ | 2.1545 | 1.24 | 100000 | 1.5895 |
49
+ | 1.5727 | 2.49 | 200000 | 1.5383 |
50
+ | 1.5368 | 3.73 | 300000 | 1.5117 |
51
+ | 1.5143 | 4.97 | 400000 | 1.4926 |
52
+ | 1.4969 | 6.22 | 500000 | 1.4789 |
53
+ | 1.4841 | 7.46 | 600000 | 1.4677 |
54
+ | 1.475 | 8.7 | 700000 | 1.4599 |
55
+ | 1.4677 | 9.95 | 800000 | 1.4533 |
56
+ | 1.4623 | 11.19 | 900000 | 1.4491 |
57
+ | 1.4576 | 12.43 | 1000000 | 1.4461 |
58
+ | 1.4544 | 13.68 | 1100000 | 1.4420 |
59
+ | 1.451 | 14.92 | 1200000 | 1.4381 |
60
+ | 1.4482 | 16.16 | 1300000 | 1.4359 |
61
+ | 1.446 | 17.4 | 1400000 | 1.4345 |
62
+ | 1.443 | 18.65 | 1500000 | 1.4320 |
63
+ | 1.4412 | 19.89 | 1600000 | 1.4295 |
64
+ | 1.4385 | 21.13 | 1700000 | 1.4278 |
65
+ | 1.4368 | 22.38 | 1800000 | 1.4249 |
66
+ | 1.4346 | 23.62 | 1900000 | 1.4237 |
67
+ | 1.433 | 24.86 | 2000000 | 1.4219 |
68
+ | 1.4315 | 26.11 | 2100000 | 1.4201 |
69
+ | 1.4297 | 27.35 | 2200000 | 1.4198 |
70
+ | 1.4282 | 28.59 | 2300000 | 1.4180 |
71
+ | 1.4266 | 29.84 | 2400000 | 1.4142 |
72
+ | 1.4253 | 31.08 | 2500000 | 1.4146 |
73
+ | 1.4238 | 32.32 | 2600000 | 1.4130 |
74
+ | 1.4228 | 33.57 | 2700000 | 1.4113 |
75
+ | 1.4221 | 34.81 | 2800000 | 1.4100 |
76
+ | 1.42 | 36.05 | 2900000 | 1.4097 |
77
+ | 1.4188 | 37.3 | 3000000 | 1.4085 |
78
+ | 1.4174 | 38.54 | 3100000 | 1.4067 |
79
+ | 1.4161 | 39.78 | 3200000 | 1.4064 |
80
+ | 1.4149 | 41.03 | 3300000 | 1.4058 |
81
+ | 1.4139 | 42.27 | 3400000 | 1.4024 |
82
+ | 1.4134 | 43.51 | 3500000 | 1.4022 |
83
+ | 1.4126 | 44.76 | 3600000 | 1.4025 |
84
+ | 1.4117 | 46.0 | 3700000 | 1.4015 |
85
+ | 1.411 | 47.24 | 3800000 | 1.4001 |
86
+ | 1.4098 | 48.49 | 3900000 | 1.3968 |
87
+ | 1.4096 | 49.73 | 4000000 | 1.3997 |
88
+ | 1.4089 | 50.97 | 4100000 | 1.3974 |
89
+ | 1.4084 | 52.21 | 4200000 | 1.3972 |
90
+ | 1.4072 | 53.46 | 4300000 | 1.3965 |
91
+ | 1.4066 | 54.7 | 4400000 | 1.3974 |
92
+ | 1.4062 | 55.94 | 4500000 | 1.3960 |
93
+ | 1.4058 | 57.19 | 4600000 | 1.3958 |
94
+ | 1.4053 | 58.43 | 4700000 | 1.3950 |
95
+ | 1.4041 | 59.67 | 4800000 | 1.3936 |
96
+ | 1.4041 | 60.92 | 4900000 | 1.3963 |
97
+ | 1.4031 | 62.16 | 5000000 | 1.3915 |
98
+ | 1.4023 | 63.4 | 5100000 | 1.3917 |
99
+ | 1.4022 | 64.65 | 5200000 | 1.3930 |
100
+ | 1.4017 | 65.89 | 5300000 | 1.3904 |
101
+ | 1.4009 | 67.13 | 5400000 | 1.3899 |
102
+ | 1.4007 | 68.38 | 5500000 | 1.3892 |
103
+ | 1.3997 | 69.62 | 5600000 | 1.3910 |
104
+ | 1.3996 | 70.86 | 5700000 | 1.3892 |
105
+ | 1.3991 | 72.11 | 5800000 | 1.3890 |
106
+ | 1.3983 | 73.35 | 5900000 | 1.3870 |
107
+ | 1.3985 | 74.59 | 6000000 | 1.3889 |
108
+ | 1.3975 | 75.84 | 6100000 | 1.3865 |
109
+ | 1.3973 | 77.08 | 6200000 | 1.3852 |
110
+ | 1.3969 | 78.32 | 6300000 | 1.3869 |
111
+ | 1.3964 | 79.57 | 6400000 | 1.3843 |
112
+ | 1.396 | 80.81 | 6500000 | 1.3853 |
113
+ | 1.3955 | 82.05 | 6600000 | 1.3844 |
114
+ | 1.3952 | 83.3 | 6700000 | 1.3863 |
115
+ | 1.395 | 84.54 | 6800000 | 1.3835 |
116
+ | 1.3948 | 85.78 | 6900000 | 1.3841 |
117
+ | 1.394 | 87.02 | 7000000 | 1.3850 |
118
+ | 1.3934 | 88.27 | 7100000 | 1.3827 |
119
+ | 1.3932 | 89.51 | 7200000 | 1.3830 |
120
+ | 1.3929 | 90.75 | 7300000 | 1.3821 |
121
+ | 1.392 | 92.0 | 7400000 | 1.3820 |
122
+ | 1.392 | 93.24 | 7500000 | 1.3837 |
123
+ | 1.3913 | 94.48 | 7600000 | 1.3817 |
124
+ | 1.3909 | 95.73 | 7700000 | 1.3836 |
125
+ | 1.3906 | 96.97 | 7800000 | 1.3811 |
126
+ | 1.3903 | 98.21 | 7900000 | 1.3806 |
127
+ | 1.3902 | 99.46 | 8000000 | 1.3807 |
128
+ | 1.3896 | 100.7 | 8100000 | 1.3804 |
129
+ | 1.3895 | 101.94 | 8200000 | 1.3805 |
130
+ | 1.3891 | 103.19 | 8300000 | 1.3821 |
131
+ | 1.3889 | 104.43 | 8400000 | 1.3833 |
132
+ | 1.3881 | 105.67 | 8500000 | 1.3788 |
133
+ | 1.388 | 106.92 | 8600000 | 1.3818 |
134
+ | 1.3876 | 108.16 | 8700000 | 1.3806 |
135
+ | 1.387 | 109.4 | 8800000 | 1.3766 |
136
+ | 1.387 | 110.65 | 8900000 | 1.3765 |
137
+ | 1.3865 | 111.89 | 9000000 | 1.3800 |
138
+ | 1.3864 | 113.13 | 9100000 | 1.3830 |
139
+ | 1.386 | 114.38 | 9200000 | 1.3770 |
140
+ | 1.3853 | 115.62 | 9300000 | 1.3771 |
141
+ | 1.3852 | 116.86 | 9400000 | 1.3772 |
142
+ | 1.3848 | 118.1 | 9500000 | 1.3771 |
143
+ | 1.384 | 119.35 | 9600000 | 1.3749 |
144
+ | 1.3843 | 120.59 | 9700000 | 1.3764 |
145
+ | 1.3836 | 121.83 | 9800000 | 1.3802 |
146
+ | 1.3833 | 123.08 | 9900000 | 1.3756 |
147
+ | 1.3831 | 124.32 | 10000000 | 1.3748 |
148
+ | 1.3821 | 125.56 | 10100000 | 1.3755 |
149
+ | 1.3817 | 126.81 | 10200000 | 1.3744 |
150
+ | 1.3819 | 128.05 | 10300000 | 1.3763 |
151
+ | 1.381 | 129.29 | 10400000 | 1.3743 |
152
+ | 1.3805 | 130.54 | 10500000 | 1.3762 |
153
+ | 1.3804 | 131.78 | 10600000 | 1.3725 |
154
+ | 1.3795 | 133.02 | 10700000 | 1.3753 |
155
+ | 1.3791 | 134.27 | 10800000 | 1.3780 |
156
+ | 1.3785 | 135.51 | 10900000 | 1.3749 |
157
+ | 1.3781 | 136.75 | 11000000 | 1.3749 |
158
+ | 1.3779 | 138.0 | 11100000 | 1.3737 |
159
+ | 1.3772 | 139.24 | 11200000 | 1.3715 |
160
+ | 1.3763 | 140.48 | 11300000 | 1.3759 |
161
+ | 1.3761 | 141.73 | 11400000 | 1.3745 |
162
+ | 1.3752 | 142.97 | 11500000 | 1.3708 |
163
+ | 1.3744 | 144.21 | 11600000 | 1.3735 |
164
+ | 1.3736 | 145.46 | 11700000 | 1.3720 |
165
+ | 1.3728 | 146.7 | 11800000 | 1.3702 |
166
+ | 1.3714 | 147.94 | 11900000 | 1.3696 |
167
+ | 1.3706 | 149.19 | 12000000 | 1.3707 |
168
+
169
+
170
+ ### Framework versions
171
+
172
+ - Transformers 4.27.3
173
+ - Pytorch 2.0.0
174
+ - Datasets 2.10.1
175
+ - Tokenizers 0.13.2