update model card README.md
Browse files
README.md
ADDED
@@ -0,0 +1,175 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
tags:
|
3 |
+
- generated_from_trainer
|
4 |
+
model-index:
|
5 |
+
- name: DNADebertaK6f
|
6 |
+
results: []
|
7 |
+
---
|
8 |
+
|
9 |
+
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
10 |
+
should probably proofread and complete it, then remove this comment. -->
|
11 |
+
|
12 |
+
# DNADebertaK6f
|
13 |
+
|
14 |
+
This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
|
15 |
+
It achieves the following results on the evaluation set:
|
16 |
+
- Loss: 1.3707
|
17 |
+
|
18 |
+
## Model description
|
19 |
+
|
20 |
+
More information needed
|
21 |
+
|
22 |
+
## Intended uses & limitations
|
23 |
+
|
24 |
+
More information needed
|
25 |
+
|
26 |
+
## Training and evaluation data
|
27 |
+
|
28 |
+
More information needed
|
29 |
+
|
30 |
+
## Training procedure
|
31 |
+
|
32 |
+
### Training hyperparameters
|
33 |
+
|
34 |
+
The following hyperparameters were used during training:
|
35 |
+
- learning_rate: 5e-05
|
36 |
+
- train_batch_size: 64
|
37 |
+
- eval_batch_size: 64
|
38 |
+
- seed: 42
|
39 |
+
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
40 |
+
- lr_scheduler_type: linear
|
41 |
+
- num_epochs: 150
|
42 |
+
- mixed_precision_training: Native AMP
|
43 |
+
|
44 |
+
### Training results
|
45 |
+
|
46 |
+
| Training Loss | Epoch | Step | Validation Loss |
|
47 |
+
|:-------------:|:------:|:--------:|:---------------:|
|
48 |
+
| 2.1545 | 1.24 | 100000 | 1.5895 |
|
49 |
+
| 1.5727 | 2.49 | 200000 | 1.5383 |
|
50 |
+
| 1.5368 | 3.73 | 300000 | 1.5117 |
|
51 |
+
| 1.5143 | 4.97 | 400000 | 1.4926 |
|
52 |
+
| 1.4969 | 6.22 | 500000 | 1.4789 |
|
53 |
+
| 1.4841 | 7.46 | 600000 | 1.4677 |
|
54 |
+
| 1.475 | 8.7 | 700000 | 1.4599 |
|
55 |
+
| 1.4677 | 9.95 | 800000 | 1.4533 |
|
56 |
+
| 1.4623 | 11.19 | 900000 | 1.4491 |
|
57 |
+
| 1.4576 | 12.43 | 1000000 | 1.4461 |
|
58 |
+
| 1.4544 | 13.68 | 1100000 | 1.4420 |
|
59 |
+
| 1.451 | 14.92 | 1200000 | 1.4381 |
|
60 |
+
| 1.4482 | 16.16 | 1300000 | 1.4359 |
|
61 |
+
| 1.446 | 17.4 | 1400000 | 1.4345 |
|
62 |
+
| 1.443 | 18.65 | 1500000 | 1.4320 |
|
63 |
+
| 1.4412 | 19.89 | 1600000 | 1.4295 |
|
64 |
+
| 1.4385 | 21.13 | 1700000 | 1.4278 |
|
65 |
+
| 1.4368 | 22.38 | 1800000 | 1.4249 |
|
66 |
+
| 1.4346 | 23.62 | 1900000 | 1.4237 |
|
67 |
+
| 1.433 | 24.86 | 2000000 | 1.4219 |
|
68 |
+
| 1.4315 | 26.11 | 2100000 | 1.4201 |
|
69 |
+
| 1.4297 | 27.35 | 2200000 | 1.4198 |
|
70 |
+
| 1.4282 | 28.59 | 2300000 | 1.4180 |
|
71 |
+
| 1.4266 | 29.84 | 2400000 | 1.4142 |
|
72 |
+
| 1.4253 | 31.08 | 2500000 | 1.4146 |
|
73 |
+
| 1.4238 | 32.32 | 2600000 | 1.4130 |
|
74 |
+
| 1.4228 | 33.57 | 2700000 | 1.4113 |
|
75 |
+
| 1.4221 | 34.81 | 2800000 | 1.4100 |
|
76 |
+
| 1.42 | 36.05 | 2900000 | 1.4097 |
|
77 |
+
| 1.4188 | 37.3 | 3000000 | 1.4085 |
|
78 |
+
| 1.4174 | 38.54 | 3100000 | 1.4067 |
|
79 |
+
| 1.4161 | 39.78 | 3200000 | 1.4064 |
|
80 |
+
| 1.4149 | 41.03 | 3300000 | 1.4058 |
|
81 |
+
| 1.4139 | 42.27 | 3400000 | 1.4024 |
|
82 |
+
| 1.4134 | 43.51 | 3500000 | 1.4022 |
|
83 |
+
| 1.4126 | 44.76 | 3600000 | 1.4025 |
|
84 |
+
| 1.4117 | 46.0 | 3700000 | 1.4015 |
|
85 |
+
| 1.411 | 47.24 | 3800000 | 1.4001 |
|
86 |
+
| 1.4098 | 48.49 | 3900000 | 1.3968 |
|
87 |
+
| 1.4096 | 49.73 | 4000000 | 1.3997 |
|
88 |
+
| 1.4089 | 50.97 | 4100000 | 1.3974 |
|
89 |
+
| 1.4084 | 52.21 | 4200000 | 1.3972 |
|
90 |
+
| 1.4072 | 53.46 | 4300000 | 1.3965 |
|
91 |
+
| 1.4066 | 54.7 | 4400000 | 1.3974 |
|
92 |
+
| 1.4062 | 55.94 | 4500000 | 1.3960 |
|
93 |
+
| 1.4058 | 57.19 | 4600000 | 1.3958 |
|
94 |
+
| 1.4053 | 58.43 | 4700000 | 1.3950 |
|
95 |
+
| 1.4041 | 59.67 | 4800000 | 1.3936 |
|
96 |
+
| 1.4041 | 60.92 | 4900000 | 1.3963 |
|
97 |
+
| 1.4031 | 62.16 | 5000000 | 1.3915 |
|
98 |
+
| 1.4023 | 63.4 | 5100000 | 1.3917 |
|
99 |
+
| 1.4022 | 64.65 | 5200000 | 1.3930 |
|
100 |
+
| 1.4017 | 65.89 | 5300000 | 1.3904 |
|
101 |
+
| 1.4009 | 67.13 | 5400000 | 1.3899 |
|
102 |
+
| 1.4007 | 68.38 | 5500000 | 1.3892 |
|
103 |
+
| 1.3997 | 69.62 | 5600000 | 1.3910 |
|
104 |
+
| 1.3996 | 70.86 | 5700000 | 1.3892 |
|
105 |
+
| 1.3991 | 72.11 | 5800000 | 1.3890 |
|
106 |
+
| 1.3983 | 73.35 | 5900000 | 1.3870 |
|
107 |
+
| 1.3985 | 74.59 | 6000000 | 1.3889 |
|
108 |
+
| 1.3975 | 75.84 | 6100000 | 1.3865 |
|
109 |
+
| 1.3973 | 77.08 | 6200000 | 1.3852 |
|
110 |
+
| 1.3969 | 78.32 | 6300000 | 1.3869 |
|
111 |
+
| 1.3964 | 79.57 | 6400000 | 1.3843 |
|
112 |
+
| 1.396 | 80.81 | 6500000 | 1.3853 |
|
113 |
+
| 1.3955 | 82.05 | 6600000 | 1.3844 |
|
114 |
+
| 1.3952 | 83.3 | 6700000 | 1.3863 |
|
115 |
+
| 1.395 | 84.54 | 6800000 | 1.3835 |
|
116 |
+
| 1.3948 | 85.78 | 6900000 | 1.3841 |
|
117 |
+
| 1.394 | 87.02 | 7000000 | 1.3850 |
|
118 |
+
| 1.3934 | 88.27 | 7100000 | 1.3827 |
|
119 |
+
| 1.3932 | 89.51 | 7200000 | 1.3830 |
|
120 |
+
| 1.3929 | 90.75 | 7300000 | 1.3821 |
|
121 |
+
| 1.392 | 92.0 | 7400000 | 1.3820 |
|
122 |
+
| 1.392 | 93.24 | 7500000 | 1.3837 |
|
123 |
+
| 1.3913 | 94.48 | 7600000 | 1.3817 |
|
124 |
+
| 1.3909 | 95.73 | 7700000 | 1.3836 |
|
125 |
+
| 1.3906 | 96.97 | 7800000 | 1.3811 |
|
126 |
+
| 1.3903 | 98.21 | 7900000 | 1.3806 |
|
127 |
+
| 1.3902 | 99.46 | 8000000 | 1.3807 |
|
128 |
+
| 1.3896 | 100.7 | 8100000 | 1.3804 |
|
129 |
+
| 1.3895 | 101.94 | 8200000 | 1.3805 |
|
130 |
+
| 1.3891 | 103.19 | 8300000 | 1.3821 |
|
131 |
+
| 1.3889 | 104.43 | 8400000 | 1.3833 |
|
132 |
+
| 1.3881 | 105.67 | 8500000 | 1.3788 |
|
133 |
+
| 1.388 | 106.92 | 8600000 | 1.3818 |
|
134 |
+
| 1.3876 | 108.16 | 8700000 | 1.3806 |
|
135 |
+
| 1.387 | 109.4 | 8800000 | 1.3766 |
|
136 |
+
| 1.387 | 110.65 | 8900000 | 1.3765 |
|
137 |
+
| 1.3865 | 111.89 | 9000000 | 1.3800 |
|
138 |
+
| 1.3864 | 113.13 | 9100000 | 1.3830 |
|
139 |
+
| 1.386 | 114.38 | 9200000 | 1.3770 |
|
140 |
+
| 1.3853 | 115.62 | 9300000 | 1.3771 |
|
141 |
+
| 1.3852 | 116.86 | 9400000 | 1.3772 |
|
142 |
+
| 1.3848 | 118.1 | 9500000 | 1.3771 |
|
143 |
+
| 1.384 | 119.35 | 9600000 | 1.3749 |
|
144 |
+
| 1.3843 | 120.59 | 9700000 | 1.3764 |
|
145 |
+
| 1.3836 | 121.83 | 9800000 | 1.3802 |
|
146 |
+
| 1.3833 | 123.08 | 9900000 | 1.3756 |
|
147 |
+
| 1.3831 | 124.32 | 10000000 | 1.3748 |
|
148 |
+
| 1.3821 | 125.56 | 10100000 | 1.3755 |
|
149 |
+
| 1.3817 | 126.81 | 10200000 | 1.3744 |
|
150 |
+
| 1.3819 | 128.05 | 10300000 | 1.3763 |
|
151 |
+
| 1.381 | 129.29 | 10400000 | 1.3743 |
|
152 |
+
| 1.3805 | 130.54 | 10500000 | 1.3762 |
|
153 |
+
| 1.3804 | 131.78 | 10600000 | 1.3725 |
|
154 |
+
| 1.3795 | 133.02 | 10700000 | 1.3753 |
|
155 |
+
| 1.3791 | 134.27 | 10800000 | 1.3780 |
|
156 |
+
| 1.3785 | 135.51 | 10900000 | 1.3749 |
|
157 |
+
| 1.3781 | 136.75 | 11000000 | 1.3749 |
|
158 |
+
| 1.3779 | 138.0 | 11100000 | 1.3737 |
|
159 |
+
| 1.3772 | 139.24 | 11200000 | 1.3715 |
|
160 |
+
| 1.3763 | 140.48 | 11300000 | 1.3759 |
|
161 |
+
| 1.3761 | 141.73 | 11400000 | 1.3745 |
|
162 |
+
| 1.3752 | 142.97 | 11500000 | 1.3708 |
|
163 |
+
| 1.3744 | 144.21 | 11600000 | 1.3735 |
|
164 |
+
| 1.3736 | 145.46 | 11700000 | 1.3720 |
|
165 |
+
| 1.3728 | 146.7 | 11800000 | 1.3702 |
|
166 |
+
| 1.3714 | 147.94 | 11900000 | 1.3696 |
|
167 |
+
| 1.3706 | 149.19 | 12000000 | 1.3707 |
|
168 |
+
|
169 |
+
|
170 |
+
### Framework versions
|
171 |
+
|
172 |
+
- Transformers 4.27.3
|
173 |
+
- Pytorch 2.0.0
|
174 |
+
- Datasets 2.10.1
|
175 |
+
- Tokenizers 0.13.2
|