Hugging Face's logo

bobox
/
DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp

PyTorch
TensorBoard
deberta-v2
Model card Files
xet
Metrics Community
bobox commited on
Commit
4b14a9c
·
verified ·
1 Parent(s): 970165e

Training in progress, step 304, checkpoint

Browse files
Files changed (17) hide show
  1. checkpoint-304/1_Pooling/config.json +10 -0
  2. checkpoint-304/README.md +833 -0
  3. checkpoint-304/added_tokens.json +3 -0
  4. checkpoint-304/config.json +35 -0
  5. checkpoint-304/config_sentence_transformers.json +10 -0
  6. checkpoint-304/modules.json +14 -0
  7. checkpoint-304/optimizer.pt +3 -0
  8. checkpoint-304/pytorch_model.bin +3 -0
  9. checkpoint-304/rng_state.pth +3 -0
  10. checkpoint-304/scheduler.pt +3 -0
  11. checkpoint-304/sentence_bert_config.json +4 -0
  12. checkpoint-304/special_tokens_map.json +51 -0
  13. checkpoint-304/spm.model +3 -0
  14. checkpoint-304/tokenizer.json +0 -0
  15. checkpoint-304/tokenizer_config.json +65 -0
  16. checkpoint-304/trainer_state.json +0 -0
  17. checkpoint-304/training_args.bin +3 -0
checkpoint-304/1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
checkpoint-304/README.md ADDED
@@ -0,0 +1,833 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: bobox/DeBERTa-small-ST-v1-test-step3
3
+ datasets: []
4
+ language: []
5
+ library_name: sentence-transformers
6
+ metrics:
7
+ - pearson_cosine
8
+ - spearman_cosine
9
+ - pearson_manhattan
10
+ - spearman_manhattan
11
+ - pearson_euclidean
12
+ - spearman_euclidean
13
+ - pearson_dot
14
+ - spearman_dot
15
+ - pearson_max
16
+ - spearman_max
17
+ pipeline_tag: sentence-similarity
18
+ tags:
19
+ - sentence-transformers
20
+ - sentence-similarity
21
+ - feature-extraction
22
+ - generated_from_trainer
23
+ - dataset_size:120849
24
+ - loss:CachedGISTEmbedLoss
25
+ widget:
26
+ - source_sentence: 'Brian Cummins, who was in his early 60s, was refereeing an Under-16s
27
+ final in Elburton, Devon, on Sunday when he fell to the ground.
28
+
29
+ He was taken to Derriford Hospital in Plymouth but died shortly after, the Devon
30
+ Junior Minor League said.
31
+
32
+ A spokesman said: "Our thoughts and condolences at this time are with his family.
33
+ The league has lost a very loyal referee."
34
+
35
+ The game between Woolwell Juniors and Tavistock was abandoned after Mr Cummins
36
+ collapsed.
37
+
38
+ His daughter Sarah said: "He was a loving father, father in law, granddad, husband
39
+ and friend to all who knew him. He loved his football and refereeing.
40
+
41
+ "He will be greatly missed by all and will forever be in our hearts."
42
+
43
+ Brian Rimes, general secretary of the league, said former Devonport dockyard worker
44
+ Mr Cummins had been a referee for the league for about 20 years.
45
+
46
+ "He was a very good referee, a man well respected by youngsters and the referee
47
+ fraternity," he said.
48
+
49
+ Mark Davies wrote on the league''s Facebook page: "Very sad news indeed and our
50
+ thoughts go out to Brian''s family and friends. It''s a shame that it takes such
51
+ awful circumstances to unite the local footballing community but in Brian we know
52
+ we have lost a true gent. RIP Brian."
53
+
54
+ Mark Evans wrote: "RIP Brian. Grassroots football has lost an amazing guy and
55
+ great referee. Deepest sympathies from all Devon FA referees."
56
+
57
+ Michael Davies tweeted: "#RIP Brian Cummins, such sad news! Top Ref, top neighbour
58
+ but most of all a top bloke! Will be sadly missed."'
59
+ sentences:
60
+ - 'Ptosis (Sagging Eyelids): Check Your Symptoms and Signs Watery Eye A drooping
61
+ or sagging of the eyelid is medically known as ptosis or blepharoptosis . Drooping
62
+ eyelids may occur on both sides (bilateral) or on one side only (unilateral),
63
+ in which case it is more easily noticed. Congenital ptosis is eyelid drooping
64
+ that is present at birth; when it develops later, it is referred to as acquired
65
+ ptosis. Depending upon the severity of the condition, drooping eyelids may be
66
+ barely noticeable or quite prominent. Some sagging of the skin and connective
67
+ tissues occurs during the normal aging process, potentially leading to drooping
68
+ of the eyelids. Other causes include conditions that affect the muscles and nerves
69
+ of the eyelid as well as conditions that affect the skin and connective tissues
70
+ of the eyelid. Rarely, tumors of the brain or eye area are the cause of drooping
71
+ eyelids. Medically Reviewed by a Doctor on 3/6/2012 Health concern on your mind?
72
+ Visit the Symptom Checker. REFERENCE: Fauci, Anthony S., et al. Harrison''s Principles
73
+ of Internal Medicine. 17th ed. United States: McGraw-Hill Professional, 2008.
74
+ Causes of Ptosis Allergy (Allergies) An allergy refers to a misguided reaction
75
+ by our immune system in response to bodily contact with certain foreign substances.
76
+ ... learn more » Botulism Botulism is an illness caused by a neurotoxin produced
77
+ by the bacterium Clostridium botulinum. There are three types of botulism:...
78
+ learn more » In This Article'
79
+ - A football referee has died after collapsing during a boys' cup final.
80
+ - The car is at the intersection while the sun is setting.
81
+ - source_sentence: sparge water temperature
82
+ sentences:
83
+ - This is easy to translate to gallons and degrees F. (for example, suggested sparge
84
+ water temperature is 167° F., which is 75° C.). It also features a stay warm
85
+ feature - after the target water temperature is hit, it will keep it at the desired
86
+ temperature as long as it is on.
87
+ - Arsenal playmaker Mesut Ozil says he will put talks over his future at the club
88
+ on hold until the summer.
89
+ - a greenhouse is used to protect plants by keeping them warm
90
+ - source_sentence: What does sunlight create for plants?
91
+ sentences:
92
+ - "a plant requires sunlight for photosynthesis. Photosynthesis occurs using the\
93
+ \ suns energy to create the plants own energy. \n sunlight creates energy for\
94
+ \ plants"
95
+ - His references in electronic music are Todd Terry , Armand Van Helden , Roger
96
+ Sanchez , Tiesto and the Epic Sax Guy.
97
+ - "if a neutral atom loses an electron then an atom with a negative charge will\
98
+ \ be formed. Ions are neutral atoms. \n ions can have a negative charge if they\
99
+ \ lose an electron"
100
+ - source_sentence: Metals, metalloids, and nonmetals are the different classes of
101
+ what?
102
+ sentences:
103
+ - when an animal sheds its fur , its fur becomes less dense
104
+ - Though there's no limit to how much you can keep in a savings account, you should
105
+ know the rules surrounding large deposits to savings accounts. When it comes to
106
+ making deposits to a bank account, $10,000 is the magic number.
107
+ - The classes of elements are metals, metalloids, and nonmetals. They are color-coded
108
+ in the table. Blue stands for metals, orange for metalloids, and green for nonmetals.
109
+ You can read about each of these three classes of elements later in the chapter,
110
+ in the lesson "Classes of Elements. ".
111
+ - source_sentence: More than 336,000 COVID-19 cases have been reported in over 190
112
+ countries .
113
+ sentences:
114
+ - Birds are four-limbed, endothermic vertebrates with wings and feathers. They produce
115
+ amniotic eggs and are the most numerous class of vertebrates.
116
+ - As of 23 March , more than 337,000 cases of COVID-19 have been reported in over
117
+ 190 countries and territories , resulting in more than 14,600 deaths and 97,000
118
+ recoveries .
119
+ - Apocalypticism Apocalypticism is the religious belief that there will be an apocalypse,
120
+ a term which originally referred to a revelation of God's will, but now usually
121
+ refers to the belief that the end of the world is imminent, even within one's
122
+ own lifetime. This belief is usually accompanied by the idea that civilization
123
+ will soon come to a tumultuous end due to some sort of catastrophic global event.
124
+ model-index:
125
+ - name: SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
126
+ results:
127
+ - task:
128
+ type: semantic-similarity
129
+ name: Semantic Similarity
130
+ dataset:
131
+ name: sts test
132
+ type: sts-test
133
+ metrics:
134
+ - type: pearson_cosine
135
+ value: 0.8842529900690873
136
+ name: Pearson Cosine
137
+ - type: spearman_cosine
138
+ value: 0.9092792206013333
139
+ name: Spearman Cosine
140
+ - type: pearson_manhattan
141
+ value: 0.9125934943438889
142
+ name: Pearson Manhattan
143
+ - type: spearman_manhattan
144
+ value: 0.9085863815724716
145
+ name: Spearman Manhattan
146
+ - type: pearson_euclidean
147
+ value: 0.9135232453954274
148
+ name: Pearson Euclidean
149
+ - type: spearman_euclidean
150
+ value: 0.9094442185020833
151
+ name: Spearman Euclidean
152
+ - type: pearson_dot
153
+ value: 0.8618630965880719
154
+ name: Pearson Dot
155
+ - type: spearman_dot
156
+ value: 0.8592595274601085
157
+ name: Spearman Dot
158
+ - type: pearson_max
159
+ value: 0.9135232453954274
160
+ name: Pearson Max
161
+ - type: spearman_max
162
+ value: 0.9094442185020833
163
+ name: Spearman Max
164
+ ---
165
+
166
+ # SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
167
+
168
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [bobox/DeBERTa-small-ST-v1-test-step3](https://huggingface.co/bobox/DeBERTa-small-ST-v1-test-step3) on the bobox/enhanced_nli-50_k dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
169
+
170
+ ## Model Details
171
+
172
+ ### Model Description
173
+ - **Model Type:** Sentence Transformer
174
+ - **Base model:** [bobox/DeBERTa-small-ST-v1-test-step3](https://huggingface.co/bobox/DeBERTa-small-ST-v1-test-step3) <!-- at revision df9aaa75fe0c2791e5ed35ff33de1689d9a5f5ff -->
175
+ - **Maximum Sequence Length:** 512 tokens
176
+ - **Output Dimensionality:** 768 tokens
177
+ - **Similarity Function:** Cosine Similarity
178
+ - **Training Dataset:**
179
+ - bobox/enhanced_nli-50_k
180
+ <!-- - **Language:** Unknown -->
181
+ <!-- - **License:** Unknown -->
182
+
183
+ ### Model Sources
184
+
185
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
186
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
187
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
188
+
189
+ ### Full Model Architecture
190
+
191
+ ```
192
+ SentenceTransformer(
193
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model
194
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
195
+ )
196
+ ```
197
+
198
+ ## Usage
199
+
200
+ ### Direct Usage (Sentence Transformers)
201
+
202
+ First install the Sentence Transformers library:
203
+
204
+ ```bash
205
+ pip install -U sentence-transformers
206
+ ```
207
+
208
+ Then you can load this model and run inference.
209
+ ```python
210
+ from sentence_transformers import SentenceTransformer
211
+
212
+ # Download from the 🤗 Hub
213
+ model = SentenceTransformer("bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp")
214
+ # Run inference
215
+ sentences = [
216
+ 'More than 336,000 COVID-19 cases have been reported in over 190 countries .',
217
+ 'As of 23 March , more than 337,000 cases of COVID-19 have been reported in over 190 countries and territories , resulting in more than 14,600 deaths and 97,000 recoveries .',
218
+ "Apocalypticism Apocalypticism is the religious belief that there will be an apocalypse, a term which originally referred to a revelation of God's will, but now usually refers to the belief that the end of the world is imminent, even within one's own lifetime. This belief is usually accompanied by the idea that civilization will soon come to a tumultuous end due to some sort of catastrophic global event.",
219
+ ]
220
+ embeddings = model.encode(sentences)
221
+ print(embeddings.shape)
222
+ # [3, 768]
223
+
224
+ # Get the similarity scores for the embeddings
225
+ similarities = model.similarity(embeddings, embeddings)
226
+ print(similarities.shape)
227
+ # [3, 3]
228
+ ```
229
+
230
+ <!--
231
+ ### Direct Usage (Transformers)
232
+
233
+ <details><summary>Click to see the direct usage in Transformers</summary>
234
+
235
+ </details>
236
+ -->
237
+
238
+ <!--
239
+ ### Downstream Usage (Sentence Transformers)
240
+
241
+ You can finetune this model on your own dataset.
242
+
243
+ <details><summary>Click to expand</summary>
244
+
245
+ </details>
246
+ -->
247
+
248
+ <!--
249
+ ### Out-of-Scope Use
250
+
251
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
252
+ -->
253
+
254
+ ## Evaluation
255
+
256
+ ### Metrics
257
+
258
+ #### Semantic Similarity
259
+ * Dataset: `sts-test`
260
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
261
+
262
+ | Metric | Value |
263
+ |:--------------------|:-----------|
264
+ | pearson_cosine | 0.8843 |
265
+ | **spearman_cosine** | **0.9093** |
266
+ | pearson_manhattan | 0.9126 |
267
+ | spearman_manhattan | 0.9086 |
268
+ | pearson_euclidean | 0.9135 |
269
+ | spearman_euclidean | 0.9094 |
270
+ | pearson_dot | 0.8619 |
271
+ | spearman_dot | 0.8593 |
272
+ | pearson_max | 0.9135 |
273
+ | spearman_max | 0.9094 |
274
+
275
+ <!--
276
+ ## Bias, Risks and Limitations
277
+
278
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
279
+ -->
280
+
281
+ <!--
282
+ ### Recommendations
283
+
284
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
285
+ -->
286
+
287
+ ## Training Details
288
+
289
+ ### Training Dataset
290
+
291
+ #### bobox/enhanced_nli-50_k
292
+
293
+ * Dataset: bobox/enhanced_nli-50_k
294
+ * Size: 120,849 training samples
295
+ * Columns: <code>sentence1</code> and <code>sentence2</code>
296
+ * Approximate statistics based on the first 1000 samples:
297
+ | | sentence1 | sentence2 |
298
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
299
+ | type | string | string |
300
+ | details | <ul><li>min: 4 tokens</li><li>mean: 33.98 tokens</li><li>max: 358 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 63.13 tokens</li><li>max: 414 tokens</li></ul> |
301
+ * Samples:
302
+ | sentence1 | sentence2 |
303
+ |:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
304
+ | <code>A lady working in a kitchen with several different types of dishes.</code> | <code>A woman is cooking and cleaning in her kitchen.</code> |
305
+ | <code>is it possible to get pregnant after delivery?</code> | <code>How soon can you get pregnant after giving birth? It's possible to get pregnant before you even have your first postpartum period, which can occur as early as four weeks after giving birth or as late as 24 weeks after baby arrives (or later), depending on whether you're breastfeeding exclusively or not.</code> |
306
+ | <code>how long does corn take to grill in foil</code> | <code>Place each corn on top of one piece the heavy-duty foil. Brush each ear generously with soft butter. Season lightly with seasoned salt or white salt and black pepper. Wrap the corn then seal the foil loosely but leave room for expansion, then cut a very small hole in the foil to allow steam to escape. Grill over medium coals for about 15-20 minutes (the larger ears may take a little longer).</code> |
307
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
308
+ ```json
309
+ {'guide': SentenceTransformer(
310
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
311
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
312
+ (2): Normalize()
313
+ ), 'temperature': 0.025}
314
+ ```
315
+
316
+ ### Evaluation Dataset
317
+
318
+ #### bobox/enhanced_nli-50_k
319
+
320
+ * Dataset: bobox/enhanced_nli-50_k
321
+ * Size: 3,052 evaluation samples
322
+ * Columns: <code>sentence1</code> and <code>sentence2</code>
323
+ * Approximate statistics based on the first 1000 samples:
324
+ | | sentence1 | sentence2 |
325
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
326
+ | type | string | string |
327
+ | details | <ul><li>min: 4 tokens</li><li>mean: 33.63 tokens</li><li>max: 328 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 60.36 tokens</li><li>max: 501 tokens</li></ul> |
328
+ * Samples:
329
+ | sentence1 | sentence2 |
330
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
331
+ | <code>The 17-year-old asked not to be named but said he lost control of his silver hatchback when he swerved to avoid a cat in Parkway, Chellaston, Derbyshire.<br>He estimated he was travelling at about 30mph when he smashed into the garage of a residential home on Friday night.<br>The owners were away at the time and the crash was reported to police.<br>The driver told BBC News: "I swerved to the right and the back of the car went out.<br>"I then swerved to the left and lost control - I couldn't bring it back.<br>"I just missed two parked cars and a tree and ended up in the wall - it was a big impact.<br>"I smelt burning, I thought it was the car, so I got out and laid down - I was in shock."<br>Neighbours reported hearing a "loud screeching" followed by a "massive bang".<br>The driver said: "It was a natural reaction to swerve to miss the cat, but I went into a state of shock and panic."<br>Derbyshire Police said the driver was due to appear before magistrates at a later date charged with driving without due care or attention.</code> | <code>A new driver who ploughed into a house, having swerved around two parked cars and a tree to avoid hitting a cat in the road, faces court.</code> |
332
+ | <code>what requirements are needed to be a psychologist?</code> | <code>To become a clinician, you must apply for registration with the College of Psychologists of Ontario, a process which requires 4 years of work experience and one year of supervised practice. In some provinces, you can be registered to practice as a psychologist with either a masters or doctoral degree.</code> |
333
+ | <code>What does sunlight create for plants?</code> | <code>a plant requires sunlight for photosynthesis. Photosynthesis occurs using the suns energy to create the plants own energy. <br> sunlight creates energy for plants</code> |
334
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
335
+ ```json
336
+ {'guide': SentenceTransformer(
337
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
338
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
339
+ (2): Normalize()
340
+ ), 'temperature': 0.025}
341
+ ```
342
+
343
+ ### Training Hyperparameters
344
+ #### Non-Default Hyperparameters
345
+
346
+ - `eval_strategy`: steps
347
+ - `per_device_train_batch_size`: 640
348
+ - `per_device_eval_batch_size`: 128
349
+ - `learning_rate`: 3.5e-05
350
+ - `weight_decay`: 0.0001
351
+ - `num_train_epochs`: 2
352
+ - `lr_scheduler_type`: cosine_with_min_lr
353
+ - `lr_scheduler_kwargs`: {'num_cycles': 0.5, 'min_lr': 5.833333333333333e-06}
354
+ - `warmup_ratio`: 0.25
355
+ - `save_safetensors`: False
356
+ - `fp16`: True
357
+ - `push_to_hub`: True
358
+ - `hub_model_id`: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp
359
+ - `hub_strategy`: all_checkpoints
360
+ - `batch_sampler`: no_duplicates
361
+
362
+ #### All Hyperparameters
363
+ <details><summary>Click to expand</summary>
364
+
365
+ - `overwrite_output_dir`: False
366
+ - `do_predict`: False
367
+ - `eval_strategy`: steps
368
+ - `prediction_loss_only`: True
369
+ - `per_device_train_batch_size`: 640
370
+ - `per_device_eval_batch_size`: 128
371
+ - `per_gpu_train_batch_size`: None
372
+ - `per_gpu_eval_batch_size`: None
373
+ - `gradient_accumulation_steps`: 1
374
+ - `eval_accumulation_steps`: None
375
+ - `torch_empty_cache_steps`: None
376
+ - `learning_rate`: 3.5e-05
377
+ - `weight_decay`: 0.0001
378
+ - `adam_beta1`: 0.9
379
+ - `adam_beta2`: 0.999
380
+ - `adam_epsilon`: 1e-08
381
+ - `max_grad_norm`: 1.0
382
+ - `num_train_epochs`: 2
383
+ - `max_steps`: -1
384
+ - `lr_scheduler_type`: cosine_with_min_lr
385
+ - `lr_scheduler_kwargs`: {'num_cycles': 0.5, 'min_lr': 5.833333333333333e-06}
386
+ - `warmup_ratio`: 0.25
387
+ - `warmup_steps`: 0
388
+ - `log_level`: passive
389
+ - `log_level_replica`: warning
390
+ - `log_on_each_node`: True
391
+ - `logging_nan_inf_filter`: True
392
+ - `save_safetensors`: False
393
+ - `save_on_each_node`: False
394
+ - `save_only_model`: False
395
+ - `restore_callback_states_from_checkpoint`: False
396
+ - `no_cuda`: False
397
+ - `use_cpu`: False
398
+ - `use_mps_device`: False
399
+ - `seed`: 42
400
+ - `data_seed`: None
401
+ - `jit_mode_eval`: False
402
+ - `use_ipex`: False
403
+ - `bf16`: False
404
+ - `fp16`: True
405
+ - `fp16_opt_level`: O1
406
+ - `half_precision_backend`: auto
407
+ - `bf16_full_eval`: False
408
+ - `fp16_full_eval`: False
409
+ - `tf32`: None
410
+ - `local_rank`: 0
411
+ - `ddp_backend`: None
412
+ - `tpu_num_cores`: None
413
+ - `tpu_metrics_debug`: False
414
+ - `debug`: []
415
+ - `dataloader_drop_last`: False
416
+ - `dataloader_num_workers`: 0
417
+ - `dataloader_prefetch_factor`: None
418
+ - `past_index`: -1
419
+ - `disable_tqdm`: False
420
+ - `remove_unused_columns`: True
421
+ - `label_names`: None
422
+ - `load_best_model_at_end`: False
423
+ - `ignore_data_skip`: False
424
+ - `fsdp`: []
425
+ - `fsdp_min_num_params`: 0
426
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
427
+ - `fsdp_transformer_layer_cls_to_wrap`: None
428
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
429
+ - `deepspeed`: None
430
+ - `label_smoothing_factor`: 0.0
431
+ - `optim`: adamw_torch
432
+ - `optim_args`: None
433
+ - `adafactor`: False
434
+ - `group_by_length`: False
435
+ - `length_column_name`: length
436
+ - `ddp_find_unused_parameters`: None
437
+ - `ddp_bucket_cap_mb`: None
438
+ - `ddp_broadcast_buffers`: False
439
+ - `dataloader_pin_memory`: True
440
+ - `dataloader_persistent_workers`: False
441
+ - `skip_memory_metrics`: True
442
+ - `use_legacy_prediction_loop`: False
443
+ - `push_to_hub`: True
444
+ - `resume_from_checkpoint`: None
445
+ - `hub_model_id`: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp
446
+ - `hub_strategy`: all_checkpoints
447
+ - `hub_private_repo`: False
448
+ - `hub_always_push`: False
449
+ - `gradient_checkpointing`: False
450
+ - `gradient_checkpointing_kwargs`: None
451
+ - `include_inputs_for_metrics`: False
452
+ - `eval_do_concat_batches`: True
453
+ - `fp16_backend`: auto
454
+ - `push_to_hub_model_id`: None
455
+ - `push_to_hub_organization`: None
456
+ - `mp_parameters`:
457
+ - `auto_find_batch_size`: False
458
+ - `full_determinism`: False
459
+ - `torchdynamo`: None
460
+ - `ray_scope`: last
461
+ - `ddp_timeout`: 1800
462
+ - `torch_compile`: False
463
+ - `torch_compile_backend`: None
464
+ - `torch_compile_mode`: None
465
+ - `dispatch_batches`: None
466
+ - `split_batches`: None
467
+ - `include_tokens_per_second`: False
468
+ - `include_num_input_tokens_seen`: False
469
+ - `neftune_noise_alpha`: None
470
+ - `optim_target_modules`: None
471
+ - `batch_eval_metrics`: False
472
+ - `eval_on_start`: False
473
+ - `eval_use_gather_object`: False
474
+ - `batch_sampler`: no_duplicates
475
+ - `multi_dataset_batch_sampler`: proportional
476
+
477
+ </details>
478
+
479
+ ### Training Logs
480
+ <details><summary>Click to expand</summary>
481
+
482
+ | Epoch | Step | Training Loss | loss | sts-test_spearman_cosine |
483
+ |:------:|:----:|:-------------:|:------:|:------------------------:|
484
+ | 0.0053 | 1 | 0.3768 | - | - |
485
+ | 0.0106 | 2 | 0.3162 | - | - |
486
+ | 0.0159 | 3 | 0.275 | - | - |
487
+ | 0.0212 | 4 | 0.293 | - | - |
488
+ | 0.0265 | 5 | 0.2437 | 0.2190 | 0.9079 |
489
+ | 0.0317 | 6 | 0.3681 | - | - |
490
+ | 0.0370 | 7 | 0.2314 | - | - |
491
+ | 0.0423 | 8 | 0.2481 | - | - |
492
+ | 0.0476 | 9 | 0.2403 | - | - |
493
+ | 0.0529 | 10 | 0.2966 | 0.2125 | 0.9079 |
494
+ | 0.0582 | 11 | 0.2867 | - | - |
495
+ | 0.0635 | 12 | 0.3413 | - | - |
496
+ | 0.0688 | 13 | 0.4119 | - | - |
497
+ | 0.0741 | 14 | 0.3118 | - | - |
498
+ | 0.0794 | 15 | 0.327 | 0.2031 | 0.9082 |
499
+ | 0.0847 | 16 | 0.3389 | - | - |
500
+ | 0.0899 | 17 | 0.2018 | - | - |
501
+ | 0.0952 | 18 | 0.2861 | - | - |
502
+ | 0.1005 | 19 | 0.2848 | - | - |
503
+ | 0.1058 | 20 | 0.2563 | 0.1943 | 0.9082 |
504
+ | 0.1111 | 21 | 0.3058 | - | - |
505
+ | 0.1164 | 22 | 0.285 | - | - |
506
+ | 0.1217 | 23 | 0.3151 | - | - |
507
+ | 0.1270 | 24 | 0.2716 | - | - |
508
+ | 0.1323 | 25 | 0.2422 | 0.1794 | 0.9082 |
509
+ | 0.1376 | 26 | 0.2858 | - | - |
510
+ | 0.1429 | 27 | 0.3211 | - | - |
511
+ | 0.1481 | 28 | 0.2158 | - | - |
512
+ | 0.1534 | 29 | 0.2811 | - | - |
513
+ | 0.1587 | 30 | 0.2063 | 0.1636 | 0.9077 |
514
+ | 0.1640 | 31 | 0.2492 | - | - |
515
+ | 0.1693 | 32 | 0.3096 | - | - |
516
+ | 0.1746 | 33 | 0.2914 | - | - |
517
+ | 0.1799 | 34 | 0.2888 | - | - |
518
+ | 0.1852 | 35 | 0.223 | 0.1532 | 0.9072 |
519
+ | 0.1905 | 36 | 0.2595 | - | - |
520
+ | 0.1958 | 37 | 0.3122 | - | - |
521
+ | 0.2011 | 38 | 0.2327 | - | - |
522
+ | 0.2063 | 39 | 0.1718 | - | - |
523
+ | 0.2116 | 40 | 0.3162 | 0.1443 | 0.9067 |
524
+ | 0.2169 | 41 | 0.296 | - | - |
525
+ | 0.2222 | 42 | 0.2821 | - | - |
526
+ | 0.2275 | 43 | 0.2069 | - | - |
527
+ | 0.2328 | 44 | 0.2573 | - | - |
528
+ | 0.2381 | 45 | 0.3119 | 0.1343 | 0.9064 |
529
+ | 0.2434 | 46 | 0.2743 | - | - |
530
+ | 0.2487 | 47 | 0.2666 | - | - |
531
+ | 0.2540 | 48 | 0.2414 | - | - |
532
+ | 0.2593 | 49 | 0.2793 | - | - |
533
+ | 0.2646 | 50 | 0.2212 | 0.1251 | 0.9068 |
534
+ | 0.2698 | 51 | 0.2071 | - | - |
535
+ | 0.2751 | 52 | 0.296 | - | - |
536
+ | 0.2804 | 53 | 0.2061 | - | - |
537
+ | 0.2857 | 54 | 0.2164 | - | - |
538
+ | 0.2910 | 55 | 0.188 | 0.1197 | 0.9072 |
539
+ | 0.2963 | 56 | 0.2411 | - | - |
540
+ | 0.3016 | 57 | 0.2031 | - | - |
541
+ | 0.3069 | 58 | 0.2438 | - | - |
542
+ | 0.3122 | 59 | 0.2417 | - | - |
543
+ | 0.3175 | 60 | 0.1515 | 0.1233 | 0.9066 |
544
+ | 0.3228 | 61 | 0.21 | - | - |
545
+ | 0.3280 | 62 | 0.21 | - | - |
546
+ | 0.3333 | 63 | 0.2157 | - | - |
547
+ | 0.3386 | 64 | 0.2138 | - | - |
548
+ | 0.3439 | 65 | 0.2403 | 0.1273 | 0.9058 |
549
+ | 0.3492 | 66 | 0.2808 | - | - |
550
+ | 0.3545 | 67 | 0.1891 | - | - |
551
+ | 0.3598 | 68 | 0.1991 | - | - |
552
+ | 0.3651 | 69 | 0.2121 | - | - |
553
+ | 0.3704 | 70 | 0.2039 | 0.1311 | 0.9066 |
554
+ | 0.3757 | 71 | 0.1986 | - | - |
555
+ | 0.3810 | 72 | 0.2925 | - | - |
556
+ | 0.3862 | 73 | 0.2527 | - | - |
557
+ | 0.3915 | 74 | 0.279 | - | - |
558
+ | 0.3968 | 75 | 0.2419 | 0.1315 | 0.9066 |
559
+ | 0.4021 | 76 | 0.2228 | - | - |
560
+ | 0.4074 | 77 | 0.2242 | - | - |
561
+ | 0.4127 | 78 | 0.2737 | - | - |
562
+ | 0.4180 | 79 | 0.2328 | - | - |
563
+ | 0.4233 | 80 | 0.2802 | 0.1262 | 0.9058 |
564
+ | 0.4286 | 81 | 0.2044 | - | - |
565
+ | 0.4339 | 82 | 0.1828 | - | - |
566
+ | 0.4392 | 83 | 0.2372 | - | - |
567
+ | 0.4444 | 84 | 0.2241 | - | - |
568
+ | 0.4497 | 85 | 0.2782 | 0.1207 | 0.9063 |
569
+ | 0.4550 | 86 | 0.3244 | - | - |
570
+ | 0.4603 | 87 | 0.2102 | - | - |
571
+ | 0.4656 | 88 | 0.2265 | - | - |
572
+ | 0.4709 | 89 | 0.2666 | - | - |
573
+ | 0.4762 | 90 | 0.23 | 0.1186 | 0.9078 |
574
+ | 0.4815 | 91 | 0.2358 | - | - |
575
+ | 0.4868 | 92 | 0.2896 | - | - |
576
+ | 0.4921 | 93 | 0.2126 | - | - |
577
+ | 0.4974 | 94 | 0.2669 | - | - |
578
+ | 0.5026 | 95 | 0.2375 | 0.1128 | 0.9087 |
579
+ | 0.5079 | 96 | 0.1903 | - | - |
580
+ | 0.5132 | 97 | 0.2507 | - | - |
581
+ | 0.5185 | 98 | 0.1897 | - | - |
582
+ | 0.5238 | 99 | 0.2775 | - | - |
583
+ | 0.5291 | 100 | 0.2098 | 0.1168 | 0.9064 |
584
+ | 0.5344 | 101 | 0.1628 | - | - |
585
+ | 0.5397 | 102 | 0.2158 | - | - |
586
+ | 0.5450 | 103 | 0.1552 | - | - |
587
+ | 0.5503 | 104 | 0.2364 | - | - |
588
+ | 0.5556 | 105 | 0.272 | 0.1178 | 0.9056 |
589
+ | 0.5608 | 106 | 0.2271 | - | - |
590
+ | 0.5661 | 107 | 0.2132 | - | - |
591
+ | 0.5714 | 108 | 0.1782 | - | - |
592
+ | 0.5767 | 109 | 0.1598 | - | - |
593
+ | 0.5820 | 110 | 0.2472 | 0.1269 | 0.9050 |
594
+ | 0.5873 | 111 | 0.2041 | - | - |
595
+ | 0.5926 | 112 | 0.2426 | - | - |
596
+ | 0.5979 | 113 | 0.2105 | - | - |
597
+ | 0.6032 | 114 | 0.1923 | - | - |
598
+ | 0.6085 | 115 | 0.2271 | 0.1233 | 0.9061 |
599
+ | 0.6138 | 116 | 0.3029 | - | - |
600
+ | 0.6190 | 117 | 0.2554 | - | - |
601
+ | 0.6243 | 118 | 0.2182 | - | - |
602
+ | 0.6296 | 119 | 0.2852 | - | - |
603
+ | 0.6349 | 120 | 0.2285 | 0.1280 | 0.9053 |
604
+ | 0.6402 | 121 | 0.218 | - | - |
605
+ | 0.6455 | 122 | 0.1841 | - | - |
606
+ | 0.6508 | 123 | 0.2629 | - | - |
607
+ | 0.6561 | 124 | 0.1749 | - | - |
608
+ | 0.6614 | 125 | 0.2417 | 0.1415 | 0.9057 |
609
+ | 0.6667 | 126 | 0.2305 | - | - |
610
+ | 0.6720 | 127 | 0.2841 | - | - |
611
+ | 0.6772 | 128 | 0.1785 | - | - |
612
+ | 0.6825 | 129 | 0.2153 | - | - |
613
+ | 0.6878 | 130 | 0.2548 | 0.1413 | 0.9079 |
614
+ | 0.6931 | 131 | 0.2059 | - | - |
615
+ | 0.6984 | 132 | 0.2073 | - | - |
616
+ | 0.7037 | 133 | 0.191 | - | - |
617
+ | 0.7090 | 134 | 0.1633 | - | - |
618
+ | 0.7143 | 135 | 0.2627 | 0.1333 | 0.9077 |
619
+ | 0.7196 | 136 | 0.2451 | - | - |
620
+ | 0.7249 | 137 | 0.1441 | - | - |
621
+ | 0.7302 | 138 | 0.2138 | - | - |
622
+ | 0.7354 | 139 | 0.2564 | - | - |
623
+ | 0.7407 | 140 | 0.1524 | 0.1323 | 0.9049 |
624
+ | 0.7460 | 141 | 0.1786 | - | - |
625
+ | 0.7513 | 142 | 0.2104 | - | - |
626
+ | 0.7566 | 143 | 0.2512 | - | - |
627
+ | 0.7619 | 144 | 0.1889 | - | - |
628
+ | 0.7672 | 145 | 0.2127 | 0.1291 | 0.9015 |
629
+ | 0.7725 | 146 | 0.2115 | - | - |
630
+ | 0.7778 | 147 | 0.179 | - | - |
631
+ | 0.7831 | 148 | 0.2188 | - | - |
632
+ | 0.7884 | 149 | 0.1687 | - | - |
633
+ | 0.7937 | 150 | 0.2265 | 0.1145 | 0.9014 |
634
+ | 0.7989 | 151 | 0.182 | - | - |
635
+ | 0.8042 | 152 | 0.1789 | - | - |
636
+ | 0.8095 | 153 | 0.225 | - | - |
637
+ | 0.8148 | 154 | 0.3015 | - | - |
638
+ | 0.8201 | 155 | 0.1656 | 0.1009 | 0.9045 |
639
+ | 0.8254 | 156 | 0.2321 | - | - |
640
+ | 0.8307 | 157 | 0.2514 | - | - |
641
+ | 0.8360 | 158 | 0.2348 | - | - |
642
+ | 0.8413 | 159 | 0.2003 | - | - |
643
+ | 0.8466 | 160 | 0.2487 | 0.0983 | 0.9057 |
644
+ | 0.8519 | 161 | 0.1674 | - | - |
645
+ | 0.8571 | 162 | 0.1968 | - | - |
646
+ | 0.8624 | 163 | 0.2237 | - | - |
647
+ | 0.8677 | 164 | 0.2636 | - | - |
648
+ | 0.8730 | 165 | 0.1858 | 0.1015 | 0.9043 |
649
+ | 0.8783 | 166 | 0.198 | - | - |
650
+ | 0.8836 | 167 | 0.2011 | - | - |
651
+ | 0.8889 | 168 | 0.2887 | - | - |
652
+ | 0.8942 | 169 | 0.2014 | - | - |
653
+ | 0.8995 | 170 | 0.2122 | 0.1002 | 0.9025 |
654
+ | 0.9048 | 171 | 0.2537 | - | - |
655
+ | 0.9101 | 172 | 0.2104 | - | - |
656
+ | 0.9153 | 173 | 0.1291 | - | - |
657
+ | 0.9206 | 174 | 0.1794 | - | - |
658
+ | 0.9259 | 175 | 0.2569 | 0.0948 | 0.9027 |
659
+ | 0.9312 | 176 | 0.26 | - | - |
660
+ | 0.9365 | 177 | 0.1665 | - | - |
661
+ | 0.9418 | 178 | 0.2012 | - | - |
662
+ | 0.9471 | 179 | 0.2171 | - | - |
663
+ | 0.9524 | 180 | 0.1554 | 0.0874 | 0.9055 |
664
+ | 0.9577 | 181 | 0.2177 | - | - |
665
+ | 0.9630 | 182 | 0.1919 | - | - |
666
+ | 0.9683 | 183 | 0.1686 | - | - |
667
+ | 0.9735 | 184 | 0.2277 | - | - |
668
+ | 0.9788 | 185 | 0.2169 | 0.0847 | 0.9089 |
669
+ | 0.9841 | 186 | 0.1991 | - | - |
670
+ | 0.9894 | 187 | 0.2373 | - | - |
671
+ | 0.9947 | 188 | 0.1636 | - | - |
672
+ | 1.0 | 189 | 0.0 | - | - |
673
+ | 1.0053 | 190 | 0.0 | 0.0846 | 0.9103 |
674
+ | 1.0106 | 191 | 0.0 | - | - |
675
+ | 1.0159 | 192 | 0.0 | - | - |
676
+ | 1.0212 | 193 | 0.0 | - | - |
677
+ | 1.0265 | 194 | 0.0 | - | - |
678
+ | 1.0317 | 195 | 0.0 | 0.0859 | 0.9098 |
679
+ | 1.0370 | 196 | 0.0 | - | - |
680
+ | 1.0423 | 197 | 0.0 | - | - |
681
+ | 1.0476 | 198 | 0.0 | - | - |
682
+ | 1.0529 | 199 | 0.0 | - | - |
683
+ | 1.0582 | 200 | 0.0 | 0.0870 | 0.9101 |
684
+ | 1.0635 | 201 | 0.0 | - | - |
685
+ | 1.0688 | 202 | 0.0 | - | - |
686
+ | 1.0741 | 203 | 0.0 | - | - |
687
+ | 1.0794 | 204 | 0.0 | - | - |
688
+ | 1.0847 | 205 | 0.0 | 0.0879 | 0.9100 |
689
+ | 1.0899 | 206 | 0.0 | - | - |
690
+ | 1.0952 | 207 | 0.0 | - | - |
691
+ | 1.1005 | 208 | 0.0 | - | - |
692
+ | 1.1058 | 209 | 0.0 | - | - |
693
+ | 1.1111 | 210 | 0.0 | 0.0884 | 0.9097 |
694
+ | 1.1164 | 211 | 0.0 | - | - |
695
+ | 1.1217 | 212 | 0.0 | - | - |
696
+ | 1.1270 | 213 | 0.0 | - | - |
697
+ | 1.1323 | 214 | 0.0 | - | - |
698
+ | 1.1376 | 215 | 0.0 | 0.0888 | 0.9096 |
699
+ | 1.1429 | 216 | 0.0 | - | - |
700
+ | 1.1481 | 217 | 0.0 | - | - |
701
+ | 1.1534 | 218 | 0.0 | - | - |
702
+ | 1.1587 | 219 | 0.0 | - | - |
703
+ | 1.1640 | 220 | 0.0 | 0.0890 | 0.9095 |
704
+ | 1.1693 | 221 | 0.0 | - | - |
705
+ | 1.1746 | 222 | 0.0 | - | - |
706
+ | 1.1799 | 223 | 0.0 | - | - |
707
+ | 1.1852 | 224 | 0.0 | - | - |
708
+ | 1.1905 | 225 | 0.0 | 0.0891 | 0.9095 |
709
+ | 1.1958 | 226 | 0.0 | - | - |
710
+ | 1.2011 | 227 | 0.0 | - | - |
711
+ | 1.2063 | 228 | 0.0 | - | - |
712
+ | 1.2116 | 229 | 0.0 | - | - |
713
+ | 1.2169 | 230 | 0.0 | 0.0892 | 0.9095 |
714
+ | 1.2222 | 231 | 0.0 | - | - |
715
+ | 1.2275 | 232 | 0.0 | - | - |
716
+ | 1.2328 | 233 | 0.0 | - | - |
717
+ | 1.2381 | 234 | 0.0 | - | - |
718
+ | 1.2434 | 235 | 0.0 | 0.0892 | 0.9094 |
719
+ | 1.2487 | 236 | 0.0 | - | - |
720
+ | 1.2540 | 237 | 0.0 | - | - |
721
+ | 1.2593 | 238 | 0.0 | - | - |
722
+ | 1.2646 | 239 | 0.0 | - | - |
723
+ | 1.2698 | 240 | 0.0 | 0.0892 | 0.9093 |
724
+ | 1.2751 | 241 | 0.0 | - | - |
725
+ | 1.2804 | 242 | 0.0 | - | - |
726
+ | 1.2857 | 243 | 0.0 | - | - |
727
+ | 1.2910 | 244 | 0.0 | - | - |
728
+ | 1.2963 | 245 | 0.0 | 0.0892 | 0.9094 |
729
+ | 1.3016 | 246 | 0.0 | - | - |
730
+ | 1.3069 | 247 | 0.0 | - | - |
731
+ | 1.3122 | 248 | 0.0 | - | - |
732
+ | 1.3175 | 249 | 0.0 | - | - |
733
+ | 1.3228 | 250 | 0.0 | 0.0892 | 0.9094 |
734
+ | 1.3280 | 251 | 0.0 | - | - |
735
+ | 1.3333 | 252 | 0.0 | - | - |
736
+ | 1.3386 | 253 | 0.0 | - | - |
737
+ | 1.3439 | 254 | 0.0 | - | - |
738
+ | 1.3492 | 255 | 0.0 | 0.0893 | 0.9093 |
739
+ | 1.3545 | 256 | 0.0 | - | - |
740
+ | 1.3598 | 257 | 0.0 | - | - |
741
+ | 1.3651 | 258 | 0.0 | - | - |
742
+ | 1.3704 | 259 | 0.0 | - | - |
743
+ | 1.3757 | 260 | 0.0 | 0.0893 | 0.9093 |
744
+ | 1.3810 | 261 | 0.0 | - | - |
745
+ | 1.3862 | 262 | 0.0 | - | - |
746
+ | 1.3915 | 263 | 0.0 | - | - |
747
+ | 1.3968 | 264 | 0.0 | - | - |
748
+ | 1.4021 | 265 | 0.0 | 0.0893 | 0.9093 |
749
+ | 1.4074 | 266 | 0.0 | - | - |
750
+ | 1.4127 | 267 | 0.0 | - | - |
751
+ | 1.4180 | 268 | 0.0 | - | - |
752
+ | 1.4233 | 269 | 0.0 | - | - |
753
+ | 1.4286 | 270 | 0.0 | 0.0893 | 0.9093 |
754
+ | 1.4339 | 271 | 0.0 | - | - |
755
+ | 1.4392 | 272 | 0.0 | - | - |
756
+ | 1.4444 | 273 | 0.0 | - | - |
757
+ | 1.4497 | 274 | 0.0 | - | - |
758
+ | 1.4550 | 275 | 0.0 | 0.0893 | 0.9093 |
759
+ | 1.4603 | 276 | 0.0 | - | - |
760
+ | 1.4656 | 277 | 0.0 | - | - |
761
+ | 1.4709 | 278 | 0.0 | - | - |
762
+ | 1.4762 | 279 | 0.0 | - | - |
763
+ | 1.4815 | 280 | 0.0 | 0.0892 | 0.9093 |
764
+ | 1.4868 | 281 | 0.0 | - | - |
765
+ | 1.4921 | 282 | 0.0 | - | - |
766
+ | 1.4974 | 283 | 0.0 | - | - |
767
+ | 1.5026 | 284 | 0.0 | - | - |
768
+ | 1.5079 | 285 | 0.0 | 0.0892 | 0.9093 |
769
+ | 1.5132 | 286 | 0.0 | - | - |
770
+ | 1.5185 | 287 | 0.0 | - | - |
771
+ | 1.5238 | 288 | 0.0 | - | - |
772
+ | 1.5291 | 289 | 0.0 | - | - |
773
+ | 1.5344 | 290 | 0.0 | 0.0892 | 0.9093 |
774
+ | 1.5397 | 291 | 0.0 | - | - |
775
+ | 1.5450 | 292 | 0.0 | - | - |
776
+ | 1.5503 | 293 | 0.0 | - | - |
777
+ | 1.5556 | 294 | 0.0 | - | - |
778
+ | 1.5608 | 295 | 0.0 | 0.0893 | 0.9093 |
779
+ | 1.5661 | 296 | 0.0 | - | - |
780
+ | 1.5714 | 297 | 0.0 | - | - |
781
+ | 1.5767 | 298 | 0.0 | - | - |
782
+ | 1.5820 | 299 | 0.0 | - | - |
783
+ | 1.5873 | 300 | 0.0 | 0.0892 | 0.9093 |
784
+ | 1.5926 | 301 | 0.0 | - | - |
785
+ | 1.5979 | 302 | 0.0 | - | - |
786
+ | 1.6032 | 303 | 0.0 | - | - |
787
+ | 1.6085 | 304 | 0.0 | - | - |
788
+
789
+ </details>
790
+
791
+ ### Framework Versions
792
+ - Python: 3.10.14
793
+ - Sentence Transformers: 3.0.1
794
+ - Transformers: 4.44.0
795
+ - PyTorch: 2.4.0
796
+ - Accelerate: 0.33.0
797
+ - Datasets: 2.21.0
798
+ - Tokenizers: 0.19.1
799
+
800
+ ## Citation
801
+
802
+ ### BibTeX
803
+
804
+ #### Sentence Transformers
805
+ ```bibtex
806
+ @inproceedings{reimers-2019-sentence-bert,
807
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
808
+ author = "Reimers, Nils and Gurevych, Iryna",
809
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
810
+ month = "11",
811
+ year = "2019",
812
+ publisher = "Association for Computational Linguistics",
813
+ url = "https://arxiv.org/abs/1908.10084",
814
+ }
815
+ ```
816
+
817
+ <!--
818
+ ## Glossary
819
+
820
+ *Clearly define terms in order to be accessible across audiences.*
821
+ -->
822
+
823
+ <!--
824
+ ## Model Card Authors
825
+
826
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
827
+ -->
828
+
829
+ <!--
830
+ ## Model Card Contact
831
+
832
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
833
+ -->
checkpoint-304/added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[MASK]": 128000
3
+ }
checkpoint-304/config.json ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bobox/DeBERTa-small-ST-v1-test-step3",
3
+ "architectures": [
4
+ "DebertaV2Model"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.1,
9
+ "hidden_size": 768,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 3072,
12
+ "layer_norm_eps": 1e-07,
13
+ "max_position_embeddings": 512,
14
+ "max_relative_positions": -1,
15
+ "model_type": "deberta-v2",
16
+ "norm_rel_ebd": "layer_norm",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 6,
19
+ "pad_token_id": 0,
20
+ "pooler_dropout": 0,
21
+ "pooler_hidden_act": "gelu",
22
+ "pooler_hidden_size": 768,
23
+ "pos_att_type": [
24
+ "p2c",
25
+ "c2p"
26
+ ],
27
+ "position_biased_input": false,
28
+ "position_buckets": 256,
29
+ "relative_attention": true,
30
+ "share_att_key": true,
31
+ "torch_dtype": "float32",
32
+ "transformers_version": "4.44.0",
33
+ "type_vocab_size": 0,
34
+ "vocab_size": 128100
35
+ }
checkpoint-304/config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.0.1",
4
+ "transformers": "4.44.0",
5
+ "pytorch": "2.4.0"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
checkpoint-304/modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ }
14
+ ]
checkpoint-304/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33c79a2eb9dc6c64386dc76883c7adfc3c28230b062af4a92d7e35d984a074f0
3
+ size 1130520122
checkpoint-304/pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2faf1d78221b8c5e82f59e610e2c9a8e5e5bbb1f42b07008ca9207d62fb350fb
3
+ size 565251810
checkpoint-304/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b9622de8b1e8357618338f87d30ec41707f836d5bfd2db8f0def65df4e91d12f
3
+ size 14244
checkpoint-304/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2d5b707ea93241730658853b06b91db135c9f42dd67b5fc4b4740c00b567e05
3
+ size 1064
checkpoint-304/sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
checkpoint-304/special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "[CLS]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "[SEP]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "[MASK]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "[PAD]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "[SEP]",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "[UNK]",
46
+ "lstrip": false,
47
+ "normalized": true,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
checkpoint-304/spm.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
3
+ size 2464616
checkpoint-304/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-304/tokenizer_config.json ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "[CLS]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "[SEP]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "[UNK]",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "128000": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "bos_token": "[CLS]",
45
+ "clean_up_tokenization_spaces": true,
46
+ "cls_token": "[CLS]",
47
+ "do_lower_case": false,
48
+ "eos_token": "[SEP]",
49
+ "mask_token": "[MASK]",
50
+ "max_length": 512,
51
+ "model_max_length": 512,
52
+ "pad_to_multiple_of": null,
53
+ "pad_token": "[PAD]",
54
+ "pad_token_type_id": 0,
55
+ "padding_side": "right",
56
+ "sep_token": "[SEP]",
57
+ "sp_model_kwargs": {},
58
+ "split_by_punct": false,
59
+ "stride": 0,
60
+ "tokenizer_class": "DebertaV2Tokenizer",
61
+ "truncation_side": "right",
62
+ "truncation_strategy": "longest_first",
63
+ "unk_token": "[UNK]",
64
+ "vocab_type": "spm"
65
+ }
checkpoint-304/trainer_state.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-304/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2968b16dc322a2bed394ae83a32300ba89607cbee354667b1d1918a368bd3a9a
3
+ size 5752