Hugging Face's logo

sparse-encoder
/
example-splade-distilbert-base-uncased-gooaq

Feature Extraction
sentence-transformers
Safetensors
English
distilbert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:99000
loss:SpladeLoss
loss:SparseMultipleNegativesRankingLoss
loss:FlopsLoss
Eval Results
text-embeddings-inference
Model card Files Community
arthurbresnu HF Staff commited on
Commit
ff6a38d
·
verified ·
1 Parent(s): a83820c

Add new SparseEncoder model

Browse files
Files changed (11) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +1728 -0
  3. config.json +23 -0
  4. config_sentence_transformers.json +11 -0
  5. model.safetensors +3 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
  8. special_tokens_map.json +7 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +56 -0
  11. vocab.txt +0 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1728 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: distilbert/distilbert-base-uncased
16
+ widget:
17
+ - text: 'The term emergent literacy signals a belief that, in a literate society,
18
+ young children even one and two year olds, are in the process of becoming literate”.
19
+ ... Gray (1956:21) notes: Functional literacy is used for the training of adults
20
+ to ''meet independently the reading and writing demands placed on them''.'
21
+ - text: Rey is seemingly confirmed as being The Chosen One per a quote by a Lucasfilm
22
+ production designer who worked on The Rise of Skywalker.
23
+ - text: are union gun safes fireproof?
24
+ - text: Fruit is an essential part of a healthy diet — and may aid weight loss. Most
25
+ fruits are low in calories while high in nutrients and fiber, which can boost
26
+ your fullness. Keep in mind that it's best to eat fruits whole rather than juiced.
27
+ What's more, simply eating fruit is not the key to weight loss.
28
+ - text: Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate
29
+ or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks
30
+ for chronic sinusitis.
31
+ datasets:
32
+ - sentence-transformers/gooaq
33
+ pipeline_tag: feature-extraction
34
+ library_name: sentence-transformers
35
+ metrics:
36
+ - dot_accuracy@1
37
+ - dot_accuracy@3
38
+ - dot_accuracy@5
39
+ - dot_accuracy@10
40
+ - dot_precision@1
41
+ - dot_precision@3
42
+ - dot_precision@5
43
+ - dot_precision@10
44
+ - dot_recall@1
45
+ - dot_recall@3
46
+ - dot_recall@5
47
+ - dot_recall@10
48
+ - dot_ndcg@10
49
+ - dot_mrr@10
50
+ - dot_map@100
51
+ - query_active_dims
52
+ - query_sparsity_ratio
53
+ - corpus_active_dims
54
+ - corpus_sparsity_ratio
55
+ co2_eq_emissions:
56
+ emissions: 1.0881870582723092
57
+ energy_consumed: 0.019418388234485075
58
+ source: codecarbon
59
+ training_type: fine-tuning
60
+ on_cloud: false
61
+ cpu_model: AMD Ryzen 9 6900HX with Radeon Graphics
62
+ ram_total_size: 30.6114501953125
63
+ hours_used: 0.174
64
+ hardware_used: 1 x NVIDIA GeForce RTX 3070 Ti Laptop GPU
65
+ model-index:
66
+ - name: splade-distilbert-base-uncased trained on GooAQ
67
+ results:
68
+ - task:
69
+ type: sparse-information-retrieval
70
+ name: Sparse Information Retrieval
71
+ dataset:
72
+ name: NanoMSMARCO
73
+ type: NanoMSMARCO
74
+ metrics:
75
+ - type: dot_accuracy@1
76
+ value: 0.22
77
+ name: Dot Accuracy@1
78
+ - type: dot_accuracy@3
79
+ value: 0.46
80
+ name: Dot Accuracy@3
81
+ - type: dot_accuracy@5
82
+ value: 0.54
83
+ name: Dot Accuracy@5
84
+ - type: dot_accuracy@10
85
+ value: 0.7
86
+ name: Dot Accuracy@10
87
+ - type: dot_precision@1
88
+ value: 0.22
89
+ name: Dot Precision@1
90
+ - type: dot_precision@3
91
+ value: 0.15333333333333332
92
+ name: Dot Precision@3
93
+ - type: dot_precision@5
94
+ value: 0.10800000000000001
95
+ name: Dot Precision@5
96
+ - type: dot_precision@10
97
+ value: 0.07
98
+ name: Dot Precision@10
99
+ - type: dot_recall@1
100
+ value: 0.22
101
+ name: Dot Recall@1
102
+ - type: dot_recall@3
103
+ value: 0.46
104
+ name: Dot Recall@3
105
+ - type: dot_recall@5
106
+ value: 0.54
107
+ name: Dot Recall@5
108
+ - type: dot_recall@10
109
+ value: 0.7
110
+ name: Dot Recall@10
111
+ - type: dot_ndcg@10
112
+ value: 0.44470504856183124
113
+ name: Dot Ndcg@10
114
+ - type: dot_mrr@10
115
+ value: 0.3652460317460317
116
+ name: Dot Mrr@10
117
+ - type: dot_map@100
118
+ value: 0.37928248813494486
119
+ name: Dot Map@100
120
+ - type: query_active_dims
121
+ value: 125.86000061035156
122
+ name: Query Active Dims
123
+ - type: query_sparsity_ratio
124
+ value: 0.9958764169906837
125
+ name: Query Sparsity Ratio
126
+ - type: corpus_active_dims
127
+ value: 296.2349853515625
128
+ name: Corpus Active Dims
129
+ - type: corpus_sparsity_ratio
130
+ value: 0.9902943783057611
131
+ name: Corpus Sparsity Ratio
132
+ - type: dot_accuracy@1
133
+ value: 0.24
134
+ name: Dot Accuracy@1
135
+ - type: dot_accuracy@3
136
+ value: 0.5
137
+ name: Dot Accuracy@3
138
+ - type: dot_accuracy@5
139
+ value: 0.58
140
+ name: Dot Accuracy@5
141
+ - type: dot_accuracy@10
142
+ value: 0.72
143
+ name: Dot Accuracy@10
144
+ - type: dot_precision@1
145
+ value: 0.24
146
+ name: Dot Precision@1
147
+ - type: dot_precision@3
148
+ value: 0.16666666666666663
149
+ name: Dot Precision@3
150
+ - type: dot_precision@5
151
+ value: 0.11600000000000002
152
+ name: Dot Precision@5
153
+ - type: dot_precision@10
154
+ value: 0.07200000000000001
155
+ name: Dot Precision@10
156
+ - type: dot_recall@1
157
+ value: 0.24
158
+ name: Dot Recall@1
159
+ - type: dot_recall@3
160
+ value: 0.5
161
+ name: Dot Recall@3
162
+ - type: dot_recall@5
163
+ value: 0.58
164
+ name: Dot Recall@5
165
+ - type: dot_recall@10
166
+ value: 0.72
167
+ name: Dot Recall@10
168
+ - type: dot_ndcg@10
169
+ value: 0.47847271089832977
170
+ name: Dot Ndcg@10
171
+ - type: dot_mrr@10
172
+ value: 0.40169047619047615
173
+ name: Dot Mrr@10
174
+ - type: dot_map@100
175
+ value: 0.4140044816294816
176
+ name: Dot Map@100
177
+ - type: query_active_dims
178
+ value: 109.69999694824219
179
+ name: Query Active Dims
180
+ - type: query_sparsity_ratio
181
+ value: 0.9964058712748758
182
+ name: Query Sparsity Ratio
183
+ - type: corpus_active_dims
184
+ value: 265.6180725097656
185
+ name: Corpus Active Dims
186
+ - type: corpus_sparsity_ratio
187
+ value: 0.9912974879591847
188
+ name: Corpus Sparsity Ratio
189
+ - task:
190
+ type: sparse-information-retrieval
191
+ name: Sparse Information Retrieval
192
+ dataset:
193
+ name: NanoNFCorpus
194
+ type: NanoNFCorpus
195
+ metrics:
196
+ - type: dot_accuracy@1
197
+ value: 0.36
198
+ name: Dot Accuracy@1
199
+ - type: dot_accuracy@3
200
+ value: 0.5
201
+ name: Dot Accuracy@3
202
+ - type: dot_accuracy@5
203
+ value: 0.56
204
+ name: Dot Accuracy@5
205
+ - type: dot_accuracy@10
206
+ value: 0.58
207
+ name: Dot Accuracy@10
208
+ - type: dot_precision@1
209
+ value: 0.36
210
+ name: Dot Precision@1
211
+ - type: dot_precision@3
212
+ value: 0.2866666666666666
213
+ name: Dot Precision@3
214
+ - type: dot_precision@5
215
+ value: 0.276
216
+ name: Dot Precision@5
217
+ - type: dot_precision@10
218
+ value: 0.20400000000000001
219
+ name: Dot Precision@10
220
+ - type: dot_recall@1
221
+ value: 0.020432228546915038
222
+ name: Dot Recall@1
223
+ - type: dot_recall@3
224
+ value: 0.05966030415500706
225
+ name: Dot Recall@3
226
+ - type: dot_recall@5
227
+ value: 0.08546529551494754
228
+ name: Dot Recall@5
229
+ - type: dot_recall@10
230
+ value: 0.10325648585391117
231
+ name: Dot Recall@10
232
+ - type: dot_ndcg@10
233
+ value: 0.2586742055175529
234
+ name: Dot Ndcg@10
235
+ - type: dot_mrr@10
236
+ value: 0.444
237
+ name: Dot Mrr@10
238
+ - type: dot_map@100
239
+ value: 0.10277044614671307
240
+ name: Dot Map@100
241
+ - type: query_active_dims
242
+ value: 160.10000610351562
243
+ name: Query Active Dims
244
+ - type: query_sparsity_ratio
245
+ value: 0.9947546030370383
246
+ name: Query Sparsity Ratio
247
+ - type: corpus_active_dims
248
+ value: 409.76904296875
249
+ name: Corpus Active Dims
250
+ - type: corpus_sparsity_ratio
251
+ value: 0.986574633281936
252
+ name: Corpus Sparsity Ratio
253
+ - type: dot_accuracy@1
254
+ value: 0.38
255
+ name: Dot Accuracy@1
256
+ - type: dot_accuracy@3
257
+ value: 0.5
258
+ name: Dot Accuracy@3
259
+ - type: dot_accuracy@5
260
+ value: 0.52
261
+ name: Dot Accuracy@5
262
+ - type: dot_accuracy@10
263
+ value: 0.66
264
+ name: Dot Accuracy@10
265
+ - type: dot_precision@1
266
+ value: 0.38
267
+ name: Dot Precision@1
268
+ - type: dot_precision@3
269
+ value: 0.29333333333333333
270
+ name: Dot Precision@3
271
+ - type: dot_precision@5
272
+ value: 0.268
273
+ name: Dot Precision@5
274
+ - type: dot_precision@10
275
+ value: 0.22799999999999998
276
+ name: Dot Precision@10
277
+ - type: dot_recall@1
278
+ value: 0.03979891140267026
279
+ name: Dot Recall@1
280
+ - type: dot_recall@3
281
+ value: 0.05843303142773433
282
+ name: Dot Recall@3
283
+ - type: dot_recall@5
284
+ value: 0.07656018207627424
285
+ name: Dot Recall@5
286
+ - type: dot_recall@10
287
+ value: 0.10998150964383814
288
+ name: Dot Recall@10
289
+ - type: dot_ndcg@10
290
+ value: 0.28162049888840096
291
+ name: Dot Ndcg@10
292
+ - type: dot_mrr@10
293
+ value: 0.4571904761904762
294
+ name: Dot Mrr@10
295
+ - type: dot_map@100
296
+ value: 0.11433559983443616
297
+ name: Dot Map@100
298
+ - type: query_active_dims
299
+ value: 140.05999755859375
300
+ name: Query Active Dims
301
+ - type: query_sparsity_ratio
302
+ value: 0.9954111789018218
303
+ name: Query Sparsity Ratio
304
+ - type: corpus_active_dims
305
+ value: 371.9038391113281
306
+ name: Corpus Active Dims
307
+ - type: corpus_sparsity_ratio
308
+ value: 0.9878152205258068
309
+ name: Corpus Sparsity Ratio
310
+ - task:
311
+ type: sparse-information-retrieval
312
+ name: Sparse Information Retrieval
313
+ dataset:
314
+ name: NanoNQ
315
+ type: NanoNQ
316
+ metrics:
317
+ - type: dot_accuracy@1
318
+ value: 0.32
319
+ name: Dot Accuracy@1
320
+ - type: dot_accuracy@3
321
+ value: 0.54
322
+ name: Dot Accuracy@3
323
+ - type: dot_accuracy@5
324
+ value: 0.64
325
+ name: Dot Accuracy@5
326
+ - type: dot_accuracy@10
327
+ value: 0.74
328
+ name: Dot Accuracy@10
329
+ - type: dot_precision@1
330
+ value: 0.32
331
+ name: Dot Precision@1
332
+ - type: dot_precision@3
333
+ value: 0.18
334
+ name: Dot Precision@3
335
+ - type: dot_precision@5
336
+ value: 0.136
337
+ name: Dot Precision@5
338
+ - type: dot_precision@10
339
+ value: 0.08
340
+ name: Dot Precision@10
341
+ - type: dot_recall@1
342
+ value: 0.3
343
+ name: Dot Recall@1
344
+ - type: dot_recall@3
345
+ value: 0.5
346
+ name: Dot Recall@3
347
+ - type: dot_recall@5
348
+ value: 0.62
349
+ name: Dot Recall@5
350
+ - type: dot_recall@10
351
+ value: 0.7
352
+ name: Dot Recall@10
353
+ - type: dot_ndcg@10
354
+ value: 0.5013957867971872
355
+ name: Dot Ndcg@10
356
+ - type: dot_mrr@10
357
+ value: 0.4491904761904762
358
+ name: Dot Mrr@10
359
+ - type: dot_map@100
360
+ value: 0.44262111936629595
361
+ name: Dot Map@100
362
+ - type: query_active_dims
363
+ value: 128.39999389648438
364
+ name: Query Active Dims
365
+ - type: query_sparsity_ratio
366
+ value: 0.9957931985487031
367
+ name: Query Sparsity Ratio
368
+ - type: corpus_active_dims
369
+ value: 359.4007873535156
370
+ name: Corpus Active Dims
371
+ - type: corpus_sparsity_ratio
372
+ value: 0.9882248611705158
373
+ name: Corpus Sparsity Ratio
374
+ - type: dot_accuracy@1
375
+ value: 0.36
376
+ name: Dot Accuracy@1
377
+ - type: dot_accuracy@3
378
+ value: 0.58
379
+ name: Dot Accuracy@3
380
+ - type: dot_accuracy@5
381
+ value: 0.66
382
+ name: Dot Accuracy@5
383
+ - type: dot_accuracy@10
384
+ value: 0.72
385
+ name: Dot Accuracy@10
386
+ - type: dot_precision@1
387
+ value: 0.36
388
+ name: Dot Precision@1
389
+ - type: dot_precision@3
390
+ value: 0.19333333333333333
391
+ name: Dot Precision@3
392
+ - type: dot_precision@5
393
+ value: 0.136
394
+ name: Dot Precision@5
395
+ - type: dot_precision@10
396
+ value: 0.078
397
+ name: Dot Precision@10
398
+ - type: dot_recall@1
399
+ value: 0.34
400
+ name: Dot Recall@1
401
+ - type: dot_recall@3
402
+ value: 0.54
403
+ name: Dot Recall@3
404
+ - type: dot_recall@5
405
+ value: 0.63
406
+ name: Dot Recall@5
407
+ - type: dot_recall@10
408
+ value: 0.69
409
+ name: Dot Recall@10
410
+ - type: dot_ndcg@10
411
+ value: 0.5189963924532662
412
+ name: Dot Ndcg@10
413
+ - type: dot_mrr@10
414
+ value: 0.4757777777777777
415
+ name: Dot Mrr@10
416
+ - type: dot_map@100
417
+ value: 0.46913515575703424
418
+ name: Dot Map@100
419
+ - type: query_active_dims
420
+ value: 115.30000305175781
421
+ name: Query Active Dims
422
+ - type: query_sparsity_ratio
423
+ value: 0.9962223968595846
424
+ name: Query Sparsity Ratio
425
+ - type: corpus_active_dims
426
+ value: 336.913818359375
427
+ name: Corpus Active Dims
428
+ - type: corpus_sparsity_ratio
429
+ value: 0.9889616074189316
430
+ name: Corpus Sparsity Ratio
431
+ - task:
432
+ type: sparse-nano-beir
433
+ name: Sparse Nano BEIR
434
+ dataset:
435
+ name: NanoBEIR mean
436
+ type: NanoBEIR_mean
437
+ metrics:
438
+ - type: dot_accuracy@1
439
+ value: 0.3
440
+ name: Dot Accuracy@1
441
+ - type: dot_accuracy@3
442
+ value: 0.5
443
+ name: Dot Accuracy@3
444
+ - type: dot_accuracy@5
445
+ value: 0.5800000000000001
446
+ name: Dot Accuracy@5
447
+ - type: dot_accuracy@10
448
+ value: 0.6733333333333332
449
+ name: Dot Accuracy@10
450
+ - type: dot_precision@1
451
+ value: 0.3
452
+ name: Dot Precision@1
453
+ - type: dot_precision@3
454
+ value: 0.20666666666666664
455
+ name: Dot Precision@3
456
+ - type: dot_precision@5
457
+ value: 0.17333333333333334
458
+ name: Dot Precision@5
459
+ - type: dot_precision@10
460
+ value: 0.11800000000000001
461
+ name: Dot Precision@10
462
+ - type: dot_recall@1
463
+ value: 0.180144076182305
464
+ name: Dot Recall@1
465
+ - type: dot_recall@3
466
+ value: 0.339886768051669
467
+ name: Dot Recall@3
468
+ - type: dot_recall@5
469
+ value: 0.4151550985049825
470
+ name: Dot Recall@5
471
+ - type: dot_recall@10
472
+ value: 0.501085495284637
473
+ name: Dot Recall@10
474
+ - type: dot_ndcg@10
475
+ value: 0.40159168029219044
476
+ name: Dot Ndcg@10
477
+ - type: dot_mrr@10
478
+ value: 0.41947883597883595
479
+ name: Dot Mrr@10
480
+ - type: dot_map@100
481
+ value: 0.30822468454931795
482
+ name: Dot Map@100
483
+ - type: query_active_dims
484
+ value: 138.12000020345053
485
+ name: Query Active Dims
486
+ - type: query_sparsity_ratio
487
+ value: 0.9954747395254749
488
+ name: Query Sparsity Ratio
489
+ - type: corpus_active_dims
490
+ value: 346.36973212643693
491
+ name: Corpus Active Dims
492
+ - type: corpus_sparsity_ratio
493
+ value: 0.9886518009263339
494
+ name: Corpus Sparsity Ratio
495
+ - type: dot_accuracy@1
496
+ value: 0.4301726844583988
497
+ name: Dot Accuracy@1
498
+ - type: dot_accuracy@3
499
+ value: 0.6182417582417583
500
+ name: Dot Accuracy@3
501
+ - type: dot_accuracy@5
502
+ value: 0.6783359497645213
503
+ name: Dot Accuracy@5
504
+ - type: dot_accuracy@10
505
+ value: 0.7722135007849293
506
+ name: Dot Accuracy@10
507
+ - type: dot_precision@1
508
+ value: 0.4301726844583988
509
+ name: Dot Precision@1
510
+ - type: dot_precision@3
511
+ value: 0.274160125588697
512
+ name: Dot Precision@3
513
+ - type: dot_precision@5
514
+ value: 0.21524646781789644
515
+ name: Dot Precision@5
516
+ - type: dot_precision@10
517
+ value: 0.1563861852433281
518
+ name: Dot Precision@10
519
+ - type: dot_recall@1
520
+ value: 0.24332694326060123
521
+ name: Dot Recall@1
522
+ - type: dot_recall@3
523
+ value: 0.38912806185875454
524
+ name: Dot Recall@3
525
+ - type: dot_recall@5
526
+ value: 0.4466126446755131
527
+ name: Dot Recall@5
528
+ - type: dot_recall@10
529
+ value: 0.5378480354517308
530
+ name: Dot Recall@10
531
+ - type: dot_ndcg@10
532
+ value: 0.48091561944614786
533
+ name: Dot Ndcg@10
534
+ - type: dot_mrr@10
535
+ value: 0.5383367720714658
536
+ name: Dot Mrr@10
537
+ - type: dot_map@100
538
+ value: 0.40550699373209664
539
+ name: Dot Map@100
540
+ - type: query_active_dims
541
+ value: 161.59013707612073
542
+ name: Query Active Dims
543
+ - type: query_sparsity_ratio
544
+ value: 0.9947057814993735
545
+ name: Query Sparsity Ratio
546
+ - type: corpus_active_dims
547
+ value: 302.84806046588795
548
+ name: Corpus Active Dims
549
+ - type: corpus_sparsity_ratio
550
+ value: 0.99007771245443
551
+ name: Corpus Sparsity Ratio
552
+ - task:
553
+ type: sparse-information-retrieval
554
+ name: Sparse Information Retrieval
555
+ dataset:
556
+ name: NanoClimateFEVER
557
+ type: NanoClimateFEVER
558
+ metrics:
559
+ - type: dot_accuracy@1
560
+ value: 0.26
561
+ name: Dot Accuracy@1
562
+ - type: dot_accuracy@3
563
+ value: 0.4
564
+ name: Dot Accuracy@3
565
+ - type: dot_accuracy@5
566
+ value: 0.42
567
+ name: Dot Accuracy@5
568
+ - type: dot_accuracy@10
569
+ value: 0.64
570
+ name: Dot Accuracy@10
571
+ - type: dot_precision@1
572
+ value: 0.26
573
+ name: Dot Precision@1
574
+ - type: dot_precision@3
575
+ value: 0.14
576
+ name: Dot Precision@3
577
+ - type: dot_precision@5
578
+ value: 0.09200000000000001
579
+ name: Dot Precision@5
580
+ - type: dot_precision@10
581
+ value: 0.08
582
+ name: Dot Precision@10
583
+ - type: dot_recall@1
584
+ value: 0.13
585
+ name: Dot Recall@1
586
+ - type: dot_recall@3
587
+ value: 0.18
588
+ name: Dot Recall@3
589
+ - type: dot_recall@5
590
+ value: 0.19
591
+ name: Dot Recall@5
592
+ - type: dot_recall@10
593
+ value: 0.30733333333333335
594
+ name: Dot Recall@10
595
+ - type: dot_ndcg@10
596
+ value: 0.2528315611912319
597
+ name: Dot Ndcg@10
598
+ - type: dot_mrr@10
599
+ value: 0.3483253968253968
600
+ name: Dot Mrr@10
601
+ - type: dot_map@100
602
+ value: 0.195000428587257
603
+ name: Dot Map@100
604
+ - type: query_active_dims
605
+ value: 215.39999389648438
606
+ name: Query Active Dims
607
+ - type: query_sparsity_ratio
608
+ value: 0.9929427955606944
609
+ name: Query Sparsity Ratio
610
+ - type: corpus_active_dims
611
+ value: 334.818359375
612
+ name: Corpus Active Dims
613
+ - type: corpus_sparsity_ratio
614
+ value: 0.9890302614712339
615
+ name: Corpus Sparsity Ratio
616
+ - task:
617
+ type: sparse-information-retrieval
618
+ name: Sparse Information Retrieval
619
+ dataset:
620
+ name: NanoDBPedia
621
+ type: NanoDBPedia
622
+ metrics:
623
+ - type: dot_accuracy@1
624
+ value: 0.54
625
+ name: Dot Accuracy@1
626
+ - type: dot_accuracy@3
627
+ value: 0.68
628
+ name: Dot Accuracy@3
629
+ - type: dot_accuracy@5
630
+ value: 0.76
631
+ name: Dot Accuracy@5
632
+ - type: dot_accuracy@10
633
+ value: 0.9
634
+ name: Dot Accuracy@10
635
+ - type: dot_precision@1
636
+ value: 0.54
637
+ name: Dot Precision@1
638
+ - type: dot_precision@3
639
+ value: 0.43333333333333335
640
+ name: Dot Precision@3
641
+ - type: dot_precision@5
642
+ value: 0.4
643
+ name: Dot Precision@5
644
+ - type: dot_precision@10
645
+ value: 0.35999999999999993
646
+ name: Dot Precision@10
647
+ - type: dot_recall@1
648
+ value: 0.04725330037285543
649
+ name: Dot Recall@1
650
+ - type: dot_recall@3
651
+ value: 0.09136010229983793
652
+ name: Dot Recall@3
653
+ - type: dot_recall@5
654
+ value: 0.12256470056683391
655
+ name: Dot Recall@5
656
+ - type: dot_recall@10
657
+ value: 0.24664786941021674
658
+ name: Dot Recall@10
659
+ - type: dot_ndcg@10
660
+ value: 0.43054704834652313
661
+ name: Dot Ndcg@10
662
+ - type: dot_mrr@10
663
+ value: 0.6440714285714284
664
+ name: Dot Mrr@10
665
+ - type: dot_map@100
666
+ value: 0.3239717199251123
667
+ name: Dot Map@100
668
+ - type: query_active_dims
669
+ value: 147.72000122070312
670
+ name: Query Active Dims
671
+ - type: query_sparsity_ratio
672
+ value: 0.9951602122658835
673
+ name: Query Sparsity Ratio
674
+ - type: corpus_active_dims
675
+ value: 295.1452331542969
676
+ name: Corpus Active Dims
677
+ - type: corpus_sparsity_ratio
678
+ value: 0.9903300821324194
679
+ name: Corpus Sparsity Ratio
680
+ - task:
681
+ type: sparse-information-retrieval
682
+ name: Sparse Information Retrieval
683
+ dataset:
684
+ name: NanoFEVER
685
+ type: NanoFEVER
686
+ metrics:
687
+ - type: dot_accuracy@1
688
+ value: 0.56
689
+ name: Dot Accuracy@1
690
+ - type: dot_accuracy@3
691
+ value: 0.78
692
+ name: Dot Accuracy@3
693
+ - type: dot_accuracy@5
694
+ value: 0.86
695
+ name: Dot Accuracy@5
696
+ - type: dot_accuracy@10
697
+ value: 0.92
698
+ name: Dot Accuracy@10
699
+ - type: dot_precision@1
700
+ value: 0.56
701
+ name: Dot Precision@1
702
+ - type: dot_precision@3
703
+ value: 0.26
704
+ name: Dot Precision@3
705
+ - type: dot_precision@5
706
+ value: 0.172
707
+ name: Dot Precision@5
708
+ - type: dot_precision@10
709
+ value: 0.096
710
+ name: Dot Precision@10
711
+ - type: dot_recall@1
712
+ value: 0.5466666666666666
713
+ name: Dot Recall@1
714
+ - type: dot_recall@3
715
+ value: 0.7466666666666666
716
+ name: Dot Recall@3
717
+ - type: dot_recall@5
718
+ value: 0.8066666666666668
719
+ name: Dot Recall@5
720
+ - type: dot_recall@10
721
+ value: 0.8766666666666667
722
+ name: Dot Recall@10
723
+ - type: dot_ndcg@10
724
+ value: 0.7202530021492869
725
+ name: Dot Ndcg@10
726
+ - type: dot_mrr@10
727
+ value: 0.6843809523809523
728
+ name: Dot Mrr@10
729
+ - type: dot_map@100
730
+ value: 0.6647642136958143
731
+ name: Dot Map@100
732
+ - type: query_active_dims
733
+ value: 201.5399932861328
734
+ name: Query Active Dims
735
+ - type: query_sparsity_ratio
736
+ value: 0.9933968942636088
737
+ name: Query Sparsity Ratio
738
+ - type: corpus_active_dims
739
+ value: 374.9945983886719
740
+ name: Corpus Active Dims
741
+ - type: corpus_sparsity_ratio
742
+ value: 0.9877139571984578
743
+ name: Corpus Sparsity Ratio
744
+ - task:
745
+ type: sparse-information-retrieval
746
+ name: Sparse Information Retrieval
747
+ dataset:
748
+ name: NanoFiQA2018
749
+ type: NanoFiQA2018
750
+ metrics:
751
+ - type: dot_accuracy@1
752
+ value: 0.36
753
+ name: Dot Accuracy@1
754
+ - type: dot_accuracy@3
755
+ value: 0.52
756
+ name: Dot Accuracy@3
757
+ - type: dot_accuracy@5
758
+ value: 0.54
759
+ name: Dot Accuracy@5
760
+ - type: dot_accuracy@10
761
+ value: 0.62
762
+ name: Dot Accuracy@10
763
+ - type: dot_precision@1
764
+ value: 0.36
765
+ name: Dot Precision@1
766
+ - type: dot_precision@3
767
+ value: 0.24666666666666667
768
+ name: Dot Precision@3
769
+ - type: dot_precision@5
770
+ value: 0.168
771
+ name: Dot Precision@5
772
+ - type: dot_precision@10
773
+ value: 0.106
774
+ name: Dot Precision@10
775
+ - type: dot_recall@1
776
+ value: 0.18857936507936507
777
+ name: Dot Recall@1
778
+ - type: dot_recall@3
779
+ value: 0.3216825396825396
780
+ name: Dot Recall@3
781
+ - type: dot_recall@5
782
+ value: 0.3532380952380953
783
+ name: Dot Recall@5
784
+ - type: dot_recall@10
785
+ value: 0.4552380952380953
786
+ name: Dot Recall@10
787
+ - type: dot_ndcg@10
788
+ value: 0.3784249151812378
789
+ name: Dot Ndcg@10
790
+ - type: dot_mrr@10
791
+ value: 0.44319047619047613
792
+ name: Dot Mrr@10
793
+ - type: dot_map@100
794
+ value: 0.31981273776184116
795
+ name: Dot Map@100
796
+ - type: query_active_dims
797
+ value: 87.62000274658203
798
+ name: Query Active Dims
799
+ - type: query_sparsity_ratio
800
+ value: 0.9971292837053083
801
+ name: Query Sparsity Ratio
802
+ - type: corpus_active_dims
803
+ value: 275.46795654296875
804
+ name: Corpus Active Dims
805
+ - type: corpus_sparsity_ratio
806
+ value: 0.9909747737191872
807
+ name: Corpus Sparsity Ratio
808
+ - task:
809
+ type: sparse-information-retrieval
810
+ name: Sparse Information Retrieval
811
+ dataset:
812
+ name: NanoHotpotQA
813
+ type: NanoHotpotQA
814
+ metrics:
815
+ - type: dot_accuracy@1
816
+ value: 0.66
817
+ name: Dot Accuracy@1
818
+ - type: dot_accuracy@3
819
+ value: 0.86
820
+ name: Dot Accuracy@3
821
+ - type: dot_accuracy@5
822
+ value: 0.92
823
+ name: Dot Accuracy@5
824
+ - type: dot_accuracy@10
825
+ value: 0.92
826
+ name: Dot Accuracy@10
827
+ - type: dot_precision@1
828
+ value: 0.66
829
+ name: Dot Precision@1
830
+ - type: dot_precision@3
831
+ value: 0.4333333333333333
832
+ name: Dot Precision@3
833
+ - type: dot_precision@5
834
+ value: 0.2879999999999999
835
+ name: Dot Precision@5
836
+ - type: dot_precision@10
837
+ value: 0.15599999999999997
838
+ name: Dot Precision@10
839
+ - type: dot_recall@1
840
+ value: 0.33
841
+ name: Dot Recall@1
842
+ - type: dot_recall@3
843
+ value: 0.65
844
+ name: Dot Recall@3
845
+ - type: dot_recall@5
846
+ value: 0.72
847
+ name: Dot Recall@5
848
+ - type: dot_recall@10
849
+ value: 0.78
850
+ name: Dot Recall@10
851
+ - type: dot_ndcg@10
852
+ value: 0.6985941766475363
853
+ name: Dot Ndcg@10
854
+ - type: dot_mrr@10
855
+ value: 0.7596666666666667
856
+ name: Dot Mrr@10
857
+ - type: dot_map@100
858
+ value: 0.632578269203448
859
+ name: Dot Map@100
860
+ - type: query_active_dims
861
+ value: 131.75999450683594
862
+ name: Query Active Dims
863
+ - type: query_sparsity_ratio
864
+ value: 0.9956831139995139
865
+ name: Query Sparsity Ratio
866
+ - type: corpus_active_dims
867
+ value: 330.9889831542969
868
+ name: Corpus Active Dims
869
+ - type: corpus_sparsity_ratio
870
+ value: 0.9891557242921729
871
+ name: Corpus Sparsity Ratio
872
+ - task:
873
+ type: sparse-information-retrieval
874
+ name: Sparse Information Retrieval
875
+ dataset:
876
+ name: NanoQuoraRetrieval
877
+ type: NanoQuoraRetrieval
878
+ metrics:
879
+ - type: dot_accuracy@1
880
+ value: 0.58
881
+ name: Dot Accuracy@1
882
+ - type: dot_accuracy@3
883
+ value: 0.76
884
+ name: Dot Accuracy@3
885
+ - type: dot_accuracy@5
886
+ value: 0.86
887
+ name: Dot Accuracy@5
888
+ - type: dot_accuracy@10
889
+ value: 0.94
890
+ name: Dot Accuracy@10
891
+ - type: dot_precision@1
892
+ value: 0.58
893
+ name: Dot Precision@1
894
+ - type: dot_precision@3
895
+ value: 0.26
896
+ name: Dot Precision@3
897
+ - type: dot_precision@5
898
+ value: 0.184
899
+ name: Dot Precision@5
900
+ - type: dot_precision@10
901
+ value: 0.11199999999999999
902
+ name: Dot Precision@10
903
+ - type: dot_recall@1
904
+ value: 0.57
905
+ name: Dot Recall@1
906
+ - type: dot_recall@3
907
+ value: 0.7233333333333334
908
+ name: Dot Recall@3
909
+ - type: dot_recall@5
910
+ value: 0.8233333333333333
911
+ name: Dot Recall@5
912
+ - type: dot_recall@10
913
+ value: 0.8953333333333333
914
+ name: Dot Recall@10
915
+ - type: dot_ndcg@10
916
+ value: 0.7379320795882585
917
+ name: Dot Ndcg@10
918
+ - type: dot_mrr@10
919
+ value: 0.6864126984126984
920
+ name: Dot Mrr@10
921
+ - type: dot_map@100
922
+ value: 0.6882004324782192
923
+ name: Dot Map@100
924
+ - type: query_active_dims
925
+ value: 56.70000076293945
926
+ name: Query Active Dims
927
+ - type: query_sparsity_ratio
928
+ value: 0.9981423235448876
929
+ name: Query Sparsity Ratio
930
+ - type: corpus_active_dims
931
+ value: 63.429447174072266
932
+ name: Corpus Active Dims
933
+ - type: corpus_sparsity_ratio
934
+ value: 0.9979218449913483
935
+ name: Corpus Sparsity Ratio
936
+ - task:
937
+ type: sparse-information-retrieval
938
+ name: Sparse Information Retrieval
939
+ dataset:
940
+ name: NanoSCIDOCS
941
+ type: NanoSCIDOCS
942
+ metrics:
943
+ - type: dot_accuracy@1
944
+ value: 0.4
945
+ name: Dot Accuracy@1
946
+ - type: dot_accuracy@3
947
+ value: 0.58
948
+ name: Dot Accuracy@3
949
+ - type: dot_accuracy@5
950
+ value: 0.66
951
+ name: Dot Accuracy@5
952
+ - type: dot_accuracy@10
953
+ value: 0.74
954
+ name: Dot Accuracy@10
955
+ - type: dot_precision@1
956
+ value: 0.4
957
+ name: Dot Precision@1
958
+ - type: dot_precision@3
959
+ value: 0.2533333333333333
960
+ name: Dot Precision@3
961
+ - type: dot_precision@5
962
+ value: 0.228
963
+ name: Dot Precision@5
964
+ - type: dot_precision@10
965
+ value: 0.154
966
+ name: Dot Precision@10
967
+ - type: dot_recall@1
968
+ value: 0.08466666666666667
969
+ name: Dot Recall@1
970
+ - type: dot_recall@3
971
+ value: 0.15866666666666668
972
+ name: Dot Recall@3
973
+ - type: dot_recall@5
974
+ value: 0.23566666666666666
975
+ name: Dot Recall@5
976
+ - type: dot_recall@10
977
+ value: 0.31666666666666665
978
+ name: Dot Recall@10
979
+ - type: dot_ndcg@10
980
+ value: 0.307302076202993
981
+ name: Dot Ndcg@10
982
+ - type: dot_mrr@10
983
+ value: 0.5031111111111111
984
+ name: Dot Mrr@10
985
+ - type: dot_map@100
986
+ value: 0.2314013330851555
987
+ name: Dot Map@100
988
+ - type: query_active_dims
989
+ value: 219.97999572753906
990
+ name: Query Active Dims
991
+ - type: query_sparsity_ratio
992
+ value: 0.9927927398031735
993
+ name: Query Sparsity Ratio
994
+ - type: corpus_active_dims
995
+ value: 370.2647399902344
996
+ name: Corpus Active Dims
997
+ - type: corpus_sparsity_ratio
998
+ value: 0.98786892274457
999
+ name: Corpus Sparsity Ratio
1000
+ - task:
1001
+ type: sparse-information-retrieval
1002
+ name: Sparse Information Retrieval
1003
+ dataset:
1004
+ name: NanoArguAna
1005
+ type: NanoArguAna
1006
+ metrics:
1007
+ - type: dot_accuracy@1
1008
+ value: 0.1
1009
+ name: Dot Accuracy@1
1010
+ - type: dot_accuracy@3
1011
+ value: 0.38
1012
+ name: Dot Accuracy@3
1013
+ - type: dot_accuracy@5
1014
+ value: 0.46
1015
+ name: Dot Accuracy@5
1016
+ - type: dot_accuracy@10
1017
+ value: 0.54
1018
+ name: Dot Accuracy@10
1019
+ - type: dot_precision@1
1020
+ value: 0.1
1021
+ name: Dot Precision@1
1022
+ - type: dot_precision@3
1023
+ value: 0.12666666666666665
1024
+ name: Dot Precision@3
1025
+ - type: dot_precision@5
1026
+ value: 0.09200000000000001
1027
+ name: Dot Precision@5
1028
+ - type: dot_precision@10
1029
+ value: 0.05400000000000001
1030
+ name: Dot Precision@10
1031
+ - type: dot_recall@1
1032
+ value: 0.1
1033
+ name: Dot Recall@1
1034
+ - type: dot_recall@3
1035
+ value: 0.38
1036
+ name: Dot Recall@3
1037
+ - type: dot_recall@5
1038
+ value: 0.46
1039
+ name: Dot Recall@5
1040
+ - type: dot_recall@10
1041
+ value: 0.54
1042
+ name: Dot Recall@10
1043
+ - type: dot_ndcg@10
1044
+ value: 0.314067080699688
1045
+ name: Dot Ndcg@10
1046
+ - type: dot_mrr@10
1047
+ value: 0.24191269841269844
1048
+ name: Dot Mrr@10
1049
+ - type: dot_map@100
1050
+ value: 0.2544871127158089
1051
+ name: Dot Map@100
1052
+ - type: query_active_dims
1053
+ value: 392.3999938964844
1054
+ name: Query Active Dims
1055
+ - type: query_sparsity_ratio
1056
+ value: 0.98714369982647
1057
+ name: Query Sparsity Ratio
1058
+ - type: corpus_active_dims
1059
+ value: 371.9895324707031
1060
+ name: Corpus Active Dims
1061
+ - type: corpus_sparsity_ratio
1062
+ value: 0.9878124129326157
1063
+ name: Corpus Sparsity Ratio
1064
+ - task:
1065
+ type: sparse-information-retrieval
1066
+ name: Sparse Information Retrieval
1067
+ dataset:
1068
+ name: NanoSciFact
1069
+ type: NanoSciFact
1070
+ metrics:
1071
+ - type: dot_accuracy@1
1072
+ value: 0.54
1073
+ name: Dot Accuracy@1
1074
+ - type: dot_accuracy@3
1075
+ value: 0.64
1076
+ name: Dot Accuracy@3
1077
+ - type: dot_accuracy@5
1078
+ value: 0.66
1079
+ name: Dot Accuracy@5
1080
+ - type: dot_accuracy@10
1081
+ value: 0.78
1082
+ name: Dot Accuracy@10
1083
+ - type: dot_precision@1
1084
+ value: 0.54
1085
+ name: Dot Precision@1
1086
+ - type: dot_precision@3
1087
+ value: 0.22
1088
+ name: Dot Precision@3
1089
+ - type: dot_precision@5
1090
+ value: 0.14400000000000002
1091
+ name: Dot Precision@5
1092
+ - type: dot_precision@10
1093
+ value: 0.08599999999999998
1094
+ name: Dot Precision@10
1095
+ - type: dot_recall@1
1096
+ value: 0.505
1097
+ name: Dot Recall@1
1098
+ - type: dot_recall@3
1099
+ value: 0.6
1100
+ name: Dot Recall@3
1101
+ - type: dot_recall@5
1102
+ value: 0.635
1103
+ name: Dot Recall@5
1104
+ - type: dot_recall@10
1105
+ value: 0.76
1106
+ name: Dot Recall@10
1107
+ - type: dot_ndcg@10
1108
+ value: 0.6330847757650383
1109
+ name: Dot Ndcg@10
1110
+ - type: dot_mrr@10
1111
+ value: 0.6099365079365079
1112
+ name: Dot Mrr@10
1113
+ - type: dot_map@100
1114
+ value: 0.5921039809068559
1115
+ name: Dot Map@100
1116
+ - type: query_active_dims
1117
+ value: 239.02000427246094
1118
+ name: Query Active Dims
1119
+ - type: query_sparsity_ratio
1120
+ value: 0.9921689271911257
1121
+ name: Query Sparsity Ratio
1122
+ - type: corpus_active_dims
1123
+ value: 362.61492919921875
1124
+ name: Corpus Active Dims
1125
+ - type: corpus_sparsity_ratio
1126
+ value: 0.9881195554288966
1127
+ name: Corpus Sparsity Ratio
1128
+ - task:
1129
+ type: sparse-information-retrieval
1130
+ name: Sparse Information Retrieval
1131
+ dataset:
1132
+ name: NanoTouche2020
1133
+ type: NanoTouche2020
1134
+ metrics:
1135
+ - type: dot_accuracy@1
1136
+ value: 0.6122448979591837
1137
+ name: Dot Accuracy@1
1138
+ - type: dot_accuracy@3
1139
+ value: 0.8571428571428571
1140
+ name: Dot Accuracy@3
1141
+ - type: dot_accuracy@5
1142
+ value: 0.9183673469387755
1143
+ name: Dot Accuracy@5
1144
+ - type: dot_accuracy@10
1145
+ value: 0.9387755102040817
1146
+ name: Dot Accuracy@10
1147
+ - type: dot_precision@1
1148
+ value: 0.6122448979591837
1149
+ name: Dot Precision@1
1150
+ - type: dot_precision@3
1151
+ value: 0.5374149659863945
1152
+ name: Dot Precision@3
1153
+ - type: dot_precision@5
1154
+ value: 0.5102040816326532
1155
+ name: Dot Precision@5
1156
+ - type: dot_precision@10
1157
+ value: 0.4510204081632653
1158
+ name: Dot Precision@10
1159
+ - type: dot_recall@1
1160
+ value: 0.04128535219959204
1161
+ name: Dot Recall@1
1162
+ - type: dot_recall@3
1163
+ value: 0.10852246408702973
1164
+ name: Dot Recall@3
1165
+ - type: dot_recall@5
1166
+ value: 0.17293473623380118
1167
+ name: Dot Recall@5
1168
+ - type: dot_recall@10
1169
+ value: 0.29415698658034994
1170
+ name: Dot Recall@10
1171
+ - type: dot_ndcg@10
1172
+ value: 0.4997767347881314
1173
+ name: Dot Ndcg@10
1174
+ - type: dot_mrr@10
1175
+ value: 0.7427113702623908
1176
+ name: Dot Mrr@10
1177
+ - type: dot_map@100
1178
+ value: 0.37179545293679184
1179
+ name: Dot Map@100
1180
+ - type: query_active_dims
1181
+ value: 41.06122589111328
1182
+ name: Query Active Dims
1183
+ - type: query_sparsity_ratio
1184
+ value: 0.9986547006784905
1185
+ name: Query Sparsity Ratio
1186
+ - type: corpus_active_dims
1187
+ value: 307.7058410644531
1188
+ name: Corpus Active Dims
1189
+ - type: corpus_sparsity_ratio
1190
+ value: 0.9899185557609445
1191
+ name: Corpus Sparsity Ratio
1192
+ ---
1193
+
1194
+ # splade-distilbert-base-uncased trained on GooAQ
1195
+
1196
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1197
+ ## Model Details
1198
+
1199
+ ### Model Description
1200
+ - **Model Type:** SPLADE Sparse Encoder
1201
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
1202
+ - **Maximum Sequence Length:** 256 tokens
1203
+ - **Output Dimensionality:** 30522 dimensions
1204
+ - **Similarity Function:** Dot Product
1205
+ - **Training Dataset:**
1206
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
1207
+ - **Language:** en
1208
+ - **License:** apache-2.0
1209
+
1210
+ ### Model Sources
1211
+
1212
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1213
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1214
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
1215
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1216
+
1217
+ ### Full Model Architecture
1218
+
1219
+ ```
1220
+ SparseEncoder(
1221
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1222
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
1223
+ )
1224
+ ```
1225
+
1226
+ ## Usage
1227
+
1228
+ ### Direct Usage (Sentence Transformers)
1229
+
1230
+ First install the Sentence Transformers library:
1231
+
1232
+ ```bash
1233
+ pip install -U sentence-transformers
1234
+ ```
1235
+
1236
+ Then you can load this model and run inference.
1237
+ ```python
1238
+ from sentence_transformers import SparseEncoder
1239
+
1240
+ # Download from the 🤗 Hub
1241
+ model = SparseEncoder("arthurbresnu/splade-distilbert-base-uncased-gooaq")
1242
+ # Run inference
1243
+ sentences = [
1244
+ 'how many days for doxycycline to work on sinus infection?',
1245
+ 'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.',
1246
+ 'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.',
1247
+ ]
1248
+ embeddings = model.encode(sentences)
1249
+ print(embeddings.shape)
1250
+ # (3, 30522)
1251
+
1252
+ # Get the similarity scores for the embeddings
1253
+ similarities = model.similarity(embeddings, embeddings)
1254
+ print(similarities.shape)
1255
+ # [3, 3]
1256
+ ```
1257
+
1258
+ <!--
1259
+ ### Direct Usage (Transformers)
1260
+
1261
+ <details><summary>Click to see the direct usage in Transformers</summary>
1262
+
1263
+ </details>
1264
+ -->
1265
+
1266
+ <!--
1267
+ ### Downstream Usage (Sentence Transformers)
1268
+
1269
+ You can finetune this model on your own dataset.
1270
+
1271
+ <details><summary>Click to expand</summary>
1272
+
1273
+ </details>
1274
+ -->
1275
+
1276
+ <!--
1277
+ ### Out-of-Scope Use
1278
+
1279
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1280
+ -->
1281
+
1282
+ ## Evaluation
1283
+
1284
+ ### Metrics
1285
+
1286
+ #### Sparse Information Retrieval
1287
+
1288
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
1289
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
1290
+
1291
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
1292
+ |:----------------------|:------------|:-------------|:----------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------|
1293
+ | dot_accuracy@1 | 0.24 | 0.38 | 0.36 | 0.26 | 0.54 | 0.56 | 0.36 | 0.66 | 0.58 | 0.4 | 0.1 | 0.54 | 0.6122 |
1294
+ | dot_accuracy@3 | 0.5 | 0.5 | 0.58 | 0.4 | 0.68 | 0.78 | 0.52 | 0.86 | 0.76 | 0.58 | 0.38 | 0.64 | 0.8571 |
1295
+ | dot_accuracy@5 | 0.58 | 0.52 | 0.66 | 0.42 | 0.76 | 0.86 | 0.54 | 0.92 | 0.86 | 0.66 | 0.46 | 0.66 | 0.9184 |
1296
+ | dot_accuracy@10 | 0.72 | 0.66 | 0.72 | 0.64 | 0.9 | 0.92 | 0.62 | 0.92 | 0.94 | 0.74 | 0.54 | 0.78 | 0.9388 |
1297
+ | dot_precision@1 | 0.24 | 0.38 | 0.36 | 0.26 | 0.54 | 0.56 | 0.36 | 0.66 | 0.58 | 0.4 | 0.1 | 0.54 | 0.6122 |
1298
+ | dot_precision@3 | 0.1667 | 0.2933 | 0.1933 | 0.14 | 0.4333 | 0.26 | 0.2467 | 0.4333 | 0.26 | 0.2533 | 0.1267 | 0.22 | 0.5374 |
1299
+ | dot_precision@5 | 0.116 | 0.268 | 0.136 | 0.092 | 0.4 | 0.172 | 0.168 | 0.288 | 0.184 | 0.228 | 0.092 | 0.144 | 0.5102 |
1300
+ | dot_precision@10 | 0.072 | 0.228 | 0.078 | 0.08 | 0.36 | 0.096 | 0.106 | 0.156 | 0.112 | 0.154 | 0.054 | 0.086 | 0.451 |
1301
+ | dot_recall@1 | 0.24 | 0.0398 | 0.34 | 0.13 | 0.0473 | 0.5467 | 0.1886 | 0.33 | 0.57 | 0.0847 | 0.1 | 0.505 | 0.0413 |
1302
+ | dot_recall@3 | 0.5 | 0.0584 | 0.54 | 0.18 | 0.0914 | 0.7467 | 0.3217 | 0.65 | 0.7233 | 0.1587 | 0.38 | 0.6 | 0.1085 |
1303
+ | dot_recall@5 | 0.58 | 0.0766 | 0.63 | 0.19 | 0.1226 | 0.8067 | 0.3532 | 0.72 | 0.8233 | 0.2357 | 0.46 | 0.635 | 0.1729 |
1304
+ | dot_recall@10 | 0.72 | 0.11 | 0.69 | 0.3073 | 0.2466 | 0.8767 | 0.4552 | 0.78 | 0.8953 | 0.3167 | 0.54 | 0.76 | 0.2942 |
1305
+ | **dot_ndcg@10** | **0.4785** | **0.2816** | **0.519** | **0.2528** | **0.4305** | **0.7203** | **0.3784** | **0.6986** | **0.7379** | **0.3073** | **0.3141** | **0.6331** | **0.4998** |
1306
+ | dot_mrr@10 | 0.4017 | 0.4572 | 0.4758 | 0.3483 | 0.6441 | 0.6844 | 0.4432 | 0.7597 | 0.6864 | 0.5031 | 0.2419 | 0.6099 | 0.7427 |
1307
+ | dot_map@100 | 0.414 | 0.1143 | 0.4691 | 0.195 | 0.324 | 0.6648 | 0.3198 | 0.6326 | 0.6882 | 0.2314 | 0.2545 | 0.5921 | 0.3718 |
1308
+ | query_active_dims | 109.7 | 140.06 | 115.3 | 215.4 | 147.72 | 201.54 | 87.62 | 131.76 | 56.7 | 219.98 | 392.4 | 239.02 | 41.0612 |
1309
+ | query_sparsity_ratio | 0.9964 | 0.9954 | 0.9962 | 0.9929 | 0.9952 | 0.9934 | 0.9971 | 0.9957 | 0.9981 | 0.9928 | 0.9871 | 0.9922 | 0.9987 |
1310
+ | corpus_active_dims | 265.6181 | 371.9038 | 336.9138 | 334.8184 | 295.1452 | 374.9946 | 275.468 | 330.989 | 63.4294 | 370.2647 | 371.9895 | 362.6149 | 307.7058 |
1311
+ | corpus_sparsity_ratio | 0.9913 | 0.9878 | 0.989 | 0.989 | 0.9903 | 0.9877 | 0.991 | 0.9892 | 0.9979 | 0.9879 | 0.9878 | 0.9881 | 0.9899 |
1312
+
1313
+ #### Sparse Nano BEIR
1314
+
1315
+ * Dataset: `NanoBEIR_mean`
1316
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1317
+ ```json
1318
+ {
1319
+ "dataset_names": [
1320
+ "msmarco",
1321
+ "nfcorpus",
1322
+ "nq"
1323
+ ]
1324
+ }
1325
+ ```
1326
+
1327
+ | Metric | Value |
1328
+ |:----------------------|:-----------|
1329
+ | dot_accuracy@1 | 0.3 |
1330
+ | dot_accuracy@3 | 0.5 |
1331
+ | dot_accuracy@5 | 0.58 |
1332
+ | dot_accuracy@10 | 0.6733 |
1333
+ | dot_precision@1 | 0.3 |
1334
+ | dot_precision@3 | 0.2067 |
1335
+ | dot_precision@5 | 0.1733 |
1336
+ | dot_precision@10 | 0.118 |
1337
+ | dot_recall@1 | 0.1801 |
1338
+ | dot_recall@3 | 0.3399 |
1339
+ | dot_recall@5 | 0.4152 |
1340
+ | dot_recall@10 | 0.5011 |
1341
+ | **dot_ndcg@10** | **0.4016** |
1342
+ | dot_mrr@10 | 0.4195 |
1343
+ | dot_map@100 | 0.3082 |
1344
+ | query_active_dims | 138.12 |
1345
+ | query_sparsity_ratio | 0.9955 |
1346
+ | corpus_active_dims | 346.3697 |
1347
+ | corpus_sparsity_ratio | 0.9887 |
1348
+
1349
+ #### Sparse Nano BEIR
1350
+
1351
+ * Dataset: `NanoBEIR_mean`
1352
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1353
+ ```json
1354
+ {
1355
+ "dataset_names": [
1356
+ "climatefever",
1357
+ "dbpedia",
1358
+ "fever",
1359
+ "fiqa2018",
1360
+ "hotpotqa",
1361
+ "msmarco",
1362
+ "nfcorpus",
1363
+ "nq",
1364
+ "quoraretrieval",
1365
+ "scidocs",
1366
+ "arguana",
1367
+ "scifact",
1368
+ "touche2020"
1369
+ ]
1370
+ }
1371
+ ```
1372
+
1373
+ | Metric | Value |
1374
+ |:----------------------|:-----------|
1375
+ | dot_accuracy@1 | 0.4302 |
1376
+ | dot_accuracy@3 | 0.6182 |
1377
+ | dot_accuracy@5 | 0.6783 |
1378
+ | dot_accuracy@10 | 0.7722 |
1379
+ | dot_precision@1 | 0.4302 |
1380
+ | dot_precision@3 | 0.2742 |
1381
+ | dot_precision@5 | 0.2152 |
1382
+ | dot_precision@10 | 0.1564 |
1383
+ | dot_recall@1 | 0.2433 |
1384
+ | dot_recall@3 | 0.3891 |
1385
+ | dot_recall@5 | 0.4466 |
1386
+ | dot_recall@10 | 0.5378 |
1387
+ | **dot_ndcg@10** | **0.4809** |
1388
+ | dot_mrr@10 | 0.5383 |
1389
+ | dot_map@100 | 0.4055 |
1390
+ | query_active_dims | 161.5901 |
1391
+ | query_sparsity_ratio | 0.9947 |
1392
+ | corpus_active_dims | 302.8481 |
1393
+ | corpus_sparsity_ratio | 0.9901 |
1394
+
1395
+ <!--
1396
+ ## Bias, Risks and Limitations
1397
+
1398
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1399
+ -->
1400
+
1401
+ <!--
1402
+ ### Recommendations
1403
+
1404
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1405
+ -->
1406
+
1407
+ ## Training Details
1408
+
1409
+ ### Training Dataset
1410
+
1411
+ #### gooaq
1412
+
1413
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1414
+ * Size: 99,000 training samples
1415
+ * Columns: <code>question</code> and <code>answer</code>
1416
+ * Approximate statistics based on the first 1000 samples:
1417
+ | | question | answer |
1418
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1419
+ | type | string | string |
1420
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.79 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.02 tokens</li><li>max: 153 tokens</li></ul> |
1421
+ * Samples:
1422
+ | question | answer |
1423
+ |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1424
+ | <code>what are the 5 characteristics of a star?</code> | <code>Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.</code> |
1425
+ | <code>are copic markers alcohol ink?</code> | <code>Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.</code> |
1426
+ | <code>what is the difference between appellate term and appellate division?</code> | <code>Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.</code> |
1427
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1428
+ ```json
1429
+ {
1430
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1431
+ "lambda_corpus": 3e-05,
1432
+ "lambda_query": 5e-05
1433
+ }
1434
+ ```
1435
+
1436
+ ### Evaluation Dataset
1437
+
1438
+ #### gooaq
1439
+
1440
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1441
+ * Size: 1,000 evaluation samples
1442
+ * Columns: <code>question</code> and <code>answer</code>
1443
+ * Approximate statistics based on the first 1000 samples:
1444
+ | | question | answer |
1445
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1446
+ | type | string | string |
1447
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.84 tokens</li><li>max: 127 tokens</li></ul> |
1448
+ * Samples:
1449
+ | question | answer |
1450
+ |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1451
+ | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code> |
1452
+ | <code>how old do you have to be to work in sc?</code> | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code> |
1453
+ | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
1454
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1455
+ ```json
1456
+ {
1457
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1458
+ "lambda_corpus": 3e-05,
1459
+ "lambda_query": 5e-05
1460
+ }
1461
+ ```
1462
+
1463
+ ### Training Hyperparameters
1464
+ #### Non-Default Hyperparameters
1465
+
1466
+ - `eval_strategy`: steps
1467
+ - `per_device_train_batch_size`: 32
1468
+ - `per_device_eval_batch_size`: 32
1469
+ - `learning_rate`: 2e-05
1470
+ - `num_train_epochs`: 1
1471
+ - `bf16`: True
1472
+ - `load_best_model_at_end`: True
1473
+ - `batch_sampler`: no_duplicates
1474
+
1475
+ #### All Hyperparameters
1476
+ <details><summary>Click to expand</summary>
1477
+
1478
+ - `overwrite_output_dir`: False
1479
+ - `do_predict`: False
1480
+ - `eval_strategy`: steps
1481
+ - `prediction_loss_only`: True
1482
+ - `per_device_train_batch_size`: 32
1483
+ - `per_device_eval_batch_size`: 32
1484
+ - `per_gpu_train_batch_size`: None
1485
+ - `per_gpu_eval_batch_size`: None
1486
+ - `gradient_accumulation_steps`: 1
1487
+ - `eval_accumulation_steps`: None
1488
+ - `torch_empty_cache_steps`: None
1489
+ - `learning_rate`: 2e-05
1490
+ - `weight_decay`: 0.0
1491
+ - `adam_beta1`: 0.9
1492
+ - `adam_beta2`: 0.999
1493
+ - `adam_epsilon`: 1e-08
1494
+ - `max_grad_norm`: 1.0
1495
+ - `num_train_epochs`: 1
1496
+ - `max_steps`: -1
1497
+ - `lr_scheduler_type`: linear
1498
+ - `lr_scheduler_kwargs`: {}
1499
+ - `warmup_ratio`: 0.0
1500
+ - `warmup_steps`: 0
1501
+ - `log_level`: passive
1502
+ - `log_level_replica`: warning
1503
+ - `log_on_each_node`: True
1504
+ - `logging_nan_inf_filter`: True
1505
+ - `save_safetensors`: True
1506
+ - `save_on_each_node`: False
1507
+ - `save_only_model`: False
1508
+ - `restore_callback_states_from_checkpoint`: False
1509
+ - `no_cuda`: False
1510
+ - `use_cpu`: False
1511
+ - `use_mps_device`: False
1512
+ - `seed`: 42
1513
+ - `data_seed`: None
1514
+ - `jit_mode_eval`: False
1515
+ - `use_ipex`: False
1516
+ - `bf16`: True
1517
+ - `fp16`: False
1518
+ - `fp16_opt_level`: O1
1519
+ - `half_precision_backend`: auto
1520
+ - `bf16_full_eval`: False
1521
+ - `fp16_full_eval`: False
1522
+ - `tf32`: None
1523
+ - `local_rank`: 0
1524
+ - `ddp_backend`: None
1525
+ - `tpu_num_cores`: None
1526
+ - `tpu_metrics_debug`: False
1527
+ - `debug`: []
1528
+ - `dataloader_drop_last`: False
1529
+ - `dataloader_num_workers`: 0
1530
+ - `dataloader_prefetch_factor`: None
1531
+ - `past_index`: -1
1532
+ - `disable_tqdm`: False
1533
+ - `remove_unused_columns`: True
1534
+ - `label_names`: None
1535
+ - `load_best_model_at_end`: True
1536
+ - `ignore_data_skip`: False
1537
+ - `fsdp`: []
1538
+ - `fsdp_min_num_params`: 0
1539
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1540
+ - `tp_size`: 0
1541
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1542
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1543
+ - `deepspeed`: None
1544
+ - `label_smoothing_factor`: 0.0
1545
+ - `optim`: adamw_torch
1546
+ - `optim_args`: None
1547
+ - `adafactor`: False
1548
+ - `group_by_length`: False
1549
+ - `length_column_name`: length
1550
+ - `ddp_find_unused_parameters`: None
1551
+ - `ddp_bucket_cap_mb`: None
1552
+ - `ddp_broadcast_buffers`: False
1553
+ - `dataloader_pin_memory`: True
1554
+ - `dataloader_persistent_workers`: False
1555
+ - `skip_memory_metrics`: True
1556
+ - `use_legacy_prediction_loop`: False
1557
+ - `push_to_hub`: False
1558
+ - `resume_from_checkpoint`: None
1559
+ - `hub_model_id`: None
1560
+ - `hub_strategy`: every_save
1561
+ - `hub_private_repo`: None
1562
+ - `hub_always_push`: False
1563
+ - `gradient_checkpointing`: False
1564
+ - `gradient_checkpointing_kwargs`: None
1565
+ - `include_inputs_for_metrics`: False
1566
+ - `include_for_metrics`: []
1567
+ - `eval_do_concat_batches`: True
1568
+ - `fp16_backend`: auto
1569
+ - `push_to_hub_model_id`: None
1570
+ - `push_to_hub_organization`: None
1571
+ - `mp_parameters`:
1572
+ - `auto_find_batch_size`: False
1573
+ - `full_determinism`: False
1574
+ - `torchdynamo`: None
1575
+ - `ray_scope`: last
1576
+ - `ddp_timeout`: 1800
1577
+ - `torch_compile`: False
1578
+ - `torch_compile_backend`: None
1579
+ - `torch_compile_mode`: None
1580
+ - `dispatch_batches`: None
1581
+ - `split_batches`: None
1582
+ - `include_tokens_per_second`: False
1583
+ - `include_num_input_tokens_seen`: False
1584
+ - `neftune_noise_alpha`: None
1585
+ - `optim_target_modules`: None
1586
+ - `batch_eval_metrics`: False
1587
+ - `eval_on_start`: False
1588
+ - `use_liger_kernel`: False
1589
+ - `eval_use_gather_object`: False
1590
+ - `average_tokens_across_devices`: False
1591
+ - `prompts`: None
1592
+ - `batch_sampler`: no_duplicates
1593
+ - `multi_dataset_batch_sampler`: proportional
1594
+
1595
+ </details>
1596
+
1597
+ ### Training Logs
1598
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
1599
+ |:----------:|:--------:|:-------------:|:---------------:|:-----------------------:|:------------------------:|:------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|
1600
+ | 0.0323 | 100 | 15.2006 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1601
+ | 0.0646 | 200 | 0.2384 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1602
+ | 0.0970 | 300 | 0.1932 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1603
+ | 0.1293 | 400 | 0.1428 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1604
+ | 0.1616 | 500 | 0.144 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1605
+ | 0.1939 | 600 | 0.1345 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1606
+ | 0.1972 | 610 | - | 0.1199 | 0.4364 | 0.2195 | 0.4998 | 0.3853 | - | - | - | - | - | - | - | - | - | - |
1607
+ | 0.2262 | 700 | 0.1406 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1608
+ | 0.2586 | 800 | 0.1012 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1609
+ | 0.2909 | 900 | 0.112 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1610
+ | 0.3232 | 1000 | 0.0736 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1611
+ | 0.3555 | 1100 | 0.0943 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1612
+ | 0.3878 | 1200 | 0.0901 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1613
+ | 0.3943 | 1220 | - | 0.1126 | 0.4706 | 0.2490 | 0.5154 | 0.4117 | - | - | - | - | - | - | - | - | - | - |
1614
+ | 0.4202 | 1300 | 0.0988 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1615
+ | 0.4525 | 1400 | 0.0953 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1616
+ | 0.4848 | 1500 | 0.1145 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1617
+ | 0.5171 | 1600 | 0.0928 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1618
+ | 0.5495 | 1700 | 0.0963 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1619
+ | 0.5818 | 1800 | 0.0724 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1620
+ | 0.5915 | 1830 | - | 0.0736 | 0.4576 | 0.2457 | 0.5015 | 0.4016 | - | - | - | - | - | - | - | - | - | - |
1621
+ | 0.6141 | 1900 | 0.0753 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1622
+ | 0.6464 | 2000 | 0.0657 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1623
+ | 0.6787 | 2100 | 0.0741 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1624
+ | 0.7111 | 2200 | 0.0671 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1625
+ | 0.7434 | 2300 | 0.1013 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1626
+ | 0.7757 | 2400 | 0.0795 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1627
+ | **0.7886** | **2440** | **-** | **0.0719** | **0.4785** | **0.2816** | **0.519** | **0.4264** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1628
+ | 0.8080 | 2500 | 0.0666 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1629
+ | 0.8403 | 2600 | 0.0589 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1630
+ | 0.8727 | 2700 | 0.0569 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1631
+ | 0.9050 | 2800 | 0.0754 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1632
+ | 0.9373 | 2900 | 0.0724 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1633
+ | 0.9696 | 3000 | 0.0658 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1634
+ | 0.9858 | 3050 | - | 0.0661 | 0.4447 | 0.2587 | 0.5014 | 0.4016 | - | - | - | - | - | - | - | - | - | - |
1635
+ | -1 | -1 | - | - | 0.4785 | 0.2816 | 0.5190 | 0.4809 | 0.2528 | 0.4305 | 0.7203 | 0.3784 | 0.6986 | 0.7379 | 0.3073 | 0.3141 | 0.6331 | 0.4998 |
1636
+
1637
+ * The bold row denotes the saved checkpoint.
1638
+
1639
+ ### Environmental Impact
1640
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1641
+ - **Energy Consumed**: 0.019 kWh
1642
+ - **Carbon Emitted**: 0.001 kg of CO2
1643
+ - **Hours Used**: 0.174 hours
1644
+
1645
+ ### Training Hardware
1646
+ - **On Cloud**: No
1647
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3070 Ti Laptop GPU
1648
+ - **CPU Model**: AMD Ryzen 9 6900HX with Radeon Graphics
1649
+ - **RAM Size**: 30.61 GB
1650
+
1651
+ ### Framework Versions
1652
+ - Python: 3.12.9
1653
+ - Sentence Transformers: 4.2.0.dev0
1654
+ - Transformers: 4.50.3
1655
+ - PyTorch: 2.6.0+cu124
1656
+ - Accelerate: 1.6.0
1657
+ - Datasets: 3.5.0
1658
+ - Tokenizers: 0.21.1
1659
+
1660
+ ## Citation
1661
+
1662
+ ### BibTeX
1663
+
1664
+ #### Sentence Transformers
1665
+ ```bibtex
1666
+ @inproceedings{reimers-2019-sentence-bert,
1667
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1668
+ author = "Reimers, Nils and Gurevych, Iryna",
1669
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1670
+ month = "11",
1671
+ year = "2019",
1672
+ publisher = "Association for Computational Linguistics",
1673
+ url = "https://arxiv.org/abs/1908.10084",
1674
+ }
1675
+ ```
1676
+
1677
+ #### SpladeLoss
1678
+ ```bibtex
1679
+ @misc{formal2022distillationhardnegativesampling,
1680
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1681
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1682
+ year={2022},
1683
+ eprint={2205.04733},
1684
+ archivePrefix={arXiv},
1685
+ primaryClass={cs.IR},
1686
+ url={https://arxiv.org/abs/2205.04733},
1687
+ }
1688
+ ```
1689
+
1690
+ #### SparseMultipleNegativesRankingLoss
1691
+ ```bibtex
1692
+ @misc{henderson2017efficient,
1693
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
1694
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
1695
+ year={2017},
1696
+ eprint={1705.00652},
1697
+ archivePrefix={arXiv},
1698
+ primaryClass={cs.CL}
1699
+ }
1700
+ ```
1701
+
1702
+ #### FlopsLoss
1703
+ ```bibtex
1704
+ @article{paria2020minimizing,
1705
+ title={Minimizing flops to learn efficient sparse representations},
1706
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1707
+ journal={arXiv preprint arXiv:2004.05665},
1708
+ year={2020}
1709
+ }
1710
+ ```
1711
+
1712
+ <!--
1713
+ ## Glossary
1714
+
1715
+ *Clearly define terms in order to be accessible across audiences.*
1716
+ -->
1717
+
1718
+ <!--
1719
+ ## Model Card Authors
1720
+
1721
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1722
+ -->
1723
+
1724
+ <!--
1725
+ ## Model Card Contact
1726
+
1727
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1728
+ -->
config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation": "gelu",
3
+ "architectures": [
4
+ "DistilBertForMaskedLM"
5
+ ],
6
+ "attention_dropout": 0.1,
7
+ "dim": 768,
8
+ "dropout": 0.1,
9
+ "hidden_dim": 3072,
10
+ "initializer_range": 0.02,
11
+ "max_position_embeddings": 512,
12
+ "model_type": "distilbert",
13
+ "n_heads": 12,
14
+ "n_layers": 6,
15
+ "pad_token_id": 0,
16
+ "qa_dropout": 0.1,
17
+ "seq_classif_dropout": 0.2,
18
+ "sinusoidal_pos_embds": false,
19
+ "tie_weights_": true,
20
+ "torch_dtype": "float32",
21
+ "transformers_version": "4.50.3",
22
+ "vocab_size": 30522
23
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.50.3",
6
+ "pytorch": "2.6.0+cu124"
7
+ },
8
+ "prompts": {},
9
+ "default_prompt_name": null,
10
+ "similarity_fn_name": "dot"
11
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de857b559488d5e8c5619291c5dd41f7feadffa0b6091cd263820a3c344c4f17
3
+ size 267954768
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff