Hugging Face's logo

aari1995
/
German_Semantic_STS_V2

Sentence Similarity
sentence-transformers
PyTorch
Safetensors
Transformers
German
bert
feature-extraction
gBERT-large
RAG
retrieval augmented generation
STS
MTEB
mteb
Eval Results
text-embeddings-inference
Model card Files Community
10
aari1995 commited on
Commit
763ede6
·
verified ·
1 Parent(s): 73e02f8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +258 -2
README.md CHANGED
@@ -14,8 +14,264 @@ tags:
14
  - retrieval augmented generation
15
  - STS
16
  - MTEB
17
-
18
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  ---
20
 
21
  # German_Semantic_STS_V2
 
14
  - retrieval augmented generation
15
  - STS
16
  - MTEB
17
+ - mteb
18
+
19
+ model-index:
20
+ - name: German_Semantic_STS_V2
21
+ results:
22
+ - dataset:
23
+ config: de
24
+ name: MTEB AmazonCounterfactualClassification
25
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
26
+ split: test
27
+ type: mteb/amazon_counterfactual
28
+ metrics:
29
+ - type: accuracy
30
+ value: 0.6700214132762312
31
+ task:
32
+ type: Classification
33
+ - dataset:
34
+ config: de
35
+ name: MTEB AmazonCounterfactualClassification
36
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
37
+ split: validation
38
+ type: mteb/amazon_counterfactual
39
+ metrics:
40
+ - type: accuracy
41
+ value: 0.6843347639484978
42
+ task:
43
+ type: Classification
44
+ - dataset:
45
+ config: de
46
+ name: MTEB AmazonReviewsClassification
47
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
48
+ split: test
49
+ type: mteb/amazon_reviews_multi
50
+ metrics:
51
+ - type: accuracy
52
+ value: 0.39092
53
+ task:
54
+ type: Classification
55
+ - dataset:
56
+ config: de
57
+ name: MTEB AmazonReviewsClassification
58
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
59
+ split: validation
60
+ type: mteb/amazon_reviews_multi
61
+ metrics:
62
+ - type: accuracy
63
+ value: 0.39146000000000003
64
+ task:
65
+ type: Classification
66
+ - dataset:
67
+ config: default
68
+ name: MTEB BlurbsClusteringP2P
69
+ revision: a2dd5b02a77de3466a3eaa98ae586b5610314496
70
+ split: test
71
+ type: slvnwhrl/blurbs-clustering-p2p
72
+ metrics:
73
+ - type: v_measure
74
+ value: 0.38680981669842135
75
+ task:
76
+ type: Clustering
77
+ - dataset:
78
+ config: default
79
+ name: MTEB BlurbsClusteringS2S
80
+ revision: 22793b6a6465bf00120ad525e38c51210858132c
81
+ split: test
82
+ type: slvnwhrl/blurbs-clustering-s2s
83
+ metrics:
84
+ - type: v_measure
85
+ value: 0.17624489937027504
86
+ task:
87
+ type: Clustering
88
+ - dataset:
89
+ config: default
90
+ name: MTEB GermanDPR
91
+ revision: 5129d02422a66be600ac89cd3e8531b4f97d347d
92
+ split: test
93
+ type: deepset/germandpr
94
+ metrics:
95
+ - type: ndcg_at_10
96
+ value: 0.72921
97
+ task:
98
+ type: Retrieval
99
+ - dataset:
100
+ config: default
101
+ name: MTEB GermanQuAD-Retrieval
102
+ revision: f5c87ae5a2e7a5106606314eef45255f03151bb3
103
+ split: test
104
+ type: mteb/germanquad-retrieval
105
+ metrics:
106
+ - type: mrr_at_5
107
+ value: 0.85316
108
+ task:
109
+ type: Retrieval
110
+ - dataset:
111
+ config: default
112
+ name: MTEB GermanSTSBenchmark
113
+ revision: e36907544d44c3a247898ed81540310442329e20
114
+ split: test
115
+ type: jinaai/german-STSbenchmark
116
+ metrics:
117
+ - type: cosine_spearman
118
+ value: 0.8467696933608695
119
+ task:
120
+ type: STS
121
+ - dataset:
122
+ config: default
123
+ name: MTEB GermanSTSBenchmark
124
+ revision: e36907544d44c3a247898ed81540310442329e20
125
+ split: validation
126
+ type: jinaai/german-STSbenchmark
127
+ metrics:
128
+ - type: cosine_spearman
129
+ value: 0.88048957974805
130
+ task:
131
+ type: STS
132
+ - dataset:
133
+ config: de
134
+ name: MTEB MassiveIntentClassification
135
+ revision: 4672e20407010da34463acc759c162ca9734bca6
136
+ split: test
137
+ type: mteb/amazon_massive_intent
138
+ metrics:
139
+ - type: accuracy
140
+ value: 0.6625084061869536
141
+ task:
142
+ type: Classification
143
+ - dataset:
144
+ config: de
145
+ name: MTEB MassiveIntentClassification
146
+ revision: 4672e20407010da34463acc759c162ca9734bca6
147
+ split: validation
148
+ type: mteb/amazon_massive_intent
149
+ metrics:
150
+ - type: accuracy
151
+ value: 0.6644859813084113
152
+ task:
153
+ type: Classification
154
+ - dataset:
155
+ config: de
156
+ name: MTEB MassiveScenarioClassification
157
+ revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
158
+ split: test
159
+ type: mteb/amazon_massive_scenario
160
+ metrics:
161
+ - type: accuracy
162
+ value: 0.7251176866173503
163
+ task:
164
+ type: Classification
165
+ - dataset:
166
+ config: de
167
+ name: MTEB MassiveScenarioClassification
168
+ revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
169
+ split: validation
170
+ type: mteb/amazon_massive_scenario
171
+ metrics:
172
+ - type: accuracy
173
+ value: 0.7202164289227742
174
+ task:
175
+ type: Classification
176
+ - dataset:
177
+ config: de
178
+ name: MTEB MTOPDomainClassification
179
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
180
+ split: test
181
+ type: mteb/mtop_domain
182
+ metrics:
183
+ - type: accuracy
184
+ value: 0.8900253592561285
185
+ task:
186
+ type: Classification
187
+ - dataset:
188
+ config: de
189
+ name: MTEB MTOPDomainClassification
190
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
191
+ split: validation
192
+ type: mteb/mtop_domain
193
+ metrics:
194
+ - type: accuracy
195
+ value: 0.8770798898071626
196
+ task:
197
+ type: Classification
198
+ - dataset:
199
+ config: de
200
+ name: MTEB MTOPIntentClassification
201
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
202
+ split: test
203
+ type: mteb/mtop_intent
204
+ metrics:
205
+ - type: accuracy
206
+ value: 0.7006198929275853
207
+ task:
208
+ type: Classification
209
+ - dataset:
210
+ config: de
211
+ name: MTEB MTOPIntentClassification
212
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
213
+ split: validation
214
+ type: mteb/mtop_intent
215
+ metrics:
216
+ - type: accuracy
217
+ value: 0.686060606060606
218
+ task:
219
+ type: Classification
220
+ - dataset:
221
+ config: de
222
+ name: MTEB PawsX
223
+ revision: 8a04d940a42cd40658986fdd8e3da561533a3646
224
+ split: test
225
+ type: google-research-datasets/paws-x
226
+ metrics:
227
+ - type: ap
228
+ value: 0.5747670853851811
229
+ task:
230
+ type: PairClassification
231
+ - dataset:
232
+ config: de
233
+ name: MTEB PawsX
234
+ revision: 8a04d940a42cd40658986fdd8e3da561533a3646
235
+ split: validation
236
+ type: google-research-datasets/paws-x
237
+ metrics:
238
+ - type: ap
239
+ value: 0.5285587710877178
240
+ task:
241
+ type: PairClassification
242
+ - dataset:
243
+ config: de
244
+ name: MTEB STS22
245
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
246
+ split: test
247
+ type: mteb/sts22-crosslingual-sts
248
+ metrics:
249
+ - type: cosine_spearman
250
+ value: 0.5063839763951755
251
+ task:
252
+ type: STS
253
+ - dataset:
254
+ config: default
255
+ name: MTEB TenKGnadClusteringP2P
256
+ revision: 5c59e41555244b7e45c9a6be2d720ab4bafae558
257
+ split: test
258
+ type: slvnwhrl/tenkgnad-clustering-p2p
259
+ metrics:
260
+ - type: v_measure
261
+ value: 0.37996685796529817
262
+ task:
263
+ type: Clustering
264
+ - dataset:
265
+ config: default
266
+ name: MTEB TenKGnadClusteringS2S
267
+ revision: 6cddbe003f12b9b140aec477b583ac4191f01786
268
+ split: test
269
+ type: slvnwhrl/tenkgnad-clustering-s2s
270
+ metrics:
271
+ - type: v_measure
272
+ value: 0.2371145428041516
273
+ task:
274
+ type: Clustering
275
  ---
276
 
277
  # German_Semantic_STS_V2