agier9 commited on
Commit
0a234d9
1 Parent(s): a72c790

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +3682 -0
README.md ADDED
@@ -0,0 +1,3682 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - mteb
4
+ - sentence-transformers
5
+ - transformers
6
+ - Qwen
7
+ - sentence-similarity
8
+ - llama-cpp
9
+ - gguf-my-repo
10
+ model-index:
11
+ - name: gte-qwen1.5-7b
12
+ results:
13
+ - task:
14
+ type: Classification
15
+ dataset:
16
+ name: MTEB AmazonCounterfactualClassification (en)
17
+ type: mteb/amazon_counterfactual
18
+ config: en
19
+ split: test
20
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
21
+ metrics:
22
+ - type: accuracy
23
+ value: 83.16417910447761
24
+ - type: ap
25
+ value: 49.37655308937739
26
+ - type: f1
27
+ value: 77.52987230462615
28
+ - task:
29
+ type: Classification
30
+ dataset:
31
+ name: MTEB AmazonPolarityClassification
32
+ type: mteb/amazon_polarity
33
+ config: default
34
+ split: test
35
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
36
+ metrics:
37
+ - type: accuracy
38
+ value: 96.6959
39
+ - type: ap
40
+ value: 94.90885739242472
41
+ - type: f1
42
+ value: 96.69477648952649
43
+ - task:
44
+ type: Classification
45
+ dataset:
46
+ name: MTEB AmazonReviewsClassification (en)
47
+ type: mteb/amazon_reviews_multi
48
+ config: en
49
+ split: test
50
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
51
+ metrics:
52
+ - type: accuracy
53
+ value: 62.168
54
+ - type: f1
55
+ value: 60.411431278343755
56
+ - task:
57
+ type: Retrieval
58
+ dataset:
59
+ name: MTEB ArguAna
60
+ type: mteb/arguana
61
+ config: default
62
+ split: test
63
+ revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
64
+ metrics:
65
+ - type: map_at_1
66
+ value: 36.415
67
+ - type: map_at_10
68
+ value: 53.505
69
+ - type: map_at_100
70
+ value: 54.013
71
+ - type: map_at_1000
72
+ value: 54.013
73
+ - type: map_at_3
74
+ value: 48.459
75
+ - type: map_at_5
76
+ value: 51.524
77
+ - type: mrr_at_1
78
+ value: 36.842000000000006
79
+ - type: mrr_at_10
80
+ value: 53.679
81
+ - type: mrr_at_100
82
+ value: 54.17999999999999
83
+ - type: mrr_at_1000
84
+ value: 54.17999999999999
85
+ - type: mrr_at_3
86
+ value: 48.613
87
+ - type: mrr_at_5
88
+ value: 51.696
89
+ - type: ndcg_at_1
90
+ value: 36.415
91
+ - type: ndcg_at_10
92
+ value: 62.644999999999996
93
+ - type: ndcg_at_100
94
+ value: 64.60000000000001
95
+ - type: ndcg_at_1000
96
+ value: 64.60000000000001
97
+ - type: ndcg_at_3
98
+ value: 52.44799999999999
99
+ - type: ndcg_at_5
100
+ value: 57.964000000000006
101
+ - type: precision_at_1
102
+ value: 36.415
103
+ - type: precision_at_10
104
+ value: 9.161
105
+ - type: precision_at_100
106
+ value: 0.996
107
+ - type: precision_at_1000
108
+ value: 0.1
109
+ - type: precision_at_3
110
+ value: 21.337
111
+ - type: precision_at_5
112
+ value: 15.476999999999999
113
+ - type: recall_at_1
114
+ value: 36.415
115
+ - type: recall_at_10
116
+ value: 91.607
117
+ - type: recall_at_100
118
+ value: 99.644
119
+ - type: recall_at_1000
120
+ value: 99.644
121
+ - type: recall_at_3
122
+ value: 64.011
123
+ - type: recall_at_5
124
+ value: 77.383
125
+ - task:
126
+ type: Clustering
127
+ dataset:
128
+ name: MTEB ArxivClusteringP2P
129
+ type: mteb/arxiv-clustering-p2p
130
+ config: default
131
+ split: test
132
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
133
+ metrics:
134
+ - type: v_measure
135
+ value: 56.40183100758549
136
+ - task:
137
+ type: Clustering
138
+ dataset:
139
+ name: MTEB ArxivClusteringS2S
140
+ type: mteb/arxiv-clustering-s2s
141
+ config: default
142
+ split: test
143
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
144
+ metrics:
145
+ - type: v_measure
146
+ value: 51.44814171373338
147
+ - task:
148
+ type: Reranking
149
+ dataset:
150
+ name: MTEB AskUbuntuDupQuestions
151
+ type: mteb/askubuntudupquestions-reranking
152
+ config: default
153
+ split: test
154
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
155
+ metrics:
156
+ - type: map
157
+ value: 66.00208703259058
158
+ - type: mrr
159
+ value: 78.95165545442553
160
+ - task:
161
+ type: STS
162
+ dataset:
163
+ name: MTEB BIOSSES
164
+ type: mteb/biosses-sts
165
+ config: default
166
+ split: test
167
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
168
+ metrics:
169
+ - type: cos_sim_pearson
170
+ value: 82.12591694410098
171
+ - type: cos_sim_spearman
172
+ value: 81.11570369802254
173
+ - type: euclidean_pearson
174
+ value: 80.91709076204458
175
+ - type: euclidean_spearman
176
+ value: 81.11570369802254
177
+ - type: manhattan_pearson
178
+ value: 80.71719561024605
179
+ - type: manhattan_spearman
180
+ value: 81.21510355327713
181
+ - task:
182
+ type: Classification
183
+ dataset:
184
+ name: MTEB Banking77Classification
185
+ type: mteb/banking77
186
+ config: default
187
+ split: test
188
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
189
+ metrics:
190
+ - type: accuracy
191
+ value: 81.67857142857142
192
+ - type: f1
193
+ value: 80.84103272994895
194
+ - task:
195
+ type: Clustering
196
+ dataset:
197
+ name: MTEB BiorxivClusteringP2P
198
+ type: mteb/biorxiv-clustering-p2p
199
+ config: default
200
+ split: test
201
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
202
+ metrics:
203
+ - type: v_measure
204
+ value: 49.008657468552016
205
+ - task:
206
+ type: Clustering
207
+ dataset:
208
+ name: MTEB BiorxivClusteringS2S
209
+ type: mteb/biorxiv-clustering-s2s
210
+ config: default
211
+ split: test
212
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
213
+ metrics:
214
+ - type: v_measure
215
+ value: 45.05901064421589
216
+ - task:
217
+ type: Retrieval
218
+ dataset:
219
+ name: MTEB CQADupstackAndroidRetrieval
220
+ type: BeIR/cqadupstack
221
+ config: default
222
+ split: test
223
+ revision: f46a197baaae43b4f621051089b82a364682dfeb
224
+ metrics:
225
+ - type: map_at_1
226
+ value: 32.694
227
+ - type: map_at_10
228
+ value: 43.895
229
+ - type: map_at_100
230
+ value: 45.797
231
+ - type: map_at_1000
232
+ value: 45.922000000000004
233
+ - type: map_at_3
234
+ value: 40.141
235
+ - type: map_at_5
236
+ value: 42.077
237
+ - type: mrr_at_1
238
+ value: 40.2
239
+ - type: mrr_at_10
240
+ value: 50.11
241
+ - type: mrr_at_100
242
+ value: 51.101
243
+ - type: mrr_at_1000
244
+ value: 51.13100000000001
245
+ - type: mrr_at_3
246
+ value: 47.735
247
+ - type: mrr_at_5
248
+ value: 48.922
249
+ - type: ndcg_at_1
250
+ value: 40.2
251
+ - type: ndcg_at_10
252
+ value: 50.449999999999996
253
+ - type: ndcg_at_100
254
+ value: 56.85
255
+ - type: ndcg_at_1000
256
+ value: 58.345
257
+ - type: ndcg_at_3
258
+ value: 45.261
259
+ - type: ndcg_at_5
260
+ value: 47.298
261
+ - type: precision_at_1
262
+ value: 40.2
263
+ - type: precision_at_10
264
+ value: 9.742
265
+ - type: precision_at_100
266
+ value: 1.6480000000000001
267
+ - type: precision_at_1000
268
+ value: 0.214
269
+ - type: precision_at_3
270
+ value: 21.841
271
+ - type: precision_at_5
272
+ value: 15.68
273
+ - type: recall_at_1
274
+ value: 32.694
275
+ - type: recall_at_10
276
+ value: 62.751999999999995
277
+ - type: recall_at_100
278
+ value: 88.619
279
+ - type: recall_at_1000
280
+ value: 97.386
281
+ - type: recall_at_3
282
+ value: 47.087
283
+ - type: recall_at_5
284
+ value: 53.108999999999995
285
+ - task:
286
+ type: Retrieval
287
+ dataset:
288
+ name: MTEB CQADupstackEnglishRetrieval
289
+ type: BeIR/cqadupstack
290
+ config: default
291
+ split: test
292
+ revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
293
+ metrics:
294
+ - type: map_at_1
295
+ value: 27.849
296
+ - type: map_at_10
297
+ value: 37.938
298
+ - type: map_at_100
299
+ value: 39.211
300
+ - type: map_at_1000
301
+ value: 39.333
302
+ - type: map_at_3
303
+ value: 35.314
304
+ - type: map_at_5
305
+ value: 36.666
306
+ - type: mrr_at_1
307
+ value: 34.904
308
+ - type: mrr_at_10
309
+ value: 43.869
310
+ - type: mrr_at_100
311
+ value: 44.614
312
+ - type: mrr_at_1000
313
+ value: 44.662
314
+ - type: mrr_at_3
315
+ value: 41.815000000000005
316
+ - type: mrr_at_5
317
+ value: 42.943
318
+ - type: ndcg_at_1
319
+ value: 34.904
320
+ - type: ndcg_at_10
321
+ value: 43.605
322
+ - type: ndcg_at_100
323
+ value: 48.339999999999996
324
+ - type: ndcg_at_1000
325
+ value: 50.470000000000006
326
+ - type: ndcg_at_3
327
+ value: 39.835
328
+ - type: ndcg_at_5
329
+ value: 41.364000000000004
330
+ - type: precision_at_1
331
+ value: 34.904
332
+ - type: precision_at_10
333
+ value: 8.222999999999999
334
+ - type: precision_at_100
335
+ value: 1.332
336
+ - type: precision_at_1000
337
+ value: 0.183
338
+ - type: precision_at_3
339
+ value: 19.575
340
+ - type: precision_at_5
341
+ value: 13.58
342
+ - type: recall_at_1
343
+ value: 27.849
344
+ - type: recall_at_10
345
+ value: 53.635
346
+ - type: recall_at_100
347
+ value: 73.932
348
+ - type: recall_at_1000
349
+ value: 87.29599999999999
350
+ - type: recall_at_3
351
+ value: 42.019
352
+ - type: recall_at_5
353
+ value: 46.58
354
+ - task:
355
+ type: Retrieval
356
+ dataset:
357
+ name: MTEB CQADupstackGamingRetrieval
358
+ type: BeIR/cqadupstack
359
+ config: default
360
+ split: test
361
+ revision: 4885aa143210c98657558c04aaf3dc47cfb54340
362
+ metrics:
363
+ - type: map_at_1
364
+ value: 29.182999999999996
365
+ - type: map_at_10
366
+ value: 41.233
367
+ - type: map_at_100
368
+ value: 42.52
369
+ - type: map_at_1000
370
+ value: 42.589
371
+ - type: map_at_3
372
+ value: 37.284
373
+ - type: map_at_5
374
+ value: 39.586
375
+ - type: mrr_at_1
376
+ value: 33.793
377
+ - type: mrr_at_10
378
+ value: 44.572
379
+ - type: mrr_at_100
380
+ value: 45.456
381
+ - type: mrr_at_1000
382
+ value: 45.497
383
+ - type: mrr_at_3
384
+ value: 41.275
385
+ - type: mrr_at_5
386
+ value: 43.278
387
+ - type: ndcg_at_1
388
+ value: 33.793
389
+ - type: ndcg_at_10
390
+ value: 47.823
391
+ - type: ndcg_at_100
392
+ value: 52.994
393
+ - type: ndcg_at_1000
394
+ value: 54.400000000000006
395
+ - type: ndcg_at_3
396
+ value: 40.82
397
+ - type: ndcg_at_5
398
+ value: 44.426
399
+ - type: precision_at_1
400
+ value: 33.793
401
+ - type: precision_at_10
402
+ value: 8.312999999999999
403
+ - type: precision_at_100
404
+ value: 1.191
405
+ - type: precision_at_1000
406
+ value: 0.136
407
+ - type: precision_at_3
408
+ value: 18.662
409
+ - type: precision_at_5
410
+ value: 13.668
411
+ - type: recall_at_1
412
+ value: 29.182999999999996
413
+ - type: recall_at_10
414
+ value: 64.14999999999999
415
+ - type: recall_at_100
416
+ value: 86.533
417
+ - type: recall_at_1000
418
+ value: 96.492
419
+ - type: recall_at_3
420
+ value: 45.7
421
+ - type: recall_at_5
422
+ value: 54.330999999999996
423
+ - task:
424
+ type: Retrieval
425
+ dataset:
426
+ name: MTEB CQADupstackGisRetrieval
427
+ type: BeIR/cqadupstack
428
+ config: default
429
+ split: test
430
+ revision: 5003b3064772da1887988e05400cf3806fe491f2
431
+ metrics:
432
+ - type: map_at_1
433
+ value: 24.389
434
+ - type: map_at_10
435
+ value: 33.858
436
+ - type: map_at_100
437
+ value: 35.081
438
+ - type: map_at_1000
439
+ value: 35.161
440
+ - type: map_at_3
441
+ value: 30.793
442
+ - type: map_at_5
443
+ value: 32.336
444
+ - type: mrr_at_1
445
+ value: 27.006000000000004
446
+ - type: mrr_at_10
447
+ value: 36.378
448
+ - type: mrr_at_100
449
+ value: 37.345
450
+ - type: mrr_at_1000
451
+ value: 37.405
452
+ - type: mrr_at_3
453
+ value: 33.578
454
+ - type: mrr_at_5
455
+ value: 34.991
456
+ - type: ndcg_at_1
457
+ value: 27.006000000000004
458
+ - type: ndcg_at_10
459
+ value: 39.612
460
+ - type: ndcg_at_100
461
+ value: 45.216
462
+ - type: ndcg_at_1000
463
+ value: 47.12
464
+ - type: ndcg_at_3
465
+ value: 33.566
466
+ - type: ndcg_at_5
467
+ value: 36.105
468
+ - type: precision_at_1
469
+ value: 27.006000000000004
470
+ - type: precision_at_10
471
+ value: 6.372999999999999
472
+ - type: precision_at_100
473
+ value: 0.968
474
+ - type: precision_at_1000
475
+ value: 0.11800000000000001
476
+ - type: precision_at_3
477
+ value: 14.501
478
+ - type: precision_at_5
479
+ value: 10.169
480
+ - type: recall_at_1
481
+ value: 24.389
482
+ - type: recall_at_10
483
+ value: 55.131
484
+ - type: recall_at_100
485
+ value: 80.315
486
+ - type: recall_at_1000
487
+ value: 94.284
488
+ - type: recall_at_3
489
+ value: 38.643
490
+ - type: recall_at_5
491
+ value: 44.725
492
+ - task:
493
+ type: Retrieval
494
+ dataset:
495
+ name: MTEB CQADupstackMathematicaRetrieval
496
+ type: BeIR/cqadupstack
497
+ config: default
498
+ split: test
499
+ revision: 90fceea13679c63fe563ded68f3b6f06e50061de
500
+ metrics:
501
+ - type: map_at_1
502
+ value: 15.845999999999998
503
+ - type: map_at_10
504
+ value: 25.019000000000002
505
+ - type: map_at_100
506
+ value: 26.478
507
+ - type: map_at_1000
508
+ value: 26.598
509
+ - type: map_at_3
510
+ value: 21.595
511
+ - type: map_at_5
512
+ value: 23.335
513
+ - type: mrr_at_1
514
+ value: 20.274
515
+ - type: mrr_at_10
516
+ value: 29.221000000000004
517
+ - type: mrr_at_100
518
+ value: 30.354999999999997
519
+ - type: mrr_at_1000
520
+ value: 30.419
521
+ - type: mrr_at_3
522
+ value: 26.161
523
+ - type: mrr_at_5
524
+ value: 27.61
525
+ - type: ndcg_at_1
526
+ value: 20.274
527
+ - type: ndcg_at_10
528
+ value: 31.014000000000003
529
+ - type: ndcg_at_100
530
+ value: 37.699
531
+ - type: ndcg_at_1000
532
+ value: 40.363
533
+ - type: ndcg_at_3
534
+ value: 24.701999999999998
535
+ - type: ndcg_at_5
536
+ value: 27.261999999999997
537
+ - type: precision_at_1
538
+ value: 20.274
539
+ - type: precision_at_10
540
+ value: 6.219
541
+ - type: precision_at_100
542
+ value: 1.101
543
+ - type: precision_at_1000
544
+ value: 0.146
545
+ - type: precision_at_3
546
+ value: 12.231
547
+ - type: precision_at_5
548
+ value: 9.129
549
+ - type: recall_at_1
550
+ value: 15.845999999999998
551
+ - type: recall_at_10
552
+ value: 45.358
553
+ - type: recall_at_100
554
+ value: 74.232
555
+ - type: recall_at_1000
556
+ value: 92.985
557
+ - type: recall_at_3
558
+ value: 28.050000000000004
559
+ - type: recall_at_5
560
+ value: 34.588
561
+ - task:
562
+ type: Retrieval
563
+ dataset:
564
+ name: MTEB CQADupstackPhysicsRetrieval
565
+ type: BeIR/cqadupstack
566
+ config: default
567
+ split: test
568
+ revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
569
+ metrics:
570
+ - type: map_at_1
571
+ value: 33.808
572
+ - type: map_at_10
573
+ value: 46.86
574
+ - type: map_at_100
575
+ value: 48.237
576
+ - type: map_at_1000
577
+ value: 48.331
578
+ - type: map_at_3
579
+ value: 42.784
580
+ - type: map_at_5
581
+ value: 45.015
582
+ - type: mrr_at_1
583
+ value: 41.771
584
+ - type: mrr_at_10
585
+ value: 52.35300000000001
586
+ - type: mrr_at_100
587
+ value: 53.102000000000004
588
+ - type: mrr_at_1000
589
+ value: 53.132999999999996
590
+ - type: mrr_at_3
591
+ value: 49.663000000000004
592
+ - type: mrr_at_5
593
+ value: 51.27
594
+ - type: ndcg_at_1
595
+ value: 41.771
596
+ - type: ndcg_at_10
597
+ value: 53.562
598
+ - type: ndcg_at_100
599
+ value: 58.809999999999995
600
+ - type: ndcg_at_1000
601
+ value: 60.23
602
+ - type: ndcg_at_3
603
+ value: 47.514
604
+ - type: ndcg_at_5
605
+ value: 50.358999999999995
606
+ - type: precision_at_1
607
+ value: 41.771
608
+ - type: precision_at_10
609
+ value: 10.038
610
+ - type: precision_at_100
611
+ value: 1.473
612
+ - type: precision_at_1000
613
+ value: 0.17600000000000002
614
+ - type: precision_at_3
615
+ value: 22.875
616
+ - type: precision_at_5
617
+ value: 16.477
618
+ - type: recall_at_1
619
+ value: 33.808
620
+ - type: recall_at_10
621
+ value: 67.721
622
+ - type: recall_at_100
623
+ value: 89.261
624
+ - type: recall_at_1000
625
+ value: 98.042
626
+ - type: recall_at_3
627
+ value: 50.807
628
+ - type: recall_at_5
629
+ value: 58.162000000000006
630
+ - task:
631
+ type: Retrieval
632
+ dataset:
633
+ name: MTEB CQADupstackProgrammersRetrieval
634
+ type: BeIR/cqadupstack
635
+ config: default
636
+ split: test
637
+ revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
638
+ metrics:
639
+ - type: map_at_1
640
+ value: 28.105000000000004
641
+ - type: map_at_10
642
+ value: 40.354
643
+ - type: map_at_100
644
+ value: 41.921
645
+ - type: map_at_1000
646
+ value: 42.021
647
+ - type: map_at_3
648
+ value: 36.532
649
+ - type: map_at_5
650
+ value: 38.671
651
+ - type: mrr_at_1
652
+ value: 34.475
653
+ - type: mrr_at_10
654
+ value: 45.342
655
+ - type: mrr_at_100
656
+ value: 46.300000000000004
657
+ - type: mrr_at_1000
658
+ value: 46.343
659
+ - type: mrr_at_3
660
+ value: 42.637
661
+ - type: mrr_at_5
662
+ value: 44.207
663
+ - type: ndcg_at_1
664
+ value: 34.475
665
+ - type: ndcg_at_10
666
+ value: 46.945
667
+ - type: ndcg_at_100
668
+ value: 52.939
669
+ - type: ndcg_at_1000
670
+ value: 54.645999999999994
671
+ - type: ndcg_at_3
672
+ value: 41.065000000000005
673
+ - type: ndcg_at_5
674
+ value: 43.832
675
+ - type: precision_at_1
676
+ value: 34.475
677
+ - type: precision_at_10
678
+ value: 8.892999999999999
679
+ - type: precision_at_100
680
+ value: 1.377
681
+ - type: precision_at_1000
682
+ value: 0.17099999999999999
683
+ - type: precision_at_3
684
+ value: 20.091
685
+ - type: precision_at_5
686
+ value: 14.452000000000002
687
+ - type: recall_at_1
688
+ value: 28.105000000000004
689
+ - type: recall_at_10
690
+ value: 61.253
691
+ - type: recall_at_100
692
+ value: 85.92
693
+ - type: recall_at_1000
694
+ value: 96.799
695
+ - type: recall_at_3
696
+ value: 45.094
697
+ - type: recall_at_5
698
+ value: 52.455
699
+ - task:
700
+ type: Retrieval
701
+ dataset:
702
+ name: MTEB CQADupstackRetrieval
703
+ type: BeIR/cqadupstack
704
+ config: default
705
+ split: test
706
+ revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
707
+ metrics:
708
+ - type: map_at_1
709
+ value: 24.613833333333332
710
+ - type: map_at_10
711
+ value: 34.763
712
+ - type: map_at_100
713
+ value: 36.17066666666667
714
+ - type: map_at_1000
715
+ value: 36.2905
716
+ - type: map_at_3
717
+ value: 31.53541666666666
718
+ - type: map_at_5
719
+ value: 33.29216666666667
720
+ - type: mrr_at_1
721
+ value: 29.48725
722
+ - type: mrr_at_10
723
+ value: 38.92066666666667
724
+ - type: mrr_at_100
725
+ value: 39.88725000000001
726
+ - type: mrr_at_1000
727
+ value: 39.9435
728
+ - type: mrr_at_3
729
+ value: 36.284083333333335
730
+ - type: mrr_at_5
731
+ value: 37.73941666666667
732
+ - type: ndcg_at_1
733
+ value: 29.48725
734
+ - type: ndcg_at_10
735
+ value: 40.635083333333334
736
+ - type: ndcg_at_100
737
+ value: 46.479416666666665
738
+ - type: ndcg_at_1000
739
+ value: 48.63308333333334
740
+ - type: ndcg_at_3
741
+ value: 35.19483333333333
742
+ - type: ndcg_at_5
743
+ value: 37.68016666666667
744
+ - type: precision_at_1
745
+ value: 29.48725
746
+ - type: precision_at_10
747
+ value: 7.406499999999998
748
+ - type: precision_at_100
749
+ value: 1.2225833333333334
750
+ - type: precision_at_1000
751
+ value: 0.16108333333333336
752
+ - type: precision_at_3
753
+ value: 16.53375
754
+ - type: precision_at_5
755
+ value: 11.919416666666665
756
+ - type: recall_at_1
757
+ value: 24.613833333333332
758
+ - type: recall_at_10
759
+ value: 53.91766666666666
760
+ - type: recall_at_100
761
+ value: 79.18
762
+ - type: recall_at_1000
763
+ value: 93.85133333333333
764
+ - type: recall_at_3
765
+ value: 38.866166666666665
766
+ - type: recall_at_5
767
+ value: 45.21275000000001
768
+ - type: map_at_1
769
+ value: 12.328999999999999
770
+ - type: map_at_10
771
+ value: 20.078
772
+ - type: map_at_100
773
+ value: 21.166999999999998
774
+ - type: map_at_1000
775
+ value: 21.308
776
+ - type: map_at_3
777
+ value: 17.702
778
+ - type: map_at_5
779
+ value: 18.725
780
+ - type: mrr_at_1
781
+ value: 13.678
782
+ - type: mrr_at_10
783
+ value: 21.859
784
+ - type: mrr_at_100
785
+ value: 22.816
786
+ - type: mrr_at_1000
787
+ value: 22.926
788
+ - type: mrr_at_3
789
+ value: 19.378
790
+ - type: mrr_at_5
791
+ value: 20.385
792
+ - type: ndcg_at_1
793
+ value: 13.678
794
+ - type: ndcg_at_10
795
+ value: 24.993000000000002
796
+ - type: ndcg_at_100
797
+ value: 30.464999999999996
798
+ - type: ndcg_at_1000
799
+ value: 33.916000000000004
800
+ - type: ndcg_at_3
801
+ value: 19.966
802
+ - type: ndcg_at_5
803
+ value: 21.712999999999997
804
+ - type: precision_at_1
805
+ value: 13.678
806
+ - type: precision_at_10
807
+ value: 4.473
808
+ - type: precision_at_100
809
+ value: 0.784
810
+ - type: precision_at_1000
811
+ value: 0.116
812
+ - type: precision_at_3
813
+ value: 9.181000000000001
814
+ - type: precision_at_5
815
+ value: 6.506
816
+ - type: recall_at_1
817
+ value: 12.328999999999999
818
+ - type: recall_at_10
819
+ value: 38.592
820
+ - type: recall_at_100
821
+ value: 63.817
822
+ - type: recall_at_1000
823
+ value: 89.67500000000001
824
+ - type: recall_at_3
825
+ value: 24.726
826
+ - type: recall_at_5
827
+ value: 28.959000000000003
828
+ - task:
829
+ type: Retrieval
830
+ dataset:
831
+ name: MTEB CQADupstackStatsRetrieval
832
+ type: BeIR/cqadupstack
833
+ config: default
834
+ split: test
835
+ revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
836
+ metrics:
837
+ - type: map_at_1
838
+ value: 25.106
839
+ - type: map_at_10
840
+ value: 33.367999999999995
841
+ - type: map_at_100
842
+ value: 34.586
843
+ - type: map_at_1000
844
+ value: 34.681
845
+ - type: map_at_3
846
+ value: 31.022
847
+ - type: map_at_5
848
+ value: 32.548
849
+ - type: mrr_at_1
850
+ value: 28.374
851
+ - type: mrr_at_10
852
+ value: 36.521
853
+ - type: mrr_at_100
854
+ value: 37.55
855
+ - type: mrr_at_1000
856
+ value: 37.614999999999995
857
+ - type: mrr_at_3
858
+ value: 34.509
859
+ - type: mrr_at_5
860
+ value: 35.836
861
+ - type: ndcg_at_1
862
+ value: 28.374
863
+ - type: ndcg_at_10
864
+ value: 37.893
865
+ - type: ndcg_at_100
866
+ value: 43.694
867
+ - type: ndcg_at_1000
868
+ value: 46.001999999999995
869
+ - type: ndcg_at_3
870
+ value: 33.825
871
+ - type: ndcg_at_5
872
+ value: 36.201
873
+ - type: precision_at_1
874
+ value: 28.374
875
+ - type: precision_at_10
876
+ value: 5.966
877
+ - type: precision_at_100
878
+ value: 0.9650000000000001
879
+ - type: precision_at_1000
880
+ value: 0.124
881
+ - type: precision_at_3
882
+ value: 14.774999999999999
883
+ - type: precision_at_5
884
+ value: 10.459999999999999
885
+ - type: recall_at_1
886
+ value: 25.106
887
+ - type: recall_at_10
888
+ value: 48.607
889
+ - type: recall_at_100
890
+ value: 74.66000000000001
891
+ - type: recall_at_1000
892
+ value: 91.562
893
+ - type: recall_at_3
894
+ value: 37.669999999999995
895
+ - type: recall_at_5
896
+ value: 43.484
897
+ - task:
898
+ type: Retrieval
899
+ dataset:
900
+ name: MTEB CQADupstackTexRetrieval
901
+ type: BeIR/cqadupstack
902
+ config: default
903
+ split: test
904
+ revision: 46989137a86843e03a6195de44b09deda022eec7
905
+ metrics:
906
+ - type: map_at_1
907
+ value: 13.755
908
+ - type: map_at_10
909
+ value: 20.756
910
+ - type: map_at_100
911
+ value: 22.05
912
+ - type: map_at_1000
913
+ value: 22.201
914
+ - type: map_at_3
915
+ value: 18.243000000000002
916
+ - type: map_at_5
917
+ value: 19.512
918
+ - type: mrr_at_1
919
+ value: 16.93
920
+ - type: mrr_at_10
921
+ value: 24.276
922
+ - type: mrr_at_100
923
+ value: 25.349
924
+ - type: mrr_at_1000
925
+ value: 25.441000000000003
926
+ - type: mrr_at_3
927
+ value: 21.897
928
+ - type: mrr_at_5
929
+ value: 23.134
930
+ - type: ndcg_at_1
931
+ value: 16.93
932
+ - type: ndcg_at_10
933
+ value: 25.508999999999997
934
+ - type: ndcg_at_100
935
+ value: 31.777
936
+ - type: ndcg_at_1000
937
+ value: 35.112
938
+ - type: ndcg_at_3
939
+ value: 20.896
940
+ - type: ndcg_at_5
941
+ value: 22.857
942
+ - type: precision_at_1
943
+ value: 16.93
944
+ - type: precision_at_10
945
+ value: 4.972
946
+ - type: precision_at_100
947
+ value: 0.963
948
+ - type: precision_at_1000
949
+ value: 0.145
950
+ - type: precision_at_3
951
+ value: 10.14
952
+ - type: precision_at_5
953
+ value: 7.536
954
+ - type: recall_at_1
955
+ value: 13.755
956
+ - type: recall_at_10
957
+ value: 36.46
958
+ - type: recall_at_100
959
+ value: 64.786
960
+ - type: recall_at_1000
961
+ value: 88.287
962
+ - type: recall_at_3
963
+ value: 23.681
964
+ - type: recall_at_5
965
+ value: 28.615000000000002
966
+ - task:
967
+ type: Retrieval
968
+ dataset:
969
+ name: MTEB CQADupstackUnixRetrieval
970
+ type: BeIR/cqadupstack
971
+ config: default
972
+ split: test
973
+ revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
974
+ metrics:
975
+ - type: map_at_1
976
+ value: 26.99
977
+ - type: map_at_10
978
+ value: 38.009
979
+ - type: map_at_100
980
+ value: 39.384
981
+ - type: map_at_1000
982
+ value: 39.481
983
+ - type: map_at_3
984
+ value: 34.593
985
+ - type: map_at_5
986
+ value: 36.449999999999996
987
+ - type: mrr_at_1
988
+ value: 31.81
989
+ - type: mrr_at_10
990
+ value: 41.943000000000005
991
+ - type: mrr_at_100
992
+ value: 42.914
993
+ - type: mrr_at_1000
994
+ value: 42.962
995
+ - type: mrr_at_3
996
+ value: 39.179
997
+ - type: mrr_at_5
998
+ value: 40.798
999
+ - type: ndcg_at_1
1000
+ value: 31.81
1001
+ - type: ndcg_at_10
1002
+ value: 44.086
1003
+ - type: ndcg_at_100
1004
+ value: 50.026
1005
+ - type: ndcg_at_1000
1006
+ value: 51.903999999999996
1007
+ - type: ndcg_at_3
1008
+ value: 38.23
1009
+ - type: ndcg_at_5
1010
+ value: 40.926
1011
+ - type: precision_at_1
1012
+ value: 31.81
1013
+ - type: precision_at_10
1014
+ value: 7.761
1015
+ - type: precision_at_100
1016
+ value: 1.205
1017
+ - type: precision_at_1000
1018
+ value: 0.148
1019
+ - type: precision_at_3
1020
+ value: 17.537
1021
+ - type: precision_at_5
1022
+ value: 12.649
1023
+ - type: recall_at_1
1024
+ value: 26.99
1025
+ - type: recall_at_10
1026
+ value: 58.467
1027
+ - type: recall_at_100
1028
+ value: 83.93
1029
+ - type: recall_at_1000
1030
+ value: 96.452
1031
+ - type: recall_at_3
1032
+ value: 42.685
1033
+ - type: recall_at_5
1034
+ value: 49.341
1035
+ - task:
1036
+ type: Retrieval
1037
+ dataset:
1038
+ name: MTEB CQADupstackWebmastersRetrieval
1039
+ type: BeIR/cqadupstack
1040
+ config: default
1041
+ split: test
1042
+ revision: 160c094312a0e1facb97e55eeddb698c0abe3571
1043
+ metrics:
1044
+ - type: map_at_1
1045
+ value: 25.312
1046
+ - type: map_at_10
1047
+ value: 35.788
1048
+ - type: map_at_100
1049
+ value: 37.616
1050
+ - type: map_at_1000
1051
+ value: 37.86
1052
+ - type: map_at_3
1053
+ value: 32.422000000000004
1054
+ - type: map_at_5
1055
+ value: 34.585
1056
+ - type: mrr_at_1
1057
+ value: 30.631999999999998
1058
+ - type: mrr_at_10
1059
+ value: 40.604
1060
+ - type: mrr_at_100
1061
+ value: 41.745
1062
+ - type: mrr_at_1000
1063
+ value: 41.788
1064
+ - type: mrr_at_3
1065
+ value: 37.582
1066
+ - type: mrr_at_5
1067
+ value: 39.499
1068
+ - type: ndcg_at_1
1069
+ value: 30.631999999999998
1070
+ - type: ndcg_at_10
1071
+ value: 42.129
1072
+ - type: ndcg_at_100
1073
+ value: 48.943
1074
+ - type: ndcg_at_1000
1075
+ value: 51.089
1076
+ - type: ndcg_at_3
1077
+ value: 36.658
1078
+ - type: ndcg_at_5
1079
+ value: 39.818999999999996
1080
+ - type: precision_at_1
1081
+ value: 30.631999999999998
1082
+ - type: precision_at_10
1083
+ value: 7.904999999999999
1084
+ - type: precision_at_100
1085
+ value: 1.664
1086
+ - type: precision_at_1000
1087
+ value: 0.256
1088
+ - type: precision_at_3
1089
+ value: 16.996
1090
+ - type: precision_at_5
1091
+ value: 12.727
1092
+ - type: recall_at_1
1093
+ value: 25.312
1094
+ - type: recall_at_10
1095
+ value: 54.886
1096
+ - type: recall_at_100
1097
+ value: 84.155
1098
+ - type: recall_at_1000
1099
+ value: 96.956
1100
+ - type: recall_at_3
1101
+ value: 40.232
1102
+ - type: recall_at_5
1103
+ value: 48.204
1104
+ - task:
1105
+ type: Retrieval
1106
+ dataset:
1107
+ name: MTEB ClimateFEVER
1108
+ type: mteb/climate-fever
1109
+ config: default
1110
+ split: test
1111
+ revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
1112
+ metrics:
1113
+ - type: map_at_1
1114
+ value: 19.147
1115
+ - type: map_at_10
1116
+ value: 33.509
1117
+ - type: map_at_100
1118
+ value: 35.573
1119
+ - type: map_at_1000
1120
+ value: 35.769
1121
+ - type: map_at_3
1122
+ value: 27.983999999999998
1123
+ - type: map_at_5
1124
+ value: 31.012
1125
+ - type: mrr_at_1
1126
+ value: 43.844
1127
+ - type: mrr_at_10
1128
+ value: 56.24
1129
+ - type: mrr_at_100
1130
+ value: 56.801
1131
+ - type: mrr_at_1000
1132
+ value: 56.826
1133
+ - type: mrr_at_3
1134
+ value: 53.290000000000006
1135
+ - type: mrr_at_5
1136
+ value: 55.13
1137
+ - type: ndcg_at_1
1138
+ value: 43.844
1139
+ - type: ndcg_at_10
1140
+ value: 43.996
1141
+ - type: ndcg_at_100
1142
+ value: 50.965
1143
+ - type: ndcg_at_1000
1144
+ value: 53.927
1145
+ - type: ndcg_at_3
1146
+ value: 37.263000000000005
1147
+ - type: ndcg_at_5
1148
+ value: 39.553
1149
+ - type: precision_at_1
1150
+ value: 43.844
1151
+ - type: precision_at_10
1152
+ value: 13.687
1153
+ - type: precision_at_100
1154
+ value: 2.139
1155
+ - type: precision_at_1000
1156
+ value: 0.269
1157
+ - type: precision_at_3
1158
+ value: 28.122000000000003
1159
+ - type: precision_at_5
1160
+ value: 21.303
1161
+ - type: recall_at_1
1162
+ value: 19.147
1163
+ - type: recall_at_10
1164
+ value: 50.449999999999996
1165
+ - type: recall_at_100
1166
+ value: 74.00099999999999
1167
+ - type: recall_at_1000
1168
+ value: 90.098
1169
+ - type: recall_at_3
1170
+ value: 33.343
1171
+ - type: recall_at_5
1172
+ value: 40.744
1173
+ - task:
1174
+ type: Retrieval
1175
+ dataset:
1176
+ name: MTEB DBPedia
1177
+ type: mteb/dbpedia
1178
+ config: default
1179
+ split: test
1180
+ revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
1181
+ metrics:
1182
+ - type: map_at_1
1183
+ value: 8.773
1184
+ - type: map_at_10
1185
+ value: 21.172
1186
+ - type: map_at_100
1187
+ value: 30.244
1188
+ - type: map_at_1000
1189
+ value: 32.127
1190
+ - type: map_at_3
1191
+ value: 14.510000000000002
1192
+ - type: map_at_5
1193
+ value: 17.483
1194
+ - type: mrr_at_1
1195
+ value: 68.25
1196
+ - type: mrr_at_10
1197
+ value: 77.33
1198
+ - type: mrr_at_100
1199
+ value: 77.529
1200
+ - type: mrr_at_1000
1201
+ value: 77.536
1202
+ - type: mrr_at_3
1203
+ value: 75.708
1204
+ - type: mrr_at_5
1205
+ value: 76.72099999999999
1206
+ - type: ndcg_at_1
1207
+ value: 60.0
1208
+ - type: ndcg_at_10
1209
+ value: 48.045
1210
+ - type: ndcg_at_100
1211
+ value: 51.620999999999995
1212
+ - type: ndcg_at_1000
1213
+ value: 58.843999999999994
1214
+ - type: ndcg_at_3
1215
+ value: 52.922000000000004
1216
+ - type: ndcg_at_5
1217
+ value: 50.27
1218
+ - type: precision_at_1
1219
+ value: 68.25
1220
+ - type: precision_at_10
1221
+ value: 37.625
1222
+ - type: precision_at_100
1223
+ value: 11.774999999999999
1224
+ - type: precision_at_1000
1225
+ value: 2.395
1226
+ - type: precision_at_3
1227
+ value: 55.25
1228
+ - type: precision_at_5
1229
+ value: 47.599999999999994
1230
+ - type: recall_at_1
1231
+ value: 8.773
1232
+ - type: recall_at_10
1233
+ value: 27.332
1234
+ - type: recall_at_100
1235
+ value: 55.48499999999999
1236
+ - type: recall_at_1000
1237
+ value: 79.886
1238
+ - type: recall_at_3
1239
+ value: 15.823
1240
+ - type: recall_at_5
1241
+ value: 20.523
1242
+ - task:
1243
+ type: Classification
1244
+ dataset:
1245
+ name: MTEB EmotionClassification
1246
+ type: mteb/emotion
1247
+ config: default
1248
+ split: test
1249
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1250
+ metrics:
1251
+ - type: accuracy
1252
+ value: 54.52999999999999
1253
+ - type: f1
1254
+ value: 47.396628088963645
1255
+ - task:
1256
+ type: Retrieval
1257
+ dataset:
1258
+ name: MTEB FEVER
1259
+ type: mteb/fever
1260
+ config: default
1261
+ split: test
1262
+ revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
1263
+ metrics:
1264
+ - type: map_at_1
1265
+ value: 85.397
1266
+ - type: map_at_10
1267
+ value: 90.917
1268
+ - type: map_at_100
1269
+ value: 91.109
1270
+ - type: map_at_1000
1271
+ value: 91.121
1272
+ - type: map_at_3
1273
+ value: 90.045
1274
+ - type: map_at_5
1275
+ value: 90.602
1276
+ - type: mrr_at_1
1277
+ value: 92.00399999999999
1278
+ - type: mrr_at_10
1279
+ value: 95.39999999999999
1280
+ - type: mrr_at_100
1281
+ value: 95.41
1282
+ - type: mrr_at_1000
1283
+ value: 95.41
1284
+ - type: mrr_at_3
1285
+ value: 95.165
1286
+ - type: mrr_at_5
1287
+ value: 95.348
1288
+ - type: ndcg_at_1
1289
+ value: 92.00399999999999
1290
+ - type: ndcg_at_10
1291
+ value: 93.345
1292
+ - type: ndcg_at_100
1293
+ value: 93.934
1294
+ - type: ndcg_at_1000
1295
+ value: 94.108
1296
+ - type: ndcg_at_3
1297
+ value: 92.32000000000001
1298
+ - type: ndcg_at_5
1299
+ value: 92.899
1300
+ - type: precision_at_1
1301
+ value: 92.00399999999999
1302
+ - type: precision_at_10
1303
+ value: 10.839
1304
+ - type: precision_at_100
1305
+ value: 1.1440000000000001
1306
+ - type: precision_at_1000
1307
+ value: 0.117
1308
+ - type: precision_at_3
1309
+ value: 34.298
1310
+ - type: precision_at_5
1311
+ value: 21.128
1312
+ - type: recall_at_1
1313
+ value: 85.397
1314
+ - type: recall_at_10
1315
+ value: 96.375
1316
+ - type: recall_at_100
1317
+ value: 98.518
1318
+ - type: recall_at_1000
1319
+ value: 99.515
1320
+ - type: recall_at_3
1321
+ value: 93.59100000000001
1322
+ - type: recall_at_5
1323
+ value: 95.134
1324
+ - task:
1325
+ type: Retrieval
1326
+ dataset:
1327
+ name: MTEB FiQA2018
1328
+ type: mteb/fiqa
1329
+ config: default
1330
+ split: test
1331
+ revision: 27a168819829fe9bcd655c2df245fb19452e8e06
1332
+ metrics:
1333
+ - type: map_at_1
1334
+ value: 27.36
1335
+ - type: map_at_10
1336
+ value: 46.847
1337
+ - type: map_at_100
1338
+ value: 49.259
1339
+ - type: map_at_1000
1340
+ value: 49.389
1341
+ - type: map_at_3
1342
+ value: 41.095
1343
+ - type: map_at_5
1344
+ value: 44.084
1345
+ - type: mrr_at_1
1346
+ value: 51.852
1347
+ - type: mrr_at_10
1348
+ value: 61.67
1349
+ - type: mrr_at_100
1350
+ value: 62.395999999999994
1351
+ - type: mrr_at_1000
1352
+ value: 62.414
1353
+ - type: mrr_at_3
1354
+ value: 59.465
1355
+ - type: mrr_at_5
1356
+ value: 60.584
1357
+ - type: ndcg_at_1
1358
+ value: 51.852
1359
+ - type: ndcg_at_10
1360
+ value: 55.311
1361
+ - type: ndcg_at_100
1362
+ value: 62.6
1363
+ - type: ndcg_at_1000
1364
+ value: 64.206
1365
+ - type: ndcg_at_3
1366
+ value: 51.159
1367
+ - type: ndcg_at_5
1368
+ value: 52.038
1369
+ - type: precision_at_1
1370
+ value: 51.852
1371
+ - type: precision_at_10
1372
+ value: 15.370000000000001
1373
+ - type: precision_at_100
1374
+ value: 2.282
1375
+ - type: precision_at_1000
1376
+ value: 0.258
1377
+ - type: precision_at_3
1378
+ value: 34.721999999999994
1379
+ - type: precision_at_5
1380
+ value: 24.846
1381
+ - type: recall_at_1
1382
+ value: 27.36
1383
+ - type: recall_at_10
1384
+ value: 63.932
1385
+ - type: recall_at_100
1386
+ value: 89.824
1387
+ - type: recall_at_1000
1388
+ value: 98.556
1389
+ - type: recall_at_3
1390
+ value: 47.227999999999994
1391
+ - type: recall_at_5
1392
+ value: 53.724000000000004
1393
+ - task:
1394
+ type: Retrieval
1395
+ dataset:
1396
+ name: MTEB HotpotQA
1397
+ type: mteb/hotpotqa
1398
+ config: default
1399
+ split: test
1400
+ revision: ab518f4d6fcca38d87c25209f94beba119d02014
1401
+ metrics:
1402
+ - type: map_at_1
1403
+ value: 40.655
1404
+ - type: map_at_10
1405
+ value: 63.824999999999996
1406
+ - type: map_at_100
1407
+ value: 64.793
1408
+ - type: map_at_1000
1409
+ value: 64.848
1410
+ - type: map_at_3
1411
+ value: 60.221000000000004
1412
+ - type: map_at_5
1413
+ value: 62.474
1414
+ - type: mrr_at_1
1415
+ value: 81.31
1416
+ - type: mrr_at_10
1417
+ value: 86.509
1418
+ - type: mrr_at_100
1419
+ value: 86.677
1420
+ - type: mrr_at_1000
1421
+ value: 86.682
1422
+ - type: mrr_at_3
1423
+ value: 85.717
1424
+ - type: mrr_at_5
1425
+ value: 86.21
1426
+ - type: ndcg_at_1
1427
+ value: 81.31
1428
+ - type: ndcg_at_10
1429
+ value: 72.251
1430
+ - type: ndcg_at_100
1431
+ value: 75.536
1432
+ - type: ndcg_at_1000
1433
+ value: 76.558
1434
+ - type: ndcg_at_3
1435
+ value: 67.291
1436
+ - type: ndcg_at_5
1437
+ value: 70.045
1438
+ - type: precision_at_1
1439
+ value: 81.31
1440
+ - type: precision_at_10
1441
+ value: 15.082999999999998
1442
+ - type: precision_at_100
1443
+ value: 1.764
1444
+ - type: precision_at_1000
1445
+ value: 0.19
1446
+ - type: precision_at_3
1447
+ value: 42.971
1448
+ - type: precision_at_5
1449
+ value: 27.956999999999997
1450
+ - type: recall_at_1
1451
+ value: 40.655
1452
+ - type: recall_at_10
1453
+ value: 75.41499999999999
1454
+ - type: recall_at_100
1455
+ value: 88.224
1456
+ - type: recall_at_1000
1457
+ value: 94.943
1458
+ - type: recall_at_3
1459
+ value: 64.456
1460
+ - type: recall_at_5
1461
+ value: 69.892
1462
+ - task:
1463
+ type: Classification
1464
+ dataset:
1465
+ name: MTEB ImdbClassification
1466
+ type: mteb/imdb
1467
+ config: default
1468
+ split: test
1469
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1470
+ metrics:
1471
+ - type: accuracy
1472
+ value: 95.58120000000001
1473
+ - type: ap
1474
+ value: 93.0407063004784
1475
+ - type: f1
1476
+ value: 95.57849992996822
1477
+ - task:
1478
+ type: Retrieval
1479
+ dataset:
1480
+ name: MTEB MSMARCO
1481
+ type: mteb/msmarco
1482
+ config: default
1483
+ split: dev
1484
+ revision: c5a29a104738b98a9e76336939199e264163d4a0
1485
+ metrics:
1486
+ - type: map_at_1
1487
+ value: 22.031
1488
+ - type: map_at_10
1489
+ value: 34.628
1490
+ - type: map_at_100
1491
+ value: 35.833
1492
+ - type: map_at_1000
1493
+ value: 35.881
1494
+ - type: map_at_3
1495
+ value: 30.619000000000003
1496
+ - type: map_at_5
1497
+ value: 32.982
1498
+ - type: mrr_at_1
1499
+ value: 22.736
1500
+ - type: mrr_at_10
1501
+ value: 35.24
1502
+ - type: mrr_at_100
1503
+ value: 36.381
1504
+ - type: mrr_at_1000
1505
+ value: 36.424
1506
+ - type: mrr_at_3
1507
+ value: 31.287
1508
+ - type: mrr_at_5
1509
+ value: 33.617000000000004
1510
+ - type: ndcg_at_1
1511
+ value: 22.736
1512
+ - type: ndcg_at_10
1513
+ value: 41.681000000000004
1514
+ - type: ndcg_at_100
1515
+ value: 47.371
1516
+ - type: ndcg_at_1000
1517
+ value: 48.555
1518
+ - type: ndcg_at_3
1519
+ value: 33.553
1520
+ - type: ndcg_at_5
1521
+ value: 37.771
1522
+ - type: precision_at_1
1523
+ value: 22.736
1524
+ - type: precision_at_10
1525
+ value: 6.625
1526
+ - type: precision_at_100
1527
+ value: 0.9450000000000001
1528
+ - type: precision_at_1000
1529
+ value: 0.105
1530
+ - type: precision_at_3
1531
+ value: 14.331
1532
+ - type: precision_at_5
1533
+ value: 10.734
1534
+ - type: recall_at_1
1535
+ value: 22.031
1536
+ - type: recall_at_10
1537
+ value: 63.378
1538
+ - type: recall_at_100
1539
+ value: 89.47699999999999
1540
+ - type: recall_at_1000
1541
+ value: 98.48400000000001
1542
+ - type: recall_at_3
1543
+ value: 41.388000000000005
1544
+ - type: recall_at_5
1545
+ value: 51.522999999999996
1546
+ - task:
1547
+ type: Classification
1548
+ dataset:
1549
+ name: MTEB MTOPDomainClassification (en)
1550
+ type: mteb/mtop_domain
1551
+ config: en
1552
+ split: test
1553
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1554
+ metrics:
1555
+ - type: accuracy
1556
+ value: 95.75239398084815
1557
+ - type: f1
1558
+ value: 95.51228043205194
1559
+ - task:
1560
+ type: Classification
1561
+ dataset:
1562
+ name: MTEB MTOPIntentClassification (en)
1563
+ type: mteb/mtop_intent
1564
+ config: en
1565
+ split: test
1566
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1567
+ metrics:
1568
+ - type: accuracy
1569
+ value: 84.25900592795259
1570
+ - type: f1
1571
+ value: 62.14790420114562
1572
+ - task:
1573
+ type: Classification
1574
+ dataset:
1575
+ name: MTEB MassiveIntentClassification (en)
1576
+ type: mteb/amazon_massive_intent
1577
+ config: en
1578
+ split: test
1579
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1580
+ metrics:
1581
+ - type: accuracy
1582
+ value: 78.47007397444519
1583
+ - type: f1
1584
+ value: 76.92133583932912
1585
+ - task:
1586
+ type: Classification
1587
+ dataset:
1588
+ name: MTEB MassiveScenarioClassification (en)
1589
+ type: mteb/amazon_massive_scenario
1590
+ config: en
1591
+ split: test
1592
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
1593
+ metrics:
1594
+ - type: accuracy
1595
+ value: 78.19098856758575
1596
+ - type: f1
1597
+ value: 78.10820805879119
1598
+ - task:
1599
+ type: Clustering
1600
+ dataset:
1601
+ name: MTEB MedrxivClusteringP2P
1602
+ type: mteb/medrxiv-clustering-p2p
1603
+ config: default
1604
+ split: test
1605
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1606
+ metrics:
1607
+ - type: v_measure
1608
+ value: 44.37013684222983
1609
+ - task:
1610
+ type: Clustering
1611
+ dataset:
1612
+ name: MTEB MedrxivClusteringS2S
1613
+ type: mteb/medrxiv-clustering-s2s
1614
+ config: default
1615
+ split: test
1616
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1617
+ metrics:
1618
+ - type: v_measure
1619
+ value: 42.003012591979704
1620
+ - task:
1621
+ type: Reranking
1622
+ dataset:
1623
+ name: MTEB MindSmallReranking
1624
+ type: mteb/mind_small
1625
+ config: default
1626
+ split: test
1627
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1628
+ metrics:
1629
+ - type: map
1630
+ value: 32.70743071063257
1631
+ - type: mrr
1632
+ value: 33.938337390083994
1633
+ - task:
1634
+ type: Retrieval
1635
+ dataset:
1636
+ name: MTEB NFCorpus
1637
+ type: mteb/nfcorpus
1638
+ config: default
1639
+ split: test
1640
+ revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
1641
+ metrics:
1642
+ - type: map_at_1
1643
+ value: 6.369
1644
+ - type: map_at_10
1645
+ value: 14.313
1646
+ - type: map_at_100
1647
+ value: 18.329
1648
+ - type: map_at_1000
1649
+ value: 20.017
1650
+ - type: map_at_3
1651
+ value: 10.257
1652
+ - type: map_at_5
1653
+ value: 12.264999999999999
1654
+ - type: mrr_at_1
1655
+ value: 49.536
1656
+ - type: mrr_at_10
1657
+ value: 58.464000000000006
1658
+ - type: mrr_at_100
1659
+ value: 59.016000000000005
1660
+ - type: mrr_at_1000
1661
+ value: 59.053
1662
+ - type: mrr_at_3
1663
+ value: 56.294999999999995
1664
+ - type: mrr_at_5
1665
+ value: 57.766
1666
+ - type: ndcg_at_1
1667
+ value: 47.678
1668
+ - type: ndcg_at_10
1669
+ value: 38.246
1670
+ - type: ndcg_at_100
1671
+ value: 35.370000000000005
1672
+ - type: ndcg_at_1000
1673
+ value: 44.517
1674
+ - type: ndcg_at_3
1675
+ value: 43.368
1676
+ - type: ndcg_at_5
1677
+ value: 41.892
1678
+ - type: precision_at_1
1679
+ value: 49.536
1680
+ - type: precision_at_10
1681
+ value: 28.235
1682
+ - type: precision_at_100
1683
+ value: 9.014999999999999
1684
+ - type: precision_at_1000
1685
+ value: 2.257
1686
+ - type: precision_at_3
1687
+ value: 40.557
1688
+ - type: precision_at_5
1689
+ value: 36.409000000000006
1690
+ - type: recall_at_1
1691
+ value: 6.369
1692
+ - type: recall_at_10
1693
+ value: 19.195999999999998
1694
+ - type: recall_at_100
1695
+ value: 37.042
1696
+ - type: recall_at_1000
1697
+ value: 69.203
1698
+ - type: recall_at_3
1699
+ value: 11.564
1700
+ - type: recall_at_5
1701
+ value: 15.264
1702
+ - task:
1703
+ type: Retrieval
1704
+ dataset:
1705
+ name: MTEB NQ
1706
+ type: mteb/nq
1707
+ config: default
1708
+ split: test
1709
+ revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
1710
+ metrics:
1711
+ - type: map_at_1
1712
+ value: 39.323
1713
+ - type: map_at_10
1714
+ value: 54.608999999999995
1715
+ - type: map_at_100
1716
+ value: 55.523
1717
+ - type: map_at_1000
1718
+ value: 55.544000000000004
1719
+ - type: map_at_3
1720
+ value: 50.580000000000005
1721
+ - type: map_at_5
1722
+ value: 53.064
1723
+ - type: mrr_at_1
1724
+ value: 44.263999999999996
1725
+ - type: mrr_at_10
1726
+ value: 57.416
1727
+ - type: mrr_at_100
1728
+ value: 58.037000000000006
1729
+ - type: mrr_at_1000
1730
+ value: 58.05200000000001
1731
+ - type: mrr_at_3
1732
+ value: 54.330999999999996
1733
+ - type: mrr_at_5
1734
+ value: 56.302
1735
+ - type: ndcg_at_1
1736
+ value: 44.263999999999996
1737
+ - type: ndcg_at_10
1738
+ value: 61.785999999999994
1739
+ - type: ndcg_at_100
1740
+ value: 65.40599999999999
1741
+ - type: ndcg_at_1000
1742
+ value: 65.859
1743
+ - type: ndcg_at_3
1744
+ value: 54.518
1745
+ - type: ndcg_at_5
1746
+ value: 58.53699999999999
1747
+ - type: precision_at_1
1748
+ value: 44.263999999999996
1749
+ - type: precision_at_10
1750
+ value: 9.652
1751
+ - type: precision_at_100
1752
+ value: 1.169
1753
+ - type: precision_at_1000
1754
+ value: 0.121
1755
+ - type: precision_at_3
1756
+ value: 24.15
1757
+ - type: precision_at_5
1758
+ value: 16.848
1759
+ - type: recall_at_1
1760
+ value: 39.323
1761
+ - type: recall_at_10
1762
+ value: 80.663
1763
+ - type: recall_at_100
1764
+ value: 96.072
1765
+ - type: recall_at_1000
1766
+ value: 99.37700000000001
1767
+ - type: recall_at_3
1768
+ value: 62.23
1769
+ - type: recall_at_5
1770
+ value: 71.379
1771
+ - task:
1772
+ type: Retrieval
1773
+ dataset:
1774
+ name: MTEB QuoraRetrieval
1775
+ type: mteb/quora
1776
+ config: default
1777
+ split: test
1778
+ revision: None
1779
+ metrics:
1780
+ - type: map_at_1
1781
+ value: 72.02499999999999
1782
+ - type: map_at_10
1783
+ value: 86.14500000000001
1784
+ - type: map_at_100
1785
+ value: 86.764
1786
+ - type: map_at_1000
1787
+ value: 86.776
1788
+ - type: map_at_3
1789
+ value: 83.249
1790
+ - type: map_at_5
1791
+ value: 85.083
1792
+ - type: mrr_at_1
1793
+ value: 82.83
1794
+ - type: mrr_at_10
1795
+ value: 88.70599999999999
1796
+ - type: mrr_at_100
1797
+ value: 88.791
1798
+ - type: mrr_at_1000
1799
+ value: 88.791
1800
+ - type: mrr_at_3
1801
+ value: 87.815
1802
+ - type: mrr_at_5
1803
+ value: 88.435
1804
+ - type: ndcg_at_1
1805
+ value: 82.84
1806
+ - type: ndcg_at_10
1807
+ value: 89.61200000000001
1808
+ - type: ndcg_at_100
1809
+ value: 90.693
1810
+ - type: ndcg_at_1000
1811
+ value: 90.752
1812
+ - type: ndcg_at_3
1813
+ value: 86.96199999999999
1814
+ - type: ndcg_at_5
1815
+ value: 88.454
1816
+ - type: precision_at_1
1817
+ value: 82.84
1818
+ - type: precision_at_10
1819
+ value: 13.600000000000001
1820
+ - type: precision_at_100
1821
+ value: 1.543
1822
+ - type: precision_at_1000
1823
+ value: 0.157
1824
+ - type: precision_at_3
1825
+ value: 38.092999999999996
1826
+ - type: precision_at_5
1827
+ value: 25.024
1828
+ - type: recall_at_1
1829
+ value: 72.02499999999999
1830
+ - type: recall_at_10
1831
+ value: 96.21600000000001
1832
+ - type: recall_at_100
1833
+ value: 99.76
1834
+ - type: recall_at_1000
1835
+ value: 99.996
1836
+ - type: recall_at_3
1837
+ value: 88.57000000000001
1838
+ - type: recall_at_5
1839
+ value: 92.814
1840
+ - task:
1841
+ type: Clustering
1842
+ dataset:
1843
+ name: MTEB RedditClustering
1844
+ type: mteb/reddit-clustering
1845
+ config: default
1846
+ split: test
1847
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
1848
+ metrics:
1849
+ - type: v_measure
1850
+ value: 73.37297191949929
1851
+ - task:
1852
+ type: Clustering
1853
+ dataset:
1854
+ name: MTEB RedditClusteringP2P
1855
+ type: mteb/reddit-clustering-p2p
1856
+ config: default
1857
+ split: test
1858
+ revision: 282350215ef01743dc01b456c7f5241fa8937f16
1859
+ metrics:
1860
+ - type: v_measure
1861
+ value: 72.50752304246946
1862
+ - task:
1863
+ type: Retrieval
1864
+ dataset:
1865
+ name: MTEB SCIDOCS
1866
+ type: mteb/scidocs
1867
+ config: default
1868
+ split: test
1869
+ revision: None
1870
+ metrics:
1871
+ - type: map_at_1
1872
+ value: 6.4479999999999995
1873
+ - type: map_at_10
1874
+ value: 17.268
1875
+ - type: map_at_100
1876
+ value: 20.502000000000002
1877
+ - type: map_at_1000
1878
+ value: 20.904
1879
+ - type: map_at_3
1880
+ value: 11.951
1881
+ - type: map_at_5
1882
+ value: 14.494000000000002
1883
+ - type: mrr_at_1
1884
+ value: 31.900000000000002
1885
+ - type: mrr_at_10
1886
+ value: 45.084999999999994
1887
+ - type: mrr_at_100
1888
+ value: 46.145
1889
+ - type: mrr_at_1000
1890
+ value: 46.164
1891
+ - type: mrr_at_3
1892
+ value: 41.6
1893
+ - type: mrr_at_5
1894
+ value: 43.76
1895
+ - type: ndcg_at_1
1896
+ value: 31.900000000000002
1897
+ - type: ndcg_at_10
1898
+ value: 27.694000000000003
1899
+ - type: ndcg_at_100
1900
+ value: 39.016
1901
+ - type: ndcg_at_1000
1902
+ value: 44.448
1903
+ - type: ndcg_at_3
1904
+ value: 26.279999999999998
1905
+ - type: ndcg_at_5
1906
+ value: 22.93
1907
+ - type: precision_at_1
1908
+ value: 31.900000000000002
1909
+ - type: precision_at_10
1910
+ value: 14.399999999999999
1911
+ - type: precision_at_100
1912
+ value: 3.082
1913
+ - type: precision_at_1000
1914
+ value: 0.436
1915
+ - type: precision_at_3
1916
+ value: 24.667
1917
+ - type: precision_at_5
1918
+ value: 20.200000000000003
1919
+ - type: recall_at_1
1920
+ value: 6.4479999999999995
1921
+ - type: recall_at_10
1922
+ value: 29.243000000000002
1923
+ - type: recall_at_100
1924
+ value: 62.547
1925
+ - type: recall_at_1000
1926
+ value: 88.40299999999999
1927
+ - type: recall_at_3
1928
+ value: 14.988000000000001
1929
+ - type: recall_at_5
1930
+ value: 20.485
1931
+ - task:
1932
+ type: STS
1933
+ dataset:
1934
+ name: MTEB SICK-R
1935
+ type: mteb/sickr-sts
1936
+ config: default
1937
+ split: test
1938
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
1939
+ metrics:
1940
+ - type: cos_sim_pearson
1941
+ value: 80.37839336866843
1942
+ - type: cos_sim_spearman
1943
+ value: 79.14737320486729
1944
+ - type: euclidean_pearson
1945
+ value: 78.74010870392799
1946
+ - type: euclidean_spearman
1947
+ value: 79.1472505448557
1948
+ - type: manhattan_pearson
1949
+ value: 78.76735626972086
1950
+ - type: manhattan_spearman
1951
+ value: 79.18509055331465
1952
+ - task:
1953
+ type: STS
1954
+ dataset:
1955
+ name: MTEB STS12
1956
+ type: mteb/sts12-sts
1957
+ config: default
1958
+ split: test
1959
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
1960
+ metrics:
1961
+ - type: cos_sim_pearson
1962
+ value: 84.98947740740309
1963
+ - type: cos_sim_spearman
1964
+ value: 76.52068694652895
1965
+ - type: euclidean_pearson
1966
+ value: 81.10952542010847
1967
+ - type: euclidean_spearman
1968
+ value: 76.52162808897668
1969
+ - type: manhattan_pearson
1970
+ value: 81.13752577872523
1971
+ - type: manhattan_spearman
1972
+ value: 76.55073892851847
1973
+ - type: cos_sim_pearson
1974
+ value: 84.99292517797305
1975
+ - type: cos_sim_spearman
1976
+ value: 76.52287451692155
1977
+ - type: euclidean_pearson
1978
+ value: 81.11616055544546
1979
+ - type: euclidean_spearman
1980
+ value: 76.525387473028
1981
+ - type: manhattan_pearson
1982
+ value: 81.14367598670032
1983
+ - type: manhattan_spearman
1984
+ value: 76.55571799438607
1985
+ - task:
1986
+ type: STS
1987
+ dataset:
1988
+ name: MTEB STS13
1989
+ type: mteb/sts13-sts
1990
+ config: default
1991
+ split: test
1992
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
1993
+ metrics:
1994
+ - type: cos_sim_pearson
1995
+ value: 88.14795728641734
1996
+ - type: cos_sim_spearman
1997
+ value: 88.62720469210905
1998
+ - type: euclidean_pearson
1999
+ value: 87.96160445129142
2000
+ - type: euclidean_spearman
2001
+ value: 88.62615925428736
2002
+ - type: manhattan_pearson
2003
+ value: 87.86760858379527
2004
+ - type: manhattan_spearman
2005
+ value: 88.5613166629411
2006
+ - task:
2007
+ type: STS
2008
+ dataset:
2009
+ name: MTEB STS14
2010
+ type: mteb/sts14-sts
2011
+ config: default
2012
+ split: test
2013
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
2014
+ metrics:
2015
+ - type: cos_sim_pearson
2016
+ value: 85.06444249948838
2017
+ - type: cos_sim_spearman
2018
+ value: 83.32346434965837
2019
+ - type: euclidean_pearson
2020
+ value: 83.86264166785146
2021
+ - type: euclidean_spearman
2022
+ value: 83.32323156068114
2023
+ - type: manhattan_pearson
2024
+ value: 83.87253909108084
2025
+ - type: manhattan_spearman
2026
+ value: 83.42760090819642
2027
+ - task:
2028
+ type: STS
2029
+ dataset:
2030
+ name: MTEB STS15
2031
+ type: mteb/sts15-sts
2032
+ config: default
2033
+ split: test
2034
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
2035
+ metrics:
2036
+ - type: cos_sim_pearson
2037
+ value: 87.00847937091636
2038
+ - type: cos_sim_spearman
2039
+ value: 87.50432670473445
2040
+ - type: euclidean_pearson
2041
+ value: 87.21611485565168
2042
+ - type: euclidean_spearman
2043
+ value: 87.50387351928698
2044
+ - type: manhattan_pearson
2045
+ value: 87.30690660623411
2046
+ - type: manhattan_spearman
2047
+ value: 87.61147161393255
2048
+ - task:
2049
+ type: STS
2050
+ dataset:
2051
+ name: MTEB STS16
2052
+ type: mteb/sts16-sts
2053
+ config: default
2054
+ split: test
2055
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2056
+ metrics:
2057
+ - type: cos_sim_pearson
2058
+ value: 85.51456553517488
2059
+ - type: cos_sim_spearman
2060
+ value: 86.39208323626035
2061
+ - type: euclidean_pearson
2062
+ value: 85.74698473006475
2063
+ - type: euclidean_spearman
2064
+ value: 86.3892506146807
2065
+ - type: manhattan_pearson
2066
+ value: 85.77493611949014
2067
+ - type: manhattan_spearman
2068
+ value: 86.42961510735024
2069
+ - task:
2070
+ type: STS
2071
+ dataset:
2072
+ name: MTEB STS17 (en-en)
2073
+ type: mteb/sts17-crosslingual-sts
2074
+ config: en-en
2075
+ split: test
2076
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2077
+ metrics:
2078
+ - type: cos_sim_pearson
2079
+ value: 88.63402051628222
2080
+ - type: cos_sim_spearman
2081
+ value: 87.78994504115502
2082
+ - type: euclidean_pearson
2083
+ value: 88.44861926968403
2084
+ - type: euclidean_spearman
2085
+ value: 87.80670473078185
2086
+ - type: manhattan_pearson
2087
+ value: 88.4773722010208
2088
+ - type: manhattan_spearman
2089
+ value: 87.85175600656768
2090
+ - task:
2091
+ type: STS
2092
+ dataset:
2093
+ name: MTEB STS22 (en)
2094
+ type: mteb/sts22-crosslingual-sts
2095
+ config: en
2096
+ split: test
2097
+ revision: eea2b4fe26a775864c896887d910b76a8098ad3f
2098
+ metrics:
2099
+ - type: cos_sim_pearson
2100
+ value: 65.9659729672951
2101
+ - type: cos_sim_spearman
2102
+ value: 66.39891735341361
2103
+ - type: euclidean_pearson
2104
+ value: 68.040150710449
2105
+ - type: euclidean_spearman
2106
+ value: 66.41777234484414
2107
+ - type: manhattan_pearson
2108
+ value: 68.16264809387305
2109
+ - type: manhattan_spearman
2110
+ value: 66.31608161700346
2111
+ - task:
2112
+ type: STS
2113
+ dataset:
2114
+ name: MTEB STSBenchmark
2115
+ type: mteb/stsbenchmark-sts
2116
+ config: default
2117
+ split: test
2118
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2119
+ metrics:
2120
+ - type: cos_sim_pearson
2121
+ value: 86.91024857159385
2122
+ - type: cos_sim_spearman
2123
+ value: 87.35031011815016
2124
+ - type: euclidean_pearson
2125
+ value: 86.94569462996033
2126
+ - type: euclidean_spearman
2127
+ value: 87.34929703462852
2128
+ - type: manhattan_pearson
2129
+ value: 86.94404111225616
2130
+ - type: manhattan_spearman
2131
+ value: 87.37827218003393
2132
+ - task:
2133
+ type: Reranking
2134
+ dataset:
2135
+ name: MTEB SciDocsRR
2136
+ type: mteb/scidocs-reranking
2137
+ config: default
2138
+ split: test
2139
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2140
+ metrics:
2141
+ - type: map
2142
+ value: 87.89077927002596
2143
+ - type: mrr
2144
+ value: 96.94650937297997
2145
+ - task:
2146
+ type: Retrieval
2147
+ dataset:
2148
+ name: MTEB SciFact
2149
+ type: mteb/scifact
2150
+ config: default
2151
+ split: test
2152
+ revision: 0228b52cf27578f30900b9e5271d331663a030d7
2153
+ metrics:
2154
+ - type: map_at_1
2155
+ value: 57.994
2156
+ - type: map_at_10
2157
+ value: 70.07100000000001
2158
+ - type: map_at_100
2159
+ value: 70.578
2160
+ - type: map_at_1000
2161
+ value: 70.588
2162
+ - type: map_at_3
2163
+ value: 67.228
2164
+ - type: map_at_5
2165
+ value: 68.695
2166
+ - type: mrr_at_1
2167
+ value: 61.333000000000006
2168
+ - type: mrr_at_10
2169
+ value: 71.342
2170
+ - type: mrr_at_100
2171
+ value: 71.739
2172
+ - type: mrr_at_1000
2173
+ value: 71.75
2174
+ - type: mrr_at_3
2175
+ value: 69.389
2176
+ - type: mrr_at_5
2177
+ value: 70.322
2178
+ - type: ndcg_at_1
2179
+ value: 61.333000000000006
2180
+ - type: ndcg_at_10
2181
+ value: 75.312
2182
+ - type: ndcg_at_100
2183
+ value: 77.312
2184
+ - type: ndcg_at_1000
2185
+ value: 77.50200000000001
2186
+ - type: ndcg_at_3
2187
+ value: 70.72
2188
+ - type: ndcg_at_5
2189
+ value: 72.616
2190
+ - type: precision_at_1
2191
+ value: 61.333000000000006
2192
+ - type: precision_at_10
2193
+ value: 10.167
2194
+ - type: precision_at_100
2195
+ value: 1.117
2196
+ - type: precision_at_1000
2197
+ value: 0.11299999999999999
2198
+ - type: precision_at_3
2199
+ value: 28.111000000000004
2200
+ - type: precision_at_5
2201
+ value: 18.333
2202
+ - type: recall_at_1
2203
+ value: 57.994
2204
+ - type: recall_at_10
2205
+ value: 89.944
2206
+ - type: recall_at_100
2207
+ value: 98.667
2208
+ - type: recall_at_1000
2209
+ value: 100.0
2210
+ - type: recall_at_3
2211
+ value: 77.694
2212
+ - type: recall_at_5
2213
+ value: 82.339
2214
+ - task:
2215
+ type: PairClassification
2216
+ dataset:
2217
+ name: MTEB SprintDuplicateQuestions
2218
+ type: mteb/sprintduplicatequestions-pairclassification
2219
+ config: default
2220
+ split: test
2221
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2222
+ metrics:
2223
+ - type: cos_sim_accuracy
2224
+ value: 99.81485148514851
2225
+ - type: cos_sim_ap
2226
+ value: 95.99339654021689
2227
+ - type: cos_sim_f1
2228
+ value: 90.45971329708354
2229
+ - type: cos_sim_precision
2230
+ value: 89.44281524926686
2231
+ - type: cos_sim_recall
2232
+ value: 91.5
2233
+ - type: dot_accuracy
2234
+ value: 99.81485148514851
2235
+ - type: dot_ap
2236
+ value: 95.990792367539
2237
+ - type: dot_f1
2238
+ value: 90.54187192118228
2239
+ - type: dot_precision
2240
+ value: 89.2233009708738
2241
+ - type: dot_recall
2242
+ value: 91.9
2243
+ - type: euclidean_accuracy
2244
+ value: 99.81386138613861
2245
+ - type: euclidean_ap
2246
+ value: 95.99403827746491
2247
+ - type: euclidean_f1
2248
+ value: 90.45971329708354
2249
+ - type: euclidean_precision
2250
+ value: 89.44281524926686
2251
+ - type: euclidean_recall
2252
+ value: 91.5
2253
+ - type: manhattan_accuracy
2254
+ value: 99.81485148514851
2255
+ - type: manhattan_ap
2256
+ value: 96.06741547889861
2257
+ - type: manhattan_f1
2258
+ value: 90.55666003976144
2259
+ - type: manhattan_precision
2260
+ value: 90.01976284584981
2261
+ - type: manhattan_recall
2262
+ value: 91.10000000000001
2263
+ - type: max_accuracy
2264
+ value: 99.81485148514851
2265
+ - type: max_ap
2266
+ value: 96.06741547889861
2267
+ - type: max_f1
2268
+ value: 90.55666003976144
2269
+ - task:
2270
+ type: Clustering
2271
+ dataset:
2272
+ name: MTEB StackExchangeClustering
2273
+ type: mteb/stackexchange-clustering
2274
+ config: default
2275
+ split: test
2276
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2277
+ metrics:
2278
+ - type: v_measure
2279
+ value: 79.0667992003181
2280
+ - task:
2281
+ type: Clustering
2282
+ dataset:
2283
+ name: MTEB StackExchangeClusteringP2P
2284
+ type: mteb/stackexchange-clustering-p2p
2285
+ config: default
2286
+ split: test
2287
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2288
+ metrics:
2289
+ - type: v_measure
2290
+ value: 49.57086425048946
2291
+ - task:
2292
+ type: Reranking
2293
+ dataset:
2294
+ name: MTEB StackOverflowDupQuestions
2295
+ type: mteb/stackoverflowdupquestions-reranking
2296
+ config: default
2297
+ split: test
2298
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2299
+ metrics:
2300
+ - type: map
2301
+ value: 53.929415255105894
2302
+ - type: mrr
2303
+ value: 54.93889790764791
2304
+ - task:
2305
+ type: Summarization
2306
+ dataset:
2307
+ name: MTEB SummEval
2308
+ type: mteb/summeval
2309
+ config: default
2310
+ split: test
2311
+ revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
2312
+ metrics:
2313
+ - type: cos_sim_pearson
2314
+ value: 31.050700527286658
2315
+ - type: cos_sim_spearman
2316
+ value: 31.46077656458546
2317
+ - type: dot_pearson
2318
+ value: 31.056448416258263
2319
+ - type: dot_spearman
2320
+ value: 31.435272601921042
2321
+ - task:
2322
+ type: Retrieval
2323
+ dataset:
2324
+ name: MTEB TRECCOVID
2325
+ type: mteb/trec-covid
2326
+ config: default
2327
+ split: test
2328
+ revision: None
2329
+ metrics:
2330
+ - type: map_at_1
2331
+ value: 0.23500000000000001
2332
+ - type: map_at_10
2333
+ value: 1.812
2334
+ - type: map_at_100
2335
+ value: 10.041
2336
+ - type: map_at_1000
2337
+ value: 24.095
2338
+ - type: map_at_3
2339
+ value: 0.643
2340
+ - type: map_at_5
2341
+ value: 1.0
2342
+ - type: mrr_at_1
2343
+ value: 86.0
2344
+ - type: mrr_at_10
2345
+ value: 92.0
2346
+ - type: mrr_at_100
2347
+ value: 92.0
2348
+ - type: mrr_at_1000
2349
+ value: 92.0
2350
+ - type: mrr_at_3
2351
+ value: 91.667
2352
+ - type: mrr_at_5
2353
+ value: 91.667
2354
+ - type: ndcg_at_1
2355
+ value: 79.0
2356
+ - type: ndcg_at_10
2357
+ value: 72.72
2358
+ - type: ndcg_at_100
2359
+ value: 55.82899999999999
2360
+ - type: ndcg_at_1000
2361
+ value: 50.72
2362
+ - type: ndcg_at_3
2363
+ value: 77.715
2364
+ - type: ndcg_at_5
2365
+ value: 75.036
2366
+ - type: precision_at_1
2367
+ value: 86.0
2368
+ - type: precision_at_10
2369
+ value: 77.60000000000001
2370
+ - type: precision_at_100
2371
+ value: 56.46
2372
+ - type: precision_at_1000
2373
+ value: 22.23
2374
+ - type: precision_at_3
2375
+ value: 82.667
2376
+ - type: precision_at_5
2377
+ value: 80.4
2378
+ - type: recall_at_1
2379
+ value: 0.23500000000000001
2380
+ - type: recall_at_10
2381
+ value: 2.046
2382
+ - type: recall_at_100
2383
+ value: 13.708
2384
+ - type: recall_at_1000
2385
+ value: 47.451
2386
+ - type: recall_at_3
2387
+ value: 0.6709999999999999
2388
+ - type: recall_at_5
2389
+ value: 1.078
2390
+ - task:
2391
+ type: Retrieval
2392
+ dataset:
2393
+ name: MTEB Touche2020
2394
+ type: mteb/touche2020
2395
+ config: default
2396
+ split: test
2397
+ revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
2398
+ metrics:
2399
+ - type: map_at_1
2400
+ value: 2.252
2401
+ - type: map_at_10
2402
+ value: 7.958
2403
+ - type: map_at_100
2404
+ value: 12.293
2405
+ - type: map_at_1000
2406
+ value: 13.832
2407
+ - type: map_at_3
2408
+ value: 4.299
2409
+ - type: map_at_5
2410
+ value: 5.514
2411
+ - type: mrr_at_1
2412
+ value: 30.612000000000002
2413
+ - type: mrr_at_10
2414
+ value: 42.329
2415
+ - type: mrr_at_100
2416
+ value: 43.506
2417
+ - type: mrr_at_1000
2418
+ value: 43.506
2419
+ - type: mrr_at_3
2420
+ value: 38.775999999999996
2421
+ - type: mrr_at_5
2422
+ value: 39.592
2423
+ - type: ndcg_at_1
2424
+ value: 28.571
2425
+ - type: ndcg_at_10
2426
+ value: 20.301
2427
+ - type: ndcg_at_100
2428
+ value: 30.703999999999997
2429
+ - type: ndcg_at_1000
2430
+ value: 43.155
2431
+ - type: ndcg_at_3
2432
+ value: 22.738
2433
+ - type: ndcg_at_5
2434
+ value: 20.515
2435
+ - type: precision_at_1
2436
+ value: 30.612000000000002
2437
+ - type: precision_at_10
2438
+ value: 17.347
2439
+ - type: precision_at_100
2440
+ value: 6.327000000000001
2441
+ - type: precision_at_1000
2442
+ value: 1.443
2443
+ - type: precision_at_3
2444
+ value: 22.448999999999998
2445
+ - type: precision_at_5
2446
+ value: 19.184
2447
+ - type: recall_at_1
2448
+ value: 2.252
2449
+ - type: recall_at_10
2450
+ value: 13.206999999999999
2451
+ - type: recall_at_100
2452
+ value: 40.372
2453
+ - type: recall_at_1000
2454
+ value: 78.071
2455
+ - type: recall_at_3
2456
+ value: 5.189
2457
+ - type: recall_at_5
2458
+ value: 7.338
2459
+ - task:
2460
+ type: Classification
2461
+ dataset:
2462
+ name: MTEB ToxicConversationsClassification
2463
+ type: mteb/toxic_conversations_50k
2464
+ config: default
2465
+ split: test
2466
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2467
+ metrics:
2468
+ - type: accuracy
2469
+ value: 78.75399999999999
2470
+ - type: ap
2471
+ value: 19.666483622175363
2472
+ - type: f1
2473
+ value: 61.575187470329176
2474
+ - task:
2475
+ type: Classification
2476
+ dataset:
2477
+ name: MTEB TweetSentimentExtractionClassification
2478
+ type: mteb/tweet_sentiment_extraction
2479
+ config: default
2480
+ split: test
2481
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2482
+ metrics:
2483
+ - type: accuracy
2484
+ value: 66.00452744765137
2485
+ - type: f1
2486
+ value: 66.18291586829227
2487
+ - task:
2488
+ type: Clustering
2489
+ dataset:
2490
+ name: MTEB TwentyNewsgroupsClustering
2491
+ type: mteb/twentynewsgroups-clustering
2492
+ config: default
2493
+ split: test
2494
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2495
+ metrics:
2496
+ - type: v_measure
2497
+ value: 51.308747717084316
2498
+ - task:
2499
+ type: PairClassification
2500
+ dataset:
2501
+ name: MTEB TwitterSemEval2015
2502
+ type: mteb/twittersemeval2015-pairclassification
2503
+ config: default
2504
+ split: test
2505
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2506
+ metrics:
2507
+ - type: cos_sim_accuracy
2508
+ value: 87.81069321094355
2509
+ - type: cos_sim_ap
2510
+ value: 79.3576921453847
2511
+ - type: cos_sim_f1
2512
+ value: 71.75811286328685
2513
+ - type: cos_sim_precision
2514
+ value: 70.89878959567345
2515
+ - type: cos_sim_recall
2516
+ value: 72.63852242744063
2517
+ - type: dot_accuracy
2518
+ value: 87.79877212850927
2519
+ - type: dot_ap
2520
+ value: 79.35550320857683
2521
+ - type: dot_f1
2522
+ value: 71.78153446033811
2523
+ - type: dot_precision
2524
+ value: 70.76923076923077
2525
+ - type: dot_recall
2526
+ value: 72.82321899736148
2527
+ - type: euclidean_accuracy
2528
+ value: 87.80473266972642
2529
+ - type: euclidean_ap
2530
+ value: 79.35792655436586
2531
+ - type: euclidean_f1
2532
+ value: 71.75672148264161
2533
+ - type: euclidean_precision
2534
+ value: 70.99690082644628
2535
+ - type: euclidean_recall
2536
+ value: 72.53298153034301
2537
+ - type: manhattan_accuracy
2538
+ value: 87.76300888120642
2539
+ - type: manhattan_ap
2540
+ value: 79.33615959143606
2541
+ - type: manhattan_f1
2542
+ value: 71.73219978746015
2543
+ - type: manhattan_precision
2544
+ value: 72.23113964686998
2545
+ - type: manhattan_recall
2546
+ value: 71.2401055408971
2547
+ - type: max_accuracy
2548
+ value: 87.81069321094355
2549
+ - type: max_ap
2550
+ value: 79.35792655436586
2551
+ - type: max_f1
2552
+ value: 71.78153446033811
2553
+ - task:
2554
+ type: PairClassification
2555
+ dataset:
2556
+ name: MTEB TwitterURLCorpus
2557
+ type: mteb/twitterurlcorpus-pairclassification
2558
+ config: default
2559
+ split: test
2560
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2561
+ metrics:
2562
+ - type: cos_sim_accuracy
2563
+ value: 89.3778864439011
2564
+ - type: cos_sim_ap
2565
+ value: 86.79005637312795
2566
+ - type: cos_sim_f1
2567
+ value: 79.14617791685293
2568
+ - type: cos_sim_precision
2569
+ value: 76.66714780600462
2570
+ - type: cos_sim_recall
2571
+ value: 81.79088389282414
2572
+ - type: dot_accuracy
2573
+ value: 89.37206504443668
2574
+ - type: dot_ap
2575
+ value: 86.78770290102123
2576
+ - type: dot_f1
2577
+ value: 79.14741392159786
2578
+ - type: dot_precision
2579
+ value: 76.6897746967071
2580
+ - type: dot_recall
2581
+ value: 81.76778564829073
2582
+ - type: euclidean_accuracy
2583
+ value: 89.37594597741297
2584
+ - type: euclidean_ap
2585
+ value: 86.7900899669397
2586
+ - type: euclidean_f1
2587
+ value: 79.13920845898953
2588
+ - type: euclidean_precision
2589
+ value: 76.62028692956528
2590
+ - type: euclidean_recall
2591
+ value: 81.8293809670465
2592
+ - type: manhattan_accuracy
2593
+ value: 89.38758877634183
2594
+ - type: manhattan_ap
2595
+ value: 86.78862564973224
2596
+ - type: manhattan_f1
2597
+ value: 79.1130985653065
2598
+ - type: manhattan_precision
2599
+ value: 76.6592041597458
2600
+ - type: manhattan_recall
2601
+ value: 81.72928857406838
2602
+ - type: max_accuracy
2603
+ value: 89.38758877634183
2604
+ - type: max_ap
2605
+ value: 86.7900899669397
2606
+ - type: max_f1
2607
+ value: 79.14741392159786
2608
+ - task:
2609
+ type: STS
2610
+ dataset:
2611
+ name: MTEB AFQMC
2612
+ type: C-MTEB/AFQMC
2613
+ config: default
2614
+ split: validation
2615
+ revision: b44c3b011063adb25877c13823db83bb193913c4
2616
+ metrics:
2617
+ - type: cos_sim_pearson
2618
+ value: 50.01571015887356
2619
+ - type: cos_sim_spearman
2620
+ value: 58.47419994907958
2621
+ - type: euclidean_pearson
2622
+ value: 55.63582004345212
2623
+ - type: euclidean_spearman
2624
+ value: 58.47514484211099
2625
+ - type: manhattan_pearson
2626
+ value: 55.58487268871911
2627
+ - type: manhattan_spearman
2628
+ value: 58.411916843600075
2629
+ - task:
2630
+ type: STS
2631
+ dataset:
2632
+ name: MTEB ATEC
2633
+ type: C-MTEB/ATEC
2634
+ config: default
2635
+ split: test
2636
+ revision: 0f319b1142f28d00e055a6770f3f726ae9b7d865
2637
+ metrics:
2638
+ - type: cos_sim_pearson
2639
+ value: 44.99231617937922
2640
+ - type: cos_sim_spearman
2641
+ value: 55.459227458516416
2642
+ - type: euclidean_pearson
2643
+ value: 52.98483376548224
2644
+ - type: euclidean_spearman
2645
+ value: 55.45938733128155
2646
+ - type: manhattan_pearson
2647
+ value: 52.946854805143964
2648
+ - type: manhattan_spearman
2649
+ value: 55.4272663113618
2650
+ - task:
2651
+ type: Classification
2652
+ dataset:
2653
+ name: MTEB AmazonReviewsClassification (zh)
2654
+ type: mteb/amazon_reviews_multi
2655
+ config: zh
2656
+ split: test
2657
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
2658
+ metrics:
2659
+ - type: accuracy
2660
+ value: 52.946000000000005
2661
+ - type: f1
2662
+ value: 49.299873931232725
2663
+ - task:
2664
+ type: STS
2665
+ dataset:
2666
+ name: MTEB BQ
2667
+ type: C-MTEB/BQ
2668
+ config: default
2669
+ split: test
2670
+ revision: e3dda5e115e487b39ec7e618c0c6a29137052a55
2671
+ metrics:
2672
+ - type: cos_sim_pearson
2673
+ value: 74.66979530294986
2674
+ - type: cos_sim_spearman
2675
+ value: 77.59153258548018
2676
+ - type: euclidean_pearson
2677
+ value: 76.5862988380262
2678
+ - type: euclidean_spearman
2679
+ value: 77.59094368703879
2680
+ - type: manhattan_pearson
2681
+ value: 76.6034419552102
2682
+ - type: manhattan_spearman
2683
+ value: 77.6000715948404
2684
+ - task:
2685
+ type: Clustering
2686
+ dataset:
2687
+ name: MTEB CLSClusteringP2P
2688
+ type: C-MTEB/CLSClusteringP2P
2689
+ config: default
2690
+ split: test
2691
+ revision: 4b6227591c6c1a73bc76b1055f3b7f3588e72476
2692
+ metrics:
2693
+ - type: v_measure
2694
+ value: 47.20931915009524
2695
+ - task:
2696
+ type: Clustering
2697
+ dataset:
2698
+ name: MTEB CLSClusteringS2S
2699
+ type: C-MTEB/CLSClusteringS2S
2700
+ config: default
2701
+ split: test
2702
+ revision: e458b3f5414b62b7f9f83499ac1f5497ae2e869f
2703
+ metrics:
2704
+ - type: v_measure
2705
+ value: 45.787353610995474
2706
+ - task:
2707
+ type: Reranking
2708
+ dataset:
2709
+ name: MTEB CMedQAv1
2710
+ type: C-MTEB/CMedQAv1-reranking
2711
+ config: default
2712
+ split: test
2713
+ revision: 8d7f1e942507dac42dc58017c1a001c3717da7df
2714
+ metrics:
2715
+ - type: map
2716
+ value: 86.37146026784607
2717
+ - type: mrr
2718
+ value: 88.52309523809524
2719
+ - task:
2720
+ type: Reranking
2721
+ dataset:
2722
+ name: MTEB CMedQAv2
2723
+ type: C-MTEB/CMedQAv2-reranking
2724
+ config: default
2725
+ split: test
2726
+ revision: 23d186750531a14a0357ca22cd92d712fd512ea0
2727
+ metrics:
2728
+ - type: map
2729
+ value: 87.40699302584699
2730
+ - type: mrr
2731
+ value: 89.51591269841269
2732
+ - task:
2733
+ type: Retrieval
2734
+ dataset:
2735
+ name: MTEB CmedqaRetrieval
2736
+ type: C-MTEB/CmedqaRetrieval
2737
+ config: default
2738
+ split: dev
2739
+ revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
2740
+ metrics:
2741
+ - type: map_at_1
2742
+ value: 24.465
2743
+ - type: map_at_10
2744
+ value: 36.689
2745
+ - type: map_at_100
2746
+ value: 38.605000000000004
2747
+ - type: map_at_1000
2748
+ value: 38.718
2749
+ - type: map_at_3
2750
+ value: 32.399
2751
+ - type: map_at_5
2752
+ value: 34.784
2753
+ - type: mrr_at_1
2754
+ value: 37.234
2755
+ - type: mrr_at_10
2756
+ value: 45.634
2757
+ - type: mrr_at_100
2758
+ value: 46.676
2759
+ - type: mrr_at_1000
2760
+ value: 46.717
2761
+ - type: mrr_at_3
2762
+ value: 42.94
2763
+ - type: mrr_at_5
2764
+ value: 44.457
2765
+ - type: ndcg_at_1
2766
+ value: 37.234
2767
+ - type: ndcg_at_10
2768
+ value: 43.469
2769
+ - type: ndcg_at_100
2770
+ value: 51.048
2771
+ - type: ndcg_at_1000
2772
+ value: 52.925999999999995
2773
+ - type: ndcg_at_3
2774
+ value: 37.942
2775
+ - type: ndcg_at_5
2776
+ value: 40.253
2777
+ - type: precision_at_1
2778
+ value: 37.234
2779
+ - type: precision_at_10
2780
+ value: 9.745
2781
+ - type: precision_at_100
2782
+ value: 1.5879999999999999
2783
+ - type: precision_at_1000
2784
+ value: 0.183
2785
+ - type: precision_at_3
2786
+ value: 21.505
2787
+ - type: precision_at_5
2788
+ value: 15.729000000000001
2789
+ - type: recall_at_1
2790
+ value: 24.465
2791
+ - type: recall_at_10
2792
+ value: 54.559999999999995
2793
+ - type: recall_at_100
2794
+ value: 85.97200000000001
2795
+ - type: recall_at_1000
2796
+ value: 98.32499999999999
2797
+ - type: recall_at_3
2798
+ value: 38.047
2799
+ - type: recall_at_5
2800
+ value: 45.08
2801
+ - task:
2802
+ type: PairClassification
2803
+ dataset:
2804
+ name: MTEB Cmnli
2805
+ type: C-MTEB/CMNLI
2806
+ config: default
2807
+ split: validation
2808
+ revision: 41bc36f332156f7adc9e38f53777c959b2ae9766
2809
+ metrics:
2810
+ - type: cos_sim_accuracy
2811
+ value: 84.50992182802165
2812
+ - type: cos_sim_ap
2813
+ value: 91.81488661281966
2814
+ - type: cos_sim_f1
2815
+ value: 85.46855802524294
2816
+ - type: cos_sim_precision
2817
+ value: 81.82207014542344
2818
+ - type: cos_sim_recall
2819
+ value: 89.4552256254384
2820
+ - type: dot_accuracy
2821
+ value: 84.50992182802165
2822
+ - type: dot_ap
2823
+ value: 91.80547588176556
2824
+ - type: dot_f1
2825
+ value: 85.46492111446794
2826
+ - type: dot_precision
2827
+ value: 81.95278969957081
2828
+ - type: dot_recall
2829
+ value: 89.29155950432546
2830
+ - type: euclidean_accuracy
2831
+ value: 84.49789536981359
2832
+ - type: euclidean_ap
2833
+ value: 91.81495039620808
2834
+ - type: euclidean_f1
2835
+ value: 85.46817317373308
2836
+ - type: euclidean_precision
2837
+ value: 81.93908193908193
2838
+ - type: euclidean_recall
2839
+ value: 89.31494037877017
2840
+ - type: manhattan_accuracy
2841
+ value: 84.46181599518941
2842
+ - type: manhattan_ap
2843
+ value: 91.85400573633447
2844
+ - type: manhattan_f1
2845
+ value: 85.54283809312146
2846
+ - type: manhattan_precision
2847
+ value: 81.51207115628971
2848
+ - type: manhattan_recall
2849
+ value: 89.99298573766659
2850
+ - type: max_accuracy
2851
+ value: 84.50992182802165
2852
+ - type: max_ap
2853
+ value: 91.85400573633447
2854
+ - type: max_f1
2855
+ value: 85.54283809312146
2856
+ - task:
2857
+ type: Retrieval
2858
+ dataset:
2859
+ name: MTEB CovidRetrieval
2860
+ type: C-MTEB/CovidRetrieval
2861
+ config: default
2862
+ split: dev
2863
+ revision: 1271c7809071a13532e05f25fb53511ffce77117
2864
+ metrics:
2865
+ - type: map_at_1
2866
+ value: 68.072
2867
+ - type: map_at_10
2868
+ value: 76.82900000000001
2869
+ - type: map_at_100
2870
+ value: 77.146
2871
+ - type: map_at_1000
2872
+ value: 77.14999999999999
2873
+ - type: map_at_3
2874
+ value: 74.939
2875
+ - type: map_at_5
2876
+ value: 76.009
2877
+ - type: mrr_at_1
2878
+ value: 68.282
2879
+ - type: mrr_at_10
2880
+ value: 76.818
2881
+ - type: mrr_at_100
2882
+ value: 77.13600000000001
2883
+ - type: mrr_at_1000
2884
+ value: 77.14
2885
+ - type: mrr_at_3
2886
+ value: 74.956
2887
+ - type: mrr_at_5
2888
+ value: 76.047
2889
+ - type: ndcg_at_1
2890
+ value: 68.282
2891
+ - type: ndcg_at_10
2892
+ value: 80.87299999999999
2893
+ - type: ndcg_at_100
2894
+ value: 82.191
2895
+ - type: ndcg_at_1000
2896
+ value: 82.286
2897
+ - type: ndcg_at_3
2898
+ value: 77.065
2899
+ - type: ndcg_at_5
2900
+ value: 78.965
2901
+ - type: precision_at_1
2902
+ value: 68.282
2903
+ - type: precision_at_10
2904
+ value: 9.452
2905
+ - type: precision_at_100
2906
+ value: 1.002
2907
+ - type: precision_at_1000
2908
+ value: 0.101
2909
+ - type: precision_at_3
2910
+ value: 27.889000000000003
2911
+ - type: precision_at_5
2912
+ value: 17.682000000000002
2913
+ - type: recall_at_1
2914
+ value: 68.072
2915
+ - type: recall_at_10
2916
+ value: 93.467
2917
+ - type: recall_at_100
2918
+ value: 99.157
2919
+ - type: recall_at_1000
2920
+ value: 99.895
2921
+ - type: recall_at_3
2922
+ value: 83.14
2923
+ - type: recall_at_5
2924
+ value: 87.67099999999999
2925
+ - task:
2926
+ type: Retrieval
2927
+ dataset:
2928
+ name: MTEB DuRetrieval
2929
+ type: C-MTEB/DuRetrieval
2930
+ config: default
2931
+ split: dev
2932
+ revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
2933
+ metrics:
2934
+ - type: map_at_1
2935
+ value: 26.107999999999997
2936
+ - type: map_at_10
2937
+ value: 78.384
2938
+ - type: map_at_100
2939
+ value: 81.341
2940
+ - type: map_at_1000
2941
+ value: 81.384
2942
+ - type: map_at_3
2943
+ value: 54.462999999999994
2944
+ - type: map_at_5
2945
+ value: 68.607
2946
+ - type: mrr_at_1
2947
+ value: 88.94999999999999
2948
+ - type: mrr_at_10
2949
+ value: 92.31
2950
+ - type: mrr_at_100
2951
+ value: 92.379
2952
+ - type: mrr_at_1000
2953
+ value: 92.38300000000001
2954
+ - type: mrr_at_3
2955
+ value: 91.85799999999999
2956
+ - type: mrr_at_5
2957
+ value: 92.146
2958
+ - type: ndcg_at_1
2959
+ value: 88.94999999999999
2960
+ - type: ndcg_at_10
2961
+ value: 86.00999999999999
2962
+ - type: ndcg_at_100
2963
+ value: 89.121
2964
+ - type: ndcg_at_1000
2965
+ value: 89.534
2966
+ - type: ndcg_at_3
2967
+ value: 84.69200000000001
2968
+ - type: ndcg_at_5
2969
+ value: 83.678
2970
+ - type: precision_at_1
2971
+ value: 88.94999999999999
2972
+ - type: precision_at_10
2973
+ value: 41.065000000000005
2974
+ - type: precision_at_100
2975
+ value: 4.781
2976
+ - type: precision_at_1000
2977
+ value: 0.488
2978
+ - type: precision_at_3
2979
+ value: 75.75
2980
+ - type: precision_at_5
2981
+ value: 63.93
2982
+ - type: recall_at_1
2983
+ value: 26.107999999999997
2984
+ - type: recall_at_10
2985
+ value: 87.349
2986
+ - type: recall_at_100
2987
+ value: 97.14699999999999
2988
+ - type: recall_at_1000
2989
+ value: 99.287
2990
+ - type: recall_at_3
2991
+ value: 56.601
2992
+ - type: recall_at_5
2993
+ value: 73.381
2994
+ - task:
2995
+ type: Retrieval
2996
+ dataset:
2997
+ name: MTEB EcomRetrieval
2998
+ type: C-MTEB/EcomRetrieval
2999
+ config: default
3000
+ split: dev
3001
+ revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
3002
+ metrics:
3003
+ - type: map_at_1
3004
+ value: 50.7
3005
+ - type: map_at_10
3006
+ value: 61.312999999999995
3007
+ - type: map_at_100
3008
+ value: 61.88399999999999
3009
+ - type: map_at_1000
3010
+ value: 61.9
3011
+ - type: map_at_3
3012
+ value: 58.983
3013
+ - type: map_at_5
3014
+ value: 60.238
3015
+ - type: mrr_at_1
3016
+ value: 50.7
3017
+ - type: mrr_at_10
3018
+ value: 61.312999999999995
3019
+ - type: mrr_at_100
3020
+ value: 61.88399999999999
3021
+ - type: mrr_at_1000
3022
+ value: 61.9
3023
+ - type: mrr_at_3
3024
+ value: 58.983
3025
+ - type: mrr_at_5
3026
+ value: 60.238
3027
+ - type: ndcg_at_1
3028
+ value: 50.7
3029
+ - type: ndcg_at_10
3030
+ value: 66.458
3031
+ - type: ndcg_at_100
3032
+ value: 69.098
3033
+ - type: ndcg_at_1000
3034
+ value: 69.539
3035
+ - type: ndcg_at_3
3036
+ value: 61.637
3037
+ - type: ndcg_at_5
3038
+ value: 63.92099999999999
3039
+ - type: precision_at_1
3040
+ value: 50.7
3041
+ - type: precision_at_10
3042
+ value: 8.260000000000002
3043
+ - type: precision_at_100
3044
+ value: 0.946
3045
+ - type: precision_at_1000
3046
+ value: 0.098
3047
+ - type: precision_at_3
3048
+ value: 23.1
3049
+ - type: precision_at_5
3050
+ value: 14.979999999999999
3051
+ - type: recall_at_1
3052
+ value: 50.7
3053
+ - type: recall_at_10
3054
+ value: 82.6
3055
+ - type: recall_at_100
3056
+ value: 94.6
3057
+ - type: recall_at_1000
3058
+ value: 98.1
3059
+ - type: recall_at_3
3060
+ value: 69.3
3061
+ - type: recall_at_5
3062
+ value: 74.9
3063
+ - task:
3064
+ type: Classification
3065
+ dataset:
3066
+ name: MTEB IFlyTek
3067
+ type: C-MTEB/IFlyTek-classification
3068
+ config: default
3069
+ split: validation
3070
+ revision: 421605374b29664c5fc098418fe20ada9bd55f8a
3071
+ metrics:
3072
+ - type: accuracy
3073
+ value: 53.76683339746056
3074
+ - type: f1
3075
+ value: 40.026100192683714
3076
+ - task:
3077
+ type: Classification
3078
+ dataset:
3079
+ name: MTEB JDReview
3080
+ type: C-MTEB/JDReview-classification
3081
+ config: default
3082
+ split: test
3083
+ revision: b7c64bd89eb87f8ded463478346f76731f07bf8b
3084
+ metrics:
3085
+ - type: accuracy
3086
+ value: 88.19887429643526
3087
+ - type: ap
3088
+ value: 59.02998120976959
3089
+ - type: f1
3090
+ value: 83.3659125921227
3091
+ - task:
3092
+ type: STS
3093
+ dataset:
3094
+ name: MTEB LCQMC
3095
+ type: C-MTEB/LCQMC
3096
+ config: default
3097
+ split: test
3098
+ revision: 17f9b096f80380fce5ed12a9be8be7784b337daf
3099
+ metrics:
3100
+ - type: cos_sim_pearson
3101
+ value: 72.53955204856854
3102
+ - type: cos_sim_spearman
3103
+ value: 76.28996886746215
3104
+ - type: euclidean_pearson
3105
+ value: 75.31184890026394
3106
+ - type: euclidean_spearman
3107
+ value: 76.28984471300522
3108
+ - type: manhattan_pearson
3109
+ value: 75.36930361638623
3110
+ - type: manhattan_spearman
3111
+ value: 76.34021995551348
3112
+ - task:
3113
+ type: Reranking
3114
+ dataset:
3115
+ name: MTEB MMarcoReranking
3116
+ type: C-MTEB/Mmarco-reranking
3117
+ config: default
3118
+ split: dev
3119
+ revision: None
3120
+ metrics:
3121
+ - type: map
3122
+ value: 23.63666512532725
3123
+ - type: mrr
3124
+ value: 22.49642857142857
3125
+ - task:
3126
+ type: Retrieval
3127
+ dataset:
3128
+ name: MTEB MMarcoRetrieval
3129
+ type: C-MTEB/MMarcoRetrieval
3130
+ config: default
3131
+ split: dev
3132
+ revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
3133
+ metrics:
3134
+ - type: map_at_1
3135
+ value: 60.645
3136
+ - type: map_at_10
3137
+ value: 69.733
3138
+ - type: map_at_100
3139
+ value: 70.11699999999999
3140
+ - type: map_at_1000
3141
+ value: 70.135
3142
+ - type: map_at_3
3143
+ value: 67.585
3144
+ - type: map_at_5
3145
+ value: 68.904
3146
+ - type: mrr_at_1
3147
+ value: 62.765
3148
+ - type: mrr_at_10
3149
+ value: 70.428
3150
+ - type: mrr_at_100
3151
+ value: 70.77
3152
+ - type: mrr_at_1000
3153
+ value: 70.785
3154
+ - type: mrr_at_3
3155
+ value: 68.498
3156
+ - type: mrr_at_5
3157
+ value: 69.69
3158
+ - type: ndcg_at_1
3159
+ value: 62.765
3160
+ - type: ndcg_at_10
3161
+ value: 73.83
3162
+ - type: ndcg_at_100
3163
+ value: 75.593
3164
+ - type: ndcg_at_1000
3165
+ value: 76.05199999999999
3166
+ - type: ndcg_at_3
3167
+ value: 69.66499999999999
3168
+ - type: ndcg_at_5
3169
+ value: 71.929
3170
+ - type: precision_at_1
3171
+ value: 62.765
3172
+ - type: precision_at_10
3173
+ value: 9.117
3174
+ - type: precision_at_100
3175
+ value: 1.0
3176
+ - type: precision_at_1000
3177
+ value: 0.104
3178
+ - type: precision_at_3
3179
+ value: 26.323
3180
+ - type: precision_at_5
3181
+ value: 16.971
3182
+ - type: recall_at_1
3183
+ value: 60.645
3184
+ - type: recall_at_10
3185
+ value: 85.907
3186
+ - type: recall_at_100
3187
+ value: 93.947
3188
+ - type: recall_at_1000
3189
+ value: 97.531
3190
+ - type: recall_at_3
3191
+ value: 74.773
3192
+ - type: recall_at_5
3193
+ value: 80.16799999999999
3194
+ - task:
3195
+ type: Classification
3196
+ dataset:
3197
+ name: MTEB MassiveIntentClassification (zh-CN)
3198
+ type: mteb/amazon_massive_intent
3199
+ config: zh-CN
3200
+ split: test
3201
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
3202
+ metrics:
3203
+ - type: accuracy
3204
+ value: 76.25084061869536
3205
+ - type: f1
3206
+ value: 73.65064492827022
3207
+ - task:
3208
+ type: Classification
3209
+ dataset:
3210
+ name: MTEB MassiveScenarioClassification (zh-CN)
3211
+ type: mteb/amazon_massive_scenario
3212
+ config: zh-CN
3213
+ split: test
3214
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3215
+ metrics:
3216
+ - type: accuracy
3217
+ value: 77.2595830531271
3218
+ - type: f1
3219
+ value: 77.15217273559321
3220
+ - task:
3221
+ type: Retrieval
3222
+ dataset:
3223
+ name: MTEB MedicalRetrieval
3224
+ type: C-MTEB/MedicalRetrieval
3225
+ config: default
3226
+ split: dev
3227
+ revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
3228
+ metrics:
3229
+ - type: map_at_1
3230
+ value: 52.400000000000006
3231
+ - type: map_at_10
3232
+ value: 58.367000000000004
3233
+ - type: map_at_100
3234
+ value: 58.913000000000004
3235
+ - type: map_at_1000
3236
+ value: 58.961
3237
+ - type: map_at_3
3238
+ value: 56.882999999999996
3239
+ - type: map_at_5
3240
+ value: 57.743
3241
+ - type: mrr_at_1
3242
+ value: 52.400000000000006
3243
+ - type: mrr_at_10
3244
+ value: 58.367000000000004
3245
+ - type: mrr_at_100
3246
+ value: 58.913000000000004
3247
+ - type: mrr_at_1000
3248
+ value: 58.961
3249
+ - type: mrr_at_3
3250
+ value: 56.882999999999996
3251
+ - type: mrr_at_5
3252
+ value: 57.743
3253
+ - type: ndcg_at_1
3254
+ value: 52.400000000000006
3255
+ - type: ndcg_at_10
3256
+ value: 61.329
3257
+ - type: ndcg_at_100
3258
+ value: 64.264
3259
+ - type: ndcg_at_1000
3260
+ value: 65.669
3261
+ - type: ndcg_at_3
3262
+ value: 58.256
3263
+ - type: ndcg_at_5
3264
+ value: 59.813
3265
+ - type: precision_at_1
3266
+ value: 52.400000000000006
3267
+ - type: precision_at_10
3268
+ value: 7.07
3269
+ - type: precision_at_100
3270
+ value: 0.851
3271
+ - type: precision_at_1000
3272
+ value: 0.096
3273
+ - type: precision_at_3
3274
+ value: 20.732999999999997
3275
+ - type: precision_at_5
3276
+ value: 13.200000000000001
3277
+ - type: recall_at_1
3278
+ value: 52.400000000000006
3279
+ - type: recall_at_10
3280
+ value: 70.7
3281
+ - type: recall_at_100
3282
+ value: 85.1
3283
+ - type: recall_at_1000
3284
+ value: 96.39999999999999
3285
+ - type: recall_at_3
3286
+ value: 62.2
3287
+ - type: recall_at_5
3288
+ value: 66.0
3289
+ - task:
3290
+ type: Classification
3291
+ dataset:
3292
+ name: MTEB MultilingualSentiment
3293
+ type: C-MTEB/MultilingualSentiment-classification
3294
+ config: default
3295
+ split: validation
3296
+ revision: 46958b007a63fdbf239b7672c25d0bea67b5ea1a
3297
+ metrics:
3298
+ - type: accuracy
3299
+ value: 77.42333333333333
3300
+ - type: f1
3301
+ value: 77.24849313989888
3302
+ - task:
3303
+ type: PairClassification
3304
+ dataset:
3305
+ name: MTEB Ocnli
3306
+ type: C-MTEB/OCNLI
3307
+ config: default
3308
+ split: validation
3309
+ revision: 66e76a618a34d6d565d5538088562851e6daa7ec
3310
+ metrics:
3311
+ - type: cos_sim_accuracy
3312
+ value: 80.12994044396319
3313
+ - type: cos_sim_ap
3314
+ value: 85.21793541189636
3315
+ - type: cos_sim_f1
3316
+ value: 81.91489361702128
3317
+ - type: cos_sim_precision
3318
+ value: 75.55753791257806
3319
+ - type: cos_sim_recall
3320
+ value: 89.44033790918691
3321
+ - type: dot_accuracy
3322
+ value: 80.12994044396319
3323
+ - type: dot_ap
3324
+ value: 85.22568672443236
3325
+ - type: dot_f1
3326
+ value: 81.91489361702128
3327
+ - type: dot_precision
3328
+ value: 75.55753791257806
3329
+ - type: dot_recall
3330
+ value: 89.44033790918691
3331
+ - type: euclidean_accuracy
3332
+ value: 80.12994044396319
3333
+ - type: euclidean_ap
3334
+ value: 85.21643342357407
3335
+ - type: euclidean_f1
3336
+ value: 81.8830242510699
3337
+ - type: euclidean_precision
3338
+ value: 74.48096885813149
3339
+ - type: euclidean_recall
3340
+ value: 90.91869060190075
3341
+ - type: manhattan_accuracy
3342
+ value: 80.5630752571738
3343
+ - type: manhattan_ap
3344
+ value: 85.27682975032671
3345
+ - type: manhattan_f1
3346
+ value: 82.03883495145631
3347
+ - type: manhattan_precision
3348
+ value: 75.92093441150045
3349
+ - type: manhattan_recall
3350
+ value: 89.22914466737065
3351
+ - type: max_accuracy
3352
+ value: 80.5630752571738
3353
+ - type: max_ap
3354
+ value: 85.27682975032671
3355
+ - type: max_f1
3356
+ value: 82.03883495145631
3357
+ - task:
3358
+ type: Classification
3359
+ dataset:
3360
+ name: MTEB OnlineShopping
3361
+ type: C-MTEB/OnlineShopping-classification
3362
+ config: default
3363
+ split: test
3364
+ revision: e610f2ebd179a8fda30ae534c3878750a96db120
3365
+ metrics:
3366
+ - type: accuracy
3367
+ value: 94.47999999999999
3368
+ - type: ap
3369
+ value: 92.81177660844013
3370
+ - type: f1
3371
+ value: 94.47045470502114
3372
+ - task:
3373
+ type: STS
3374
+ dataset:
3375
+ name: MTEB PAWSX
3376
+ type: C-MTEB/PAWSX
3377
+ config: default
3378
+ split: test
3379
+ revision: 9c6a90e430ac22b5779fb019a23e820b11a8b5e1
3380
+ metrics:
3381
+ - type: cos_sim_pearson
3382
+ value: 46.13154582182421
3383
+ - type: cos_sim_spearman
3384
+ value: 50.21718723757444
3385
+ - type: euclidean_pearson
3386
+ value: 49.41535243569054
3387
+ - type: euclidean_spearman
3388
+ value: 50.21831909208907
3389
+ - type: manhattan_pearson
3390
+ value: 49.50756578601167
3391
+ - type: manhattan_spearman
3392
+ value: 50.229118655684566
3393
+ - task:
3394
+ type: STS
3395
+ dataset:
3396
+ name: MTEB QBQTC
3397
+ type: C-MTEB/QBQTC
3398
+ config: default
3399
+ split: test
3400
+ revision: 790b0510dc52b1553e8c49f3d2afb48c0e5c48b7
3401
+ metrics:
3402
+ - type: cos_sim_pearson
3403
+ value: 30.787794367421956
3404
+ - type: cos_sim_spearman
3405
+ value: 31.81774306987836
3406
+ - type: euclidean_pearson
3407
+ value: 29.809436608089495
3408
+ - type: euclidean_spearman
3409
+ value: 31.817379098812165
3410
+ - type: manhattan_pearson
3411
+ value: 30.377027186607787
3412
+ - type: manhattan_spearman
3413
+ value: 32.42286865176827
3414
+ - task:
3415
+ type: STS
3416
+ dataset:
3417
+ name: MTEB STS22 (zh)
3418
+ type: mteb/sts22-crosslingual-sts
3419
+ config: zh
3420
+ split: test
3421
+ revision: eea2b4fe26a775864c896887d910b76a8098ad3f
3422
+ metrics:
3423
+ - type: cos_sim_pearson
3424
+ value: 61.29839896616376
3425
+ - type: cos_sim_spearman
3426
+ value: 67.36328213286453
3427
+ - type: euclidean_pearson
3428
+ value: 64.33899267794008
3429
+ - type: euclidean_spearman
3430
+ value: 67.36552580196211
3431
+ - type: manhattan_pearson
3432
+ value: 65.20010308796022
3433
+ - type: manhattan_spearman
3434
+ value: 67.50982972902
3435
+ - task:
3436
+ type: STS
3437
+ dataset:
3438
+ name: MTEB STSB
3439
+ type: C-MTEB/STSB
3440
+ config: default
3441
+ split: test
3442
+ revision: 0cde68302b3541bb8b3c340dc0644b0b745b3dc0
3443
+ metrics:
3444
+ - type: cos_sim_pearson
3445
+ value: 81.23278996774297
3446
+ - type: cos_sim_spearman
3447
+ value: 81.369375466486
3448
+ - type: euclidean_pearson
3449
+ value: 79.91030863727944
3450
+ - type: euclidean_spearman
3451
+ value: 81.36824495466793
3452
+ - type: manhattan_pearson
3453
+ value: 79.88047052896854
3454
+ - type: manhattan_spearman
3455
+ value: 81.3369604332008
3456
+ - task:
3457
+ type: Reranking
3458
+ dataset:
3459
+ name: MTEB T2Reranking
3460
+ type: C-MTEB/T2Reranking
3461
+ config: default
3462
+ split: dev
3463
+ revision: 76631901a18387f85eaa53e5450019b87ad58ef9
3464
+ metrics:
3465
+ - type: map
3466
+ value: 68.109205221286
3467
+ - type: mrr
3468
+ value: 78.40703619520477
3469
+ - task:
3470
+ type: Retrieval
3471
+ dataset:
3472
+ name: MTEB T2Retrieval
3473
+ type: C-MTEB/T2Retrieval
3474
+ config: default
3475
+ split: dev
3476
+ revision: 8731a845f1bf500a4f111cf1070785c793d10e64
3477
+ metrics:
3478
+ - type: map_at_1
3479
+ value: 26.704
3480
+ - type: map_at_10
3481
+ value: 75.739
3482
+ - type: map_at_100
3483
+ value: 79.606
3484
+ - type: map_at_1000
3485
+ value: 79.666
3486
+ - type: map_at_3
3487
+ value: 52.803
3488
+ - type: map_at_5
3489
+ value: 65.068
3490
+ - type: mrr_at_1
3491
+ value: 88.48899999999999
3492
+ - type: mrr_at_10
3493
+ value: 91.377
3494
+ - type: mrr_at_100
3495
+ value: 91.474
3496
+ - type: mrr_at_1000
3497
+ value: 91.47800000000001
3498
+ - type: mrr_at_3
3499
+ value: 90.846
3500
+ - type: mrr_at_5
3501
+ value: 91.18
3502
+ - type: ndcg_at_1
3503
+ value: 88.48899999999999
3504
+ - type: ndcg_at_10
3505
+ value: 83.581
3506
+ - type: ndcg_at_100
3507
+ value: 87.502
3508
+ - type: ndcg_at_1000
3509
+ value: 88.1
3510
+ - type: ndcg_at_3
3511
+ value: 84.433
3512
+ - type: ndcg_at_5
3513
+ value: 83.174
3514
+ - type: precision_at_1
3515
+ value: 88.48899999999999
3516
+ - type: precision_at_10
3517
+ value: 41.857
3518
+ - type: precision_at_100
3519
+ value: 5.039
3520
+ - type: precision_at_1000
3521
+ value: 0.517
3522
+ - type: precision_at_3
3523
+ value: 73.938
3524
+ - type: precision_at_5
3525
+ value: 62.163000000000004
3526
+ - type: recall_at_1
3527
+ value: 26.704
3528
+ - type: recall_at_10
3529
+ value: 83.092
3530
+ - type: recall_at_100
3531
+ value: 95.659
3532
+ - type: recall_at_1000
3533
+ value: 98.779
3534
+ - type: recall_at_3
3535
+ value: 54.678000000000004
3536
+ - type: recall_at_5
3537
+ value: 68.843
3538
+ - task:
3539
+ type: Classification
3540
+ dataset:
3541
+ name: MTEB TNews
3542
+ type: C-MTEB/TNews-classification
3543
+ config: default
3544
+ split: validation
3545
+ revision: 317f262bf1e6126357bbe89e875451e4b0938fe4
3546
+ metrics:
3547
+ - type: accuracy
3548
+ value: 51.235
3549
+ - type: f1
3550
+ value: 48.14373844331604
3551
+ - task:
3552
+ type: Clustering
3553
+ dataset:
3554
+ name: MTEB ThuNewsClusteringP2P
3555
+ type: C-MTEB/ThuNewsClusteringP2P
3556
+ config: default
3557
+ split: test
3558
+ revision: 5798586b105c0434e4f0fe5e767abe619442cf93
3559
+ metrics:
3560
+ - type: v_measure
3561
+ value: 87.42930040493792
3562
+ - task:
3563
+ type: Clustering
3564
+ dataset:
3565
+ name: MTEB ThuNewsClusteringS2S
3566
+ type: C-MTEB/ThuNewsClusteringS2S
3567
+ config: default
3568
+ split: test
3569
+ revision: 8a8b2caeda43f39e13c4bc5bea0f8a667896e10d
3570
+ metrics:
3571
+ - type: v_measure
3572
+ value: 87.90254094650042
3573
+ - task:
3574
+ type: Retrieval
3575
+ dataset:
3576
+ name: MTEB VideoRetrieval
3577
+ type: C-MTEB/VideoRetrieval
3578
+ config: default
3579
+ split: dev
3580
+ revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
3581
+ metrics:
3582
+ - type: map_at_1
3583
+ value: 54.900000000000006
3584
+ - type: map_at_10
3585
+ value: 64.92
3586
+ - type: map_at_100
3587
+ value: 65.424
3588
+ - type: map_at_1000
3589
+ value: 65.43900000000001
3590
+ - type: map_at_3
3591
+ value: 63.132999999999996
3592
+ - type: map_at_5
3593
+ value: 64.208
3594
+ - type: mrr_at_1
3595
+ value: 54.900000000000006
3596
+ - type: mrr_at_10
3597
+ value: 64.92
3598
+ - type: mrr_at_100
3599
+ value: 65.424
3600
+ - type: mrr_at_1000
3601
+ value: 65.43900000000001
3602
+ - type: mrr_at_3
3603
+ value: 63.132999999999996
3604
+ - type: mrr_at_5
3605
+ value: 64.208
3606
+ - type: ndcg_at_1
3607
+ value: 54.900000000000006
3608
+ - type: ndcg_at_10
3609
+ value: 69.41199999999999
3610
+ - type: ndcg_at_100
3611
+ value: 71.824
3612
+ - type: ndcg_at_1000
3613
+ value: 72.301
3614
+ - type: ndcg_at_3
3615
+ value: 65.79700000000001
3616
+ - type: ndcg_at_5
3617
+ value: 67.713
3618
+ - type: precision_at_1
3619
+ value: 54.900000000000006
3620
+ - type: precision_at_10
3621
+ value: 8.33
3622
+ - type: precision_at_100
3623
+ value: 0.9450000000000001
3624
+ - type: precision_at_1000
3625
+ value: 0.098
3626
+ - type: precision_at_3
3627
+ value: 24.5
3628
+ - type: precision_at_5
3629
+ value: 15.620000000000001
3630
+ - type: recall_at_1
3631
+ value: 54.900000000000006
3632
+ - type: recall_at_10
3633
+ value: 83.3
3634
+ - type: recall_at_100
3635
+ value: 94.5
3636
+ - type: recall_at_1000
3637
+ value: 98.4
3638
+ - type: recall_at_3
3639
+ value: 73.5
3640
+ - type: recall_at_5
3641
+ value: 78.10000000000001
3642
+ - task:
3643
+ type: Classification
3644
+ dataset:
3645
+ name: MTEB Waimai
3646
+ type: C-MTEB/waimai-classification
3647
+ config: default
3648
+ split: test
3649
+ revision: 339287def212450dcaa9df8c22bf93e9980c7023
3650
+ metrics:
3651
+ - type: accuracy
3652
+ value: 88.63
3653
+ - type: ap
3654
+ value: 73.78658340897097
3655
+ - type: f1
3656
+ value: 87.16764294033919
3657
+ ---
3658
+
3659
+ # agier9/gte-Qwen1.5-7B-instruct-Q5_K_M-GGUF
3660
+ This model was converted to GGUF format from [`Alibaba-NLP/gte-Qwen1.5-7B-instruct`](https://huggingface.co/Alibaba-NLP/gte-Qwen1.5-7B-instruct) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
3661
+ Refer to the [original model card](https://huggingface.co/Alibaba-NLP/gte-Qwen1.5-7B-instruct) for more details on the model.
3662
+ ## Use with llama.cpp
3663
+ Install llama.cpp through brew.
3664
+ ```bash
3665
+ brew install ggerganov/ggerganov/llama.cpp
3666
+ ```
3667
+ Invoke the llama.cpp server or the CLI.
3668
+ CLI:
3669
+ ```bash
3670
+ llama-cli --hf-repo agier9/gte-Qwen1.5-7B-instruct-Q5_K_M-GGUF --model gte-qwen1.5-7b-instruct-q5_k_m.gguf -p "The meaning to life and the universe is"
3671
+ ```
3672
+ Server:
3673
+ ```bash
3674
+ llama-server --hf-repo agier9/gte-Qwen1.5-7B-instruct-Q5_K_M-GGUF --model gte-qwen1.5-7b-instruct-q5_k_m.gguf -c 2048
3675
+ ```
3676
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
3677
+ ```
3678
+ git clone https://github.com/ggerganov/llama.cpp && \
3679
+ cd llama.cpp && \
3680
+ make && \
3681
+ ./main -m gte-qwen1.5-7b-instruct-q5_k_m.gguf -n 128
3682
+ ```