Spaces:
Build error
Build error
| tags: | |
| - mteb | |
| model-index: | |
| - name: multilingual-e5-base | |
| results: | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_counterfactual | |
| name: MTEB AmazonCounterfactualClassification (en) | |
| config: en | |
| split: test | |
| revision: e8379541af4e31359cca9fbcf4b00f2671dba205 | |
| metrics: | |
| - type: accuracy | |
| value: 78.97014925373135 | |
| - type: ap | |
| value: 43.69351129103008 | |
| - type: f1 | |
| value: 73.38075030070492 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_counterfactual | |
| name: MTEB AmazonCounterfactualClassification (de) | |
| config: de | |
| split: test | |
| revision: e8379541af4e31359cca9fbcf4b00f2671dba205 | |
| metrics: | |
| - type: accuracy | |
| value: 71.7237687366167 | |
| - type: ap | |
| value: 82.22089859962671 | |
| - type: f1 | |
| value: 69.95532758884401 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_counterfactual | |
| name: MTEB AmazonCounterfactualClassification (en-ext) | |
| config: en-ext | |
| split: test | |
| revision: e8379541af4e31359cca9fbcf4b00f2671dba205 | |
| metrics: | |
| - type: accuracy | |
| value: 79.65517241379312 | |
| - type: ap | |
| value: 28.507918657094738 | |
| - type: f1 | |
| value: 66.84516013726119 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_counterfactual | |
| name: MTEB AmazonCounterfactualClassification (ja) | |
| config: ja | |
| split: test | |
| revision: e8379541af4e31359cca9fbcf4b00f2671dba205 | |
| metrics: | |
| - type: accuracy | |
| value: 73.32976445396146 | |
| - type: ap | |
| value: 20.720481637566014 | |
| - type: f1 | |
| value: 59.78002763416003 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_polarity | |
| name: MTEB AmazonPolarityClassification | |
| config: default | |
| split: test | |
| revision: e2d317d38cd51312af73b3d32a06d1a08b442046 | |
| metrics: | |
| - type: accuracy | |
| value: 90.63775 | |
| - type: ap | |
| value: 87.22277903861716 | |
| - type: f1 | |
| value: 90.60378636386807 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_reviews_multi | |
| name: MTEB AmazonReviewsClassification (en) | |
| config: en | |
| split: test | |
| revision: 1399c76144fd37290681b995c656ef9b2e06e26d | |
| metrics: | |
| - type: accuracy | |
| value: 44.546 | |
| - type: f1 | |
| value: 44.05666638370923 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_reviews_multi | |
| name: MTEB AmazonReviewsClassification (de) | |
| config: de | |
| split: test | |
| revision: 1399c76144fd37290681b995c656ef9b2e06e26d | |
| metrics: | |
| - type: accuracy | |
| value: 41.828 | |
| - type: f1 | |
| value: 41.2710255644252 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_reviews_multi | |
| name: MTEB AmazonReviewsClassification (es) | |
| config: es | |
| split: test | |
| revision: 1399c76144fd37290681b995c656ef9b2e06e26d | |
| metrics: | |
| - type: accuracy | |
| value: 40.534 | |
| - type: f1 | |
| value: 39.820743174270326 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_reviews_multi | |
| name: MTEB AmazonReviewsClassification (fr) | |
| config: fr | |
| split: test | |
| revision: 1399c76144fd37290681b995c656ef9b2e06e26d | |
| metrics: | |
| - type: accuracy | |
| value: 39.684 | |
| - type: f1 | |
| value: 39.11052682815307 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_reviews_multi | |
| name: MTEB AmazonReviewsClassification (ja) | |
| config: ja | |
| split: test | |
| revision: 1399c76144fd37290681b995c656ef9b2e06e26d | |
| metrics: | |
| - type: accuracy | |
| value: 37.436 | |
| - type: f1 | |
| value: 37.07082931930871 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_reviews_multi | |
| name: MTEB AmazonReviewsClassification (zh) | |
| config: zh | |
| split: test | |
| revision: 1399c76144fd37290681b995c656ef9b2e06e26d | |
| metrics: | |
| - type: accuracy | |
| value: 37.226000000000006 | |
| - type: f1 | |
| value: 36.65372077739185 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: arguana | |
| name: MTEB ArguAna | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 22.831000000000003 | |
| - type: map_at_10 | |
| value: 36.42 | |
| - type: map_at_100 | |
| value: 37.699 | |
| - type: map_at_1000 | |
| value: 37.724000000000004 | |
| - type: map_at_3 | |
| value: 32.207 | |
| - type: map_at_5 | |
| value: 34.312 | |
| - type: mrr_at_1 | |
| value: 23.257 | |
| - type: mrr_at_10 | |
| value: 36.574 | |
| - type: mrr_at_100 | |
| value: 37.854 | |
| - type: mrr_at_1000 | |
| value: 37.878 | |
| - type: mrr_at_3 | |
| value: 32.385000000000005 | |
| - type: mrr_at_5 | |
| value: 34.48 | |
| - type: ndcg_at_1 | |
| value: 22.831000000000003 | |
| - type: ndcg_at_10 | |
| value: 44.230000000000004 | |
| - type: ndcg_at_100 | |
| value: 49.974000000000004 | |
| - type: ndcg_at_1000 | |
| value: 50.522999999999996 | |
| - type: ndcg_at_3 | |
| value: 35.363 | |
| - type: ndcg_at_5 | |
| value: 39.164 | |
| - type: precision_at_1 | |
| value: 22.831000000000003 | |
| - type: precision_at_10 | |
| value: 6.935 | |
| - type: precision_at_100 | |
| value: 0.9520000000000001 | |
| - type: precision_at_1000 | |
| value: 0.099 | |
| - type: precision_at_3 | |
| value: 14.841 | |
| - type: precision_at_5 | |
| value: 10.754 | |
| - type: recall_at_1 | |
| value: 22.831000000000003 | |
| - type: recall_at_10 | |
| value: 69.346 | |
| - type: recall_at_100 | |
| value: 95.235 | |
| - type: recall_at_1000 | |
| value: 99.36 | |
| - type: recall_at_3 | |
| value: 44.523 | |
| - type: recall_at_5 | |
| value: 53.769999999999996 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/arxiv-clustering-p2p | |
| name: MTEB ArxivClusteringP2P | |
| config: default | |
| split: test | |
| revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d | |
| metrics: | |
| - type: v_measure | |
| value: 40.27789869854063 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/arxiv-clustering-s2s | |
| name: MTEB ArxivClusteringS2S | |
| config: default | |
| split: test | |
| revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 | |
| metrics: | |
| - type: v_measure | |
| value: 35.41979463347428 | |
| - task: | |
| type: Reranking | |
| dataset: | |
| type: mteb/askubuntudupquestions-reranking | |
| name: MTEB AskUbuntuDupQuestions | |
| config: default | |
| split: test | |
| revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 | |
| metrics: | |
| - type: map | |
| value: 58.22752045109304 | |
| - type: mrr | |
| value: 71.51112430198303 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/biosses-sts | |
| name: MTEB BIOSSES | |
| config: default | |
| split: test | |
| revision: d3fb88f8f02e40887cd149695127462bbcf29b4a | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 84.71147646622866 | |
| - type: cos_sim_spearman | |
| value: 85.059167046486 | |
| - type: euclidean_pearson | |
| value: 75.88421613600647 | |
| - type: euclidean_spearman | |
| value: 75.12821787150585 | |
| - type: manhattan_pearson | |
| value: 75.22005646957604 | |
| - type: manhattan_spearman | |
| value: 74.42880434453272 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/bucc-bitext-mining | |
| name: MTEB BUCC (de-en) | |
| config: de-en | |
| split: test | |
| revision: d51519689f32196a32af33b075a01d0e7c51e252 | |
| metrics: | |
| - type: accuracy | |
| value: 99.23799582463465 | |
| - type: f1 | |
| value: 99.12665274878218 | |
| - type: precision | |
| value: 99.07098121085595 | |
| - type: recall | |
| value: 99.23799582463465 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/bucc-bitext-mining | |
| name: MTEB BUCC (fr-en) | |
| config: fr-en | |
| split: test | |
| revision: d51519689f32196a32af33b075a01d0e7c51e252 | |
| metrics: | |
| - type: accuracy | |
| value: 97.88685890380806 | |
| - type: f1 | |
| value: 97.59336708489249 | |
| - type: precision | |
| value: 97.44662117543473 | |
| - type: recall | |
| value: 97.88685890380806 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/bucc-bitext-mining | |
| name: MTEB BUCC (ru-en) | |
| config: ru-en | |
| split: test | |
| revision: d51519689f32196a32af33b075a01d0e7c51e252 | |
| metrics: | |
| - type: accuracy | |
| value: 97.47142362313821 | |
| - type: f1 | |
| value: 97.1989377670015 | |
| - type: precision | |
| value: 97.06384944001847 | |
| - type: recall | |
| value: 97.47142362313821 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/bucc-bitext-mining | |
| name: MTEB BUCC (zh-en) | |
| config: zh-en | |
| split: test | |
| revision: d51519689f32196a32af33b075a01d0e7c51e252 | |
| metrics: | |
| - type: accuracy | |
| value: 98.4728804634018 | |
| - type: f1 | |
| value: 98.2973494821836 | |
| - type: precision | |
| value: 98.2095839915745 | |
| - type: recall | |
| value: 98.4728804634018 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/banking77 | |
| name: MTEB Banking77Classification | |
| config: default | |
| split: test | |
| revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 | |
| metrics: | |
| - type: accuracy | |
| value: 82.74025974025975 | |
| - type: f1 | |
| value: 82.67420447730439 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/biorxiv-clustering-p2p | |
| name: MTEB BiorxivClusteringP2P | |
| config: default | |
| split: test | |
| revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 | |
| metrics: | |
| - type: v_measure | |
| value: 35.0380848063507 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/biorxiv-clustering-s2s | |
| name: MTEB BiorxivClusteringS2S | |
| config: default | |
| split: test | |
| revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 | |
| metrics: | |
| - type: v_measure | |
| value: 29.45956405670166 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackAndroidRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 32.122 | |
| - type: map_at_10 | |
| value: 42.03 | |
| - type: map_at_100 | |
| value: 43.364000000000004 | |
| - type: map_at_1000 | |
| value: 43.474000000000004 | |
| - type: map_at_3 | |
| value: 38.804 | |
| - type: map_at_5 | |
| value: 40.585 | |
| - type: mrr_at_1 | |
| value: 39.914 | |
| - type: mrr_at_10 | |
| value: 48.227 | |
| - type: mrr_at_100 | |
| value: 49.018 | |
| - type: mrr_at_1000 | |
| value: 49.064 | |
| - type: mrr_at_3 | |
| value: 45.994 | |
| - type: mrr_at_5 | |
| value: 47.396 | |
| - type: ndcg_at_1 | |
| value: 39.914 | |
| - type: ndcg_at_10 | |
| value: 47.825 | |
| - type: ndcg_at_100 | |
| value: 52.852 | |
| - type: ndcg_at_1000 | |
| value: 54.891 | |
| - type: ndcg_at_3 | |
| value: 43.517 | |
| - type: ndcg_at_5 | |
| value: 45.493 | |
| - type: precision_at_1 | |
| value: 39.914 | |
| - type: precision_at_10 | |
| value: 8.956 | |
| - type: precision_at_100 | |
| value: 1.388 | |
| - type: precision_at_1000 | |
| value: 0.182 | |
| - type: precision_at_3 | |
| value: 20.791999999999998 | |
| - type: precision_at_5 | |
| value: 14.821000000000002 | |
| - type: recall_at_1 | |
| value: 32.122 | |
| - type: recall_at_10 | |
| value: 58.294999999999995 | |
| - type: recall_at_100 | |
| value: 79.726 | |
| - type: recall_at_1000 | |
| value: 93.099 | |
| - type: recall_at_3 | |
| value: 45.017 | |
| - type: recall_at_5 | |
| value: 51.002 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackEnglishRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 29.677999999999997 | |
| - type: map_at_10 | |
| value: 38.684000000000005 | |
| - type: map_at_100 | |
| value: 39.812999999999995 | |
| - type: map_at_1000 | |
| value: 39.945 | |
| - type: map_at_3 | |
| value: 35.831 | |
| - type: map_at_5 | |
| value: 37.446 | |
| - type: mrr_at_1 | |
| value: 37.771 | |
| - type: mrr_at_10 | |
| value: 44.936 | |
| - type: mrr_at_100 | |
| value: 45.583 | |
| - type: mrr_at_1000 | |
| value: 45.634 | |
| - type: mrr_at_3 | |
| value: 42.771 | |
| - type: mrr_at_5 | |
| value: 43.994 | |
| - type: ndcg_at_1 | |
| value: 37.771 | |
| - type: ndcg_at_10 | |
| value: 44.059 | |
| - type: ndcg_at_100 | |
| value: 48.192 | |
| - type: ndcg_at_1000 | |
| value: 50.375 | |
| - type: ndcg_at_3 | |
| value: 40.172000000000004 | |
| - type: ndcg_at_5 | |
| value: 41.899 | |
| - type: precision_at_1 | |
| value: 37.771 | |
| - type: precision_at_10 | |
| value: 8.286999999999999 | |
| - type: precision_at_100 | |
| value: 1.322 | |
| - type: precision_at_1000 | |
| value: 0.178 | |
| - type: precision_at_3 | |
| value: 19.406000000000002 | |
| - type: precision_at_5 | |
| value: 13.745 | |
| - type: recall_at_1 | |
| value: 29.677999999999997 | |
| - type: recall_at_10 | |
| value: 53.071 | |
| - type: recall_at_100 | |
| value: 70.812 | |
| - type: recall_at_1000 | |
| value: 84.841 | |
| - type: recall_at_3 | |
| value: 41.016000000000005 | |
| - type: recall_at_5 | |
| value: 46.22 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackGamingRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 42.675000000000004 | |
| - type: map_at_10 | |
| value: 53.93599999999999 | |
| - type: map_at_100 | |
| value: 54.806999999999995 | |
| - type: map_at_1000 | |
| value: 54.867 | |
| - type: map_at_3 | |
| value: 50.934000000000005 | |
| - type: map_at_5 | |
| value: 52.583 | |
| - type: mrr_at_1 | |
| value: 48.339 | |
| - type: mrr_at_10 | |
| value: 57.265 | |
| - type: mrr_at_100 | |
| value: 57.873 | |
| - type: mrr_at_1000 | |
| value: 57.906 | |
| - type: mrr_at_3 | |
| value: 55.193000000000005 | |
| - type: mrr_at_5 | |
| value: 56.303000000000004 | |
| - type: ndcg_at_1 | |
| value: 48.339 | |
| - type: ndcg_at_10 | |
| value: 59.19799999999999 | |
| - type: ndcg_at_100 | |
| value: 62.743 | |
| - type: ndcg_at_1000 | |
| value: 63.99399999999999 | |
| - type: ndcg_at_3 | |
| value: 54.367 | |
| - type: ndcg_at_5 | |
| value: 56.548 | |
| - type: precision_at_1 | |
| value: 48.339 | |
| - type: precision_at_10 | |
| value: 9.216000000000001 | |
| - type: precision_at_100 | |
| value: 1.1809999999999998 | |
| - type: precision_at_1000 | |
| value: 0.134 | |
| - type: precision_at_3 | |
| value: 23.72 | |
| - type: precision_at_5 | |
| value: 16.025 | |
| - type: recall_at_1 | |
| value: 42.675000000000004 | |
| - type: recall_at_10 | |
| value: 71.437 | |
| - type: recall_at_100 | |
| value: 86.803 | |
| - type: recall_at_1000 | |
| value: 95.581 | |
| - type: recall_at_3 | |
| value: 58.434 | |
| - type: recall_at_5 | |
| value: 63.754 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackGisRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 23.518 | |
| - type: map_at_10 | |
| value: 30.648999999999997 | |
| - type: map_at_100 | |
| value: 31.508999999999997 | |
| - type: map_at_1000 | |
| value: 31.604 | |
| - type: map_at_3 | |
| value: 28.247 | |
| - type: map_at_5 | |
| value: 29.65 | |
| - type: mrr_at_1 | |
| value: 25.650000000000002 | |
| - type: mrr_at_10 | |
| value: 32.771 | |
| - type: mrr_at_100 | |
| value: 33.554 | |
| - type: mrr_at_1000 | |
| value: 33.629999999999995 | |
| - type: mrr_at_3 | |
| value: 30.433 | |
| - type: mrr_at_5 | |
| value: 31.812 | |
| - type: ndcg_at_1 | |
| value: 25.650000000000002 | |
| - type: ndcg_at_10 | |
| value: 34.929 | |
| - type: ndcg_at_100 | |
| value: 39.382 | |
| - type: ndcg_at_1000 | |
| value: 41.913 | |
| - type: ndcg_at_3 | |
| value: 30.292 | |
| - type: ndcg_at_5 | |
| value: 32.629999999999995 | |
| - type: precision_at_1 | |
| value: 25.650000000000002 | |
| - type: precision_at_10 | |
| value: 5.311 | |
| - type: precision_at_100 | |
| value: 0.792 | |
| - type: precision_at_1000 | |
| value: 0.105 | |
| - type: precision_at_3 | |
| value: 12.58 | |
| - type: precision_at_5 | |
| value: 8.994 | |
| - type: recall_at_1 | |
| value: 23.518 | |
| - type: recall_at_10 | |
| value: 46.19 | |
| - type: recall_at_100 | |
| value: 67.123 | |
| - type: recall_at_1000 | |
| value: 86.442 | |
| - type: recall_at_3 | |
| value: 33.678000000000004 | |
| - type: recall_at_5 | |
| value: 39.244 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackMathematicaRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 15.891 | |
| - type: map_at_10 | |
| value: 22.464000000000002 | |
| - type: map_at_100 | |
| value: 23.483 | |
| - type: map_at_1000 | |
| value: 23.613 | |
| - type: map_at_3 | |
| value: 20.080000000000002 | |
| - type: map_at_5 | |
| value: 21.526 | |
| - type: mrr_at_1 | |
| value: 20.025000000000002 | |
| - type: mrr_at_10 | |
| value: 26.712999999999997 | |
| - type: mrr_at_100 | |
| value: 27.650000000000002 | |
| - type: mrr_at_1000 | |
| value: 27.737000000000002 | |
| - type: mrr_at_3 | |
| value: 24.274 | |
| - type: mrr_at_5 | |
| value: 25.711000000000002 | |
| - type: ndcg_at_1 | |
| value: 20.025000000000002 | |
| - type: ndcg_at_10 | |
| value: 27.028999999999996 | |
| - type: ndcg_at_100 | |
| value: 32.064 | |
| - type: ndcg_at_1000 | |
| value: 35.188 | |
| - type: ndcg_at_3 | |
| value: 22.512999999999998 | |
| - type: ndcg_at_5 | |
| value: 24.89 | |
| - type: precision_at_1 | |
| value: 20.025000000000002 | |
| - type: precision_at_10 | |
| value: 4.776 | |
| - type: precision_at_100 | |
| value: 0.8500000000000001 | |
| - type: precision_at_1000 | |
| value: 0.125 | |
| - type: precision_at_3 | |
| value: 10.531 | |
| - type: precision_at_5 | |
| value: 7.811 | |
| - type: recall_at_1 | |
| value: 15.891 | |
| - type: recall_at_10 | |
| value: 37.261 | |
| - type: recall_at_100 | |
| value: 59.12 | |
| - type: recall_at_1000 | |
| value: 81.356 | |
| - type: recall_at_3 | |
| value: 24.741 | |
| - type: recall_at_5 | |
| value: 30.753999999999998 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackPhysicsRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 27.544 | |
| - type: map_at_10 | |
| value: 36.283 | |
| - type: map_at_100 | |
| value: 37.467 | |
| - type: map_at_1000 | |
| value: 37.574000000000005 | |
| - type: map_at_3 | |
| value: 33.528999999999996 | |
| - type: map_at_5 | |
| value: 35.028999999999996 | |
| - type: mrr_at_1 | |
| value: 34.166999999999994 | |
| - type: mrr_at_10 | |
| value: 41.866 | |
| - type: mrr_at_100 | |
| value: 42.666 | |
| - type: mrr_at_1000 | |
| value: 42.716 | |
| - type: mrr_at_3 | |
| value: 39.541 | |
| - type: mrr_at_5 | |
| value: 40.768 | |
| - type: ndcg_at_1 | |
| value: 34.166999999999994 | |
| - type: ndcg_at_10 | |
| value: 41.577 | |
| - type: ndcg_at_100 | |
| value: 46.687 | |
| - type: ndcg_at_1000 | |
| value: 48.967 | |
| - type: ndcg_at_3 | |
| value: 37.177 | |
| - type: ndcg_at_5 | |
| value: 39.097 | |
| - type: precision_at_1 | |
| value: 34.166999999999994 | |
| - type: precision_at_10 | |
| value: 7.420999999999999 | |
| - type: precision_at_100 | |
| value: 1.165 | |
| - type: precision_at_1000 | |
| value: 0.154 | |
| - type: precision_at_3 | |
| value: 17.291999999999998 | |
| - type: precision_at_5 | |
| value: 12.166 | |
| - type: recall_at_1 | |
| value: 27.544 | |
| - type: recall_at_10 | |
| value: 51.99399999999999 | |
| - type: recall_at_100 | |
| value: 73.738 | |
| - type: recall_at_1000 | |
| value: 89.33 | |
| - type: recall_at_3 | |
| value: 39.179 | |
| - type: recall_at_5 | |
| value: 44.385999999999996 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackProgrammersRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 26.661 | |
| - type: map_at_10 | |
| value: 35.475 | |
| - type: map_at_100 | |
| value: 36.626999999999995 | |
| - type: map_at_1000 | |
| value: 36.741 | |
| - type: map_at_3 | |
| value: 32.818000000000005 | |
| - type: map_at_5 | |
| value: 34.397 | |
| - type: mrr_at_1 | |
| value: 32.647999999999996 | |
| - type: mrr_at_10 | |
| value: 40.784 | |
| - type: mrr_at_100 | |
| value: 41.602 | |
| - type: mrr_at_1000 | |
| value: 41.661 | |
| - type: mrr_at_3 | |
| value: 38.68 | |
| - type: mrr_at_5 | |
| value: 39.838 | |
| - type: ndcg_at_1 | |
| value: 32.647999999999996 | |
| - type: ndcg_at_10 | |
| value: 40.697 | |
| - type: ndcg_at_100 | |
| value: 45.799 | |
| - type: ndcg_at_1000 | |
| value: 48.235 | |
| - type: ndcg_at_3 | |
| value: 36.516 | |
| - type: ndcg_at_5 | |
| value: 38.515 | |
| - type: precision_at_1 | |
| value: 32.647999999999996 | |
| - type: precision_at_10 | |
| value: 7.202999999999999 | |
| - type: precision_at_100 | |
| value: 1.1360000000000001 | |
| - type: precision_at_1000 | |
| value: 0.151 | |
| - type: precision_at_3 | |
| value: 17.314 | |
| - type: precision_at_5 | |
| value: 12.145999999999999 | |
| - type: recall_at_1 | |
| value: 26.661 | |
| - type: recall_at_10 | |
| value: 50.995000000000005 | |
| - type: recall_at_100 | |
| value: 73.065 | |
| - type: recall_at_1000 | |
| value: 89.781 | |
| - type: recall_at_3 | |
| value: 39.073 | |
| - type: recall_at_5 | |
| value: 44.395 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 25.946583333333333 | |
| - type: map_at_10 | |
| value: 33.79725 | |
| - type: map_at_100 | |
| value: 34.86408333333333 | |
| - type: map_at_1000 | |
| value: 34.9795 | |
| - type: map_at_3 | |
| value: 31.259999999999998 | |
| - type: map_at_5 | |
| value: 32.71541666666666 | |
| - type: mrr_at_1 | |
| value: 30.863749999999996 | |
| - type: mrr_at_10 | |
| value: 37.99183333333333 | |
| - type: mrr_at_100 | |
| value: 38.790499999999994 | |
| - type: mrr_at_1000 | |
| value: 38.85575000000001 | |
| - type: mrr_at_3 | |
| value: 35.82083333333333 | |
| - type: mrr_at_5 | |
| value: 37.07533333333333 | |
| - type: ndcg_at_1 | |
| value: 30.863749999999996 | |
| - type: ndcg_at_10 | |
| value: 38.52141666666667 | |
| - type: ndcg_at_100 | |
| value: 43.17966666666667 | |
| - type: ndcg_at_1000 | |
| value: 45.64608333333333 | |
| - type: ndcg_at_3 | |
| value: 34.333000000000006 | |
| - type: ndcg_at_5 | |
| value: 36.34975 | |
| - type: precision_at_1 | |
| value: 30.863749999999996 | |
| - type: precision_at_10 | |
| value: 6.598999999999999 | |
| - type: precision_at_100 | |
| value: 1.0502500000000001 | |
| - type: precision_at_1000 | |
| value: 0.14400000000000002 | |
| - type: precision_at_3 | |
| value: 15.557583333333334 | |
| - type: precision_at_5 | |
| value: 11.020000000000001 | |
| - type: recall_at_1 | |
| value: 25.946583333333333 | |
| - type: recall_at_10 | |
| value: 48.36991666666666 | |
| - type: recall_at_100 | |
| value: 69.02408333333334 | |
| - type: recall_at_1000 | |
| value: 86.43858333333331 | |
| - type: recall_at_3 | |
| value: 36.4965 | |
| - type: recall_at_5 | |
| value: 41.76258333333334 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackStatsRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 22.431 | |
| - type: map_at_10 | |
| value: 28.889 | |
| - type: map_at_100 | |
| value: 29.642000000000003 | |
| - type: map_at_1000 | |
| value: 29.742 | |
| - type: map_at_3 | |
| value: 26.998 | |
| - type: map_at_5 | |
| value: 28.172000000000004 | |
| - type: mrr_at_1 | |
| value: 25.307000000000002 | |
| - type: mrr_at_10 | |
| value: 31.763 | |
| - type: mrr_at_100 | |
| value: 32.443 | |
| - type: mrr_at_1000 | |
| value: 32.531 | |
| - type: mrr_at_3 | |
| value: 29.959000000000003 | |
| - type: mrr_at_5 | |
| value: 31.063000000000002 | |
| - type: ndcg_at_1 | |
| value: 25.307000000000002 | |
| - type: ndcg_at_10 | |
| value: 32.586999999999996 | |
| - type: ndcg_at_100 | |
| value: 36.5 | |
| - type: ndcg_at_1000 | |
| value: 39.133 | |
| - type: ndcg_at_3 | |
| value: 29.25 | |
| - type: ndcg_at_5 | |
| value: 31.023 | |
| - type: precision_at_1 | |
| value: 25.307000000000002 | |
| - type: precision_at_10 | |
| value: 4.954 | |
| - type: precision_at_100 | |
| value: 0.747 | |
| - type: precision_at_1000 | |
| value: 0.104 | |
| - type: precision_at_3 | |
| value: 12.577 | |
| - type: precision_at_5 | |
| value: 8.741999999999999 | |
| - type: recall_at_1 | |
| value: 22.431 | |
| - type: recall_at_10 | |
| value: 41.134 | |
| - type: recall_at_100 | |
| value: 59.28600000000001 | |
| - type: recall_at_1000 | |
| value: 78.857 | |
| - type: recall_at_3 | |
| value: 31.926 | |
| - type: recall_at_5 | |
| value: 36.335 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackTexRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 17.586 | |
| - type: map_at_10 | |
| value: 23.304 | |
| - type: map_at_100 | |
| value: 24.159 | |
| - type: map_at_1000 | |
| value: 24.281 | |
| - type: map_at_3 | |
| value: 21.316 | |
| - type: map_at_5 | |
| value: 22.383 | |
| - type: mrr_at_1 | |
| value: 21.645 | |
| - type: mrr_at_10 | |
| value: 27.365000000000002 | |
| - type: mrr_at_100 | |
| value: 28.108 | |
| - type: mrr_at_1000 | |
| value: 28.192 | |
| - type: mrr_at_3 | |
| value: 25.482 | |
| - type: mrr_at_5 | |
| value: 26.479999999999997 | |
| - type: ndcg_at_1 | |
| value: 21.645 | |
| - type: ndcg_at_10 | |
| value: 27.306 | |
| - type: ndcg_at_100 | |
| value: 31.496000000000002 | |
| - type: ndcg_at_1000 | |
| value: 34.53 | |
| - type: ndcg_at_3 | |
| value: 23.73 | |
| - type: ndcg_at_5 | |
| value: 25.294 | |
| - type: precision_at_1 | |
| value: 21.645 | |
| - type: precision_at_10 | |
| value: 4.797 | |
| - type: precision_at_100 | |
| value: 0.8059999999999999 | |
| - type: precision_at_1000 | |
| value: 0.121 | |
| - type: precision_at_3 | |
| value: 10.850999999999999 | |
| - type: precision_at_5 | |
| value: 7.736 | |
| - type: recall_at_1 | |
| value: 17.586 | |
| - type: recall_at_10 | |
| value: 35.481 | |
| - type: recall_at_100 | |
| value: 54.534000000000006 | |
| - type: recall_at_1000 | |
| value: 76.456 | |
| - type: recall_at_3 | |
| value: 25.335 | |
| - type: recall_at_5 | |
| value: 29.473 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackUnixRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 25.095 | |
| - type: map_at_10 | |
| value: 32.374 | |
| - type: map_at_100 | |
| value: 33.537 | |
| - type: map_at_1000 | |
| value: 33.634 | |
| - type: map_at_3 | |
| value: 30.089 | |
| - type: map_at_5 | |
| value: 31.433 | |
| - type: mrr_at_1 | |
| value: 29.198 | |
| - type: mrr_at_10 | |
| value: 36.01 | |
| - type: mrr_at_100 | |
| value: 37.022 | |
| - type: mrr_at_1000 | |
| value: 37.083 | |
| - type: mrr_at_3 | |
| value: 33.94 | |
| - type: mrr_at_5 | |
| value: 35.148 | |
| - type: ndcg_at_1 | |
| value: 29.198 | |
| - type: ndcg_at_10 | |
| value: 36.729 | |
| - type: ndcg_at_100 | |
| value: 42.114000000000004 | |
| - type: ndcg_at_1000 | |
| value: 44.592 | |
| - type: ndcg_at_3 | |
| value: 32.644 | |
| - type: ndcg_at_5 | |
| value: 34.652 | |
| - type: precision_at_1 | |
| value: 29.198 | |
| - type: precision_at_10 | |
| value: 5.970000000000001 | |
| - type: precision_at_100 | |
| value: 0.967 | |
| - type: precision_at_1000 | |
| value: 0.129 | |
| - type: precision_at_3 | |
| value: 14.396999999999998 | |
| - type: precision_at_5 | |
| value: 10.093 | |
| - type: recall_at_1 | |
| value: 25.095 | |
| - type: recall_at_10 | |
| value: 46.392 | |
| - type: recall_at_100 | |
| value: 69.706 | |
| - type: recall_at_1000 | |
| value: 87.738 | |
| - type: recall_at_3 | |
| value: 35.303000000000004 | |
| - type: recall_at_5 | |
| value: 40.441 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackWebmastersRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 26.857999999999997 | |
| - type: map_at_10 | |
| value: 34.066 | |
| - type: map_at_100 | |
| value: 35.671 | |
| - type: map_at_1000 | |
| value: 35.881 | |
| - type: map_at_3 | |
| value: 31.304 | |
| - type: map_at_5 | |
| value: 32.885 | |
| - type: mrr_at_1 | |
| value: 32.411 | |
| - type: mrr_at_10 | |
| value: 38.987 | |
| - type: mrr_at_100 | |
| value: 39.894 | |
| - type: mrr_at_1000 | |
| value: 39.959 | |
| - type: mrr_at_3 | |
| value: 36.626999999999995 | |
| - type: mrr_at_5 | |
| value: 38.011 | |
| - type: ndcg_at_1 | |
| value: 32.411 | |
| - type: ndcg_at_10 | |
| value: 39.208 | |
| - type: ndcg_at_100 | |
| value: 44.626 | |
| - type: ndcg_at_1000 | |
| value: 47.43 | |
| - type: ndcg_at_3 | |
| value: 35.091 | |
| - type: ndcg_at_5 | |
| value: 37.119 | |
| - type: precision_at_1 | |
| value: 32.411 | |
| - type: precision_at_10 | |
| value: 7.51 | |
| - type: precision_at_100 | |
| value: 1.486 | |
| - type: precision_at_1000 | |
| value: 0.234 | |
| - type: precision_at_3 | |
| value: 16.14 | |
| - type: precision_at_5 | |
| value: 11.976 | |
| - type: recall_at_1 | |
| value: 26.857999999999997 | |
| - type: recall_at_10 | |
| value: 47.407 | |
| - type: recall_at_100 | |
| value: 72.236 | |
| - type: recall_at_1000 | |
| value: 90.77 | |
| - type: recall_at_3 | |
| value: 35.125 | |
| - type: recall_at_5 | |
| value: 40.522999999999996 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: BeIR/cqadupstack | |
| name: MTEB CQADupstackWordpressRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 21.3 | |
| - type: map_at_10 | |
| value: 27.412999999999997 | |
| - type: map_at_100 | |
| value: 28.29 | |
| - type: map_at_1000 | |
| value: 28.398 | |
| - type: map_at_3 | |
| value: 25.169999999999998 | |
| - type: map_at_5 | |
| value: 26.496 | |
| - type: mrr_at_1 | |
| value: 23.29 | |
| - type: mrr_at_10 | |
| value: 29.215000000000003 | |
| - type: mrr_at_100 | |
| value: 30.073 | |
| - type: mrr_at_1000 | |
| value: 30.156 | |
| - type: mrr_at_3 | |
| value: 26.956000000000003 | |
| - type: mrr_at_5 | |
| value: 28.38 | |
| - type: ndcg_at_1 | |
| value: 23.29 | |
| - type: ndcg_at_10 | |
| value: 31.113000000000003 | |
| - type: ndcg_at_100 | |
| value: 35.701 | |
| - type: ndcg_at_1000 | |
| value: 38.505 | |
| - type: ndcg_at_3 | |
| value: 26.727 | |
| - type: ndcg_at_5 | |
| value: 29.037000000000003 | |
| - type: precision_at_1 | |
| value: 23.29 | |
| - type: precision_at_10 | |
| value: 4.787 | |
| - type: precision_at_100 | |
| value: 0.763 | |
| - type: precision_at_1000 | |
| value: 0.11100000000000002 | |
| - type: precision_at_3 | |
| value: 11.091 | |
| - type: precision_at_5 | |
| value: 7.985 | |
| - type: recall_at_1 | |
| value: 21.3 | |
| - type: recall_at_10 | |
| value: 40.782000000000004 | |
| - type: recall_at_100 | |
| value: 62.13999999999999 | |
| - type: recall_at_1000 | |
| value: 83.012 | |
| - type: recall_at_3 | |
| value: 29.131 | |
| - type: recall_at_5 | |
| value: 34.624 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: climate-fever | |
| name: MTEB ClimateFEVER | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 9.631 | |
| - type: map_at_10 | |
| value: 16.634999999999998 | |
| - type: map_at_100 | |
| value: 18.23 | |
| - type: map_at_1000 | |
| value: 18.419 | |
| - type: map_at_3 | |
| value: 13.66 | |
| - type: map_at_5 | |
| value: 15.173 | |
| - type: mrr_at_1 | |
| value: 21.368000000000002 | |
| - type: mrr_at_10 | |
| value: 31.56 | |
| - type: mrr_at_100 | |
| value: 32.58 | |
| - type: mrr_at_1000 | |
| value: 32.633 | |
| - type: mrr_at_3 | |
| value: 28.241 | |
| - type: mrr_at_5 | |
| value: 30.225 | |
| - type: ndcg_at_1 | |
| value: 21.368000000000002 | |
| - type: ndcg_at_10 | |
| value: 23.855999999999998 | |
| - type: ndcg_at_100 | |
| value: 30.686999999999998 | |
| - type: ndcg_at_1000 | |
| value: 34.327000000000005 | |
| - type: ndcg_at_3 | |
| value: 18.781 | |
| - type: ndcg_at_5 | |
| value: 20.73 | |
| - type: precision_at_1 | |
| value: 21.368000000000002 | |
| - type: precision_at_10 | |
| value: 7.564 | |
| - type: precision_at_100 | |
| value: 1.496 | |
| - type: precision_at_1000 | |
| value: 0.217 | |
| - type: precision_at_3 | |
| value: 13.876 | |
| - type: precision_at_5 | |
| value: 11.062 | |
| - type: recall_at_1 | |
| value: 9.631 | |
| - type: recall_at_10 | |
| value: 29.517 | |
| - type: recall_at_100 | |
| value: 53.452 | |
| - type: recall_at_1000 | |
| value: 74.115 | |
| - type: recall_at_3 | |
| value: 17.605999999999998 | |
| - type: recall_at_5 | |
| value: 22.505 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: dbpedia-entity | |
| name: MTEB DBPedia | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 8.885 | |
| - type: map_at_10 | |
| value: 18.798000000000002 | |
| - type: map_at_100 | |
| value: 26.316 | |
| - type: map_at_1000 | |
| value: 27.869 | |
| - type: map_at_3 | |
| value: 13.719000000000001 | |
| - type: map_at_5 | |
| value: 15.716 | |
| - type: mrr_at_1 | |
| value: 66 | |
| - type: mrr_at_10 | |
| value: 74.263 | |
| - type: mrr_at_100 | |
| value: 74.519 | |
| - type: mrr_at_1000 | |
| value: 74.531 | |
| - type: mrr_at_3 | |
| value: 72.458 | |
| - type: mrr_at_5 | |
| value: 73.321 | |
| - type: ndcg_at_1 | |
| value: 53.87499999999999 | |
| - type: ndcg_at_10 | |
| value: 40.355999999999995 | |
| - type: ndcg_at_100 | |
| value: 44.366 | |
| - type: ndcg_at_1000 | |
| value: 51.771 | |
| - type: ndcg_at_3 | |
| value: 45.195 | |
| - type: ndcg_at_5 | |
| value: 42.187000000000005 | |
| - type: precision_at_1 | |
| value: 66 | |
| - type: precision_at_10 | |
| value: 31.75 | |
| - type: precision_at_100 | |
| value: 10.11 | |
| - type: precision_at_1000 | |
| value: 1.9800000000000002 | |
| - type: precision_at_3 | |
| value: 48.167 | |
| - type: precision_at_5 | |
| value: 40.050000000000004 | |
| - type: recall_at_1 | |
| value: 8.885 | |
| - type: recall_at_10 | |
| value: 24.471999999999998 | |
| - type: recall_at_100 | |
| value: 49.669000000000004 | |
| - type: recall_at_1000 | |
| value: 73.383 | |
| - type: recall_at_3 | |
| value: 14.872 | |
| - type: recall_at_5 | |
| value: 18.262999999999998 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/emotion | |
| name: MTEB EmotionClassification | |
| config: default | |
| split: test | |
| revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 | |
| metrics: | |
| - type: accuracy | |
| value: 45.18 | |
| - type: f1 | |
| value: 40.26878691789978 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: fever | |
| name: MTEB FEVER | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 62.751999999999995 | |
| - type: map_at_10 | |
| value: 74.131 | |
| - type: map_at_100 | |
| value: 74.407 | |
| - type: map_at_1000 | |
| value: 74.423 | |
| - type: map_at_3 | |
| value: 72.329 | |
| - type: map_at_5 | |
| value: 73.555 | |
| - type: mrr_at_1 | |
| value: 67.282 | |
| - type: mrr_at_10 | |
| value: 78.292 | |
| - type: mrr_at_100 | |
| value: 78.455 | |
| - type: mrr_at_1000 | |
| value: 78.458 | |
| - type: mrr_at_3 | |
| value: 76.755 | |
| - type: mrr_at_5 | |
| value: 77.839 | |
| - type: ndcg_at_1 | |
| value: 67.282 | |
| - type: ndcg_at_10 | |
| value: 79.443 | |
| - type: ndcg_at_100 | |
| value: 80.529 | |
| - type: ndcg_at_1000 | |
| value: 80.812 | |
| - type: ndcg_at_3 | |
| value: 76.281 | |
| - type: ndcg_at_5 | |
| value: 78.235 | |
| - type: precision_at_1 | |
| value: 67.282 | |
| - type: precision_at_10 | |
| value: 10.078 | |
| - type: precision_at_100 | |
| value: 1.082 | |
| - type: precision_at_1000 | |
| value: 0.11199999999999999 | |
| - type: precision_at_3 | |
| value: 30.178 | |
| - type: precision_at_5 | |
| value: 19.232 | |
| - type: recall_at_1 | |
| value: 62.751999999999995 | |
| - type: recall_at_10 | |
| value: 91.521 | |
| - type: recall_at_100 | |
| value: 95.997 | |
| - type: recall_at_1000 | |
| value: 97.775 | |
| - type: recall_at_3 | |
| value: 83.131 | |
| - type: recall_at_5 | |
| value: 87.93299999999999 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: fiqa | |
| name: MTEB FiQA2018 | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 18.861 | |
| - type: map_at_10 | |
| value: 30.252000000000002 | |
| - type: map_at_100 | |
| value: 32.082 | |
| - type: map_at_1000 | |
| value: 32.261 | |
| - type: map_at_3 | |
| value: 25.909 | |
| - type: map_at_5 | |
| value: 28.296 | |
| - type: mrr_at_1 | |
| value: 37.346000000000004 | |
| - type: mrr_at_10 | |
| value: 45.802 | |
| - type: mrr_at_100 | |
| value: 46.611999999999995 | |
| - type: mrr_at_1000 | |
| value: 46.659 | |
| - type: mrr_at_3 | |
| value: 43.056 | |
| - type: mrr_at_5 | |
| value: 44.637 | |
| - type: ndcg_at_1 | |
| value: 37.346000000000004 | |
| - type: ndcg_at_10 | |
| value: 38.169 | |
| - type: ndcg_at_100 | |
| value: 44.864 | |
| - type: ndcg_at_1000 | |
| value: 47.974 | |
| - type: ndcg_at_3 | |
| value: 33.619 | |
| - type: ndcg_at_5 | |
| value: 35.317 | |
| - type: precision_at_1 | |
| value: 37.346000000000004 | |
| - type: precision_at_10 | |
| value: 10.693999999999999 | |
| - type: precision_at_100 | |
| value: 1.775 | |
| - type: precision_at_1000 | |
| value: 0.231 | |
| - type: precision_at_3 | |
| value: 22.325 | |
| - type: precision_at_5 | |
| value: 16.852 | |
| - type: recall_at_1 | |
| value: 18.861 | |
| - type: recall_at_10 | |
| value: 45.672000000000004 | |
| - type: recall_at_100 | |
| value: 70.60499999999999 | |
| - type: recall_at_1000 | |
| value: 89.216 | |
| - type: recall_at_3 | |
| value: 30.361 | |
| - type: recall_at_5 | |
| value: 36.998999999999995 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: hotpotqa | |
| name: MTEB HotpotQA | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 37.852999999999994 | |
| - type: map_at_10 | |
| value: 59.961 | |
| - type: map_at_100 | |
| value: 60.78 | |
| - type: map_at_1000 | |
| value: 60.843 | |
| - type: map_at_3 | |
| value: 56.39999999999999 | |
| - type: map_at_5 | |
| value: 58.646 | |
| - type: mrr_at_1 | |
| value: 75.70599999999999 | |
| - type: mrr_at_10 | |
| value: 82.321 | |
| - type: mrr_at_100 | |
| value: 82.516 | |
| - type: mrr_at_1000 | |
| value: 82.525 | |
| - type: mrr_at_3 | |
| value: 81.317 | |
| - type: mrr_at_5 | |
| value: 81.922 | |
| - type: ndcg_at_1 | |
| value: 75.70599999999999 | |
| - type: ndcg_at_10 | |
| value: 68.557 | |
| - type: ndcg_at_100 | |
| value: 71.485 | |
| - type: ndcg_at_1000 | |
| value: 72.71600000000001 | |
| - type: ndcg_at_3 | |
| value: 63.524 | |
| - type: ndcg_at_5 | |
| value: 66.338 | |
| - type: precision_at_1 | |
| value: 75.70599999999999 | |
| - type: precision_at_10 | |
| value: 14.463000000000001 | |
| - type: precision_at_100 | |
| value: 1.677 | |
| - type: precision_at_1000 | |
| value: 0.184 | |
| - type: precision_at_3 | |
| value: 40.806 | |
| - type: precision_at_5 | |
| value: 26.709 | |
| - type: recall_at_1 | |
| value: 37.852999999999994 | |
| - type: recall_at_10 | |
| value: 72.316 | |
| - type: recall_at_100 | |
| value: 83.842 | |
| - type: recall_at_1000 | |
| value: 91.999 | |
| - type: recall_at_3 | |
| value: 61.209 | |
| - type: recall_at_5 | |
| value: 66.77199999999999 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/imdb | |
| name: MTEB ImdbClassification | |
| config: default | |
| split: test | |
| revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 | |
| metrics: | |
| - type: accuracy | |
| value: 85.46039999999999 | |
| - type: ap | |
| value: 79.9812521351881 | |
| - type: f1 | |
| value: 85.31722909702084 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: msmarco | |
| name: MTEB MSMARCO | |
| config: default | |
| split: dev | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 22.704 | |
| - type: map_at_10 | |
| value: 35.329 | |
| - type: map_at_100 | |
| value: 36.494 | |
| - type: map_at_1000 | |
| value: 36.541000000000004 | |
| - type: map_at_3 | |
| value: 31.476 | |
| - type: map_at_5 | |
| value: 33.731 | |
| - type: mrr_at_1 | |
| value: 23.294999999999998 | |
| - type: mrr_at_10 | |
| value: 35.859 | |
| - type: mrr_at_100 | |
| value: 36.968 | |
| - type: mrr_at_1000 | |
| value: 37.008 | |
| - type: mrr_at_3 | |
| value: 32.085 | |
| - type: mrr_at_5 | |
| value: 34.299 | |
| - type: ndcg_at_1 | |
| value: 23.324 | |
| - type: ndcg_at_10 | |
| value: 42.274 | |
| - type: ndcg_at_100 | |
| value: 47.839999999999996 | |
| - type: ndcg_at_1000 | |
| value: 48.971 | |
| - type: ndcg_at_3 | |
| value: 34.454 | |
| - type: ndcg_at_5 | |
| value: 38.464 | |
| - type: precision_at_1 | |
| value: 23.324 | |
| - type: precision_at_10 | |
| value: 6.648 | |
| - type: precision_at_100 | |
| value: 0.9440000000000001 | |
| - type: precision_at_1000 | |
| value: 0.104 | |
| - type: precision_at_3 | |
| value: 14.674999999999999 | |
| - type: precision_at_5 | |
| value: 10.850999999999999 | |
| - type: recall_at_1 | |
| value: 22.704 | |
| - type: recall_at_10 | |
| value: 63.660000000000004 | |
| - type: recall_at_100 | |
| value: 89.29899999999999 | |
| - type: recall_at_1000 | |
| value: 97.88900000000001 | |
| - type: recall_at_3 | |
| value: 42.441 | |
| - type: recall_at_5 | |
| value: 52.04 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_domain | |
| name: MTEB MTOPDomainClassification (en) | |
| config: en | |
| split: test | |
| revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf | |
| metrics: | |
| - type: accuracy | |
| value: 93.1326949384405 | |
| - type: f1 | |
| value: 92.89743579612082 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_domain | |
| name: MTEB MTOPDomainClassification (de) | |
| config: de | |
| split: test | |
| revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf | |
| metrics: | |
| - type: accuracy | |
| value: 89.62524654832347 | |
| - type: f1 | |
| value: 88.65106082263151 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_domain | |
| name: MTEB MTOPDomainClassification (es) | |
| config: es | |
| split: test | |
| revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf | |
| metrics: | |
| - type: accuracy | |
| value: 90.59039359573046 | |
| - type: f1 | |
| value: 90.31532892105662 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_domain | |
| name: MTEB MTOPDomainClassification (fr) | |
| config: fr | |
| split: test | |
| revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf | |
| metrics: | |
| - type: accuracy | |
| value: 86.21046038208581 | |
| - type: f1 | |
| value: 86.41459529813113 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_domain | |
| name: MTEB MTOPDomainClassification (hi) | |
| config: hi | |
| split: test | |
| revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf | |
| metrics: | |
| - type: accuracy | |
| value: 87.3180351380423 | |
| - type: f1 | |
| value: 86.71383078226444 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_domain | |
| name: MTEB MTOPDomainClassification (th) | |
| config: th | |
| split: test | |
| revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf | |
| metrics: | |
| - type: accuracy | |
| value: 86.24231464737792 | |
| - type: f1 | |
| value: 86.31845567592403 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_intent | |
| name: MTEB MTOPIntentClassification (en) | |
| config: en | |
| split: test | |
| revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba | |
| metrics: | |
| - type: accuracy | |
| value: 75.27131782945736 | |
| - type: f1 | |
| value: 57.52079940417103 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_intent | |
| name: MTEB MTOPIntentClassification (de) | |
| config: de | |
| split: test | |
| revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba | |
| metrics: | |
| - type: accuracy | |
| value: 71.2341504649197 | |
| - type: f1 | |
| value: 51.349951558039244 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_intent | |
| name: MTEB MTOPIntentClassification (es) | |
| config: es | |
| split: test | |
| revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba | |
| metrics: | |
| - type: accuracy | |
| value: 71.27418278852569 | |
| - type: f1 | |
| value: 50.1714985749095 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_intent | |
| name: MTEB MTOPIntentClassification (fr) | |
| config: fr | |
| split: test | |
| revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba | |
| metrics: | |
| - type: accuracy | |
| value: 67.68243031631694 | |
| - type: f1 | |
| value: 50.1066160836192 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_intent | |
| name: MTEB MTOPIntentClassification (hi) | |
| config: hi | |
| split: test | |
| revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba | |
| metrics: | |
| - type: accuracy | |
| value: 69.2362854069559 | |
| - type: f1 | |
| value: 48.821279948766424 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/mtop_intent | |
| name: MTEB MTOPIntentClassification (th) | |
| config: th | |
| split: test | |
| revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba | |
| metrics: | |
| - type: accuracy | |
| value: 71.71428571428571 | |
| - type: f1 | |
| value: 53.94611389496195 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (af) | |
| config: af | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 59.97646267652992 | |
| - type: f1 | |
| value: 57.26797883561521 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (am) | |
| config: am | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 53.65501008742435 | |
| - type: f1 | |
| value: 50.416258382177034 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ar) | |
| config: ar | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 57.45796906523201 | |
| - type: f1 | |
| value: 53.306690547422185 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (az) | |
| config: az | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 62.59246805648957 | |
| - type: f1 | |
| value: 59.818381969051494 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (bn) | |
| config: bn | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 61.126429051782104 | |
| - type: f1 | |
| value: 58.25993593933026 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (cy) | |
| config: cy | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 50.057162071284466 | |
| - type: f1 | |
| value: 46.96095728790911 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (da) | |
| config: da | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 66.64425016812375 | |
| - type: f1 | |
| value: 62.858291698755764 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (de) | |
| config: de | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 66.08944182918628 | |
| - type: f1 | |
| value: 62.44639030604241 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (el) | |
| config: el | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 64.68056489576328 | |
| - type: f1 | |
| value: 61.775326758789504 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (en) | |
| config: en | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 72.11163416274377 | |
| - type: f1 | |
| value: 69.70789096927015 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (es) | |
| config: es | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 68.40282447881641 | |
| - type: f1 | |
| value: 66.38492065671895 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (fa) | |
| config: fa | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 67.24613315400134 | |
| - type: f1 | |
| value: 64.3348019501336 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (fi) | |
| config: fi | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 65.78345662407531 | |
| - type: f1 | |
| value: 62.21279452354622 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (fr) | |
| config: fr | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 67.9455279085407 | |
| - type: f1 | |
| value: 65.48193124964094 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (he) | |
| config: he | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 62.05110961667788 | |
| - type: f1 | |
| value: 58.097856564684534 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (hi) | |
| config: hi | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 64.95292535305985 | |
| - type: f1 | |
| value: 62.09182174767901 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (hu) | |
| config: hu | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 64.97310020174848 | |
| - type: f1 | |
| value: 61.14252567730396 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (hy) | |
| config: hy | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 60.08069939475453 | |
| - type: f1 | |
| value: 57.044041742492034 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (id) | |
| config: id | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 66.63752521856085 | |
| - type: f1 | |
| value: 63.889340907205316 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (is) | |
| config: is | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 56.385339609952936 | |
| - type: f1 | |
| value: 53.449033750088304 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (it) | |
| config: it | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 68.93073301950234 | |
| - type: f1 | |
| value: 65.9884357824104 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ja) | |
| config: ja | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 68.94418291862812 | |
| - type: f1 | |
| value: 66.48740222583132 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (jv) | |
| config: jv | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 54.26025554808339 | |
| - type: f1 | |
| value: 50.19562815100793 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ka) | |
| config: ka | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 48.98789509078682 | |
| - type: f1 | |
| value: 46.65788438676836 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (km) | |
| config: km | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 44.68728984532616 | |
| - type: f1 | |
| value: 41.642419349541996 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (kn) | |
| config: kn | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 59.19300605245461 | |
| - type: f1 | |
| value: 55.8626492442437 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ko) | |
| config: ko | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 66.33826496301278 | |
| - type: f1 | |
| value: 63.89499791648792 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (lv) | |
| config: lv | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 60.33960995292536 | |
| - type: f1 | |
| value: 57.15242464180892 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ml) | |
| config: ml | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 63.09347679892402 | |
| - type: f1 | |
| value: 59.64733214063841 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (mn) | |
| config: mn | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 58.75924680564896 | |
| - type: f1 | |
| value: 55.96585692366827 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ms) | |
| config: ms | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 62.48486886348352 | |
| - type: f1 | |
| value: 59.45143559032946 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (my) | |
| config: my | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 58.56422326832549 | |
| - type: f1 | |
| value: 54.96368702901926 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (nb) | |
| config: nb | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 66.18022864828512 | |
| - type: f1 | |
| value: 63.05369805040634 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (nl) | |
| config: nl | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 67.30329522528581 | |
| - type: f1 | |
| value: 64.06084612020727 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (pl) | |
| config: pl | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 68.36919973100201 | |
| - type: f1 | |
| value: 65.12154124788887 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (pt) | |
| config: pt | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 68.98117014122394 | |
| - type: f1 | |
| value: 66.41847559806962 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ro) | |
| config: ro | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 65.53799596503026 | |
| - type: f1 | |
| value: 62.17067330740817 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ru) | |
| config: ru | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 69.01815736381977 | |
| - type: f1 | |
| value: 66.24988369607843 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (sl) | |
| config: sl | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 62.34700739744452 | |
| - type: f1 | |
| value: 59.957933424941636 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (sq) | |
| config: sq | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 61.23402824478815 | |
| - type: f1 | |
| value: 57.98836976018471 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (sv) | |
| config: sv | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 68.54068594485541 | |
| - type: f1 | |
| value: 65.43849680666855 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (sw) | |
| config: sw | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 55.998655010087425 | |
| - type: f1 | |
| value: 52.83737515406804 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ta) | |
| config: ta | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 58.71217215870882 | |
| - type: f1 | |
| value: 55.051794977833026 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (te) | |
| config: te | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 59.724277067921996 | |
| - type: f1 | |
| value: 56.33485571838306 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (th) | |
| config: th | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 65.59515803631473 | |
| - type: f1 | |
| value: 64.96772366193588 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (tl) | |
| config: tl | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 60.860793544048406 | |
| - type: f1 | |
| value: 58.148845819115394 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (tr) | |
| config: tr | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 67.40753194351043 | |
| - type: f1 | |
| value: 63.18903778054698 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (ur) | |
| config: ur | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 61.52320107599194 | |
| - type: f1 | |
| value: 58.356144563398516 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (vi) | |
| config: vi | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 66.17014122394083 | |
| - type: f1 | |
| value: 63.919964062638925 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (zh-CN) | |
| config: zh-CN | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 69.15601882985878 | |
| - type: f1 | |
| value: 67.01451905761371 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_intent | |
| name: MTEB MassiveIntentClassification (zh-TW) | |
| config: zh-TW | |
| split: test | |
| revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 | |
| metrics: | |
| - type: accuracy | |
| value: 64.65030262273034 | |
| - type: f1 | |
| value: 64.14420425129063 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (af) | |
| config: af | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 65.08742434431743 | |
| - type: f1 | |
| value: 63.044060042311756 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (am) | |
| config: am | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 58.52387357094821 | |
| - type: f1 | |
| value: 56.82398588814534 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ar) | |
| config: ar | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 62.239408204438476 | |
| - type: f1 | |
| value: 61.92570286170469 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (az) | |
| config: az | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 63.74915938130463 | |
| - type: f1 | |
| value: 62.130740689396276 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (bn) | |
| config: bn | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 65.00336247478144 | |
| - type: f1 | |
| value: 63.71080635228055 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (cy) | |
| config: cy | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 52.837928715534645 | |
| - type: f1 | |
| value: 50.390741680320836 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (da) | |
| config: da | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 72.42098184263618 | |
| - type: f1 | |
| value: 71.41355113538995 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (de) | |
| config: de | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 71.95359784801613 | |
| - type: f1 | |
| value: 71.42699340156742 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (el) | |
| config: el | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.18157363819772 | |
| - type: f1 | |
| value: 69.74836113037671 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (en) | |
| config: en | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 77.08137188971082 | |
| - type: f1 | |
| value: 76.78000685068261 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (es) | |
| config: es | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 71.5030262273033 | |
| - type: f1 | |
| value: 71.71620130425673 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (fa) | |
| config: fa | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.24546065904505 | |
| - type: f1 | |
| value: 69.07638311730359 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (fi) | |
| config: fi | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 69.12911903160726 | |
| - type: f1 | |
| value: 68.32651736539815 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (fr) | |
| config: fr | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 71.89307330195025 | |
| - type: f1 | |
| value: 71.33986549860187 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (he) | |
| config: he | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 67.44451916610626 | |
| - type: f1 | |
| value: 66.90192664503866 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (hi) | |
| config: hi | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 69.16274377942166 | |
| - type: f1 | |
| value: 68.01090953775066 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (hu) | |
| config: hu | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.75319435104237 | |
| - type: f1 | |
| value: 70.18035309201403 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (hy) | |
| config: hy | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 63.14391392064559 | |
| - type: f1 | |
| value: 61.48286540778145 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (id) | |
| config: id | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.70275722932078 | |
| - type: f1 | |
| value: 70.26164779846495 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (is) | |
| config: is | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 60.93813046402153 | |
| - type: f1 | |
| value: 58.8852862116525 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (it) | |
| config: it | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 72.320107599193 | |
| - type: f1 | |
| value: 72.19836409602924 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ja) | |
| config: ja | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 74.65366509751176 | |
| - type: f1 | |
| value: 74.55188288799579 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (jv) | |
| config: jv | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 59.694014794889036 | |
| - type: f1 | |
| value: 58.11353311721067 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ka) | |
| config: ka | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 54.37457969065231 | |
| - type: f1 | |
| value: 52.81306134311697 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (km) | |
| config: km | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 48.3086751849361 | |
| - type: f1 | |
| value: 45.396449765419376 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (kn) | |
| config: kn | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 62.151983860121064 | |
| - type: f1 | |
| value: 60.31762544281696 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ko) | |
| config: ko | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 72.44788164088769 | |
| - type: f1 | |
| value: 71.68150151736367 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (lv) | |
| config: lv | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 62.81439139206455 | |
| - type: f1 | |
| value: 62.06735559105593 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ml) | |
| config: ml | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 68.04303967720242 | |
| - type: f1 | |
| value: 66.68298851670133 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (mn) | |
| config: mn | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 61.43913920645595 | |
| - type: f1 | |
| value: 60.25605977560783 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ms) | |
| config: ms | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 66.90316072629456 | |
| - type: f1 | |
| value: 65.1325924692381 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (my) | |
| config: my | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 61.63752521856086 | |
| - type: f1 | |
| value: 59.14284778039585 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (nb) | |
| config: nb | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 71.63080026899797 | |
| - type: f1 | |
| value: 70.89771864626877 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (nl) | |
| config: nl | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 72.10827168796234 | |
| - type: f1 | |
| value: 71.71954219691159 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (pl) | |
| config: pl | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.59515803631471 | |
| - type: f1 | |
| value: 70.05040128099003 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (pt) | |
| config: pt | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.83389374579691 | |
| - type: f1 | |
| value: 70.84877936562735 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ro) | |
| config: ro | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 69.18628110289173 | |
| - type: f1 | |
| value: 68.97232927921841 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ru) | |
| config: ru | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 72.99260255548083 | |
| - type: f1 | |
| value: 72.85139492157732 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (sl) | |
| config: sl | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 65.26227303295225 | |
| - type: f1 | |
| value: 65.08833655469431 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (sq) | |
| config: sq | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 66.48621385339611 | |
| - type: f1 | |
| value: 64.43483199071298 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (sv) | |
| config: sv | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 73.14391392064559 | |
| - type: f1 | |
| value: 72.2580822579741 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (sw) | |
| config: sw | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 59.88567585743107 | |
| - type: f1 | |
| value: 58.3073765932569 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ta) | |
| config: ta | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 62.38399462004034 | |
| - type: f1 | |
| value: 60.82139544252606 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (te) | |
| config: te | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 62.58574310692671 | |
| - type: f1 | |
| value: 60.71443370385374 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (th) | |
| config: th | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 71.61398789509079 | |
| - type: f1 | |
| value: 70.99761812049401 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (tl) | |
| config: tl | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 62.73705447209146 | |
| - type: f1 | |
| value: 61.680849331794796 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (tr) | |
| config: tr | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 71.66778749159381 | |
| - type: f1 | |
| value: 71.17320646080115 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (ur) | |
| config: ur | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 64.640215198386 | |
| - type: f1 | |
| value: 63.301805157015444 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (vi) | |
| config: vi | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.00672494956288 | |
| - type: f1 | |
| value: 70.26005548582106 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (zh-CN) | |
| config: zh-CN | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 75.42030934767989 | |
| - type: f1 | |
| value: 75.2074842882598 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/amazon_massive_scenario | |
| name: MTEB MassiveScenarioClassification (zh-TW) | |
| config: zh-TW | |
| split: test | |
| revision: 7d571f92784cd94a019292a1f45445077d0ef634 | |
| metrics: | |
| - type: accuracy | |
| value: 70.69266980497646 | |
| - type: f1 | |
| value: 70.94103167391192 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/medrxiv-clustering-p2p | |
| name: MTEB MedrxivClusteringP2P | |
| config: default | |
| split: test | |
| revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 | |
| metrics: | |
| - type: v_measure | |
| value: 28.91697191169135 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/medrxiv-clustering-s2s | |
| name: MTEB MedrxivClusteringS2S | |
| config: default | |
| split: test | |
| revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 | |
| metrics: | |
| - type: v_measure | |
| value: 28.434000079573313 | |
| - task: | |
| type: Reranking | |
| dataset: | |
| type: mteb/mind_small | |
| name: MTEB MindSmallReranking | |
| config: default | |
| split: test | |
| revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69 | |
| metrics: | |
| - type: map | |
| value: 30.96683513343383 | |
| - type: mrr | |
| value: 31.967364078714834 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: nfcorpus | |
| name: MTEB NFCorpus | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 5.5280000000000005 | |
| - type: map_at_10 | |
| value: 11.793 | |
| - type: map_at_100 | |
| value: 14.496999999999998 | |
| - type: map_at_1000 | |
| value: 15.783 | |
| - type: map_at_3 | |
| value: 8.838 | |
| - type: map_at_5 | |
| value: 10.07 | |
| - type: mrr_at_1 | |
| value: 43.653 | |
| - type: mrr_at_10 | |
| value: 51.531000000000006 | |
| - type: mrr_at_100 | |
| value: 52.205 | |
| - type: mrr_at_1000 | |
| value: 52.242999999999995 | |
| - type: mrr_at_3 | |
| value: 49.431999999999995 | |
| - type: mrr_at_5 | |
| value: 50.470000000000006 | |
| - type: ndcg_at_1 | |
| value: 42.415000000000006 | |
| - type: ndcg_at_10 | |
| value: 32.464999999999996 | |
| - type: ndcg_at_100 | |
| value: 28.927999999999997 | |
| - type: ndcg_at_1000 | |
| value: 37.629000000000005 | |
| - type: ndcg_at_3 | |
| value: 37.845 | |
| - type: ndcg_at_5 | |
| value: 35.147 | |
| - type: precision_at_1 | |
| value: 43.653 | |
| - type: precision_at_10 | |
| value: 23.932000000000002 | |
| - type: precision_at_100 | |
| value: 7.17 | |
| - type: precision_at_1000 | |
| value: 1.967 | |
| - type: precision_at_3 | |
| value: 35.397 | |
| - type: precision_at_5 | |
| value: 29.907 | |
| - type: recall_at_1 | |
| value: 5.5280000000000005 | |
| - type: recall_at_10 | |
| value: 15.568000000000001 | |
| - type: recall_at_100 | |
| value: 28.54 | |
| - type: recall_at_1000 | |
| value: 59.864 | |
| - type: recall_at_3 | |
| value: 9.822000000000001 | |
| - type: recall_at_5 | |
| value: 11.726 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: nq | |
| name: MTEB NQ | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 37.041000000000004 | |
| - type: map_at_10 | |
| value: 52.664 | |
| - type: map_at_100 | |
| value: 53.477 | |
| - type: map_at_1000 | |
| value: 53.505 | |
| - type: map_at_3 | |
| value: 48.510999999999996 | |
| - type: map_at_5 | |
| value: 51.036 | |
| - type: mrr_at_1 | |
| value: 41.338 | |
| - type: mrr_at_10 | |
| value: 55.071000000000005 | |
| - type: mrr_at_100 | |
| value: 55.672 | |
| - type: mrr_at_1000 | |
| value: 55.689 | |
| - type: mrr_at_3 | |
| value: 51.82 | |
| - type: mrr_at_5 | |
| value: 53.852 | |
| - type: ndcg_at_1 | |
| value: 41.338 | |
| - type: ndcg_at_10 | |
| value: 60.01800000000001 | |
| - type: ndcg_at_100 | |
| value: 63.409000000000006 | |
| - type: ndcg_at_1000 | |
| value: 64.017 | |
| - type: ndcg_at_3 | |
| value: 52.44799999999999 | |
| - type: ndcg_at_5 | |
| value: 56.571000000000005 | |
| - type: precision_at_1 | |
| value: 41.338 | |
| - type: precision_at_10 | |
| value: 9.531 | |
| - type: precision_at_100 | |
| value: 1.145 | |
| - type: precision_at_1000 | |
| value: 0.12 | |
| - type: precision_at_3 | |
| value: 23.416 | |
| - type: precision_at_5 | |
| value: 16.46 | |
| - type: recall_at_1 | |
| value: 37.041000000000004 | |
| - type: recall_at_10 | |
| value: 79.76299999999999 | |
| - type: recall_at_100 | |
| value: 94.39 | |
| - type: recall_at_1000 | |
| value: 98.851 | |
| - type: recall_at_3 | |
| value: 60.465 | |
| - type: recall_at_5 | |
| value: 69.906 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: quora | |
| name: MTEB QuoraRetrieval | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 69.952 | |
| - type: map_at_10 | |
| value: 83.758 | |
| - type: map_at_100 | |
| value: 84.406 | |
| - type: map_at_1000 | |
| value: 84.425 | |
| - type: map_at_3 | |
| value: 80.839 | |
| - type: map_at_5 | |
| value: 82.646 | |
| - type: mrr_at_1 | |
| value: 80.62 | |
| - type: mrr_at_10 | |
| value: 86.947 | |
| - type: mrr_at_100 | |
| value: 87.063 | |
| - type: mrr_at_1000 | |
| value: 87.064 | |
| - type: mrr_at_3 | |
| value: 85.96000000000001 | |
| - type: mrr_at_5 | |
| value: 86.619 | |
| - type: ndcg_at_1 | |
| value: 80.63 | |
| - type: ndcg_at_10 | |
| value: 87.64800000000001 | |
| - type: ndcg_at_100 | |
| value: 88.929 | |
| - type: ndcg_at_1000 | |
| value: 89.054 | |
| - type: ndcg_at_3 | |
| value: 84.765 | |
| - type: ndcg_at_5 | |
| value: 86.291 | |
| - type: precision_at_1 | |
| value: 80.63 | |
| - type: precision_at_10 | |
| value: 13.314 | |
| - type: precision_at_100 | |
| value: 1.525 | |
| - type: precision_at_1000 | |
| value: 0.157 | |
| - type: precision_at_3 | |
| value: 37.1 | |
| - type: precision_at_5 | |
| value: 24.372 | |
| - type: recall_at_1 | |
| value: 69.952 | |
| - type: recall_at_10 | |
| value: 94.955 | |
| - type: recall_at_100 | |
| value: 99.38 | |
| - type: recall_at_1000 | |
| value: 99.96000000000001 | |
| - type: recall_at_3 | |
| value: 86.60600000000001 | |
| - type: recall_at_5 | |
| value: 90.997 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/reddit-clustering | |
| name: MTEB RedditClustering | |
| config: default | |
| split: test | |
| revision: 24640382cdbf8abc73003fb0fa6d111a705499eb | |
| metrics: | |
| - type: v_measure | |
| value: 42.41329517878427 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/reddit-clustering-p2p | |
| name: MTEB RedditClusteringP2P | |
| config: default | |
| split: test | |
| revision: 282350215ef01743dc01b456c7f5241fa8937f16 | |
| metrics: | |
| - type: v_measure | |
| value: 55.171278362748666 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: scidocs | |
| name: MTEB SCIDOCS | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 4.213 | |
| - type: map_at_10 | |
| value: 9.895 | |
| - type: map_at_100 | |
| value: 11.776 | |
| - type: map_at_1000 | |
| value: 12.084 | |
| - type: map_at_3 | |
| value: 7.2669999999999995 | |
| - type: map_at_5 | |
| value: 8.620999999999999 | |
| - type: mrr_at_1 | |
| value: 20.8 | |
| - type: mrr_at_10 | |
| value: 31.112000000000002 | |
| - type: mrr_at_100 | |
| value: 32.274 | |
| - type: mrr_at_1000 | |
| value: 32.35 | |
| - type: mrr_at_3 | |
| value: 28.133000000000003 | |
| - type: mrr_at_5 | |
| value: 29.892999999999997 | |
| - type: ndcg_at_1 | |
| value: 20.8 | |
| - type: ndcg_at_10 | |
| value: 17.163999999999998 | |
| - type: ndcg_at_100 | |
| value: 24.738 | |
| - type: ndcg_at_1000 | |
| value: 30.316 | |
| - type: ndcg_at_3 | |
| value: 16.665 | |
| - type: ndcg_at_5 | |
| value: 14.478 | |
| - type: precision_at_1 | |
| value: 20.8 | |
| - type: precision_at_10 | |
| value: 8.74 | |
| - type: precision_at_100 | |
| value: 1.963 | |
| - type: precision_at_1000 | |
| value: 0.33 | |
| - type: precision_at_3 | |
| value: 15.467 | |
| - type: precision_at_5 | |
| value: 12.6 | |
| - type: recall_at_1 | |
| value: 4.213 | |
| - type: recall_at_10 | |
| value: 17.698 | |
| - type: recall_at_100 | |
| value: 39.838 | |
| - type: recall_at_1000 | |
| value: 66.893 | |
| - type: recall_at_3 | |
| value: 9.418 | |
| - type: recall_at_5 | |
| value: 12.773000000000001 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sickr-sts | |
| name: MTEB SICK-R | |
| config: default | |
| split: test | |
| revision: a6ea5a8cab320b040a23452cc28066d9beae2cee | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 82.90453315738294 | |
| - type: cos_sim_spearman | |
| value: 78.51197850080254 | |
| - type: euclidean_pearson | |
| value: 80.09647123597748 | |
| - type: euclidean_spearman | |
| value: 78.63548011514061 | |
| - type: manhattan_pearson | |
| value: 80.10645285675231 | |
| - type: manhattan_spearman | |
| value: 78.57861806068901 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts12-sts | |
| name: MTEB STS12 | |
| config: default | |
| split: test | |
| revision: a0d554a64d88156834ff5ae9920b964011b16384 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 84.2616156846401 | |
| - type: cos_sim_spearman | |
| value: 76.69713867850156 | |
| - type: euclidean_pearson | |
| value: 77.97948563800394 | |
| - type: euclidean_spearman | |
| value: 74.2371211567807 | |
| - type: manhattan_pearson | |
| value: 77.69697879669705 | |
| - type: manhattan_spearman | |
| value: 73.86529778022278 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts13-sts | |
| name: MTEB STS13 | |
| config: default | |
| split: test | |
| revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 77.0293269315045 | |
| - type: cos_sim_spearman | |
| value: 78.02555120584198 | |
| - type: euclidean_pearson | |
| value: 78.25398100379078 | |
| - type: euclidean_spearman | |
| value: 78.66963870599464 | |
| - type: manhattan_pearson | |
| value: 78.14314682167348 | |
| - type: manhattan_spearman | |
| value: 78.57692322969135 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts14-sts | |
| name: MTEB STS14 | |
| config: default | |
| split: test | |
| revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 79.16989925136942 | |
| - type: cos_sim_spearman | |
| value: 76.5996225327091 | |
| - type: euclidean_pearson | |
| value: 77.8319003279786 | |
| - type: euclidean_spearman | |
| value: 76.42824009468998 | |
| - type: manhattan_pearson | |
| value: 77.69118862737736 | |
| - type: manhattan_spearman | |
| value: 76.25568104762812 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts15-sts | |
| name: MTEB STS15 | |
| config: default | |
| split: test | |
| revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 87.42012286935325 | |
| - type: cos_sim_spearman | |
| value: 88.15654297884122 | |
| - type: euclidean_pearson | |
| value: 87.34082819427852 | |
| - type: euclidean_spearman | |
| value: 88.06333589547084 | |
| - type: manhattan_pearson | |
| value: 87.25115596784842 | |
| - type: manhattan_spearman | |
| value: 87.9559927695203 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts16-sts | |
| name: MTEB STS16 | |
| config: default | |
| split: test | |
| revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 82.88222044996712 | |
| - type: cos_sim_spearman | |
| value: 84.28476589061077 | |
| - type: euclidean_pearson | |
| value: 83.17399758058309 | |
| - type: euclidean_spearman | |
| value: 83.85497357244542 | |
| - type: manhattan_pearson | |
| value: 83.0308397703786 | |
| - type: manhattan_spearman | |
| value: 83.71554539935046 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (ko-ko) | |
| config: ko-ko | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 80.20682986257339 | |
| - type: cos_sim_spearman | |
| value: 79.94567120362092 | |
| - type: euclidean_pearson | |
| value: 79.43122480368902 | |
| - type: euclidean_spearman | |
| value: 79.94802077264987 | |
| - type: manhattan_pearson | |
| value: 79.32653021527081 | |
| - type: manhattan_spearman | |
| value: 79.80961146709178 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (ar-ar) | |
| config: ar-ar | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 74.46578144394383 | |
| - type: cos_sim_spearman | |
| value: 74.52496637472179 | |
| - type: euclidean_pearson | |
| value: 72.2903807076809 | |
| - type: euclidean_spearman | |
| value: 73.55549359771645 | |
| - type: manhattan_pearson | |
| value: 72.09324837709393 | |
| - type: manhattan_spearman | |
| value: 73.36743103606581 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (en-ar) | |
| config: en-ar | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 71.37272335116 | |
| - type: cos_sim_spearman | |
| value: 71.26702117766037 | |
| - type: euclidean_pearson | |
| value: 67.114829954434 | |
| - type: euclidean_spearman | |
| value: 66.37938893947761 | |
| - type: manhattan_pearson | |
| value: 66.79688574095246 | |
| - type: manhattan_spearman | |
| value: 66.17292828079667 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (en-de) | |
| config: en-de | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 80.61016770129092 | |
| - type: cos_sim_spearman | |
| value: 82.08515426632214 | |
| - type: euclidean_pearson | |
| value: 80.557340361131 | |
| - type: euclidean_spearman | |
| value: 80.37585812266175 | |
| - type: manhattan_pearson | |
| value: 80.6782873404285 | |
| - type: manhattan_spearman | |
| value: 80.6678073032024 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (en-en) | |
| config: en-en | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 87.00150745350108 | |
| - type: cos_sim_spearman | |
| value: 87.83441972211425 | |
| - type: euclidean_pearson | |
| value: 87.94826702308792 | |
| - type: euclidean_spearman | |
| value: 87.46143974860725 | |
| - type: manhattan_pearson | |
| value: 87.97560344306105 | |
| - type: manhattan_spearman | |
| value: 87.5267102829796 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (en-tr) | |
| config: en-tr | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 64.76325252267235 | |
| - type: cos_sim_spearman | |
| value: 63.32615095463905 | |
| - type: euclidean_pearson | |
| value: 64.07920669155716 | |
| - type: euclidean_spearman | |
| value: 61.21409893072176 | |
| - type: manhattan_pearson | |
| value: 64.26308625680016 | |
| - type: manhattan_spearman | |
| value: 61.2438185254079 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (es-en) | |
| config: es-en | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 75.82644463022595 | |
| - type: cos_sim_spearman | |
| value: 76.50381269945073 | |
| - type: euclidean_pearson | |
| value: 75.1328548315934 | |
| - type: euclidean_spearman | |
| value: 75.63761139408453 | |
| - type: manhattan_pearson | |
| value: 75.18610101241407 | |
| - type: manhattan_spearman | |
| value: 75.30669266354164 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (es-es) | |
| config: es-es | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 87.49994164686832 | |
| - type: cos_sim_spearman | |
| value: 86.73743986245549 | |
| - type: euclidean_pearson | |
| value: 86.8272894387145 | |
| - type: euclidean_spearman | |
| value: 85.97608491000507 | |
| - type: manhattan_pearson | |
| value: 86.74960140396779 | |
| - type: manhattan_spearman | |
| value: 85.79285984190273 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (fr-en) | |
| config: fr-en | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 79.58172210788469 | |
| - type: cos_sim_spearman | |
| value: 80.17516468334607 | |
| - type: euclidean_pearson | |
| value: 77.56537843470504 | |
| - type: euclidean_spearman | |
| value: 77.57264627395521 | |
| - type: manhattan_pearson | |
| value: 78.09703521695943 | |
| - type: manhattan_spearman | |
| value: 78.15942760916954 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (it-en) | |
| config: it-en | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 79.7589932931751 | |
| - type: cos_sim_spearman | |
| value: 80.15210089028162 | |
| - type: euclidean_pearson | |
| value: 77.54135223516057 | |
| - type: euclidean_spearman | |
| value: 77.52697996368764 | |
| - type: manhattan_pearson | |
| value: 77.65734439572518 | |
| - type: manhattan_spearman | |
| value: 77.77702992016121 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts17-crosslingual-sts | |
| name: MTEB STS17 (nl-en) | |
| config: nl-en | |
| split: test | |
| revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 79.16682365511267 | |
| - type: cos_sim_spearman | |
| value: 79.25311267628506 | |
| - type: euclidean_pearson | |
| value: 77.54882036762244 | |
| - type: euclidean_spearman | |
| value: 77.33212935194827 | |
| - type: manhattan_pearson | |
| value: 77.98405516064015 | |
| - type: manhattan_spearman | |
| value: 77.85075717865719 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (en) | |
| config: en | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 59.10473294775917 | |
| - type: cos_sim_spearman | |
| value: 61.82780474476838 | |
| - type: euclidean_pearson | |
| value: 45.885111672377256 | |
| - type: euclidean_spearman | |
| value: 56.88306351932454 | |
| - type: manhattan_pearson | |
| value: 46.101218127323186 | |
| - type: manhattan_spearman | |
| value: 56.80953694186333 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (de) | |
| config: de | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 45.781923079584146 | |
| - type: cos_sim_spearman | |
| value: 55.95098449691107 | |
| - type: euclidean_pearson | |
| value: 25.4571031323205 | |
| - type: euclidean_spearman | |
| value: 49.859978118078935 | |
| - type: manhattan_pearson | |
| value: 25.624938455041384 | |
| - type: manhattan_spearman | |
| value: 49.99546185049401 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (es) | |
| config: es | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 60.00618133997907 | |
| - type: cos_sim_spearman | |
| value: 66.57896677718321 | |
| - type: euclidean_pearson | |
| value: 42.60118466388821 | |
| - type: euclidean_spearman | |
| value: 62.8210759715209 | |
| - type: manhattan_pearson | |
| value: 42.63446860604094 | |
| - type: manhattan_spearman | |
| value: 62.73803068925271 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (pl) | |
| config: pl | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 28.460759121626943 | |
| - type: cos_sim_spearman | |
| value: 34.13459007469131 | |
| - type: euclidean_pearson | |
| value: 6.0917739325525195 | |
| - type: euclidean_spearman | |
| value: 27.9947262664867 | |
| - type: manhattan_pearson | |
| value: 6.16877864169911 | |
| - type: manhattan_spearman | |
| value: 28.00664163971514 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (tr) | |
| config: tr | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 57.42546621771696 | |
| - type: cos_sim_spearman | |
| value: 63.699663168970474 | |
| - type: euclidean_pearson | |
| value: 38.12085278789738 | |
| - type: euclidean_spearman | |
| value: 58.12329140741536 | |
| - type: manhattan_pearson | |
| value: 37.97364549443335 | |
| - type: manhattan_spearman | |
| value: 57.81545502318733 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (ar) | |
| config: ar | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 46.82241380954213 | |
| - type: cos_sim_spearman | |
| value: 57.86569456006391 | |
| - type: euclidean_pearson | |
| value: 31.80480070178813 | |
| - type: euclidean_spearman | |
| value: 52.484000620130104 | |
| - type: manhattan_pearson | |
| value: 31.952708554646097 | |
| - type: manhattan_spearman | |
| value: 52.8560972356195 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (ru) | |
| config: ru | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 52.00447170498087 | |
| - type: cos_sim_spearman | |
| value: 60.664116225735164 | |
| - type: euclidean_pearson | |
| value: 33.87382555421702 | |
| - type: euclidean_spearman | |
| value: 55.74649067458667 | |
| - type: manhattan_pearson | |
| value: 33.99117246759437 | |
| - type: manhattan_spearman | |
| value: 55.98749034923899 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (zh) | |
| config: zh | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 58.06497233105448 | |
| - type: cos_sim_spearman | |
| value: 65.62968801135676 | |
| - type: euclidean_pearson | |
| value: 47.482076613243905 | |
| - type: euclidean_spearman | |
| value: 62.65137791498299 | |
| - type: manhattan_pearson | |
| value: 47.57052626104093 | |
| - type: manhattan_spearman | |
| value: 62.436916516613294 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (fr) | |
| config: fr | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 70.49397298562575 | |
| - type: cos_sim_spearman | |
| value: 74.79604041187868 | |
| - type: euclidean_pearson | |
| value: 49.661891561317795 | |
| - type: euclidean_spearman | |
| value: 70.31535537621006 | |
| - type: manhattan_pearson | |
| value: 49.553715741850006 | |
| - type: manhattan_spearman | |
| value: 70.24779344636806 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (de-en) | |
| config: de-en | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 55.640574515348696 | |
| - type: cos_sim_spearman | |
| value: 54.927959317689 | |
| - type: euclidean_pearson | |
| value: 29.00139666967476 | |
| - type: euclidean_spearman | |
| value: 41.86386566971605 | |
| - type: manhattan_pearson | |
| value: 29.47411067730344 | |
| - type: manhattan_spearman | |
| value: 42.337438424952786 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (es-en) | |
| config: es-en | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 68.14095292259312 | |
| - type: cos_sim_spearman | |
| value: 73.99017581234789 | |
| - type: euclidean_pearson | |
| value: 46.46304297872084 | |
| - type: euclidean_spearman | |
| value: 60.91834114800041 | |
| - type: manhattan_pearson | |
| value: 47.07072666338692 | |
| - type: manhattan_spearman | |
| value: 61.70415727977926 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (it) | |
| config: it | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 73.27184653359575 | |
| - type: cos_sim_spearman | |
| value: 77.76070252418626 | |
| - type: euclidean_pearson | |
| value: 62.30586577544778 | |
| - type: euclidean_spearman | |
| value: 75.14246629110978 | |
| - type: manhattan_pearson | |
| value: 62.328196884927046 | |
| - type: manhattan_spearman | |
| value: 75.1282792981433 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (pl-en) | |
| config: pl-en | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 71.59448528829957 | |
| - type: cos_sim_spearman | |
| value: 70.37277734222123 | |
| - type: euclidean_pearson | |
| value: 57.63145565721123 | |
| - type: euclidean_spearman | |
| value: 66.10113048304427 | |
| - type: manhattan_pearson | |
| value: 57.18897811586808 | |
| - type: manhattan_spearman | |
| value: 66.5595511215901 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (zh-en) | |
| config: zh-en | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 66.37520607720838 | |
| - type: cos_sim_spearman | |
| value: 69.92282148997948 | |
| - type: euclidean_pearson | |
| value: 40.55768770125291 | |
| - type: euclidean_spearman | |
| value: 55.189128944669605 | |
| - type: manhattan_pearson | |
| value: 41.03566433468883 | |
| - type: manhattan_spearman | |
| value: 55.61251893174558 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (es-it) | |
| config: es-it | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 57.791929533771835 | |
| - type: cos_sim_spearman | |
| value: 66.45819707662093 | |
| - type: euclidean_pearson | |
| value: 39.03686018511092 | |
| - type: euclidean_spearman | |
| value: 56.01282695640428 | |
| - type: manhattan_pearson | |
| value: 38.91586623619632 | |
| - type: manhattan_spearman | |
| value: 56.69394943612747 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (de-fr) | |
| config: de-fr | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 47.82224468473866 | |
| - type: cos_sim_spearman | |
| value: 59.467307194781164 | |
| - type: euclidean_pearson | |
| value: 27.428459190256145 | |
| - type: euclidean_spearman | |
| value: 60.83463107397519 | |
| - type: manhattan_pearson | |
| value: 27.487391578496638 | |
| - type: manhattan_spearman | |
| value: 61.281380460246496 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (de-pl) | |
| config: de-pl | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 16.306666792752644 | |
| - type: cos_sim_spearman | |
| value: 39.35486427252405 | |
| - type: euclidean_pearson | |
| value: -2.7887154897955435 | |
| - type: euclidean_spearman | |
| value: 27.1296051831719 | |
| - type: manhattan_pearson | |
| value: -3.202291270581297 | |
| - type: manhattan_spearman | |
| value: 26.32895849218158 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/sts22-crosslingual-sts | |
| name: MTEB STS22 (fr-pl) | |
| config: fr-pl | |
| split: test | |
| revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 59.67006803805076 | |
| - type: cos_sim_spearman | |
| value: 73.24670207647144 | |
| - type: euclidean_pearson | |
| value: 46.91884681500483 | |
| - type: euclidean_spearman | |
| value: 16.903085094570333 | |
| - type: manhattan_pearson | |
| value: 46.88391675325812 | |
| - type: manhattan_spearman | |
| value: 28.17180849095055 | |
| - task: | |
| type: STS | |
| dataset: | |
| type: mteb/stsbenchmark-sts | |
| name: MTEB STSBenchmark | |
| config: default | |
| split: test | |
| revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 83.79555591223837 | |
| - type: cos_sim_spearman | |
| value: 85.63658602085185 | |
| - type: euclidean_pearson | |
| value: 85.22080894037671 | |
| - type: euclidean_spearman | |
| value: 85.54113580167038 | |
| - type: manhattan_pearson | |
| value: 85.1639505960118 | |
| - type: manhattan_spearman | |
| value: 85.43502665436196 | |
| - task: | |
| type: Reranking | |
| dataset: | |
| type: mteb/scidocs-reranking | |
| name: MTEB SciDocsRR | |
| config: default | |
| split: test | |
| revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab | |
| metrics: | |
| - type: map | |
| value: 80.73900991689766 | |
| - type: mrr | |
| value: 94.81624131133934 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: scifact | |
| name: MTEB SciFact | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 55.678000000000004 | |
| - type: map_at_10 | |
| value: 65.135 | |
| - type: map_at_100 | |
| value: 65.824 | |
| - type: map_at_1000 | |
| value: 65.852 | |
| - type: map_at_3 | |
| value: 62.736000000000004 | |
| - type: map_at_5 | |
| value: 64.411 | |
| - type: mrr_at_1 | |
| value: 58.333 | |
| - type: mrr_at_10 | |
| value: 66.5 | |
| - type: mrr_at_100 | |
| value: 67.053 | |
| - type: mrr_at_1000 | |
| value: 67.08 | |
| - type: mrr_at_3 | |
| value: 64.944 | |
| - type: mrr_at_5 | |
| value: 65.89399999999999 | |
| - type: ndcg_at_1 | |
| value: 58.333 | |
| - type: ndcg_at_10 | |
| value: 69.34700000000001 | |
| - type: ndcg_at_100 | |
| value: 72.32 | |
| - type: ndcg_at_1000 | |
| value: 73.014 | |
| - type: ndcg_at_3 | |
| value: 65.578 | |
| - type: ndcg_at_5 | |
| value: 67.738 | |
| - type: precision_at_1 | |
| value: 58.333 | |
| - type: precision_at_10 | |
| value: 9.033 | |
| - type: precision_at_100 | |
| value: 1.0670000000000002 | |
| - type: precision_at_1000 | |
| value: 0.11199999999999999 | |
| - type: precision_at_3 | |
| value: 25.444 | |
| - type: precision_at_5 | |
| value: 16.933 | |
| - type: recall_at_1 | |
| value: 55.678000000000004 | |
| - type: recall_at_10 | |
| value: 80.72200000000001 | |
| - type: recall_at_100 | |
| value: 93.93299999999999 | |
| - type: recall_at_1000 | |
| value: 99.333 | |
| - type: recall_at_3 | |
| value: 70.783 | |
| - type: recall_at_5 | |
| value: 75.978 | |
| - task: | |
| type: PairClassification | |
| dataset: | |
| type: mteb/sprintduplicatequestions-pairclassification | |
| name: MTEB SprintDuplicateQuestions | |
| config: default | |
| split: test | |
| revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 | |
| metrics: | |
| - type: cos_sim_accuracy | |
| value: 99.74653465346535 | |
| - type: cos_sim_ap | |
| value: 93.01476369929063 | |
| - type: cos_sim_f1 | |
| value: 86.93009118541033 | |
| - type: cos_sim_precision | |
| value: 88.09034907597535 | |
| - type: cos_sim_recall | |
| value: 85.8 | |
| - type: dot_accuracy | |
| value: 99.22970297029703 | |
| - type: dot_ap | |
| value: 51.58725659485144 | |
| - type: dot_f1 | |
| value: 53.51351351351352 | |
| - type: dot_precision | |
| value: 58.235294117647065 | |
| - type: dot_recall | |
| value: 49.5 | |
| - type: euclidean_accuracy | |
| value: 99.74356435643564 | |
| - type: euclidean_ap | |
| value: 92.40332894384368 | |
| - type: euclidean_f1 | |
| value: 86.97838109602817 | |
| - type: euclidean_precision | |
| value: 87.46208291203236 | |
| - type: euclidean_recall | |
| value: 86.5 | |
| - type: manhattan_accuracy | |
| value: 99.73069306930694 | |
| - type: manhattan_ap | |
| value: 92.01320815721121 | |
| - type: manhattan_f1 | |
| value: 86.4135864135864 | |
| - type: manhattan_precision | |
| value: 86.32734530938124 | |
| - type: manhattan_recall | |
| value: 86.5 | |
| - type: max_accuracy | |
| value: 99.74653465346535 | |
| - type: max_ap | |
| value: 93.01476369929063 | |
| - type: max_f1 | |
| value: 86.97838109602817 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/stackexchange-clustering | |
| name: MTEB StackExchangeClustering | |
| config: default | |
| split: test | |
| revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 | |
| metrics: | |
| - type: v_measure | |
| value: 55.2660514302523 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/stackexchange-clustering-p2p | |
| name: MTEB StackExchangeClusteringP2P | |
| config: default | |
| split: test | |
| revision: 815ca46b2622cec33ccafc3735d572c266efdb44 | |
| metrics: | |
| - type: v_measure | |
| value: 30.4637783572547 | |
| - task: | |
| type: Reranking | |
| dataset: | |
| type: mteb/stackoverflowdupquestions-reranking | |
| name: MTEB StackOverflowDupQuestions | |
| config: default | |
| split: test | |
| revision: e185fbe320c72810689fc5848eb6114e1ef5ec69 | |
| metrics: | |
| - type: map | |
| value: 49.41377758357637 | |
| - type: mrr | |
| value: 50.138451213818854 | |
| - task: | |
| type: Summarization | |
| dataset: | |
| type: mteb/summeval | |
| name: MTEB SummEval | |
| config: default | |
| split: test | |
| revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c | |
| metrics: | |
| - type: cos_sim_pearson | |
| value: 28.887846011166594 | |
| - type: cos_sim_spearman | |
| value: 30.10823258355903 | |
| - type: dot_pearson | |
| value: 12.888049550236385 | |
| - type: dot_spearman | |
| value: 12.827495903098123 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: trec-covid | |
| name: MTEB TRECCOVID | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 0.21 | |
| - type: map_at_10 | |
| value: 1.667 | |
| - type: map_at_100 | |
| value: 9.15 | |
| - type: map_at_1000 | |
| value: 22.927 | |
| - type: map_at_3 | |
| value: 0.573 | |
| - type: map_at_5 | |
| value: 0.915 | |
| - type: mrr_at_1 | |
| value: 80 | |
| - type: mrr_at_10 | |
| value: 87.167 | |
| - type: mrr_at_100 | |
| value: 87.167 | |
| - type: mrr_at_1000 | |
| value: 87.167 | |
| - type: mrr_at_3 | |
| value: 85.667 | |
| - type: mrr_at_5 | |
| value: 87.167 | |
| - type: ndcg_at_1 | |
| value: 76 | |
| - type: ndcg_at_10 | |
| value: 69.757 | |
| - type: ndcg_at_100 | |
| value: 52.402 | |
| - type: ndcg_at_1000 | |
| value: 47.737 | |
| - type: ndcg_at_3 | |
| value: 71.866 | |
| - type: ndcg_at_5 | |
| value: 72.225 | |
| - type: precision_at_1 | |
| value: 80 | |
| - type: precision_at_10 | |
| value: 75 | |
| - type: precision_at_100 | |
| value: 53.959999999999994 | |
| - type: precision_at_1000 | |
| value: 21.568 | |
| - type: precision_at_3 | |
| value: 76.667 | |
| - type: precision_at_5 | |
| value: 78 | |
| - type: recall_at_1 | |
| value: 0.21 | |
| - type: recall_at_10 | |
| value: 1.9189999999999998 | |
| - type: recall_at_100 | |
| value: 12.589 | |
| - type: recall_at_1000 | |
| value: 45.312000000000005 | |
| - type: recall_at_3 | |
| value: 0.61 | |
| - type: recall_at_5 | |
| value: 1.019 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (sqi-eng) | |
| config: sqi-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 92.10000000000001 | |
| - type: f1 | |
| value: 90.06 | |
| - type: precision | |
| value: 89.17333333333333 | |
| - type: recall | |
| value: 92.10000000000001 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (fry-eng) | |
| config: fry-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 56.06936416184971 | |
| - type: f1 | |
| value: 50.87508028259473 | |
| - type: precision | |
| value: 48.97398843930635 | |
| - type: recall | |
| value: 56.06936416184971 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (kur-eng) | |
| config: kur-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 57.3170731707317 | |
| - type: f1 | |
| value: 52.96080139372822 | |
| - type: precision | |
| value: 51.67861124382864 | |
| - type: recall | |
| value: 57.3170731707317 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tur-eng) | |
| config: tur-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.3 | |
| - type: f1 | |
| value: 92.67333333333333 | |
| - type: precision | |
| value: 91.90833333333333 | |
| - type: recall | |
| value: 94.3 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (deu-eng) | |
| config: deu-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 97.7 | |
| - type: f1 | |
| value: 97.07333333333332 | |
| - type: precision | |
| value: 96.79500000000002 | |
| - type: recall | |
| value: 97.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (nld-eng) | |
| config: nld-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.69999999999999 | |
| - type: f1 | |
| value: 93.2 | |
| - type: precision | |
| value: 92.48333333333333 | |
| - type: recall | |
| value: 94.69999999999999 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ron-eng) | |
| config: ron-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 92.9 | |
| - type: f1 | |
| value: 91.26666666666667 | |
| - type: precision | |
| value: 90.59444444444445 | |
| - type: recall | |
| value: 92.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ang-eng) | |
| config: ang-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 34.32835820895522 | |
| - type: f1 | |
| value: 29.074180380150533 | |
| - type: precision | |
| value: 28.068207322920596 | |
| - type: recall | |
| value: 34.32835820895522 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ido-eng) | |
| config: ido-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 78.5 | |
| - type: f1 | |
| value: 74.3945115995116 | |
| - type: precision | |
| value: 72.82967843459222 | |
| - type: recall | |
| value: 78.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (jav-eng) | |
| config: jav-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 66.34146341463415 | |
| - type: f1 | |
| value: 61.2469400518181 | |
| - type: precision | |
| value: 59.63977756660683 | |
| - type: recall | |
| value: 66.34146341463415 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (isl-eng) | |
| config: isl-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 80.9 | |
| - type: f1 | |
| value: 76.90349206349207 | |
| - type: precision | |
| value: 75.32921568627451 | |
| - type: recall | |
| value: 80.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (slv-eng) | |
| config: slv-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 84.93317132442284 | |
| - type: f1 | |
| value: 81.92519105034295 | |
| - type: precision | |
| value: 80.71283920615635 | |
| - type: recall | |
| value: 84.93317132442284 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (cym-eng) | |
| config: cym-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 71.1304347826087 | |
| - type: f1 | |
| value: 65.22394755003451 | |
| - type: precision | |
| value: 62.912422360248435 | |
| - type: recall | |
| value: 71.1304347826087 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (kaz-eng) | |
| config: kaz-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 79.82608695652173 | |
| - type: f1 | |
| value: 75.55693581780538 | |
| - type: precision | |
| value: 73.79420289855072 | |
| - type: recall | |
| value: 79.82608695652173 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (est-eng) | |
| config: est-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 74 | |
| - type: f1 | |
| value: 70.51022222222223 | |
| - type: precision | |
| value: 69.29673599347512 | |
| - type: recall | |
| value: 74 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (heb-eng) | |
| config: heb-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 78.7 | |
| - type: f1 | |
| value: 74.14238095238095 | |
| - type: precision | |
| value: 72.27214285714285 | |
| - type: recall | |
| value: 78.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (gla-eng) | |
| config: gla-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 48.97466827503016 | |
| - type: f1 | |
| value: 43.080330405420874 | |
| - type: precision | |
| value: 41.36505499593557 | |
| - type: recall | |
| value: 48.97466827503016 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (mar-eng) | |
| config: mar-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 89.60000000000001 | |
| - type: f1 | |
| value: 86.62333333333333 | |
| - type: precision | |
| value: 85.225 | |
| - type: recall | |
| value: 89.60000000000001 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (lat-eng) | |
| config: lat-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 45.2 | |
| - type: f1 | |
| value: 39.5761253006253 | |
| - type: precision | |
| value: 37.991358436312 | |
| - type: recall | |
| value: 45.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (bel-eng) | |
| config: bel-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 89.5 | |
| - type: f1 | |
| value: 86.70333333333333 | |
| - type: precision | |
| value: 85.53166666666667 | |
| - type: recall | |
| value: 89.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (pms-eng) | |
| config: pms-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 50.095238095238095 | |
| - type: f1 | |
| value: 44.60650460650461 | |
| - type: precision | |
| value: 42.774116796477045 | |
| - type: recall | |
| value: 50.095238095238095 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (gle-eng) | |
| config: gle-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 63.4 | |
| - type: f1 | |
| value: 58.35967261904762 | |
| - type: precision | |
| value: 56.54857142857143 | |
| - type: recall | |
| value: 63.4 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (pes-eng) | |
| config: pes-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 89.2 | |
| - type: f1 | |
| value: 87.075 | |
| - type: precision | |
| value: 86.12095238095239 | |
| - type: recall | |
| value: 89.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (nob-eng) | |
| config: nob-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 96.8 | |
| - type: f1 | |
| value: 95.90333333333334 | |
| - type: precision | |
| value: 95.50833333333333 | |
| - type: recall | |
| value: 96.8 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (bul-eng) | |
| config: bul-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 90.9 | |
| - type: f1 | |
| value: 88.6288888888889 | |
| - type: precision | |
| value: 87.61607142857142 | |
| - type: recall | |
| value: 90.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (cbk-eng) | |
| config: cbk-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 65.2 | |
| - type: f1 | |
| value: 60.54377630539395 | |
| - type: precision | |
| value: 58.89434482711381 | |
| - type: recall | |
| value: 65.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (hun-eng) | |
| config: hun-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 87 | |
| - type: f1 | |
| value: 84.32412698412699 | |
| - type: precision | |
| value: 83.25527777777778 | |
| - type: recall | |
| value: 87 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (uig-eng) | |
| config: uig-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 68.7 | |
| - type: f1 | |
| value: 63.07883541295306 | |
| - type: precision | |
| value: 61.06117424242426 | |
| - type: recall | |
| value: 68.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (rus-eng) | |
| config: rus-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 93.7 | |
| - type: f1 | |
| value: 91.78333333333335 | |
| - type: precision | |
| value: 90.86666666666667 | |
| - type: recall | |
| value: 93.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (spa-eng) | |
| config: spa-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 97.7 | |
| - type: f1 | |
| value: 96.96666666666667 | |
| - type: precision | |
| value: 96.61666666666667 | |
| - type: recall | |
| value: 97.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (hye-eng) | |
| config: hye-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 88.27493261455525 | |
| - type: f1 | |
| value: 85.90745732255168 | |
| - type: precision | |
| value: 84.91389637616052 | |
| - type: recall | |
| value: 88.27493261455525 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tel-eng) | |
| config: tel-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 90.5982905982906 | |
| - type: f1 | |
| value: 88.4900284900285 | |
| - type: precision | |
| value: 87.57122507122507 | |
| - type: recall | |
| value: 90.5982905982906 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (afr-eng) | |
| config: afr-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 89.5 | |
| - type: f1 | |
| value: 86.90769841269842 | |
| - type: precision | |
| value: 85.80178571428571 | |
| - type: recall | |
| value: 89.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (mon-eng) | |
| config: mon-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 82.5 | |
| - type: f1 | |
| value: 78.36796536796538 | |
| - type: precision | |
| value: 76.82196969696969 | |
| - type: recall | |
| value: 82.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (arz-eng) | |
| config: arz-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 71.48846960167715 | |
| - type: f1 | |
| value: 66.78771089148448 | |
| - type: precision | |
| value: 64.98302885095339 | |
| - type: recall | |
| value: 71.48846960167715 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (hrv-eng) | |
| config: hrv-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.1 | |
| - type: f1 | |
| value: 92.50333333333333 | |
| - type: precision | |
| value: 91.77499999999999 | |
| - type: recall | |
| value: 94.1 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (nov-eng) | |
| config: nov-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 71.20622568093385 | |
| - type: f1 | |
| value: 66.83278891450098 | |
| - type: precision | |
| value: 65.35065777283677 | |
| - type: recall | |
| value: 71.20622568093385 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (gsw-eng) | |
| config: gsw-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 48.717948717948715 | |
| - type: f1 | |
| value: 43.53146853146853 | |
| - type: precision | |
| value: 42.04721204721204 | |
| - type: recall | |
| value: 48.717948717948715 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (nds-eng) | |
| config: nds-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 58.5 | |
| - type: f1 | |
| value: 53.8564991863928 | |
| - type: precision | |
| value: 52.40329436122275 | |
| - type: recall | |
| value: 58.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ukr-eng) | |
| config: ukr-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 90.8 | |
| - type: f1 | |
| value: 88.29 | |
| - type: precision | |
| value: 87.09166666666667 | |
| - type: recall | |
| value: 90.8 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (uzb-eng) | |
| config: uzb-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 67.28971962616822 | |
| - type: f1 | |
| value: 62.63425307817832 | |
| - type: precision | |
| value: 60.98065939771546 | |
| - type: recall | |
| value: 67.28971962616822 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (lit-eng) | |
| config: lit-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 78.7 | |
| - type: f1 | |
| value: 75.5264472455649 | |
| - type: precision | |
| value: 74.38205086580086 | |
| - type: recall | |
| value: 78.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ina-eng) | |
| config: ina-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 88.7 | |
| - type: f1 | |
| value: 86.10809523809525 | |
| - type: precision | |
| value: 85.07602564102565 | |
| - type: recall | |
| value: 88.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (lfn-eng) | |
| config: lfn-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 56.99999999999999 | |
| - type: f1 | |
| value: 52.85487521402737 | |
| - type: precision | |
| value: 51.53985162713104 | |
| - type: recall | |
| value: 56.99999999999999 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (zsm-eng) | |
| config: zsm-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94 | |
| - type: f1 | |
| value: 92.45333333333333 | |
| - type: precision | |
| value: 91.79166666666667 | |
| - type: recall | |
| value: 94 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ita-eng) | |
| config: ita-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 92.30000000000001 | |
| - type: f1 | |
| value: 90.61333333333333 | |
| - type: precision | |
| value: 89.83333333333331 | |
| - type: recall | |
| value: 92.30000000000001 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (cmn-eng) | |
| config: cmn-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.69999999999999 | |
| - type: f1 | |
| value: 93.34555555555555 | |
| - type: precision | |
| value: 92.75416666666668 | |
| - type: recall | |
| value: 94.69999999999999 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (lvs-eng) | |
| config: lvs-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 80.2 | |
| - type: f1 | |
| value: 76.6563035113035 | |
| - type: precision | |
| value: 75.3014652014652 | |
| - type: recall | |
| value: 80.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (glg-eng) | |
| config: glg-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 84.7 | |
| - type: f1 | |
| value: 82.78689263765207 | |
| - type: precision | |
| value: 82.06705086580087 | |
| - type: recall | |
| value: 84.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ceb-eng) | |
| config: ceb-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 50.33333333333333 | |
| - type: f1 | |
| value: 45.461523661523664 | |
| - type: precision | |
| value: 43.93545574795575 | |
| - type: recall | |
| value: 50.33333333333333 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (bre-eng) | |
| config: bre-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 6.6000000000000005 | |
| - type: f1 | |
| value: 5.442121400446441 | |
| - type: precision | |
| value: 5.146630385487529 | |
| - type: recall | |
| value: 6.6000000000000005 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ben-eng) | |
| config: ben-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 85 | |
| - type: f1 | |
| value: 81.04666666666667 | |
| - type: precision | |
| value: 79.25 | |
| - type: recall | |
| value: 85 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (swg-eng) | |
| config: swg-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 47.32142857142857 | |
| - type: f1 | |
| value: 42.333333333333336 | |
| - type: precision | |
| value: 40.69196428571429 | |
| - type: recall | |
| value: 47.32142857142857 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (arq-eng) | |
| config: arq-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 30.735455543358945 | |
| - type: f1 | |
| value: 26.73616790022338 | |
| - type: precision | |
| value: 25.397823220451283 | |
| - type: recall | |
| value: 30.735455543358945 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (kab-eng) | |
| config: kab-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 25.1 | |
| - type: f1 | |
| value: 21.975989896371022 | |
| - type: precision | |
| value: 21.059885632257203 | |
| - type: recall | |
| value: 25.1 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (fra-eng) | |
| config: fra-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.3 | |
| - type: f1 | |
| value: 92.75666666666666 | |
| - type: precision | |
| value: 92.06166666666665 | |
| - type: recall | |
| value: 94.3 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (por-eng) | |
| config: por-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.1 | |
| - type: f1 | |
| value: 92.74 | |
| - type: precision | |
| value: 92.09166666666667 | |
| - type: recall | |
| value: 94.1 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tat-eng) | |
| config: tat-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 71.3 | |
| - type: f1 | |
| value: 66.922442002442 | |
| - type: precision | |
| value: 65.38249567099568 | |
| - type: recall | |
| value: 71.3 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (oci-eng) | |
| config: oci-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 40.300000000000004 | |
| - type: f1 | |
| value: 35.78682789299971 | |
| - type: precision | |
| value: 34.66425128716588 | |
| - type: recall | |
| value: 40.300000000000004 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (pol-eng) | |
| config: pol-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 96 | |
| - type: f1 | |
| value: 94.82333333333334 | |
| - type: precision | |
| value: 94.27833333333334 | |
| - type: recall | |
| value: 96 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (war-eng) | |
| config: war-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 51.1 | |
| - type: f1 | |
| value: 47.179074753133584 | |
| - type: precision | |
| value: 46.06461044702424 | |
| - type: recall | |
| value: 51.1 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (aze-eng) | |
| config: aze-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 87.7 | |
| - type: f1 | |
| value: 84.71 | |
| - type: precision | |
| value: 83.46166666666667 | |
| - type: recall | |
| value: 87.7 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (vie-eng) | |
| config: vie-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 95.8 | |
| - type: f1 | |
| value: 94.68333333333334 | |
| - type: precision | |
| value: 94.13333333333334 | |
| - type: recall | |
| value: 95.8 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (nno-eng) | |
| config: nno-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 85.39999999999999 | |
| - type: f1 | |
| value: 82.5577380952381 | |
| - type: precision | |
| value: 81.36833333333334 | |
| - type: recall | |
| value: 85.39999999999999 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (cha-eng) | |
| config: cha-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 21.16788321167883 | |
| - type: f1 | |
| value: 16.948865627297987 | |
| - type: precision | |
| value: 15.971932568647897 | |
| - type: recall | |
| value: 21.16788321167883 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (mhr-eng) | |
| config: mhr-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 6.9 | |
| - type: f1 | |
| value: 5.515526831658907 | |
| - type: precision | |
| value: 5.141966366966367 | |
| - type: recall | |
| value: 6.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (dan-eng) | |
| config: dan-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 93.2 | |
| - type: f1 | |
| value: 91.39666666666668 | |
| - type: precision | |
| value: 90.58666666666667 | |
| - type: recall | |
| value: 93.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ell-eng) | |
| config: ell-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 92.2 | |
| - type: f1 | |
| value: 89.95666666666666 | |
| - type: precision | |
| value: 88.92833333333333 | |
| - type: recall | |
| value: 92.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (amh-eng) | |
| config: amh-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 79.76190476190477 | |
| - type: f1 | |
| value: 74.93386243386244 | |
| - type: precision | |
| value: 73.11011904761904 | |
| - type: recall | |
| value: 79.76190476190477 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (pam-eng) | |
| config: pam-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 8.799999999999999 | |
| - type: f1 | |
| value: 6.921439712248537 | |
| - type: precision | |
| value: 6.489885109680683 | |
| - type: recall | |
| value: 8.799999999999999 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (hsb-eng) | |
| config: hsb-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 45.75569358178054 | |
| - type: f1 | |
| value: 40.34699501312631 | |
| - type: precision | |
| value: 38.57886764719063 | |
| - type: recall | |
| value: 45.75569358178054 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (srp-eng) | |
| config: srp-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 91.4 | |
| - type: f1 | |
| value: 89.08333333333333 | |
| - type: precision | |
| value: 88.01666666666668 | |
| - type: recall | |
| value: 91.4 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (epo-eng) | |
| config: epo-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 93.60000000000001 | |
| - type: f1 | |
| value: 92.06690476190477 | |
| - type: precision | |
| value: 91.45095238095239 | |
| - type: recall | |
| value: 93.60000000000001 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (kzj-eng) | |
| config: kzj-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 7.5 | |
| - type: f1 | |
| value: 6.200363129378736 | |
| - type: precision | |
| value: 5.89115314822466 | |
| - type: recall | |
| value: 7.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (awa-eng) | |
| config: awa-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 73.59307359307358 | |
| - type: f1 | |
| value: 68.38933553219267 | |
| - type: precision | |
| value: 66.62698412698413 | |
| - type: recall | |
| value: 73.59307359307358 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (fao-eng) | |
| config: fao-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 69.8473282442748 | |
| - type: f1 | |
| value: 64.72373682297346 | |
| - type: precision | |
| value: 62.82834214131924 | |
| - type: recall | |
| value: 69.8473282442748 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (mal-eng) | |
| config: mal-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 97.5254730713246 | |
| - type: f1 | |
| value: 96.72489082969432 | |
| - type: precision | |
| value: 96.33672974284326 | |
| - type: recall | |
| value: 97.5254730713246 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ile-eng) | |
| config: ile-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 75.6 | |
| - type: f1 | |
| value: 72.42746031746033 | |
| - type: precision | |
| value: 71.14036630036631 | |
| - type: recall | |
| value: 75.6 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (bos-eng) | |
| config: bos-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 91.24293785310734 | |
| - type: f1 | |
| value: 88.86064030131826 | |
| - type: precision | |
| value: 87.73540489642184 | |
| - type: recall | |
| value: 91.24293785310734 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (cor-eng) | |
| config: cor-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 6.2 | |
| - type: f1 | |
| value: 4.383083659794954 | |
| - type: precision | |
| value: 4.027861324289673 | |
| - type: recall | |
| value: 6.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (cat-eng) | |
| config: cat-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 86.8 | |
| - type: f1 | |
| value: 84.09428571428572 | |
| - type: precision | |
| value: 83.00333333333333 | |
| - type: recall | |
| value: 86.8 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (eus-eng) | |
| config: eus-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 60.699999999999996 | |
| - type: f1 | |
| value: 56.1584972394755 | |
| - type: precision | |
| value: 54.713456330903135 | |
| - type: recall | |
| value: 60.699999999999996 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (yue-eng) | |
| config: yue-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 84.2 | |
| - type: f1 | |
| value: 80.66190476190475 | |
| - type: precision | |
| value: 79.19690476190476 | |
| - type: recall | |
| value: 84.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (swe-eng) | |
| config: swe-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 93.2 | |
| - type: f1 | |
| value: 91.33 | |
| - type: precision | |
| value: 90.45 | |
| - type: recall | |
| value: 93.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (dtp-eng) | |
| config: dtp-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 6.3 | |
| - type: f1 | |
| value: 5.126828976748276 | |
| - type: precision | |
| value: 4.853614328966668 | |
| - type: recall | |
| value: 6.3 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (kat-eng) | |
| config: kat-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 81.76943699731903 | |
| - type: f1 | |
| value: 77.82873739308057 | |
| - type: precision | |
| value: 76.27622452019234 | |
| - type: recall | |
| value: 81.76943699731903 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (jpn-eng) | |
| config: jpn-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 92.30000000000001 | |
| - type: f1 | |
| value: 90.29666666666665 | |
| - type: precision | |
| value: 89.40333333333334 | |
| - type: recall | |
| value: 92.30000000000001 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (csb-eng) | |
| config: csb-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 29.249011857707508 | |
| - type: f1 | |
| value: 24.561866096392947 | |
| - type: precision | |
| value: 23.356583740215456 | |
| - type: recall | |
| value: 29.249011857707508 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (xho-eng) | |
| config: xho-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 77.46478873239437 | |
| - type: f1 | |
| value: 73.23943661971832 | |
| - type: precision | |
| value: 71.66666666666667 | |
| - type: recall | |
| value: 77.46478873239437 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (orv-eng) | |
| config: orv-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 20.35928143712575 | |
| - type: f1 | |
| value: 15.997867865075824 | |
| - type: precision | |
| value: 14.882104658301346 | |
| - type: recall | |
| value: 20.35928143712575 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ind-eng) | |
| config: ind-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 92.2 | |
| - type: f1 | |
| value: 90.25999999999999 | |
| - type: precision | |
| value: 89.45333333333335 | |
| - type: recall | |
| value: 92.2 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tuk-eng) | |
| config: tuk-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 23.15270935960591 | |
| - type: f1 | |
| value: 19.65673625772148 | |
| - type: precision | |
| value: 18.793705293464992 | |
| - type: recall | |
| value: 23.15270935960591 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (max-eng) | |
| config: max-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 59.154929577464785 | |
| - type: f1 | |
| value: 52.3868463305083 | |
| - type: precision | |
| value: 50.14938113529662 | |
| - type: recall | |
| value: 59.154929577464785 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (swh-eng) | |
| config: swh-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 70.51282051282051 | |
| - type: f1 | |
| value: 66.8089133089133 | |
| - type: precision | |
| value: 65.37645687645687 | |
| - type: recall | |
| value: 70.51282051282051 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (hin-eng) | |
| config: hin-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 94.6 | |
| - type: f1 | |
| value: 93 | |
| - type: precision | |
| value: 92.23333333333333 | |
| - type: recall | |
| value: 94.6 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (dsb-eng) | |
| config: dsb-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 38.62212943632568 | |
| - type: f1 | |
| value: 34.3278276962583 | |
| - type: precision | |
| value: 33.07646935732408 | |
| - type: recall | |
| value: 38.62212943632568 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ber-eng) | |
| config: ber-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 28.1 | |
| - type: f1 | |
| value: 23.579609223054604 | |
| - type: precision | |
| value: 22.39622774921555 | |
| - type: recall | |
| value: 28.1 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tam-eng) | |
| config: tam-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 88.27361563517914 | |
| - type: f1 | |
| value: 85.12486427795874 | |
| - type: precision | |
| value: 83.71335504885994 | |
| - type: recall | |
| value: 88.27361563517914 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (slk-eng) | |
| config: slk-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 88.6 | |
| - type: f1 | |
| value: 86.39928571428571 | |
| - type: precision | |
| value: 85.4947557997558 | |
| - type: recall | |
| value: 88.6 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tgl-eng) | |
| config: tgl-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 86.5 | |
| - type: f1 | |
| value: 83.77952380952381 | |
| - type: precision | |
| value: 82.67602564102565 | |
| - type: recall | |
| value: 86.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ast-eng) | |
| config: ast-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 79.52755905511812 | |
| - type: f1 | |
| value: 75.3055868016498 | |
| - type: precision | |
| value: 73.81889763779527 | |
| - type: recall | |
| value: 79.52755905511812 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (mkd-eng) | |
| config: mkd-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 77.9 | |
| - type: f1 | |
| value: 73.76261904761905 | |
| - type: precision | |
| value: 72.11670995670995 | |
| - type: recall | |
| value: 77.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (khm-eng) | |
| config: khm-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 53.8781163434903 | |
| - type: f1 | |
| value: 47.25804051288816 | |
| - type: precision | |
| value: 45.0603482390186 | |
| - type: recall | |
| value: 53.8781163434903 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ces-eng) | |
| config: ces-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 91.10000000000001 | |
| - type: f1 | |
| value: 88.88 | |
| - type: precision | |
| value: 87.96333333333334 | |
| - type: recall | |
| value: 91.10000000000001 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tzl-eng) | |
| config: tzl-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 38.46153846153847 | |
| - type: f1 | |
| value: 34.43978243978244 | |
| - type: precision | |
| value: 33.429487179487175 | |
| - type: recall | |
| value: 38.46153846153847 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (urd-eng) | |
| config: urd-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 88.9 | |
| - type: f1 | |
| value: 86.19888888888887 | |
| - type: precision | |
| value: 85.07440476190476 | |
| - type: recall | |
| value: 88.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (ara-eng) | |
| config: ara-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 85.9 | |
| - type: f1 | |
| value: 82.58857142857143 | |
| - type: precision | |
| value: 81.15666666666667 | |
| - type: recall | |
| value: 85.9 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (kor-eng) | |
| config: kor-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 86.8 | |
| - type: f1 | |
| value: 83.36999999999999 | |
| - type: precision | |
| value: 81.86833333333333 | |
| - type: recall | |
| value: 86.8 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (yid-eng) | |
| config: yid-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 68.51415094339622 | |
| - type: f1 | |
| value: 63.195000099481234 | |
| - type: precision | |
| value: 61.394033442972116 | |
| - type: recall | |
| value: 68.51415094339622 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (fin-eng) | |
| config: fin-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 88.5 | |
| - type: f1 | |
| value: 86.14603174603175 | |
| - type: precision | |
| value: 85.1162037037037 | |
| - type: recall | |
| value: 88.5 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (tha-eng) | |
| config: tha-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 95.62043795620438 | |
| - type: f1 | |
| value: 94.40389294403892 | |
| - type: precision | |
| value: 93.7956204379562 | |
| - type: recall | |
| value: 95.62043795620438 | |
| - task: | |
| type: BitextMining | |
| dataset: | |
| type: mteb/tatoeba-bitext-mining | |
| name: MTEB Tatoeba (wuu-eng) | |
| config: wuu-eng | |
| split: test | |
| revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 | |
| metrics: | |
| - type: accuracy | |
| value: 81.8 | |
| - type: f1 | |
| value: 78.6532178932179 | |
| - type: precision | |
| value: 77.46348795840176 | |
| - type: recall | |
| value: 81.8 | |
| - task: | |
| type: Retrieval | |
| dataset: | |
| type: webis-touche2020 | |
| name: MTEB Touche2020 | |
| config: default | |
| split: test | |
| revision: None | |
| metrics: | |
| - type: map_at_1 | |
| value: 2.603 | |
| - type: map_at_10 | |
| value: 8.5 | |
| - type: map_at_100 | |
| value: 12.985 | |
| - type: map_at_1000 | |
| value: 14.466999999999999 | |
| - type: map_at_3 | |
| value: 4.859999999999999 | |
| - type: map_at_5 | |
| value: 5.817 | |
| - type: mrr_at_1 | |
| value: 28.571 | |
| - type: mrr_at_10 | |
| value: 42.331 | |
| - type: mrr_at_100 | |
| value: 43.592999999999996 | |
| - type: mrr_at_1000 | |
| value: 43.592999999999996 | |
| - type: mrr_at_3 | |
| value: 38.435 | |
| - type: mrr_at_5 | |
| value: 39.966 | |
| - type: ndcg_at_1 | |
| value: 26.531 | |
| - type: ndcg_at_10 | |
| value: 21.353 | |
| - type: ndcg_at_100 | |
| value: 31.087999999999997 | |
| - type: ndcg_at_1000 | |
| value: 43.163000000000004 | |
| - type: ndcg_at_3 | |
| value: 22.999 | |
| - type: ndcg_at_5 | |
| value: 21.451 | |
| - type: precision_at_1 | |
| value: 28.571 | |
| - type: precision_at_10 | |
| value: 19.387999999999998 | |
| - type: precision_at_100 | |
| value: 6.265 | |
| - type: precision_at_1000 | |
| value: 1.4160000000000001 | |
| - type: precision_at_3 | |
| value: 24.490000000000002 | |
| - type: precision_at_5 | |
| value: 21.224 | |
| - type: recall_at_1 | |
| value: 2.603 | |
| - type: recall_at_10 | |
| value: 14.474 | |
| - type: recall_at_100 | |
| value: 40.287 | |
| - type: recall_at_1000 | |
| value: 76.606 | |
| - type: recall_at_3 | |
| value: 5.978 | |
| - type: recall_at_5 | |
| value: 7.819 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/toxic_conversations_50k | |
| name: MTEB ToxicConversationsClassification | |
| config: default | |
| split: test | |
| revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c | |
| metrics: | |
| - type: accuracy | |
| value: 69.7848 | |
| - type: ap | |
| value: 13.661023167088224 | |
| - type: f1 | |
| value: 53.61686134460943 | |
| - task: | |
| type: Classification | |
| dataset: | |
| type: mteb/tweet_sentiment_extraction | |
| name: MTEB TweetSentimentExtractionClassification | |
| config: default | |
| split: test | |
| revision: d604517c81ca91fe16a244d1248fc021f9ecee7a | |
| metrics: | |
| - type: accuracy | |
| value: 61.28183361629882 | |
| - type: f1 | |
| value: 61.55481034919965 | |
| - task: | |
| type: Clustering | |
| dataset: | |
| type: mteb/twentynewsgroups-clustering | |
| name: MTEB TwentyNewsgroupsClustering | |
| config: default | |
| split: test | |
| revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 | |
| metrics: | |
| - type: v_measure | |
| value: 35.972128420092396 | |
| - task: | |
| type: PairClassification | |
| dataset: | |
| type: mteb/twittersemeval2015-pairclassification | |
| name: MTEB TwitterSemEval2015 | |
| config: default | |
| split: test | |
| revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 | |
| metrics: | |
| - type: cos_sim_accuracy | |
| value: 85.59933241938367 | |
| - type: cos_sim_ap | |
| value: 72.20760361208136 | |
| - type: cos_sim_f1 | |
| value: 66.4447731755424 | |
| - type: cos_sim_precision | |
| value: 62.35539102267469 | |
| - type: cos_sim_recall | |
| value: 71.10817941952506 | |
| - type: dot_accuracy | |
| value: 78.98313166835548 | |
| - type: dot_ap | |
| value: 44.492521645493795 | |
| - type: dot_f1 | |
| value: 45.814889336016094 | |
| - type: dot_precision | |
| value: 37.02439024390244 | |
| - type: dot_recall | |
| value: 60.07915567282321 | |
| - type: euclidean_accuracy | |
| value: 85.3907134767837 | |
| - type: euclidean_ap | |
| value: 71.53847289080343 | |
| - type: euclidean_f1 | |
| value: 65.95952206778834 | |
| - type: euclidean_precision | |
| value: 61.31006346328196 | |
| - type: euclidean_recall | |
| value: 71.37203166226914 | |
| - type: manhattan_accuracy | |
| value: 85.40859510043511 | |
| - type: manhattan_ap | |
| value: 71.49664104395515 | |
| - type: manhattan_f1 | |
| value: 65.98569969356485 | |
| - type: manhattan_precision | |
| value: 63.928748144482924 | |
| - type: manhattan_recall | |
| value: 68.17941952506597 | |
| - type: max_accuracy | |
| value: 85.59933241938367 | |
| - type: max_ap | |
| value: 72.20760361208136 | |
| - type: max_f1 | |
| value: 66.4447731755424 | |
| - task: | |
| type: PairClassification | |
| dataset: | |
| type: mteb/twitterurlcorpus-pairclassification | |
| name: MTEB TwitterURLCorpus | |
| config: default | |
| split: test | |
| revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf | |
| metrics: | |
| - type: cos_sim_accuracy | |
| value: 88.83261536073273 | |
| - type: cos_sim_ap | |
| value: 85.48178133644264 | |
| - type: cos_sim_f1 | |
| value: 77.87816307403935 | |
| - type: cos_sim_precision | |
| value: 75.88953021114926 | |
| - type: cos_sim_recall | |
| value: 79.97382198952879 | |
| - type: dot_accuracy | |
| value: 79.76287499514883 | |
| - type: dot_ap | |
| value: 59.17438838475084 | |
| - type: dot_f1 | |
| value: 56.34566667855996 | |
| - type: dot_precision | |
| value: 52.50349092359864 | |
| - type: dot_recall | |
| value: 60.794579611949494 | |
| - type: euclidean_accuracy | |
| value: 88.76857996662397 | |
| - type: euclidean_ap | |
| value: 85.22764834359887 | |
| - type: euclidean_f1 | |
| value: 77.65379751543554 | |
| - type: euclidean_precision | |
| value: 75.11152683839401 | |
| - type: euclidean_recall | |
| value: 80.37419156144134 | |
| - type: manhattan_accuracy | |
| value: 88.6987231730508 | |
| - type: manhattan_ap | |
| value: 85.18907981724007 | |
| - type: manhattan_f1 | |
| value: 77.51967028849757 | |
| - type: manhattan_precision | |
| value: 75.49992701795358 | |
| - type: manhattan_recall | |
| value: 79.65044656606098 | |
| - type: max_accuracy | |
| value: 88.83261536073273 | |
| - type: max_ap | |
| value: 85.48178133644264 | |
| - type: max_f1 | |
| value: 77.87816307403935 | |
| language: | |
| - multilingual | |
| - af | |
| - am | |
| - ar | |
| - as | |
| - az | |
| - be | |
| - bg | |
| - bn | |
| - br | |
| - bs | |
| - ca | |
| - cs | |
| - cy | |
| - da | |
| - de | |
| - el | |
| - en | |
| - eo | |
| - es | |
| - et | |
| - eu | |
| - fa | |
| - fi | |
| - fr | |
| - fy | |
| - ga | |
| - gd | |
| - gl | |
| - gu | |
| - ha | |
| - he | |
| - hi | |
| - hr | |
| - hu | |
| - hy | |
| - id | |
| - is | |
| - it | |
| - ja | |
| - jv | |
| - ka | |
| - kk | |
| - km | |
| - kn | |
| - ko | |
| - ku | |
| - ky | |
| - la | |
| - lo | |
| - lt | |
| - lv | |
| - mg | |
| - mk | |
| - ml | |
| - mn | |
| - mr | |
| - ms | |
| - my | |
| - ne | |
| - nl | |
| - 'no' | |
| - om | |
| - or | |
| - pa | |
| - pl | |
| - ps | |
| - pt | |
| - ro | |
| - ru | |
| - sa | |
| - sd | |
| - si | |
| - sk | |
| - sl | |
| - so | |
| - sq | |
| - sr | |
| - su | |
| - sv | |
| - sw | |
| - ta | |
| - te | |
| - th | |
| - tl | |
| - tr | |
| - ug | |
| - uk | |
| - ur | |
| - uz | |
| - vi | |
| - xh | |
| - yi | |
| - zh | |
| license: mit | |
| ## Multilingual-E5-base | |
| [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://arxiv.org/pdf/2212.03533.pdf). | |
| Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022 | |
| This model has 12 layers and the embedding size is 768. | |
| ## Usage | |
| Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. | |
| ```python | |
| import torch.nn.functional as F | |
| from torch import Tensor | |
| from transformers import AutoTokenizer, AutoModel | |
| def average_pool(last_hidden_states: Tensor, | |
| attention_mask: Tensor) -> Tensor: | |
| last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0) | |
| return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None] | |
| # Each input text should start with "query: " or "passage: ", even for non-English texts. | |
| # For tasks other than retrieval, you can simply use the "query: " prefix. | |
| input_texts = ['query: how much protein should a female eat', | |
| 'query: 南瓜的家常做法', | |
| "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.", | |
| "passage: 1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右,放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅"] | |
| tokenizer = AutoTokenizer.from_pretrained('intfloat/multilingual-e5-base') | |
| model = AutoModel.from_pretrained('intfloat/multilingual-e5-base') | |
| # Tokenize the input texts | |
| batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt') | |
| outputs = model(**batch_dict) | |
| embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask']) | |
| # (Optionally) normalize embeddings | |
| embeddings = F.normalize(embeddings, p=2, dim=1) | |
| scores = (embeddings[:2] @ embeddings[2:].T) * 100 | |
| print(scores.tolist()) | |
| ``` | |
| ## Supported Languages | |
| This model is initialized from [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) | |
| and continually trained on a mixture of multilingual datasets. | |
| It supports 100 languages from xlm-roberta, | |
| but low-resource languages may see performance degradation. | |
| ## Training Details | |
| Please refer to our paper at [https://arxiv.org/pdf/2212.03533.pdf](https://arxiv.org/pdf/2212.03533.pdf). | |
| ## Benchmark Evaluation | |
| Check out [unilm/e5](https://github.com/microsoft/unilm/tree/master/e5) to reproduce evaluation results | |
| on the [BEIR](https://arxiv.org/abs/2104.08663) and [MTEB benchmark](https://arxiv.org/abs/2210.07316). | |
| ## Citation | |
| If you find our paper or models helpful, please consider cite as follows: | |
| ``` | |
| @article{wang2022text, | |
| title={Text Embeddings by Weakly-Supervised Contrastive Pre-training}, | |
| author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Jiao, Binxing and Yang, Linjun and Jiang, Daxin and Majumder, Rangan and Wei, Furu}, | |
| journal={arXiv preprint arXiv:2212.03533}, | |
| year={2022} | |
| } | |
| ``` | |
| ## Limitations | |
| Long texts will be truncated to at most 512 tokens. |