--- model-index: - name: brahmairesearch/slx-v0.1 results: - dataset: config: en-ext name: MTEB AmazonCounterfactualClassification (en-ext) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: test type: mteb/amazon_counterfactual metrics: - type: accuracy value: 65.59220389805098 - type: ap value: 16.81423101691947 - type: ap_weighted value: 16.81423101691947 - type: f1 value: 53.71729938392706 - type: f1_weighted value: 72.20850142432688 - type: main_score value: 65.59220389805098 task: type: Classification - dataset: config: en name: MTEB AmazonCounterfactualClassification (en) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: test type: mteb/amazon_counterfactual metrics: - type: accuracy value: 63.64179104477612 - type: ap value: 26.921335760715966 - type: ap_weighted value: 26.921335760715966 - type: f1 value: 57.64569779029121 - type: f1_weighted value: 67.27976461433454 - type: main_score value: 63.64179104477612 task: type: Classification - dataset: config: de name: MTEB AmazonCounterfactualClassification (de) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: test type: mteb/amazon_counterfactual metrics: - type: accuracy value: 57.81584582441114 - type: ap value: 73.96447293747234 - type: ap_weighted value: 73.96447293747234 - type: f1 value: 56.086772577787755 - type: f1_weighted value: 59.41839602591139 - type: main_score value: 57.81584582441114 task: type: Classification - dataset: config: ja name: MTEB AmazonCounterfactualClassification (ja) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: test type: mteb/amazon_counterfactual metrics: - type: accuracy value: 60.89935760171306 - type: ap value: 13.241905671485688 - type: ap_weighted value: 13.241905671485688 - type: f1 value: 48.847713021626774 - type: f1_weighted value: 68.43256893578159 - type: main_score value: 60.89935760171306 task: type: Classification - dataset: config: en-ext name: MTEB AmazonCounterfactualClassification (en-ext) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: validation type: mteb/amazon_counterfactual metrics: - type: accuracy value: 63.46846846846847 - type: ap value: 15.500820900257253 - type: ap_weighted value: 15.500820900257253 - type: f1 value: 51.829238493255424 - type: f1_weighted value: 70.67890974680999 - type: main_score value: 63.46846846846847 task: type: Classification - dataset: config: en name: MTEB AmazonCounterfactualClassification (en) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: validation type: mteb/amazon_counterfactual metrics: - type: accuracy value: 61.462686567164184 - type: ap value: 22.787749709369066 - type: ap_weighted value: 22.787749709369066 - type: f1 value: 54.398879015183766 - type: f1_weighted value: 66.03261824563178 - type: main_score value: 61.462686567164184 task: type: Classification - dataset: config: de name: MTEB AmazonCounterfactualClassification (de) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: validation type: mteb/amazon_counterfactual metrics: - type: accuracy value: 57.98283261802576 - type: ap value: 73.53358224916597 - type: ap_weighted value: 73.53358224916597 - type: f1 value: 55.63777177215615 - type: f1_weighted value: 59.597057376484194 - type: main_score value: 57.98283261802576 task: type: Classification - dataset: config: ja name: MTEB AmazonCounterfactualClassification (ja) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: validation type: mteb/amazon_counterfactual metrics: - type: accuracy value: 59.377682403433475 - type: ap value: 11.144115013790348 - type: ap_weighted value: 11.144115013790348 - type: f1 value: 46.238094801012494 - type: f1_weighted value: 67.41908331890755 - type: main_score value: 59.377682403433475 task: type: Classification - dataset: config: default name: MTEB AmazonPolarityClassification (default) revision: e2d317d38cd51312af73b3d32a06d1a08b442046 split: test type: mteb/amazon_polarity metrics: - type: accuracy value: 64.259025 - type: ap value: 59.266682504413595 - type: ap_weighted value: 59.266682504413595 - type: f1 value: 63.94155696126964 - type: f1_weighted value: 63.94155696126964 - type: main_score value: 64.259025 task: type: Classification - dataset: config: en name: MTEB AmazonReviewsClassification (en) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 30.848000000000003 - type: f1 value: 30.43009944052308 - type: f1_weighted value: 30.43009944052308 - type: main_score value: 30.848000000000003 task: type: Classification - dataset: config: de name: MTEB AmazonReviewsClassification (de) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 26.444000000000003 - type: f1 value: 26.08196367948359 - type: f1_weighted value: 26.08196367948359 - type: main_score value: 26.444000000000003 task: type: Classification - dataset: config: es name: MTEB AmazonReviewsClassification (es) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 27.351999999999997 - type: f1 value: 26.925355188458816 - type: f1_weighted value: 26.925355188458816 - type: main_score value: 27.351999999999997 task: type: Classification - dataset: config: fr name: MTEB AmazonReviewsClassification (fr) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 26.876 - type: f1 value: 26.598456848307034 - type: f1_weighted value: 26.598456848307038 - type: main_score value: 26.876 task: type: Classification - dataset: config: ja name: MTEB AmazonReviewsClassification (ja) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 23.779999999999994 - type: f1 value: 23.498318699105575 - type: f1_weighted value: 23.498318699105575 - type: main_score value: 23.779999999999994 task: type: Classification - dataset: config: zh name: MTEB AmazonReviewsClassification (zh) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 23.671999999999997 - type: f1 value: 23.219332570442802 - type: f1_weighted value: 23.219332570442802 - type: main_score value: 23.671999999999997 task: type: Classification - dataset: config: en name: MTEB AmazonReviewsClassification (en) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: validation type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 30.302 - type: f1 value: 29.983005188434415 - type: f1_weighted value: 29.983005188434415 - type: main_score value: 30.302 task: type: Classification - dataset: config: de name: MTEB AmazonReviewsClassification (de) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: validation type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 26.734 - type: f1 value: 26.36749853440715 - type: f1_weighted value: 26.36749853440715 - type: main_score value: 26.734 task: type: Classification - dataset: config: es name: MTEB AmazonReviewsClassification (es) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: validation type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 27.230000000000004 - type: f1 value: 26.77695940498618 - type: f1_weighted value: 26.776959404986183 - type: main_score value: 27.230000000000004 task: type: Classification - dataset: config: fr name: MTEB AmazonReviewsClassification (fr) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: validation type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 26.998000000000005 - type: f1 value: 26.710220629952016 - type: f1_weighted value: 26.710220629952016 - type: main_score value: 26.998000000000005 task: type: Classification - dataset: config: ja name: MTEB AmazonReviewsClassification (ja) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: validation type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 23.754 - type: f1 value: 23.46121752248106 - type: f1_weighted value: 23.46121752248106 - type: main_score value: 23.754 task: type: Classification - dataset: config: zh name: MTEB AmazonReviewsClassification (zh) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: validation type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 23.128 - type: f1 value: 22.642833134438163 - type: f1_weighted value: 22.642833134438163 - type: main_score value: 23.128 task: type: Classification - dataset: config: default name: MTEB ArguAna (default) revision: c22ab2a51041ffd869aaddef7af8d8215647e41a split: test type: mteb/arguana metrics: - type: main_score value: 50.166999999999994 - type: map_at_1 value: 25.605 - type: map_at_10 value: 41.165 - type: map_at_100 value: 42.230000000000004 - type: map_at_1000 value: 42.241 - type: map_at_20 value: 42.009 - type: map_at_3 value: 35.965 - type: map_at_5 value: 38.981 - type: mrr_at_1 value: 26.03129445234708 - type: mrr_at_10 value: 41.340569893201476 - type: mrr_at_100 value: 42.404387843997725 - type: mrr_at_1000 value: 42.415330484075966 - type: mrr_at_20 value: 42.18326840322522 - type: mrr_at_3 value: 36.10715979137032 - type: mrr_at_5 value: 39.12991939307729 - type: nauc_map_at_1000_diff1 value: 11.79240946563101 - type: nauc_map_at_1000_max value: -1.710270555143247 - type: nauc_map_at_1000_std value: -6.176978224681317 - type: nauc_map_at_100_diff1 value: 11.80102860550546 - type: nauc_map_at_100_max value: -1.6868871420671758 - type: nauc_map_at_100_std value: -6.161835334784752 - type: nauc_map_at_10_diff1 value: 11.38320219583136 - type: nauc_map_at_10_max value: -1.86833617297681 - type: nauc_map_at_10_std value: -6.310959275748389 - type: nauc_map_at_1_diff1 value: 15.127834783684404 - type: nauc_map_at_1_max value: -2.698529001436304 - type: nauc_map_at_1_std value: -6.380124976036761 - type: nauc_map_at_20_diff1 value: 11.682604563128333 - type: nauc_map_at_20_max value: -1.7092031240854961 - type: nauc_map_at_20_std value: -6.247205826670096 - type: nauc_map_at_3_diff1 value: 12.209991942694055 - type: nauc_map_at_3_max value: -1.4520060777192374 - type: nauc_map_at_3_std value: -6.488975648815049 - type: nauc_map_at_5_diff1 value: 11.278322255170867 - type: nauc_map_at_5_max value: -2.3286853004722894 - type: nauc_map_at_5_std value: -6.4661832552053085 - type: nauc_mrr_at_1000_diff1 value: 10.404448873848906 - type: nauc_mrr_at_1000_max value: -2.5691360137833272 - type: nauc_mrr_at_1000_std value: -6.110131384746746 - type: nauc_mrr_at_100_diff1 value: 10.413547230720669 - type: nauc_mrr_at_100_max value: -2.545515473991115 - type: nauc_mrr_at_100_std value: -6.095048045810124 - type: nauc_mrr_at_10_diff1 value: 10.01017546961717 - type: nauc_mrr_at_10_max value: -2.750088666160259 - type: nauc_mrr_at_10_std value: -6.300546755168433 - type: nauc_mrr_at_1_diff1 value: 13.694695183834565 - type: nauc_mrr_at_1_max value: -3.2629164055170774 - type: nauc_mrr_at_1_std value: -6.0916939845233475 - type: nauc_mrr_at_20_diff1 value: 10.305948492563749 - type: nauc_mrr_at_20_max value: -2.5611935016932743 - type: nauc_mrr_at_20_std value: -6.180513505874478 - type: nauc_mrr_at_3_diff1 value: 10.628768671682717 - type: nauc_mrr_at_3_max value: -2.580374767191263 - type: nauc_mrr_at_3_std value: -6.64080922029136 - type: nauc_mrr_at_5_diff1 value: 9.901973480934398 - type: nauc_mrr_at_5_max value: -3.219485330030026 - type: nauc_mrr_at_5_std value: -6.37273290858359 - type: nauc_ndcg_at_1000_diff1 value: 11.350532451515136 - type: nauc_ndcg_at_1000_max value: -1.1246559265562561 - type: nauc_ndcg_at_1000_std value: -5.615586258686471 - type: nauc_ndcg_at_100_diff1 value: 11.535796848928232 - type: nauc_ndcg_at_100_max value: -0.5247636861072896 - type: nauc_ndcg_at_100_std value: -5.175870339874143 - type: nauc_ndcg_at_10_diff1 value: 9.850403353114398 - type: nauc_ndcg_at_10_max value: -1.173410220148898 - type: nauc_ndcg_at_10_std value: -6.060986238386804 - type: nauc_ndcg_at_1_diff1 value: 15.127834783684404 - type: nauc_ndcg_at_1_max value: -2.698529001436304 - type: nauc_ndcg_at_1_std value: -6.380124976036761 - type: nauc_ndcg_at_20_diff1 value: 10.92332827898689 - type: nauc_ndcg_at_20_max value: -0.5604462367633576 - type: nauc_ndcg_at_20_std value: -5.828378388299274 - type: nauc_ndcg_at_3_diff1 value: 11.535159087904137 - type: nauc_ndcg_at_3_max value: -0.8097811834424876 - type: nauc_ndcg_at_3_std value: -6.317462979151113 - type: nauc_ndcg_at_5_diff1 value: 9.888045596844593 - type: nauc_ndcg_at_5_max value: -2.323203935463137 - type: nauc_ndcg_at_5_std value: -6.330378743194537 - type: nauc_precision_at_1000_diff1 value: -23.92620682863098 - type: nauc_precision_at_1000_max value: 13.422855934028885 - type: nauc_precision_at_1000_std value: 66.55596232602036 - type: nauc_precision_at_100_diff1 value: 17.358373323749067 - type: nauc_precision_at_100_max value: 50.15880988336681 - type: nauc_precision_at_100_std value: 46.757781571204916 - type: nauc_precision_at_10_diff1 value: 2.253255859825255 - type: nauc_precision_at_10_max value: 2.5784898691061175 - type: nauc_precision_at_10_std value: -4.9121989534305515 - type: nauc_precision_at_1_diff1 value: 15.127834783684404 - type: nauc_precision_at_1_max value: -2.698529001436304 - type: nauc_precision_at_1_std value: -6.380124976036761 - type: nauc_precision_at_20_diff1 value: 5.6318283797917905 - type: nauc_precision_at_20_max value: 12.700179942961162 - type: nauc_precision_at_20_std value: -1.467276762621626 - type: nauc_precision_at_3_diff1 value: 9.71950360663784 - type: nauc_precision_at_3_max value: 1.0574403269454646 - type: nauc_precision_at_3_std value: -5.778592397335912 - type: nauc_precision_at_5_diff1 value: 5.392563197207298 - type: nauc_precision_at_5_max value: -2.419134195197182 - type: nauc_precision_at_5_std value: -5.881534742887281 - type: nauc_recall_at_1000_diff1 value: -23.926206828624526 - type: nauc_recall_at_1000_max value: 13.422855934029233 - type: nauc_recall_at_1000_std value: 66.55596232601879 - type: nauc_recall_at_100_diff1 value: 17.358373323747863 - type: nauc_recall_at_100_max value: 50.158809883365386 - type: nauc_recall_at_100_std value: 46.75778157120503 - type: nauc_recall_at_10_diff1 value: 2.2532558598252876 - type: nauc_recall_at_10_max value: 2.57848986910615 - type: nauc_recall_at_10_std value: -4.912198953430499 - type: nauc_recall_at_1_diff1 value: 15.127834783684404 - type: nauc_recall_at_1_max value: -2.698529001436304 - type: nauc_recall_at_1_std value: -6.380124976036761 - type: nauc_recall_at_20_diff1 value: 5.631828379791899 - type: nauc_recall_at_20_max value: 12.700179942961059 - type: nauc_recall_at_20_std value: -1.4672767626217327 - type: nauc_recall_at_3_diff1 value: 9.71950360663783 - type: nauc_recall_at_3_max value: 1.0574403269454553 - type: nauc_recall_at_3_std value: -5.7785923973359115 - type: nauc_recall_at_5_diff1 value: 5.392563197207263 - type: nauc_recall_at_5_max value: -2.4191341951972087 - type: nauc_recall_at_5_std value: -5.881534742887311 - type: ndcg_at_1 value: 25.605 - type: ndcg_at_10 value: 50.166999999999994 - type: ndcg_at_100 value: 54.534000000000006 - type: ndcg_at_1000 value: 54.772 - type: ndcg_at_20 value: 53.159 - type: ndcg_at_3 value: 39.434000000000005 - type: ndcg_at_5 value: 44.876 - type: precision_at_1 value: 25.605 - type: precision_at_10 value: 7.908999999999999 - type: precision_at_100 value: 0.9769999999999999 - type: precision_at_1000 value: 0.1 - type: precision_at_20 value: 4.537999999999999 - type: precision_at_3 value: 16.500999999999998 - type: precision_at_5 value: 12.546 - type: recall_at_1 value: 25.605 - type: recall_at_10 value: 79.09 - type: recall_at_100 value: 97.724 - type: recall_at_1000 value: 99.502 - type: recall_at_20 value: 90.754 - type: recall_at_3 value: 49.502 - type: recall_at_5 value: 62.731 task: type: Retrieval - dataset: config: default name: MTEB ArxivClusteringP2P (default) revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d split: test type: mteb/arxiv-clustering-p2p metrics: - type: main_score value: 46.47356874334095 - type: v_measure value: 46.47356874334095 - type: v_measure_std value: 14.130000402938242 task: type: Clustering - dataset: config: default name: MTEB ArxivClusteringS2S (default) revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 split: test type: mteb/arxiv-clustering-s2s metrics: - type: main_score value: 37.668382184658086 - type: v_measure value: 37.668382184658086 - type: v_measure_std value: 14.422481353717659 task: type: Clustering - dataset: config: default name: MTEB Banking77Classification (default) revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 split: test type: mteb/banking77 metrics: - type: accuracy value: 80.04220779220779 - type: f1 value: 79.37884047247287 - type: f1_weighted value: 79.37884047247287 - type: main_score value: 80.04220779220779 task: type: Classification - dataset: config: default name: MTEB BiorxivClusteringP2P (default) revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 split: test type: mteb/biorxiv-clustering-p2p metrics: - type: main_score value: 38.36772570768531 - type: v_measure value: 38.36772570768531 - type: v_measure_std value: 0.757123491643444 task: type: Clustering - dataset: config: default name: MTEB BiorxivClusteringS2S (default) revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 split: test type: mteb/biorxiv-clustering-s2s metrics: - type: main_score value: 32.877041490913825 - type: v_measure value: 32.877041490913825 - type: v_measure_std value: 0.7543369230787144 task: type: Clustering - dataset: config: default name: MTEB CQADupstackAndroidRetrieval (default) revision: f46a197baaae43b4f621051089b82a364682dfeb split: test type: mteb/cqadupstack-android metrics: - type: main_score value: 53.82900000000001 - type: map_at_1 value: 34.660000000000004 - type: map_at_10 value: 46.938 - type: map_at_100 value: 48.435 - type: map_at_1000 value: 48.555 - type: map_at_20 value: 47.74 - type: map_at_3 value: 43.034 - type: map_at_5 value: 45.055 - type: mrr_at_1 value: 42.77539341917024 - type: mrr_at_10 value: 53.09807661738992 - type: mrr_at_100 value: 53.68797698584606 - type: mrr_at_1000 value: 53.728658519510255 - type: mrr_at_20 value: 53.406401026336844 - type: mrr_at_3 value: 50.47687172150691 - type: mrr_at_5 value: 51.971864568431094 - type: nauc_map_at_1000_diff1 value: 50.339438270595124 - type: nauc_map_at_1000_max value: 43.23580099659964 - type: nauc_map_at_1000_std value: -4.443430079055073 - type: nauc_map_at_100_diff1 value: 50.31288284732284 - type: nauc_map_at_100_max value: 43.222755445839276 - type: nauc_map_at_100_std value: -4.418807021239256 - type: nauc_map_at_10_diff1 value: 50.56104970478486 - type: nauc_map_at_10_max value: 42.6266791147141 - type: nauc_map_at_10_std value: -4.980766222304502 - type: nauc_map_at_1_diff1 value: 54.888348136203646 - type: nauc_map_at_1_max value: 36.80610602276261 - type: nauc_map_at_1_std value: -7.771623626380834 - type: nauc_map_at_20_diff1 value: 50.35074054443037 - type: nauc_map_at_20_max value: 42.953032257345086 - type: nauc_map_at_20_std value: -4.821787093076811 - type: nauc_map_at_3_diff1 value: 51.239250560446116 - type: nauc_map_at_3_max value: 40.35471642815388 - type: nauc_map_at_3_std value: -6.646877308574401 - type: nauc_map_at_5_diff1 value: 50.496300745722756 - type: nauc_map_at_5_max value: 41.850857768498756 - type: nauc_map_at_5_std value: -5.404018922303552 - type: nauc_mrr_at_1000_diff1 value: 49.881328279967285 - type: nauc_mrr_at_1000_max value: 46.097835021931175 - type: nauc_mrr_at_1000_std value: -2.804152721492059 - type: nauc_mrr_at_100_diff1 value: 49.858526100177876 - type: nauc_mrr_at_100_max value: 46.09840964633914 - type: nauc_mrr_at_100_std value: -2.7999853970281348 - type: nauc_mrr_at_10_diff1 value: 49.93411634590759 - type: nauc_mrr_at_10_max value: 46.18386740859704 - type: nauc_mrr_at_10_std value: -2.8076410876971183 - type: nauc_mrr_at_1_diff1 value: 53.69555843067244 - type: nauc_mrr_at_1_max value: 45.05416656111299 - type: nauc_mrr_at_1_std value: -4.545996010008188 - type: nauc_mrr_at_20_diff1 value: 49.777795732597255 - type: nauc_mrr_at_20_max value: 46.06899511122978 - type: nauc_mrr_at_20_std value: -2.8788594167873023 - type: nauc_mrr_at_3_diff1 value: 50.492982183354314 - type: nauc_mrr_at_3_max value: 45.785055561526654 - type: nauc_mrr_at_3_std value: -3.5794494150490306 - type: nauc_mrr_at_5_diff1 value: 49.42113979258591 - type: nauc_mrr_at_5_max value: 46.181441772578005 - type: nauc_mrr_at_5_std value: -2.564347515524424 - type: nauc_ndcg_at_1000_diff1 value: 48.51134144021698 - type: nauc_ndcg_at_1000_max value: 45.21247700795964 - type: nauc_ndcg_at_1000_std value: -1.9910218646027125 - type: nauc_ndcg_at_100_diff1 value: 47.97366158568629 - type: nauc_ndcg_at_100_max value: 45.284592745752995 - type: nauc_ndcg_at_100_std value: -1.5111571184477774 - type: nauc_ndcg_at_10_diff1 value: 48.76231762294791 - type: nauc_ndcg_at_10_max value: 44.64822017157136 - type: nauc_ndcg_at_10_std value: -3.1795387687509757 - type: nauc_ndcg_at_1_diff1 value: 53.69555843067244 - type: nauc_ndcg_at_1_max value: 45.05416656111299 - type: nauc_ndcg_at_1_std value: -4.545996010008188 - type: nauc_ndcg_at_20_diff1 value: 47.936887203849174 - type: nauc_ndcg_at_20_max value: 44.48528835952385 - type: nauc_ndcg_at_20_std value: -3.2878343948840754 - type: nauc_ndcg_at_3_diff1 value: 48.95438268525682 - type: nauc_ndcg_at_3_max value: 42.82166912587241 - type: nauc_ndcg_at_3_std value: -4.452980871878634 - type: nauc_ndcg_at_5_diff1 value: 48.040466488093344 - type: nauc_ndcg_at_5_max value: 44.11878407368567 - type: nauc_ndcg_at_5_std value: -2.927818810673812 - type: nauc_precision_at_1000_diff1 value: -14.566583997940322 - type: nauc_precision_at_1000_max value: -0.918804040217281 - type: nauc_precision_at_1000_std value: 1.1992139320417226 - type: nauc_precision_at_100_diff1 value: -9.893301864794328 - type: nauc_precision_at_100_max value: 10.623906964961217 - type: nauc_precision_at_100_std value: 10.064779696033806 - type: nauc_precision_at_10_diff1 value: 6.023861912263093 - type: nauc_precision_at_10_max value: 29.11455745368896 - type: nauc_precision_at_10_std value: 10.328214515351226 - type: nauc_precision_at_1_diff1 value: 53.69555843067244 - type: nauc_precision_at_1_max value: 45.05416656111299 - type: nauc_precision_at_1_std value: -4.545996010008188 - type: nauc_precision_at_20_diff1 value: -0.19647947200302293 - type: nauc_precision_at_20_max value: 22.792436348540797 - type: nauc_precision_at_20_std value: 9.061504712470217 - type: nauc_precision_at_3_diff1 value: 27.9608600333444 - type: nauc_precision_at_3_max value: 40.970323543946456 - type: nauc_precision_at_3_std value: 1.8438060839666899 - type: nauc_precision_at_5_diff1 value: 15.84255637113644 - type: nauc_precision_at_5_max value: 37.500727811082854 - type: nauc_precision_at_5_std value: 8.03910737246483 - type: nauc_recall_at_1000_diff1 value: 24.090697018928587 - type: nauc_recall_at_1000_max value: 73.14142934283402 - type: nauc_recall_at_1000_std value: 55.79929618150117 - type: nauc_recall_at_100_diff1 value: 29.857260058277358 - type: nauc_recall_at_100_max value: 48.81400298946208 - type: nauc_recall_at_100_std value: 18.538526239613542 - type: nauc_recall_at_10_diff1 value: 40.68103165111888 - type: nauc_recall_at_10_max value: 41.84487898013782 - type: nauc_recall_at_10_std value: -0.565909316767332 - type: nauc_recall_at_1_diff1 value: 54.888348136203646 - type: nauc_recall_at_1_max value: 36.80610602276261 - type: nauc_recall_at_1_std value: -7.771623626380834 - type: nauc_recall_at_20_diff1 value: 36.107932349955526 - type: nauc_recall_at_20_max value: 41.10565729843375 - type: nauc_recall_at_20_std value: -0.507369837769159 - type: nauc_recall_at_3_diff1 value: 45.07374176578747 - type: nauc_recall_at_3_max value: 37.769608961603005 - type: nauc_recall_at_3_std value: -5.464466204317328 - type: nauc_recall_at_5_diff1 value: 40.670419340485566 - type: nauc_recall_at_5_max value: 40.96043054994941 - type: nauc_recall_at_5_std value: -0.79976340870195 - type: ndcg_at_1 value: 42.775 - type: ndcg_at_10 value: 53.82900000000001 - type: ndcg_at_100 value: 58.74700000000001 - type: ndcg_at_1000 value: 60.309000000000005 - type: ndcg_at_20 value: 55.544000000000004 - type: ndcg_at_3 value: 48.487 - type: ndcg_at_5 value: 50.722 - type: precision_at_1 value: 42.775 - type: precision_at_10 value: 10.629 - type: precision_at_100 value: 1.652 - type: precision_at_1000 value: 0.209 - type: precision_at_20 value: 6.202 - type: precision_at_3 value: 23.366999999999997 - type: precision_at_5 value: 16.967 - type: recall_at_1 value: 34.660000000000004 - type: recall_at_10 value: 66.465 - type: recall_at_100 value: 87.559 - type: recall_at_1000 value: 97.18299999999999 - type: recall_at_20 value: 72.564 - type: recall_at_3 value: 51.01 - type: recall_at_5 value: 57.412 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackEnglishRetrieval (default) revision: ad9991cb51e31e31e430383c75ffb2885547b5f0 split: test type: mteb/cqadupstack-english metrics: - type: main_score value: 47.776 - type: map_at_1 value: 31.180999999999997 - type: map_at_10 value: 41.802 - type: map_at_100 value: 43.294 - type: map_at_1000 value: 43.438 - type: map_at_20 value: 42.629 - type: map_at_3 value: 38.668 - type: map_at_5 value: 40.559 - type: mrr_at_1 value: 39.490445859872615 - type: mrr_at_10 value: 48.38117986047922 - type: mrr_at_100 value: 49.09349219641033 - type: mrr_at_1000 value: 49.14315583534605 - type: mrr_at_20 value: 48.8095411057515 - type: mrr_at_3 value: 46.13588110403397 - type: mrr_at_5 value: 47.61358811040339 - type: nauc_map_at_1000_diff1 value: 49.44353218086297 - type: nauc_map_at_1000_max value: 46.00802280640446 - type: nauc_map_at_1000_std value: -6.439705189752024 - type: nauc_map_at_100_diff1 value: 49.484618732724634 - type: nauc_map_at_100_max value: 45.97344254091007 - type: nauc_map_at_100_std value: -6.553883735865738 - type: nauc_map_at_10_diff1 value: 49.73827377129964 - type: nauc_map_at_10_max value: 44.80720760764555 - type: nauc_map_at_10_std value: -8.560849258023518 - type: nauc_map_at_1_diff1 value: 54.89660403213069 - type: nauc_map_at_1_max value: 37.249410464928694 - type: nauc_map_at_1_std value: -13.977909182823515 - type: nauc_map_at_20_diff1 value: 49.687120778362065 - type: nauc_map_at_20_max value: 45.443506532069904 - type: nauc_map_at_20_std value: -7.568509029940879 - type: nauc_map_at_3_diff1 value: 50.89143851346375 - type: nauc_map_at_3_max value: 42.914305225298556 - type: nauc_map_at_3_std value: -10.86916575143883 - type: nauc_map_at_5_diff1 value: 49.898323469941644 - type: nauc_map_at_5_max value: 43.771367872507945 - type: nauc_map_at_5_std value: -9.813577503710816 - type: nauc_mrr_at_1000_diff1 value: 48.033344235259904 - type: nauc_mrr_at_1000_max value: 47.531177627486635 - type: nauc_mrr_at_1000_std value: -1.955679550360315 - type: nauc_mrr_at_100_diff1 value: 48.02321810138372 - type: nauc_mrr_at_100_max value: 47.535853824033744 - type: nauc_mrr_at_100_std value: -1.94289219204237 - type: nauc_mrr_at_10_diff1 value: 47.94020541903534 - type: nauc_mrr_at_10_max value: 47.43728027081464 - type: nauc_mrr_at_10_std value: -2.259178114481939 - type: nauc_mrr_at_1_diff1 value: 52.27503616517619 - type: nauc_mrr_at_1_max value: 45.06074105733823 - type: nauc_mrr_at_1_std value: -5.7377695949500245 - type: nauc_mrr_at_20_diff1 value: 48.03620297384247 - type: nauc_mrr_at_20_max value: 47.55872695584821 - type: nauc_mrr_at_20_std value: -2.034445052818207 - type: nauc_mrr_at_3_diff1 value: 48.859898225026605 - type: nauc_mrr_at_3_max value: 47.303607386271665 - type: nauc_mrr_at_3_std value: -3.2957071258586046 - type: nauc_mrr_at_5_diff1 value: 47.95858711262797 - type: nauc_mrr_at_5_max value: 47.31116600948832 - type: nauc_mrr_at_5_std value: -2.5820093721058734 - type: nauc_ndcg_at_1000_diff1 value: 46.96312568077269 - type: nauc_ndcg_at_1000_max value: 48.560727008968826 - type: nauc_ndcg_at_1000_std value: -0.4012569951094544 - type: nauc_ndcg_at_100_diff1 value: 46.913971925229944 - type: nauc_ndcg_at_100_max value: 48.7348027137932 - type: nauc_ndcg_at_100_std value: -0.3693179964972252 - type: nauc_ndcg_at_10_diff1 value: 47.05397879637747 - type: nauc_ndcg_at_10_max value: 47.186393355060105 - type: nauc_ndcg_at_10_std value: -4.226025177523436 - type: nauc_ndcg_at_1_diff1 value: 52.27503616517619 - type: nauc_ndcg_at_1_max value: 45.06074105733823 - type: nauc_ndcg_at_1_std value: -5.7377695949500245 - type: nauc_ndcg_at_20_diff1 value: 47.434914014226045 - type: nauc_ndcg_at_20_max value: 48.19048106740082 - type: nauc_ndcg_at_20_std value: -2.738959718307571 - type: nauc_ndcg_at_3_diff1 value: 48.52829797445635 - type: nauc_ndcg_at_3_max value: 46.61870515960141 - type: nauc_ndcg_at_3_std value: -5.399579996544238 - type: nauc_ndcg_at_5_diff1 value: 47.03264255703224 - type: nauc_ndcg_at_5_max value: 46.48251257722442 - type: nauc_ndcg_at_5_std value: -5.209610776344875 - type: nauc_precision_at_1000_diff1 value: -18.482812709864366 - type: nauc_precision_at_1000_max value: 10.386743152083614 - type: nauc_precision_at_1000_std value: 33.84712693143723 - type: nauc_precision_at_100_diff1 value: -10.375010055412929 - type: nauc_precision_at_100_max value: 26.01411743304198 - type: nauc_precision_at_100_std value: 41.104425433091855 - type: nauc_precision_at_10_diff1 value: 9.870205262081164 - type: nauc_precision_at_10_max value: 40.852557164414286 - type: nauc_precision_at_10_std value: 24.499871481702353 - type: nauc_precision_at_1_diff1 value: 52.27503616517619 - type: nauc_precision_at_1_max value: 45.06074105733823 - type: nauc_precision_at_1_std value: -5.7377695949500245 - type: nauc_precision_at_20_diff1 value: 2.9119655830761553 - type: nauc_precision_at_20_max value: 36.75317153063545 - type: nauc_precision_at_20_std value: 31.91941895489882 - type: nauc_precision_at_3_diff1 value: 29.276623750077814 - type: nauc_precision_at_3_max value: 48.19178171533935 - type: nauc_precision_at_3_std value: 10.69118139694828 - type: nauc_precision_at_5_diff1 value: 18.630812871426908 - type: nauc_precision_at_5_max value: 43.837708095342016 - type: nauc_precision_at_5_std value: 16.949209975740935 - type: nauc_recall_at_1000_diff1 value: 26.24448042293446 - type: nauc_recall_at_1000_max value: 60.33665737882512 - type: nauc_recall_at_1000_std value: 43.22679612133859 - type: nauc_recall_at_100_diff1 value: 32.815940701950794 - type: nauc_recall_at_100_max value: 52.580546526531194 - type: nauc_recall_at_100_std value: 21.46922275288912 - type: nauc_recall_at_10_diff1 value: 39.148857860161165 - type: nauc_recall_at_10_max value: 45.153827460455894 - type: nauc_recall_at_10_std value: -1.950539651814339 - type: nauc_recall_at_1_diff1 value: 54.89660403213069 - type: nauc_recall_at_1_max value: 37.249410464928694 - type: nauc_recall_at_1_std value: -13.977909182823515 - type: nauc_recall_at_20_diff1 value: 39.23611115234944 - type: nauc_recall_at_20_max value: 48.84676239763795 - type: nauc_recall_at_20_std value: 4.404569205404032 - type: nauc_recall_at_3_diff1 value: 45.62757328218809 - type: nauc_recall_at_3_max value: 43.520129499410864 - type: nauc_recall_at_3_std value: -9.149827641664626 - type: nauc_recall_at_5_diff1 value: 40.947956603151056 - type: nauc_recall_at_5_max value: 43.64403265109561 - type: nauc_recall_at_5_std value: -6.476025374647841 - type: ndcg_at_1 value: 39.489999999999995 - type: ndcg_at_10 value: 47.776 - type: ndcg_at_100 value: 52.705 - type: ndcg_at_1000 value: 54.830999999999996 - type: ndcg_at_20 value: 49.763000000000005 - type: ndcg_at_3 value: 43.649 - type: ndcg_at_5 value: 45.885 - type: precision_at_1 value: 39.489999999999995 - type: precision_at_10 value: 9.121 - type: precision_at_100 value: 1.504 - type: precision_at_1000 value: 0.2 - type: precision_at_20 value: 5.449 - type: precision_at_3 value: 21.38 - type: precision_at_5 value: 15.35 - type: recall_at_1 value: 31.180999999999997 - type: recall_at_10 value: 57.714 - type: recall_at_100 value: 78.342 - type: recall_at_1000 value: 91.586 - type: recall_at_20 value: 64.917 - type: recall_at_3 value: 45.255 - type: recall_at_5 value: 51.459999999999994 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackGamingRetrieval (default) revision: 4885aa143210c98657558c04aaf3dc47cfb54340 split: test type: mteb/cqadupstack-gaming metrics: - type: main_score value: 56.923 - type: map_at_1 value: 38.732 - type: map_at_10 value: 51.03 - type: map_at_100 value: 52.078 - type: map_at_1000 value: 52.132 - type: map_at_20 value: 51.648 - type: map_at_3 value: 47.735 - type: map_at_5 value: 49.562 - type: mrr_at_1 value: 44.075235109717866 - type: mrr_at_10 value: 54.24349405383888 - type: mrr_at_100 value: 54.92548762360754 - type: mrr_at_1000 value: 54.953180989670734 - type: mrr_at_20 value: 54.65337227412258 - type: mrr_at_3 value: 51.765935214211076 - type: mrr_at_5 value: 53.2110762800418 - type: nauc_map_at_1000_diff1 value: 51.90809944536219 - type: nauc_map_at_1000_max value: 42.584041397418915 - type: nauc_map_at_1000_std value: -4.092796385362112 - type: nauc_map_at_100_diff1 value: 51.87489219454552 - type: nauc_map_at_100_max value: 42.56321622647399 - type: nauc_map_at_100_std value: -4.108671096246823 - type: nauc_map_at_10_diff1 value: 51.94456556858801 - type: nauc_map_at_10_max value: 42.019642277457834 - type: nauc_map_at_10_std value: -5.093848267830008 - type: nauc_map_at_1_diff1 value: 56.18384355337904 - type: nauc_map_at_1_max value: 35.92739920256724 - type: nauc_map_at_1_std value: -7.700954384698089 - type: nauc_map_at_20_diff1 value: 51.84091110199576 - type: nauc_map_at_20_max value: 42.3843420189171 - type: nauc_map_at_20_std value: -4.40444644798754 - type: nauc_map_at_3_diff1 value: 52.08439941713987 - type: nauc_map_at_3_max value: 40.21039214935397 - type: nauc_map_at_3_std value: -7.324738964281617 - type: nauc_map_at_5_diff1 value: 51.82629501690293 - type: nauc_map_at_5_max value: 41.10065065673012 - type: nauc_map_at_5_std value: -6.261842403214812 - type: nauc_mrr_at_1000_diff1 value: 53.279508868349645 - type: nauc_mrr_at_1000_max value: 45.25364956742305 - type: nauc_mrr_at_1000_std value: -1.8430883364000474 - type: nauc_mrr_at_100_diff1 value: 53.25983484799026 - type: nauc_mrr_at_100_max value: 45.25962889286108 - type: nauc_mrr_at_100_std value: -1.8211114380578641 - type: nauc_mrr_at_10_diff1 value: 53.27160810895254 - type: nauc_mrr_at_10_max value: 45.278243219167614 - type: nauc_mrr_at_10_std value: -2.0022809788792197 - type: nauc_mrr_at_1_diff1 value: 57.03534737937047 - type: nauc_mrr_at_1_max value: 42.41929180637128 - type: nauc_mrr_at_1_std value: -5.559470907525906 - type: nauc_mrr_at_20_diff1 value: 53.212121600393495 - type: nauc_mrr_at_20_max value: 45.25492529730299 - type: nauc_mrr_at_20_std value: -1.8709180680768236 - type: nauc_mrr_at_3_diff1 value: 53.66732897537923 - type: nauc_mrr_at_3_max value: 45.213018524091744 - type: nauc_mrr_at_3_std value: -2.6545754944664726 - type: nauc_mrr_at_5_diff1 value: 53.11553937075799 - type: nauc_mrr_at_5_max value: 45.21311015517629 - type: nauc_mrr_at_5_std value: -2.3047852346440485 - type: nauc_ndcg_at_1000_diff1 value: 51.22084372891484 - type: nauc_ndcg_at_1000_max value: 45.033616473157096 - type: nauc_ndcg_at_1000_std value: -0.49537334935387783 - type: nauc_ndcg_at_100_diff1 value: 50.662111242206734 - type: nauc_ndcg_at_100_max value: 45.20193284684836 - type: nauc_ndcg_at_100_std value: 0.06588193709667538 - type: nauc_ndcg_at_10_diff1 value: 50.7967609561699 - type: nauc_ndcg_at_10_max value: 44.44451781209191 - type: nauc_ndcg_at_10_std value: -2.2422184861488756 - type: nauc_ndcg_at_1_diff1 value: 57.03534737937047 - type: nauc_ndcg_at_1_max value: 42.41929180637128 - type: nauc_ndcg_at_1_std value: -5.559470907525906 - type: nauc_ndcg_at_20_diff1 value: 50.40874342321644 - type: nauc_ndcg_at_20_max value: 44.82829895621773 - type: nauc_ndcg_at_20_std value: -1.0316251943203758 - type: nauc_ndcg_at_3_diff1 value: 51.19864887663497 - type: nauc_ndcg_at_3_max value: 42.86706753256436 - type: nauc_ndcg_at_3_std value: -5.012473010526891 - type: nauc_ndcg_at_5_diff1 value: 50.50949069494513 - type: nauc_ndcg_at_5_max value: 43.3779262922725 - type: nauc_ndcg_at_5_std value: -3.9043550992546048 - type: nauc_precision_at_1000_diff1 value: -8.649623125586269 - type: nauc_precision_at_1000_max value: 21.555836290383134 - type: nauc_precision_at_1000_std value: 31.65585615100276 - type: nauc_precision_at_100_diff1 value: -3.7069964551748122 - type: nauc_precision_at_100_max value: 28.923976741147765 - type: nauc_precision_at_100_std value: 32.54838990025281 - type: nauc_precision_at_10_diff1 value: 16.349707064050946 - type: nauc_precision_at_10_max value: 39.132645459531986 - type: nauc_precision_at_10_std value: 16.306513403680004 - type: nauc_precision_at_1_diff1 value: 57.03534737937047 - type: nauc_precision_at_1_max value: 42.41929180637128 - type: nauc_precision_at_1_std value: -5.559470907525906 - type: nauc_precision_at_20_diff1 value: 7.5905952134108094 - type: nauc_precision_at_20_max value: 35.8825094829391 - type: nauc_precision_at_20_std value: 24.48349957693172 - type: nauc_precision_at_3_diff1 value: 32.80092363376732 - type: nauc_precision_at_3_max value: 43.3774140381366 - type: nauc_precision_at_3_std value: 1.7501259447120148 - type: nauc_precision_at_5_diff1 value: 25.28973395899848 - type: nauc_precision_at_5_max value: 41.69668551811128 - type: nauc_precision_at_5_std value: 7.244670538771375 - type: nauc_recall_at_1000_diff1 value: 36.08978686599952 - type: nauc_recall_at_1000_max value: 60.956553966508785 - type: nauc_recall_at_1000_std value: 49.63396504888992 - type: nauc_recall_at_100_diff1 value: 35.857979478786056 - type: nauc_recall_at_100_max value: 54.20184925232567 - type: nauc_recall_at_100_std value: 28.673958901408014 - type: nauc_recall_at_10_diff1 value: 43.0128169608147 - type: nauc_recall_at_10_max value: 45.696402596156545 - type: nauc_recall_at_10_std value: 2.6482104533880717 - type: nauc_recall_at_1_diff1 value: 56.18384355337904 - type: nauc_recall_at_1_max value: 35.92739920256724 - type: nauc_recall_at_1_std value: -7.700954384698089 - type: nauc_recall_at_20_diff1 value: 39.89455258008386 - type: nauc_recall_at_20_max value: 48.35270191375881 - type: nauc_recall_at_20_std value: 9.673486702133298 - type: nauc_recall_at_3_diff1 value: 46.67802970279263 - type: nauc_recall_at_3_max value: 41.314599757779426 - type: nauc_recall_at_3_std value: -6.557047140009625 - type: nauc_recall_at_5_diff1 value: 43.54326665692751 - type: nauc_recall_at_5_max value: 42.76486301935587 - type: nauc_recall_at_5_std value: -2.91989328244513 - type: ndcg_at_1 value: 44.074999999999996 - type: ndcg_at_10 value: 56.923 - type: ndcg_at_100 value: 61.004999999999995 - type: ndcg_at_1000 value: 62.12800000000001 - type: ndcg_at_20 value: 58.618 - type: ndcg_at_3 value: 51.381 - type: ndcg_at_5 value: 54.027 - type: precision_at_1 value: 44.074999999999996 - type: precision_at_10 value: 9.21 - type: precision_at_100 value: 1.221 - type: precision_at_1000 value: 0.136 - type: precision_at_20 value: 5.1499999999999995 - type: precision_at_3 value: 23.009 - type: precision_at_5 value: 15.748999999999999 - type: recall_at_1 value: 38.732 - type: recall_at_10 value: 71.154 - type: recall_at_100 value: 88.676 - type: recall_at_1000 value: 96.718 - type: recall_at_20 value: 77.242 - type: recall_at_3 value: 56.288000000000004 - type: recall_at_5 value: 62.792 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackGisRetrieval (default) revision: 5003b3064772da1887988e05400cf3806fe491f2 split: test type: mteb/cqadupstack-gis metrics: - type: main_score value: 41.31 - type: map_at_1 value: 26.837 - type: map_at_10 value: 35.959 - type: map_at_100 value: 37.172 - type: map_at_1000 value: 37.241 - type: map_at_20 value: 36.7 - type: map_at_3 value: 33.027 - type: map_at_5 value: 34.699000000000005 - type: mrr_at_1 value: 29.37853107344633 - type: mrr_at_10 value: 38.30732669715721 - type: mrr_at_100 value: 39.33054657529381 - type: mrr_at_1000 value: 39.379667885813205 - type: mrr_at_20 value: 38.95885516282221 - type: mrr_at_3 value: 35.59322033898305 - type: mrr_at_5 value: 37.09039548022599 - type: nauc_map_at_1000_diff1 value: 33.824985266541105 - type: nauc_map_at_1000_max value: 27.38354315602597 - type: nauc_map_at_1000_std value: -4.183937989318261 - type: nauc_map_at_100_diff1 value: 33.826642921299836 - type: nauc_map_at_100_max value: 27.391547143969703 - type: nauc_map_at_100_std value: -4.1622733750723535 - type: nauc_map_at_10_diff1 value: 33.74139790310016 - type: nauc_map_at_10_max value: 26.988973573292906 - type: nauc_map_at_10_std value: -4.6158886383681486 - type: nauc_map_at_1_diff1 value: 40.73105610223564 - type: nauc_map_at_1_max value: 26.117980671724606 - type: nauc_map_at_1_std value: -7.823817113802216 - type: nauc_map_at_20_diff1 value: 33.77805724779642 - type: nauc_map_at_20_max value: 27.282978783733174 - type: nauc_map_at_20_std value: -4.186095094578366 - type: nauc_map_at_3_diff1 value: 36.097185072940704 - type: nauc_map_at_3_max value: 26.874307388587006 - type: nauc_map_at_3_std value: -6.020142942595767 - type: nauc_map_at_5_diff1 value: 34.25776532088489 - type: nauc_map_at_5_max value: 26.843408999370062 - type: nauc_map_at_5_std value: -5.3182865274497555 - type: nauc_mrr_at_1000_diff1 value: 31.07446539700584 - type: nauc_mrr_at_1000_max value: 28.23648431351983 - type: nauc_mrr_at_1000_std value: -1.9730030997144297 - type: nauc_mrr_at_100_diff1 value: 31.05879961364511 - type: nauc_mrr_at_100_max value: 28.257937702133447 - type: nauc_mrr_at_100_std value: -1.9401890833311664 - type: nauc_mrr_at_10_diff1 value: 30.851921082461164 - type: nauc_mrr_at_10_max value: 28.019748398612656 - type: nauc_mrr_at_10_std value: -2.1501394995405905 - type: nauc_mrr_at_1_diff1 value: 37.49227872151747 - type: nauc_mrr_at_1_max value: 28.674240705315903 - type: nauc_mrr_at_1_std value: -4.35056486689729 - type: nauc_mrr_at_20_diff1 value: 30.982876689431553 - type: nauc_mrr_at_20_max value: 28.18728312649274 - type: nauc_mrr_at_20_std value: -1.9400536131807824 - type: nauc_mrr_at_3_diff1 value: 32.80644035401081 - type: nauc_mrr_at_3_max value: 28.31201785250565 - type: nauc_mrr_at_3_std value: -3.0715346813687603 - type: nauc_mrr_at_5_diff1 value: 31.215681881852785 - type: nauc_mrr_at_5_max value: 27.853131873520315 - type: nauc_mrr_at_5_std value: -2.7267492848551864 - type: nauc_ndcg_at_1000_diff1 value: 30.882552399077433 - type: nauc_ndcg_at_1000_max value: 28.592799066356868 - type: nauc_ndcg_at_1000_std value: -1.1610130632370996 - type: nauc_ndcg_at_100_diff1 value: 30.52925170726371 - type: nauc_ndcg_at_100_max value: 29.01788121102285 - type: nauc_ndcg_at_100_std value: -0.36382102937662036 - type: nauc_ndcg_at_10_diff1 value: 29.646957531268058 - type: nauc_ndcg_at_10_max value: 27.25820710857276 - type: nauc_ndcg_at_10_std value: -2.1740807696394246 - type: nauc_ndcg_at_1_diff1 value: 37.49227872151747 - type: nauc_ndcg_at_1_max value: 28.674240705315903 - type: nauc_ndcg_at_1_std value: -4.35056486689729 - type: nauc_ndcg_at_20_diff1 value: 29.81954001580238 - type: nauc_ndcg_at_20_max value: 28.12655829166776 - type: nauc_ndcg_at_20_std value: -0.8426496817280011 - type: nauc_ndcg_at_3_diff1 value: 33.79994800316125 - type: nauc_ndcg_at_3_max value: 27.566086245731903 - type: nauc_ndcg_at_3_std value: -4.531429933937334 - type: nauc_ndcg_at_5_diff1 value: 30.796050416879833 - type: nauc_ndcg_at_5_max value: 26.987450353856836 - type: nauc_ndcg_at_5_std value: -3.6785320861681536 - type: nauc_precision_at_1000_diff1 value: -6.667048342551707 - type: nauc_precision_at_1000_max value: 12.74665735228754 - type: nauc_precision_at_1000_std value: 14.388810690231695 - type: nauc_precision_at_100_diff1 value: -0.9315554525058245 - type: nauc_precision_at_100_max value: 23.785783803695086 - type: nauc_precision_at_100_std value: 18.017934826958072 - type: nauc_precision_at_10_diff1 value: 10.897625176899407 - type: nauc_precision_at_10_max value: 28.302337751018168 - type: nauc_precision_at_10_std value: 7.593887792320358 - type: nauc_precision_at_1_diff1 value: 37.49227872151747 - type: nauc_precision_at_1_max value: 28.674240705315903 - type: nauc_precision_at_1_std value: -4.35056486689729 - type: nauc_precision_at_20_diff1 value: 7.1553681942590055 - type: nauc_precision_at_20_max value: 29.03199232231963 - type: nauc_precision_at_20_std value: 14.400319742846573 - type: nauc_precision_at_3_diff1 value: 25.195444969267704 - type: nauc_precision_at_3_max value: 30.412936949743763 - type: nauc_precision_at_3_std value: 0.3718140966352182 - type: nauc_precision_at_5_diff1 value: 16.08508800784734 - type: nauc_precision_at_5_max value: 28.184570311016234 - type: nauc_precision_at_5_std value: 2.4001788379725673 - type: nauc_recall_at_1000_diff1 value: 15.251030527386675 - type: nauc_recall_at_1000_max value: 44.68626014260891 - type: nauc_recall_at_1000_std value: 31.912046734047696 - type: nauc_recall_at_100_diff1 value: 18.625221846410803 - type: nauc_recall_at_100_max value: 38.12448370446821 - type: nauc_recall_at_100_std value: 20.795359725619754 - type: nauc_recall_at_10_diff1 value: 18.354876530522134 - type: nauc_recall_at_10_max value: 25.551602682680922 - type: nauc_recall_at_10_std value: 3.220738051054369 - type: nauc_recall_at_1_diff1 value: 40.73105610223564 - type: nauc_recall_at_1_max value: 26.117980671724606 - type: nauc_recall_at_1_std value: -7.823817113802216 - type: nauc_recall_at_20_diff1 value: 17.48367397137018 - type: nauc_recall_at_20_max value: 28.613212769951218 - type: nauc_recall_at_20_std value: 9.367130329617583 - type: nauc_recall_at_3_diff1 value: 30.22639273579569 - type: nauc_recall_at_3_max value: 26.672584329288853 - type: nauc_recall_at_3_std value: -3.1070994652973067 - type: nauc_recall_at_5_diff1 value: 22.609561401532595 - type: nauc_recall_at_5_max value: 25.243108633000972 - type: nauc_recall_at_5_std value: -1.3641313534725177 - type: ndcg_at_1 value: 29.378999999999998 - type: ndcg_at_10 value: 41.31 - type: ndcg_at_100 value: 47.058 - type: ndcg_at_1000 value: 48.777 - type: ndcg_at_20 value: 43.811 - type: ndcg_at_3 value: 35.564 - type: ndcg_at_5 value: 38.384 - type: precision_at_1 value: 29.378999999999998 - type: precision_at_10 value: 6.361999999999999 - type: precision_at_100 value: 0.98 - type: precision_at_1000 value: 0.117 - type: precision_at_20 value: 3.785 - type: precision_at_3 value: 15.028 - type: precision_at_5 value: 10.667 - type: recall_at_1 value: 26.837 - type: recall_at_10 value: 55.667 - type: recall_at_100 value: 81.843 - type: recall_at_1000 value: 94.707 - type: recall_at_20 value: 64.99000000000001 - type: recall_at_3 value: 40.049 - type: recall_at_5 value: 46.92 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackMathematicaRetrieval (default) revision: 90fceea13679c63fe563ded68f3b6f06e50061de split: test type: mteb/cqadupstack-mathematica metrics: - type: main_score value: 29.286 - type: map_at_1 value: 15.142 - type: map_at_10 value: 23.727999999999998 - type: map_at_100 value: 25.137999999999998 - type: map_at_1000 value: 25.256 - type: map_at_20 value: 24.555 - type: map_at_3 value: 20.673 - type: map_at_5 value: 22.325999999999997 - type: mrr_at_1 value: 18.407960199004975 - type: mrr_at_10 value: 27.512289741767354 - type: mrr_at_100 value: 28.67133144523845 - type: mrr_at_1000 value: 28.734556130499765 - type: mrr_at_20 value: 28.242773767515576 - type: mrr_at_3 value: 24.58540630182421 - type: mrr_at_5 value: 26.25207296849088 - type: nauc_map_at_1000_diff1 value: 29.830495875741686 - type: nauc_map_at_1000_max value: 29.844871544709754 - type: nauc_map_at_1000_std value: 4.531584325691709 - type: nauc_map_at_100_diff1 value: 29.783099563199478 - type: nauc_map_at_100_max value: 29.870567523376064 - type: nauc_map_at_100_std value: 4.575369775572174 - type: nauc_map_at_10_diff1 value: 30.328052451658177 - type: nauc_map_at_10_max value: 29.68196864841043 - type: nauc_map_at_10_std value: 4.041487315029131 - type: nauc_map_at_1_diff1 value: 34.06103409502576 - type: nauc_map_at_1_max value: 26.542597994455168 - type: nauc_map_at_1_std value: 0.13222518698558713 - type: nauc_map_at_20_diff1 value: 29.671969728646207 - type: nauc_map_at_20_max value: 29.56469190452673 - type: nauc_map_at_20_std value: 4.270584332187552 - type: nauc_map_at_3_diff1 value: 31.886663324961994 - type: nauc_map_at_3_max value: 28.931210892147874 - type: nauc_map_at_3_std value: 1.792647218529753 - type: nauc_map_at_5_diff1 value: 30.130974453903004 - type: nauc_map_at_5_max value: 28.404743926409616 - type: nauc_map_at_5_std value: 2.548774969391927 - type: nauc_mrr_at_1000_diff1 value: 31.628830222794146 - type: nauc_mrr_at_1000_max value: 31.91487774004666 - type: nauc_mrr_at_1000_std value: 6.370961988994502 - type: nauc_mrr_at_100_diff1 value: 31.606059731944256 - type: nauc_mrr_at_100_max value: 31.910838943893637 - type: nauc_mrr_at_100_std value: 6.377656370563926 - type: nauc_mrr_at_10_diff1 value: 31.977420533671218 - type: nauc_mrr_at_10_max value: 32.11665186256517 - type: nauc_mrr_at_10_std value: 6.277016093709463 - type: nauc_mrr_at_1_diff1 value: 37.478663352559366 - type: nauc_mrr_at_1_max value: 31.176249723296767 - type: nauc_mrr_at_1_std value: 3.302936413131115 - type: nauc_mrr_at_20_diff1 value: 31.484955715846052 - type: nauc_mrr_at_20_max value: 31.80392932975274 - type: nauc_mrr_at_20_std value: 6.348841589653416 - type: nauc_mrr_at_3_diff1 value: 33.0234707676869 - type: nauc_mrr_at_3_max value: 31.66594429342623 - type: nauc_mrr_at_3_std value: 4.386409179222804 - type: nauc_mrr_at_5_diff1 value: 31.812421283740065 - type: nauc_mrr_at_5_max value: 31.311323549308206 - type: nauc_mrr_at_5_std value: 5.642758734528096 - type: nauc_ndcg_at_1000_diff1 value: 28.02913984018856 - type: nauc_ndcg_at_1000_max value: 31.14720296628386 - type: nauc_ndcg_at_1000_std value: 8.245505565605681 - type: nauc_ndcg_at_100_diff1 value: 27.133234480898416 - type: nauc_ndcg_at_100_max value: 31.734486588931297 - type: nauc_ndcg_at_100_std value: 9.151097698922285 - type: nauc_ndcg_at_10_diff1 value: 28.838857669076322 - type: nauc_ndcg_at_10_max value: 31.115220545303117 - type: nauc_ndcg_at_10_std value: 6.713247141105111 - type: nauc_ndcg_at_1_diff1 value: 37.478663352559366 - type: nauc_ndcg_at_1_max value: 31.176249723296767 - type: nauc_ndcg_at_1_std value: 3.302936413131115 - type: nauc_ndcg_at_20_diff1 value: 26.838210799213876 - type: nauc_ndcg_at_20_max value: 30.439021621824924 - type: nauc_ndcg_at_20_std value: 7.311545065277699 - type: nauc_ndcg_at_3_diff1 value: 31.401802099009004 - type: nauc_ndcg_at_3_max value: 29.953906005830135 - type: nauc_ndcg_at_3_std value: 2.5890022039011202 - type: nauc_ndcg_at_5_diff1 value: 28.6330418628233 - type: nauc_ndcg_at_5_max value: 29.00802098619434 - type: nauc_ndcg_at_5_std value: 4.227912244901072 - type: nauc_precision_at_1000_diff1 value: 2.5911901580875867 - type: nauc_precision_at_1000_max value: 4.0518538327169855 - type: nauc_precision_at_1000_std value: 1.7758268328761477 - type: nauc_precision_at_100_diff1 value: 8.980077140205331 - type: nauc_precision_at_100_max value: 22.724340263254195 - type: nauc_precision_at_100_std value: 14.664473151056562 - type: nauc_precision_at_10_diff1 value: 21.763297367222552 - type: nauc_precision_at_10_max value: 32.92900970570401 - type: nauc_precision_at_10_std value: 13.497567973388566 - type: nauc_precision_at_1_diff1 value: 37.478663352559366 - type: nauc_precision_at_1_max value: 31.176249723296767 - type: nauc_precision_at_1_std value: 3.302936413131115 - type: nauc_precision_at_20_diff1 value: 14.21580175290909 - type: nauc_precision_at_20_max value: 28.968198467692787 - type: nauc_precision_at_20_std value: 13.791402369534932 - type: nauc_precision_at_3_diff1 value: 29.256654567402435 - type: nauc_precision_at_3_max value: 32.24444511852006 - type: nauc_precision_at_3_std value: 5.595987236310881 - type: nauc_precision_at_5_diff1 value: 23.48548965604621 - type: nauc_precision_at_5_max value: 29.559989698787266 - type: nauc_precision_at_5_std value: 7.973397491873005 - type: nauc_recall_at_1000_diff1 value: 8.77350731884266 - type: nauc_recall_at_1000_max value: 30.982963576717697 - type: nauc_recall_at_1000_std value: 39.847794579683 - type: nauc_recall_at_100_diff1 value: 12.323323397833159 - type: nauc_recall_at_100_max value: 33.62294233066083 - type: nauc_recall_at_100_std value: 26.54562660936417 - type: nauc_recall_at_10_diff1 value: 22.641004774225816 - type: nauc_recall_at_10_max value: 30.594205495355244 - type: nauc_recall_at_10_std value: 11.310901090263238 - type: nauc_recall_at_1_diff1 value: 34.06103409502576 - type: nauc_recall_at_1_max value: 26.542597994455168 - type: nauc_recall_at_1_std value: 0.13222518698558713 - type: nauc_recall_at_20_diff1 value: 15.689605490947223 - type: nauc_recall_at_20_max value: 27.746801753725965 - type: nauc_recall_at_20_std value: 13.370839851272823 - type: nauc_recall_at_3_diff1 value: 27.553486937655663 - type: nauc_recall_at_3_max value: 28.01270402033647 - type: nauc_recall_at_3_std value: 1.6810358030711698 - type: nauc_recall_at_5_diff1 value: 21.661158520439603 - type: nauc_recall_at_5_max value: 25.659495548646337 - type: nauc_recall_at_5_std value: 5.487851983581348 - type: ndcg_at_1 value: 18.407999999999998 - type: ndcg_at_10 value: 29.286 - type: ndcg_at_100 value: 35.753 - type: ndcg_at_1000 value: 38.541 - type: ndcg_at_20 value: 32.029999999999994 - type: ndcg_at_3 value: 23.599 - type: ndcg_at_5 value: 26.262 - type: precision_at_1 value: 18.407999999999998 - type: precision_at_10 value: 5.697 - type: precision_at_100 value: 1.034 - type: precision_at_1000 value: 0.14100000000000001 - type: precision_at_20 value: 3.6189999999999998 - type: precision_at_3 value: 11.567 - type: precision_at_5 value: 8.781 - type: recall_at_1 value: 15.142 - type: recall_at_10 value: 42.476 - type: recall_at_100 value: 70.22699999999999 - type: recall_at_1000 value: 90.02799999999999 - type: recall_at_20 value: 52.278999999999996 - type: recall_at_3 value: 27.056 - type: recall_at_5 value: 33.663 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackPhysicsRetrieval (default) revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4 split: test type: mteb/cqadupstack-physics metrics: - type: main_score value: 46.982 - type: map_at_1 value: 29.142000000000003 - type: map_at_10 value: 40.735 - type: map_at_100 value: 42.155 - type: map_at_1000 value: 42.27 - type: map_at_20 value: 41.498000000000005 - type: map_at_3 value: 37.491 - type: map_at_5 value: 39.475 - type: mrr_at_1 value: 35.514918190567855 - type: mrr_at_10 value: 46.31582565653788 - type: mrr_at_100 value: 47.14427440178252 - type: mrr_at_1000 value: 47.18423700218728 - type: mrr_at_20 value: 46.78013051884466 - type: mrr_at_3 value: 43.96855951235162 - type: mrr_at_5 value: 45.33525826114854 - type: nauc_map_at_1000_diff1 value: 48.376841720659684 - type: nauc_map_at_1000_max value: 36.76054797105518 - type: nauc_map_at_1000_std value: -5.310244782841096 - type: nauc_map_at_100_diff1 value: 48.39009284893546 - type: nauc_map_at_100_max value: 36.756939642759676 - type: nauc_map_at_100_std value: -5.3360132498714155 - type: nauc_map_at_10_diff1 value: 48.301337883081494 - type: nauc_map_at_10_max value: 35.84362178782953 - type: nauc_map_at_10_std value: -6.680396768130203 - type: nauc_map_at_1_diff1 value: 54.10938668812014 - type: nauc_map_at_1_max value: 33.5569415932719 - type: nauc_map_at_1_std value: -9.158006006882797 - type: nauc_map_at_20_diff1 value: 48.37273154648706 - type: nauc_map_at_20_max value: 36.375946064959514 - type: nauc_map_at_20_std value: -5.941356185910192 - type: nauc_map_at_3_diff1 value: 49.44052440546684 - type: nauc_map_at_3_max value: 35.29983997477593 - type: nauc_map_at_3_std value: -7.926122642140471 - type: nauc_map_at_5_diff1 value: 48.425401884249546 - type: nauc_map_at_5_max value: 35.76738899038316 - type: nauc_map_at_5_std value: -7.086066122233774 - type: nauc_mrr_at_1000_diff1 value: 47.739498443882525 - type: nauc_mrr_at_1000_max value: 38.08125408730989 - type: nauc_mrr_at_1000_std value: -2.2895275028822875 - type: nauc_mrr_at_100_diff1 value: 47.71344530559796 - type: nauc_mrr_at_100_max value: 38.08097573778073 - type: nauc_mrr_at_100_std value: -2.283043912301711 - type: nauc_mrr_at_10_diff1 value: 47.570348799278975 - type: nauc_mrr_at_10_max value: 37.79140951321141 - type: nauc_mrr_at_10_std value: -2.669668464347039 - type: nauc_mrr_at_1_diff1 value: 54.362704882881786 - type: nauc_mrr_at_1_max value: 38.0436856766311 - type: nauc_mrr_at_1_std value: -3.7228255611457297 - type: nauc_mrr_at_20_diff1 value: 47.61687084521078 - type: nauc_mrr_at_20_max value: 37.98181088604165 - type: nauc_mrr_at_20_std value: -2.451306358135568 - type: nauc_mrr_at_3_diff1 value: 48.29760542894089 - type: nauc_mrr_at_3_max value: 38.320196024196484 - type: nauc_mrr_at_3_std value: -2.608893715829596 - type: nauc_mrr_at_5_diff1 value: 47.52445262256707 - type: nauc_mrr_at_5_max value: 37.88269893600532 - type: nauc_mrr_at_5_std value: -2.6463159987466938 - type: nauc_ndcg_at_1000_diff1 value: 46.468406214074975 - type: nauc_ndcg_at_1000_max value: 38.46677359985886 - type: nauc_ndcg_at_1000_std value: -1.4983478106237946 - type: nauc_ndcg_at_100_diff1 value: 46.173532773312665 - type: nauc_ndcg_at_100_max value: 38.5543502306479 - type: nauc_ndcg_at_100_std value: -1.2145160553641832 - type: nauc_ndcg_at_10_diff1 value: 45.903006767468966 - type: nauc_ndcg_at_10_max value: 35.65042776694142 - type: nauc_ndcg_at_10_std value: -5.878538482190709 - type: nauc_ndcg_at_1_diff1 value: 54.362704882881786 - type: nauc_ndcg_at_1_max value: 38.0436856766311 - type: nauc_ndcg_at_1_std value: -3.7228255611457297 - type: nauc_ndcg_at_20_diff1 value: 46.009956025569906 - type: nauc_ndcg_at_20_max value: 37.09175828227361 - type: nauc_ndcg_at_20_std value: -3.9903455853106147 - type: nauc_ndcg_at_3_diff1 value: 47.20874990734948 - type: nauc_ndcg_at_3_max value: 36.7478305444307 - type: nauc_ndcg_at_3_std value: -5.556906879694707 - type: nauc_ndcg_at_5_diff1 value: 45.938325614437616 - type: nauc_ndcg_at_5_max value: 36.158427238686485 - type: nauc_ndcg_at_5_std value: -5.663685225247423 - type: nauc_precision_at_1000_diff1 value: -20.17554621466442 - type: nauc_precision_at_1000_max value: 3.718674752259679 - type: nauc_precision_at_1000_std value: 20.311126678632473 - type: nauc_precision_at_100_diff1 value: -9.77163051571759 - type: nauc_precision_at_100_max value: 18.342524884598333 - type: nauc_precision_at_100_std value: 29.35841582729325 - type: nauc_precision_at_10_diff1 value: 13.680163039035545 - type: nauc_precision_at_10_max value: 29.16355469101684 - type: nauc_precision_at_10_std value: 13.68130771479034 - type: nauc_precision_at_1_diff1 value: 54.362704882881786 - type: nauc_precision_at_1_max value: 38.0436856766311 - type: nauc_precision_at_1_std value: -3.7228255611457297 - type: nauc_precision_at_20_diff1 value: 6.2946432484262616 - type: nauc_precision_at_20_max value: 27.909497591400235 - type: nauc_precision_at_20_std value: 20.52292294725688 - type: nauc_precision_at_3_diff1 value: 31.1332087743737 - type: nauc_precision_at_3_max value: 37.86322291336991 - type: nauc_precision_at_3_std value: 5.327614107392234 - type: nauc_precision_at_5_diff1 value: 20.73434520971964 - type: nauc_precision_at_5_max value: 33.9824188155478 - type: nauc_precision_at_5_std value: 10.495836005987448 - type: nauc_recall_at_1000_diff1 value: 41.56310016395119 - type: nauc_recall_at_1000_max value: 58.40933956558557 - type: nauc_recall_at_1000_std value: 47.97490616720571 - type: nauc_recall_at_100_diff1 value: 32.94606451556238 - type: nauc_recall_at_100_max value: 41.9600123794069 - type: nauc_recall_at_100_std value: 18.888894721063252 - type: nauc_recall_at_10_diff1 value: 37.467808311529566 - type: nauc_recall_at_10_max value: 29.680696266674904 - type: nauc_recall_at_10_std value: -8.981727422390986 - type: nauc_recall_at_1_diff1 value: 54.10938668812014 - type: nauc_recall_at_1_max value: 33.5569415932719 - type: nauc_recall_at_1_std value: -9.158006006882797 - type: nauc_recall_at_20_diff1 value: 36.37336600186495 - type: nauc_recall_at_20_max value: 33.76755814232919 - type: nauc_recall_at_20_std value: -2.491259189562959 - type: nauc_recall_at_3_diff1 value: 42.42921625891564 - type: nauc_recall_at_3_max value: 32.1863591696642 - type: nauc_recall_at_3_std value: -9.472176740832754 - type: nauc_recall_at_5_diff1 value: 38.90086250705934 - type: nauc_recall_at_5_max value: 31.42899308569517 - type: nauc_recall_at_5_std value: -9.346282658103304 - type: ndcg_at_1 value: 35.515 - type: ndcg_at_10 value: 46.982 - type: ndcg_at_100 value: 52.913 - type: ndcg_at_1000 value: 54.759 - type: ndcg_at_20 value: 49.296 - type: ndcg_at_3 value: 42.164 - type: ndcg_at_5 value: 44.674 - type: precision_at_1 value: 35.515 - type: precision_at_10 value: 8.624 - type: precision_at_100 value: 1.377 - type: precision_at_1000 value: 0.173 - type: precision_at_20 value: 5.058 - type: precision_at_3 value: 20.468 - type: precision_at_5 value: 14.649000000000001 - type: recall_at_1 value: 29.142000000000003 - type: recall_at_10 value: 59.693 - type: recall_at_100 value: 84.84899999999999 - type: recall_at_1000 value: 96.451 - type: recall_at_20 value: 68.015 - type: recall_at_3 value: 46.086 - type: recall_at_5 value: 52.556000000000004 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackProgrammersRetrieval (default) revision: 6184bc1440d2dbc7612be22b50686b8826d22b32 split: test type: mteb/cqadupstack-programmers metrics: - type: main_score value: 38.968 - type: map_at_1 value: 22.081999999999997 - type: map_at_10 value: 32.74 - type: map_at_100 value: 34.108 - type: map_at_1000 value: 34.233000000000004 - type: map_at_20 value: 33.521 - type: map_at_3 value: 29.282999999999998 - type: map_at_5 value: 31.127 - type: mrr_at_1 value: 26.71232876712329 - type: mrr_at_10 value: 37.271598898311225 - type: mrr_at_100 value: 38.14029311656465 - type: mrr_at_1000 value: 38.20075599073657 - type: mrr_at_20 value: 37.769608485035874 - type: mrr_at_3 value: 34.58904109589041 - type: mrr_at_5 value: 36.0216894977169 - type: nauc_map_at_1000_diff1 value: 44.041894545810635 - type: nauc_map_at_1000_max value: 41.17362572555018 - type: nauc_map_at_1000_std value: 8.767356428542685 - type: nauc_map_at_100_diff1 value: 44.04056956484395 - type: nauc_map_at_100_max value: 41.18083594913976 - type: nauc_map_at_100_std value: 8.752657132178738 - type: nauc_map_at_10_diff1 value: 44.363453458247925 - type: nauc_map_at_10_max value: 40.75918614278003 - type: nauc_map_at_10_std value: 7.772306317028583 - type: nauc_map_at_1_diff1 value: 51.272585822020254 - type: nauc_map_at_1_max value: 37.645585910953386 - type: nauc_map_at_1_std value: 3.2801156746839997 - type: nauc_map_at_20_diff1 value: 44.03116886698072 - type: nauc_map_at_20_max value: 41.02335530846115 - type: nauc_map_at_20_std value: 8.342361143255655 - type: nauc_map_at_3_diff1 value: 44.70706747094596 - type: nauc_map_at_3_max value: 39.030651844405575 - type: nauc_map_at_3_std value: 5.884095706583685 - type: nauc_map_at_5_diff1 value: 44.4544981640302 - type: nauc_map_at_5_max value: 39.635461711896916 - type: nauc_map_at_5_std value: 6.632250546113723 - type: nauc_mrr_at_1000_diff1 value: 42.81708824392129 - type: nauc_mrr_at_1000_max value: 42.57093551420639 - type: nauc_mrr_at_1000_std value: 11.441585136182013 - type: nauc_mrr_at_100_diff1 value: 42.81275458475952 - type: nauc_mrr_at_100_max value: 42.582787877166965 - type: nauc_mrr_at_100_std value: 11.44975978035579 - type: nauc_mrr_at_10_diff1 value: 42.72063226967484 - type: nauc_mrr_at_10_max value: 42.556953195342615 - type: nauc_mrr_at_10_std value: 11.105965017912668 - type: nauc_mrr_at_1_diff1 value: 50.5113885243745 - type: nauc_mrr_at_1_max value: 41.93828222541368 - type: nauc_mrr_at_1_std value: 9.159606325340556 - type: nauc_mrr_at_20_diff1 value: 42.77647311397092 - type: nauc_mrr_at_20_max value: 42.53014349662046 - type: nauc_mrr_at_20_std value: 11.260887899494547 - type: nauc_mrr_at_3_diff1 value: 42.5769549053029 - type: nauc_mrr_at_3_max value: 41.862496724071704 - type: nauc_mrr_at_3_std value: 10.210790947470494 - type: nauc_mrr_at_5_diff1 value: 42.95929464184399 - type: nauc_mrr_at_5_max value: 42.300359084486274 - type: nauc_mrr_at_5_std value: 10.467983162075953 - type: nauc_ndcg_at_1000_diff1 value: 41.39614751382601 - type: nauc_ndcg_at_1000_max value: 42.6186909427122 - type: nauc_ndcg_at_1000_std value: 13.121900374296718 - type: nauc_ndcg_at_100_diff1 value: 41.235561668154006 - type: nauc_ndcg_at_100_max value: 42.994411802403185 - type: nauc_ndcg_at_100_std value: 13.658642895310704 - type: nauc_ndcg_at_10_diff1 value: 41.79496737016758 - type: nauc_ndcg_at_10_max value: 41.890657768164786 - type: nauc_ndcg_at_10_std value: 10.231134586943591 - type: nauc_ndcg_at_1_diff1 value: 50.5113885243745 - type: nauc_ndcg_at_1_max value: 41.93828222541368 - type: nauc_ndcg_at_1_std value: 9.159606325340556 - type: nauc_ndcg_at_20_diff1 value: 41.186060008922745 - type: nauc_ndcg_at_20_max value: 42.172791223117336 - type: nauc_ndcg_at_20_std value: 11.34527301101956 - type: nauc_ndcg_at_3_diff1 value: 41.681565698112294 - type: nauc_ndcg_at_3_max value: 39.845797667995306 - type: nauc_ndcg_at_3_std value: 7.940462212664345 - type: nauc_ndcg_at_5_diff1 value: 41.967719742308994 - type: nauc_ndcg_at_5_max value: 40.225831354460404 - type: nauc_ndcg_at_5_std value: 8.261587245067236 - type: nauc_precision_at_1000_diff1 value: -12.943400339828257 - type: nauc_precision_at_1000_max value: -1.1246092172976385 - type: nauc_precision_at_1000_std value: 9.16540771536695 - type: nauc_precision_at_100_diff1 value: 1.0038742911334513 - type: nauc_precision_at_100_max value: 19.43548294193961 - type: nauc_precision_at_100_std value: 22.80991154538675 - type: nauc_precision_at_10_diff1 value: 21.219025348692206 - type: nauc_precision_at_10_max value: 38.12618998197481 - type: nauc_precision_at_10_std value: 19.666478896577768 - type: nauc_precision_at_1_diff1 value: 50.5113885243745 - type: nauc_precision_at_1_max value: 41.93828222541368 - type: nauc_precision_at_1_std value: 9.159606325340556 - type: nauc_precision_at_20_diff1 value: 12.157890222623376 - type: nauc_precision_at_20_max value: 31.805010750506018 - type: nauc_precision_at_20_std value: 22.04108879323118 - type: nauc_precision_at_3_diff1 value: 30.42582719684319 - type: nauc_precision_at_3_max value: 40.56014217819822 - type: nauc_precision_at_3_std value: 13.992907571064775 - type: nauc_precision_at_5_diff1 value: 26.390186333811126 - type: nauc_precision_at_5_max value: 38.91468839125907 - type: nauc_precision_at_5_std value: 15.715475645777822 - type: nauc_recall_at_1000_diff1 value: 7.224229291228593 - type: nauc_recall_at_1000_max value: 44.352809276303745 - type: nauc_recall_at_1000_std value: 59.80498706867754 - type: nauc_recall_at_100_diff1 value: 26.880858714601658 - type: nauc_recall_at_100_max value: 44.69680037474241 - type: nauc_recall_at_100_std value: 33.36901319726501 - type: nauc_recall_at_10_diff1 value: 33.972463386849014 - type: nauc_recall_at_10_max value: 39.932831237464626 - type: nauc_recall_at_10_std value: 13.234453796470213 - type: nauc_recall_at_1_diff1 value: 51.272585822020254 - type: nauc_recall_at_1_max value: 37.645585910953386 - type: nauc_recall_at_1_std value: 3.2801156746839997 - type: nauc_recall_at_20_diff1 value: 31.42963785503753 - type: nauc_recall_at_20_max value: 40.02728183031791 - type: nauc_recall_at_20_std value: 16.412858575517493 - type: nauc_recall_at_3_diff1 value: 36.14293146571133 - type: nauc_recall_at_3_max value: 36.181181225211205 - type: nauc_recall_at_3_std value: 6.450986128210194 - type: nauc_recall_at_5_diff1 value: 35.55122852349323 - type: nauc_recall_at_5_max value: 36.55019755945847 - type: nauc_recall_at_5_std value: 7.672244545272601 - type: ndcg_at_1 value: 26.712000000000003 - type: ndcg_at_10 value: 38.968 - type: ndcg_at_100 value: 44.64 - type: ndcg_at_1000 value: 47.193000000000005 - type: ndcg_at_20 value: 41.203 - type: ndcg_at_3 value: 33.311 - type: ndcg_at_5 value: 35.76 - type: precision_at_1 value: 26.712000000000003 - type: precision_at_10 value: 7.534000000000001 - type: precision_at_100 value: 1.2149999999999999 - type: precision_at_1000 value: 0.163 - type: precision_at_20 value: 4.566 - type: precision_at_3 value: 16.476 - type: precision_at_5 value: 12.009 - type: recall_at_1 value: 22.081999999999997 - type: recall_at_10 value: 52.859 - type: recall_at_100 value: 76.812 - type: recall_at_1000 value: 94.248 - type: recall_at_20 value: 60.378 - type: recall_at_3 value: 36.964999999999996 - type: recall_at_5 value: 43.338 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackRetrieval (default) revision: CQADupstackRetrieval_is_a_combined_dataset split: test type: CQADupstackRetrieval_is_a_combined_dataset metrics: - type: main_score value: 41.32475 - type: ndcg_at_10 value: 41.32475 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackStatsRetrieval (default) revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a split: test type: mteb/cqadupstack-stats metrics: - type: main_score value: 35.379 - type: map_at_1 value: 23.147000000000002 - type: map_at_10 value: 30.867 - type: map_at_100 value: 31.961000000000002 - type: map_at_1000 value: 32.074999999999996 - type: map_at_20 value: 31.451 - type: map_at_3 value: 28.598000000000003 - type: map_at_5 value: 29.715000000000003 - type: mrr_at_1 value: 26.07361963190184 - type: mrr_at_10 value: 33.78596990943617 - type: mrr_at_100 value: 34.69551634061485 - type: mrr_at_1000 value: 34.77295317920054 - type: mrr_at_20 value: 34.28410324873221 - type: mrr_at_3 value: 31.901840490797547 - type: mrr_at_5 value: 32.70705521472393 - type: nauc_map_at_1000_diff1 value: 42.51208917704474 - type: nauc_map_at_1000_max value: 36.32219027239691 - type: nauc_map_at_1000_std value: 3.7599632795506417 - type: nauc_map_at_100_diff1 value: 42.50198316718319 - type: nauc_map_at_100_max value: 36.28499029077374 - type: nauc_map_at_100_std value: 3.7301933622893144 - type: nauc_map_at_10_diff1 value: 42.335456854540816 - type: nauc_map_at_10_max value: 35.9706844819072 - type: nauc_map_at_10_std value: 3.0121166680691034 - type: nauc_map_at_1_diff1 value: 48.21792116617763 - type: nauc_map_at_1_max value: 32.422399618563205 - type: nauc_map_at_1_std value: -2.2529709040289214 - type: nauc_map_at_20_diff1 value: 42.6392464609064 - type: nauc_map_at_20_max value: 36.30364987133835 - type: nauc_map_at_20_std value: 3.4854467402144116 - type: nauc_map_at_3_diff1 value: 42.31934484905233 - type: nauc_map_at_3_max value: 34.49722205197702 - type: nauc_map_at_3_std value: 1.3826788859308912 - type: nauc_map_at_5_diff1 value: 41.695914770923025 - type: nauc_map_at_5_max value: 34.9293262652828 - type: nauc_map_at_5_std value: 2.268052263977006 - type: nauc_mrr_at_1000_diff1 value: 42.953573052747785 - type: nauc_mrr_at_1000_max value: 39.23622168446037 - type: nauc_mrr_at_1000_std value: 7.062229019755459 - type: nauc_mrr_at_100_diff1 value: 42.94054236395847 - type: nauc_mrr_at_100_max value: 39.22993003862566 - type: nauc_mrr_at_100_std value: 7.064981248046304 - type: nauc_mrr_at_10_diff1 value: 42.97514355223379 - type: nauc_mrr_at_10_max value: 39.18214683331138 - type: nauc_mrr_at_10_std value: 6.611133670202421 - type: nauc_mrr_at_1_diff1 value: 48.41118406440152 - type: nauc_mrr_at_1_max value: 39.19901795383339 - type: nauc_mrr_at_1_std value: 4.184238273499772 - type: nauc_mrr_at_20_diff1 value: 43.06740774183079 - type: nauc_mrr_at_20_max value: 39.25274434382343 - type: nauc_mrr_at_20_std value: 6.927365321881451 - type: nauc_mrr_at_3_diff1 value: 42.48514891484537 - type: nauc_mrr_at_3_max value: 38.491353279339776 - type: nauc_mrr_at_3_std value: 5.930261116335607 - type: nauc_mrr_at_5_diff1 value: 42.33101067028797 - type: nauc_mrr_at_5_max value: 38.37707622201782 - type: nauc_mrr_at_5_std value: 6.033822417562119 - type: nauc_ndcg_at_1000_diff1 value: 41.154900505541065 - type: nauc_ndcg_at_1000_max value: 38.24520451359673 - type: nauc_ndcg_at_1000_std value: 8.053909931241048 - type: nauc_ndcg_at_100_diff1 value: 41.008224147466706 - type: nauc_ndcg_at_100_max value: 38.03003963108958 - type: nauc_ndcg_at_100_std value: 8.029055010625735 - type: nauc_ndcg_at_10_diff1 value: 41.38757336254797 - type: nauc_ndcg_at_10_max value: 37.69449355625365 - type: nauc_ndcg_at_10_std value: 5.501776175296825 - type: nauc_ndcg_at_1_diff1 value: 48.41118406440152 - type: nauc_ndcg_at_1_max value: 39.19901795383339 - type: nauc_ndcg_at_1_std value: 4.184238273499772 - type: nauc_ndcg_at_20_diff1 value: 42.07413489567003 - type: nauc_ndcg_at_20_max value: 38.36958336029682 - type: nauc_ndcg_at_20_std value: 6.725957802525416 - type: nauc_ndcg_at_3_diff1 value: 40.55437805117007 - type: nauc_ndcg_at_3_max value: 35.88449899248541 - type: nauc_ndcg_at_3_std value: 3.6748580378083586 - type: nauc_ndcg_at_5_diff1 value: 40.010471092618246 - type: nauc_ndcg_at_5_max value: 35.88766927207923 - type: nauc_ndcg_at_5_std value: 4.207367413979825 - type: nauc_precision_at_1000_diff1 value: -0.627954206482575 - type: nauc_precision_at_1000_max value: 16.87253851929619 - type: nauc_precision_at_1000_std value: 20.006671502432432 - type: nauc_precision_at_100_diff1 value: 14.668946009188208 - type: nauc_precision_at_100_max value: 29.742017118053397 - type: nauc_precision_at_100_std value: 25.20619621555537 - type: nauc_precision_at_10_diff1 value: 28.297723813286847 - type: nauc_precision_at_10_max value: 43.75323355613635 - type: nauc_precision_at_10_std value: 21.41675895539336 - type: nauc_precision_at_1_diff1 value: 48.41118406440152 - type: nauc_precision_at_1_max value: 39.19901795383339 - type: nauc_precision_at_1_std value: 4.184238273499772 - type: nauc_precision_at_20_diff1 value: 28.180742632762257 - type: nauc_precision_at_20_max value: 42.1557651237426 - type: nauc_precision_at_20_std value: 23.76852281392542 - type: nauc_precision_at_3_diff1 value: 32.14428304746157 - type: nauc_precision_at_3_max value: 41.05702645399837 - type: nauc_precision_at_3_std value: 13.525455649176054 - type: nauc_precision_at_5_diff1 value: 27.518551234674852 - type: nauc_precision_at_5_max value: 40.244849049438955 - type: nauc_precision_at_5_std value: 16.955839126853174 - type: nauc_recall_at_1000_diff1 value: 23.387951594003543 - type: nauc_recall_at_1000_max value: 40.53645639097819 - type: nauc_recall_at_1000_std value: 38.15241870854853 - type: nauc_recall_at_100_diff1 value: 31.320417507693122 - type: nauc_recall_at_100_max value: 37.61668703163837 - type: nauc_recall_at_100_std value: 21.3973156433774 - type: nauc_recall_at_10_diff1 value: 37.53430570520243 - type: nauc_recall_at_10_max value: 38.450456066919294 - type: nauc_recall_at_10_std value: 8.777785896723772 - type: nauc_recall_at_1_diff1 value: 48.21792116617763 - type: nauc_recall_at_1_max value: 32.422399618563205 - type: nauc_recall_at_1_std value: -2.2529709040289214 - type: nauc_recall_at_20_diff1 value: 38.90462139302631 - type: nauc_recall_at_20_max value: 39.777813182962014 - type: nauc_recall_at_20_std value: 12.368553526936706 - type: nauc_recall_at_3_diff1 value: 36.12960620845285 - type: nauc_recall_at_3_max value: 33.625725775340115 - type: nauc_recall_at_3_std value: 3.1934379790504277 - type: nauc_recall_at_5_diff1 value: 34.68344816631666 - type: nauc_recall_at_5_max value: 34.096665584351086 - type: nauc_recall_at_5_std value: 4.965244979957592 - type: ndcg_at_1 value: 26.074 - type: ndcg_at_10 value: 35.379 - type: ndcg_at_100 value: 40.668 - type: ndcg_at_1000 value: 43.271 - type: ndcg_at_20 value: 37.3 - type: ndcg_at_3 value: 31.291000000000004 - type: ndcg_at_5 value: 32.828 - type: precision_at_1 value: 26.074 - type: precision_at_10 value: 5.782 - type: precision_at_100 value: 0.9159999999999999 - type: precision_at_1000 value: 0.121 - type: precision_at_20 value: 3.39 - type: precision_at_3 value: 13.905999999999999 - type: precision_at_5 value: 9.508999999999999 - type: recall_at_1 value: 23.147000000000002 - type: recall_at_10 value: 46.308 - type: recall_at_100 value: 70.529 - type: recall_at_1000 value: 89.53 - type: recall_at_20 value: 53.510000000000005 - type: recall_at_3 value: 34.504000000000005 - type: recall_at_5 value: 38.472 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackTexRetrieval (default) revision: 46989137a86843e03a6195de44b09deda022eec7 split: test type: mteb/cqadupstack-tex metrics: - type: main_score value: 30.519000000000002 - type: map_at_1 value: 17.573 - type: map_at_10 value: 25.480999999999998 - type: map_at_100 value: 26.740000000000002 - type: map_at_1000 value: 26.881 - type: map_at_20 value: 26.125999999999998 - type: map_at_3 value: 22.962 - type: map_at_5 value: 24.366 - type: mrr_at_1 value: 21.782518926359256 - type: mrr_at_10 value: 29.551570915129073 - type: mrr_at_100 value: 30.563788618793907 - type: mrr_at_1000 value: 30.64724659416222 - type: mrr_at_20 value: 30.1023182624383 - type: mrr_at_3 value: 27.248222069281947 - type: mrr_at_5 value: 28.540376233080984 - type: nauc_map_at_1000_diff1 value: 31.881175131210888 - type: nauc_map_at_1000_max value: 29.300696375017903 - type: nauc_map_at_1000_std value: 3.8823292626839456 - type: nauc_map_at_100_diff1 value: 31.91438127161964 - type: nauc_map_at_100_max value: 29.252651714818867 - type: nauc_map_at_100_std value: 3.8321881954789876 - type: nauc_map_at_10_diff1 value: 32.171669036580084 - type: nauc_map_at_10_max value: 28.901773247218653 - type: nauc_map_at_10_std value: 3.326466043614621 - type: nauc_map_at_1_diff1 value: 37.444204623648204 - type: nauc_map_at_1_max value: 27.97756203885853 - type: nauc_map_at_1_std value: 0.36949231909922875 - type: nauc_map_at_20_diff1 value: 31.979938373253127 - type: nauc_map_at_20_max value: 29.04729001511327 - type: nauc_map_at_20_std value: 3.4743150945293406 - type: nauc_map_at_3_diff1 value: 32.915907037588454 - type: nauc_map_at_3_max value: 28.113160819217768 - type: nauc_map_at_3_std value: 2.1821842033363197 - type: nauc_map_at_5_diff1 value: 32.31836184904865 - type: nauc_map_at_5_max value: 28.382450988689257 - type: nauc_map_at_5_std value: 2.647063248227251 - type: nauc_mrr_at_1000_diff1 value: 30.340557224068306 - type: nauc_mrr_at_1000_max value: 30.055801929203373 - type: nauc_mrr_at_1000_std value: 4.664666667473129 - type: nauc_mrr_at_100_diff1 value: 30.350033762326877 - type: nauc_mrr_at_100_max value: 30.053743987411675 - type: nauc_mrr_at_100_std value: 4.661666112298184 - type: nauc_mrr_at_10_diff1 value: 30.487849925791462 - type: nauc_mrr_at_10_max value: 30.07149856764245 - type: nauc_mrr_at_10_std value: 4.44378828352727 - type: nauc_mrr_at_1_diff1 value: 35.16105289700225 - type: nauc_mrr_at_1_max value: 30.496689088104585 - type: nauc_mrr_at_1_std value: 1.8902302298962788 - type: nauc_mrr_at_20_diff1 value: 30.25575973094628 - type: nauc_mrr_at_20_max value: 29.929532643775552 - type: nauc_mrr_at_20_std value: 4.504962325540269 - type: nauc_mrr_at_3_diff1 value: 30.711071703778686 - type: nauc_mrr_at_3_max value: 29.663764134099473 - type: nauc_mrr_at_3_std value: 3.320193337280269 - type: nauc_mrr_at_5_diff1 value: 30.476922599219552 - type: nauc_mrr_at_5_max value: 29.85199369614983 - type: nauc_mrr_at_5_std value: 3.9318122348403928 - type: nauc_ndcg_at_1000_diff1 value: 29.24537085275522 - type: nauc_ndcg_at_1000_max value: 30.46819189338767 - type: nauc_ndcg_at_1000_std value: 7.106119481191128 - type: nauc_ndcg_at_100_diff1 value: 29.555127473225106 - type: nauc_ndcg_at_100_max value: 30.313025639748854 - type: nauc_ndcg_at_100_std value: 7.100492334790892 - type: nauc_ndcg_at_10_diff1 value: 30.172626798643286 - type: nauc_ndcg_at_10_max value: 29.569399019833753 - type: nauc_ndcg_at_10_std value: 5.333855439010787 - type: nauc_ndcg_at_1_diff1 value: 35.16105289700225 - type: nauc_ndcg_at_1_max value: 30.496689088104585 - type: nauc_ndcg_at_1_std value: 1.8902302298962788 - type: nauc_ndcg_at_20_diff1 value: 29.601538139574433 - type: nauc_ndcg_at_20_max value: 29.511195260623875 - type: nauc_ndcg_at_20_std value: 5.59012025163207 - type: nauc_ndcg_at_3_diff1 value: 30.692765366524306 - type: nauc_ndcg_at_3_max value: 28.78218499164891 - type: nauc_ndcg_at_3_std value: 3.1036538657542896 - type: nauc_ndcg_at_5_diff1 value: 30.23085391402633 - type: nauc_ndcg_at_5_max value: 28.812813896142124 - type: nauc_ndcg_at_5_std value: 3.946855139104198 - type: nauc_precision_at_1000_diff1 value: -5.807690994963469 - type: nauc_precision_at_1000_max value: 20.20712256250324 - type: nauc_precision_at_1000_std value: 14.321049866221422 - type: nauc_precision_at_100_diff1 value: 3.9668914526416437 - type: nauc_precision_at_100_max value: 26.84931463902196 - type: nauc_precision_at_100_std value: 16.787333115848682 - type: nauc_precision_at_10_diff1 value: 17.595096225964767 - type: nauc_precision_at_10_max value: 31.638328874193032 - type: nauc_precision_at_10_std value: 11.100027061781638 - type: nauc_precision_at_1_diff1 value: 35.16105289700225 - type: nauc_precision_at_1_max value: 30.496689088104585 - type: nauc_precision_at_1_std value: 1.8902302298962788 - type: nauc_precision_at_20_diff1 value: 12.906680878575328 - type: nauc_precision_at_20_max value: 29.696422699114635 - type: nauc_precision_at_20_std value: 12.017829462542974 - type: nauc_precision_at_3_diff1 value: 24.474722384406093 - type: nauc_precision_at_3_max value: 30.295991671763854 - type: nauc_precision_at_3_std value: 5.359770976628988 - type: nauc_precision_at_5_diff1 value: 20.9210542273083 - type: nauc_precision_at_5_max value: 30.803816020272418 - type: nauc_precision_at_5_std value: 7.650041221503042 - type: nauc_recall_at_1000_diff1 value: 7.868010062781431 - type: nauc_recall_at_1000_max value: 34.4127297805766 - type: nauc_recall_at_1000_std value: 32.83607435392193 - type: nauc_recall_at_100_diff1 value: 21.18500764361835 - type: nauc_recall_at_100_max value: 30.24295887819678 - type: nauc_recall_at_100_std value: 18.463080010741866 - type: nauc_recall_at_10_diff1 value: 25.17302253212331 - type: nauc_recall_at_10_max value: 27.36615448456723 - type: nauc_recall_at_10_std value: 8.69018868943956 - type: nauc_recall_at_1_diff1 value: 37.444204623648204 - type: nauc_recall_at_1_max value: 27.97756203885853 - type: nauc_recall_at_1_std value: 0.36949231909922875 - type: nauc_recall_at_20_diff1 value: 23.076180124742322 - type: nauc_recall_at_20_max value: 26.759809589883787 - type: nauc_recall_at_20_std value: 9.509183248844606 - type: nauc_recall_at_3_diff1 value: 27.559573848327812 - type: nauc_recall_at_3_max value: 25.585037245374835 - type: nauc_recall_at_3_std value: 3.403071467061742 - type: nauc_recall_at_5_diff1 value: 25.939753995717375 - type: nauc_recall_at_5_max value: 25.715892642199158 - type: nauc_recall_at_5_std value: 4.895015921257684 - type: ndcg_at_1 value: 21.783 - type: ndcg_at_10 value: 30.519000000000002 - type: ndcg_at_100 value: 36.449 - type: ndcg_at_1000 value: 39.476 - type: ndcg_at_20 value: 32.57 - type: ndcg_at_3 value: 26.104 - type: ndcg_at_5 value: 28.142 - type: precision_at_1 value: 21.783 - type: precision_at_10 value: 5.716 - type: precision_at_100 value: 1.036 - type: precision_at_1000 value: 0.149 - type: precision_at_20 value: 3.5000000000000004 - type: precision_at_3 value: 12.629000000000001 - type: precision_at_5 value: 9.188 - type: recall_at_1 value: 17.573 - type: recall_at_10 value: 41.565999999999995 - type: recall_at_100 value: 68.31099999999999 - type: recall_at_1000 value: 89.66 - type: recall_at_20 value: 49.033 - type: recall_at_3 value: 28.998 - type: recall_at_5 value: 34.409 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackUnixRetrieval (default) revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53 split: test type: mteb/cqadupstack-unix metrics: - type: main_score value: 41.31 - type: map_at_1 value: 25.393 - type: map_at_10 value: 35.408 - type: map_at_100 value: 36.765 - type: map_at_1000 value: 36.870000000000005 - type: map_at_20 value: 36.186 - type: map_at_3 value: 31.858999999999998 - type: map_at_5 value: 34.088 - type: mrr_at_1 value: 30.410447761194032 - type: mrr_at_10 value: 39.62523691068467 - type: mrr_at_100 value: 40.58641984797069 - type: mrr_at_1000 value: 40.64414774113457 - type: mrr_at_20 value: 40.186787569713786 - type: mrr_at_3 value: 36.53606965174129 - type: mrr_at_5 value: 38.5136815920398 - type: nauc_map_at_1000_diff1 value: 45.3237924349066 - type: nauc_map_at_1000_max value: 42.882551041895475 - type: nauc_map_at_1000_std value: -2.061776932133938 - type: nauc_map_at_100_diff1 value: 45.31179559669483 - type: nauc_map_at_100_max value: 42.88055143410839 - type: nauc_map_at_100_std value: -2.0619544266001766 - type: nauc_map_at_10_diff1 value: 45.246469302227034 - type: nauc_map_at_10_max value: 42.43898453050511 - type: nauc_map_at_10_std value: -2.601046837280869 - type: nauc_map_at_1_diff1 value: 51.10553001605208 - type: nauc_map_at_1_max value: 41.25214004071871 - type: nauc_map_at_1_std value: -5.9190349301612075 - type: nauc_map_at_20_diff1 value: 45.231373982382955 - type: nauc_map_at_20_max value: 42.66345871982157 - type: nauc_map_at_20_std value: -2.26108364692999 - type: nauc_map_at_3_diff1 value: 46.23862364083718 - type: nauc_map_at_3_max value: 41.88503103268463 - type: nauc_map_at_3_std value: -4.084509652842847 - type: nauc_map_at_5_diff1 value: 45.76253491098928 - type: nauc_map_at_5_max value: 42.11909174581756 - type: nauc_map_at_5_std value: -3.152593546431723 - type: nauc_mrr_at_1000_diff1 value: 45.16287764406163 - type: nauc_mrr_at_1000_max value: 44.28490255584458 - type: nauc_mrr_at_1000_std value: -0.851874205124679 - type: nauc_mrr_at_100_diff1 value: 45.14662264595562 - type: nauc_mrr_at_100_max value: 44.28754816492547 - type: nauc_mrr_at_100_std value: -0.827775845705049 - type: nauc_mrr_at_10_diff1 value: 45.05127336584147 - type: nauc_mrr_at_10_max value: 44.25978209317813 - type: nauc_mrr_at_10_std value: -0.9734920470308909 - type: nauc_mrr_at_1_diff1 value: 50.63689908117468 - type: nauc_mrr_at_1_max value: 44.57852646244725 - type: nauc_mrr_at_1_std value: -4.059128543246773 - type: nauc_mrr_at_20_diff1 value: 45.044852569128444 - type: nauc_mrr_at_20_max value: 44.20947897727891 - type: nauc_mrr_at_20_std value: -0.8526642429773674 - type: nauc_mrr_at_3_diff1 value: 45.45435070485086 - type: nauc_mrr_at_3_max value: 44.257326484777224 - type: nauc_mrr_at_3_std value: -2.1608895747068093 - type: nauc_mrr_at_5_diff1 value: 45.24789730211493 - type: nauc_mrr_at_5_max value: 44.10858723544344 - type: nauc_mrr_at_5_std value: -1.3663611847409747 - type: nauc_ndcg_at_1000_diff1 value: 43.50803190980708 - type: nauc_ndcg_at_1000_max value: 43.66343887264139 - type: nauc_ndcg_at_1000_std value: 0.8232157246006608 - type: nauc_ndcg_at_100_diff1 value: 43.21201807639179 - type: nauc_ndcg_at_100_max value: 43.66944092609106 - type: nauc_ndcg_at_100_std value: 1.2771093902231845 - type: nauc_ndcg_at_10_diff1 value: 42.99494568521883 - type: nauc_ndcg_at_10_max value: 42.80862067566825 - type: nauc_ndcg_at_10_std value: -0.6520481559808629 - type: nauc_ndcg_at_1_diff1 value: 50.63689908117468 - type: nauc_ndcg_at_1_max value: 44.57852646244725 - type: nauc_ndcg_at_1_std value: -4.059128543246773 - type: nauc_ndcg_at_20_diff1 value: 42.83217912527002 - type: nauc_ndcg_at_20_max value: 43.161124041011675 - type: nauc_ndcg_at_20_std value: 0.407283232341166 - type: nauc_ndcg_at_3_diff1 value: 44.56968073360208 - type: nauc_ndcg_at_3_max value: 42.93029489205904 - type: nauc_ndcg_at_3_std value: -3.1396422724943855 - type: nauc_ndcg_at_5_diff1 value: 44.02970798279963 - type: nauc_ndcg_at_5_max value: 42.51569472321148 - type: nauc_ndcg_at_5_std value: -1.757501561649831 - type: nauc_precision_at_1000_diff1 value: -10.598699181466678 - type: nauc_precision_at_1000_max value: 1.784813164595759 - type: nauc_precision_at_1000_std value: 7.16918104334376 - type: nauc_precision_at_100_diff1 value: 2.473069128343708 - type: nauc_precision_at_100_max value: 19.13842474812851 - type: nauc_precision_at_100_std value: 13.648804134138286 - type: nauc_precision_at_10_diff1 value: 20.442243228917654 - type: nauc_precision_at_10_max value: 36.5015082373227 - type: nauc_precision_at_10_std value: 8.070385983398559 - type: nauc_precision_at_1_diff1 value: 50.63689908117468 - type: nauc_precision_at_1_max value: 44.57852646244725 - type: nauc_precision_at_1_std value: -4.059128543246773 - type: nauc_precision_at_20_diff1 value: 14.744878445902772 - type: nauc_precision_at_20_max value: 32.527747228894064 - type: nauc_precision_at_20_std value: 11.67200027534835 - type: nauc_precision_at_3_diff1 value: 35.09915972547601 - type: nauc_precision_at_3_max value: 43.08085093225614 - type: nauc_precision_at_3_std value: 0.9578865822763707 - type: nauc_precision_at_5_diff1 value: 29.124386207816823 - type: nauc_precision_at_5_max value: 40.68095646716276 - type: nauc_precision_at_5_std value: 5.077110628260498 - type: nauc_recall_at_1000_diff1 value: 22.38553348188135 - type: nauc_recall_at_1000_max value: 41.789440224180886 - type: nauc_recall_at_1000_std value: 44.11372608956349 - type: nauc_recall_at_100_diff1 value: 30.944884049597054 - type: nauc_recall_at_100_max value: 39.74965194127809 - type: nauc_recall_at_100_std value: 17.335161783835254 - type: nauc_recall_at_10_diff1 value: 33.77972605382363 - type: nauc_recall_at_10_max value: 37.96828171185741 - type: nauc_recall_at_10_std value: 2.878193454017939 - type: nauc_recall_at_1_diff1 value: 51.10553001605208 - type: nauc_recall_at_1_max value: 41.25214004071871 - type: nauc_recall_at_1_std value: -5.9190349301612075 - type: nauc_recall_at_20_diff1 value: 32.15944402025048 - type: nauc_recall_at_20_max value: 38.553123596617475 - type: nauc_recall_at_20_std value: 7.185913655178715 - type: nauc_recall_at_3_diff1 value: 39.30789421964286 - type: nauc_recall_at_3_max value: 39.38989571990551 - type: nauc_recall_at_3_std value: -2.4776813018682304 - type: nauc_recall_at_5_diff1 value: 37.23603695010937 - type: nauc_recall_at_5_max value: 38.24172443271927 - type: nauc_recall_at_5_std value: 0.41240611008696 - type: ndcg_at_1 value: 30.409999999999997 - type: ndcg_at_10 value: 41.31 - type: ndcg_at_100 value: 47.317 - type: ndcg_at_1000 value: 49.451 - type: ndcg_at_20 value: 43.678 - type: ndcg_at_3 value: 35.156 - type: ndcg_at_5 value: 38.550000000000004 - type: precision_at_1 value: 30.409999999999997 - type: precision_at_10 value: 7.285 - type: precision_at_100 value: 1.16 - type: precision_at_1000 value: 0.145 - type: precision_at_20 value: 4.3 - type: precision_at_3 value: 16.2 - type: precision_at_5 value: 12.015 - type: recall_at_1 value: 25.393 - type: recall_at_10 value: 54.955 - type: recall_at_100 value: 81.074 - type: recall_at_1000 value: 95.517 - type: recall_at_20 value: 63.336000000000006 - type: recall_at_3 value: 38.646 - type: recall_at_5 value: 47.155 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackWebmastersRetrieval (default) revision: 160c094312a0e1facb97e55eeddb698c0abe3571 split: test type: mteb/cqadupstack-webmasters metrics: - type: main_score value: 40.467 - type: map_at_1 value: 25.219 - type: map_at_10 value: 34.317 - type: map_at_100 value: 36.099 - type: map_at_1000 value: 36.339 - type: map_at_20 value: 35.207 - type: map_at_3 value: 31.118000000000002 - type: map_at_5 value: 32.759 - type: mrr_at_1 value: 30.039525691699602 - type: mrr_at_10 value: 39.2148189974277 - type: mrr_at_100 value: 40.2161521379955 - type: mrr_at_1000 value: 40.265703712297686 - type: mrr_at_20 value: 39.77515544005589 - type: mrr_at_3 value: 36.29776021080369 - type: mrr_at_5 value: 37.85902503293808 - type: nauc_map_at_1000_diff1 value: 42.249285823910625 - type: nauc_map_at_1000_max value: 37.81729357104585 - type: nauc_map_at_1000_std value: 3.378421859371135 - type: nauc_map_at_100_diff1 value: 42.26973168859294 - type: nauc_map_at_100_max value: 37.71021506487753 - type: nauc_map_at_100_std value: 3.0599597011125255 - type: nauc_map_at_10_diff1 value: 42.62764810062953 - type: nauc_map_at_10_max value: 37.70978577391228 - type: nauc_map_at_10_std value: 1.8639905372216925 - type: nauc_map_at_1_diff1 value: 50.640732359157305 - type: nauc_map_at_1_max value: 33.30575957486884 - type: nauc_map_at_1_std value: -1.6408504568688285 - type: nauc_map_at_20_diff1 value: 42.49959845025793 - type: nauc_map_at_20_max value: 37.434835390720735 - type: nauc_map_at_20_std value: 2.128763652038146 - type: nauc_map_at_3_diff1 value: 43.93770137567739 - type: nauc_map_at_3_max value: 36.363959841283624 - type: nauc_map_at_3_std value: 0.5712143645184623 - type: nauc_map_at_5_diff1 value: 42.987763840097216 - type: nauc_map_at_5_max value: 37.07794713015937 - type: nauc_map_at_5_std value: 1.097852086925236 - type: nauc_mrr_at_1000_diff1 value: 40.66271045590558 - type: nauc_mrr_at_1000_max value: 38.08244696996338 - type: nauc_mrr_at_1000_std value: 6.051657410597092 - type: nauc_mrr_at_100_diff1 value: 40.619228571947225 - type: nauc_mrr_at_100_max value: 38.07828766827722 - type: nauc_mrr_at_100_std value: 6.069557283626944 - type: nauc_mrr_at_10_diff1 value: 40.485799180131835 - type: nauc_mrr_at_10_max value: 38.327695119460344 - type: nauc_mrr_at_10_std value: 6.01317781659517 - type: nauc_mrr_at_1_diff1 value: 46.729900872794204 - type: nauc_mrr_at_1_max value: 36.14627468706107 - type: nauc_mrr_at_1_std value: 4.402796904496076 - type: nauc_mrr_at_20_diff1 value: 40.65612564643004 - type: nauc_mrr_at_20_max value: 37.97791721709834 - type: nauc_mrr_at_20_std value: 5.802723977451494 - type: nauc_mrr_at_3_diff1 value: 40.821252306592314 - type: nauc_mrr_at_3_max value: 37.908667331976744 - type: nauc_mrr_at_3_std value: 5.316096828314028 - type: nauc_mrr_at_5_diff1 value: 41.05340039412131 - type: nauc_mrr_at_5_max value: 38.37605719831689 - type: nauc_mrr_at_5_std value: 5.213483061859129 - type: nauc_ndcg_at_1000_diff1 value: 39.88147024776181 - type: nauc_ndcg_at_1000_max value: 39.42772441796562 - type: nauc_ndcg_at_1000_std value: 7.4990876900763865 - type: nauc_ndcg_at_100_diff1 value: 38.89054295519447 - type: nauc_ndcg_at_100_max value: 38.92308897211485 - type: nauc_ndcg_at_100_std value: 7.885233156116329 - type: nauc_ndcg_at_10_diff1 value: 38.95442786328181 - type: nauc_ndcg_at_10_max value: 38.96665154095573 - type: nauc_ndcg_at_10_std value: 4.9802982912762435 - type: nauc_ndcg_at_1_diff1 value: 46.729900872794204 - type: nauc_ndcg_at_1_max value: 36.14627468706107 - type: nauc_ndcg_at_1_std value: 4.402796904496076 - type: nauc_ndcg_at_20_diff1 value: 39.17829178553532 - type: nauc_ndcg_at_20_max value: 37.4931955956734 - type: nauc_ndcg_at_20_std value: 4.596018393131905 - type: nauc_ndcg_at_3_diff1 value: 40.271638727924035 - type: nauc_ndcg_at_3_max value: 37.9826745644439 - type: nauc_ndcg_at_3_std value: 3.3572598270925216 - type: nauc_ndcg_at_5_diff1 value: 39.61366758008502 - type: nauc_ndcg_at_5_max value: 38.24456377281972 - type: nauc_ndcg_at_5_std value: 3.4817608168910166 - type: nauc_precision_at_1000_diff1 value: -11.622465140715828 - type: nauc_precision_at_1000_max value: 15.224960412516204 - type: nauc_precision_at_1000_std value: 34.73491517319145 - type: nauc_precision_at_100_diff1 value: -8.371026383253232 - type: nauc_precision_at_100_max value: 17.691072594176628 - type: nauc_precision_at_100_std value: 33.068086276168074 - type: nauc_precision_at_10_diff1 value: 11.176335364060336 - type: nauc_precision_at_10_max value: 37.13096692438884 - type: nauc_precision_at_10_std value: 20.553307297887073 - type: nauc_precision_at_1_diff1 value: 46.729900872794204 - type: nauc_precision_at_1_max value: 36.14627468706107 - type: nauc_precision_at_1_std value: 4.402796904496076 - type: nauc_precision_at_20_diff1 value: 3.693449256817259 - type: nauc_precision_at_20_max value: 27.01823261508418 - type: nauc_precision_at_20_std value: 23.20858961605156 - type: nauc_precision_at_3_diff1 value: 27.556711531090126 - type: nauc_precision_at_3_max value: 42.1929766320672 - type: nauc_precision_at_3_std value: 10.47253267125295 - type: nauc_precision_at_5_diff1 value: 20.191187550201118 - type: nauc_precision_at_5_max value: 40.28760124538548 - type: nauc_precision_at_5_std value: 13.938138706548745 - type: nauc_recall_at_1000_diff1 value: 28.858547583461792 - type: nauc_recall_at_1000_max value: 64.82322977199058 - type: nauc_recall_at_1000_std value: 62.95541267815066 - type: nauc_recall_at_100_diff1 value: 18.389599474131128 - type: nauc_recall_at_100_max value: 38.283013442171 - type: nauc_recall_at_100_std value: 32.53288294020897 - type: nauc_recall_at_10_diff1 value: 28.938406924345855 - type: nauc_recall_at_10_max value: 37.201339069853056 - type: nauc_recall_at_10_std value: 6.711404696968165 - type: nauc_recall_at_1_diff1 value: 50.640732359157305 - type: nauc_recall_at_1_max value: 33.30575957486884 - type: nauc_recall_at_1_std value: -1.6408504568688285 - type: nauc_recall_at_20_diff1 value: 27.15677059349287 - type: nauc_recall_at_20_max value: 29.95311918377292 - type: nauc_recall_at_20_std value: 5.269984079212893 - type: nauc_recall_at_3_diff1 value: 36.02691929591682 - type: nauc_recall_at_3_max value: 34.955041088527615 - type: nauc_recall_at_3_std value: 0.33989578439054857 - type: nauc_recall_at_5_diff1 value: 32.94914901826653 - type: nauc_recall_at_5_max value: 35.810195894176786 - type: nauc_recall_at_5_std value: 0.6467685151302079 - type: ndcg_at_1 value: 30.04 - type: ndcg_at_10 value: 40.467 - type: ndcg_at_100 value: 46.918 - type: ndcg_at_1000 value: 49.263 - type: ndcg_at_20 value: 42.853 - type: ndcg_at_3 value: 34.976 - type: ndcg_at_5 value: 37.345 - type: precision_at_1 value: 30.04 - type: precision_at_10 value: 7.786999999999999 - type: precision_at_100 value: 1.638 - type: precision_at_1000 value: 0.249 - type: precision_at_20 value: 5.0 - type: precision_at_3 value: 16.206 - type: precision_at_5 value: 11.976 - type: recall_at_1 value: 25.219 - type: recall_at_10 value: 52.443 - type: recall_at_100 value: 80.523 - type: recall_at_1000 value: 95.025 - type: recall_at_20 value: 61.317 - type: recall_at_3 value: 37.216 - type: recall_at_5 value: 43.086999999999996 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackWordpressRetrieval (default) revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4 split: test type: mteb/cqadupstack-wordpress metrics: - type: main_score value: 33.147999999999996 - type: map_at_1 value: 20.801 - type: map_at_10 value: 28.371000000000002 - type: map_at_100 value: 29.483999999999998 - type: map_at_1000 value: 29.602 - type: map_at_20 value: 29.015 - type: map_at_3 value: 25.790999999999997 - type: map_at_5 value: 27.025 - type: mrr_at_1 value: 22.73567467652495 - type: mrr_at_10 value: 30.274990464454422 - type: mrr_at_100 value: 31.240500066009304 - type: mrr_at_1000 value: 31.319717738953777 - type: mrr_at_20 value: 30.803996383475962 - type: mrr_at_3 value: 27.84966112138016 - type: mrr_at_5 value: 28.958718422674057 - type: nauc_map_at_1000_diff1 value: 30.950985415079412 - type: nauc_map_at_1000_max value: 31.59549203568236 - type: nauc_map_at_1000_std value: 4.803056268621205 - type: nauc_map_at_100_diff1 value: 30.94723046406967 - type: nauc_map_at_100_max value: 31.57394480311361 - type: nauc_map_at_100_std value: 4.837512208711833 - type: nauc_map_at_10_diff1 value: 31.171401087039285 - type: nauc_map_at_10_max value: 31.638390655676897 - type: nauc_map_at_10_std value: 4.534982253282165 - type: nauc_map_at_1_diff1 value: 38.237149754210726 - type: nauc_map_at_1_max value: 31.191178791236453 - type: nauc_map_at_1_std value: 0.12298428255657214 - type: nauc_map_at_20_diff1 value: 30.841529177079995 - type: nauc_map_at_20_max value: 31.599269377431604 - type: nauc_map_at_20_std value: 4.678845170315568 - type: nauc_map_at_3_diff1 value: 32.43547187741804 - type: nauc_map_at_3_max value: 30.41318267496344 - type: nauc_map_at_3_std value: 2.772894233835619 - type: nauc_map_at_5_diff1 value: 32.09431780702127 - type: nauc_map_at_5_max value: 31.4564477685958 - type: nauc_map_at_5_std value: 3.937401514669942 - type: nauc_mrr_at_1000_diff1 value: 30.606652910465677 - type: nauc_mrr_at_1000_max value: 33.142269356318806 - type: nauc_mrr_at_1000_std value: 6.022676880875486 - type: nauc_mrr_at_100_diff1 value: 30.565395214575187 - type: nauc_mrr_at_100_max value: 33.13271265557326 - type: nauc_mrr_at_100_std value: 6.048419255062242 - type: nauc_mrr_at_10_diff1 value: 30.579355991470248 - type: nauc_mrr_at_10_max value: 33.187118274483886 - type: nauc_mrr_at_10_std value: 5.705001216556541 - type: nauc_mrr_at_1_diff1 value: 39.311857430553474 - type: nauc_mrr_at_1_max value: 33.97087354489543 - type: nauc_mrr_at_1_std value: 1.6330160686638415 - type: nauc_mrr_at_20_diff1 value: 30.416078716055406 - type: nauc_mrr_at_20_max value: 33.15138236143527 - type: nauc_mrr_at_20_std value: 6.023106429375021 - type: nauc_mrr_at_3_diff1 value: 32.40012457172046 - type: nauc_mrr_at_3_max value: 32.758960987870964 - type: nauc_mrr_at_3_std value: 4.244944069148985 - type: nauc_mrr_at_5_diff1 value: 31.510030613661343 - type: nauc_mrr_at_5_max value: 33.234760081794775 - type: nauc_mrr_at_5_std value: 5.318059562676151 - type: nauc_ndcg_at_1000_diff1 value: 28.4386212441825 - type: nauc_ndcg_at_1000_max value: 32.30754779983019 - type: nauc_ndcg_at_1000_std value: 8.17776352566968 - type: nauc_ndcg_at_100_diff1 value: 27.132214912589582 - type: nauc_ndcg_at_100_max value: 31.625856416323956 - type: nauc_ndcg_at_100_std value: 9.042769717455885 - type: nauc_ndcg_at_10_diff1 value: 27.473488318876065 - type: nauc_ndcg_at_10_max value: 32.14506071223697 - type: nauc_ndcg_at_10_std value: 7.135810912180236 - type: nauc_ndcg_at_1_diff1 value: 39.311857430553474 - type: nauc_ndcg_at_1_max value: 33.97087354489543 - type: nauc_ndcg_at_1_std value: 1.6330160686638415 - type: nauc_ndcg_at_20_diff1 value: 26.394868881104934 - type: nauc_ndcg_at_20_max value: 31.85402555846015 - type: nauc_ndcg_at_20_std value: 7.795555059602305 - type: nauc_ndcg_at_3_diff1 value: 30.454593372388928 - type: nauc_ndcg_at_3_max value: 30.611639348279894 - type: nauc_ndcg_at_3_std value: 3.964523006177655 - type: nauc_ndcg_at_5_diff1 value: 29.540536953174794 - type: nauc_ndcg_at_5_max value: 31.934893980894923 - type: nauc_ndcg_at_5_std value: 5.829188982388009 - type: nauc_precision_at_1000_diff1 value: -17.3936895310082 - type: nauc_precision_at_1000_max value: 5.628308780206078 - type: nauc_precision_at_1000_std value: 2.955268381292835 - type: nauc_precision_at_100_diff1 value: 2.918251291838532 - type: nauc_precision_at_100_max value: 21.776965300942503 - type: nauc_precision_at_100_std value: 21.660217753595028 - type: nauc_precision_at_10_diff1 value: 15.801364706043461 - type: nauc_precision_at_10_max value: 33.624629609247265 - type: nauc_precision_at_10_std value: 15.047003127768848 - type: nauc_precision_at_1_diff1 value: 39.311857430553474 - type: nauc_precision_at_1_max value: 33.97087354489543 - type: nauc_precision_at_1_std value: 1.6330160686638415 - type: nauc_precision_at_20_diff1 value: 9.513628374160124 - type: nauc_precision_at_20_max value: 30.042069637616407 - type: nauc_precision_at_20_std value: 16.240137732903406 - type: nauc_precision_at_3_diff1 value: 25.162961789515652 - type: nauc_precision_at_3_max value: 31.508280123911387 - type: nauc_precision_at_3_std value: 8.185817168342629 - type: nauc_precision_at_5_diff1 value: 22.259694033823866 - type: nauc_precision_at_5_max value: 34.20401982691483 - type: nauc_precision_at_5_std value: 12.132427527020509 - type: nauc_recall_at_1000_diff1 value: 35.64252862292649 - type: nauc_recall_at_1000_max value: 45.125446579637156 - type: nauc_recall_at_1000_std value: 44.01656756138821 - type: nauc_recall_at_100_diff1 value: 12.738708654455664 - type: nauc_recall_at_100_max value: 27.455639671124654 - type: nauc_recall_at_100_std value: 26.965778999592672 - type: nauc_recall_at_10_diff1 value: 17.106987038385054 - type: nauc_recall_at_10_max value: 31.39153322965062 - type: nauc_recall_at_10_std value: 13.55126572530736 - type: nauc_recall_at_1_diff1 value: 38.237149754210726 - type: nauc_recall_at_1_max value: 31.191178791236453 - type: nauc_recall_at_1_std value: 0.12298428255657214 - type: nauc_recall_at_20_diff1 value: 12.47022101356733 - type: nauc_recall_at_20_max value: 29.940503509143472 - type: nauc_recall_at_20_std value: 16.01775569765924 - type: nauc_recall_at_3_diff1 value: 24.941265281565013 - type: nauc_recall_at_3_max value: 27.770084920697585 - type: nauc_recall_at_3_std value: 5.298432806419127 - type: nauc_recall_at_5_diff1 value: 23.196111299796783 - type: nauc_recall_at_5_max value: 31.165645983532862 - type: nauc_recall_at_5_std value: 9.630557988742119 - type: ndcg_at_1 value: 22.736 - type: ndcg_at_10 value: 33.147999999999996 - type: ndcg_at_100 value: 38.711 - type: ndcg_at_1000 value: 41.498000000000005 - type: ndcg_at_20 value: 35.286 - type: ndcg_at_3 value: 28.016000000000002 - type: ndcg_at_5 value: 30.011 - type: precision_at_1 value: 22.736 - type: precision_at_10 value: 5.379 - type: precision_at_100 value: 0.876 - type: precision_at_1000 value: 0.125 - type: precision_at_20 value: 3.2070000000000003 - type: precision_at_3 value: 11.953 - type: precision_at_5 value: 8.466 - type: recall_at_1 value: 20.801 - type: recall_at_10 value: 46.134 - type: recall_at_100 value: 72.151 - type: recall_at_1000 value: 92.648 - type: recall_at_20 value: 54.208 - type: recall_at_3 value: 32.061 - type: recall_at_5 value: 36.781000000000006 task: type: Retrieval - dataset: config: default name: MTEB ClimateFEVER (default) revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380 split: test type: mteb/climate-fever metrics: - type: main_score value: 20.272000000000002 - type: map_at_1 value: 7.9159999999999995 - type: map_at_10 value: 13.769 - type: map_at_100 value: 15.447 - type: map_at_1000 value: 15.634 - type: map_at_20 value: 14.584 - type: map_at_3 value: 11.234 - type: map_at_5 value: 12.581999999999999 - type: mrr_at_1 value: 17.71986970684039 - type: mrr_at_10 value: 27.315262912982785 - type: mrr_at_100 value: 28.561447384716445 - type: mrr_at_1000 value: 28.615464057754238 - type: mrr_at_20 value: 28.05807905035424 - type: mrr_at_3 value: 23.995656894679694 - type: mrr_at_5 value: 25.910966340933765 - type: nauc_map_at_1000_diff1 value: 19.27164407066605 - type: nauc_map_at_1000_max value: 39.41684350220827 - type: nauc_map_at_1000_std value: 14.316937322743758 - type: nauc_map_at_100_diff1 value: 19.248468726547568 - type: nauc_map_at_100_max value: 39.34794716638515 - type: nauc_map_at_100_std value: 14.201768964502925 - type: nauc_map_at_10_diff1 value: 19.683986719050274 - type: nauc_map_at_10_max value: 38.399204402852966 - type: nauc_map_at_10_std value: 11.085951407127387 - type: nauc_map_at_1_diff1 value: 27.04827437676845 - type: nauc_map_at_1_max value: 38.633105955884005 - type: nauc_map_at_1_std value: 5.938865181063489 - type: nauc_map_at_20_diff1 value: 19.360005560838747 - type: nauc_map_at_20_max value: 38.82413953246029 - type: nauc_map_at_20_std value: 12.754242668354054 - type: nauc_map_at_3_diff1 value: 20.894319202919544 - type: nauc_map_at_3_max value: 37.249524247063235 - type: nauc_map_at_3_std value: 6.843478307744917 - type: nauc_map_at_5_diff1 value: 20.27906594864198 - type: nauc_map_at_5_max value: 37.54544135335632 - type: nauc_map_at_5_std value: 8.531082034593195 - type: nauc_mrr_at_1000_diff1 value: 16.711410139123505 - type: nauc_mrr_at_1000_max value: 36.86483960867672 - type: nauc_mrr_at_1000_std value: 15.882643863172216 - type: nauc_mrr_at_100_diff1 value: 16.68281852636272 - type: nauc_mrr_at_100_max value: 36.87086265744777 - type: nauc_mrr_at_100_std value: 15.928290728923649 - type: nauc_mrr_at_10_diff1 value: 16.71042298660718 - type: nauc_mrr_at_10_max value: 36.61850549472591 - type: nauc_mrr_at_10_std value: 15.215625786453261 - type: nauc_mrr_at_1_diff1 value: 21.34560742722612 - type: nauc_mrr_at_1_max value: 35.23794351748137 - type: nauc_mrr_at_1_std value: 10.275184170527861 - type: nauc_mrr_at_20_diff1 value: 16.697159507381322 - type: nauc_mrr_at_20_max value: 36.82721248687378 - type: nauc_mrr_at_20_std value: 15.924913729811388 - type: nauc_mrr_at_3_diff1 value: 17.2413614094298 - type: nauc_mrr_at_3_max value: 36.02624449447232 - type: nauc_mrr_at_3_std value: 12.642327717244067 - type: nauc_mrr_at_5_diff1 value: 16.95145771063532 - type: nauc_mrr_at_5_max value: 36.34805513361277 - type: nauc_mrr_at_5_std value: 14.20542978979385 - type: nauc_ndcg_at_1000_diff1 value: 16.068715421681784 - type: nauc_ndcg_at_1000_max value: 41.32612347157758 - type: nauc_ndcg_at_1000_std value: 24.78081286872656 - type: nauc_ndcg_at_100_diff1 value: 15.233309050933297 - type: nauc_ndcg_at_100_max value: 40.42753224168349 - type: nauc_ndcg_at_100_std value: 24.305443776274192 - type: nauc_ndcg_at_10_diff1 value: 16.670460078695964 - type: nauc_ndcg_at_10_max value: 38.622868150049406 - type: nauc_ndcg_at_10_std value: 16.125904572906634 - type: nauc_ndcg_at_1_diff1 value: 21.34560742722612 - type: nauc_ndcg_at_1_max value: 35.23794351748137 - type: nauc_ndcg_at_1_std value: 10.275184170527861 - type: nauc_ndcg_at_20_diff1 value: 16.080658587383432 - type: nauc_ndcg_at_20_max value: 39.25652773329176 - type: nauc_ndcg_at_20_std value: 19.643708653239592 - type: nauc_ndcg_at_3_diff1 value: 17.672008602530248 - type: nauc_ndcg_at_3_max value: 36.1308953627282 - type: nauc_ndcg_at_3_std value: 9.42112318215692 - type: nauc_ndcg_at_5_diff1 value: 17.762402594079937 - type: nauc_ndcg_at_5_max value: 37.33867622232897 - type: nauc_ndcg_at_5_std value: 11.899504255587376 - type: nauc_precision_at_1000_diff1 value: -0.5331282207902904 - type: nauc_precision_at_1000_max value: 19.59281809586525 - type: nauc_precision_at_1000_std value: 33.920228812394626 - type: nauc_precision_at_100_diff1 value: 0.7454530517085408 - type: nauc_precision_at_100_max value: 28.13749861343301 - type: nauc_precision_at_100_std value: 40.28621226626079 - type: nauc_precision_at_10_diff1 value: 8.011193964539697 - type: nauc_precision_at_10_max value: 34.42259186378951 - type: nauc_precision_at_10_std value: 26.713919326095976 - type: nauc_precision_at_1_diff1 value: 21.34560742722612 - type: nauc_precision_at_1_max value: 35.23794351748137 - type: nauc_precision_at_1_std value: 10.275184170527861 - type: nauc_precision_at_20_diff1 value: 5.723382814616091 - type: nauc_precision_at_20_max value: 32.583990730667686 - type: nauc_precision_at_20_std value: 33.18884905285113 - type: nauc_precision_at_3_diff1 value: 12.12489342689621 - type: nauc_precision_at_3_max value: 34.4378910447675 - type: nauc_precision_at_3_std value: 13.830198340805433 - type: nauc_precision_at_5_diff1 value: 11.262919485267677 - type: nauc_precision_at_5_max value: 34.57032588255472 - type: nauc_precision_at_5_std value: 19.029127585142927 - type: nauc_recall_at_1000_diff1 value: 8.055138775756063 - type: nauc_recall_at_1000_max value: 36.705342422891505 - type: nauc_recall_at_1000_std value: 41.71634343710187 - type: nauc_recall_at_100_diff1 value: 5.3789172239715395 - type: nauc_recall_at_100_max value: 33.29134082588753 - type: nauc_recall_at_100_std value: 36.08081458503018 - type: nauc_recall_at_10_diff1 value: 11.06279620172739 - type: nauc_recall_at_10_max value: 35.01017831291135 - type: nauc_recall_at_10_std value: 19.431017047690844 - type: nauc_recall_at_1_diff1 value: 27.04827437676845 - type: nauc_recall_at_1_max value: 38.633105955884005 - type: nauc_recall_at_1_std value: 5.938865181063489 - type: nauc_recall_at_20_diff1 value: 9.231470165981193 - type: nauc_recall_at_20_max value: 33.734669757329925 - type: nauc_recall_at_20_std value: 25.21327790996102 - type: nauc_recall_at_3_diff1 value: 15.825652907571802 - type: nauc_recall_at_3_max value: 35.40629897302576 - type: nauc_recall_at_3_std value: 8.185163099422358 - type: nauc_recall_at_5_diff1 value: 13.925296061018297 - type: nauc_recall_at_5_max value: 34.35920557431228 - type: nauc_recall_at_5_std value: 11.983898742305827 - type: ndcg_at_1 value: 17.72 - type: ndcg_at_10 value: 20.272000000000002 - type: ndcg_at_100 value: 27.748 - type: ndcg_at_1000 value: 31.457 - type: ndcg_at_20 value: 22.861 - type: ndcg_at_3 value: 15.742 - type: ndcg_at_5 value: 17.494 - type: precision_at_1 value: 17.72 - type: precision_at_10 value: 6.554 - type: precision_at_100 value: 1.438 - type: precision_at_1000 value: 0.212 - type: precision_at_20 value: 4.332 - type: precision_at_3 value: 11.705 - type: precision_at_5 value: 9.511 - type: recall_at_1 value: 7.9159999999999995 - type: recall_at_10 value: 25.389 - type: recall_at_100 value: 52.042 - type: recall_at_1000 value: 73.166 - type: recall_at_20 value: 32.914 - type: recall_at_3 value: 14.585999999999999 - type: recall_at_5 value: 19.145 task: type: Retrieval - dataset: config: default name: MTEB DBPedia (default) revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659 split: dev type: mteb/dbpedia metrics: - type: main_score value: 36.741 - type: map_at_1 value: 8.518 - type: map_at_10 value: 16.805 - type: map_at_100 value: 22.688 - type: map_at_1000 value: 24.034 - type: map_at_20 value: 18.989 - type: map_at_3 value: 11.62 - type: map_at_5 value: 13.632 - type: mrr_at_1 value: 56.71641791044776 - type: mrr_at_10 value: 67.47927031509121 - type: mrr_at_100 value: 67.68036491984809 - type: mrr_at_1000 value: 67.71866110452235 - type: mrr_at_20 value: 67.55782491053505 - type: mrr_at_3 value: 65.92039800995025 - type: mrr_at_5 value: 66.66666666666666 - type: nauc_map_at_1000_diff1 value: 38.91309544886292 - type: nauc_map_at_1000_max value: 33.25327638790899 - type: nauc_map_at_1000_std value: -4.839456770653688 - type: nauc_map_at_100_diff1 value: 39.470095651936624 - type: nauc_map_at_100_max value: 31.6392453321697 - type: nauc_map_at_100_std value: -8.96416600201913 - type: nauc_map_at_10_diff1 value: 48.72413101964486 - type: nauc_map_at_10_max value: 24.221984845780593 - type: nauc_map_at_10_std value: -18.439525457479917 - type: nauc_map_at_1_diff1 value: 56.29522627864423 - type: nauc_map_at_1_max value: 2.032697167465618 - type: nauc_map_at_1_std value: -21.150111280187122 - type: nauc_map_at_20_diff1 value: 46.19388124154824 - type: nauc_map_at_20_max value: 28.220863411684395 - type: nauc_map_at_20_std value: -16.492554560268378 - type: nauc_map_at_3_diff1 value: 51.9225025148335 - type: nauc_map_at_3_max value: 13.639624579280879 - type: nauc_map_at_3_std value: -19.528434007929427 - type: nauc_map_at_5_diff1 value: 51.534134358833214 - type: nauc_map_at_5_max value: 19.822522893387344 - type: nauc_map_at_5_std value: -20.434559861361485 - type: nauc_mrr_at_1000_diff1 value: 51.93113835872477 - type: nauc_mrr_at_1000_max value: 54.32437428254132 - type: nauc_mrr_at_1000_std value: 5.2577802702668315 - type: nauc_mrr_at_100_diff1 value: 51.9059098333946 - type: nauc_mrr_at_100_max value: 54.3692994529951 - type: nauc_mrr_at_100_std value: 5.331395943562554 - type: nauc_mrr_at_10_diff1 value: 51.87485930238953 - type: nauc_mrr_at_10_max value: 54.54221250482985 - type: nauc_mrr_at_10_std value: 5.480126002292635 - type: nauc_mrr_at_1_diff1 value: 54.76665426289065 - type: nauc_mrr_at_1_max value: 43.297228503319396 - type: nauc_mrr_at_1_std value: 1.2859466120943177 - type: nauc_mrr_at_20_diff1 value: 52.062528659927786 - type: nauc_mrr_at_20_max value: 54.5770909107215 - type: nauc_mrr_at_20_std value: 5.546497370615925 - type: nauc_mrr_at_3_diff1 value: 54.1005935805453 - type: nauc_mrr_at_3_max value: 54.34057761998554 - type: nauc_mrr_at_3_std value: 0.21706581272032274 - type: nauc_mrr_at_5_diff1 value: 53.179463268815496 - type: nauc_mrr_at_5_max value: 55.721499296650926 - type: nauc_mrr_at_5_std value: 4.261656144434617 - type: nauc_ndcg_at_1000_diff1 value: 36.590546507362184 - type: nauc_ndcg_at_1000_max value: 47.736470639972715 - type: nauc_ndcg_at_1000_std value: 11.859823360897009 - type: nauc_ndcg_at_100_diff1 value: 38.0278717369385 - type: nauc_ndcg_at_100_max value: 42.987806870329834 - type: nauc_ndcg_at_100_std value: -0.8175387159806664 - type: nauc_ndcg_at_10_diff1 value: 41.69729579253183 - type: nauc_ndcg_at_10_max value: 39.74061193498346 - type: nauc_ndcg_at_10_std value: -4.062017740706537 - type: nauc_ndcg_at_1_diff1 value: 48.64546696798436 - type: nauc_ndcg_at_1_max value: 31.342716759901656 - type: nauc_ndcg_at_1_std value: -2.5410201667884045 - type: nauc_ndcg_at_20_diff1 value: 42.64476678174895 - type: nauc_ndcg_at_20_max value: 38.59473045117711 - type: nauc_ndcg_at_20_std value: -8.435063130217593 - type: nauc_ndcg_at_3_diff1 value: 37.135301571661294 - type: nauc_ndcg_at_3_max value: 36.87208354019284 - type: nauc_ndcg_at_3_std value: -2.7431351676630613 - type: nauc_ndcg_at_5_diff1 value: 39.72580848038363 - type: nauc_ndcg_at_5_max value: 39.6775641287642 - type: nauc_ndcg_at_5_std value: -4.118332260805727 - type: nauc_precision_at_1000_diff1 value: -26.092565956641273 - type: nauc_precision_at_1000_max value: 19.581157629965798 - type: nauc_precision_at_1000_std value: 54.13868562253408 - type: nauc_precision_at_100_diff1 value: -23.885066773251197 - type: nauc_precision_at_100_max value: 25.40568210909806 - type: nauc_precision_at_100_std value: 45.70352750469782 - type: nauc_precision_at_10_diff1 value: -7.812878775603055 - type: nauc_precision_at_10_max value: 39.333622055903525 - type: nauc_precision_at_10_std value: 22.237660677257402 - type: nauc_precision_at_1_diff1 value: 54.76665426289065 - type: nauc_precision_at_1_max value: 43.297228503319396 - type: nauc_precision_at_1_std value: 1.2859466120943177 - type: nauc_precision_at_20_diff1 value: -10.970224909606511 - type: nauc_precision_at_20_max value: 37.3041054296315 - type: nauc_precision_at_20_std value: 27.22315967301695 - type: nauc_precision_at_3_diff1 value: 17.76029053099922 - type: nauc_precision_at_3_max value: 50.65384554488439 - type: nauc_precision_at_3_std value: 15.558585289301638 - type: nauc_precision_at_5_diff1 value: 8.247336865521545 - type: nauc_precision_at_5_max value: 51.37335018224255 - type: nauc_precision_at_5_std value: 14.58519131797533 - type: nauc_recall_at_1000_diff1 value: 14.738115664684562 - type: nauc_recall_at_1000_max value: 33.67225069826756 - type: nauc_recall_at_1000_std value: 23.43446988901876 - type: nauc_recall_at_100_diff1 value: 19.890629659776625 - type: nauc_recall_at_100_max value: 29.224410455958814 - type: nauc_recall_at_100_std value: -2.0806276568086144 - type: nauc_recall_at_10_diff1 value: 38.427006027875144 - type: nauc_recall_at_10_max value: 19.924708152983968 - type: nauc_recall_at_10_std value: -16.630217466078644 - type: nauc_recall_at_1_diff1 value: 56.29522627864423 - type: nauc_recall_at_1_max value: 2.032697167465618 - type: nauc_recall_at_1_std value: -21.150111280187122 - type: nauc_recall_at_20_diff1 value: 37.14428498437845 - type: nauc_recall_at_20_max value: 25.145823335978513 - type: nauc_recall_at_20_std value: -14.42493782371082 - type: nauc_recall_at_3_diff1 value: 47.52928646265923 - type: nauc_recall_at_3_max value: 12.680033579427134 - type: nauc_recall_at_3_std value: -21.288969511674864 - type: nauc_recall_at_5_diff1 value: 49.48113347640814 - type: nauc_recall_at_5_max value: 21.348244496594557 - type: nauc_recall_at_5_std value: -23.32820771790651 - type: ndcg_at_1 value: 47.015 - type: ndcg_at_10 value: 36.741 - type: ndcg_at_100 value: 41.513 - type: ndcg_at_1000 value: 49.504 - type: ndcg_at_20 value: 35.787 - type: ndcg_at_3 value: 39.969 - type: ndcg_at_5 value: 38.07 - type: precision_at_1 value: 56.716 - type: precision_at_10 value: 27.761000000000003 - type: precision_at_100 value: 8.343 - type: precision_at_1000 value: 1.4909999999999999 - type: precision_at_20 value: 19.925 - type: precision_at_3 value: 38.806000000000004 - type: precision_at_5 value: 34.327999999999996 - type: recall_at_1 value: 8.518 - type: recall_at_10 value: 24.04 - type: recall_at_100 value: 47.696 - type: recall_at_1000 value: 73.98 - type: recall_at_20 value: 30.036 - type: recall_at_3 value: 12.928 - type: recall_at_5 value: 16.247 task: type: Retrieval - dataset: config: default name: MTEB DBPedia (default) revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659 split: test type: mteb/dbpedia metrics: - type: main_score value: 32.33 - type: map_at_1 value: 7.172000000000001 - type: map_at_10 value: 14.507 - type: map_at_100 value: 20.094 - type: map_at_1000 value: 21.357 - type: map_at_20 value: 16.544 - type: map_at_3 value: 10.45 - type: map_at_5 value: 12.135 - type: mrr_at_1 value: 54.50000000000001 - type: mrr_at_10 value: 63.956646825396824 - type: mrr_at_100 value: 64.46710719535605 - type: mrr_at_1000 value: 64.49017448257285 - type: mrr_at_20 value: 64.29454397189498 - type: mrr_at_3 value: 61.541666666666664 - type: mrr_at_5 value: 63.12916666666667 - type: nauc_map_at_1000_diff1 value: 17.98869657338149 - type: nauc_map_at_1000_max value: 30.333054783890713 - type: nauc_map_at_1000_std value: 28.27600264564019 - type: nauc_map_at_100_diff1 value: 18.30992505565338 - type: nauc_map_at_100_max value: 28.247081175265638 - type: nauc_map_at_100_std value: 25.219665647719737 - type: nauc_map_at_10_diff1 value: 24.602841428268228 - type: nauc_map_at_10_max value: 19.198550685025463 - type: nauc_map_at_10_std value: 4.94101336989231 - type: nauc_map_at_1_diff1 value: 42.48207044156705 - type: nauc_map_at_1_max value: 11.178685700020784 - type: nauc_map_at_1_std value: -11.7133145725654 - type: nauc_map_at_20_diff1 value: 22.485265149042345 - type: nauc_map_at_20_max value: 22.30085094149233 - type: nauc_map_at_20_std value: 12.002784564238516 - type: nauc_map_at_3_diff1 value: 32.59887029757679 - type: nauc_map_at_3_max value: 14.376368303570612 - type: nauc_map_at_3_std value: -7.812707129845258 - type: nauc_map_at_5_diff1 value: 28.420076313160227 - type: nauc_map_at_5_max value: 16.35175739634549 - type: nauc_map_at_5_std value: -2.2859112790508633 - type: nauc_mrr_at_1000_diff1 value: 33.16285404785331 - type: nauc_mrr_at_1000_max value: 52.95307100316335 - type: nauc_mrr_at_1000_std value: 35.6708854579188 - type: nauc_mrr_at_100_diff1 value: 33.163975798622324 - type: nauc_mrr_at_100_max value: 52.94924599844868 - type: nauc_mrr_at_100_std value: 35.681725458149536 - type: nauc_mrr_at_10_diff1 value: 33.052961411322926 - type: nauc_mrr_at_10_max value: 52.87719597200036 - type: nauc_mrr_at_10_std value: 35.4649058995902 - type: nauc_mrr_at_1_diff1 value: 38.1491930696872 - type: nauc_mrr_at_1_max value: 51.415881617631634 - type: nauc_mrr_at_1_std value: 31.665817021306292 - type: nauc_mrr_at_20_diff1 value: 33.1786209686742 - type: nauc_mrr_at_20_max value: 52.97979514533936 - type: nauc_mrr_at_20_std value: 35.720708427743745 - type: nauc_mrr_at_3_diff1 value: 32.0527395678109 - type: nauc_mrr_at_3_max value: 52.30552818184112 - type: nauc_mrr_at_3_std value: 34.717929240152436 - type: nauc_mrr_at_5_diff1 value: 32.60246941607667 - type: nauc_mrr_at_5_max value: 52.90891823393802 - type: nauc_mrr_at_5_std value: 35.73817768953035 - type: nauc_ndcg_at_1000_diff1 value: 17.777717528764786 - type: nauc_ndcg_at_1000_max value: 41.17736287839547 - type: nauc_ndcg_at_1000_std value: 39.42093133426984 - type: nauc_ndcg_at_100_diff1 value: 18.7009852260721 - type: nauc_ndcg_at_100_max value: 35.070493992114486 - type: nauc_ndcg_at_100_std value: 33.29594459213619 - type: nauc_ndcg_at_10_diff1 value: 20.24863550289221 - type: nauc_ndcg_at_10_max value: 37.61721766753526 - type: nauc_ndcg_at_10_std value: 29.002032871576546 - type: nauc_ndcg_at_1_diff1 value: 35.99290323798346 - type: nauc_ndcg_at_1_max value: 42.10390853824893 - type: nauc_ndcg_at_1_std value: 23.49738590016146 - type: nauc_ndcg_at_20_diff1 value: 21.98574066507013 - type: nauc_ndcg_at_20_max value: 34.74490524420729 - type: nauc_ndcg_at_20_std value: 26.979787645217034 - type: nauc_ndcg_at_3_diff1 value: 24.109022760008195 - type: nauc_ndcg_at_3_max value: 41.57735669263441 - type: nauc_ndcg_at_3_std value: 26.48137639996327 - type: nauc_ndcg_at_5_diff1 value: 21.1115968406177 - type: nauc_ndcg_at_5_max value: 39.79689482635169 - type: nauc_ndcg_at_5_std value: 28.71417077380657 - type: nauc_precision_at_1000_diff1 value: -6.308626101041377 - type: nauc_precision_at_1000_max value: 23.736634275582723 - type: nauc_precision_at_1000_std value: 35.17703177177103 - type: nauc_precision_at_100_diff1 value: -5.991432241727243 - type: nauc_precision_at_100_max value: 36.632326384787945 - type: nauc_precision_at_100_std value: 55.36164388714797 - type: nauc_precision_at_10_diff1 value: -2.003639167604807 - type: nauc_precision_at_10_max value: 39.9605047862184 - type: nauc_precision_at_10_std value: 48.203273333600976 - type: nauc_precision_at_1_diff1 value: 38.1491930696872 - type: nauc_precision_at_1_max value: 51.415881617631634 - type: nauc_precision_at_1_std value: 31.665817021306292 - type: nauc_precision_at_20_diff1 value: -2.0019880382597592 - type: nauc_precision_at_20_max value: 39.77280564724441 - type: nauc_precision_at_20_std value: 52.93777268797039 - type: nauc_precision_at_3_diff1 value: 7.926089484917107 - type: nauc_precision_at_3_max value: 43.17631014195846 - type: nauc_precision_at_3_std value: 36.50361169083805 - type: nauc_precision_at_5_diff1 value: 0.935141231734079 - type: nauc_precision_at_5_max value: 43.43686608909768 - type: nauc_precision_at_5_std value: 45.153047523435355 - type: nauc_recall_at_1000_diff1 value: 4.341876247276381 - type: nauc_recall_at_1000_max value: 27.248539409823575 - type: nauc_recall_at_1000_std value: 38.96443511840041 - type: nauc_recall_at_100_diff1 value: 9.09368046295337 - type: nauc_recall_at_100_max value: 20.212709370157558 - type: nauc_recall_at_100_std value: 29.27989964554077 - type: nauc_recall_at_10_diff1 value: 15.702200814005119 - type: nauc_recall_at_10_max value: 11.354521056668917 - type: nauc_recall_at_10_std value: 1.9460926024560252 - type: nauc_recall_at_1_diff1 value: 42.48207044156705 - type: nauc_recall_at_1_max value: 11.178685700020784 - type: nauc_recall_at_1_std value: -11.7133145725654 - type: nauc_recall_at_20_diff1 value: 15.751803189017796 - type: nauc_recall_at_20_max value: 14.082256061576908 - type: nauc_recall_at_20_std value: 8.698741608220557 - type: nauc_recall_at_3_diff1 value: 27.177853206728642 - type: nauc_recall_at_3_max value: 11.883811151473985 - type: nauc_recall_at_3_std value: -8.753487515467821 - type: nauc_recall_at_5_diff1 value: 20.88189722918793 - type: nauc_recall_at_5_max value: 12.165134797296266 - type: nauc_recall_at_5_std value: -2.9050270843186716 - type: ndcg_at_1 value: 42.375 - type: ndcg_at_10 value: 32.33 - type: ndcg_at_100 value: 36.370000000000005 - type: ndcg_at_1000 value: 43.596000000000004 - type: ndcg_at_20 value: 31.557000000000002 - type: ndcg_at_3 value: 35.006 - type: ndcg_at_5 value: 33.35 - type: precision_at_1 value: 54.50000000000001 - type: precision_at_10 value: 26.424999999999997 - type: precision_at_100 value: 8.24 - type: precision_at_1000 value: 1.7659999999999998 - type: precision_at_20 value: 19.262 - type: precision_at_3 value: 38.667 - type: precision_at_5 value: 33.0 - type: recall_at_1 value: 7.172000000000001 - type: recall_at_10 value: 19.922 - type: recall_at_100 value: 43.273 - type: recall_at_1000 value: 67.157 - type: recall_at_20 value: 25.602999999999998 - type: recall_at_3 value: 11.521 - type: recall_at_5 value: 14.667 task: type: Retrieval - dataset: config: default name: MTEB EmotionClassification (default) revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 split: test type: mteb/emotion metrics: - type: accuracy value: 40.83 - type: f1 value: 37.18194175166477 - type: f1_weighted value: 42.95575332555824 - type: main_score value: 40.83 task: type: Classification - dataset: config: default name: MTEB EmotionClassification (default) revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 split: validation type: mteb/emotion metrics: - type: accuracy value: 40.55 - type: f1 value: 37.433406191273036 - type: f1_weighted value: 42.28073807897865 - type: main_score value: 40.55 task: type: Classification - dataset: config: default name: MTEB FEVER (default) revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 split: dev type: mteb/fever metrics: - type: main_score value: 52.03 - type: map_at_1 value: 34.943999999999996 - type: map_at_10 value: 45.769999999999996 - type: map_at_100 value: 46.466 - type: map_at_1000 value: 46.501999999999995 - type: map_at_20 value: 46.211999999999996 - type: map_at_3 value: 42.751 - type: map_at_5 value: 44.509 - type: mrr_at_1 value: 37.80378037803781 - type: mrr_at_10 value: 48.93870339414894 - type: mrr_at_100 value: 49.601175839993346 - type: mrr_at_1000 value: 49.63131452135376 - type: mrr_at_20 value: 49.36802873435527 - type: mrr_at_3 value: 45.844584458445844 - type: mrr_at_5 value: 47.672517251725175 - type: nauc_map_at_1000_diff1 value: 40.54062035165064 - type: nauc_map_at_1000_max value: 27.289178560605333 - type: nauc_map_at_1000_std value: 5.568130411014613 - type: nauc_map_at_100_diff1 value: 40.53294239730666 - type: nauc_map_at_100_max value: 27.303094343476264 - type: nauc_map_at_100_std value: 5.580846319474926 - type: nauc_map_at_10_diff1 value: 40.36832372050692 - type: nauc_map_at_10_max value: 27.181736026891624 - type: nauc_map_at_10_std value: 5.278484971252747 - type: nauc_map_at_1_diff1 value: 43.63870052897785 - type: nauc_map_at_1_max value: 23.91172545582507 - type: nauc_map_at_1_std value: 1.0551066222390522 - type: nauc_map_at_20_diff1 value: 40.477298918415485 - type: nauc_map_at_20_max value: 27.320301914916662 - type: nauc_map_at_20_std value: 5.523811856269391 - type: nauc_map_at_3_diff1 value: 40.51977087679908 - type: nauc_map_at_3_max value: 25.754504604995727 - type: nauc_map_at_3_std value: 3.2633589338880533 - type: nauc_map_at_5_diff1 value: 40.32976587618264 - type: nauc_map_at_5_max value: 26.448666343780392 - type: nauc_map_at_5_std value: 4.402864599093908 - type: nauc_mrr_at_1000_diff1 value: 41.918408557950485 - type: nauc_mrr_at_1000_max value: 28.906736122718556 - type: nauc_mrr_at_1000_std value: 5.553304289160072 - type: nauc_mrr_at_100_diff1 value: 41.909263695140915 - type: nauc_mrr_at_100_max value: 28.927291145915586 - type: nauc_mrr_at_100_std value: 5.5758003987110785 - type: nauc_mrr_at_10_diff1 value: 41.78054335093107 - type: nauc_mrr_at_10_max value: 28.875286254271337 - type: nauc_mrr_at_10_std value: 5.400103314799414 - type: nauc_mrr_at_1_diff1 value: 45.41307124018666 - type: nauc_mrr_at_1_max value: 25.42705936857816 - type: nauc_mrr_at_1_std value: 1.1261509938529668 - type: nauc_mrr_at_20_diff1 value: 41.85355521719339 - type: nauc_mrr_at_20_max value: 28.954337133137397 - type: nauc_mrr_at_20_std value: 5.5680811472526255 - type: nauc_mrr_at_3_diff1 value: 41.87597266839382 - type: nauc_mrr_at_3_max value: 27.364838471220192 - type: nauc_mrr_at_3_std value: 3.421015527947961 - type: nauc_mrr_at_5_diff1 value: 41.720199314616735 - type: nauc_mrr_at_5_max value: 28.16125782835219 - type: nauc_mrr_at_5_std value: 4.535741115937104 - type: nauc_ndcg_at_1000_diff1 value: 40.0862551794478 - type: nauc_ndcg_at_1000_max value: 29.620465023882847 - type: nauc_ndcg_at_1000_std value: 9.116463212141856 - type: nauc_ndcg_at_100_diff1 value: 39.91403782740219 - type: nauc_ndcg_at_100_max value: 30.126314736130627 - type: nauc_ndcg_at_100_std value: 9.629679229500052 - type: nauc_ndcg_at_10_diff1 value: 39.31693698451662 - type: nauc_ndcg_at_10_max value: 29.88649188330779 - type: nauc_ndcg_at_10_std value: 8.379670725952097 - type: nauc_ndcg_at_1_diff1 value: 45.41307124018666 - type: nauc_ndcg_at_1_max value: 25.42705936857816 - type: nauc_ndcg_at_1_std value: 1.1261509938529668 - type: nauc_ndcg_at_20_diff1 value: 39.63543393169759 - type: nauc_ndcg_at_20_max value: 30.395928596281337 - type: nauc_ndcg_at_20_std value: 9.331827354152152 - type: nauc_ndcg_at_3_diff1 value: 39.66004638243799 - type: nauc_ndcg_at_3_max value: 26.693162446383678 - type: nauc_ndcg_at_3_std value: 4.019479662872945 - type: nauc_ndcg_at_5_diff1 value: 39.29942630190977 - type: nauc_ndcg_at_5_max value: 28.03021086473212 - type: nauc_ndcg_at_5_std value: 6.133973202905711 - type: nauc_precision_at_1000_diff1 value: 6.790391477963487 - type: nauc_precision_at_1000_max value: 23.785347381042847 - type: nauc_precision_at_1000_std value: 32.029397732885975 - type: nauc_precision_at_100_diff1 value: 20.119850476874824 - type: nauc_precision_at_100_max value: 37.503871561687006 - type: nauc_precision_at_100_std value: 36.65793346223648 - type: nauc_precision_at_10_diff1 value: 30.875126515581474 - type: nauc_precision_at_10_max value: 40.41257688668288 - type: nauc_precision_at_10_std value: 23.73873082405173 - type: nauc_precision_at_1_diff1 value: 45.41307124018666 - type: nauc_precision_at_1_max value: 25.42705936857816 - type: nauc_precision_at_1_std value: 1.1261509938529668 - type: nauc_precision_at_20_diff1 value: 28.669583804112513 - type: nauc_precision_at_20_max value: 42.81841781000904 - type: nauc_precision_at_20_std value: 30.511249469538328 - type: nauc_precision_at_3_diff1 value: 35.695753378227444 - type: nauc_precision_at_3_max value: 29.356372089093437 - type: nauc_precision_at_3_std value: 7.264008666110257 - type: nauc_precision_at_5_diff1 value: 33.78933799034119 - type: nauc_precision_at_5_max value: 33.30247271400044 - type: nauc_precision_at_5_std value: 13.624181787527073 - type: nauc_recall_at_1000_diff1 value: 24.81993368943268 - type: nauc_recall_at_1000_max value: 32.64499600999935 - type: nauc_recall_at_1000_std value: 39.8723613927536 - type: nauc_recall_at_100_diff1 value: 28.838479610811767 - type: nauc_recall_at_100_max value: 39.31061939017268 - type: nauc_recall_at_100_std value: 35.334519993269694 - type: nauc_recall_at_10_diff1 value: 31.365769093693665 - type: nauc_recall_at_10_max value: 36.357564167301135 - type: nauc_recall_at_10_std value: 19.567161580054368 - type: nauc_recall_at_1_diff1 value: 43.63870052897785 - type: nauc_recall_at_1_max value: 23.91172545582507 - type: nauc_recall_at_1_std value: 1.0551066222390522 - type: nauc_recall_at_20_diff1 value: 31.07931048339613 - type: nauc_recall_at_20_max value: 39.3409835430064 - type: nauc_recall_at_20_std value: 26.017221663087625 - type: nauc_recall_at_3_diff1 value: 34.870082028464864 - type: nauc_recall_at_3_max value: 26.964701318509526 - type: nauc_recall_at_3_std value: 5.750625392443932 - type: nauc_recall_at_5_diff1 value: 33.09810558988621 - type: nauc_recall_at_5_max value: 30.114089305272714 - type: nauc_recall_at_5_std value: 11.136566513395767 - type: ndcg_at_1 value: 37.804 - type: ndcg_at_10 value: 52.03 - type: ndcg_at_100 value: 55.342 - type: ndcg_at_1000 value: 56.297 - type: ndcg_at_20 value: 53.574999999999996 - type: ndcg_at_3 value: 45.954 - type: ndcg_at_5 value: 49.051 - type: precision_at_1 value: 37.804 - type: precision_at_10 value: 7.559 - type: precision_at_100 value: 0.935 - type: precision_at_1000 value: 0.10300000000000001 - type: precision_at_20 value: 4.119 - type: precision_at_3 value: 19.022 - type: precision_at_5 value: 13.102 - type: recall_at_1 value: 34.943999999999996 - type: recall_at_10 value: 68.516 - type: recall_at_100 value: 83.55900000000001 - type: recall_at_1000 value: 90.824 - type: recall_at_20 value: 74.38900000000001 - type: recall_at_3 value: 52.032000000000004 - type: recall_at_5 value: 59.507 task: type: Retrieval - dataset: config: default name: MTEB FEVER (default) revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 split: test type: mteb/fever metrics: - type: main_score value: 51.934000000000005 - type: map_at_1 value: 34.076 - type: map_at_10 value: 45.482 - type: map_at_100 value: 46.269 - type: map_at_1000 value: 46.309 - type: map_at_20 value: 45.999 - type: map_at_3 value: 42.614000000000004 - type: map_at_5 value: 44.314 - type: mrr_at_1 value: 36.528652865286524 - type: mrr_at_10 value: 48.512785802389764 - type: mrr_at_100 value: 49.22606699053963 - type: mrr_at_1000 value: 49.257250294016124 - type: mrr_at_20 value: 48.98993069550024 - type: mrr_at_3 value: 45.57955795579558 - type: mrr_at_5 value: 47.34373437343734 - type: nauc_map_at_1000_diff1 value: 38.06411619881442 - type: nauc_map_at_1000_max value: 26.08940745236635 - type: nauc_map_at_1000_std value: 6.680760121868734 - type: nauc_map_at_100_diff1 value: 38.05029780772629 - type: nauc_map_at_100_max value: 26.09201602974643 - type: nauc_map_at_100_std value: 6.686772430837484 - type: nauc_map_at_10_diff1 value: 37.94073108438308 - type: nauc_map_at_10_max value: 26.010720595761715 - type: nauc_map_at_10_std value: 6.388307711704795 - type: nauc_map_at_1_diff1 value: 41.856370709349505 - type: nauc_map_at_1_max value: 22.120448152871685 - type: nauc_map_at_1_std value: 2.14480027163088 - type: nauc_map_at_20_diff1 value: 38.019326179975145 - type: nauc_map_at_20_max value: 26.101738560941307 - type: nauc_map_at_20_std value: 6.649402862599178 - type: nauc_map_at_3_diff1 value: 38.28916036819592 - type: nauc_map_at_3_max value: 25.282244701628976 - type: nauc_map_at_3_std value: 5.358107636585845 - type: nauc_map_at_5_diff1 value: 38.0319766931337 - type: nauc_map_at_5_max value: 25.760510571451654 - type: nauc_map_at_5_std value: 5.959226091718213 - type: nauc_mrr_at_1000_diff1 value: 40.14990337723866 - type: nauc_mrr_at_1000_max value: 27.52396080338908 - type: nauc_mrr_at_1000_std value: 5.905561916071595 - type: nauc_mrr_at_100_diff1 value: 40.138095201620885 - type: nauc_mrr_at_100_max value: 27.534919650845975 - type: nauc_mrr_at_100_std value: 5.921610771714778 - type: nauc_mrr_at_10_diff1 value: 40.042888225151884 - type: nauc_mrr_at_10_max value: 27.51884538283777 - type: nauc_mrr_at_10_std value: 5.7346191171159955 - type: nauc_mrr_at_1_diff1 value: 44.057159032541755 - type: nauc_mrr_at_1_max value: 23.378181020350205 - type: nauc_mrr_at_1_std value: 1.1184427932175494 - type: nauc_mrr_at_20_diff1 value: 40.10748214477916 - type: nauc_mrr_at_20_max value: 27.56420590984982 - type: nauc_mrr_at_20_std value: 5.92587573833964 - type: nauc_mrr_at_3_diff1 value: 40.35104109379557 - type: nauc_mrr_at_3_max value: 26.839657427564866 - type: nauc_mrr_at_3_std value: 4.842798077251789 - type: nauc_mrr_at_5_diff1 value: 40.120921909189065 - type: nauc_mrr_at_5_max value: 27.263579151858597 - type: nauc_mrr_at_5_std value: 5.33348700481928 - type: nauc_ndcg_at_1000_diff1 value: 37.49363482187989 - type: nauc_ndcg_at_1000_max value: 28.253157018498303 - type: nauc_ndcg_at_1000_std value: 9.548391600192229 - type: nauc_ndcg_at_100_diff1 value: 37.06429542530781 - type: nauc_ndcg_at_100_max value: 28.432696909982752 - type: nauc_ndcg_at_100_std value: 9.914672381291032 - type: nauc_ndcg_at_10_diff1 value: 36.62394158558396 - type: nauc_ndcg_at_10_max value: 28.091717808644788 - type: nauc_ndcg_at_10_std value: 8.593519390900884 - type: nauc_ndcg_at_1_diff1 value: 44.057159032541755 - type: nauc_ndcg_at_1_max value: 23.378181020350205 - type: nauc_ndcg_at_1_std value: 1.1184427932175494 - type: nauc_ndcg_at_20_diff1 value: 36.8375539418876 - type: nauc_ndcg_at_20_max value: 28.4653022439827 - type: nauc_ndcg_at_20_std value: 9.62779954290926 - type: nauc_ndcg_at_3_diff1 value: 37.507844796905424 - type: nauc_ndcg_at_3_max value: 26.621845005094237 - type: nauc_ndcg_at_3_std value: 6.311129269656924 - type: nauc_ndcg_at_5_diff1 value: 36.9499559138595 - type: nauc_ndcg_at_5_max value: 27.478742195672982 - type: nauc_ndcg_at_5_std value: 7.455941612523616 - type: nauc_precision_at_1000_diff1 value: 4.078793640824701 - type: nauc_precision_at_1000_max value: 20.00343523272489 - type: nauc_precision_at_1000_std value: 20.485736390122142 - type: nauc_precision_at_100_diff1 value: 14.17308181643292 - type: nauc_precision_at_100_max value: 31.74202484387519 - type: nauc_precision_at_100_std value: 27.502413567021478 - type: nauc_precision_at_10_diff1 value: 27.43865858943569 - type: nauc_precision_at_10_max value: 35.589513603311566 - type: nauc_precision_at_10_std value: 17.186128232419385 - type: nauc_precision_at_1_diff1 value: 44.057159032541755 - type: nauc_precision_at_1_max value: 23.378181020350205 - type: nauc_precision_at_1_std value: 1.1184427932175494 - type: nauc_precision_at_20_diff1 value: 23.512273491099464 - type: nauc_precision_at_20_max value: 36.4099385522305 - type: nauc_precision_at_20_std value: 23.38611190971049 - type: nauc_precision_at_3_diff1 value: 34.45481574669334 - type: nauc_precision_at_3_max value: 31.254230439494616 - type: nauc_precision_at_3_std value: 9.083954939005855 - type: nauc_precision_at_5_diff1 value: 31.5255657670166 - type: nauc_precision_at_5_max value: 33.513795593306064 - type: nauc_precision_at_5_std value: 12.268919737890467 - type: nauc_recall_at_1000_diff1 value: 21.034235565510095 - type: nauc_recall_at_1000_max value: 38.63663184408291 - type: nauc_recall_at_1000_std value: 45.51157667913507 - type: nauc_recall_at_100_diff1 value: 22.66108749212852 - type: nauc_recall_at_100_max value: 36.80398383289957 - type: nauc_recall_at_100_std value: 33.73455640625873 - type: nauc_recall_at_10_diff1 value: 27.131812527654887 - type: nauc_recall_at_10_max value: 32.196430920874285 - type: nauc_recall_at_10_std value: 17.075213496563453 - type: nauc_recall_at_1_diff1 value: 41.856370709349505 - type: nauc_recall_at_1_max value: 22.120448152871685 - type: nauc_recall_at_1_std value: 2.14480027163088 - type: nauc_recall_at_20_diff1 value: 25.929563215115486 - type: nauc_recall_at_20_max value: 34.652054105580135 - type: nauc_recall_at_20_std value: 24.184493516549775 - type: nauc_recall_at_3_diff1 value: 32.22327879284907 - type: nauc_recall_at_3_max value: 28.18077707672782 - type: nauc_recall_at_3_std value: 9.447537554097932 - type: nauc_recall_at_5_diff1 value: 29.95857292782884 - type: nauc_recall_at_5_max value: 30.180855826849314 - type: nauc_recall_at_5_std value: 12.493035764779222 - type: ndcg_at_1 value: 36.529 - type: ndcg_at_10 value: 51.934000000000005 - type: ndcg_at_100 value: 55.525000000000006 - type: ndcg_at_1000 value: 56.568 - type: ndcg_at_20 value: 53.746 - type: ndcg_at_3 value: 46.169 - type: ndcg_at_5 value: 49.163000000000004 - type: precision_at_1 value: 36.529 - type: precision_at_10 value: 7.5649999999999995 - type: precision_at_100 value: 0.947 - type: precision_at_1000 value: 0.105 - type: precision_at_20 value: 4.184 - type: precision_at_3 value: 19.326999999999998 - type: precision_at_5 value: 13.239999999999998 - type: recall_at_1 value: 34.076 - type: recall_at_10 value: 69.009 - type: recall_at_100 value: 85.047 - type: recall_at_1000 value: 92.962 - type: recall_at_20 value: 75.94800000000001 - type: recall_at_3 value: 53.446000000000005 - type: recall_at_5 value: 60.622 task: type: Retrieval - dataset: config: default name: MTEB FEVER (default) revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 split: train type: mteb/fever metrics: - type: main_score value: 45.507999999999996 - type: map_at_1 value: 26.979 - type: map_at_10 value: 38.607 - type: map_at_100 value: 39.531 - type: map_at_1000 value: 39.579 - type: map_at_20 value: 39.177 - type: map_at_3 value: 35.287 - type: map_at_5 value: 37.252 - type: mrr_at_1 value: 30.104726345505874 - type: mrr_at_10 value: 42.35555280910896 - type: mrr_at_100 value: 43.217533931012895 - type: mrr_at_1000 value: 43.252575699572596 - type: mrr_at_20 value: 42.902576525355975 - type: mrr_at_3 value: 38.98081534772182 - type: mrr_at_5 value: 40.99315484321404 - type: nauc_map_at_1000_diff1 value: 27.90071174527769 - type: nauc_map_at_1000_max value: 19.43450287678522 - type: nauc_map_at_1000_std value: 1.0478682545377263 - type: nauc_map_at_100_diff1 value: 27.889168491127947 - type: nauc_map_at_100_max value: 19.436828906145358 - type: nauc_map_at_100_std value: 1.05155715237615 - type: nauc_map_at_10_diff1 value: 27.840339888168252 - type: nauc_map_at_10_max value: 19.329180467515787 - type: nauc_map_at_10_std value: 0.7171877649239929 - type: nauc_map_at_1_diff1 value: 31.738179514738864 - type: nauc_map_at_1_max value: 15.968106583150215 - type: nauc_map_at_1_std value: -2.570993256036204 - type: nauc_map_at_20_diff1 value: 27.845284489690027 - type: nauc_map_at_20_max value: 19.40752573037618 - type: nauc_map_at_20_std value: 0.9717636792007718 - type: nauc_map_at_3_diff1 value: 28.044016784749825 - type: nauc_map_at_3_max value: 18.145169413574557 - type: nauc_map_at_3_std value: -0.7999143466737352 - type: nauc_map_at_5_diff1 value: 27.85994131302655 - type: nauc_map_at_5_max value: 18.910766958000796 - type: nauc_map_at_5_std value: 0.06864766875494689 - type: nauc_mrr_at_1000_diff1 value: 28.93587018349831 - type: nauc_mrr_at_1000_max value: 20.39155500244368 - type: nauc_mrr_at_1000_std value: 0.06072071663002824 - type: nauc_mrr_at_100_diff1 value: 28.92572607525496 - type: nauc_mrr_at_100_max value: 20.404607849703098 - type: nauc_mrr_at_100_std value: 0.07776005419330084 - type: nauc_mrr_at_10_diff1 value: 28.84843951949083 - type: nauc_mrr_at_10_max value: 20.399412540137465 - type: nauc_mrr_at_10_std value: -0.09691968843711217 - type: nauc_mrr_at_1_diff1 value: 32.92934503399022 - type: nauc_mrr_at_1_max value: 16.941727815666827 - type: nauc_mrr_at_1_std value: -3.3732010441778253 - type: nauc_mrr_at_20_diff1 value: 28.877436267558192 - type: nauc_mrr_at_20_max value: 20.432633686382445 - type: nauc_mrr_at_20_std value: 0.08162211942258811 - type: nauc_mrr_at_3_diff1 value: 29.05219557839034 - type: nauc_mrr_at_3_max value: 19.286442075009553 - type: nauc_mrr_at_3_std value: -1.4920090443959984 - type: nauc_mrr_at_5_diff1 value: 28.84976179309284 - type: nauc_mrr_at_5_max value: 20.03989971433926 - type: nauc_mrr_at_5_std value: -0.6327717473113279 - type: nauc_ndcg_at_1000_diff1 value: 27.263814137108294 - type: nauc_ndcg_at_1000_max value: 21.54444104314226 - type: nauc_ndcg_at_1000_std value: 3.9345593404348325 - type: nauc_ndcg_at_100_diff1 value: 26.920118374587148 - type: nauc_ndcg_at_100_max value: 21.744889824732063 - type: nauc_ndcg_at_100_std value: 4.28614804478737 - type: nauc_ndcg_at_10_diff1 value: 26.607429479330936 - type: nauc_ndcg_at_10_max value: 21.3810679241016 - type: nauc_ndcg_at_10_std value: 2.8694133195492135 - type: nauc_ndcg_at_1_diff1 value: 32.92934503399022 - type: nauc_ndcg_at_1_max value: 16.941727815666827 - type: nauc_ndcg_at_1_std value: -3.3732010441778253 - type: nauc_ndcg_at_20_diff1 value: 26.5967994647727 - type: nauc_ndcg_at_20_max value: 21.659688473792883 - type: nauc_ndcg_at_20_std value: 3.842346776620175 - type: nauc_ndcg_at_3_diff1 value: 27.158766570572386 - type: nauc_ndcg_at_3_max value: 19.119402545875683 - type: nauc_ndcg_at_3_std value: -0.35953162925164833 - type: nauc_ndcg_at_5_diff1 value: 26.730692135636065 - type: nauc_ndcg_at_5_max value: 20.414969250885925 - type: nauc_ndcg_at_5_std value: 1.2752832955418998 - type: nauc_precision_at_1000_diff1 value: 2.561063684584515 - type: nauc_precision_at_1000_max value: 19.186729674644486 - type: nauc_precision_at_1000_std value: 18.093647810661352 - type: nauc_precision_at_100_diff1 value: 9.316421561811165 - type: nauc_precision_at_100_max value: 27.738024328392086 - type: nauc_precision_at_100_std value: 22.433924792860076 - type: nauc_precision_at_10_diff1 value: 19.426331365751906 - type: nauc_precision_at_10_max value: 28.635390613812916 - type: nauc_precision_at_10_std value: 10.855946901322524 - type: nauc_precision_at_1_diff1 value: 32.92934503399022 - type: nauc_precision_at_1_max value: 16.941727815666827 - type: nauc_precision_at_1_std value: -3.3732010441778253 - type: nauc_precision_at_20_diff1 value: 16.34675014694496 - type: nauc_precision_at_20_max value: 29.832850703812035 - type: nauc_precision_at_20_std value: 16.553248881871074 - type: nauc_precision_at_3_diff1 value: 23.995435131526925 - type: nauc_precision_at_3_max value: 22.11997270130166 - type: nauc_precision_at_3_std value: 0.8503676668029546 - type: nauc_precision_at_5_diff1 value: 22.001243194385456 - type: nauc_precision_at_5_max value: 25.475925553993733 - type: nauc_precision_at_5_std value: 5.022727930559935 - type: nauc_recall_at_1000_diff1 value: 14.645333801590885 - type: nauc_recall_at_1000_max value: 30.164138362969688 - type: nauc_recall_at_1000_std value: 38.22549566467189 - type: nauc_recall_at_100_diff1 value: 15.503889282866288 - type: nauc_recall_at_100_max value: 29.086237873053495 - type: nauc_recall_at_100_std value: 27.06734385373416 - type: nauc_recall_at_10_diff1 value: 19.403300497172893 - type: nauc_recall_at_10_max value: 25.248649908264113 - type: nauc_recall_at_10_std value: 10.936513113166264 - type: nauc_recall_at_1_diff1 value: 31.738179514738864 - type: nauc_recall_at_1_max value: 15.968106583150215 - type: nauc_recall_at_1_std value: -2.570993256036204 - type: nauc_recall_at_20_diff1 value: 17.733156124123276 - type: nauc_recall_at_20_max value: 26.746759836363143 - type: nauc_recall_at_20_std value: 16.75017273825386 - type: nauc_recall_at_3_diff1 value: 22.68171478313316 - type: nauc_recall_at_3_max value: 19.636825377253857 - type: nauc_recall_at_3_std value: 1.5757483953126163 - type: nauc_recall_at_5_diff1 value: 20.974147223847623 - type: nauc_recall_at_5_max value: 22.322448079695377 - type: nauc_recall_at_5_std value: 5.398452974444173 - type: ndcg_at_1 value: 30.104999999999997 - type: ndcg_at_10 value: 45.507999999999996 - type: ndcg_at_100 value: 49.88 - type: ndcg_at_1000 value: 51.064 - type: ndcg_at_20 value: 47.504000000000005 - type: ndcg_at_3 value: 38.913 - type: ndcg_at_5 value: 42.329 - type: precision_at_1 value: 30.104999999999997 - type: precision_at_10 value: 7.199 - type: precision_at_100 value: 0.962 - type: precision_at_1000 value: 0.109 - type: precision_at_20 value: 4.051 - type: precision_at_3 value: 17.087 - type: precision_at_5 value: 12.16 - type: recall_at_1 value: 26.979 - type: recall_at_10 value: 63.366 - type: recall_at_100 value: 83.062 - type: recall_at_1000 value: 91.847 - type: recall_at_20 value: 70.938 - type: recall_at_3 value: 45.544000000000004 - type: recall_at_5 value: 53.822 task: type: Retrieval - dataset: config: default name: MTEB FiQA2018 (default) revision: 27a168819829fe9bcd655c2df245fb19452e8e06 split: dev type: mteb/fiqa metrics: - type: main_score value: 38.815 - type: map_at_1 value: 20.335 - type: map_at_10 value: 31.272 - type: map_at_100 value: 33.190999999999995 - type: map_at_1000 value: 33.344 - type: map_at_20 value: 32.256 - type: map_at_3 value: 27.567000000000004 - type: map_at_5 value: 29.458000000000002 - type: mrr_at_1 value: 36.199999999999996 - type: mrr_at_10 value: 45.32039682539683 - type: mrr_at_100 value: 46.062971677642594 - type: mrr_at_1000 value: 46.112522460479234 - type: mrr_at_20 value: 45.753339829245405 - type: mrr_at_3 value: 42.96666666666667 - type: mrr_at_5 value: 44.276666666666664 - type: nauc_map_at_1000_diff1 value: 39.09686251702312 - type: nauc_map_at_1000_max value: 23.718502764114042 - type: nauc_map_at_1000_std value: -5.127196142411253 - type: nauc_map_at_100_diff1 value: 39.09364693819265 - type: nauc_map_at_100_max value: 23.653458304303854 - type: nauc_map_at_100_std value: -5.177316371185933 - type: nauc_map_at_10_diff1 value: 39.31442389333748 - type: nauc_map_at_10_max value: 21.74468737473981 - type: nauc_map_at_10_std value: -6.696579950725854 - type: nauc_map_at_1_diff1 value: 47.43702042188656 - type: nauc_map_at_1_max value: 16.34626334329432 - type: nauc_map_at_1_std value: -13.86785028017293 - type: nauc_map_at_20_diff1 value: 39.17778732888042 - type: nauc_map_at_20_max value: 22.46617212441066 - type: nauc_map_at_20_std value: -6.34830253856367 - type: nauc_map_at_3_diff1 value: 39.15300459049625 - type: nauc_map_at_3_max value: 19.302611306768988 - type: nauc_map_at_3_std value: -8.794316331911508 - type: nauc_map_at_5_diff1 value: 38.782168864662474 - type: nauc_map_at_5_max value: 20.107759031023953 - type: nauc_map_at_5_std value: -8.188426276758069 - type: nauc_mrr_at_1000_diff1 value: 44.08705555634411 - type: nauc_mrr_at_1000_max value: 33.33076293762814 - type: nauc_mrr_at_1000_std value: -3.1398513714145397 - type: nauc_mrr_at_100_diff1 value: 44.06582470169942 - type: nauc_mrr_at_100_max value: 33.344145740218686 - type: nauc_mrr_at_100_std value: -3.08986027603163 - type: nauc_mrr_at_10_diff1 value: 44.30974388499585 - type: nauc_mrr_at_10_max value: 33.406651307005205 - type: nauc_mrr_at_10_std value: -3.2928612666079515 - type: nauc_mrr_at_1_diff1 value: 51.40538032179158 - type: nauc_mrr_at_1_max value: 31.412039525256464 - type: nauc_mrr_at_1_std value: -8.907602241057901 - type: nauc_mrr_at_20_diff1 value: 43.980675465268725 - type: nauc_mrr_at_20_max value: 33.32527070543209 - type: nauc_mrr_at_20_std value: -3.2339919035351694 - type: nauc_mrr_at_3_diff1 value: 44.647084271055625 - type: nauc_mrr_at_3_max value: 32.45045666872438 - type: nauc_mrr_at_3_std value: -4.529436120509803 - type: nauc_mrr_at_5_diff1 value: 43.67143288169307 - type: nauc_mrr_at_5_max value: 32.44875364546683 - type: nauc_mrr_at_5_std value: -3.9584779431763346 - type: nauc_ndcg_at_1000_diff1 value: 38.354081675214225 - type: nauc_ndcg_at_1000_max value: 30.245302655913598 - type: nauc_ndcg_at_1000_std value: 0.7890765259603763 - type: nauc_ndcg_at_100_diff1 value: 37.817904227181856 - type: nauc_ndcg_at_100_max value: 30.02565642657375 - type: nauc_ndcg_at_100_std value: 1.7134329123978582 - type: nauc_ndcg_at_10_diff1 value: 38.58070071452654 - type: nauc_ndcg_at_10_max value: 26.03860386223017 - type: nauc_ndcg_at_10_std value: -2.807697044805227 - type: nauc_ndcg_at_1_diff1 value: 51.40538032179158 - type: nauc_ndcg_at_1_max value: 31.412039525256464 - type: nauc_ndcg_at_1_std value: -8.907602241057901 - type: nauc_ndcg_at_20_diff1 value: 37.84931412054451 - type: nauc_ndcg_at_20_max value: 26.611532647536706 - type: nauc_ndcg_at_20_std value: -2.551835640081501 - type: nauc_ndcg_at_3_diff1 value: 37.58344478410785 - type: nauc_ndcg_at_3_max value: 25.888699537427723 - type: nauc_ndcg_at_3_std value: -5.6005333462847995 - type: nauc_ndcg_at_5_diff1 value: 37.20037762553382 - type: nauc_ndcg_at_5_max value: 24.195277511419018 - type: nauc_ndcg_at_5_std value: -5.303803702718122 - type: nauc_precision_at_1000_diff1 value: -4.230348587018459 - type: nauc_precision_at_1000_max value: 31.819624500936953 - type: nauc_precision_at_1000_std value: 20.312378085793235 - type: nauc_precision_at_100_diff1 value: 2.3773304611561423 - type: nauc_precision_at_100_max value: 37.26249793571357 - type: nauc_precision_at_100_std value: 23.155582812623898 - type: nauc_precision_at_10_diff1 value: 15.88778259182176 - type: nauc_precision_at_10_max value: 33.63617175318708 - type: nauc_precision_at_10_std value: 10.546198556997263 - type: nauc_precision_at_1_diff1 value: 51.40538032179158 - type: nauc_precision_at_1_max value: 31.412039525256464 - type: nauc_precision_at_1_std value: -8.907602241057901 - type: nauc_precision_at_20_diff1 value: 10.449607655939202 - type: nauc_precision_at_20_max value: 33.7352181569909 - type: nauc_precision_at_20_std value: 12.571650811884084 - type: nauc_precision_at_3_diff1 value: 22.99546456955021 - type: nauc_precision_at_3_max value: 31.02368858204585 - type: nauc_precision_at_3_std value: 2.5868277272596245 - type: nauc_precision_at_5_diff1 value: 17.907774229925252 - type: nauc_precision_at_5_max value: 30.63973977791014 - type: nauc_precision_at_5_std value: 3.8676417305257025 - type: nauc_recall_at_1000_diff1 value: 23.63898544479131 - type: nauc_recall_at_1000_max value: 39.32223559294475 - type: nauc_recall_at_1000_std value: 37.00507997584423 - type: nauc_recall_at_100_diff1 value: 23.929610307083696 - type: nauc_recall_at_100_max value: 33.47151501002676 - type: nauc_recall_at_100_std value: 25.8591375037135 - type: nauc_recall_at_10_diff1 value: 29.42318630440556 - type: nauc_recall_at_10_max value: 22.089958081883086 - type: nauc_recall_at_10_std value: 3.27054006421318 - type: nauc_recall_at_1_diff1 value: 47.43702042188656 - type: nauc_recall_at_1_max value: 16.34626334329432 - type: nauc_recall_at_1_std value: -13.86785028017293 - type: nauc_recall_at_20_diff1 value: 25.975696003491827 - type: nauc_recall_at_20_max value: 22.787194656795947 - type: nauc_recall_at_20_std value: 4.675488568813387 - type: nauc_recall_at_3_diff1 value: 30.55655187585358 - type: nauc_recall_at_3_max value: 16.946450217238578 - type: nauc_recall_at_3_std value: -5.74601816493791 - type: nauc_recall_at_5_diff1 value: 27.85153909140805 - type: nauc_recall_at_5_max value: 16.326181395506627 - type: nauc_recall_at_5_std value: -4.086355622548156 - type: ndcg_at_1 value: 36.199999999999996 - type: ndcg_at_10 value: 38.815 - type: ndcg_at_100 value: 45.626 - type: ndcg_at_1000 value: 48.607 - type: ndcg_at_20 value: 41.229 - type: ndcg_at_3 value: 34.974 - type: ndcg_at_5 value: 36.027 - type: precision_at_1 value: 36.199999999999996 - type: precision_at_10 value: 10.280000000000001 - type: precision_at_100 value: 1.764 - type: precision_at_1000 value: 0.22399999999999998 - type: precision_at_20 value: 6.260000000000001 - type: precision_at_3 value: 22.0 - type: precision_at_5 value: 16.12 - type: recall_at_1 value: 20.335 - type: recall_at_10 value: 46.654 - type: recall_at_100 value: 72.045 - type: recall_at_1000 value: 91.269 - type: recall_at_20 value: 53.835 - type: recall_at_3 value: 32.669 - type: recall_at_5 value: 37.972 task: type: Retrieval - dataset: config: default name: MTEB FiQA2018 (default) revision: 27a168819829fe9bcd655c2df245fb19452e8e06 split: test type: mteb/fiqa metrics: - type: main_score value: 36.867 - type: map_at_1 value: 17.14 - type: map_at_10 value: 29.141000000000002 - type: map_at_100 value: 30.956 - type: map_at_1000 value: 31.159 - type: map_at_20 value: 30.072 - type: map_at_3 value: 25.188 - type: map_at_5 value: 27.506999999999998 - type: mrr_at_1 value: 34.72222222222222 - type: mrr_at_10 value: 44.51223544973544 - type: mrr_at_100 value: 45.42790088583724 - type: mrr_at_1000 value: 45.471809231059844 - type: mrr_at_20 value: 45.091318234378086 - type: mrr_at_3 value: 42.001028806584365 - type: mrr_at_5 value: 43.40534979423868 - type: nauc_map_at_1000_diff1 value: 34.5997613709207 - type: nauc_map_at_1000_max value: 29.271778485903077 - type: nauc_map_at_1000_std value: -3.1727467448529327 - type: nauc_map_at_100_diff1 value: 34.554319942549405 - type: nauc_map_at_100_max value: 29.135316364116893 - type: nauc_map_at_100_std value: -3.2318945894116595 - type: nauc_map_at_10_diff1 value: 34.792406662543804 - type: nauc_map_at_10_max value: 28.23419192864 - type: nauc_map_at_10_std value: -4.272346935010467 - type: nauc_map_at_1_diff1 value: 42.29082590818265 - type: nauc_map_at_1_max value: 22.639965829727384 - type: nauc_map_at_1_std value: -6.491951835687574 - type: nauc_map_at_20_diff1 value: 34.68427524059249 - type: nauc_map_at_20_max value: 28.665312874057904 - type: nauc_map_at_20_std value: -3.7912174506529306 - type: nauc_map_at_3_diff1 value: 37.34454537931914 - type: nauc_map_at_3_max value: 26.51145758791601 - type: nauc_map_at_3_std value: -5.988050398564233 - type: nauc_map_at_5_diff1 value: 36.239606626498436 - type: nauc_map_at_5_max value: 28.06332898952531 - type: nauc_map_at_5_std value: -5.117662764730185 - type: nauc_mrr_at_1000_diff1 value: 39.90793047345469 - type: nauc_mrr_at_1000_max value: 36.81917469749475 - type: nauc_mrr_at_1000_std value: -0.3231989591220033 - type: nauc_mrr_at_100_diff1 value: 39.864015740614924 - type: nauc_mrr_at_100_max value: 36.8047799137878 - type: nauc_mrr_at_100_std value: -0.29755042898763223 - type: nauc_mrr_at_10_diff1 value: 40.08579428751901 - type: nauc_mrr_at_10_max value: 36.796457802782996 - type: nauc_mrr_at_10_std value: -0.5536913949365608 - type: nauc_mrr_at_1_diff1 value: 45.04268958292539 - type: nauc_mrr_at_1_max value: 36.400317672526775 - type: nauc_mrr_at_1_std value: -2.686996861248284 - type: nauc_mrr_at_20_diff1 value: 39.82612669032365 - type: nauc_mrr_at_20_max value: 36.69705897717761 - type: nauc_mrr_at_20_std value: -0.4874377274349413 - type: nauc_mrr_at_3_diff1 value: 40.63402967371404 - type: nauc_mrr_at_3_max value: 36.639468909645906 - type: nauc_mrr_at_3_std value: -0.6826447878038451 - type: nauc_mrr_at_5_diff1 value: 40.35943695180751 - type: nauc_mrr_at_5_max value: 37.27151129074283 - type: nauc_mrr_at_5_std value: -0.7022269240751977 - type: nauc_ndcg_at_1000_diff1 value: 34.29778273967179 - type: nauc_ndcg_at_1000_max value: 33.60387421296041 - type: nauc_ndcg_at_1000_std value: 1.5784888442985348 - type: nauc_ndcg_at_100_diff1 value: 32.868484463640385 - type: nauc_ndcg_at_100_max value: 32.20835014990049 - type: nauc_ndcg_at_100_std value: 1.6443061101102199 - type: nauc_ndcg_at_10_diff1 value: 33.739914351355644 - type: nauc_ndcg_at_10_max value: 29.835483426992717 - type: nauc_ndcg_at_10_std value: -2.2105753291581305 - type: nauc_ndcg_at_1_diff1 value: 45.04268958292539 - type: nauc_ndcg_at_1_max value: 36.400317672526775 - type: nauc_ndcg_at_1_std value: -2.686996861248284 - type: nauc_ndcg_at_20_diff1 value: 33.091200318256405 - type: nauc_ndcg_at_20_max value: 30.30579094215661 - type: nauc_ndcg_at_20_std value: -1.1235201999007105 - type: nauc_ndcg_at_3_diff1 value: 36.56385805439159 - type: nauc_ndcg_at_3_max value: 32.483956717192505 - type: nauc_ndcg_at_3_std value: -3.6144143743586556 - type: nauc_ndcg_at_5_diff1 value: 34.89655956044515 - type: nauc_ndcg_at_5_max value: 31.02603211763708 - type: nauc_ndcg_at_5_std value: -3.1752399107619946 - type: nauc_precision_at_1000_diff1 value: 0.3236590900649295 - type: nauc_precision_at_1000_max value: 27.104634809451156 - type: nauc_precision_at_1000_std value: 11.778520603089962 - type: nauc_precision_at_100_diff1 value: 3.9447294506839174 - type: nauc_precision_at_100_max value: 31.227338950602707 - type: nauc_precision_at_100_std value: 13.77417003497627 - type: nauc_precision_at_10_diff1 value: 14.530292534293594 - type: nauc_precision_at_10_max value: 33.445331953391324 - type: nauc_precision_at_10_std value: 4.744663300034478 - type: nauc_precision_at_1_diff1 value: 45.04268958292539 - type: nauc_precision_at_1_max value: 36.400317672526775 - type: nauc_precision_at_1_std value: -2.686996861248284 - type: nauc_precision_at_20_diff1 value: 10.59165816378097 - type: nauc_precision_at_20_max value: 32.70515238034688 - type: nauc_precision_at_20_std value: 8.709471804938254 - type: nauc_precision_at_3_diff1 value: 28.667804160578804 - type: nauc_precision_at_3_max value: 37.1571013838137 - type: nauc_precision_at_3_std value: -1.4143179294066177 - type: nauc_precision_at_5_diff1 value: 21.82901016239852 - type: nauc_precision_at_5_max value: 36.582403285700835 - type: nauc_precision_at_5_std value: 1.1870648076304775 - type: nauc_recall_at_1000_diff1 value: 18.96043919824667 - type: nauc_recall_at_1000_max value: 33.945765157259046 - type: nauc_recall_at_1000_std value: 36.3186174302301 - type: nauc_recall_at_100_diff1 value: 12.093586422309906 - type: nauc_recall_at_100_max value: 22.887428460018633 - type: nauc_recall_at_100_std value: 16.93109588705723 - type: nauc_recall_at_10_diff1 value: 21.03576962038248 - type: nauc_recall_at_10_max value: 20.29671222282212 - type: nauc_recall_at_10_std value: -0.33083371279028084 - type: nauc_recall_at_1_diff1 value: 42.29082590818265 - type: nauc_recall_at_1_max value: 22.639965829727384 - type: nauc_recall_at_1_std value: -6.491951835687574 - type: nauc_recall_at_20_diff1 value: 17.38458169914841 - type: nauc_recall_at_20_max value: 19.627837586826082 - type: nauc_recall_at_20_std value: 2.3805728682334526 - type: nauc_recall_at_3_diff1 value: 29.28228006768503 - type: nauc_recall_at_3_max value: 22.866032768464663 - type: nauc_recall_at_3_std value: -4.417326928891438 - type: nauc_recall_at_5_diff1 value: 25.76912388055655 - type: nauc_recall_at_5_max value: 23.087890328997634 - type: nauc_recall_at_5_std value: -2.4408790779942664 - type: ndcg_at_1 value: 34.721999999999994 - type: ndcg_at_10 value: 36.867 - type: ndcg_at_100 value: 43.931 - type: ndcg_at_1000 value: 47.276 - type: ndcg_at_20 value: 39.432 - type: ndcg_at_3 value: 33.18 - type: ndcg_at_5 value: 34.504000000000005 - type: precision_at_1 value: 34.721999999999994 - type: precision_at_10 value: 10.448 - type: precision_at_100 value: 1.778 - type: precision_at_1000 value: 0.23600000000000002 - type: precision_at_20 value: 6.296 - type: precision_at_3 value: 22.377 - type: precision_at_5 value: 16.759 - type: recall_at_1 value: 17.14 - type: recall_at_10 value: 44.131 - type: recall_at_100 value: 70.60600000000001 - type: recall_at_1000 value: 90.672 - type: recall_at_20 value: 51.995000000000005 - type: recall_at_3 value: 30.536 - type: recall_at_5 value: 36.706 task: type: Retrieval - dataset: config: default name: MTEB FiQA2018 (default) revision: 27a168819829fe9bcd655c2df245fb19452e8e06 split: train type: mteb/fiqa metrics: - type: main_score value: 36.609 - type: map_at_1 value: 17.941 - type: map_at_10 value: 29.221999999999998 - type: map_at_100 value: 31.122 - type: map_at_1000 value: 31.305 - type: map_at_20 value: 30.237000000000002 - type: map_at_3 value: 25.374000000000002 - type: map_at_5 value: 27.422 - type: mrr_at_1 value: 33.96363636363636 - type: mrr_at_10 value: 42.89349927849928 - type: mrr_at_100 value: 43.799377407484336 - type: mrr_at_1000 value: 43.84799036352109 - type: mrr_at_20 value: 43.40050730549745 - type: mrr_at_3 value: 40.29090909090909 - type: mrr_at_5 value: 41.76636363636364 - type: nauc_map_at_1000_diff1 value: 38.31311265305888 - type: nauc_map_at_1000_max value: 27.796611741675825 - type: nauc_map_at_1000_std value: -0.5003293769017596 - type: nauc_map_at_100_diff1 value: 38.30329718195317 - type: nauc_map_at_100_max value: 27.67541927614693 - type: nauc_map_at_100_std value: -0.5698259499765883 - type: nauc_map_at_10_diff1 value: 38.37764336989788 - type: nauc_map_at_10_max value: 26.41684565430942 - type: nauc_map_at_10_std value: -2.227388823202941 - type: nauc_map_at_1_diff1 value: 43.584307937822494 - type: nauc_map_at_1_max value: 19.122356537834335 - type: nauc_map_at_1_std value: -5.768218333994525 - type: nauc_map_at_20_diff1 value: 38.334305440452496 - type: nauc_map_at_20_max value: 27.10217340122031 - type: nauc_map_at_20_std value: -1.3391204810507156 - type: nauc_map_at_3_diff1 value: 39.23894015593411 - type: nauc_map_at_3_max value: 23.59508795412976 - type: nauc_map_at_3_std value: -4.709964052658204 - type: nauc_map_at_5_diff1 value: 38.64662605958464 - type: nauc_map_at_5_max value: 25.09311144593201 - type: nauc_map_at_5_std value: -3.5135197704430126 - type: nauc_mrr_at_1000_diff1 value: 41.10436711457144 - type: nauc_mrr_at_1000_max value: 34.51060780140401 - type: nauc_mrr_at_1000_std value: 1.5284128508937864 - type: nauc_mrr_at_100_diff1 value: 41.09013437372231 - type: nauc_mrr_at_100_max value: 34.51469113741547 - type: nauc_mrr_at_100_std value: 1.5493835226225707 - type: nauc_mrr_at_10_diff1 value: 41.08319551271548 - type: nauc_mrr_at_10_max value: 34.37624451245267 - type: nauc_mrr_at_10_std value: 1.1419873008771384 - type: nauc_mrr_at_1_diff1 value: 45.80451390127011 - type: nauc_mrr_at_1_max value: 34.5836075937778 - type: nauc_mrr_at_1_std value: -1.2520732162148687 - type: nauc_mrr_at_20_diff1 value: 40.9963963346833 - type: nauc_mrr_at_20_max value: 34.45146973858243 - type: nauc_mrr_at_20_std value: 1.49800506844926 - type: nauc_mrr_at_3_diff1 value: 41.67576934001856 - type: nauc_mrr_at_3_max value: 34.220331609509586 - type: nauc_mrr_at_3_std value: 0.2681645968292583 - type: nauc_mrr_at_5_diff1 value: 41.25801597151194 - type: nauc_mrr_at_5_max value: 34.28051747187494 - type: nauc_mrr_at_5_std value: 0.7512368738269359 - type: nauc_ndcg_at_1000_diff1 value: 37.33571905289884 - type: nauc_ndcg_at_1000_max value: 32.386682549906645 - type: nauc_ndcg_at_1000_std value: 5.178519279028532 - type: nauc_ndcg_at_100_diff1 value: 36.964657701725564 - type: nauc_ndcg_at_100_max value: 31.285424273176197 - type: nauc_ndcg_at_100_std value: 5.2425868283455985 - type: nauc_ndcg_at_10_diff1 value: 37.27732435458774 - type: nauc_ndcg_at_10_max value: 28.84472459395174 - type: nauc_ndcg_at_10_std value: 0.4065378878005523 - type: nauc_ndcg_at_1_diff1 value: 45.80451390127011 - type: nauc_ndcg_at_1_max value: 34.5836075937778 - type: nauc_ndcg_at_1_std value: -1.2520732162148687 - type: nauc_ndcg_at_20_diff1 value: 36.941897541551505 - type: nauc_ndcg_at_20_max value: 29.63888858566235 - type: nauc_ndcg_at_20_std value: 2.372793909457969 - type: nauc_ndcg_at_3_diff1 value: 37.90735452944114 - type: nauc_ndcg_at_3_max value: 29.948026969129543 - type: nauc_ndcg_at_3_std value: -1.0585917576906725 - type: nauc_ndcg_at_5_diff1 value: 37.59875176994766 - type: nauc_ndcg_at_5_max value: 28.617625143556264 - type: nauc_ndcg_at_5_std value: -0.9202152226855952 - type: nauc_precision_at_1000_diff1 value: -1.842530043167204 - type: nauc_precision_at_1000_max value: 31.736279234645604 - type: nauc_precision_at_1000_std value: 20.47877783043743 - type: nauc_precision_at_100_diff1 value: 5.537122423054518 - type: nauc_precision_at_100_max value: 36.50948967541072 - type: nauc_precision_at_100_std value: 23.89557428126227 - type: nauc_precision_at_10_diff1 value: 18.643305289257178 - type: nauc_precision_at_10_max value: 38.50576761086419 - type: nauc_precision_at_10_std value: 12.13351332500471 - type: nauc_precision_at_1_diff1 value: 45.80451390127011 - type: nauc_precision_at_1_max value: 34.5836075937778 - type: nauc_precision_at_1_std value: -1.2520732162148687 - type: nauc_precision_at_20_diff1 value: 14.278241914224024 - type: nauc_precision_at_20_max value: 38.46094800402192 - type: nauc_precision_at_20_std value: 16.98733564794925 - type: nauc_precision_at_3_diff1 value: 28.04687109150208 - type: nauc_precision_at_3_max value: 37.208872365419154 - type: nauc_precision_at_3_std value: 4.456711276122235 - type: nauc_precision_at_5_diff1 value: 23.839704326874223 - type: nauc_precision_at_5_max value: 38.2698854604425 - type: nauc_precision_at_5_std value: 7.929623249647425 - type: nauc_recall_at_1000_diff1 value: 18.255196640208556 - type: nauc_recall_at_1000_max value: 29.049424513466704 - type: nauc_recall_at_1000_std value: 40.88677106793111 - type: nauc_recall_at_100_diff1 value: 22.13278638781853 - type: nauc_recall_at_100_max value: 23.170448924733204 - type: nauc_recall_at_100_std value: 21.807403505494804 - type: nauc_recall_at_10_diff1 value: 28.08883464717724 - type: nauc_recall_at_10_max value: 21.404728891014717 - type: nauc_recall_at_10_std value: 2.2419813761011773 - type: nauc_recall_at_1_diff1 value: 43.584307937822494 - type: nauc_recall_at_1_max value: 19.122356537834335 - type: nauc_recall_at_1_std value: -5.768218333994525 - type: nauc_recall_at_20_diff1 value: 25.649658018758327 - type: nauc_recall_at_20_max value: 21.944991459430334 - type: nauc_recall_at_20_std value: 7.8871985597412095 - type: nauc_recall_at_3_diff1 value: 32.750731455991236 - type: nauc_recall_at_3_max value: 20.017873825740125 - type: nauc_recall_at_3_std value: -3.8959541909716857 - type: nauc_recall_at_5_diff1 value: 30.296818458928897 - type: nauc_recall_at_5_max value: 20.547813354339194 - type: nauc_recall_at_5_std value: -1.7022461655111982 - type: ndcg_at_1 value: 33.964 - type: ndcg_at_10 value: 36.609 - type: ndcg_at_100 value: 43.814 - type: ndcg_at_1000 value: 46.961999999999996 - type: ndcg_at_20 value: 39.222 - type: ndcg_at_3 value: 32.414 - type: ndcg_at_5 value: 33.824 - type: precision_at_1 value: 33.964 - type: precision_at_10 value: 10.096 - type: precision_at_100 value: 1.7420000000000002 - type: precision_at_1000 value: 0.22899999999999998 - type: precision_at_20 value: 6.146 - type: precision_at_3 value: 21.309 - type: precision_at_5 value: 15.822 - type: recall_at_1 value: 17.941 - type: recall_at_10 value: 44.019999999999996 - type: recall_at_100 value: 71.57900000000001 - type: recall_at_1000 value: 90.688 - type: recall_at_20 value: 52.214000000000006 - type: recall_at_3 value: 30.152 - type: recall_at_5 value: 35.882 task: type: Retrieval - dataset: config: default name: MTEB ImdbClassification (default) revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 split: test type: mteb/imdb metrics: - type: accuracy value: 61.763999999999996 - type: ap value: 57.19090390623197 - type: ap_weighted value: 57.19090390623197 - type: f1 value: 61.41638740871057 - type: f1_weighted value: 61.41638740871057 - type: main_score value: 61.763999999999996 task: type: Classification - dataset: config: en name: MTEB MTOPDomainClassification (en) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 91.67806657546741 - type: f1 value: 91.255563138932 - type: f1_weighted value: 91.67838466213225 - type: main_score value: 91.67806657546741 task: type: Classification - dataset: config: de name: MTEB MTOPDomainClassification (de) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 70.47337278106508 - type: f1 value: 67.2214808141468 - type: f1_weighted value: 70.40312934833233 - type: main_score value: 70.47337278106508 task: type: Classification - dataset: config: es name: MTEB MTOPDomainClassification (es) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 72.99199466310873 - type: f1 value: 70.49410629424256 - type: f1_weighted value: 73.17064507343018 - type: main_score value: 72.99199466310873 task: type: Classification - dataset: config: fr name: MTEB MTOPDomainClassification (fr) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 75.09865330410273 - type: f1 value: 72.27554297058502 - type: f1_weighted value: 75.09636807556879 - type: main_score value: 75.09865330410273 task: type: Classification - dataset: config: hi name: MTEB MTOPDomainClassification (hi) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 40.73861599139475 - type: f1 value: 38.73178840632667 - type: f1_weighted value: 39.80798049928233 - type: main_score value: 40.73861599139475 task: type: Classification - dataset: config: th name: MTEB MTOPDomainClassification (th) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 16.40867992766727 - type: f1 value: 12.678924539856768 - type: f1_weighted value: 13.167968855792815 - type: main_score value: 16.40867992766727 task: type: Classification - dataset: config: en name: MTEB MTOPDomainClassification (en) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: validation type: mteb/mtop_domain metrics: - type: accuracy value: 91.96868008948546 - type: f1 value: 91.96167311423682 - type: f1_weighted value: 91.93715165013899 - type: main_score value: 91.96868008948546 task: type: Classification - dataset: config: de name: MTEB MTOPDomainClassification (de) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: validation type: mteb/mtop_domain metrics: - type: accuracy value: 70.39118457300276 - type: f1 value: 67.76272194382894 - type: f1_weighted value: 70.25335384362819 - type: main_score value: 70.39118457300276 task: type: Classification - dataset: config: es name: MTEB MTOPDomainClassification (es) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: validation type: mteb/mtop_domain metrics: - type: accuracy value: 73.35297969875573 - type: f1 value: 71.78213608249169 - type: f1_weighted value: 73.36604149635156 - type: main_score value: 73.35297969875573 task: type: Classification - dataset: config: fr name: MTEB MTOPDomainClassification (fr) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: validation type: mteb/mtop_domain metrics: - type: accuracy value: 75.57387444514902 - type: f1 value: 73.82790466941917 - type: f1_weighted value: 75.53887538348995 - type: main_score value: 75.57387444514902 task: type: Classification - dataset: config: hi name: MTEB MTOPDomainClassification (hi) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: validation type: mteb/mtop_domain metrics: - type: accuracy value: 39.05566600397614 - type: f1 value: 37.43458014268371 - type: f1_weighted value: 38.01336076899539 - type: main_score value: 39.05566600397614 task: type: Classification - dataset: config: th name: MTEB MTOPDomainClassification (th) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: validation type: mteb/mtop_domain metrics: - type: accuracy value: 17.08557749850389 - type: f1 value: 12.890503127470817 - type: f1_weighted value: 13.244607545895454 - type: main_score value: 17.08557749850389 task: type: Classification - dataset: config: en name: MTEB MTOPIntentClassification (en) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 61.55494756041951 - type: f1 value: 42.98989289014766 - type: f1_weighted value: 63.811676869594066 - type: main_score value: 61.55494756041951 task: type: Classification - dataset: config: de name: MTEB MTOPIntentClassification (de) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 45.70019723865878 - type: f1 value: 26.66317456263853 - type: f1_weighted value: 50.353986633624025 - type: main_score value: 45.70019723865878 task: type: Classification - dataset: config: es name: MTEB MTOPIntentClassification (es) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 44.18945963975983 - type: f1 value: 28.25625231122317 - type: f1_weighted value: 47.783139674233944 - type: main_score value: 44.18945963975983 task: type: Classification - dataset: config: fr name: MTEB MTOPIntentClassification (fr) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 39.67115565299092 - type: f1 value: 26.291493274214993 - type: f1_weighted value: 43.37932154450208 - type: main_score value: 39.67115565299092 task: type: Classification - dataset: config: hi name: MTEB MTOPIntentClassification (hi) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 18.694872714234492 - type: f1 value: 11.148407283390524 - type: f1_weighted value: 20.105783711366733 - type: main_score value: 18.694872714234492 task: type: Classification - dataset: config: th name: MTEB MTOPIntentClassification (th) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 4.929475587703436 - type: f1 value: 1.7790992902986407 - type: f1_weighted value: 4.492570333550388 - type: main_score value: 4.929475587703436 task: type: Classification - dataset: config: en name: MTEB MTOPIntentClassification (en) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: validation type: mteb/mtop_intent metrics: - type: accuracy value: 63.29753914988815 - type: f1 value: 43.9610667106009 - type: f1_weighted value: 65.65735539710052 - type: main_score value: 63.29753914988815 task: type: Classification - dataset: config: de name: MTEB MTOPIntentClassification (de) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: validation type: mteb/mtop_intent metrics: - type: accuracy value: 45.26170798898072 - type: f1 value: 26.741527246560192 - type: f1_weighted value: 49.742655367297125 - type: main_score value: 45.26170798898072 task: type: Classification - dataset: config: es name: MTEB MTOPIntentClassification (es) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: validation type: mteb/mtop_intent metrics: - type: accuracy value: 44.89849377865095 - type: f1 value: 27.92276489012525 - type: f1_weighted value: 48.82599076071894 - type: main_score value: 44.89849377865095 task: type: Classification - dataset: config: fr name: MTEB MTOPIntentClassification (fr) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: validation type: mteb/mtop_intent metrics: - type: accuracy value: 41.122384273937854 - type: f1 value: 26.51788565339205 - type: f1_weighted value: 45.37347241302537 - type: main_score value: 41.122384273937854 task: type: Classification - dataset: config: hi name: MTEB MTOPIntentClassification (hi) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: validation type: mteb/mtop_intent metrics: - type: accuracy value: 18.295228628230614 - type: f1 value: 10.62841192331815 - type: f1_weighted value: 20.10020306825243 - type: main_score value: 18.295228628230614 task: type: Classification - dataset: config: th name: MTEB MTOPIntentClassification (th) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: validation type: mteb/mtop_intent metrics: - type: accuracy value: 4.679832435667265 - type: f1 value: 1.4100910111798104 - type: f1_weighted value: 4.484127843515124 - type: main_score value: 4.679832435667265 task: type: Classification - dataset: config: sv name: MTEB MassiveIntentClassification (sv) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 38.086751849361136 - type: f1 value: 36.11433662841115 - type: f1_weighted value: 36.93038369402614 - type: main_score value: 38.086751849361136 task: type: Classification - dataset: config: tl name: MTEB MassiveIntentClassification (tl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 37.92199058507062 - type: f1 value: 34.60519784488147 - type: f1_weighted value: 35.950268585418996 - type: main_score value: 37.92199058507062 task: type: Classification - dataset: config: mn name: MTEB MassiveIntentClassification (mn) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 20.349697377269674 - type: f1 value: 19.861798467575174 - type: f1_weighted value: 18.70291426330083 - type: main_score value: 20.349697377269674 task: type: Classification - dataset: config: bn name: MTEB MassiveIntentClassification (bn) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 13.103564223268327 - type: f1 value: 10.555573966204612 - type: f1_weighted value: 11.869621416732963 - type: main_score value: 13.103564223268327 task: type: Classification - dataset: config: nb name: MTEB MassiveIntentClassification (nb) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 39.364492266308 - type: f1 value: 37.28880607134204 - type: f1_weighted value: 39.01204053143529 - type: main_score value: 39.364492266308 task: type: Classification - dataset: config: kn name: MTEB MassiveIntentClassification (kn) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 3.137188971082717 - type: f1 value: 1.803764694912577 - type: f1_weighted value: 1.7679872665866818 - type: main_score value: 3.137188971082717 task: type: Classification - dataset: config: ur name: MTEB MassiveIntentClassification (ur) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 14.421654337592466 - type: f1 value: 13.031038633914898 - type: f1_weighted value: 13.789862802298627 - type: main_score value: 14.421654337592466 task: type: Classification - dataset: config: id name: MTEB MassiveIntentClassification (id) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 39.02488231338265 - type: f1 value: 37.86673210576318 - type: f1_weighted value: 37.823236315873395 - type: main_score value: 39.02488231338265 task: type: Classification - dataset: config: hu name: MTEB MassiveIntentClassification (hu) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 35.69266980497646 - type: f1 value: 32.98593515150782 - type: f1_weighted value: 33.90511358743444 - type: main_score value: 35.69266980497646 task: type: Classification - dataset: config: sw name: MTEB MassiveIntentClassification (sw) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 34.98318762609281 - type: f1 value: 33.25847632986703 - type: f1_weighted value: 34.076934200097995 - type: main_score value: 34.98318762609281 task: type: Classification - dataset: config: af name: MTEB MassiveIntentClassification (af) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 37.45124411566913 - type: f1 value: 35.21970315557138 - type: f1_weighted value: 36.656779928093414 - type: main_score value: 37.45124411566913 task: type: Classification - dataset: config: fa name: MTEB MassiveIntentClassification (fa) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 19.09549428379287 - type: f1 value: 16.245850854202946 - type: f1_weighted value: 16.50701036721965 - type: main_score value: 19.09549428379287 task: type: Classification - dataset: config: ka name: MTEB MassiveIntentClassification (ka) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 9.071956960322797 - type: f1 value: 6.8882995804813625 - type: f1_weighted value: 6.7010161888617885 - type: main_score value: 9.071956960322797 task: type: Classification - dataset: config: fi name: MTEB MassiveIntentClassification (fi) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 38.36583725622057 - type: f1 value: 36.03704649473293 - type: f1_weighted value: 36.88952299930921 - type: main_score value: 38.36583725622057 task: type: Classification - dataset: config: de name: MTEB MassiveIntentClassification (de) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 43.43981170141224 - type: f1 value: 39.39202567196366 - type: f1_weighted value: 42.30160905333017 - type: main_score value: 43.43981170141224 task: type: Classification - dataset: config: cy name: MTEB MassiveIntentClassification (cy) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 34.54270342972428 - type: f1 value: 31.837608673605907 - type: f1_weighted value: 33.72760611633263 - type: main_score value: 34.54270342972428 task: type: Classification - dataset: config: ar name: MTEB MassiveIntentClassification (ar) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 19.045057162071284 - type: f1 value: 16.31803976047297 - type: f1_weighted value: 18.729068739019098 - type: main_score value: 19.045057162071284 task: type: Classification - dataset: config: sl name: MTEB MassiveIntentClassification (sl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.70477471418965 - type: f1 value: 34.93749444423004 - type: f1_weighted value: 35.48898823367354 - type: main_score value: 36.70477471418965 task: type: Classification - dataset: config: ru name: MTEB MassiveIntentClassification (ru) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 27.579018157363823 - type: f1 value: 25.535991275273595 - type: f1_weighted value: 25.358345328783244 - type: main_score value: 27.579018157363823 task: type: Classification - dataset: config: is name: MTEB MassiveIntentClassification (is) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 29.94620040349697 - type: f1 value: 28.408315569879456 - type: f1_weighted value: 28.81573218712015 - type: main_score value: 29.94620040349697 task: type: Classification - dataset: config: sq name: MTEB MassiveIntentClassification (sq) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 40.69939475453935 - type: f1 value: 38.477734363969404 - type: f1_weighted value: 39.68395794831442 - type: main_score value: 40.69939475453935 task: type: Classification - dataset: config: hi name: MTEB MassiveIntentClassification (hi) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 17.69670477471419 - type: f1 value: 15.137821954421135 - type: f1_weighted value: 15.510410368460576 - type: main_score value: 17.69670477471419 task: type: Classification - dataset: config: he name: MTEB MassiveIntentClassification (he) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 22.478143913920647 - type: f1 value: 19.045337279029027 - type: f1_weighted value: 20.912483391495744 - type: main_score value: 22.478143913920647 task: type: Classification - dataset: config: jv name: MTEB MassiveIntentClassification (jv) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 35.914593140551446 - type: f1 value: 33.26097776927041 - type: f1_weighted value: 34.564460184085924 - type: main_score value: 35.914593140551446 task: type: Classification - dataset: config: ml name: MTEB MassiveIntentClassification (ml) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 2.8715534633490245 - type: f1 value: 0.8826899264059154 - type: f1_weighted value: 1.5500961747802322 - type: main_score value: 2.8715534633490245 task: type: Classification - dataset: config: ms name: MTEB MassiveIntentClassification (ms) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 35.067249495628786 - type: f1 value: 32.71762741548739 - type: f1_weighted value: 33.65549567716296 - type: main_score value: 35.067249495628786 task: type: Classification - dataset: config: th name: MTEB MassiveIntentClassification (th) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 11.257565568258237 - type: f1 value: 6.699654567486038 - type: f1_weighted value: 8.520348967670168 - type: main_score value: 11.257565568258237 task: type: Classification - dataset: config: zh-TW name: MTEB MassiveIntentClassification (zh-TW) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 22.558843308675186 - type: f1 value: 19.167706620238604 - type: f1_weighted value: 20.696119220934033 - type: main_score value: 22.558843308675186 task: type: Classification - dataset: config: da name: MTEB MassiveIntentClassification (da) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 41.00201748486886 - type: f1 value: 39.04982458105756 - type: f1_weighted value: 40.24257427134743 - type: main_score value: 41.00201748486886 task: type: Classification - dataset: config: nl name: MTEB MassiveIntentClassification (nl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 40.195023537323465 - type: f1 value: 37.167146410844765 - type: f1_weighted value: 39.20176678895285 - type: main_score value: 40.195023537323465 task: type: Classification - dataset: config: te name: MTEB MassiveIntentClassification (te) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 2.4579690652320108 - type: f1 value: 1.0961357689202462 - type: f1_weighted value: 1.6357127179714857 - type: main_score value: 2.4579690652320108 task: type: Classification - dataset: config: km name: MTEB MassiveIntentClassification (km) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 4.889038332212508 - type: f1 value: 1.6988884567635383 - type: f1_weighted value: 2.1397976733510236 - type: main_score value: 4.889038332212508 task: type: Classification - dataset: config: my name: MTEB MassiveIntentClassification (my) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 4.236718224613316 - type: f1 value: 1.365086613554998 - type: f1_weighted value: 2.1871017188572592 - type: main_score value: 4.236718224613316 task: type: Classification - dataset: config: ko name: MTEB MassiveIntentClassification (ko) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 16.045729657027568 - type: f1 value: 14.667848010256822 - type: f1_weighted value: 14.905917887308808 - type: main_score value: 16.045729657027568 task: type: Classification - dataset: config: lv name: MTEB MassiveIntentClassification (lv) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.97041022192333 - type: f1 value: 35.23835305062667 - type: f1_weighted value: 35.864159034804246 - type: main_score value: 36.97041022192333 task: type: Classification - dataset: config: ro name: MTEB MassiveIntentClassification (ro) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 40.53799596503026 - type: f1 value: 37.99155380435985 - type: f1_weighted value: 39.26420797633916 - type: main_score value: 40.53799596503026 task: type: Classification - dataset: config: ta name: MTEB MassiveIntentClassification (ta) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 11.314727639542706 - type: f1 value: 8.332310541951045 - type: f1_weighted value: 8.656954174949675 - type: main_score value: 11.314727639542706 task: type: Classification - dataset: config: ja name: MTEB MassiveIntentClassification (ja) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 31.866173503698725 - type: f1 value: 28.373846581008888 - type: f1_weighted value: 30.75079780523907 - type: main_score value: 31.866173503698725 task: type: Classification - dataset: config: en name: MTEB MassiveIntentClassification (en) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 66.93678547410894 - type: f1 value: 64.47560446428436 - type: f1_weighted value: 65.872078022765 - type: main_score value: 66.93678547410894 task: type: Classification - dataset: config: it name: MTEB MassiveIntentClassification (it) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 41.59381304640215 - type: f1 value: 40.573246109791775 - type: f1_weighted value: 39.37933707666826 - type: main_score value: 41.59381304640215 task: type: Classification - dataset: config: fr name: MTEB MassiveIntentClassification (fr) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 42.545393409549426 - type: f1 value: 39.971291296333675 - type: f1_weighted value: 41.059568632107236 - type: main_score value: 42.545393409549426 task: type: Classification - dataset: config: az name: MTEB MassiveIntentClassification (az) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 30.63214525891056 - type: f1 value: 29.837265257918045 - type: f1_weighted value: 30.25849671667742 - type: main_score value: 30.63214525891056 task: type: Classification - dataset: config: tr name: MTEB MassiveIntentClassification (tr) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 33.75924680564896 - type: f1 value: 32.01435255557262 - type: f1_weighted value: 32.719918426691144 - type: main_score value: 33.75924680564896 task: type: Classification - dataset: config: vi name: MTEB MassiveIntentClassification (vi) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 37.0880968392737 - type: f1 value: 35.56236146174331 - type: f1_weighted value: 35.20606888414104 - type: main_score value: 37.0880968392737 task: type: Classification - dataset: config: pl name: MTEB MassiveIntentClassification (pl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.065904505716205 - type: f1 value: 32.62705085748215 - type: f1_weighted value: 35.41890132681854 - type: main_score value: 36.065904505716205 task: type: Classification - dataset: config: zh-CN name: MTEB MassiveIntentClassification (zh-CN) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 24.40147948890384 - type: f1 value: 19.47454731610046 - type: f1_weighted value: 22.92591950140824 - type: main_score value: 24.40147948890384 task: type: Classification - dataset: config: am name: MTEB MassiveIntentClassification (am) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 2.6227303295225286 - type: f1 value: 1.1775617480474232 - type: f1_weighted value: 1.1872876016953893 - type: main_score value: 2.6227303295225286 task: type: Classification - dataset: config: pt name: MTEB MassiveIntentClassification (pt) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 43.7626092804304 - type: f1 value: 42.092891715123194 - type: f1_weighted value: 42.498389773010075 - type: main_score value: 43.7626092804304 task: type: Classification - dataset: config: el name: MTEB MassiveIntentClassification (el) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 24.18628110289173 - type: f1 value: 20.39372975821234 - type: f1_weighted value: 22.94692663780649 - type: main_score value: 24.18628110289173 task: type: Classification - dataset: config: hy name: MTEB MassiveIntentClassification (hy) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 7.616005379959651 - type: f1 value: 6.343861198263652 - type: f1_weighted value: 6.77160484781972 - type: main_score value: 7.616005379959651 task: type: Classification - dataset: config: es name: MTEB MassiveIntentClassification (es) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 39.87558843308675 - type: f1 value: 38.67761288837852 - type: f1_weighted value: 38.28774727977292 - type: main_score value: 39.87558843308675 task: type: Classification - dataset: config: sv name: MTEB MassiveIntentClassification (sv) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 37.99803246433842 - type: f1 value: 35.571161617931 - type: f1_weighted value: 36.5486545723221 - type: main_score value: 37.99803246433842 task: type: Classification - dataset: config: tl name: MTEB MassiveIntentClassification (tl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 38.09149040826365 - type: f1 value: 33.912274295119246 - type: f1_weighted value: 36.556756266143424 - type: main_score value: 38.09149040826365 task: type: Classification - dataset: config: mn name: MTEB MassiveIntentClassification (mn) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 18.883423512051152 - type: f1 value: 19.041570438181193 - type: f1_weighted value: 17.536161028823024 - type: main_score value: 18.883423512051152 task: type: Classification - dataset: config: bn name: MTEB MassiveIntentClassification (bn) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 12.095425479586815 - type: f1 value: 9.44042542776349 - type: f1_weighted value: 10.652119351710839 - type: main_score value: 12.095425479586815 task: type: Classification - dataset: config: nb name: MTEB MassiveIntentClassification (nb) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 38.180029513034924 - type: f1 value: 35.58485149410359 - type: f1_weighted value: 37.72851368367536 - type: main_score value: 38.180029513034924 task: type: Classification - dataset: config: kn name: MTEB MassiveIntentClassification (kn) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 3.3939990162321694 - type: f1 value: 1.7394879433186086 - type: f1_weighted value: 1.892238297322192 - type: main_score value: 3.3939990162321694 task: type: Classification - dataset: config: ur name: MTEB MassiveIntentClassification (ur) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 15.115592720118054 - type: f1 value: 13.46005538295957 - type: f1_weighted value: 14.16919822946092 - type: main_score value: 15.115592720118054 task: type: Classification - dataset: config: id name: MTEB MassiveIntentClassification (id) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 38.603049680275454 - type: f1 value: 37.08063906822963 - type: f1_weighted value: 37.65472896582819 - type: main_score value: 38.603049680275454 task: type: Classification - dataset: config: hu name: MTEB MassiveIntentClassification (hu) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 33.78750614854894 - type: f1 value: 30.662355819597785 - type: f1_weighted value: 32.26765980385126 - type: main_score value: 33.78750614854894 task: type: Classification - dataset: config: sw name: MTEB MassiveIntentClassification (sw) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 33.87112641416626 - type: f1 value: 31.313377740863498 - type: f1_weighted value: 33.287299550709 - type: main_score value: 33.87112641416626 task: type: Classification - dataset: config: af name: MTEB MassiveIntentClassification (af) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.527299557304474 - type: f1 value: 33.02759229029607 - type: f1_weighted value: 36.016677318648846 - type: main_score value: 36.527299557304474 task: type: Classification - dataset: config: fa name: MTEB MassiveIntentClassification (fa) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 19.65076242006886 - type: f1 value: 16.549837961668384 - type: f1_weighted value: 17.132067511217453 - type: main_score value: 19.65076242006886 task: type: Classification - dataset: config: ka name: MTEB MassiveIntentClassification (ka) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 8.55386128873586 - type: f1 value: 6.415373906918898 - type: f1_weighted value: 6.60471921971805 - type: main_score value: 8.55386128873586 task: type: Classification - dataset: config: fi name: MTEB MassiveIntentClassification (fi) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.96507624200689 - type: f1 value: 33.99800370106888 - type: f1_weighted value: 36.00766842653996 - type: main_score value: 36.96507624200689 task: type: Classification - dataset: config: de name: MTEB MassiveIntentClassification (de) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 42.21347761928185 - type: f1 value: 37.74171313861854 - type: f1_weighted value: 41.43749251098024 - type: main_score value: 42.21347761928185 task: type: Classification - dataset: config: cy name: MTEB MassiveIntentClassification (cy) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 33.68421052631579 - type: f1 value: 31.188626291842482 - type: f1_weighted value: 33.249819486045396 - type: main_score value: 33.68421052631579 task: type: Classification - dataset: config: ar name: MTEB MassiveIntentClassification (ar) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 18.017707820954254 - type: f1 value: 15.558588275187308 - type: f1_weighted value: 17.54058461476561 - type: main_score value: 18.017707820954254 task: type: Classification - dataset: config: sl name: MTEB MassiveIntentClassification (sl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.45351696999508 - type: f1 value: 34.233566716511376 - type: f1_weighted value: 35.36719539566772 - type: main_score value: 36.45351696999508 task: type: Classification - dataset: config: ru name: MTEB MassiveIntentClassification (ru) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 27.142154451549438 - type: f1 value: 24.311088224979716 - type: f1_weighted value: 24.84555275904934 - type: main_score value: 27.142154451549438 task: type: Classification - dataset: config: is name: MTEB MassiveIntentClassification (is) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 29.508116084604037 - type: f1 value: 27.86281238749086 - type: f1_weighted value: 28.407351060310482 - type: main_score value: 29.508116084604037 task: type: Classification - dataset: config: sq name: MTEB MassiveIntentClassification (sq) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 40.23610427939007 - type: f1 value: 37.95893196593081 - type: f1_weighted value: 39.39603192315687 - type: main_score value: 40.23610427939007 task: type: Classification - dataset: config: hi name: MTEB MassiveIntentClassification (hi) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 17.60452533202164 - type: f1 value: 15.495897100004944 - type: f1_weighted value: 15.596388275508538 - type: main_score value: 17.60452533202164 task: type: Classification - dataset: config: he name: MTEB MassiveIntentClassification (he) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 20.5607476635514 - type: f1 value: 17.675374159708515 - type: f1_weighted value: 19.432282025476216 - type: main_score value: 20.5607476635514 task: type: Classification - dataset: config: jv name: MTEB MassiveIntentClassification (jv) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 35.60255779636006 - type: f1 value: 32.78297558762236 - type: f1_weighted value: 34.32527584584245 - type: main_score value: 35.60255779636006 task: type: Classification - dataset: config: ml name: MTEB MassiveIntentClassification (ml) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 2.5430398425971474 - type: f1 value: 0.7706095852996598 - type: f1_weighted value: 1.2018821749877568 - type: main_score value: 2.5430398425971474 task: type: Classification - dataset: config: ms name: MTEB MassiveIntentClassification (ms) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 34.13674372848008 - type: f1 value: 31.719688822492387 - type: f1_weighted value: 33.07325807880738 - type: main_score value: 34.13674372848008 task: type: Classification - dataset: config: th name: MTEB MassiveIntentClassification (th) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 10.30004918839154 - type: f1 value: 6.0202899457504 - type: f1_weighted value: 7.74369598826699 - type: main_score value: 10.30004918839154 task: type: Classification - dataset: config: zh-TW name: MTEB MassiveIntentClassification (zh-TW) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 24.10723069355632 - type: f1 value: 19.538712594354973 - type: f1_weighted value: 22.09409475414598 - type: main_score value: 24.10723069355632 task: type: Classification - dataset: config: da name: MTEB MassiveIntentClassification (da) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 39.6704377766847 - type: f1 value: 37.55589462060647 - type: f1_weighted value: 38.91373315382284 - type: main_score value: 39.6704377766847 task: type: Classification - dataset: config: nl name: MTEB MassiveIntentClassification (nl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 38.170191834727 - type: f1 value: 34.725273063201136 - type: f1_weighted value: 37.095919315908134 - type: main_score value: 38.170191834727 task: type: Classification - dataset: config: te name: MTEB MassiveIntentClassification (te) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 2.247909493359567 - type: f1 value: 0.8785673795123102 - type: f1_weighted value: 1.470066565713873 - type: main_score value: 2.247909493359567 task: type: Classification - dataset: config: km name: MTEB MassiveIntentClassification (km) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 4.505656665027054 - type: f1 value: 1.4939433603970276 - type: f1_weighted value: 1.8350229640902833 - type: main_score value: 4.505656665027054 task: type: Classification - dataset: config: my name: MTEB MassiveIntentClassification (my) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 4.485981308411215 - type: f1 value: 1.5284494127111021 - type: f1_weighted value: 2.272480025721987 - type: main_score value: 4.485981308411215 task: type: Classification - dataset: config: ko name: MTEB MassiveIntentClassification (ko) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 16.089522872602068 - type: f1 value: 14.43473898813613 - type: f1_weighted value: 15.329305231025254 - type: main_score value: 16.089522872602068 task: type: Classification - dataset: config: lv name: MTEB MassiveIntentClassification (lv) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 36.227250368912934 - type: f1 value: 34.00381760992141 - type: f1_weighted value: 35.552698832679816 - type: main_score value: 36.227250368912934 task: type: Classification - dataset: config: ro name: MTEB MassiveIntentClassification (ro) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 39.296606000983765 - type: f1 value: 36.25940749915232 - type: f1_weighted value: 38.20146669419212 - type: main_score value: 39.296606000983765 task: type: Classification - dataset: config: ta name: MTEB MassiveIntentClassification (ta) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 11.43138219380226 - type: f1 value: 8.015827788153656 - type: f1_weighted value: 8.778813149533917 - type: main_score value: 11.43138219380226 task: type: Classification - dataset: config: ja name: MTEB MassiveIntentClassification (ja) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 30.52139695031972 - type: f1 value: 26.287093284533835 - type: f1_weighted value: 30.035549978599217 - type: main_score value: 30.52139695031972 task: type: Classification - dataset: config: en name: MTEB MassiveIntentClassification (en) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 67.93900639449089 - type: f1 value: 63.595691964600896 - type: f1_weighted value: 66.60062895081047 - type: main_score value: 67.93900639449089 task: type: Classification - dataset: config: it name: MTEB MassiveIntentClassification (it) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 40.84112149532711 - type: f1 value: 39.156304457716494 - type: f1_weighted value: 39.012909931390624 - type: main_score value: 40.84112149532711 task: type: Classification - dataset: config: fr name: MTEB MassiveIntentClassification (fr) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 41.94786030496802 - type: f1 value: 38.495265937479395 - type: f1_weighted value: 40.444292818958026 - type: main_score value: 41.94786030496802 task: type: Classification - dataset: config: az name: MTEB MassiveIntentClassification (az) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 28.74077717658633 - type: f1 value: 28.2493939628517 - type: f1_weighted value: 28.748153549609935 - type: main_score value: 28.74077717658633 task: type: Classification - dataset: config: tr name: MTEB MassiveIntentClassification (tr) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 33.09394982784063 - type: f1 value: 30.529832436386812 - type: f1_weighted value: 32.35594077483882 - type: main_score value: 33.09394982784063 task: type: Classification - dataset: config: vi name: MTEB MassiveIntentClassification (vi) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 37.363502213477624 - type: f1 value: 34.523304417413 - type: f1_weighted value: 35.4545002559176 - type: main_score value: 37.363502213477624 task: type: Classification - dataset: config: pl name: MTEB MassiveIntentClassification (pl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 34.96802754549926 - type: f1 value: 30.92574226486116 - type: f1_weighted value: 34.22995777308236 - type: main_score value: 34.96802754549926 task: type: Classification - dataset: config: zh-CN name: MTEB MassiveIntentClassification (zh-CN) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 25.91244466305952 - type: f1 value: 20.296802475937273 - type: f1_weighted value: 24.0961502305895 - type: main_score value: 25.91244466305952 task: type: Classification - dataset: config: am name: MTEB MassiveIntentClassification (am) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 2.4446630595179535 - type: f1 value: 1.014404183852614 - type: f1_weighted value: 1.1197496207269353 - type: main_score value: 2.4446630595179535 task: type: Classification - dataset: config: pt name: MTEB MassiveIntentClassification (pt) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 42.59714707329071 - type: f1 value: 40.90151347161461 - type: f1_weighted value: 41.328070472204516 - type: main_score value: 42.59714707329071 task: type: Classification - dataset: config: el name: MTEB MassiveIntentClassification (el) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 22.49877029021151 - type: f1 value: 19.510859895669057 - type: f1_weighted value: 21.319652891287575 - type: main_score value: 22.49877029021151 task: type: Classification - dataset: config: hy name: MTEB MassiveIntentClassification (hy) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 7.132316773241515 - type: f1 value: 5.463897630896959 - type: f1_weighted value: 6.112087831194722 - type: main_score value: 7.132316773241515 task: type: Classification - dataset: config: es name: MTEB MassiveIntentClassification (es) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 40.6345302508608 - type: f1 value: 38.90613233845256 - type: f1_weighted value: 38.89331878659355 - type: main_score value: 40.6345302508608 task: type: Classification - dataset: config: sv name: MTEB MassiveScenarioClassification (sv) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 42.94889038332212 - type: f1 value: 39.92279896396741 - type: f1_weighted value: 42.81501645437316 - type: main_score value: 42.94889038332212 task: type: Classification - dataset: config: tl name: MTEB MassiveScenarioClassification (tl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 45.689307330195014 - type: f1 value: 42.28756767910468 - type: f1_weighted value: 44.75170639258308 - type: main_score value: 45.689307330195014 task: type: Classification - dataset: config: mn name: MTEB MassiveScenarioClassification (mn) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 25.47074646940148 - type: f1 value: 23.552199678741083 - type: f1_weighted value: 24.505334719658176 - type: main_score value: 25.47074646940148 task: type: Classification - dataset: config: bn name: MTEB MassiveScenarioClassification (bn) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 20.5581708137189 - type: f1 value: 17.429451637349384 - type: f1_weighted value: 18.791290411838688 - type: main_score value: 20.5581708137189 task: type: Classification - dataset: config: nb name: MTEB MassiveScenarioClassification (nb) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 44.66711499663752 - type: f1 value: 42.18130687898006 - type: f1_weighted value: 44.699898238606934 - type: main_score value: 44.66711499663752 task: type: Classification - dataset: config: kn name: MTEB MassiveScenarioClassification (kn) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.8480161398789505 - type: f1 value: 5.10837206129005 - type: f1_weighted value: 5.442449997785076 - type: main_score value: 7.8480161398789505 task: type: Classification - dataset: config: ur name: MTEB MassiveScenarioClassification (ur) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 23.725622057834567 - type: f1 value: 20.827908973901494 - type: f1_weighted value: 23.138568493821328 - type: main_score value: 23.725622057834567 task: type: Classification - dataset: config: id name: MTEB MassiveScenarioClassification (id) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.45662407531943 - type: f1 value: 41.28033604461074 - type: f1_weighted value: 43.501973363018244 - type: main_score value: 43.45662407531943 task: type: Classification - dataset: config: hu name: MTEB MassiveScenarioClassification (hu) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 41.61062542030935 - type: f1 value: 38.43297104331772 - type: f1_weighted value: 40.7669700275472 - type: main_score value: 41.61062542030935 task: type: Classification - dataset: config: sw name: MTEB MassiveScenarioClassification (sw) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.322125084061874 - type: f1 value: 39.6211632212371 - type: f1_weighted value: 42.367281241374585 - type: main_score value: 43.322125084061874 task: type: Classification - dataset: config: af name: MTEB MassiveScenarioClassification (af) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.87020847343644 - type: f1 value: 41.427117150156676 - type: f1_weighted value: 43.879857280798575 - type: main_score value: 43.87020847343644 task: type: Classification - dataset: config: fa name: MTEB MassiveScenarioClassification (fa) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 23.96772024209818 - type: f1 value: 21.123340579548803 - type: f1_weighted value: 22.459078400625554 - type: main_score value: 23.96772024209818 task: type: Classification - dataset: config: ka name: MTEB MassiveScenarioClassification (ka) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 14.922663080026902 - type: f1 value: 12.625046398705834 - type: f1_weighted value: 13.208082038502415 - type: main_score value: 14.922663080026902 task: type: Classification - dataset: config: fi name: MTEB MassiveScenarioClassification (fi) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 42.38399462004034 - type: f1 value: 38.751110741837934 - type: f1_weighted value: 42.35679472298691 - type: main_score value: 42.38399462004034 task: type: Classification - dataset: config: de name: MTEB MassiveScenarioClassification (de) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 51.46940147948891 - type: f1 value: 48.161468971752576 - type: f1_weighted value: 50.70367870119378 - type: main_score value: 51.46940147948891 task: type: Classification - dataset: config: cy name: MTEB MassiveScenarioClassification (cy) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 39.004707464694015 - type: f1 value: 35.53806288814433 - type: f1_weighted value: 38.65014652888815 - type: main_score value: 39.004707464694015 task: type: Classification - dataset: config: ar name: MTEB MassiveScenarioClassification (ar) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 25.98520511096167 - type: f1 value: 24.385748402000768 - type: f1_weighted value: 26.072306868085604 - type: main_score value: 25.98520511096167 task: type: Classification - dataset: config: sl name: MTEB MassiveScenarioClassification (sl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 41.90316072629455 - type: f1 value: 39.6868264721523 - type: f1_weighted value: 41.22071520745331 - type: main_score value: 41.90316072629455 task: type: Classification - dataset: config: ru name: MTEB MassiveScenarioClassification (ru) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 30.464021519838596 - type: f1 value: 28.646951236504954 - type: f1_weighted value: 28.83842973933925 - type: main_score value: 30.464021519838596 task: type: Classification - dataset: config: is name: MTEB MassiveScenarioClassification (is) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 36.11970410221923 - type: f1 value: 34.342908333441194 - type: f1_weighted value: 35.518852396328384 - type: main_score value: 36.11970410221923 task: type: Classification - dataset: config: sq name: MTEB MassiveScenarioClassification (sq) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 47.20578345662407 - type: f1 value: 44.49009211178284 - type: f1_weighted value: 46.95455303245114 - type: main_score value: 47.20578345662407 task: type: Classification - dataset: config: hi name: MTEB MassiveScenarioClassification (hi) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 23.708809683927374 - type: f1 value: 21.74578684210376 - type: f1_weighted value: 22.473324333506838 - type: main_score value: 23.708809683927374 task: type: Classification - dataset: config: he name: MTEB MassiveScenarioClassification (he) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 24.008069939475448 - type: f1 value: 22.220630738064763 - type: f1_weighted value: 23.500947443709617 - type: main_score value: 24.008069939475448 task: type: Classification - dataset: config: jv name: MTEB MassiveScenarioClassification (jv) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.58776059179556 - type: f1 value: 40.12797109898581 - type: f1_weighted value: 43.09644178069442 - type: main_score value: 43.58776059179556 task: type: Classification - dataset: config: ml name: MTEB MassiveScenarioClassification (ml) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.673167451244114 - type: f1 value: 3.511854125479099 - type: f1_weighted value: 4.265638725280987 - type: main_score value: 7.673167451244114 task: type: Classification - dataset: config: ms name: MTEB MassiveScenarioClassification (ms) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.67182246133153 - type: f1 value: 39.655809226546545 - type: f1_weighted value: 43.29137560706826 - type: main_score value: 43.67182246133153 task: type: Classification - dataset: config: th name: MTEB MassiveScenarioClassification (th) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 19.502353732347004 - type: f1 value: 14.334939233252669 - type: f1_weighted value: 16.37184741839454 - type: main_score value: 19.502353732347004 task: type: Classification - dataset: config: zh-TW name: MTEB MassiveScenarioClassification (zh-TW) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 31.176866173503697 - type: f1 value: 30.619127089356617 - type: f1_weighted value: 29.056975568522585 - type: main_score value: 31.176866173503697 task: type: Classification - dataset: config: da name: MTEB MassiveScenarioClassification (da) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 47.017484868863484 - type: f1 value: 44.62052303129514 - type: f1_weighted value: 46.91154914611081 - type: main_score value: 47.017484868863484 task: type: Classification - dataset: config: nl name: MTEB MassiveScenarioClassification (nl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 48.42972427706792 - type: f1 value: 45.12440632448532 - type: f1_weighted value: 48.25021698338815 - type: main_score value: 48.42972427706792 task: type: Classification - dataset: config: te name: MTEB MassiveScenarioClassification (te) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.948890383322126 - type: f1 value: 3.9403181349064584 - type: f1_weighted value: 4.705877373977869 - type: main_score value: 7.948890383322126 task: type: Classification - dataset: config: km name: MTEB MassiveScenarioClassification (km) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 9.246805648957633 - type: f1 value: 4.129891064498778 - type: f1_weighted value: 5.602493897517791 - type: main_score value: 9.246805648957633 task: type: Classification - dataset: config: my name: MTEB MassiveScenarioClassification (my) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 10.605245460659045 - type: f1 value: 5.809876826200103 - type: f1_weighted value: 6.253432663208695 - type: main_score value: 10.605245460659045 task: type: Classification - dataset: config: ko name: MTEB MassiveScenarioClassification (ko) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 20.299260255548084 - type: f1 value: 18.44287875551232 - type: f1_weighted value: 19.12844812544342 - type: main_score value: 20.299260255548084 task: type: Classification - dataset: config: lv name: MTEB MassiveScenarioClassification (lv) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 40.43039677202421 - type: f1 value: 38.92467028982438 - type: f1_weighted value: 39.42730020848229 - type: main_score value: 40.43039677202421 task: type: Classification - dataset: config: ro name: MTEB MassiveScenarioClassification (ro) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 48.23133826496301 - type: f1 value: 45.4324412715274 - type: f1_weighted value: 47.97005141245953 - type: main_score value: 48.23133826496301 task: type: Classification - dataset: config: ta name: MTEB MassiveScenarioClassification (ta) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 17.370544720914594 - type: f1 value: 14.578479484069462 - type: f1_weighted value: 14.448203779557739 - type: main_score value: 17.370544720914594 task: type: Classification - dataset: config: ja name: MTEB MassiveScenarioClassification (ja) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 37.299932750504375 - type: f1 value: 35.808981859994304 - type: f1_weighted value: 36.207622383681745 - type: main_score value: 37.299932750504375 task: type: Classification - dataset: config: en name: MTEB MassiveScenarioClassification (en) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 73.80968392737056 - type: f1 value: 73.47140983203266 - type: f1_weighted value: 73.82205293897218 - type: main_score value: 73.80968392737056 task: type: Classification - dataset: config: it name: MTEB MassiveScenarioClassification (it) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 49.8016139878951 - type: f1 value: 46.14008662393035 - type: f1_weighted value: 49.17675388568613 - type: main_score value: 49.8016139878951 task: type: Classification - dataset: config: fr name: MTEB MassiveScenarioClassification (fr) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 51.14324142568931 - type: f1 value: 48.85145683203006 - type: f1_weighted value: 50.37835045676896 - type: main_score value: 51.14324142568931 task: type: Classification - dataset: config: az name: MTEB MassiveScenarioClassification (az) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 35.58507061197041 - type: f1 value: 33.542237010785854 - type: f1_weighted value: 34.93497874727777 - type: main_score value: 35.58507061197041 task: type: Classification - dataset: config: tr name: MTEB MassiveScenarioClassification (tr) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 38.853396099529256 - type: f1 value: 38.42403488960939 - type: f1_weighted value: 37.959951965899265 - type: main_score value: 38.853396099529256 task: type: Classification - dataset: config: vi name: MTEB MassiveScenarioClassification (vi) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 40.474108944182916 - type: f1 value: 38.470544281299496 - type: f1_weighted value: 39.664332190167286 - type: main_score value: 40.474108944182916 task: type: Classification - dataset: config: pl name: MTEB MassiveScenarioClassification (pl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.8231338264963 - type: f1 value: 41.3296940713539 - type: f1_weighted value: 43.88699524977039 - type: main_score value: 43.8231338264963 task: type: Classification - dataset: config: zh-CN name: MTEB MassiveScenarioClassification (zh-CN) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 33.648285137861464 - type: f1 value: 31.47898170038156 - type: f1_weighted value: 32.81256195033526 - type: main_score value: 33.648285137861464 task: type: Classification - dataset: config: am name: MTEB MassiveScenarioClassification (am) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.572293207800941 - type: f1 value: 3.683258159199767 - type: f1_weighted value: 3.966547308224644 - type: main_score value: 7.572293207800941 task: type: Classification - dataset: config: pt name: MTEB MassiveScenarioClassification (pt) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 50.716207128446534 - type: f1 value: 48.42212962506996 - type: f1_weighted value: 49.989471189982055 - type: main_score value: 50.716207128446534 task: type: Classification - dataset: config: el name: MTEB MassiveScenarioClassification (el) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 31.297915265635506 - type: f1 value: 26.951282088939394 - type: f1_weighted value: 31.737054842062918 - type: main_score value: 31.297915265635506 task: type: Classification - dataset: config: hy name: MTEB MassiveScenarioClassification (hy) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 13.026227303295226 - type: f1 value: 11.207195434897372 - type: f1_weighted value: 11.85104323420264 - type: main_score value: 13.026227303295226 task: type: Classification - dataset: config: es name: MTEB MassiveScenarioClassification (es) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 48.99798251513114 - type: f1 value: 46.34397271008467 - type: f1_weighted value: 48.626317696475084 - type: main_score value: 48.99798251513114 task: type: Classification - dataset: config: sv name: MTEB MassiveScenarioClassification (sv) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 42.91687161829808 - type: f1 value: 40.834447863798964 - type: f1_weighted value: 42.49909938470801 - type: main_score value: 42.91687161829808 task: type: Classification - dataset: config: tl name: MTEB MassiveScenarioClassification (tl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 46.35022134776193 - type: f1 value: 43.87163502391931 - type: f1_weighted value: 45.21092209683288 - type: main_score value: 46.35022134776193 task: type: Classification - dataset: config: mn name: MTEB MassiveScenarioClassification (mn) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 24.146581406787995 - type: f1 value: 22.771605688136933 - type: f1_weighted value: 22.88849491826262 - type: main_score value: 24.146581406787995 task: type: Classification - dataset: config: bn name: MTEB MassiveScenarioClassification (bn) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 19.758976881455975 - type: f1 value: 16.426836405449265 - type: f1_weighted value: 17.9501743960883 - type: main_score value: 19.758976881455975 task: type: Classification - dataset: config: nb name: MTEB MassiveScenarioClassification (nb) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.39399901623217 - type: f1 value: 42.08575847291634 - type: f1_weighted value: 42.985579530310545 - type: main_score value: 43.39399901623217 task: type: Classification - dataset: config: kn name: MTEB MassiveScenarioClassification (kn) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 8.352188883423512 - type: f1 value: 5.4319870750035655 - type: f1_weighted value: 5.76179219962643 - type: main_score value: 8.352188883423512 task: type: Classification - dataset: config: ur name: MTEB MassiveScenarioClassification (ur) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 23.330054107230694 - type: f1 value: 20.76106092272521 - type: f1_weighted value: 22.500458051747803 - type: main_score value: 23.330054107230694 task: type: Classification - dataset: config: id name: MTEB MassiveScenarioClassification (id) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.03492375799311 - type: f1 value: 42.04623013147629 - type: f1_weighted value: 42.714825173227844 - type: main_score value: 43.03492375799311 task: type: Classification - dataset: config: hu name: MTEB MassiveScenarioClassification (hu) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 39.28184948352189 - type: f1 value: 37.28207272023906 - type: f1_weighted value: 38.66596196857041 - type: main_score value: 39.28184948352189 task: type: Classification - dataset: config: sw name: MTEB MassiveScenarioClassification (sw) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 41.83472700442695 - type: f1 value: 38.437121211879344 - type: f1_weighted value: 40.81552197018151 - type: main_score value: 41.83472700442695 task: type: Classification - dataset: config: af name: MTEB MassiveScenarioClassification (af) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 43.55632070831284 - type: f1 value: 41.9077592573678 - type: f1_weighted value: 43.19055309721229 - type: main_score value: 43.55632070831284 task: type: Classification - dataset: config: fa name: MTEB MassiveScenarioClassification (fa) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 22.911952779144123 - type: f1 value: 20.577786095120953 - type: f1_weighted value: 21.398823308631364 - type: main_score value: 22.911952779144123 task: type: Classification - dataset: config: ka name: MTEB MassiveScenarioClassification (ka) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 14.13182488932612 - type: f1 value: 11.966053928003031 - type: f1_weighted value: 12.270142836001305 - type: main_score value: 14.13182488932612 task: type: Classification - dataset: config: fi name: MTEB MassiveScenarioClassification (fi) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 40.718150516478104 - type: f1 value: 38.17397699913776 - type: f1_weighted value: 40.52057648170645 - type: main_score value: 40.718150516478104 task: type: Classification - dataset: config: de name: MTEB MassiveScenarioClassification (de) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 50.0491883915396 - type: f1 value: 47.457452418209215 - type: f1_weighted value: 49.29879443565884 - type: main_score value: 50.0491883915396 task: type: Classification - dataset: config: cy name: MTEB MassiveScenarioClassification (cy) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 38.77520905066404 - type: f1 value: 36.306659367248585 - type: f1_weighted value: 38.18076670087933 - type: main_score value: 38.77520905066404 task: type: Classification - dataset: config: ar name: MTEB MassiveScenarioClassification (ar) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 24.372848007870147 - type: f1 value: 23.375574019717764 - type: f1_weighted value: 24.450726012967433 - type: main_score value: 24.372848007870147 task: type: Classification - dataset: config: sl name: MTEB MassiveScenarioClassification (sl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 41.41662567634039 - type: f1 value: 39.927941364142164 - type: f1_weighted value: 40.707199435222265 - type: main_score value: 41.41662567634039 task: type: Classification - dataset: config: ru name: MTEB MassiveScenarioClassification (ru) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 29.950811608460405 - type: f1 value: 28.09393050390538 - type: f1_weighted value: 27.96901633099781 - type: main_score value: 29.950811608460405 task: type: Classification - dataset: config: is name: MTEB MassiveScenarioClassification (is) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 35.194294146581406 - type: f1 value: 34.500540449666644 - type: f1_weighted value: 34.62959263582815 - type: main_score value: 35.194294146581406 task: type: Classification - dataset: config: sq name: MTEB MassiveScenarioClassification (sq) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 45.489424495818994 - type: f1 value: 44.133464380702 - type: f1_weighted value: 45.307039276499495 - type: main_score value: 45.489424495818994 task: type: Classification - dataset: config: hi name: MTEB MassiveScenarioClassification (hi) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 23.148057058534185 - type: f1 value: 21.028366501452783 - type: f1_weighted value: 21.75365289495613 - type: main_score value: 23.148057058534185 task: type: Classification - dataset: config: he name: MTEB MassiveScenarioClassification (he) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 23.92031480570585 - type: f1 value: 22.631072481214698 - type: f1_weighted value: 23.317461061943888 - type: main_score value: 23.92031480570585 task: type: Classification - dataset: config: jv name: MTEB MassiveScenarioClassification (jv) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 42.818494835218885 - type: f1 value: 40.65508383583419 - type: f1_weighted value: 42.03672746056766 - type: main_score value: 42.818494835218885 task: type: Classification - dataset: config: ml name: MTEB MassiveScenarioClassification (ml) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.481554353172652 - type: f1 value: 3.2851114662893375 - type: f1_weighted value: 4.060224724153365 - type: main_score value: 7.481554353172652 task: type: Classification - dataset: config: ms name: MTEB MassiveScenarioClassification (ms) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 42.3905558288244 - type: f1 value: 39.349177885755594 - type: f1_weighted value: 41.921424413948465 - type: main_score value: 42.3905558288244 task: type: Classification - dataset: config: th name: MTEB MassiveScenarioClassification (th) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 18.844072798819482 - type: f1 value: 14.460068928964509 - type: f1_weighted value: 15.901333931287981 - type: main_score value: 18.844072798819482 task: type: Classification - dataset: config: zh-TW name: MTEB MassiveScenarioClassification (zh-TW) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 33.266109198229216 - type: f1 value: 32.52632049354786 - type: f1_weighted value: 30.807752942648815 - type: main_score value: 33.266109198229216 task: type: Classification - dataset: config: da name: MTEB MassiveScenarioClassification (da) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 44.67781603541564 - type: f1 value: 43.960109885386714 - type: f1_weighted value: 44.664994077212654 - type: main_score value: 44.67781603541564 task: type: Classification - dataset: config: nl name: MTEB MassiveScenarioClassification (nl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 46.52238071815052 - type: f1 value: 44.30085571629839 - type: f1_weighted value: 46.05732031190093 - type: main_score value: 46.52238071815052 task: type: Classification - dataset: config: te name: MTEB MassiveScenarioClassification (te) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.575012297097886 - type: f1 value: 3.8269769496067125 - type: f1_weighted value: 4.3790932390435096 - type: main_score value: 7.575012297097886 task: type: Classification - dataset: config: km name: MTEB MassiveScenarioClassification (km) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 8.839153959665518 - type: f1 value: 4.053025964209311 - type: f1_weighted value: 5.367721221788085 - type: main_score value: 8.839153959665518 task: type: Classification - dataset: config: my name: MTEB MassiveScenarioClassification (my) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 11.185440236104279 - type: f1 value: 5.98907179062831 - type: f1_weighted value: 6.3865116891207645 - type: main_score value: 11.185440236104279 task: type: Classification - dataset: config: ko name: MTEB MassiveScenarioClassification (ko) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 20.59026069847516 - type: f1 value: 18.62087210756979 - type: f1_weighted value: 18.954901563434518 - type: main_score value: 20.59026069847516 task: type: Classification - dataset: config: lv name: MTEB MassiveScenarioClassification (lv) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 40.16232169208067 - type: f1 value: 39.110919025814816 - type: f1_weighted value: 39.15526244690224 - type: main_score value: 40.16232169208067 task: type: Classification - dataset: config: ro name: MTEB MassiveScenarioClassification (ro) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 47.62420068863748 - type: f1 value: 45.701219028475535 - type: f1_weighted value: 47.081144935159855 - type: main_score value: 47.62420068863748 task: type: Classification - dataset: config: ta name: MTEB MassiveScenarioClassification (ta) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 17.57993113625184 - type: f1 value: 14.397162397916391 - type: f1_weighted value: 14.187973554808195 - type: main_score value: 17.57993113625184 task: type: Classification - dataset: config: ja name: MTEB MassiveScenarioClassification (ja) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 37.63895720609936 - type: f1 value: 36.431466368006234 - type: f1_weighted value: 36.15231062631648 - type: main_score value: 37.63895720609936 task: type: Classification - dataset: config: en name: MTEB MassiveScenarioClassification (en) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 73.91047712739794 - type: f1 value: 73.42316335755856 - type: f1_weighted value: 73.976091221308 - type: main_score value: 73.91047712739794 task: type: Classification - dataset: config: it name: MTEB MassiveScenarioClassification (it) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 49.37038858829316 - type: f1 value: 46.91139679497153 - type: f1_weighted value: 48.54309440848084 - type: main_score value: 49.37038858829316 task: type: Classification - dataset: config: fr name: MTEB MassiveScenarioClassification (fr) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 50.59026069847516 - type: f1 value: 48.97620841672786 - type: f1_weighted value: 49.716628221739406 - type: main_score value: 50.59026069847516 task: type: Classification - dataset: config: az name: MTEB MassiveScenarioClassification (az) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 35.12051155927201 - type: f1 value: 33.78932792402176 - type: f1_weighted value: 34.49013486890936 - type: main_score value: 35.12051155927201 task: type: Classification - dataset: config: tr name: MTEB MassiveScenarioClassification (tr) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 38.1947860304968 - type: f1 value: 38.11966822206283 - type: f1_weighted value: 37.374415624640065 - type: main_score value: 38.1947860304968 task: type: Classification - dataset: config: vi name: MTEB MassiveScenarioClassification (vi) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 40.8903098868667 - type: f1 value: 39.28164808311247 - type: f1_weighted value: 39.886451969691336 - type: main_score value: 40.8903098868667 task: type: Classification - dataset: config: pl name: MTEB MassiveScenarioClassification (pl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 42.50368912936547 - type: f1 value: 41.20560095847174 - type: f1_weighted value: 42.54084069975687 - type: main_score value: 42.50368912936547 task: type: Classification - dataset: config: zh-CN name: MTEB MassiveScenarioClassification (zh-CN) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 35.15986227250368 - type: f1 value: 32.968890810625226 - type: f1_weighted value: 34.10229959264426 - type: main_score value: 35.15986227250368 task: type: Classification - dataset: config: am name: MTEB MassiveScenarioClassification (am) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 7.373339891785538 - type: f1 value: 3.5650569721440686 - type: f1_weighted value: 3.917440311279737 - type: main_score value: 7.373339891785538 task: type: Classification - dataset: config: pt name: MTEB MassiveScenarioClassification (pt) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 49.51303492375799 - type: f1 value: 48.380650227868685 - type: f1_weighted value: 48.39701204701856 - type: main_score value: 49.51303492375799 task: type: Classification - dataset: config: el name: MTEB MassiveScenarioClassification (el) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 29.29168716182981 - type: f1 value: 26.485015790639988 - type: f1_weighted value: 29.506958764764725 - type: main_score value: 29.29168716182981 task: type: Classification - dataset: config: hy name: MTEB MassiveScenarioClassification (hy) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 12.36104279390064 - type: f1 value: 10.471347998073597 - type: f1_weighted value: 10.934154601387776 - type: main_score value: 12.36104279390064 task: type: Classification - dataset: config: es name: MTEB MassiveScenarioClassification (es) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 49.30152484013773 - type: f1 value: 47.483089482534794 - type: f1_weighted value: 48.737862819379586 - type: main_score value: 49.30152484013773 task: type: Classification - dataset: config: default name: MTEB MedrxivClusteringP2P (default) revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 split: test type: mteb/medrxiv-clustering-p2p metrics: - type: main_score value: 34.38933185926366 - type: v_measure value: 34.38933185926366 - type: v_measure_std value: 1.5084000520185472 task: type: Clustering - dataset: config: default name: MTEB MedrxivClusteringS2S (default) revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 split: test type: mteb/medrxiv-clustering-s2s metrics: - type: main_score value: 31.86351621903201 - type: v_measure value: 31.86351621903201 - type: v_measure_std value: 1.7336971951213678 task: type: Clustering - dataset: config: default name: MTEB RedditClustering (default) revision: 24640382cdbf8abc73003fb0fa6d111a705499eb split: test type: mteb/reddit-clustering metrics: - type: main_score value: 50.68847759621761 - type: v_measure value: 50.68847759621761 - type: v_measure_std value: 4.599084917172616 task: type: Clustering - dataset: config: default name: MTEB RedditClusteringP2P (default) revision: 385e3cb46b4cfa89021f56c4380204149d0efe33 split: test type: mteb/reddit-clustering-p2p metrics: - type: main_score value: 54.798454195773715 - type: v_measure value: 54.798454195773715 - type: v_measure_std value: 11.285422691786078 task: type: Clustering - dataset: config: default name: MTEB StackExchangeClustering (default) revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 split: test type: mteb/stackexchange-clustering metrics: - type: main_score value: 53.13685840246208 - type: v_measure value: 53.13685840246208 - type: v_measure_std value: 5.607312786136468 task: type: Clustering - dataset: config: default name: MTEB StackExchangeClusteringP2P (default) revision: 815ca46b2622cec33ccafc3735d572c266efdb44 split: test type: mteb/stackexchange-clustering-p2p metrics: - type: main_score value: 34.248876064067204 - type: v_measure value: 34.248876064067204 - type: v_measure_std value: 1.5582893002039717 task: type: Clustering - dataset: config: default name: MTEB ToxicConversationsClassification (default) revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de split: test type: mteb/toxic_conversations_50k metrics: - type: accuracy value: 62.0947265625 - type: ap value: 10.421906368823997 - type: ap_weighted value: 10.421906368823997 - type: f1 value: 47.46373293445412 - type: f1_weighted value: 70.41497186754889 - type: main_score value: 62.0947265625 task: type: Classification - dataset: config: default name: MTEB TweetSentimentExtractionClassification (default) revision: d604517c81ca91fe16a244d1248fc021f9ecee7a split: test type: mteb/tweet_sentiment_extraction metrics: - type: accuracy value: 54.04357668364459 - type: f1 value: 54.26674207276355 - type: f1_weighted value: 53.57672500028157 - type: main_score value: 54.04357668364459 task: type: Classification - dataset: config: default name: MTEB TwentyNewsgroupsClustering (default) revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 split: test type: mteb/twentynewsgroups-clustering metrics: - type: main_score value: 46.49021100388057 - type: v_measure value: 46.49021100388057 - type: v_measure_std value: 0.983303557519297 task: type: Clustering tags: - mteb - sentence-transformers - feature-extraction - sentence-similarity - transformers language: en license: apache-2.0 library_name: sentence-transformers pipeline_tag: sentence-similarity --- # SLX-v0.1 **SLX-v0.1** is an advanced model developed by [BRAHMAI Research](https://brahmai.in), specifically designed for mapping sentences and paragraphs into a 384-dimensional dense vector space. This model is ideal for tasks such as clustering, semantic search, and sentence similarity. ## Usage with Sentence-Transformers Using SLX-v0.1 is straightforward with the [Sentence-Transformers](https://www.SBERT.net) library. Follow the instructions below to get started: ### Installation ```bash pip install -U sentence-transformers ``` ### Example ```python from sentence_transformers import SentenceTransformer sentences = ["This is an example sentence", "Each sentence is converted"] model = SentenceTransformer('brahmairesearch/slx-v0.1') embeddings = model.encode(sentences) print(embeddings) ``` ## Usage with Hugging Face Transformers If you prefer to use Hugging Face Transformers without the Sentence-Transformers library, you can still utilize the SLX-v0.1 model. Below is a guide on how to process input and apply mean pooling to obtain sentence embeddings: ```python from transformers import AutoTokenizer, AutoModel import torch import torch.nn.functional as F # Mean Pooling - Takes attention mask into account for correct averaging def mean_pooling(model_output, attention_mask): token_embeddings = model_output[0] # Token embeddings from the model output input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float() return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9) # Sentences for embedding sentences = ['This is an example sentence', 'Each sentence is converted'] # Load model from Hugging Face Hub tokenizer = AutoTokenizer.from_pretrained('brahmairesearch/slx-v0.1') model = AutoModel.from_pretrained('brahmairesearch/slx-v0.1') # Tokenize sentences encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt') # Compute token embeddings with torch.no_grad(): model_output = model(**encoded_input) # Perform pooling sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask']) # Normalize embeddings sentence_embeddings = F.normalize(sentence_embeddings, p=2, dim=1) print("Sentence embeddings:") print(sentence_embeddings) ``` ## Intended Use Cases SLX-v0.1 is designed for encoding sentences and short paragraphs into dense vectors that capture their semantic information. These embeddings can be used for: - Information retrieval - Clustering - Sentence similarity tasks Please note that input text longer than 256 word pieces will be truncated by default. ## Training Procedure ### Pre-training SLX-v0.1 is based on the pre-trained model [`nreimers/MiniLM-L6-H384-uncased`](https://huggingface.co/nreimers/MiniLM-L6-H384-uncased). For more information on the pre-training process, please refer to the original model card. ### Fine-tuning The model was fine-tuned using a contrastive learning objective. Cosine similarity is computed between all possible sentence pairs within a batch, and cross-entropy loss is applied using the true pairs. Following fine-tuning, the model underwent transfer learning using the [`dunzhang/stella_en_400M_v5`](https://huggingface.co/dunzhang/stella_en_400M_v5) model with an internally curated dataset, optimizing it for English language tasks. ## Contact For inquiries or feedback, feel free to reach out to us at hello@brahmai.in. Best regards, The [BRAHMAI](https://brahmai.in) Team