metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:50000
  - loss:CachedGISTEmbedLoss
base_model: microsoft/mpnet-base
widget:
  - source_sentence: what does the accounts receivable turnover measure?
    sentences:
      - >-
        The accounts receivable turnover ratio is an accounting measure used to
        quantify a company's effectiveness in collecting its receivables or
        money owed by clients. The ratio shows how well a company uses and
        manages the credit it extends to customers and how quickly that
        short-term debt is collected or is paid.
      - >-
        Capital budgeting, and investment appraisal, is the planning process
        used to determine whether an organization's long term investments such
        as new machinery, replacement of machinery, new plants, new products,
        and research development projects are worth the funding of cash through
        the firm's capitalization structure ( ...
      - >-
        The accounts receivable turnover ratio is an accounting measure used to
        quantify a company's effectiveness in collecting its receivables or
        money owed by clients. The ratio shows how well a company uses and
        manages the credit it extends to customers and how quickly that
        short-term debt is collected or is paid.
  - source_sentence: does gabapentin cause liver problems?
    sentences:
      - >-
        Gabapentin has no appreciable liver metabolism, yet, suspected cases of
        gabapentin-induced hepatotoxicity have been reported. Per literature
        review, two cases of possible gabapentin-induced liver injury have been
        reported.
      - >-
        Strongholds are a type of story mission which only unlocks after enough
        progression through the game. There are three Stronghold's during the
        first section of progression through The Division 2. You'll need to
        complete the first two and have reached level 30 before being able to
        unlock the final Stronghold.
      - >-
        The most-common side effects attributed to Gabapentin include mild
        sedation, ataxia, and occasional diarrhea. Sedation can be minimized by
        tapering from a smaller starting dose to the desired dose. When treating
        seizures, it is ideal to wean off the drug to reduce the risk of
        withdrawal seizures.
  - source_sentence: how long should you wait to give blood after eating?
    sentences:
      - >-
        Until the bleeding has stopped it is natural to taste blood or to see
        traces of blood in your saliva. You may stop using gauze after the flow
        stops – usually around 8 hours after surgery.
      - >-
        Before donation The first and most important rule—never donate blood on
        an empty stomach. “Eat a wholesome meal about 2-3 hours before donating
        to keep your blood sugar stable," says Dr Chaturvedi. The timing of the
        meal is important too. You need to allow the food to be digested
        properly before the blood is drawn.
      - >-
        While grid computing involves virtualizing computing resources to store
        massive amounts of data, whereas cloud computing is where an application
        doesn't access resources directly, rather it accesses them through a
        service over the internet. ...
  - source_sentence: what is the difference between chicken francese and chicken marsala?
    sentences:
      - >-
        Chicken is the species name, equivalent to our “human.” Rooster is an
        adult male, equivalent to “man.” Hen is an adult female, equivalent to
        “woman.” Cockerel is a juvenile male, equivalent to “boy/young man.”
      - What is 99 kg in pounds? - 99 kg is equal to 218.26 pounds.
      - >-
        The difference between the two is for Francese, the chicken breast is
        first dipped in flour, then into a beaten egg mixture, before being
        cooked. For piccata, the chicken is first dipped in egg and then in
        flour. Both are then simmered in a lemony butter sauce, but the piccata
        sauce includes capers.”
  - source_sentence: what energy is released when coal is burned?
    sentences:
      - >-
        When coal is burned, it reacts with the oxygen in the air. This chemical
        reaction converts the stored solar energy into thermal energy, which is
        released as heat. But it also produces carbon dioxide and methane.
      - >-
        When coal is burned it releases a number of airborne toxins and
        pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides,
        particulates, and various other heavy metals.
      - >-
        Squad Building Challenges allow you to exchange sets of players for
        coins, packs, and special items in FUT 20. Each of these challenges come
        with specific requirements, such as including players from certain
        teams. ... Live SBCs are time-limited challenges which often give out
        unique, high-rated versions of players.
datasets:
  - tomaarsen/gooaq-hard-negatives
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
co2_eq_emissions:
  emissions: 40.416471447949384
  energy_consumed: 0.10397803831199579
  source: codecarbon
  training_type: fine-tuning
  on_cloud: false
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
  ram_total_size: 31.777088165283203
  hours_used: 0.273
  hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
  - name: MPNet base trained on Natural Questions pairs
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoClimateFEVER
          type: NanoClimateFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.22
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.44
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.72
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.22
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666663
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11599999999999999
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.092
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.09333333333333332
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.195
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.22666666666666666
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.36733333333333335
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.27334708954752546
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.36268253968253966
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2030635178169122
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoDBPedia
          type: NanoDBPedia
        metrics:
          - type: cosine_accuracy@1
            value: 0.44
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.66
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.74
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.82
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.44
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.38666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.37200000000000005
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.34600000000000003
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03040608825384637
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.07615846857205867
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.11999750129731426
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.2193590803164296
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.388302838821125
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5594126984126983
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.278242819200185
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFEVER
          type: NanoFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.38
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.54
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.58
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.68
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.38
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.12
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.37
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.52
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.57
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.66
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5161962245159489
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.47610317460317453
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.47704035866094685
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFiQA2018
          type: NanoFiQA2018
        metrics:
          - type: cosine_accuracy@1
            value: 0.26
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.52
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.58
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.26
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.16
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09799999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.13433333333333333
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3226904761904762
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3653571428571428
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.43073809523809525
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.34083869249027804
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3756904761904761
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2830059503847294
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoHotpotQA
          type: NanoHotpotQA
        metrics:
          - type: cosine_accuracy@1
            value: 0.34
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.58
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.64
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.74
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.21333333333333332
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14400000000000002
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.094
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.17
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.32
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.36
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.47
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3823677764194786
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4719365079365079
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3071047037648779
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoMSMARCO
          type: NanoMSMARCO
        metrics:
          - type: cosine_accuracy@1
            value: 0.12
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.34
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.54
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.66
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.12
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.11333333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10800000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.066
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.12
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.34
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.54
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.66
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.36933631896924085
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.27802380952380945
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2903297758489702
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNFCorpus
          type: NanoNFCorpus
        metrics:
          - type: cosine_accuracy@1
            value: 0.32
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.42
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.44
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.48
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.32
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.188
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13799999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.012173283062756207
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.02074558886495681
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.026655271941004092
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.03744446828268134
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.16981476360614464
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3736904761904762
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.04845896733139879
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNQ
          type: NanoNQ
        metrics:
          - type: cosine_accuracy@1
            value: 0.16
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.36
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.46
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.56
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.16
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.11999999999999998
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.09200000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.05800000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.15
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.34
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.43
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.53
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.33694488555577967
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.28549206349206346
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2889544490538325
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoQuoraRetrieval
          type: NanoQuoraRetrieval
        metrics:
          - type: cosine_accuracy@1
            value: 0.82
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.92
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.96
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.82
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3733333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.244
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13399999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7206666666666667
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.862
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8993333333333333
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9566666666666666
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8834196907213419
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8638888888888888
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8576174787744555
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSCIDOCS
          type: NanoSCIDOCS
        metrics:
          - type: cosine_accuracy@1
            value: 0.34
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.48
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.54
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.66
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2533333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.21600000000000003
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.148
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07066666666666668
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.15766666666666668
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.22266666666666668
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.30566666666666664
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.2911964961614548
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4294920634920634
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.23186478719609563
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoArguAna
          type: NanoArguAna
        metrics:
          - type: cosine_accuracy@1
            value: 0.18
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.54
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.62
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.84
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.18
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.124
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08399999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.54
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.62
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.84
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.49499964917078587
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3864126984126984
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3939847733965381
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSciFact
          type: NanoSciFact
        metrics:
          - type: cosine_accuracy@1
            value: 0.36
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.46
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.48
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.36
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666663
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10400000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.066
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.325
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.44
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.46
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.585
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.45625099735000324
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4282142857142857
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.42683234725999136
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoTouche2020
          type: NanoTouche2020
        metrics:
          - type: cosine_accuracy@1
            value: 0.5306122448979592
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7346938775510204
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8367346938775511
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9591836734693877
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5306122448979592
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.45578231292517
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.4081632653061224
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.3489795918367347
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03881638827876476
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.10112189874472176
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.14360203271733188
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.2368499712298808
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.402135622609889
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6536767087787496
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3149629356234365
            name: Cosine Map@100
      - task:
          type: nano-beir
          name: Nano BEIR
        dataset:
          name: NanoBEIR mean
          type: NanoBEIR_mean
        metrics:
          - type: cosine_accuracy@1
            value: 0.3438932496075353
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5349764521193093
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.601287284144427
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7122448979591838
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3438932496075353
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.23454735740450025
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18432025117739403
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13407535321821037
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.1857996738150283
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3257986999260677
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3834060473445738
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.4845429447487502
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4080885419953074
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.45728587625526396
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.338574066485567
            name: Cosine Map@100

MPNet base trained on Natural Questions pairs

This is a sentence-transformers model finetuned from microsoft/mpnet-base on the gooaq-hard-negatives dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: microsoft/mpnet-base
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- gooaq-hard-negatives
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/mpnet-base-nq-cgist-triplet-mask")
# Run inference
sentences = [
    'what energy is released when coal is burned?',
    'When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane.',
    'When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Datasets: NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
Evaluated with InformationRetrievalEvaluator

Metric	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
cosine_accuracy@1	0.22	0.44	0.38	0.26	0.34	0.12	0.32	0.16	0.82	0.34	0.18	0.36	0.5306
cosine_accuracy@3	0.44	0.66	0.54	0.5	0.58	0.34	0.42	0.36	0.9	0.48	0.54	0.46	0.7347
cosine_accuracy@5	0.5	0.74	0.58	0.52	0.64	0.54	0.44	0.46	0.92	0.54	0.62	0.48	0.8367
cosine_accuracy@10	0.72	0.82	0.68	0.58	0.74	0.66	0.48	0.56	0.96	0.66	0.84	0.6	0.9592
cosine_precision@1	0.22	0.44	0.38	0.26	0.34	0.12	0.32	0.16	0.82	0.34	0.18	0.36	0.5306
cosine_precision@3	0.1667	0.3867	0.18	0.22	0.2133	0.1133	0.22	0.12	0.3733	0.2533	0.18	0.1667	0.4558
cosine_precision@5	0.116	0.372	0.12	0.16	0.144	0.108	0.188	0.092	0.244	0.216	0.124	0.104	0.4082
cosine_precision@10	0.092	0.346	0.07	0.098	0.094	0.066	0.138	0.058	0.134	0.148	0.084	0.066	0.349
cosine_recall@1	0.0933	0.0304	0.37	0.1343	0.17	0.12	0.0122	0.15	0.7207	0.0707	0.18	0.325	0.0388
cosine_recall@3	0.195	0.0762	0.52	0.3227	0.32	0.34	0.0207	0.34	0.862	0.1577	0.54	0.44	0.1011
cosine_recall@5	0.2267	0.12	0.57	0.3654	0.36	0.54	0.0267	0.43	0.8993	0.2227	0.62	0.46	0.1436
cosine_recall@10	0.3673	0.2194	0.66	0.4307	0.47	0.66	0.0374	0.53	0.9567	0.3057	0.84	0.585	0.2368
cosine_ndcg@10	0.2733	0.3883	0.5162	0.3408	0.3824	0.3693	0.1698	0.3369	0.8834	0.2912	0.495	0.4563	0.4021
cosine_mrr@10	0.3627	0.5594	0.4761	0.3757	0.4719	0.278	0.3737	0.2855	0.8639	0.4295	0.3864	0.4282	0.6537
cosine_map@100	0.2031	0.2782	0.477	0.283	0.3071	0.2903	0.0485	0.289	0.8576	0.2319	0.394	0.4268	0.315

Nano BEIR

Dataset: NanoBEIR_mean
Evaluated with NanoBEIREvaluator

Metric	Value
cosine_accuracy@1	0.3439
cosine_accuracy@3	0.535
cosine_accuracy@5	0.6013
cosine_accuracy@10	0.7122
cosine_precision@1	0.3439
cosine_precision@3	0.2345
cosine_precision@5	0.1843
cosine_precision@10	0.1341
cosine_recall@1	0.1858
cosine_recall@3	0.3258
cosine_recall@5	0.3834
cosine_recall@10	0.4845
cosine_ndcg@10	0.4081
cosine_mrr@10	0.4573
cosine_map@100	0.3386

Training Details

Training Dataset

gooaq-hard-negatives

Dataset: gooaq-hard-negatives at 87594a1
Size: 50,000 training samples
Columns: question, answer, and negative

Approximate statistics based on the first 1000 samples:

	question	answer	negative
type	string	string	string
details	min: 8 tokens mean: 11.53 tokens max: 28 tokens	min: 14 tokens mean: 59.79 tokens max: 150 tokens	min: 15 tokens mean: 58.76 tokens max: 143 tokens

Samples:

question	answer	negative
`what is the difference between calories from fat and total fat?`	`Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories.`	`Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories.`
`what is the difference between return transcript and account transcript?`	`A tax return transcript usually meets the needs of lending institutions offering mortgages and student loans. ... Tax Account Transcript - shows basic data such as return type, marital status, adjusted gross income, taxable income and all payment types. It also shows changes made after you filed your original return.`	`Trial balance is not a financial statement whereas a balance sheet is a financial statement. Trial balance is solely used for internal purposes whereas a balance sheet is used for purposes other than internal i.e. external. In a trial balance, each and every account is divided into debit (dr.) and credit (cr.)`
`how long does my dog need to fast before sedation?`	`Now, guidelines are aimed towards 6-8 hours before surgery. This pre-op fasting time is much more beneficial for your pets because you have enough food in there to neutralize the stomach acid, preventing it from coming up the esophagus that causes regurgitation under anesthetic.`	`Try not to let your pooch rapidly wolf down his/her food! Do not let the dog play or exercise (e.g. go for a walk) for at least two hours after having a meal. Ensure continuous fresh water is available to avoid your pet gulping down a large amount after eating.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.01}

Evaluation Dataset

gooaq-hard-negatives

Dataset: gooaq-hard-negatives at 87594a1
Size: 10,048,700 evaluation samples
Columns: question, answer, and negative

Approximate statistics based on the first 1000 samples:

	question	answer	negative
type	string	string	string
details	min: 8 tokens mean: 11.61 tokens max: 21 tokens	min: 16 tokens mean: 58.16 tokens max: 131 tokens	min: 14 tokens mean: 57.98 tokens max: 157 tokens

Samples:

question	answer	negative
`how is height width and length written?`	`The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width.`	`The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. That's important.`
`what is the difference between pork shoulder and loin?`	`All the recipes I've found for pulled pork recommends a shoulder/butt. Shoulders take longer to cook than a loin, because they're tougher. Loins are lean, while shoulders have marbled fat inside.`	`They are extracted from the loin, which runs from the hip to the shoulder, and it has a small strip of meat called the tenderloin. Unlike other pork, this pork chop is cut from four major sections, which are the shoulder, also known as the blade chops, ribs chops, loin chops, and the last, which is the sirloin chops.`
`is the yin yang symbol religious?`	`The ubiquitous yin-yang symbol holds its roots in Taoism/Daoism, a Chinese religion and philosophy. The yin, the dark swirl, is associated with shadows, femininity, and the trough of a wave; the yang, the light swirl, represents brightness, passion and growth.`	`Yin energy is in the calm colors around you, in the soft music, in the soothing sound of a water fountain, or the relaxing images of water. Yang (active energy) is the feng shui energy expressed in strong, vibrant sounds and colors, bright lights, upward moving energy, tall plants, etc.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.01}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 2048
per_device_eval_batch_size: 2048
learning_rate: 2e-05
num_train_epochs: 1
warmup_ratio: 0.1
seed: 12
bf16: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 2048
per_device_eval_batch_size: 2048
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 12
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoClimateFEVER_cosine_ndcg@10	NanoDBPedia_cosine_ndcg@10	NanoFEVER_cosine_ndcg@10	NanoFiQA2018_cosine_ndcg@10	NanoHotpotQA_cosine_ndcg@10	NanoMSMARCO_cosine_ndcg@10	NanoNFCorpus_cosine_ndcg@10	NanoNQ_cosine_ndcg@10	NanoQuoraRetrieval_cosine_ndcg@10	NanoSCIDOCS_cosine_ndcg@10	NanoArguAna_cosine_ndcg@10	NanoSciFact_cosine_ndcg@10	NanoTouche2020_cosine_ndcg@10	NanoBEIR_mean_cosine_ndcg@10
0.04	1	11.5143	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2	5	9.4399	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4	10	5.5951	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6	15	3.7416	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8	20	2.8021	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
1.0	25	2.2003	1.3332	0.2733	0.3883	0.5162	0.3408	0.3824	0.3693	0.1698	0.3369	0.8834	0.2912	0.4950	0.4563	0.4021	0.4081

Environmental Impact

Carbon emissions were measured using CodeCarbon.

Energy Consumed: 0.104 kWh
Carbon Emitted: 0.040 kg of CO2
Hours Used: 0.273 hours

Training Hardware

On Cloud: No
GPU Model: 1 x NVIDIA GeForce RTX 3090
CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
RAM Size: 31.78 GB

Framework Versions

Python: 3.11.6
Sentence Transformers: 3.5.0.dev0
Transformers: 4.49.0
PyTorch: 2.6.0+cu124
Accelerate: 1.5.1
Datasets: 3.3.2
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}