README.md

metadata

language:
  - de
  - en
  - es
  - fr
  - it
  - nl
  - pl
  - pt
  - ru
  - zh
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:51741
  - loss:CoSENTLoss
base_model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2
widget:
  - source_sentence: Starsza para azjatycka pozuje z noworodkiem przy stole obiadowym.
    sentences:
      - Koszykarz ma zamiar zdobyć punkty dla swojej drużyny.
      - Grupa starszych osób pozuje wokół stołu w jadalni.
      - Możliwe, że układ słoneczny taki jak nasz może istnieć poza galaktyką.
  - source_sentence: >-
      Englisch arbeitet überall mit Menschen, die Dinge kaufen und verkaufen,
      und in der Gastfreundschaft und im Tourismusgeschäft.
    sentences:
      - >-
        Ich bin in Maharashtra (einschließlich Mumbai) und Andhra Pradesh
        herumgereist, und ich hatte kein Problem damit, nur mit Englisch
        auszukommen.
      - >-
        Ein griechischsprachiger Sklave (δούλος, doulos) würde seinen Herrn,
        glaube ich, κύριος nennen (translit: kurios; Herr, Herr, Herr, Herr;
        Vokativform: κύριε).
      - Das Paar lag auf dem Bett.
  - source_sentence: >-
      Si vous vous comprenez et comprenez votre ennemi, vous aurez beaucoup plus
      de chances de gagner n'importe quelle bataille.
    sentences:
      - >-
        Outre les probabilités de gagner une bataille théorique, cette citation
        a une autre signification : l'importance de connaître/comprendre les
        autres.
      - Une femme et un chien se promènent ensemble.
      - Un homme joue de la guitare.
  - source_sentence: Un homme joue de la harpe.
    sentences:
      - Une femme joue de la guitare.
      - une femme a un enfant.
      - Un groupe de personnes est debout et assis sur le sol la nuit.
  - source_sentence: Dois cães a lutar na neve.
    sentences:
      - Dois cães brincam na neve.
      - Pode sempre perguntar, então é a escolha do autor a aceitar ou não.
      - Um gato está a caminhar sobre chão de madeira dura.
datasets:
  - PhilipMay/stsb_multi_mt
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - pearson_cosine
  - spearman_cosine
model-index:
  - name: >-
      SentenceTransformer based on
      sentence-transformers/paraphrase-multilingual-mpnet-base-v2
    results:
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: sts eval
          type: sts-eval
        metrics:
          - type: pearson_cosine
            value: 0.8423180648713237
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8595850000432059
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8420181975402647
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8630073561241816
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8405171361303234
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8594948677596693
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8375312155777364
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8583531749722014
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8397619344296936
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8592894281053397
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8302450119489335
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8477495437950113
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8403036335437926
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8618318944578455
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.838706056263606
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8574971366611375
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8413052113094718
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8611085200053895
            name: Spearman Cosine
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: sts test
          type: sts-test
        metrics:
          - type: pearson_cosine
            value: 0.7456938524838218
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7483592546028903
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.7237526314017121
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7169355021670776
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.7669235794906317
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7631313253470643
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8298244150963187
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8324038122126458
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.7166564070706897
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7227801582959456
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.7855295239932334
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7934626158625494
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.8386050236111093
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.8275901416546908
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.779112011887379
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7729611139511264
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.7878478906763803
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7846990470347196
            name: Spearman Cosine
          - type: pearson_cosine
            value: 0.7882844791307567
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.7878180406501333
            name: Spearman Cosine

SentenceTransformer based on sentence-transformers/paraphrase-multilingual-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-mpnet-base-v2 on the multi_stsb_de, multi_stsb_es, multi_stsb_fr, multi_stsb_it, multi_stsb_nl, multi_stsb_pl, multi_stsb_pt, multi_stsb_ru and multi_stsb_zh datasets. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Datasets:
  • multi_stsb_de
  • multi_stsb_es
  • multi_stsb_fr
  • multi_stsb_it
  • multi_stsb_nl
  • multi_stsb_pl
  • multi_stsb_pt
  • multi_stsb_ru
  • multi_stsb_zh
• Languages: de, en, es, fr, it, nl, pl, pt, ru, zh

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): MultiHeadGeneralizedPooling(
    (P): ModuleList(
      (0-7): 8 x Linear(in_features=768, out_features=96, bias=True)
    )
    (W1): ModuleList(
      (0-7): 8 x Linear(in_features=96, out_features=384, bias=True)
    )
    (W2): ModuleList(
      (0-7): 8 x Linear(in_features=384, out_features=96, bias=True)
    )
  )
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RomainDarous/large_directFourEpoch_additivePooling_noisedInit_stsModel")
# Run inference
sentences = [
    'Dois cães a lutar na neve.',
    'Dois cães brincam na neve.',
    'Pode sempre perguntar, então é a escolha do autor a aceitar ou não.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

• Datasets: sts-eval, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test and sts-test
• Evaluated with EmbeddingSimilarityEvaluator

Metric	sts-eval	sts-test
pearson_cosine	0.8423	0.7883
spearman_cosine	0.8596	0.7878

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.842
spearman_cosine	0.863

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8405
spearman_cosine	0.8595

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8375
spearman_cosine	0.8584

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8398
spearman_cosine	0.8593

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8302
spearman_cosine	0.8477

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8403
spearman_cosine	0.8618

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8387
spearman_cosine	0.8575

Semantic Similarity

• Dataset: sts-eval
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8413
spearman_cosine	0.8611

Training Details

Training Datasets

multi_stsb_de

multi_stsb_de

• Dataset: multi_stsb_de at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 5 tokens mean: 11.58 tokens max: 37 tokens	min: 6 tokens mean: 11.53 tokens max: 36 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Ein Flugzeug hebt gerade ab.	Ein Flugzeug hebt gerade ab.	1.0
  Ein Mann spielt eine große Flöte.	Ein Mann spielt eine Flöte.	0.7599999904632568
  Ein Mann streicht geriebenen Käse auf eine Pizza.	Ein Mann streicht geriebenen Käse auf eine ungekochte Pizza.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_es

multi_stsb_es

• Dataset: multi_stsb_es at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 7 tokens mean: 12.21 tokens max: 33 tokens	min: 7 tokens mean: 12.07 tokens max: 31 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Un avión está despegando.	Un avión está despegando.	1.0
  Un hombre está tocando una gran flauta.	Un hombre está tocando una flauta.	0.7599999904632568
  Un hombre está untando queso rallado en una pizza.	Un hombre está untando queso rallado en una pizza cruda.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_fr

multi_stsb_fr

• Dataset: multi_stsb_fr at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 12.6 tokens max: 33 tokens	min: 6 tokens mean: 12.49 tokens max: 32 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Un avion est en train de décoller.	Un avion est en train de décoller.	1.0
  Un homme joue d'une grande flûte.	Un homme joue de la flûte.	0.7599999904632568
  Un homme étale du fromage râpé sur une pizza.	Un homme étale du fromage râpé sur une pizza non cuite.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_it

multi_stsb_it

• Dataset: multi_stsb_it at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 7 tokens mean: 12.77 tokens max: 36 tokens	min: 8 tokens mean: 12.69 tokens max: 30 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Un aereo sta decollando.	Un aereo sta decollando.	1.0
  Un uomo sta suonando un grande flauto.	Un uomo sta suonando un flauto.	0.7599999904632568
  Un uomo sta spalmando del formaggio a pezzetti su una pizza.	Un uomo sta spalmando del formaggio a pezzetti su una pizza non cotta.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_nl

multi_stsb_nl

• Dataset: multi_stsb_nl at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 11.67 tokens max: 33 tokens	min: 6 tokens mean: 11.55 tokens max: 29 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Er gaat een vliegtuig opstijgen.	Er gaat een vliegtuig opstijgen.	1.0
  Een man speelt een grote fluit.	Een man speelt fluit.	0.7599999904632568
  Een man smeert geraspte kaas op een pizza.	Een man strooit geraspte kaas op een ongekookte pizza.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_pl

multi_stsb_pl

• Dataset: multi_stsb_pl at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 5 tokens mean: 12.2 tokens max: 39 tokens	min: 5 tokens mean: 12.11 tokens max: 35 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Samolot wystartował.	Samolot wystartował.	1.0
  Człowiek gra na dużym flecie.	Człowiek gra na flecie.	0.7599999904632568
  Mężczyzna rozsiewa na pizzy rozdrobniony ser.	Mężczyzna rozsiewa rozdrobniony ser na niegotowanej pizzy.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_pt

multi_stsb_pt

• Dataset: multi_stsb_pt at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 7 tokens mean: 12.33 tokens max: 34 tokens	min: 7 tokens mean: 12.29 tokens max: 32 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Um avião está a descolar.	Um avião aéreo está a descolar.	1.0
  Um homem está a tocar uma grande flauta.	Um homem está a tocar uma flauta.	0.7599999904632568
  Um homem está a espalhar queijo desfiado numa pizza.	Um homem está a espalhar queijo desfiado sobre uma pizza não cozida.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_ru

multi_stsb_ru

• Dataset: multi_stsb_ru at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 5 tokens mean: 11.19 tokens max: 39 tokens	min: 5 tokens mean: 11.17 tokens max: 26 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  Самолет взлетает.	Взлетает самолет.	1.0
  Человек играет на большой флейте.	Человек играет на флейте.	0.7599999904632568
  Мужчина разбрасывает сыр на пиццу.	Мужчина разбрасывает измельченный сыр на вареную пиццу.	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_zh

multi_stsb_zh

• Dataset: multi_stsb_zh at 3acaa3d
• Size: 5,749 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 10.7 tokens max: 32 tokens	min: 7 tokens mean: 10.79 tokens max: 26 tokens	min: 0.0 mean: 0.45 max: 1.0

• Samples:

  sentence1	sentence2	score
  一架飞机正在起飞。	一架飞机正在起飞。	1.0
  一个男人正在吹一支大笛子。	一个人在吹笛子。	0.7599999904632568
  一名男子正在比萨饼上涂抹奶酪丝。	一名男子正在将奶酪丝涂抹在未熟的披萨上。	0.7599999904632568

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Datasets

multi_stsb_de

multi_stsb_de

• Dataset: multi_stsb_de at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 5 tokens mean: 18.25 tokens max: 47 tokens	min: 6 tokens mean: 18.25 tokens max: 54 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Ein Mann mit einem Schutzhelm tanzt.	Ein Mann mit einem Schutzhelm tanzt.	1.0
  Ein kleines Kind reitet auf einem Pferd.	Ein Kind reitet auf einem Pferd.	0.949999988079071
  Ein Mann verfüttert eine Maus an eine Schlange.	Der Mann füttert die Schlange mit einer Maus.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_es

multi_stsb_es

• Dataset: multi_stsb_es at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 7 tokens mean: 17.98 tokens max: 47 tokens	min: 7 tokens mean: 17.86 tokens max: 47 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Un hombre con un casco está bailando.	Un hombre con un casco está bailando.	1.0
  Un niño pequeño está montando a caballo.	Un niño está montando a caballo.	0.949999988079071
  Un hombre está alimentando a una serpiente con un ratón.	El hombre está alimentando a la serpiente con un ratón.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_fr

multi_stsb_fr

• Dataset: multi_stsb_fr at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 19.7 tokens max: 49 tokens	min: 6 tokens mean: 19.65 tokens max: 51 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Un homme avec un casque de sécurité est en train de danser.	Un homme portant un casque de sécurité est en train de danser.	1.0
  Un jeune enfant monte à cheval.	Un enfant monte à cheval.	0.949999988079071
  Un homme donne une souris à un serpent.	L'homme donne une souris au serpent.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_it

multi_stsb_it

• Dataset: multi_stsb_it at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 18.42 tokens max: 46 tokens	min: 8 tokens mean: 18.43 tokens max: 53 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Un uomo con l'elmetto sta ballando.	Un uomo che indossa un elmetto sta ballando.	1.0
  Un bambino piccolo sta cavalcando un cavallo.	Un bambino sta cavalcando un cavallo.	0.949999988079071
  Un uomo sta dando da mangiare un topo a un serpente.	L'uomo sta dando da mangiare un topo al serpente.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_nl

multi_stsb_nl

• Dataset: multi_stsb_nl at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 5 tokens mean: 17.88 tokens max: 50 tokens	min: 6 tokens mean: 17.71 tokens max: 51 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Een man met een helm is aan het dansen.	Een man met een helm is aan het dansen.	1.0
  Een jong kind rijdt op een paard.	Een kind rijdt op een paard.	0.949999988079071
  Een man voedt een muis aan een slang.	De man voert een muis aan de slang.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_pl

multi_stsb_pl

• Dataset: multi_stsb_pl at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 18.54 tokens max: 46 tokens	min: 6 tokens mean: 18.43 tokens max: 54 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Tańczy mężczyzna w twardym kapeluszu.	Tańczy mężczyzna w twardym kapeluszu.	1.0
  Małe dziecko jedzie na koniu.	Dziecko jedzie na koniu.	0.949999988079071
  Człowiek karmi węża myszką.	Ten człowiek karmi węża myszką.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_pt

multi_stsb_pt

• Dataset: multi_stsb_pt at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 7 tokens mean: 18.22 tokens max: 46 tokens	min: 7 tokens mean: 18.11 tokens max: 46 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Um homem de chapéu duro está a dançar.	Um homem com um capacete está a dançar.	1.0
  Uma criança pequena está a montar a cavalo.	Uma criança está a montar a cavalo.	0.949999988079071
  Um homem está a alimentar um rato a uma cobra.	O homem está a alimentar a cobra com um rato.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_ru

multi_stsb_ru

• Dataset: multi_stsb_ru at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 17.92 tokens max: 49 tokens	min: 5 tokens mean: 17.75 tokens max: 47 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  Человек в твердой шляпе танцует.	Мужчина в твердой шляпе танцует.	1.0
  Маленький ребенок едет верхом на лошади.	Ребенок едет на лошади.	0.949999988079071
  Мужчина кормит мышь змее.	Человек кормит змею мышью.	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

multi_stsb_zh

multi_stsb_zh

• Dataset: multi_stsb_zh at 3acaa3d
• Size: 1,500 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 6 tokens mean: 15.37 tokens max: 46 tokens	min: 5 tokens mean: 15.24 tokens max: 46 tokens	min: 0.0 mean: 0.42 max: 1.0

• Samples:

  sentence1	sentence2	score
  一个戴着硬帽子的人在跳舞。	一个戴着硬帽的人在跳舞。	1.0
  一个小孩子在骑马。	一个孩子在骑马。	0.949999988079071
  一个人正在用老鼠喂蛇。	那人正在给蛇喂老鼠。	1.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 4
• warmup_ratio: 0.1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	multi stsb de loss	multi stsb es loss	multi stsb fr loss	multi stsb it loss	multi stsb nl loss	multi stsb pl loss	multi stsb pt loss	multi stsb ru loss	multi stsb zh loss	sts-eval_spearman_cosine	sts-test_spearman_cosine
4.0	12960	3.7859	6.5030	6.5739	6.7230	6.8049	6.6585	6.8389	6.6333	6.7102	6.3148	0.8611	-
-1	-1	-	-	-	-	-	-	-	-	-	-	-	0.7878

Framework Versions

• Python: 3.10.13
• Sentence Transformers: 3.4.1
• Transformers: 4.48.2
• PyTorch: 2.1.2+cu121
• Accelerate: 1.3.0
• Datasets: 2.16.1
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}