stsb-all-MiniLM-L6-v2

This model is a fine-tuned version of sentence-transformers/all-MiniLM-L6-v2 on the Semantic Textual Similarity Benchmark (STS-B) dataset.

It achieves the following results on the evaluation set:

• Loss: 0.0307
• Pearson: 0.8287

Model description

This model is fine-tuned from the pre-trained sentence-transformers/all-MiniLM-L6-v2 on the Semantic Textual Similarity Benchmark (STS-B) dataset. It is designed to compute similarity scores between pairs of sentences, returning a continuous score between 0 and 1, where 1 represents maximum semantic similarity.

The model generates embeddings for input sentences and can be used for tasks such as text similarity, sentence clustering, or semantic search.

Training and evaluation data

The model was trained on the STS-B dataset using the following splits:

Train set: 5,749 examples Validation set: 1,500 examples Test set: 1,379 examples Each example consists of two sentences and a similarity score (from 0 to 1) indicating their semantic closeness.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Pearson
No log	1.0	360	0.0354	0.7935
0.0483	2.0	720	0.0391	0.8124
0.021	3.0	1080	0.0332	0.8206
0.021	4.0	1440	0.0296	0.8296
0.0155	5.0	1800	0.0307	0.8287

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0