metadata

base_model: bobox/DeBERTa-small-ST-v1-test-step3
datasets: []
language: []
library_name: sentence-transformers
metrics:
  - pearson_cosine
  - spearman_cosine
  - pearson_manhattan
  - spearman_manhattan
  - pearson_euclidean
  - spearman_euclidean
  - pearson_dot
  - spearman_dot
  - pearson_max
  - spearman_max
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:120849
  - loss:CachedGISTEmbedLoss
widget:
  - source_sentence: >-
      "Today I lost those who for 24 years I called...my family," said Enes
      Kanter of the Oklahoma City Thunder.

      Turkish President Recep Tayyip Erdogan blames Mr Gulen for inciting a
      failed coup last month and is seeking the cleric's extradition to Turkey.

      Mr Gulen, who has a large following, denies being involved in the coup.

      Kanter's father, Mehmet, disowned his son in a letter published on Monday
      by Sabah, a pro-government newspaper.

      Mehmet Kanter wrote his son had been "hypnotised" by the Gulen movement.

      "With a feeling of shame I apologise to our president and the Turkish
      people for having such a son," the letter said.

      Q&A on the Gulen movement

      Mr Gulen is regarded by followers as a spiritual leader and sometimes
      described as Turkey's second most powerful man.

      Enes Kanter has been a vocal supporter of Mr Gulen on Twitter.

      The movement - known in Turkey as Hizmet, or service - runs schools all
      over Turkey and around the world, including in Turkic former Soviet
      republics, Muslim countries such as Pakistan and Western nations including
      Romania and the US, where it runs more than 100 schools.

      In May 2016, the Turkish government formally declared the Gulen movement a
      terrorist organisation.

      After the failed coup, suspected Gulen supporters in Turkey were purged in
      a wave of arrests.

      Western nations have been critical of the government's response to the
      coup. US officials have said they will extradite Mr Gulen only if Turkey
      provides evidence.
    sentences:
      - "The Thinker | Rodin Museum H. 189 cm ; W. 98 cm ; D. 140 cm S.2838 When conceived in 1880 in its original size (approx. 70 cm) as the crowning element of\_The Gates of Hell\_, seated on the tympanum ,\_The Thinker\_was entitled\_The Poet. He represented Dante, author of the\_Divine Comedy\_which had inspired\_The Gates, leaning forward to observe the circles of Hell, while meditating on his work.\_The Thinker\_was therefore initially both a being with a tortured body, almost a damned soul, and a free-thinking man, determined to transcend his suffering through poetry. The pose of this figure owes much to Carpeaux’s\_Ugolino\_(1861) and to the seated portrait of Lorenzo de’ Medici carved by Michelangelo (1526-31). \_ While remaining in place on the monumental\_Gates of Hell,\_The Thinker\_was exhibited individually in 1888 and thus became an independent work. Enlarged in 1904, its colossal version proved even more popular: this image of a man lost in thought, but whose powerful body suggests a great capacity for action, has became one of the most celebrated sculptures ever known. Numerous casts exist worldwide, including the one now in the gardens of the Musée Rodin, a gift to the City of Paris installed outside the Panthéon in 1906, and another in the gardens of Rodin’s house in Meudon, on the tomb of the sculptor and his wife. George Bernard Shaw in the Pose of \"The Thinker\" Rodin, the Monument to Victor Hugo and The Thinker Rodin's \"Thinker\" in Dr Linde's Garden in Lübeck"
      - >-
        An American basketball player has cut ties with his Turkish family over
        his support for Pennsylvania-based preacher Fethullah Gulen.
      - Police are investigating a death at a bus stop in Fife.
  - source_sentence: Two adorable birds perched on a piece of bamboo.
    sentences:
      - Two birds are sitting perched on a tree limb
      - A young boy with a spoon looking at a birthday cupcake.
      - >-
        As part of his attempt to turn the Austrian right , Dessaix ordered a
        battalion to move along the Aire stream near Tairier and Crache .
  - source_sentence: how do venom snake keepers make money?
    sentences:
      - "The USDA regulates who can buy and sell snake venom. It is very important to learn about these regulations so that you can operate properly. On average, snake milkers make around $2,500 per month, but snake venom is an expensive market. One gram of certain types of snake venom can sell for $2,000.If you are crazy enough to capture, milk, and breed snakes, please take the precaution to wear protective clothing and always have antivenom close at hand.nake milkers have an insane job. They â\x80\x9Cmilkâ\x80\x9D snakes for their venom. This means that every single day, a snake milker handles deadly, venomous snakes. Itâ\x80\x99s a hands on job where you put your fingers millimeters away from the sharp, fangs of asps, vipers, cobras, corals, mambas, kraits, and rattlesnakes."
      - a greenhouse is used to protect plants by keeping them warm
      - >-
        Nashville Mayor Megan Barry has said her 22-year-old son died of what
        appeared to be a drug overdose, according to a family statement.
  - source_sentence: Adult bees include workers, a queen and what other type?
    sentences:
      - |-
        matter vibrating can cause sound. Thus, sound is a wave in air . 
         matter vibrating can cause a wave in air
      - >-
        His references in electronic music are Todd Terry , Armand Van Helden ,
        Roger Sanchez , Tiesto and the Epic Sax Guy.
      - >-
        Look at the honeybees in Figure below . Honeybees live in colonies that
        may consist of thousands of individual bees. Generally, there are three
        types of adult bees in a colony: workers, a queen, and drones.
  - source_sentence: can an object have constant non zero velocity and changing acceleration?
    sentences:
      - when an animal sheds its fur , its fur becomes less dense
      - >-
        Acceleration is defined as the time derivative of the velocity; if the
        velocity is unchanging the acceleration is zero. Velocity is a vector,
        speed is a scalar magnitude of the vector. If the velocity vector
        changes direction you can have constant speed (not velocity) with a
        non-zero acceleration.
      - >-
        Acne treatment is individual and customized to the type of acne you
        have. On average, mild acne responds in 1-2 months, moderate acne
        responds in 2-4 months and severe acne can take 4-6 months to clear,
        granted that the most effective measures can be used.
model-index:
  - name: SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
    results:
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: sts test
          type: sts-test
        metrics:
          - type: pearson_cosine
            value: 0.874702030760496
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.9021036690960521
            name: Spearman Cosine
          - type: pearson_manhattan
            value: 0.9071871020121037
            name: Pearson Manhattan
          - type: spearman_manhattan
            value: 0.9048875555646884
            name: Spearman Manhattan
          - type: pearson_euclidean
            value: 0.9065531106539271
            name: Pearson Euclidean
          - type: spearman_euclidean
            value: 0.9043656663543417
            name: Spearman Euclidean
          - type: pearson_dot
            value: 0.8555439931828537
            name: Pearson Dot
          - type: spearman_dot
            value: 0.8598106392441436
            name: Spearman Dot
          - type: pearson_max
            value: 0.9071871020121037
            name: Pearson Max
          - type: spearman_max
            value: 0.9048875555646884
            name: Spearman Max

SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3

This is a sentence-transformers model finetuned from bobox/DeBERTa-small-ST-v1-test-step3 on the bobox/enhanced_nli-50_k dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: bobox/DeBERTa-small-ST-v1-test-step3
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Training Dataset:
- bobox/enhanced_nli-50_k

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-checkpoints-tmp")
# Run inference
sentences = [
    'can an object have constant non zero velocity and changing acceleration?',
    'Acceleration is defined as the time derivative of the velocity; if the velocity is unchanging the acceleration is zero. Velocity is a vector, speed is a scalar magnitude of the vector. If the velocity vector changes direction you can have constant speed (not velocity) with a non-zero acceleration.',
    'Acne treatment is individual and customized to the type of acne you have. On average, mild acne responds in 1-2 months, moderate acne responds in 2-4 months and severe acne can take 4-6 months to clear, granted that the most effective measures can be used.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

Dataset: sts-test
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8747
spearman_cosine	0.9021
pearson_manhattan	0.9072
spearman_manhattan	0.9049
pearson_euclidean	0.9066
spearman_euclidean	0.9044
pearson_dot	0.8555
spearman_dot	0.8598
pearson_max	0.9072
spearman_max	0.9049

Training Details

Training Dataset

bobox/enhanced_nli-50_k

Dataset: bobox/enhanced_nli-50_k
Size: 120,849 training samples
Columns: sentence1 and sentence2
Approximate statistics based on the first 1000 samples:
sentence1 sentence2
type string string
details
min: 4 tokens
mean: 32.01 tokens
max: 336 tokens

min: 2 tokens
mean: 60.45 tokens
max: 512 tokens

	sentence1	sentence2
type	string	string
details	min: 4 tokens mean: 32.01 tokens max: 336 tokens	min: 2 tokens mean: 60.45 tokens max: 512 tokens

Samples:

sentence1	sentence2
`A lady working in a kitchen with several different types of dishes.`	`A woman is cooking and cleaning in her kitchen.`
`can you renew your licence online sa?`	`You can renew your licence online for as long as your photo is valid. Renew your driver's licence online with a mySA GOV account. With a mySA GOV account, you can access a legally compliant digital licence through the mySA GOV app.`
`how can coconut oil lower cholesterol`	`It has been shown that lauric acid increases the good HDL cholesterol in the blood to help improve cholesterol ratio levels. Coconut oil lowers cholesterol by promoting its conversion to pregnenolone, a molecule that is a precursor to many of the hormones our bodies need. Coconut can help restore normal thyroid function. When the thyroid does not function optimally, it can contribute to higher levels of bad cholesterol.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.025}

Evaluation Dataset

bobox/enhanced_nli-50_k

Dataset: bobox/enhanced_nli-50_k
Size: 3,052 evaluation samples
Columns: sentence1 and sentence2
Approximate statistics based on the first 1000 samples:
sentence1 sentence2
type string string
details
min: 4 tokens
mean: 32.91 tokens
max: 342 tokens

min: 2 tokens
mean: 60.3 tokens
max: 408 tokens

	sentence1	sentence2
type	string	string
details	min: 4 tokens mean: 32.91 tokens max: 342 tokens	min: 2 tokens mean: 60.3 tokens max: 408 tokens

Samples:

sentence1	sentence2
The body was found in the River Avon in Bath, Avon and Somerset Police said. Officers said although formal identification had not yet taken place, Henry Burke's family had been told. Earlier officers said they were looking for Mr Burke, who was last seen leaving a nightclub in George Street late on Thursday. A force spokesman said the death was being treated as unexplained and inquiries were continuing. Mr Burke's girlfriend, Em Comley, earlier said he had been texting her "throughout the night" but then the messages suddenly stopped just after midnight.	`A man's body has been found in a river after search and rescue teams were called in to try and find a missing 19-year-old student.`
`what happens when the president of united states is impeached?`	`Parliament votes on the proposal by secret ballot, and if two thirds of all representatives agree, the president is impeached. Once impeached, the president's powers are suspended, and the Constitutional Court decides whether or not the President should be removed from office.`
`What can feed at more than one trophic level?`	`Many consumers feed at more than one trophic level.. Nuts are also consumed by deer, turkey, foxes, wood ducks and squirrels. wood ducks can feed at more than one trophic level`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.025}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 960
per_device_eval_batch_size: 128
learning_rate: 3.5e-05
weight_decay: 0.0001
num_train_epochs: 2
lr_scheduler_type: cosine_with_min_lr
lr_scheduler_kwargs: {'num_cycles': 0.5, 'min_lr': 5.833333333333333e-06}
warmup_ratio: 0.25
save_safetensors: False
fp16: True
push_to_hub: True
hub_model_id: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-checkpoints-tmp
hub_strategy: all_checkpoints
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 960
per_device_eval_batch_size: 128
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 3.5e-05
weight_decay: 0.0001
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: cosine_with_min_lr
lr_scheduler_kwargs: {'num_cycles': 0.5, 'min_lr': 5.833333333333333e-06}
warmup_ratio: 0.25
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: True
resume_from_checkpoint: None
hub_model_id: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-checkpoints-tmp
hub_strategy: all_checkpoints
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	loss	sts-test_spearman_cosine
0.0079	1	0.404	-	-
0.0159	2	0.3185	-	-
0.0238	3	0.2821	-	-
0.0317	4	0.4036	-	-
0.0397	5	0.3442	0.1253	0.9078
0.0476	6	0.4145	-	-
0.0556	7	0.4224	-	-
0.0635	8	0.4048	-	-
0.0714	9	0.3899	-	-
0.0794	10	0.4127	0.1237	0.9079
0.0873	11	0.3496	-	-
0.0952	12	0.3731	-	-
0.1032	13	0.3929	-	-
0.1111	14	0.2957	-	-
0.1190	15	0.3324	0.1206	0.9083
0.1270	16	0.3341	-	-
0.1349	17	0.3466	-	-
0.1429	18	0.3558	-	-
0.1508	19	0.2634	-	-
0.1587	20	0.3095	0.1156	0.9088
0.1667	21	0.2973	-	-
0.1746	22	0.2884	-	-
0.1825	23	0.3697	-	-
0.1905	24	0.2683	-	-
0.1984	25	0.3026	0.1096	0.9088
0.2063	26	0.2441	-	-
0.2143	27	0.3145	-	-
0.2222	28	0.3119	-	-
0.2302	29	0.2766	-	-
0.2381	30	0.3343	0.1054	0.9084
0.2460	31	0.344	-	-
0.2540	32	0.3005	-	-
0.2619	33	0.2526	-	-
0.2698	34	0.2422	-	-
0.2778	35	0.3447	0.1022	0.9072
0.2857	36	0.2809	-	-
0.2937	37	0.2836	-	-
0.3016	38	0.2878	-	-
0.3095	39	0.2738	-	-
0.3175	40	0.2806	0.1003	0.9065
0.3254	41	0.2797	-	-
0.3333	42	0.3217	-	-
0.3413	43	0.2544	-	-
0.3492	44	0.3203	-	-
0.3571	45	0.2987	0.0990	0.9064
0.3651	46	0.2765	-	-
0.3730	47	0.2716	-	-
0.3810	48	0.3726	-	-
0.3889	49	0.2963	-	-
0.3968	50	0.2784	0.0952	0.9072
0.4048	51	0.2437	-	-
0.4127	52	0.2258	-	-
0.4206	53	0.2821	-	-
0.4286	54	0.249	-	-
0.4365	55	0.2813	0.0928	0.9080
0.4444	56	0.3003	-	-
0.4524	57	0.2812	-	-
0.4603	58	0.2619	-	-
0.4683	59	0.299	-	-
0.4762	60	0.2706	0.0927	0.9088
0.4841	61	0.297	-	-
0.4921	62	0.2906	-	-
0.5	63	0.2914	-	-
0.5079	64	0.2669	-	-
0.5159	65	0.2723	0.0946	0.9093
0.5238	66	0.3194	-	-
0.5317	67	0.3585	-	-
0.5397	68	0.2843	-	-
0.5476	69	0.1916	-	-
0.5556	70	0.351	0.0971	0.9104
0.5635	71	0.3105	-	-
0.5714	72	0.2847	-	-
0.5794	73	0.2641	-	-
0.5873	74	0.3305	-	-
0.5952	75	0.2461	0.0965	0.9096
0.6032	76	0.259	-	-
0.6111	77	0.2506	-	-
0.6190	78	0.2832	-	-
0.6270	79	0.3322	-	-
0.6349	80	0.2533	0.1001	0.9089
0.6429	81	0.2349	-	-
0.6508	82	0.2748	-	-
0.6587	83	0.223	-	-
0.6667	84	0.2416	-	-
0.6746	85	0.2637	0.1034	0.9082
0.6825	86	0.2856	-	-
0.6905	87	0.2476	-	-
0.6984	88	0.2427	-	-
0.7063	89	0.2614	-	-
0.7143	90	0.26	0.1032	0.9088
0.7222	91	0.1862	-	-
0.7302	92	0.267	-	-
0.7381	93	0.2175	-	-
0.7460	94	0.2079	-	-
0.7540	95	0.2562	0.0999	0.9086
0.7619	96	0.2516	-	-
0.7698	97	0.2956	-	-
0.7778	98	0.2733	-	-
0.7857	99	0.2919	-	-
0.7937	100	0.2997	0.1032	0.9069
0.8016	101	0.2276	-	-
0.8095	102	0.2582	-	-
0.8175	103	0.2559	-	-
0.8254	104	0.2864	-	-
0.8333	105	0.2839	0.1074	0.9076
0.8413	106	0.2549	-	-
0.8492	107	0.2826	-	-
0.8571	108	0.2334	-	-
0.8651	109	0.2632	-	-
0.8730	110	0.2255	0.1090	0.9056
0.8810	111	0.2589	-	-
0.8889	112	0.2569	-	-
0.8968	113	0.2797	-	-
0.9048	114	0.2742	-	-
0.9127	115	0.2295	0.1070	0.9014
0.9206	116	0.2047	-	-
0.9286	117	0.2577	-	-
0.9365	118	0.2614	-	-
0.9444	119	0.2722	-	-
0.9524	120	0.1927	0.1024	0.9008
0.9603	121	0.2649	-	-
0.9683	122	0.2386	-	-
0.9762	123	0.2801	-	-
0.9841	124	0.2583	-	-
0.9921	125	0.3076	0.0949	0.9016
1.0	126	0.5477	-	-
1.0079	127	0.0031	-	-
1.0159	128	0.0	-	-
1.0238	129	0.0	-	-
1.0317	130	0.0	0.0955	0.9021

Framework Versions

Python: 3.10.14
Sentence Transformers: 3.0.1
Transformers: 4.44.0
PyTorch: 2.4.0
Accelerate: 0.33.0
Datasets: 2.21.0
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}