metadata

base_model: bobox/DeBERTa-small-ST-v1-test-step3
datasets: []
language: []
library_name: sentence-transformers
metrics:
  - pearson_cosine
  - spearman_cosine
  - pearson_manhattan
  - spearman_manhattan
  - pearson_euclidean
  - spearman_euclidean
  - pearson_dot
  - spearman_dot
  - pearson_max
  - spearman_max
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:163205
  - loss:CachedGISTEmbedLoss
widget:
  - source_sentence: interview question on why you want to leave your current job?
    sentences:
      - >-
        Carrier proteins bind and carry the molecules across the cell membrane.
        These proteins bind a molecule on one side of the membrane, change shape
        as they carry the molecule across the membrane, and deposit the molecule
        on the other side of the membrane. Even though a protein is involved in
        both these methods of transport, neither method requires energy.
        Therefore these are still types of passive transport.
      - >-
        ['Desire to learn.', 'Desire to take on more responsibility.', 'Desire
        to take on less responsibility.', 'Desire to relocate.', 'Desire for a
        career change.', 'Desire to gain a new skill or grow a current skill.',
        'Company reorganization has led to change in job content.']
      - >-
        The small intestine is a narrow tube that starts at the stomach and ends
        at the large intestine. In adults, it’s about 7 meters (23 feet) long.
        Most chemical digestion and almost all nutrient absorption take place in
        the small intestine.
  - source_sentence: >-
      They say a number of people from the Mujahideen-e-Khalq (MEK) group were
      injured at Camp Liberty in Baghdad.

      Baghdad has in the past repeatedly denied attacking the group.

      MEK members fought with Iraq against Iran in the 1980s, but have since
      fallen out with the current Iraqi government.

      In an emailed message, the Paris-based National Council of Resistance of
      Iran (NCRI), the MEK's parent group, said dozens of missiles hit the camp
      on Thursday evening.

      It said two residents were killed and a third later died in hospital of
      his wounds.

      The US condemned the attack "in the strongest terms" and urged Iraq to
      better protect the camp.

      An Iranian-backed Shia militia, al-Mukhtar Army, said it had fired rockets
      at the camp, Reuters news agency reported.

      The camp is located in a former US military base, near Baghdad's airport.

      The Iraqi authorities have made no public comments on the report. However,
      one security official was quoted by the Associated Press as saying four
      rockets hit the camp, injuring two people.

      In September, the MEK accused Iraqi forces of attacking Camp Ashraf
      north-east of Baghdad and killing 52 of the group's members.

      In recent years, Baghdad has been trying to dismantle MEK camps and eject
      the group.

      Iran considers the MEK a terrorist group.

      The group was removed from the US state department's list of terrorist
      organisations last year.
    sentences:
      - >-
        A rocket attack has killed three members of an Iranian opposition group
        in Iraq, the group and its parent organisation say.
      - >-
        Directions See How It's Made. 1  Preheat the oven to 375 degrees. 2 
        Heat a large skillet over medium heat. 3  Add the spinach.  Spread the
        spinach mixture evenly on the bottom of the prepared baking 1  dish.
        Bake until the egg whites are set, about 25 minutes.  Let the casserole
        sit for about 5 minutes, and then cut into pieces and 1  serve. Submit a
        Correction.
      - >-
        Toll free phone number: 011-44-871-246-0002. Ryan Airlines
        www.ryanair.com is a low-fare airlines headquartered in Dublin, Ireland.
        Transporting over 103 million passengers last year there are 1600 daily
        flights with 185 destinations. The Ryan fleet consists of 300 new Boeing
        737-800 aircraft in operation with 283 738 aircraft on order.
  - source_sentence: In what unit is heat measured in?
    sentences:
      - >-
        The heat that is either absorbed or released is measured in joules. The
        mass is measured in grams. The change in temperature is given by , where
        is the final temperature and is the initial temperature.
      - >-
        The nitrogen atom of a primary amine is bonded to two hydrogen atoms and
        one carbon. The nitrogen atom of a secondary amine is bonded to one
        hydrogen and two carbons. The nitrogen atom of a tertiary amine is
        bonded to three carbon atoms. Amines are typically named by a common
        system rather than by IUPAC guidelines. The common system for naming
        amines along with several examples is shown below.
      - >-
        Seattle Symphony Live @ Benaroya Hall — Windborne's The Music of David
        Bowie: A Rock Symphony with the Seattle Symphony Tuesday, 10 January,
        2017 7:30PM Join conductor Brent Havens and a full rock band on a
        symphonic musical odyssey that explores the incredible range of David
        Bowie’s Music.
  - source_sentence: meristematic tissue definition
    sentences:
      - >-
        In this chapter, you saw how pressure and buoyancy of fluids can be used
        to make work easier — from raising a car on a lift to floating a ship on
        the ocean. Devices that make work easier are called machines in physics.
      - A Land Rover is splashing water as it crosses a river.
      - "meristem. n. 1. (Botany) a plant tissue responsible for growth, whose cells divide and differentiate to form the tissues and organs of the plant. Meristems occur within the stem (see cambium) and leaves and at the tips of stems and roots. [C19: from Greek meristos divided, from merizein to divide, from meris portion]. (Ë\x88mÉ\x9Br É\x99Ë\x8CstÉ\x9Bm)."
  - source_sentence: >-
      More than 190 countries and territories around the world had confirmed
      coronavirus cases by March 22 , 2020 .
    sentences:
      - when electricity flows to a light bulb , the light bulb will come on
      - >-
        As of 22 March , more than 337,000 cases of COVID-19 have been reported
        in over 190 countries and territories , resulting in more than 14,400
        deaths and 96,000 recoveries .
      - a greenhouse is used to protect plants by keeping them warm
model-index:
  - name: SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
    results:
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: sts test
          type: sts-test
        metrics:
          - type: pearson_cosine
            value: 0.8836376979322419
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.9078868298544011
            name: Spearman Cosine
          - type: pearson_manhattan
            value: 0.9106248996071287
            name: Pearson Manhattan
          - type: spearman_manhattan
            value: 0.9071474495136926
            name: Spearman Manhattan
          - type: pearson_euclidean
            value: 0.9098265834859519
            name: Pearson Euclidean
          - type: spearman_euclidean
            value: 0.9063156130669492
            name: Spearman Euclidean
          - type: pearson_dot
            value: 0.8710695777275684
            name: Pearson Dot
          - type: spearman_dot
            value: 0.8773200625274038
            name: Spearman Dot
          - type: pearson_max
            value: 0.9106248996071287
            name: Pearson Max
          - type: spearman_max
            value: 0.9078868298544011
            name: Spearman Max

SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3

This is a sentence-transformers model finetuned from bobox/DeBERTa-small-ST-v1-test-step3 on the bobox/enhanced_nli-50_k dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: bobox/DeBERTa-small-ST-v1-test-step3
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Training Dataset:
- bobox/enhanced_nli-50_k

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-Ft-checkpoints-tmp")
# Run inference
sentences = [
    'More than 190 countries and territories around the world had confirmed coronavirus cases by March 22 , 2020 .',
    'As of 22 March , more than 337,000 cases of COVID-19 have been reported in over 190 countries and territories , resulting in more than 14,400 deaths and 96,000 recoveries .',
    'a greenhouse is used to protect plants by keeping them warm',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

Dataset: sts-test
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8836
spearman_cosine	0.9079
pearson_manhattan	0.9106
spearman_manhattan	0.9071
pearson_euclidean	0.9098
spearman_euclidean	0.9063
pearson_dot	0.8711
spearman_dot	0.8773
pearson_max	0.9106
spearman_max	0.9079

Training Details

Training Dataset

bobox/enhanced_nli-50_k

Dataset: bobox/enhanced_nli-50_k
Size: 163,205 training samples
Columns: sentence1 and sentence2
Approximate statistics based on the first 1000 samples:
sentence1 sentence2
type string string
details
min: 4 tokens
mean: 36.63 tokens
max: 370 tokens

min: 2 tokens
mean: 54.93 tokens
max: 363 tokens

	sentence1	sentence2
type	string	string
details	min: 4 tokens mean: 36.63 tokens max: 370 tokens	min: 2 tokens mean: 54.93 tokens max: 363 tokens

Samples:

sentence1	sentence2
`apple customer number`	`Apple Customer Service. 800-676-2775. Before dialing Apple Customer Service. We've tried for years to eliminate that 44 minute wait for you. But the fact is that it can still take many phone calls to Apple to resolve your issue. Now you can hire GetHuman to do all the work on the phone for you.`
`Molly 's Game grossed more than $ 28.5 million in the US and Canada and less than $ 24.65 in other countries .`	`, Molly 's Game has grossed $ 28.8 million in the United States and Canada , and $ 24.6 million in other territories , for a worldwide total of $ 53.4 million .`
`mawk definition`	`Definitions for mawk. Here are all the possible meanings and translations of the word mawk. Wiktionary(0.00 / 0 votes)Rate this definition: Origin: From mawk, mauk, a contraction of mathek, from maÃ°kr, a diminutive of a base from maÃ¾a- (Old English maÃ¾a), from Indo-European *math-, moth- used in reference to insects and vermin. Cognate with Danish madike, Swedish mask, archaic English maddock (modern maggot).`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.025}

Evaluation Dataset

bobox/enhanced_nli-50_k

Dataset: bobox/enhanced_nli-50_k
Size: 3,052 evaluation samples
Columns: sentence1 and sentence2
Approximate statistics based on the first 1000 samples:
sentence1 sentence2
type string string
details
min: 4 tokens
mean: 33.13 tokens
max: 313 tokens

min: 2 tokens
mean: 58.53 tokens
max: 422 tokens

	sentence1	sentence2
type	string	string
details	min: 4 tokens mean: 33.13 tokens max: 313 tokens	min: 2 tokens mean: 58.53 tokens max: 422 tokens

Samples:

sentence1	sentence2
Dwight Beare, 27, from Melbourne, Australia, was killed in the crash near to the 16th milestone on the TT course on the Isle of Man. Mr Beare suffered fatal spine and back injuries on the west of the island on 4 June at about 14:10 BST. A verdict of misadventure was recorded by Coroner John Needham at Douglas Court House earlier. Mr Needham said: "I am not in a position to give an exact reason for the loss of control but Mr Beare would have died almost instantaneously". The inquest heard from a witness who said the carpenter and his passenger, Benjamin Binns, were thrown into the air after their vehicle hit the road side. Mr Binns, who was airlifted to hospital with a broken leg, said before the race both he and Mr Beare had been "excited and confident". "I will miss my friend dearly and his memory will live on," he said. No defects were found on the vehicle. Mr Beare moved to Onchan on the Isle of Man to pursue his road racing passion. He made his TT debut in 2014 finishing 12th in the second race of the week, with his father Noel as his passenger. He returned in 2015 when he came 17th.	`A TT racer died after being thrown from his sidecar when he hit the side of a road, an inquest has heard.`
`are yeezy adidas or nike?`	`Adidas Yeezy is a fashion collaboration between the German sportswear brand Adidas and American designer Kanye West. The collaboration has become notable for its high-end sneakers, and the Yeezy Boost sneaker line has been considered one of the most influential sneaker brands in the world.`
`what remains are changed into natural gas by heat and pressure change?`	`heat and pressure change the remains of prehistoric living things into natural gas. Dinosaurs and Other Prehistoric Creatures Dinosaurs are just one group of prehistoric animals. heat and pressure change the remains of dinosaurs into natural gas`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.025}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 320
per_device_eval_batch_size: 128
learning_rate: 2e-05
weight_decay: 0.0001
num_train_epochs: 2
lr_scheduler_type: cosine_with_restarts
lr_scheduler_kwargs: {'num_cycles': 3}
warmup_ratio: 0.25
save_safetensors: False
fp16: True
push_to_hub: True
hub_model_id: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-Ft-checkpoints-tmp
hub_strategy: all_checkpoints
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 320
per_device_eval_batch_size: 128
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0001
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: cosine_with_restarts
lr_scheduler_kwargs: {'num_cycles': 3}
warmup_ratio: 0.25
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: True
resume_from_checkpoint: None
hub_model_id: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-Ft-checkpoints-tmp
hub_strategy: all_checkpoints
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	loss	sts-test_spearman_cosine
0.0020	1	0.107	-	-
0.0039	2	0.1529	-	-
0.0059	3	0.1874	-	-
0.0078	4	0.1682	-	-
0.0098	5	0.1438	0.1470	0.9078
0.0117	6	0.2961	-	-
0.0137	7	0.3019	-	-
0.0157	8	0.1184	-	-
0.0176	9	0.3176	-	-
0.0196	10	0.2234	0.1468	0.9078
0.0215	11	0.1881	-	-
0.0235	12	0.1593	-	-
0.0254	13	0.1833	-	-
0.0274	14	0.1352	-	-
0.0294	15	0.3143	0.1462	0.9077
0.0313	16	0.1583	-	-
0.0333	17	0.2015	-	-
0.0352	18	0.1476	-	-
0.0372	19	0.1676	-	-
0.0391	20	0.1525	0.1454	0.9079
0.0411	21	0.1717	-	-
0.0431	22	0.198	-	-
0.0450	23	0.3062	-	-
0.0470	24	0.1241	-	-
0.0489	25	0.1087	0.1446	0.9082
0.0509	26	0.1767	-	-
0.0528	27	0.1951	-	-
0.0548	28	0.1621	-	-
0.0568	29	0.221	-	-
0.0587	30	0.2241	0.1435	0.9083
0.0607	31	0.2093	-	-
0.0626	32	0.1615	-	-
0.0646	33	0.1615	-	-
0.0665	34	0.1772	-	-
0.0685	35	0.2324	0.1423	0.9084
0.0705	36	0.2611	-	-
0.0724	37	0.214	-	-
0.0744	38	0.1985	-	-
0.0763	39	0.1855	-	-
0.0783	40	0.1234	0.1410	0.9085
0.0802	41	0.1492	-	-
0.0822	42	0.2022	-	-
0.0841	43	0.2146	-	-
0.0861	44	0.1688	-	-
0.0881	45	0.175	0.1396	0.9087
0.0900	46	0.2123	-	-
0.0920	47	0.1118	-	-
0.0939	48	0.3009	-	-
0.0959	49	0.1071	-	-
0.0978	50	0.2608	0.1382	0.9085
0.0998	51	0.1368	-	-
0.1018	52	0.2307	-	-
0.1037	53	0.1366	-	-
0.1057	54	0.1857	-	-
0.1076	55	0.2155	0.1367	0.9085
0.1096	56	0.2022	-	-
0.1115	57	0.2076	-	-
0.1135	58	0.4133	-	-
0.1155	59	0.1823	-	-
0.1174	60	0.1136	0.1353	0.9088
0.1194	61	0.1687	-	-
0.1213	62	0.1591	-	-
0.1233	63	0.1653	-	-
0.1252	64	0.1799	-	-
0.1272	65	0.1578	0.1337	0.9090
0.1292	66	0.1844	-	-
0.1311	67	0.1489	-	-
0.1331	68	0.1845	-	-
0.1350	69	0.1364	-	-
0.1370	70	0.1584	0.1321	0.9086
0.1389	71	0.2279	-	-
0.1409	72	0.2028	-	-
0.1429	73	0.2291	-	-
0.1448	74	0.2419	-	-
0.1468	75	0.1329	0.1306	0.9083
0.1487	76	0.204	-	-
0.1507	77	0.2239	-	-
0.1526	78	0.2181	-	-
0.1546	79	0.1285	-	-
0.1566	80	0.1067	0.1292	0.9079
0.1585	81	0.1189	-	-
0.1605	82	0.236	-	-
0.1624	83	0.1584	-	-
0.1644	84	0.1925	-	-
0.1663	85	0.129	0.1278	0.9079
0.1683	86	0.1376	-	-
0.1703	87	0.1691	-	-
0.1722	88	0.1045	-	-
0.1742	89	0.165	-	-
0.1761	90	0.2926	0.1267	0.9076
0.1781	91	0.1048	-	-
0.1800	92	0.1596	-	-
0.1820	93	0.2474	-	-
0.1840	94	0.1652	-	-
0.1859	95	0.2483	0.1253	0.9076
0.1879	96	0.1623	-	-
0.1898	97	0.1955	-	-
0.1918	98	0.2023	-	-
0.1937	99	0.1886	-	-
0.1957	100	0.1284	0.1229	0.9079
0.1977	101	0.2005	-	-
0.1996	102	0.2301	-	-
0.2016	103	0.2249	-	-

Framework Versions

Python: 3.10.14
Sentence Transformers: 3.0.1
Transformers: 4.44.0
PyTorch: 2.4.0
Accelerate: 0.33.0
Datasets: 2.21.0
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}