SentenceTransformer
This is a sentence-transformers model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Maximum Sequence Length: 128 tokens
- Output Dimensionality: 768 dimensions
- Similarity Function: Cosine Similarity
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: RobertaModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("nikatonika/chatbot_biencoder")
# Run inference
sentences = [
'Does this have anything to do with Addie?',
'Lets say yes.',
'Check it out, no one will tell me where Emily is, so Im gonna send 72 longstem, red roses to Emilys parents house, one for',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
Evaluation
Metrics
Triplet
- Datasets:
dev_evaluator
andfinal_evaluator
- Evaluated with
TripletEvaluator
Metric | dev_evaluator | final_evaluator |
---|---|---|
cosine_accuracy | 0.5451 | 0.8827 |
Training Details
Training Dataset
Unnamed Dataset
- Size: 5,933 training samples
- Columns:
sentence_0
,sentence_1
, andsentence_2
- Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1 sentence_2 type string string string details - min: 3 tokens
- mean: 13.46 tokens
- max: 34 tokens
- min: 5 tokens
- mean: 19.83 tokens
- max: 51 tokens
- min: 3 tokens
- mean: 19.0 tokens
- max: 50 tokens
- Samples:
sentence_0 sentence_1 sentence_2 specifically told you not to assume . Can we at least assume that Im not dying tomorrow? Whereas this kid...
PET rEveals sEveral more hotspots. But theyre nonspecific...
Well, I did mention the Mars Rover incident to that FBI agent and probably cost Howard his security clearance.
How can we do that if we know youre not?
You dont know anything! Except, hopefully, our Patient on anticonvulsive medication has a seizure.
Now come on. Well, Im glad we worked things out.
Why? No way youre just doing her a favour.
ER is standing room only. Which means Camerons bound to make a mistake. Find it so I can blackmail her. As far as you know, this is way more than
You know what you should do? Take a vacation.
- Loss:
TripletLoss
with these parameters:{ "distance_metric": "TripletDistanceMetric.EUCLIDEAN", "triplet_margin": 5 }
Training Hyperparameters
Non-Default Hyperparameters
num_train_epochs
: 8multi_dataset_batch_sampler
: round_robin
All Hyperparameters
Click to expand
overwrite_output_dir
: Falsedo_predict
: Falseeval_strategy
: noprediction_loss_only
: Trueper_device_train_batch_size
: 8per_device_eval_batch_size
: 8per_gpu_train_batch_size
: Noneper_gpu_eval_batch_size
: Nonegradient_accumulation_steps
: 1eval_accumulation_steps
: Nonetorch_empty_cache_steps
: Nonelearning_rate
: 5e-05weight_decay
: 0.0adam_beta1
: 0.9adam_beta2
: 0.999adam_epsilon
: 1e-08max_grad_norm
: 1num_train_epochs
: 8max_steps
: -1lr_scheduler_type
: linearlr_scheduler_kwargs
: {}warmup_ratio
: 0.0warmup_steps
: 0log_level
: passivelog_level_replica
: warninglog_on_each_node
: Truelogging_nan_inf_filter
: Truesave_safetensors
: Truesave_on_each_node
: Falsesave_only_model
: Falserestore_callback_states_from_checkpoint
: Falseno_cuda
: Falseuse_cpu
: Falseuse_mps_device
: Falseseed
: 42data_seed
: Nonejit_mode_eval
: Falseuse_ipex
: Falsebf16
: Falsefp16
: Falsefp16_opt_level
: O1half_precision_backend
: autobf16_full_eval
: Falsefp16_full_eval
: Falsetf32
: Nonelocal_rank
: 0ddp_backend
: Nonetpu_num_cores
: Nonetpu_metrics_debug
: Falsedebug
: []dataloader_drop_last
: Falsedataloader_num_workers
: 0dataloader_prefetch_factor
: Nonepast_index
: -1disable_tqdm
: Falseremove_unused_columns
: Truelabel_names
: Noneload_best_model_at_end
: Falseignore_data_skip
: Falsefsdp
: []fsdp_min_num_params
: 0fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap
: Noneaccelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed
: Nonelabel_smoothing_factor
: 0.0optim
: adamw_torchoptim_args
: Noneadafactor
: Falsegroup_by_length
: Falselength_column_name
: lengthddp_find_unused_parameters
: Noneddp_bucket_cap_mb
: Noneddp_broadcast_buffers
: Falsedataloader_pin_memory
: Truedataloader_persistent_workers
: Falseskip_memory_metrics
: Trueuse_legacy_prediction_loop
: Falsepush_to_hub
: Falseresume_from_checkpoint
: Nonehub_model_id
: Nonehub_strategy
: every_savehub_private_repo
: Nonehub_always_push
: Falsegradient_checkpointing
: Falsegradient_checkpointing_kwargs
: Noneinclude_inputs_for_metrics
: Falseinclude_for_metrics
: []eval_do_concat_batches
: Truefp16_backend
: autopush_to_hub_model_id
: Nonepush_to_hub_organization
: Nonemp_parameters
:auto_find_batch_size
: Falsefull_determinism
: Falsetorchdynamo
: Noneray_scope
: lastddp_timeout
: 1800torch_compile
: Falsetorch_compile_backend
: Nonetorch_compile_mode
: Nonedispatch_batches
: Nonesplit_batches
: Noneinclude_tokens_per_second
: Falseinclude_num_input_tokens_seen
: Falseneftune_noise_alpha
: Noneoptim_target_modules
: Nonebatch_eval_metrics
: Falseeval_on_start
: Falseuse_liger_kernel
: Falseeval_use_gather_object
: Falseaverage_tokens_across_devices
: Falseprompts
: Nonebatch_sampler
: batch_samplermulti_dataset_batch_sampler
: round_robin
Training Logs
Epoch | Step | Training Loss | dev_evaluator_cosine_accuracy | final_evaluator_cosine_accuracy |
---|---|---|---|---|
-1 | -1 | - | 0.5451 | - |
0.6739 | 500 | 3.4522 | - | - |
1.3477 | 1000 | 1.8387 | - | - |
2.0216 | 1500 | 1.5216 | - | - |
2.6954 | 2000 | 1.0493 | - | - |
3.3693 | 2500 | 0.8555 | - | - |
4.0431 | 3000 | 0.7493 | - | - |
4.7170 | 3500 | 0.5685 | - | - |
5.3908 | 4000 | 0.503 | - | - |
6.0647 | 4500 | 0.3924 | - | - |
6.7385 | 5000 | 0.3252 | - | - |
7.4124 | 5500 | 0.29 | - | - |
-1 | -1 | - | - | 0.8827 |
0.6739 | 500 | 0.3696 | - | - |
1.3477 | 1000 | 0.4362 | - | - |
2.0216 | 1500 | 0.3908 | - | - |
2.6954 | 2000 | 0.2616 | - | - |
3.3693 | 2500 | 0.2105 | - | - |
4.0431 | 3000 | 0.1877 | - | - |
4.7170 | 3500 | 0.1406 | - | - |
5.3908 | 4000 | 0.1141 | - | - |
6.0647 | 4500 | 0.1136 | - | - |
6.7385 | 5000 | 0.0708 | - | - |
7.4124 | 5500 | 0.0638 | - | - |
Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.4.1
- Transformers: 4.48.3
- PyTorch: 2.5.1+cu124
- Accelerate: 1.3.0
- Datasets: 3.3.2
- Tokenizers: 0.21.0
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
TripletLoss
@misc{hermans2017defense,
title={In Defense of the Triplet Loss for Person Re-Identification},
author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
year={2017},
eprint={1703.07737},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
- Downloads last month
- 80
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
Evaluation results
- Cosine Accuracy on dev evaluatorself-reported0.545
- Cosine Accuracy on final evaluatorself-reported0.883