SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sgadagin/fine_tuned_sbert")
# Run inference
sentences = [
    'Security Event : Hack In Paris (16-17 June, 2011)\n\n\nHack In Paris is an international and corporate security event that will take place in Disneyland Paris® fromJune 16th to 17th of 2011. Please refer to the homepage to get up-to-date information about the event.\n\nTopics\nThe following list contains major topics the conference will cover. Please consider submitting even if the subject of your research is not listed here.\nAdvances in reverse engineering\nVulnerability research and exploitation\nPenetration testing and security assessment\nMalware analysis and new trends in malicous codes\nForensics, IT crime & law enforcement\nPrivacy issues: LOPPSI, HADOPI, …\nLow-level hacking (console security & mobile devices)\nRisk management and ISO 27001\nDates\nJanuary 20: CFP announced\nMarch 30: Submission deadline\nApril 15: Notification sent to authors\nApril 17: Program announcement\nJune 16-17: Hack In Paris\nJune 18: Nuit du Hack\nMore Information: https://hackinparis.com\n\n',
    'Security Event : Hack In Paris (16-17 June, 2011)\n\n\nHack In Paris is an international and corporate security event that will take place in Disneyland Paris® fromJune 16th to 17th of 2011. Please refer to the homepage to get up-to-date information about the event.\n\nTopics\nThe following list contains major topics the conference will cover. Please consider submitting even if the subject of your research is not listed here.\nAdvances in reverse engineering\nVulnerability research and exploitation\nPenetration testing and security assessment\nMalware analysis and new trends in malicous codes\nForensics, IT crime & law enforcement\nPrivacy issues: LOPPSI, HADOPI, …\nLow-level hacking (console security & mobile devices)\nRisk management and ISO 27001\nDates\nJanuary 20: CFP announced\nMarch 30: Submission deadline\nApril 15: Notification sent to authors\nApril 17: Program announcement\nJune 16-17: Hack In Paris\nJune 18: Nuit du Hack\nMore Information: https://hackinparis.com\n\n',
    'Google is going to shut down its social media network Google+ after the company suffered a massive data breach that exposed the private data of hundreds of thousands of Google Plus users to third-party developers.\n\nAccording to the tech giant, a security vulnerability in one of Google+\'s People APIs allowed third-party developers to access data for more than 500,000 users, including their usernames, email addresses, occupation, date of birth, profile photos, and gender-related information.\n\nSince Google+ servers do not keep API logs for more than two weeks, the company cannot confirm the number of users impacted by the vulnerability.\n\nHowever, Google assured its users that the company found no evidence that any developer was aware of this bug, or that the profile data was misused by any of the 438 developers that could have had access.\n"However, we ran a detailed analysis over the two weeks prior to patching the bug, and from that analysis, the Profiles of up to 500,000 Google+ accounts were potentially affected. Our analysis showed that up to 438 applications may have used this API," Google said in blog post published today.\nThe vulnerability was open since 2015 and fixed after Google discovered it in March 2018, but the company chose not to disclose the breach to the public—at the time when Facebook was being roasted for Cambridge Analytica scandal.\n\nThough Google has not revealed the technical details of the security vulnerability, the nature of the flaw seems to be something very similar to Facebook API flaw that recently allowed unauthorized developers to access private data from Facebook users.\n\nBesides admitting the security breach, Google also announced that the company is shutting down its social media network, acknowledging that Google+ failed to gain broad adoption or significant traction with consumers.\n"The consumer version of Google+ currently has low usage and engagement: 90 percent of Google+ user sessions are less than five seconds," Google said.\nIn response, the company has decided to shut down Google+ for consumers by the end of August 2019. However, Google+ will continue as a product for Enterprise users.\n\nGoogle Introduces New Privacy Controls Over Third-Party App Permissions\n\nAs part of its "Project Strobe," Google engineers also reviewed third-party developer access to Google account and Android device data; and has accordingly now introduced some new privacy controls.\n\nWhen a third-party app prompts users for access to their Google account data, clicking "Allow" button approves all requested permissions at once, leaving an opportunity for malicious apps to trick users into giving away powerful permissions.\nBut now Google has updated its Account Permissions system that asks for each requested permission individually rather than all at once, giving users more control over what type of account data they choose to share with each app.\n\nSince APIs can also allow developers to access users\' extremely sensitive data, like that of Gmail account, Google has limited access to Gmail API only for apps that directly enhance email functionality—such as email clients, email backup services and productivity services.\n\nGoogle shares fell over 2 percent to $1134.23 after the data breach reports.\n\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 3,742 training samples
Columns: sentence_0, sentence_1, and label

Approximate statistics based on the first 1000 samples:

	sentence_0	sentence_1	label
type	string	string	int
details	min: 37 tokens mean: 252.46 tokens max: 256 tokens	min: 37 tokens mean: 252.46 tokens max: 256 tokens	0: ~35.20% 1: ~10.30% 2: ~17.90% 3: ~36.60%

Samples:

sentence_0	sentence_1	label
U.S. online fashion retailer SHEIN has admitted that the company has suffered a significant data breach after unknown hackers stole personally identifiable information (PII) of almost 6.5 million customers. Based in North Brunswick and founded in 2008, SHEIN has become one of the largest online fashion retailers that ships to more than 80 countries worldwide. The site has been initially designed to produce "affordable" and trendy fashion clothing for women. SHEIN revealed last weekend that its servers had been targeted by a "concerted criminal cyber-attack" that began in June this year and lasted until August 22, when the company was finally made aware of the potential theft. Soon after that, the company scanned its servers to remove all possible backdoored entry points, leveraging which hackers could again infiltrate the servers. SHEIN assured its customers that the website is now safe to visit. Hackers Stole Over 6.42 Million SHEIN Customers' Data Although details about the inci...	U.S. online fashion retailer SHEIN has admitted that the company has suffered a significant data breach after unknown hackers stole personally identifiable information (PII) of almost 6.5 million customers. Based in North Brunswick and founded in 2008, SHEIN has become one of the largest online fashion retailers that ships to more than 80 countries worldwide. The site has been initially designed to produce "affordable" and trendy fashion clothing for women. SHEIN revealed last weekend that its servers had been targeted by a "concerted criminal cyber-attack" that began in June this year and lasted until August 22, when the company was finally made aware of the potential theft. Soon after that, the company scanned its servers to remove all possible backdoored entry points, leveraging which hackers could again infiltrate the servers. SHEIN assured its customers that the website is now safe to visit. Hackers Stole Over 6.42 Million SHEIN Customers' Data Although details about the inci...	`1`
A location based Social Networking platform with 45 million users,'Foursquare' was vulnerable to the primary email address disclosed. Foursquare is a Smartphone application that gives you details of nearby cafes, bars, shops, parks using GPS location and also tells about your friends nearby. According to a Penetration tester and hacker 'Jamal Eddine', an attacker can extract email addresses of all 45 million users just by using a few lines of scripting tool. Basically the flaw exists in the Invitation system of the Foursquare app. While testing the app, he found that invitation received on the recipient's end actually disclosing the sender's email address, as shown above. Invitation URL: https://foursquare.com/mehdi?action=acceptFriendship&expires=1378920415&src=wtbfe&uid=64761059&sig=mmlx96RwGrQ2fJAg4OWZhAWnDvc%3D Where 'uid' parameter represents the sender's profile ID. Hacker noticed that the parameter in the Invitation URL can be modified in order to spoof the sender profile i...	A location based Social Networking platform with 45 million users,'Foursquare' was vulnerable to the primary email address disclosed. Foursquare is a Smartphone application that gives you details of nearby cafes, bars, shops, parks using GPS location and also tells about your friends nearby. According to a Penetration tester and hacker 'Jamal Eddine', an attacker can extract email addresses of all 45 million users just by using a few lines of scripting tool. Basically the flaw exists in the Invitation system of the Foursquare app. While testing the app, he found that invitation received on the recipient's end actually disclosing the sender's email address, as shown above. Invitation URL: https://foursquare.com/mehdi?action=acceptFriendship&expires=1378920415&src=wtbfe&uid=64761059&sig=mmlx96RwGrQ2fJAg4OWZhAWnDvc%3D Where 'uid' parameter represents the sender's profile ID. Hacker noticed that the parameter in the Invitation URL can be modified in order to spoof the sender profile i...	`1`
Earlier this week Dropbox team unveiled details of three critical vulnerabilities in Apple macOS operating system, which altogether could allow a remote attacker to execute malicious code on a targeted Mac computer just by convincing a victim into visiting a malicious web page. The reported vulnerabilities were originally discovered by Syndis, a cybersecurity firm hired by Dropbox to conduct simulated penetration testing attacks as Red Team on the company's IT infrastructure, including Apple software used by Dropbox. The vulnerabilities were discovered and disclosed to Apple security team in February this year, which were then patched by Apple just over one month later with the release of its March security updates. DropBox applauded Apple for its quick response to its bug report. According to DropBox, the vulnerabilities discovered by Syndis didn't just affect its macOS fleet, but also affected all Safari users running the latest version of the web browser and operating system at t...	Earlier this week Dropbox team unveiled details of three critical vulnerabilities in Apple macOS operating system, which altogether could allow a remote attacker to execute malicious code on a targeted Mac computer just by convincing a victim into visiting a malicious web page. The reported vulnerabilities were originally discovered by Syndis, a cybersecurity firm hired by Dropbox to conduct simulated penetration testing attacks as Red Team on the company's IT infrastructure, including Apple software used by Dropbox. The vulnerabilities were discovered and disclosed to Apple security team in February this year, which were then patched by Apple just over one month later with the release of its March security updates. DropBox applauded Apple for its quick response to its bug report. According to DropBox, the vulnerabilities discovered by Syndis didn't just affect its macOS fleet, but also affected all Safari users running the latest version of the web browser and operating system at t...	`3`

Loss: SoftmaxLoss

Training Hyperparameters

Non-Default Hyperparameters

multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
1.0684	500	1.2186
2.1368	1000	1.145

Framework Versions

Python: 3.12.9
Sentence Transformers: 3.4.1
Transformers: 4.49.0
PyTorch: 2.6.0
Accelerate: 1.4.0
Datasets: 3.3.2
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers and SoftmaxLoss

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

sgadagin
/

fine_tuned_sbert