shivamgoel97/finetune

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'What was the new name given to the Administrative Charge on June 22, 2022?',
    'Context: customer. Do not tell the customer that the Customer Agreement only allows termination of a contract without an ETF if the change has a material adverse effect on the customer is to the calling plan. If the customer is undecided, the Supervisor/Support Coordinator must apply the credit and note the account accordingly. Refusal to Pay There is no "Refusal to Pay" process for the TUSC. Let customers know that, if they do not pay the surcharge, it continues to show as a balance owed on the account, which can cause interruption of their service for failing to pay billed charges.Table: <table border="1" cellpadding="5" cellspacing="0" width="100%"><tbody><tr><td>\xa0</td><td><b>6/22/22 Changes</b></td><td><b>12/1/22 Changes</b></td></tr><tr><td valign="top"><b>What Changed?</b></td><td valign="top">The Administrative Charge was renamed the Administrative and Telco Recovery Charge.<br/><br/>The charge was increased to $3.30 per voice line.\xa0</td><td valign="top">The charge increased to $1.40 per data-only line.<br/></td></tr><tr><td valign="top"><b>Customer Notifications</b></td><td valign="top">Beginning 5/23/2022 and running through 6/22/2022, the following appeared on all Consumer Bills:<br/><br/>Notice of Changes to Administrative Charge:<br/>Effective June 23, 2022, the monthly Verizon Wireless Administrative and Telco Recovery Charge (formerly the Verizon Wireless Administrative Charge) for voice-capable devices will increase from $1.95 to $3.30 per line. The charge for data-only devices will remain $0.06. The increased charge and updated name will appear on your next bill. For further information regarding the “Admin &amp; Telco Recovery Charge,” review the “Explanation of Surcharges” section of the bill.</td><td>Beginning 11/1/2022 and running through 11/30/2022, the following appeared on all Consumer Bills:<br/><br/>Notice of Changes to Administrative and Telco Recovery Charge:<br/>Effective December 1, 2022, the monthly Verizon Wireless Administrative and Telco Recovery Charge for data-only plans will increase from $0.06 to $1.40 per line. If you have a device on a data-only plan on your account, the increased charge will appear on your next bill. Customers on 5G Home internet and LTE Home Internet plans will not see a monthly increase. The charge for plans with voice service will remain $3.30 per line. For further information regarding the “Admin &amp; Telco Recovery Charge,” review the “Explanation of Surcharges” section of the bill.<br/><br/></td></tr><tr><td valign="top"><b>Positioning for Customer Inquiries</b></td><td valign="top">Mr/Ms Customer,<br/>I want to help you understand the Administrative and Telco Recovery Charge and other fees or surcharges on your bill.<br/>The notification that you received states that the Administrative and Telco Recovery Charge will increase by $1.35 per voice line to $3.30, effective your June bill.\xa0<br/><br/>Verizon reviews the Administrative and Telco Recovery Charge from time to time and makes adjustments to recover some of the administrative and telco expenses and costs of complying with regulatory requirements we incur to provide service.<br/>Other fees included on your bill are listed out by line item on your bill and you can reference them at any time. I would be happy to assist with any additional questions you might have on your bill.</td><td>Mr/Ms Customer,<br/>I want to help you understand the Administrative and Telco Recovery Charge and other fees or surcharges on your bill.<br/>The notification that you received states that the Administrative and Telco Recovery Charge will increase from $0.06 to $1.40 per data-only line, effective your December bill.<br/><br/>Verizon reviews the Administrative and Telco Recovery Charge from time to time and makes adjustments to recover some of the administrative and telco expenses and costs of complying with regulatory requirements we incur to provide service.<br/>Other fees included on your bill are listed out by line item on your bill and you can reference them at any time. I would be happy to assist with any additional questions you<br/>might have on your bill.</td></tr></tbody></table>',
    'To access Payment Hub for Retail Sales: Users do not need to submit an Automated CS Request (ACSR) for basic access. Use the Payment Hub application to submit a Payment Investigation on behalf of the customer. Retail sales can access Payment Hub in the following manner: Search InfoManager for ‘Payment Hub Tool’ My POS > External Services > Payment Hub Note: To report a Payment Hub app error, use either option below: Open a Self-Help Ticket and select category: Payment Hub Call IT Support at 866-899-4872 and a ticket is opened and dispatched to the Payment Hub Support team for resolution.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5718
cosine_accuracy@3	0.7685
cosine_accuracy@5	0.8271
cosine_accuracy@10	0.887
cosine_precision@1	0.5718
cosine_precision@3	0.2562
cosine_precision@5	0.1654
cosine_precision@10	0.0887
cosine_recall@1	0.5718
cosine_recall@3	0.7685
cosine_recall@5	0.8271
cosine_recall@10	0.887
cosine_ndcg@10	0.731
cosine_mrr@10	0.6809
cosine_map@100	0.685

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,999 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 10 tokens mean: 24.57 tokens max: 53 tokens	min: 95 tokens mean: 229.55 tokens max: 256 tokens

• Samples:

  sentence_0

  sentence_1

  What is the discount offered for customers who activate or upgrade to a new Smartphone on a Small Business Plan?

  Context: who activate or upgrade to a new Smartphone on a Small Business Plan. Upgrade eligible. $5 line access discount for customers who activate or upgrade to a new Smartphone on a Small Business Plan. Upgrade eligible. Upgrade eligible. Offer Options Code Code Description Discount Final Cost Contract Consumer Business Employees N/AAccess Line $5N/A1 Yr N/AAccess Line $5N/A2 Yr Offer Options Code Code Description Discount Final Cost Contract Consumer Business Employees N/AAccess Line $5N/A1 Yr N/AAccess Line $5N/A2 Yr Offer Options Offer Options Code Code Description Discount Final Cost Contract Consumer Business Employees N/AAccess Line $5N/A1 Yr N/AAccess Line $5N/A2 YrTable: What is the line access charge for a smartphone customer under contract who moves from a non-MORE Everything plan to a Small Business Plan after applying the $5 line access discount? Customers who activate or upgrade to a new smartphone on a Small Business Plan and sign a 1 or 2-year contract automatically receive a $5 line access discount: Available on all Small Business Plan 25, 50, 100 line plans. Available for business customers who are eligible to purchase a Small Business Plan. Smartphone customers under contract moving from a non-MORE Everything plan to a Small Business Plan automatically receive a $5 line access discount. The discount applies to qualifying MDNs while the customer remains on an eligible plan with an eligible device. Note: Smartphone line access charge discounted to $35 (after the $5 line access credit). Customer invoice reflects a $40 line access charge and a $5 promotional credit. The promotion code automatically attaches in all POS systems. Who is eligible to receive the $5 discount when changing to a Small Business Plan? Available for business customers eligible to purchase a Small Business Plan, including: State and Local Government Contracts where legally approved NASPO ValuePoint accounts Note: Discount is automatically applied to new activations, Add a Lines (AALs) and upgrades with a smartphone purchase and a 1 or 2-year contract. Existing customers currently under contract and on a non-MORE Everything plan who complete a plan change to a Small Business Plan automatically receive the $5 discount. Existing smartphone customers under contract and on eligible devices and plans may request the discount. Discount can be applied through all front-end systems and My Business. Loss: MultipleNegativesRankingLoss with these parameters: { "scale": 20.0, "similarity_fct": "cos_sim" } Training Hyperparameters Non-Default Hyperparameters eval_strategy: steps per_device_train_batch_size: 10 per_device_eval_batch_size: 10 num_train_epochs: 2 multi_dataset_batch_sampler: round_robin All Hyperparameters Click to expand overwrite_output_dir: False do_predict: False eval_strategy: steps prediction_loss_only: True per_device_train_batch_size: 10 per_device_eval_batch_size: 10 per_gpu_train_batch_size: None per_gpu_eval_batch_size: None gradient_accumulation_steps: 1 eval_accumulation_steps: None torch_empty_cache_steps: None learning_rate: 5e-05 weight_decay: 0.0 adam_beta1: 0.9 adam_beta2: 0.999 adam_epsilon: 1e-08 max_grad_norm: 1 num_train_epochs: 2 max_steps: -1 lr_scheduler_type: linear lr_scheduler_kwargs: {} warmup_ratio: 0.0 warmup_steps: 0 log_level: passive log_level_replica: warning log_on_each_node: True logging_nan_inf_filter: True save_safetensors: True save_on_each_node: False save_only_model: False restore_callback_states_from_checkpoint: False no_cuda: False use_cpu: False use_mps_device: False seed: 42 data_seed: None jit_mode_eval: False use_ipex: False bf16: False fp16: False fp16_opt_level: O1 half_precision_backend: auto bf16_full_eval: False fp16_full_eval: False tf32: None local_rank: 0 ddp_backend: None tpu_num_cores: None tpu_metrics_debug: False debug: [] dataloader_drop_last: False dataloader_num_workers: 0 dataloader_prefetch_factor: None past_index: -1 disable_tqdm: False remove_unused_columns: True label_names: None load_best_model_at_end: False ignore_data_skip: False fsdp: [] fsdp_min_num_params: 0 fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} fsdp_transformer_layer_cls_to_wrap: None accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} deepspeed: None label_smoothing_factor: 0.0 optim: adamw_torch optim_args: None adafactor: False group_by_length: False length_column_name: length ddp_find_unused_parameters: None ddp_bucket_cap_mb: None ddp_broadcast_buffers: False dataloader_pin_memory: True dataloader_persistent_workers: False skip_memory_metrics: True use_legacy_prediction_loop: False push_to_hub: False resume_from_checkpoint: None hub_model_id: None hub_strategy: every_save hub_private_repo: None hub_always_push: False gradient_checkpointing: False gradient_checkpointing_kwargs: None include_inputs_for_metrics: False include_for_metrics: [] eval_do_concat_batches: True fp16_backend: auto push_to_hub_model_id: None push_to_hub_organization: None mp_parameters: auto_find_batch_size: False full_determinism: False torchdynamo: None ray_scope: last ddp_timeout: 1800 torch_compile: False torch_compile_backend: None torch_compile_mode: None dispatch_batches: None split_batches: None include_tokens_per_second: False include_num_input_tokens_seen: False neftune_noise_alpha: None optim_target_modules: None batch_eval_metrics: False eval_on_start: False use_liger_kernel: False eval_use_gather_object: False average_tokens_across_devices: False prompts: None batch_sampler: batch_sampler multi_dataset_batch_sampler: round_robin Training Logs Epoch Step Training Loss cosine_ndcg@10 0.1 50 - 0.6653 0.2 100 - 0.6952 0.3 150 - 0.7010 0.4 200 - 0.7031 0.5 250 - 0.7038 0.6 300 - 0.7051 0.7 350 - 0.7110 0.8 400 - 0.7111 0.9 450 - 0.7151 1.0 500 0.1446 0.7231 1.1 550 - 0.7198 1.2 600 - 0.7192 1.3 650 - 0.7223 1.4 700 - 0.7207 1.5 750 - 0.7242 1.6 800 - 0.7264 1.7 850 - 0.7272 1.8 900 - 0.7287 1.9 950 - 0.7296 2.0 1000 0.068 0.7310 Framework Versions Python: 3.10.12 Sentence Transformers: 3.3.1 Transformers: 4.47.1 PyTorch: 2.5.1+cu121 Accelerate: 1.2.1 Datasets: 3.2.0 Tokenizers: 0.21.0 Citation BibTeX Sentence Transformers @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } MultipleNegativesRankingLoss @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} }