metadata

language:
  - en
license: apache-2.0
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:282883
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
datasets: []
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
widget:
  - source_sentence: Mwanamke anashona.
    sentences:
      - Mwanamke akishona blanketi pamoja.
      - Lakini Sir James alimkatiza.
      - >-
        Kwa kuongezea, wabunifu mashuhuri sasa wana maduka ya rejareja katika
        hoteli kadhaa za ununuzi.
  - source_sentence: Mtandao huwawezesha watu kununua vitu.
    sentences:
      - Mwanamke fulani anakata kipande cha jibini.
      - >-
        Hakuna hata mmoja wa wafadhili hawa anayeweza kuruhusu kuacha sasa,
        haswa na uchumi unaoteseka ndani na kitaifa.
      - >-
        Kwa vyovyote vile, kama ningekuwa na nia ya kununua kitabu hicho,
        ningekuwa na nafasi nzuri zaidi ya kujadili bei nzuri.
  - source_sentence: Je, kweli wewe ni hivyo gullible, Dave Hanson?
    sentences:
      - >-
        Mwanamume amesimama katika mashua paddling kuelekea pwani lined na
        fanicha na vitu vingine kubwa.
      - Dave Hanson, je, unaamini kila kitu wanachosema?
      - Wasichana watatu wakifanya mchezo wa kuigiza jukwaani.
  - source_sentence: Wanandoa wakitembea pamoja.
    sentences:
      - >-
        Mwanamume aliyevalia koti la manjano anatembea kando ya gari la kubebea
        watu.
      - Wenzi wa ndoa wazee wanatembea barabarani wakishikamana mikono.
      - Msichana mdogo anapanda kwenye kifaa cha kamba.
  - source_sentence: Kuna masuala ya sera.
    sentences:
      - >-
        Mwanamke mwenye makunyanzi sana akishikilia miwani yake na kutembea
        kwenye barabara ya jiji.
      - Mwanamume anayeigiza kwa ajili ya umati wa watu.
      - Masuala ya sera ya mbinu nyingi na maombi.
pipeline_tag: sentence-similarity
model-index:
  - name: BGE base Swahili Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.26803894120641386
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.3499618223466531
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.3858806312038687
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.43318910664291166
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.26803894120641386
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.11665394078221768
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.07717612624077373
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.043318910664291166
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.26803894120641386
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3499618223466531
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3858806312038687
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.43318910664291166
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.34611891078942064
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3188061049905684
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3251959746415499
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.26552557902774243
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.34601679816747266
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.3810766098243828
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.4290850089081191
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.26552557902774243
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.11533893272249085
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.07621532196487656
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.04290850089081191
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.26552557902774243
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.34601679816747266
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3810766098243828
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.4290850089081191
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3425120728009226
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.31538232445349546
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.32174207802147353
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.2576355306693815
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.33790404683125475
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.37165945533214556
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.41950878086026977
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.2576355306693815
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1126346822770849
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.07433189106642912
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.041950878086026974
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.2576355306693815
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.33790404683125475
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.37165945533214556
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.41950878086026977
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3338740008089949
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.30705069547968683
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3134101334652913
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.24494146093153474
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.3218694324255536
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.3557202850598117
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.402901501654365
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.24494146093153474
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.10728981080851785
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.07114405701196233
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0402901501654365
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.24494146093153474
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3218694324255536
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3557202850598117
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.402901501654365
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3191027723891013
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.2928823673781056
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.299205934269314
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.21936243318910664
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.2918045304148638
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.3234601679816747
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.3698778315092899
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.21936243318910664
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.0972681768049546
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.06469203359633495
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.03698778315092899
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.21936243318910664
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.2918045304148638
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3234601679816747
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.3698778315092899
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.2897472677253453
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.26472963050495346
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2710377326397304
            name: Cosine Map@100

BGE base Swahili Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sartifyllc/bge-base-swahili-matryoshka")
# Run inference
sentences = [
    'Kuna masuala ya sera.',
    'Masuala ya sera ya mbinu nyingi na maombi.',
    'Mwanamke mwenye makunyanzi sana akishikilia miwani yake na kutembea kwenye barabara ya jiji.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.268
cosine_accuracy@3	0.35
cosine_accuracy@5	0.3859
cosine_accuracy@10	0.4332
cosine_precision@1	0.268
cosine_precision@3	0.1167
cosine_precision@5	0.0772
cosine_precision@10	0.0433
cosine_recall@1	0.268
cosine_recall@3	0.35
cosine_recall@5	0.3859
cosine_recall@10	0.4332
cosine_ndcg@10	0.3461
cosine_mrr@10	0.3188
cosine_map@100	0.3252

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2655
cosine_accuracy@3	0.346
cosine_accuracy@5	0.3811
cosine_accuracy@10	0.4291
cosine_precision@1	0.2655
cosine_precision@3	0.1153
cosine_precision@5	0.0762
cosine_precision@10	0.0429
cosine_recall@1	0.2655
cosine_recall@3	0.346
cosine_recall@5	0.3811
cosine_recall@10	0.4291
cosine_ndcg@10	0.3425
cosine_mrr@10	0.3154
cosine_map@100	0.3217

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2576
cosine_accuracy@3	0.3379
cosine_accuracy@5	0.3717
cosine_accuracy@10	0.4195
cosine_precision@1	0.2576
cosine_precision@3	0.1126
cosine_precision@5	0.0743
cosine_precision@10	0.042
cosine_recall@1	0.2576
cosine_recall@3	0.3379
cosine_recall@5	0.3717
cosine_recall@10	0.4195
cosine_ndcg@10	0.3339
cosine_mrr@10	0.3071
cosine_map@100	0.3134

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2449
cosine_accuracy@3	0.3219
cosine_accuracy@5	0.3557
cosine_accuracy@10	0.4029
cosine_precision@1	0.2449
cosine_precision@3	0.1073
cosine_precision@5	0.0711
cosine_precision@10	0.0403
cosine_recall@1	0.2449
cosine_recall@3	0.3219
cosine_recall@5	0.3557
cosine_recall@10	0.4029
cosine_ndcg@10	0.3191
cosine_mrr@10	0.2929
cosine_map@100	0.2992

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2194
cosine_accuracy@3	0.2918
cosine_accuracy@5	0.3235
cosine_accuracy@10	0.3699
cosine_precision@1	0.2194
cosine_precision@3	0.0973
cosine_precision@5	0.0647
cosine_precision@10	0.037
cosine_recall@1	0.2194
cosine_recall@3	0.2918
cosine_recall@5	0.3235
cosine_recall@10	0.3699
cosine_ndcg@10	0.2897
cosine_mrr@10	0.2647
cosine_map@100	0.271

Training Details

Training Dataset

Unnamed Dataset

Size: 282,883 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 5 tokens
mean: 20.1 tokens
max: 78 tokens

min: 6 tokens
mean: 38.64 tokens
max: 184 tokens

	positive	anchor
type	string	string
details	min: 5 tokens mean: 20.1 tokens max: 78 tokens	min: 6 tokens mean: 38.64 tokens max: 184 tokens

Samples:

positive	anchor
`Alingoja mtu huyo mwingine arudi.`	`Ca'daan alingoja hadi alasiri yote mtu huyo atoke tena.`
`Sheria hiyo huanzisha mfululizo wa ukaguzi wa majaribio.`	`Sheria hiyo pia inatoa sheria ya kudhibiti kwa ajili ya mashirika fulani ambayo yanahitaji kuandaa taarifa za kifedha za mashirika yote na kuzisimamisha kwa wakaguzi wa jumla.`
`Mbwa anakimbia na kuruka nje.`	`Mbwa mwenye rangi ya kahawia anaruka na kukimbia shambani.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 4
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
0.0181	10	9.7089	-	-	-	-	-
0.0362	20	9.2806	-	-	-	-	-
0.0543	30	8.8905	-	-	-	-	-
0.0724	40	7.9651	-	-	-	-	-
0.0905	50	7.4201	-	-	-	-	-
0.1086	60	6.8346	-	-	-	-	-
0.1267	70	6.515	-	-	-	-	-
0.1448	80	6.2009	-	-	-	-	-
0.1629	90	5.8256	-	-	-	-	-
0.1810	100	5.549	-	-	-	-	-
0.1991	110	5.1667	-	-	-	-	-
0.2172	120	5.2684	-	-	-	-	-
0.2353	130	5.0678	-	-	-	-	-
0.2534	140	4.9183	-	-	-	-	-
0.2715	150	4.844	-	-	-	-	-
0.2896	160	4.5427	-	-	-	-	-
0.3077	170	4.3324	-	-	-	-	-
0.3258	180	4.4963	-	-	-	-	-
0.3439	190	4.1704	-	-	-	-	-
0.3620	200	4.1285	-	-	-	-	-
0.3800	210	4.0235	-	-	-	-	-
0.3981	220	4.0738	-	-	-	-	-
0.4162	230	3.9132	-	-	-	-	-
0.4343	240	3.9682	-	-	-	-	-
0.4524	250	3.7542	-	-	-	-	-
0.4705	260	3.6508	-	-	-	-	-
0.4886	270	3.7596	-	-	-	-	-
0.5067	280	3.5596	-	-	-	-	-
0.5248	290	3.5077	-	-	-	-	-
0.5429	300	3.3831	-	-	-	-	-
0.5610	310	3.4	-	-	-	-	-
0.5791	320	3.296	-	-	-	-	-
0.5972	330	3.3646	-	-	-	-	-
0.6153	340	3.3533	-	-	-	-	-
0.6334	350	3.2171	-	-	-	-	-
0.6515	360	3.2324	-	-	-	-	-
0.6696	370	3.1544	-	-	-	-	-
0.6877	380	3.3393	-	-	-	-	-
0.7058	390	3.0864	-	-	-	-	-
0.7239	400	3.1069	-	-	-	-	-
0.7420	410	3.0722	-	-	-	-	-
0.7601	420	3.1446	-	-	-	-	-
0.7782	430	3.0847	-	-	-	-	-
0.7963	440	3.0331	-	-	-	-	-
0.8144	450	3.0197	-	-	-	-	-
0.8325	460	2.9667	-	-	-	-	-
0.8506	470	2.8331	-	-	-	-	-
0.8687	480	2.9333	-	-	-	-	-
0.8868	490	2.8714	-	-	-	-	-
0.9049	500	2.8578	-	-	-	-	-
0.9230	510	2.9689	-	-	-	-	-
0.9411	520	2.7977	-	-	-	-	-
0.9592	530	2.9832	-	-	-	-	-
0.9773	540	2.9761	-	-	-	-	-
0.9954	550	2.7711	-	-	-	-	-
0.9990	552	-	0.2772	0.2954	0.3052	0.2445	0.3080
1.0135	560	2.7194	-	-	-	-	-
1.0316	570	2.8489	-	-	-	-	-
1.0497	580	2.6559	-	-	-	-	-
1.0678	590	2.6239	-	-	-	-	-
1.0859	600	2.7081	-	-	-	-	-
1.1039	610	2.6581	-	-	-	-	-
1.1220	620	2.7709	-	-	-	-	-
1.1401	630	2.6191	-	-	-	-	-
1.1582	640	2.6712	-	-	-	-	-
1.1763	650	2.5445	-	-	-	-	-
1.1944	660	2.5264	-	-	-	-	-
1.2125	670	2.5782	-	-	-	-	-
1.2306	680	2.5652	-	-	-	-	-
1.2487	690	2.6229	-	-	-	-	-
1.2668	700	2.5557	-	-	-	-	-
1.2849	710	2.5251	-	-	-	-	-
1.3030	720	2.4555	-	-	-	-	-
1.3211	730	2.5335	-	-	-	-	-
1.3392	740	2.5027	-	-	-	-	-
1.3573	750	2.3569	-	-	-	-	-
1.3754	760	2.4255	-	-	-	-	-
1.3935	770	2.4626	-	-	-	-	-
1.4116	780	2.363	-	-	-	-	-
1.4297	790	2.4	-	-	-	-	-
1.4478	800	2.3317	-	-	-	-	-
1.4659	810	2.2922	-	-	-	-	-
1.4840	820	2.4086	-	-	-	-	-
1.5021	830	2.3166	-	-	-	-	-
1.5202	840	2.3401	-	-	-	-	-
1.5383	850	2.1951	-	-	-	-	-
1.5564	860	2.214	-	-	-	-	-
1.5745	870	2.1859	-	-	-	-	-
1.5926	880	2.3605	-	-	-	-	-
1.6107	890	2.2528	-	-	-	-	-
1.6288	900	2.2759	-	-	-	-	-
1.6469	910	2.1458	-	-	-	-	-
1.6650	920	2.187	-	-	-	-	-
1.6831	930	2.3406	-	-	-	-	-
1.7012	940	2.2151	-	-	-	-	-
1.7193	950	2.2971	-	-	-	-	-
1.7374	960	2.2736	-	-	-	-	-
1.7555	970	2.2329	-	-	-	-	-
1.7736	980	2.2602	-	-	-	-	-
1.7917	990	2.2402	-	-	-	-	-
1.8098	1000	2.1971	-	-	-	-	-
1.8278	1010	2.1642	-	-	-	-	-
1.8459	1020	2.1274	-	-	-	-	-
1.8640	1030	2.1833	-	-	-	-	-
1.8821	1040	2.156	-	-	-	-	-
1.9002	1050	2.1252	-	-	-	-	-
1.9183	1060	2.161	-	-	-	-	-
1.9364	1070	2.1267	-	-	-	-	-
1.9545	1080	2.2017	-	-	-	-	-
1.9726	1090	2.3044	-	-	-	-	-
1.9907	1100	2.161	-	-	-	-	-
1.9998	1105	-	0.2928	0.3085	0.3165	0.2632	0.3204
2.0088	1110	2.0594	-	-	-	-	-
2.0269	1120	2.2277	-	-	-	-	-
2.0450	1130	2.1591	-	-	-	-	-
2.0631	1140	2.0396	-	-	-	-	-
2.0812	1150	2.1007	-	-	-	-	-
2.0993	1160	2.0705	-	-	-	-	-
2.1174	1170	2.0894	-	-	-	-	-
2.1355	1180	2.0677	-	-	-	-	-
2.1536	1190	2.0893	-	-	-	-	-
2.1717	1200	1.984	-	-	-	-	-
2.1898	1210	1.9206	-	-	-	-	-
2.2079	1220	2.132	-	-	-	-	-
2.2260	1230	2.0457	-	-	-	-	-
2.2441	1240	2.1428	-	-	-	-	-
2.2622	1250	2.1116	-	-	-	-	-
2.2803	1260	1.993	-	-	-	-	-
2.2984	1270	2.0181	-	-	-	-	-
2.3165	1280	1.9742	-	-	-	-	-
2.3346	1290	2.081	-	-	-	-	-
2.3527	1300	1.9107	-	-	-	-	-
2.3708	1310	1.9507	-	-	-	-	-
2.3889	1320	1.9844	-	-	-	-	-
2.4070	1330	2.0035	-	-	-	-	-
2.4251	1340	1.9121	-	-	-	-	-
2.4432	1350	2.0057	-	-	-	-	-
2.4613	1360	1.9323	-	-	-	-	-
2.4794	1370	1.9216	-	-	-	-	-
2.4975	1380	1.995	-	-	-	-	-
2.5156	1390	1.9285	-	-	-	-	-
2.5337	1400	1.8886	-	-	-	-	-
2.5517	1410	1.8298	-	-	-	-	-
2.5698	1420	1.8452	-	-	-	-	-
2.5879	1430	1.9488	-	-	-	-	-
2.6060	1440	1.8928	-	-	-	-	-
2.6241	1450	2.0101	-	-	-	-	-
2.6422	1460	1.7591	-	-	-	-	-
2.6603	1470	1.9177	-	-	-	-	-
2.6784	1480	1.9329	-	-	-	-	-
2.6965	1490	1.8978	-	-	-	-	-
2.7146	1500	1.9589	-	-	-	-	-
2.7327	1510	1.9744	-	-	-	-	-
2.7508	1520	1.9272	-	-	-	-	-
2.7689	1530	1.9234	-	-	-	-	-
2.7870	1540	1.9667	-	-	-	-	-
2.8051	1550	1.853	-	-	-	-	-
2.8232	1560	1.9191	-	-	-	-	-
2.8413	1570	1.8083	-	-	-	-	-
2.8594	1580	1.8543	-	-	-	-	-
2.8775	1590	1.9091	-	-	-	-	-
2.8956	1600	1.8079	-	-	-	-	-
2.9137	1610	1.8992	-	-	-	-	-
2.9318	1620	1.8742	-	-	-	-	-
2.9499	1630	1.9313	-	-	-	-	-
2.9680	1640	1.9832	-	-	-	-	-
2.9861	1650	1.9037	-	-	-	-	-
2.9988	1657	-	0.2982	0.3130	0.3211	0.2697	0.3247
3.0042	1660	1.7924	-	-	-	-	-
3.0223	1670	1.9677	-	-	-	-	-
3.0404	1680	1.9123	-	-	-	-	-
3.0585	1690	1.7691	-	-	-	-	-
3.0766	1700	1.8822	-	-	-	-	-
3.0947	1710	1.8543	-	-	-	-	-
3.1128	1720	1.8127	-	-	-	-	-
3.1309	1730	1.8844	-	-	-	-	-
3.1490	1740	1.911	-	-	-	-	-
3.1671	1750	1.7695	-	-	-	-	-
3.1852	1760	1.8134	-	-	-	-	-
3.2033	1770	1.7794	-	-	-	-	-
3.2214	1780	1.8851	-	-	-	-	-
3.2395	1790	1.8381	-	-	-	-	-
3.2576	1800	1.9184	-	-	-	-	-
3.2756	1810	1.8074	-	-	-	-	-
3.2937	1820	1.8236	-	-	-	-	-
3.3118	1830	1.8203	-	-	-	-	-
3.3299	1840	1.8874	-	-	-	-	-
3.3480	1850	1.7457	-	-	-	-	-
3.3661	1860	1.7933	-	-	-	-	-
3.3842	1870	1.759	-	-	-	-	-
3.4023	1880	1.8514	-	-	-	-	-
3.4204	1890	1.8163	-	-	-	-	-
3.4385	1900	1.8299	-	-	-	-	-
3.4566	1910	1.8112	-	-	-	-	-
3.4747	1920	1.7446	-	-	-	-	-
3.4928	1930	1.8314	-	-	-	-	-
3.5109	1940	1.742	-	-	-	-	-
3.5290	1950	1.7519	-	-	-	-	-
3.5471	1960	1.722	-	-	-	-	-
3.5652	1970	1.7454	-	-	-	-	-
3.5833	1980	1.7875	-	-	-	-	-
3.6014	1990	1.7596	-	-	-	-	-
3.6195	2000	1.8348	-	-	-	-	-
3.6376	2010	1.6954	-	-	-	-	-
3.6557	2020	1.7334	-	-	-	-	-
3.6738	2030	1.8318	-	-	-	-	-
3.6919	2040	1.7982	-	-	-	-	-
3.7100	2050	1.7987	-	-	-	-	-
3.7281	2060	1.8402	-	-	-	-	-
3.7462	2070	1.8569	-	-	-	-	-
3.7643	2080	1.8285	-	-	-	-	-
3.7824	2090	1.8652	-	-	-	-	-
3.8005	2100	1.7731	-	-	-	-	-
3.8186	2110	1.8697	-	-	-	-	-
3.8367	2120	1.6953	-	-	-	-	-
3.8548	2130	1.7493	-	-	-	-	-
3.8729	2140	1.8031	-	-	-	-	-
3.8910	2150	1.7053	-	-	-	-	-
3.9091	2160	1.8436	-	-	-	-	-
3.9272	2170	1.7572	-	-	-	-	-
3.9453	2180	1.7797	-	-	-	-	-
3.9634	2190	1.8827	-	-	-	-	-
3.9815	2200	1.8678	-	-	-	-	-
3.9959	2208	-	0.2992	0.3134	0.3217	0.271	0.3252

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.12
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2+cu121
Accelerate: 0.31.0
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}