--- base_model: agentlans/multilingual-e5-small-aligned library_name: sentence-transformers pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:3000000 - loss:CoSENTLoss widget: - source_sentence: Jesus answered them. sentences: - ישוע ענה להם. - आत्ताच नीघ. - Мы надеялись, что дождь прекратится до обеда. - source_sentence: Foreign books are sold at the shop. sentences: - Tak, det er alt. - Корабль бросил якорь. - Les livres étrangers sont vendus à la boutique. - source_sentence: Cats usually hate dogs. sentences: - Куда вы ходили в прошлое воскресенье? - >- The bottles of beer that I brought to the party were redundant; the host's family owned a brewery. - Mir tut der Arm weh. - source_sentence: How foolish I was not to discover that simple lie! sentences: - Tenho umas perguntas pra fazer, mas não quero te incomodar. - Mi piacciono di più le mele. - Quel idiot j'étais de n'avoir pas découvert ce simple mensonge ! - source_sentence: Esta es mi amiga Rachel, fuimos al instituto juntos. sentences: - Το σχολείο μας έχει εννιά τάξεις. - When applying to American universities, your TOEFL score is only one factor. - Je n'ai pas encore pris ma décision. license: mit --- # SentenceTransformer based on agentlans/multilingual-e5-small-aligned This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [agentlans/multilingual-e5-small-aligned](https://huggingface.co/agentlans/multilingual-e5-small-aligned). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. - One of the smallest multilingual embedding models on Huggingface - This model is aligned which means translations have similar embeddings compared to unrelated sentences - Finetuned on 1,000,000 randomly selected sentence pairs downloaded from Tatoeba 2024-09-26 - Includes pairs where one or both sentences are non-English - For each pair, two negative examples were generated ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [agentlans/multilingual-e5-small-aligned](https://huggingface.co/agentlans/multilingual-e5-small-aligned) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 384 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("agentlans/multilingual-e5-small-aligned-v2") # Run inference sentences = [ 'Esta es mi amiga Rachel, fuimos al instituto juntos.', "Je n'ai pas encore pris ma décision.", 'When applying to American universities, your TOEFL score is only one factor.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 3,000,000 training samples * Columns: sentence_0, sentence_1, and label * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | label | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | | | | * Samples: | sentence_0 | sentence_1 | label | |:------------------------------------------|:-----------------------------------------|:-----------------| | Bring your friends with you. | Traga seus amigos com você. | 1.0 | | I've been there already. | Você tem algo mais barato? | 0.0 | | All my homework is done. | माझा सगळा होमवर्क झाला आहे. | 1.0 | * Loss: [CoSENTLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "pairwise_cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: no - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 3 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs
Click to expand | Epoch | Step | Training Loss | |:------:|:------:|:-------------:| | 0.0053 | 500 | 0.835 | | 0.0107 | 1000 | 0.7012 | | 0.016 | 1500 | 0.6765 | | 0.0213 | 2000 | 0.4654 | | 0.0267 | 2500 | 0.7546 | | 0.032 | 3000 | 0.6098 | | 0.0373 | 3500 | 0.644 | | 0.0427 | 4000 | 0.5318 | | 0.048 | 4500 | 0.5638 | | 0.0533 | 5000 | 0.5556 | | 0.0587 | 5500 | 0.5165 | | 0.064 | 6000 | 0.4083 | | 0.0693 | 6500 | 0.4683 | | 0.0747 | 7000 | 0.5414 | | 0.08 | 7500 | 0.4678 | | 0.0853 | 8000 | 0.4225 | | 0.0907 | 8500 | 0.4552 | | 0.096 | 9000 | 0.4551 | | 0.1013 | 9500 | 0.4347 | | 0.1067 | 10000 | 0.292 | | 0.112 | 10500 | 0.4677 | | 0.1173 | 11000 | 0.3567 | | 0.1227 | 11500 | 0.4663 | | 0.128 | 12000 | 0.4333 | | 0.1333 | 12500 | 0.375 | | 0.1387 | 13000 | 0.4183 | | 0.144 | 13500 | 0.5745 | | 0.1493 | 14000 | 0.4569 | | 0.1547 | 14500 | 0.426 | | 0.16 | 15000 | 0.4903 | | 0.1653 | 15500 | 0.4287 | | 0.1707 | 16000 | 0.4375 | | 0.176 | 16500 | 0.377 | | 0.1813 | 17000 | 0.3848 | | 0.1867 | 17500 | 0.3366 | | 0.192 | 18000 | 0.3784 | | 0.1973 | 18500 | 0.399 | | 0.2027 | 19000 | 0.3798 | | 0.208 | 19500 | 0.3275 | | 0.2133 | 20000 | 0.3594 | | 0.2187 | 20500 | 0.3555 | | 0.224 | 21000 | 0.3565 | | 0.2293 | 21500 | 0.4264 | | 0.2347 | 22000 | 0.4138 | | 0.24 | 22500 | 0.3149 | | 0.2453 | 23000 | 0.3397 | | 0.2507 | 23500 | 0.359 | | 0.256 | 24000 | 0.3311 | | 0.2613 | 24500 | 0.3632 | | 0.2667 | 25000 | 0.366 | | 0.272 | 25500 | 0.2899 | | 0.2773 | 26000 | 0.2611 | | 0.2827 | 26500 | 0.3497 | | 0.288 | 27000 | 0.3534 | | 0.2933 | 27500 | 0.273 | | 0.2987 | 28000 | 0.3199 | | 0.304 | 28500 | 0.2527 | | 0.3093 | 29000 | 0.2755 | | 0.3147 | 29500 | 0.3684 | | 0.32 | 30000 | 0.347 | | 0.3253 | 30500 | 0.2537 | | 0.3307 | 31000 | 0.3665 | | 0.336 | 31500 | 0.2512 | | 0.3413 | 32000 | 0.2913 | | 0.3467 | 32500 | 0.2619 | | 0.352 | 33000 | 0.2573 | | 0.3573 | 33500 | 0.3036 | | 0.3627 | 34000 | 0.3388 | | 0.368 | 34500 | 0.2384 | | 0.3733 | 35000 | 0.31 | | 0.3787 | 35500 | 0.3461 | | 0.384 | 36000 | 0.378 | | 0.3893 | 36500 | 0.2409 | | 0.3947 | 37000 | 0.2969 | | 0.4 | 37500 | 0.2881 | | 0.4053 | 38000 | 0.3612 | | 0.4107 | 38500 | 0.2662 | | 0.416 | 39000 | 0.2796 | | 0.4213 | 39500 | 0.3298 | | 0.4267 | 40000 | 0.2828 | | 0.432 | 40500 | 0.2367 | | 0.4373 | 41000 | 0.2661 | | 0.4427 | 41500 | 0.393 | | 0.448 | 42000 | 0.2875 | | 0.4533 | 42500 | 0.203 | | 0.4587 | 43000 | 0.3211 | | 0.464 | 43500 | 0.3404 | | 0.4693 | 44000 | 0.315 | | 0.4747 | 44500 | 0.3018 | | 0.48 | 45000 | 0.2491 | | 0.4853 | 45500 | 0.2584 | | 0.4907 | 46000 | 0.2583 | | 0.496 | 46500 | 0.3447 | | 0.5013 | 47000 | 0.4332 | | 0.5067 | 47500 | 0.297 | | 0.512 | 48000 | 0.2697 | | 0.5173 | 48500 | 0.2349 | | 0.5227 | 49000 | 0.2176 | | 0.528 | 49500 | 0.2775 | | 0.5333 | 50000 | 0.2508 | | 0.5387 | 50500 | 0.291 | | 0.544 | 51000 | 0.2672 | | 0.5493 | 51500 | 0.2638 | | 0.5547 | 52000 | 0.2877 | | 0.56 | 52500 | 0.2758 | | 0.5653 | 53000 | 0.264 | | 0.5707 | 53500 | 0.2372 | | 0.576 | 54000 | 0.3384 | | 0.5813 | 54500 | 0.2459 | | 0.5867 | 55000 | 0.3047 | | 0.592 | 55500 | 0.1926 | | 0.5973 | 56000 | 0.2573 | | 0.6027 | 56500 | 0.2816 | | 0.608 | 57000 | 0.285 | | 0.6133 | 57500 | 0.2397 | | 0.6187 | 58000 | 0.1935 | | 0.624 | 58500 | 0.3281 | | 0.6293 | 59000 | 0.3306 | | 0.6347 | 59500 | 0.2067 | | 0.64 | 60000 | 0.2483 | | 0.6453 | 60500 | 0.2719 | | 0.6507 | 61000 | 0.2585 | | 0.656 | 61500 | 0.2385 | | 0.6613 | 62000 | 0.2229 | | 0.6667 | 62500 | 0.2311 | | 0.672 | 63000 | 0.2664 | | 0.6773 | 63500 | 0.209 | | 0.6827 | 64000 | 0.2643 | | 0.688 | 64500 | 0.2108 | | 0.6933 | 65000 | 0.3063 | | 0.6987 | 65500 | 0.1802 | | 0.704 | 66000 | 0.2285 | | 0.7093 | 66500 | 0.2065 | | 0.7147 | 67000 | 0.2467 | | 0.72 | 67500 | 0.2178 | | 0.7253 | 68000 | 0.2217 | | 0.7307 | 68500 | 0.2549 | | 0.736 | 69000 | 0.2026 | | 0.7413 | 69500 | 0.2609 | | 0.7467 | 70000 | 0.2393 | | 0.752 | 70500 | 0.1958 | | 0.7573 | 71000 | 0.2214 | | 0.7627 | 71500 | 0.2079 | | 0.768 | 72000 | 0.1574 | | 0.7733 | 72500 | 0.2356 | | 0.7787 | 73000 | 0.1864 | | 0.784 | 73500 | 0.257 | | 0.7893 | 74000 | 0.2149 | | 0.7947 | 74500 | 0.2519 | | 0.8 | 75000 | 0.2746 | | 0.8053 | 75500 | 0.2145 | | 0.8107 | 76000 | 0.2732 | | 0.816 | 76500 | 0.2456 | | 0.8213 | 77000 | 0.1841 | | 0.8267 | 77500 | 0.1876 | | 0.832 | 78000 | 0.2661 | | 0.8373 | 78500 | 0.1293 | | 0.8427 | 79000 | 0.2018 | | 0.848 | 79500 | 0.1854 | | 0.8533 | 80000 | 0.1644 | | 0.8587 | 80500 | 0.1844 | | 0.864 | 81000 | 0.1937 | | 0.8693 | 81500 | 0.1486 | | 0.8747 | 82000 | 0.244 | | 0.88 | 82500 | 0.131 | | 0.8853 | 83000 | 0.215 | | 0.8907 | 83500 | 0.2398 | | 0.896 | 84000 | 0.2014 | | 0.9013 | 84500 | 0.1703 | | 0.9067 | 85000 | 0.2009 | | 0.912 | 85500 | 0.1712 | | 0.9173 | 86000 | 0.2649 | | 0.9227 | 86500 | 0.2149 | | 0.928 | 87000 | 0.1912 | | 0.9333 | 87500 | 0.1902 | | 0.9387 | 88000 | 0.2609 | | 0.944 | 88500 | 0.1846 | | 0.9493 | 89000 | 0.1485 | | 0.9547 | 89500 | 0.2076 | | 0.96 | 90000 | 0.2449 | | 0.9653 | 90500 | 0.2025 | | 0.9707 | 91000 | 0.2635 | | 0.976 | 91500 | 0.2596 | | 0.9813 | 92000 | 0.2221 | | 0.9867 | 92500 | 0.2168 | | 0.992 | 93000 | 0.192 | | 0.9973 | 93500 | 0.1966 | | 1.0027 | 94000 | 0.2112 | | 1.008 | 94500 | 0.1628 | | 1.0133 | 95000 | 0.1059 | | 1.0187 | 95500 | 0.1403 | | 1.024 | 96000 | 0.1726 | | 1.0293 | 96500 | 0.1973 | | 1.0347 | 97000 | 0.1682 | | 1.04 | 97500 | 0.1319 | | 1.0453 | 98000 | 0.1427 | | 1.0507 | 98500 | 0.1448 | | 1.056 | 99000 | 0.1215 | | 1.0613 | 99500 | 0.1064 | | 1.0667 | 100000 | 0.0856 | | 1.072 | 100500 | 0.1046 | | 1.0773 | 101000 | 0.1127 | | 1.0827 | 101500 | 0.0988 | | 1.088 | 102000 | 0.1598 | | 1.0933 | 102500 | 0.1592 | | 1.0987 | 103000 | 0.1122 | | 1.104 | 103500 | 0.0771 | | 1.1093 | 104000 | 0.1355 | | 1.1147 | 104500 | 0.1265 | | 1.12 | 105000 | 0.1464 | | 1.1253 | 105500 | 0.1578 | | 1.1307 | 106000 | 0.1017 | | 1.1360 | 106500 | 0.1047 | | 1.1413 | 107000 | 0.1865 | | 1.1467 | 107500 | 0.1721 | | 1.152 | 108000 | 0.1096 | | 1.1573 | 108500 | 0.181 | | 1.1627 | 109000 | 0.1261 | | 1.168 | 109500 | 0.1111 | | 1.1733 | 110000 | 0.1286 | | 1.1787 | 110500 | 0.1014 | | 1.184 | 111000 | 0.1033 | | 1.1893 | 111500 | 0.1124 | | 1.1947 | 112000 | 0.1316 | | 1.2 | 112500 | 0.1147 | | 1.2053 | 113000 | 0.095 | | 1.2107 | 113500 | 0.1074 | | 1.216 | 114000 | 0.1183 | | 1.2213 | 114500 | 0.1219 | | 1.2267 | 115000 | 0.1264 | | 1.232 | 115500 | 0.1339 | | 1.2373 | 116000 | 0.0903 | | 1.2427 | 116500 | 0.0923 | | 1.248 | 117000 | 0.1028 | | 1.2533 | 117500 | 0.093 | | 1.2587 | 118000 | 0.1024 | | 1.264 | 118500 | 0.1107 | | 1.2693 | 119000 | 0.1078 | | 1.2747 | 119500 | 0.0469 | | 1.28 | 120000 | 0.107 | | 1.2853 | 120500 | 0.1578 | | 1.2907 | 121000 | 0.1012 | | 1.296 | 121500 | 0.064 | | 1.3013 | 122000 | 0.0816 | | 1.3067 | 122500 | 0.0656 | | 1.312 | 123000 | 0.1314 | | 1.3173 | 123500 | 0.1345 | | 1.3227 | 124000 | 0.1057 | | 1.328 | 124500 | 0.1051 | | 1.3333 | 125000 | 0.1246 | | 1.3387 | 125500 | 0.0827 | | 1.3440 | 126000 | 0.0763 | | 1.3493 | 126500 | 0.0887 | | 1.3547 | 127000 | 0.1332 | | 1.3600 | 127500 | 0.0939 | | 1.3653 | 128000 | 0.087 | | 1.3707 | 128500 | 0.0671 | | 1.376 | 129000 | 0.1377 | | 1.3813 | 129500 | 0.1066 | | 1.3867 | 130000 | 0.1224 | | 1.392 | 130500 | 0.0797 | | 1.3973 | 131000 | 0.0712 | | 1.4027 | 131500 | 0.1141 | | 1.408 | 132000 | 0.1045 | | 1.4133 | 132500 | 0.0894 | | 1.4187 | 133000 | 0.0897 | | 1.424 | 133500 | 0.0779 | | 1.4293 | 134000 | 0.0944 | | 1.4347 | 134500 | 0.0674 | | 1.44 | 135000 | 0.1532 | | 1.4453 | 135500 | 0.0771 | | 1.4507 | 136000 | 0.1154 | | 1.456 | 136500 | 0.1159 | | 1.4613 | 137000 | 0.147 | | 1.4667 | 137500 | 0.0925 | | 1.472 | 138000 | 0.0985 | | 1.4773 | 138500 | 0.1023 | | 1.4827 | 139000 | 0.082 | | 1.488 | 139500 | 0.0947 | | 1.4933 | 140000 | 0.0901 | | 1.4987 | 140500 | 0.127 | | 1.504 | 141000 | 0.1584 | | 1.5093 | 141500 | 0.0734 | | 1.5147 | 142000 | 0.1065 | | 1.52 | 142500 | 0.0568 | | 1.5253 | 143000 | 0.1081 | | 1.5307 | 143500 | 0.0727 | | 1.536 | 144000 | 0.1346 | | 1.5413 | 144500 | 0.0894 | | 1.5467 | 145000 | 0.0739 | | 1.552 | 145500 | 0.0926 | | 1.5573 | 146000 | 0.0984 | | 1.5627 | 146500 | 0.0975 | | 1.568 | 147000 | 0.0839 | | 1.5733 | 147500 | 0.1053 | | 1.5787 | 148000 | 0.1369 | | 1.584 | 148500 | 0.093 | | 1.5893 | 149000 | 0.1008 | | 1.5947 | 149500 | 0.0981 | | 1.6 | 150000 | 0.1071 | | 1.6053 | 150500 | 0.0955 | | 1.6107 | 151000 | 0.0901 | | 1.616 | 151500 | 0.0803 | | 1.6213 | 152000 | 0.1119 | | 1.6267 | 152500 | 0.0679 | | 1.6320 | 153000 | 0.1135 | | 1.6373 | 153500 | 0.0768 | | 1.6427 | 154000 | 0.0837 | | 1.6480 | 154500 | 0.0857 | | 1.6533 | 155000 | 0.0928 | | 1.6587 | 155500 | 0.0808 | | 1.6640 | 156000 | 0.0823 | | 1.6693 | 156500 | 0.0713 | | 1.6747 | 157000 | 0.0892 | | 1.6800 | 157500 | 0.0914 | | 1.6853 | 158000 | 0.0735 | | 1.6907 | 158500 | 0.0827 | | 1.696 | 159000 | 0.1006 | | 1.7013 | 159500 | 0.0837 | | 1.7067 | 160000 | 0.0812 | | 1.712 | 160500 | 0.1056 | | 1.7173 | 161000 | 0.0878 | | 1.7227 | 161500 | 0.0625 | | 1.728 | 162000 | 0.0965 | | 1.7333 | 162500 | 0.1121 | | 1.7387 | 163000 | 0.0794 | | 1.744 | 163500 | 0.0969 | | 1.7493 | 164000 | 0.0696 | | 1.7547 | 164500 | 0.083 | | 1.76 | 165000 | 0.0702 | | 1.7653 | 165500 | 0.0768 | | 1.7707 | 166000 | 0.0632 | | 1.776 | 166500 | 0.0714 | | 1.7813 | 167000 | 0.1 | | 1.7867 | 167500 | 0.0665 | | 1.792 | 168000 | 0.1139 | | 1.7973 | 168500 | 0.1032 | | 1.8027 | 169000 | 0.0983 | | 1.808 | 169500 | 0.0812 | | 1.8133 | 170000 | 0.0996 | | 1.8187 | 170500 | 0.0872 | | 1.8240 | 171000 | 0.0612 | | 1.8293 | 171500 | 0.1038 | | 1.8347 | 172000 | 0.0558 | | 1.8400 | 172500 | 0.0595 | | 1.8453 | 173000 | 0.0558 | | 1.8507 | 173500 | 0.0717 | | 1.8560 | 174000 | 0.058 | | 1.8613 | 174500 | 0.0745 | | 1.8667 | 175000 | 0.0749 | | 1.8720 | 175500 | 0.074 | | 1.8773 | 176000 | 0.0792 | | 1.8827 | 176500 | 0.0574 | | 1.888 | 177000 | 0.0968 | | 1.8933 | 177500 | 0.0755 | | 1.8987 | 178000 | 0.0852 | | 1.904 | 178500 | 0.0502 | | 1.9093 | 179000 | 0.0699 | | 1.9147 | 179500 | 0.0793 | | 1.92 | 180000 | 0.113 | | 1.9253 | 180500 | 0.0708 | | 1.9307 | 181000 | 0.0815 | | 1.936 | 181500 | 0.0962 | | 1.9413 | 182000 | 0.083 | | 1.9467 | 182500 | 0.0761 | | 1.952 | 183000 | 0.0776 | | 1.9573 | 183500 | 0.0811 | | 1.9627 | 184000 | 0.1159 | | 1.968 | 184500 | 0.081 | | 1.9733 | 185000 | 0.146 | | 1.9787 | 185500 | 0.0715 | | 1.984 | 186000 | 0.12 | | 1.9893 | 186500 | 0.0692 | | 1.9947 | 187000 | 0.07 | | 2.0 | 187500 | 0.0935 | | 2.0053 | 188000 | 0.0848 | | 2.0107 | 188500 | 0.0474 | | 2.016 | 189000 | 0.0417 | | 2.0213 | 189500 | 0.04 | | 2.0267 | 190000 | 0.1139 | | 2.032 | 190500 | 0.0553 | | 2.0373 | 191000 | 0.0495 | | 2.0427 | 191500 | 0.0613 | | 2.048 | 192000 | 0.0379 | | 2.0533 | 192500 | 0.0487 | | 2.0587 | 193000 | 0.0417 | | 2.064 | 193500 | 0.0249 | | 2.0693 | 194000 | 0.0418 | | 2.0747 | 194500 | 0.043 | | 2.08 | 195000 | 0.051 | | 2.0853 | 195500 | 0.0339 | | 2.0907 | 196000 | 0.0519 | | 2.096 | 196500 | 0.0878 | | 2.1013 | 197000 | 0.0432 | | 2.1067 | 197500 | 0.0185 | | 2.112 | 198000 | 0.085 | | 2.1173 | 198500 | 0.0601 | | 2.1227 | 199000 | 0.0935 | | 2.128 | 199500 | 0.0538 | | 2.1333 | 200000 | 0.0445 | | 2.1387 | 200500 | 0.0499 | | 2.144 | 201000 | 0.1029 | | 2.1493 | 201500 | 0.0758 | | 2.1547 | 202000 | 0.0648 | | 2.16 | 202500 | 0.0612 | | 2.1653 | 203000 | 0.0618 | | 2.1707 | 203500 | 0.0566 | | 2.176 | 204000 | 0.0179 | | 2.1813 | 204500 | 0.0557 | | 2.1867 | 205000 | 0.0321 | | 2.192 | 205500 | 0.0562 | | 2.1973 | 206000 | 0.0673 | | 2.2027 | 206500 | 0.0286 | | 2.208 | 207000 | 0.0284 | | 2.2133 | 207500 | 0.0595 | | 2.2187 | 208000 | 0.0693 | | 2.224 | 208500 | 0.065 | | 2.2293 | 209000 | 0.0546 | | 2.2347 | 209500 | 0.0467 | | 2.24 | 210000 | 0.0353 | | 2.2453 | 210500 | 0.0475 | | 2.2507 | 211000 | 0.0451 | | 2.2560 | 211500 | 0.0348 | | 2.2613 | 212000 | 0.031 | | 2.2667 | 212500 | 0.0294 | | 2.2720 | 213000 | 0.0462 | | 2.2773 | 213500 | 0.0376 | | 2.2827 | 214000 | 0.0607 | | 2.288 | 214500 | 0.041 | | 2.2933 | 215000 | 0.0462 | | 2.2987 | 215500 | 0.0285 | | 2.304 | 216000 | 0.0177 | | 2.3093 | 216500 | 0.0577 | | 2.3147 | 217000 | 0.0368 | | 2.32 | 217500 | 0.041 | | 2.3253 | 218000 | 0.0469 | | 2.3307 | 218500 | 0.0669 | | 2.336 | 219000 | 0.0288 | | 2.3413 | 219500 | 0.0283 | | 2.3467 | 220000 | 0.0293 | | 2.352 | 220500 | 0.0364 | | 2.3573 | 221000 | 0.0431 | | 2.3627 | 221500 | 0.0478 | | 2.368 | 222000 | 0.0223 | | 2.3733 | 222500 | 0.0464 | | 2.3787 | 223000 | 0.0598 | | 2.384 | 223500 | 0.0716 | | 2.3893 | 224000 | 0.0445 | | 2.3947 | 224500 | 0.0356 | | 2.4 | 225000 | 0.0344 | | 2.4053 | 225500 | 0.0729 | | 2.4107 | 226000 | 0.0256 | | 2.416 | 226500 | 0.0383 | | 2.4213 | 227000 | 0.0445 | | 2.4267 | 227500 | 0.0286 | | 2.432 | 228000 | 0.0216 | | 2.4373 | 228500 | 0.0299 | | 2.4427 | 229000 | 0.0674 | | 2.448 | 229500 | 0.0353 | | 2.4533 | 230000 | 0.0403 | | 2.4587 | 230500 | 0.0693 | | 2.464 | 231000 | 0.0701 | | 2.4693 | 231500 | 0.0506 | | 2.4747 | 232000 | 0.0374 | | 2.48 | 232500 | 0.0511 | | 2.4853 | 233000 | 0.047 | | 2.4907 | 233500 | 0.0231 | | 2.496 | 234000 | 0.0513 | | 2.5013 | 234500 | 0.0955 | | 2.5067 | 235000 | 0.049 | | 2.512 | 235500 | 0.048 | | 2.5173 | 236000 | 0.0302 | | 2.5227 | 236500 | 0.0207 | | 2.528 | 237000 | 0.0357 | | 2.5333 | 237500 | 0.0297 | | 2.5387 | 238000 | 0.0554 | | 2.544 | 238500 | 0.0386 | | 2.5493 | 239000 | 0.0249 | | 2.5547 | 239500 | 0.0432 | | 2.56 | 240000 | 0.0539 | | 2.5653 | 240500 | 0.0348 | | 2.5707 | 241000 | 0.0233 | | 2.576 | 241500 | 0.0702 | | 2.5813 | 242000 | 0.0393 | | 2.5867 | 242500 | 0.0625 | | 2.592 | 243000 | 0.0197 | | 2.5973 | 243500 | 0.0399 | | 2.6027 | 244000 | 0.0495 | | 2.608 | 244500 | 0.0407 | | 2.6133 | 245000 | 0.0412 | | 2.6187 | 245500 | 0.0234 | | 2.624 | 246000 | 0.0559 | | 2.6293 | 246500 | 0.0555 | | 2.6347 | 247000 | 0.0328 | | 2.64 | 247500 | 0.0375 | | 2.6453 | 248000 | 0.0257 | | 2.6507 | 248500 | 0.0212 | | 2.656 | 249000 | 0.0633 | | 2.6613 | 249500 | 0.0268 | | 2.6667 | 250000 | 0.0354 | | 2.672 | 250500 | 0.0341 | | 2.6773 | 251000 | 0.0337 | | 2.6827 | 251500 | 0.0519 | | 2.6880 | 252000 | 0.0386 | | 2.6933 | 252500 | 0.0603 | | 2.6987 | 253000 | 0.0358 | | 2.7040 | 253500 | 0.0352 | | 2.7093 | 254000 | 0.0448 | | 2.7147 | 254500 | 0.037 | | 2.7200 | 255000 | 0.0375 | | 2.7253 | 255500 | 0.04 | | 2.7307 | 256000 | 0.0729 | | 2.7360 | 256500 | 0.0246 | | 2.7413 | 257000 | 0.045 | | 2.7467 | 257500 | 0.0333 | | 2.752 | 258000 | 0.0212 | | 2.7573 | 258500 | 0.0458 | | 2.7627 | 259000 | 0.048 | | 2.768 | 259500 | 0.0287 | | 2.7733 | 260000 | 0.0345 | | 2.7787 | 260500 | 0.0459 | | 2.784 | 261000 | 0.0449 | | 2.7893 | 261500 | 0.0518 | | 2.7947 | 262000 | 0.0433 | | 2.8 | 262500 | 0.0572 | | 2.8053 | 263000 | 0.0357 | | 2.8107 | 263500 | 0.0394 | | 2.816 | 264000 | 0.0531 | | 2.8213 | 264500 | 0.0294 | | 2.8267 | 265000 | 0.039 | | 2.832 | 265500 | 0.0505 | | 2.8373 | 266000 | 0.0167 | | 2.8427 | 266500 | 0.031 | | 2.848 | 267000 | 0.0362 | | 2.8533 | 267500 | 0.0246 | | 2.8587 | 268000 | 0.0317 | | 2.864 | 268500 | 0.0296 | | 2.8693 | 269000 | 0.0297 | | 2.8747 | 269500 | 0.0517 | | 2.88 | 270000 | 0.019 | | 2.8853 | 270500 | 0.0358 | | 2.8907 | 271000 | 0.0589 | | 2.896 | 271500 | 0.031 | | 2.9013 | 272000 | 0.0421 | | 2.9067 | 272500 | 0.0422 | | 2.912 | 273000 | 0.016 | | 2.9173 | 273500 | 0.0645 | | 2.9227 | 274000 | 0.0514 | | 2.928 | 274500 | 0.0173 | | 2.9333 | 275000 | 0.0432 | | 2.9387 | 275500 | 0.0594 | | 2.944 | 276000 | 0.0228 | | 2.9493 | 276500 | 0.0152 | | 2.9547 | 277000 | 0.0579 | | 2.96 | 277500 | 0.0578 | | 2.9653 | 278000 | 0.0246 | | 2.9707 | 278500 | 0.0609 | | 2.976 | 279000 | 0.0613 | | 2.9813 | 279500 | 0.0589 | | 2.9867 | 280000 | 0.047 | | 2.992 | 280500 | 0.0264 | | 2.9973 | 281000 | 0.0464 |
### Framework Versions - Python: 3.10.12 - Sentence Transformers: 3.3.0 - Transformers: 4.46.3 - PyTorch: 2.5.1+cu124 - Accelerate: 1.1.1 - Datasets: 3.2.0 - Tokenizers: 0.20.3 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### CoSENTLoss ```bibtex @online{kexuefm-8847, title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT}, author={Su Jianlin}, year={2022}, month={Jan}, url={https://kexue.fm/archives/8847}, } ```