|
--- |
|
base_model: intfloat/multilingual-e5-base |
|
datasets: |
|
- rztk/rozetka_positive_pairs |
|
language: [] |
|
library_name: sentence-transformers |
|
metrics: |
|
- dot_accuracy@1 |
|
- dot_accuracy@3 |
|
- dot_accuracy@5 |
|
- dot_accuracy@10 |
|
- dot_precision@1 |
|
- dot_precision@3 |
|
- dot_precision@5 |
|
- dot_precision@10 |
|
- dot_recall@1 |
|
- dot_recall@3 |
|
- dot_recall@5 |
|
- dot_recall@10 |
|
- dot_ndcg@10 |
|
- dot_mrr@10 |
|
- dot_map@100 |
|
- dot_ndcg@1 |
|
- dot_mrr@1 |
|
pipeline_tag: sentence-similarity |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:44800 |
|
- loss:RZTKMatryoshka2dLoss |
|
widget: |
|
- source_sentence: папка планшет |
|
sentences: |
|
- <category>Сифони</category><brand>Alcaplast</brand><options><option_title>Гарантія</option_title><option_value>24 |
|
місяці офіційної гарантії від виробника</option_value><option_title>Кількість |
|
вантажних місць</option_title><option_value>1</option_value><option_title>Країна-виробник |
|
товару</option_title><option_value>Чехія</option_value><option_title>Призначення</option_title><option_value>Для |
|
душових піддонів</option_value><option_title>Матеріал</option_title><option_value>Пластик</option_value><option_title>Вид</option_title><option_value>Пляшковий</option_value><option_title>Під'єднані |
|
до пральної машини</option_title><option_value>Немає</option_value><option_title>Колір</option_title><option_value>Білий |
|
+ Хром</option_value><option_title>Тип</option_title><option_value>Сифон</option_value><option_title>Теги</option_title><option_value>недорогий |
|
сифон</option_value><option_title>відкривання/перекриття зливних отворів</option_title><option_value>Неперекривний</option_value><option_title>Різновид |
|
гідрозатвора</option_title><option_value>Мокрий (без мембрани)</option_value><option_title>Діаметр |
|
під'єднання</option_title><option_value>90 мм</option_value><option_title>Діаметр |
|
патрубка в каналізацію</option_title><option_value>40 мм</option_value><option_title>Переливання</option_title><option_value>Без |
|
функції переливу</option_value><option_title>Тип гарантійного талона</option_title><option_value>Гарантія |
|
по чеку</option_value><option_title>Доставка Premium</option_title><option_title>Доставка</option_title><option_value>Доставка |
|
в магазини ROZETKA</option_value></options> |
|
- Сифон для душевого поддона ALCA PLAST A49CR (8594045930627) |
|
- <category>Папки-планшеты</category><brand>iTEM</brand><options><option_title>Формат</option_title><option_value>A4</option_value><option_title>Материал</option_title><option_value>Картон</option_value><option_title>Страна |
|
регистрации бренда</option_title><option_value>Украина</option_value><option_title>Страна-производитель |
|
товара</option_title><option_value>Украина</option_value></options> |
|
- source_sentence: вино игристое |
|
sentences: |
|
- Женские резиновые сапоги Demar HAWAI LADY 0076V 36 (23.8 см) Черные (5901232011374) |
|
- Верстак складной Ryobi RWB01 |
|
- Вино ігристе Adamanti біле напівсолодке 0.75 л 12.5% (4860004073259) |
|
- source_sentence: елка искуственная |
|
sentences: |
|
- <category>Підставки та столики для ноутбуків</category><brand>UFT</brand><options><option_title>Вид</option_title><option_value>Столики</option_value><option_title>Охолодження</option_title><option_value>Активне</option_value><option_title>Максимальна |
|
діагональ ноутбука</option_title><option_value>16"</option_value><option_title>Колір</option_title><option_value>Синій</option_value><option_title>Матеріал</option_title><option_value>Метал</option_value><option_title>Кількість |
|
вантажних місць</option_title><option_value>1</option_value></options> |
|
- Декоративная елка, 90см (122-F12) |
|
- Конструктор LEGO Minecraft Гарбузова ферма 257 деталей (21248) |
|
- source_sentence: переходник |
|
sentences: |
|
- Штучна ялинка «Ніка» 1.8 м |
|
- Набір інструментів NEO торцевих головок 108 шт., 1, 4, 1/2 "CrV (08-666) |
|
- <category>Кабели и адаптеры</category><brand>Protech</brand><options><option_title>Гарантия</option_title><option_value>6 |
|
месяцев</option_value><option_title>Длина</option_title><option_value>0.2 м</option_value><option_title>Тип</option_title><option_value>Адаптеры |
|
(Переходники)</option_value><option_title>Количество грузовых мест</option_title><option_value>1</option_value><option_title>Страна |
|
регистрации бренда</option_title><option_value>Китай</option_value><option_title>Страна-производитель |
|
товара</option_title><option_value>Китай</option_value><option_title>Цвет</option_title><option_value>Серебристый</option_value><option_title>Тип |
|
гарантийного талона</option_title><option_value>Гарантия по чеку</option_value><option_title>Доставка |
|
Premium</option_title><option_title>Тип коннектора 1</option_title><option_value>USB |
|
Type-C</option_value><option_title>Тип коннектора 2</option_title><option_value>USB</option_value></options> |
|
- source_sentence: поилка для детей |
|
sentences: |
|
- Шафа розпашній Fenster Оксфорд Лагуна |
|
- <category>Аксессуары для наушников</category><brand>ArmorStandart</brand><options><option_title>Гарантия</option_title><option_value>14 |
|
дней</option_value><option_title>Тип наушников</option_title><option_value>Вкладыши</option_value><option_title>Вид</option_title><option_value>Чехлы</option_value><option_title>Цвет</option_title><option_value>Dark |
|
Green</option_value><option_title>Количество грузовых мест</option_title><option_value>1</option_value><option_title>Страна |
|
регистрации бренда</option_title><option_value>Украина</option_value><option_title>Страна-производитель |
|
товара</option_title><option_value>Китай</option_value><option_title>Тип гарантийного |
|
талона</option_title><option_value>Гарантия по чеку</option_value><option_title>Материал</option_title><option_value>Силикон</option_value><option_title>Доставка |
|
Premium</option_title><option_title>Совместимая серия</option_title><option_value>Apple |
|
AirPods</option_value><option_title>Доставка</option_title><option_value>Доставка |
|
в магазины ROZETKA</option_value></options> |
|
- <category>Поїльники та непроливайки</category><brand>Nuk</brand><options><option_title>Стать |
|
дитини</option_title><option_value>Хлопчик</option_value><option_title>Стать дитини</option_title><option_value>Дівчинка</option_value><option_title>Кількість |
|
вантажних місць</option_title><option_value>1</option_value><option_title>Країна |
|
реєстрації бренда</option_title><option_value>Німеччина</option_value><option_title>Країна-виробник |
|
товару</option_title><option_value>Німеччина</option_value><option_title>Об'єм, |
|
мл</option_title><option_value>300</option_value><option_title>Матеріал</option_title><option_value>Пластик</option_value><option_title>Колір</option_title><option_value>Блакитний</option_value><option_title>Тип</option_title><option_value>Поїльник</option_value><option_title>Тип |
|
гарантійного талона</option_title><option_value>Гарантія по чеку</option_value><option_title>Доставка |
|
Premium</option_title></options> |
|
model-index: |
|
- name: SentenceTransformer based on intfloat/multilingual-e5-base |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: rusisms uk title |
|
type: rusisms-uk-title |
|
metrics: |
|
- type: dot_accuracy@1 |
|
value: 0.5428571428571428 |
|
name: Dot Accuracy@1 |
|
- type: dot_accuracy@3 |
|
value: 0.6888888888888889 |
|
name: Dot Accuracy@3 |
|
- type: dot_accuracy@5 |
|
value: 0.7492063492063492 |
|
name: Dot Accuracy@5 |
|
- type: dot_accuracy@10 |
|
value: 0.8 |
|
name: Dot Accuracy@10 |
|
- type: dot_precision@1 |
|
value: 0.5428571428571428 |
|
name: Dot Precision@1 |
|
- type: dot_precision@3 |
|
value: 0.5216931216931217 |
|
name: Dot Precision@3 |
|
- type: dot_precision@5 |
|
value: 0.5034920634920634 |
|
name: Dot Precision@5 |
|
- type: dot_precision@10 |
|
value: 0.47682539682539676 |
|
name: Dot Precision@10 |
|
- type: dot_recall@1 |
|
value: 0.009248137199056617 |
|
name: Dot Recall@1 |
|
- type: dot_recall@3 |
|
value: 0.023803562659985587 |
|
name: Dot Recall@3 |
|
- type: dot_recall@5 |
|
value: 0.03509680885707945 |
|
name: Dot Recall@5 |
|
- type: dot_recall@10 |
|
value: 0.05987127144737185 |
|
name: Dot Recall@10 |
|
- type: dot_ndcg@10 |
|
value: 0.4936504584984999 |
|
name: Dot Ndcg@10 |
|
- type: dot_mrr@10 |
|
value: 0.6286608717561099 |
|
name: Dot Mrr@10 |
|
- type: dot_map@100 |
|
value: 0.14035920755466383 |
|
name: Dot Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: 'rusisms uk title matryoshka dim 768 ' |
|
type: rusisms-uk-title--matryoshka_dim-768-- |
|
metrics: |
|
- type: dot_accuracy@1 |
|
value: 0.1619047619047619 |
|
name: Dot Accuracy@1 |
|
- type: dot_precision@1 |
|
value: 0.1619047619047619 |
|
name: Dot Precision@1 |
|
- type: dot_recall@1 |
|
value: 0.0020219082190057404 |
|
name: Dot Recall@1 |
|
- type: dot_ndcg@1 |
|
value: 0.1619047619047619 |
|
name: Dot Ndcg@1 |
|
- type: dot_mrr@1 |
|
value: 0.1619047619047619 |
|
name: Dot Mrr@1 |
|
- type: dot_map@100 |
|
value: 0.02128340409566104 |
|
name: Dot Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: 'rusisms uk title matryoshka dim 512 ' |
|
type: rusisms-uk-title--matryoshka_dim-512-- |
|
metrics: |
|
- type: dot_accuracy@1 |
|
value: 0.14603174603174604 |
|
name: Dot Accuracy@1 |
|
- type: dot_precision@1 |
|
value: 0.14603174603174604 |
|
name: Dot Precision@1 |
|
- type: dot_recall@1 |
|
value: 0.0016964404522008209 |
|
name: Dot Recall@1 |
|
- type: dot_ndcg@1 |
|
value: 0.14603174603174604 |
|
name: Dot Ndcg@1 |
|
- type: dot_mrr@1 |
|
value: 0.14603174603174604 |
|
name: Dot Mrr@1 |
|
- type: dot_map@100 |
|
value: 0.015212846443877073 |
|
name: Dot Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: 'rusisms uk title matryoshka dim 256 ' |
|
type: rusisms-uk-title--matryoshka_dim-256-- |
|
metrics: |
|
- type: dot_accuracy@1 |
|
value: 0.10158730158730159 |
|
name: Dot Accuracy@1 |
|
- type: dot_precision@1 |
|
value: 0.10158730158730159 |
|
name: Dot Precision@1 |
|
- type: dot_recall@1 |
|
value: 0.0012653450153450154 |
|
name: Dot Recall@1 |
|
- type: dot_ndcg@1 |
|
value: 0.10158730158730159 |
|
name: Dot Ndcg@1 |
|
- type: dot_mrr@1 |
|
value: 0.10158730158730159 |
|
name: Dot Mrr@1 |
|
- type: dot_map@100 |
|
value: 0.011952854173853285 |
|
name: Dot Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: 'rusisms uk title matryoshka dim 128 ' |
|
type: rusisms-uk-title--matryoshka_dim-128-- |
|
metrics: |
|
- type: dot_accuracy@1 |
|
value: 0.05396825396825397 |
|
name: Dot Accuracy@1 |
|
- type: dot_precision@1 |
|
value: 0.05396825396825397 |
|
name: Dot Precision@1 |
|
- type: dot_recall@1 |
|
value: 0.0007494719994719994 |
|
name: Dot Recall@1 |
|
- type: dot_ndcg@1 |
|
value: 0.05396825396825397 |
|
name: Dot Ndcg@1 |
|
- type: dot_mrr@1 |
|
value: 0.05396825396825397 |
|
name: Dot Mrr@1 |
|
- type: dot_map@100 |
|
value: 0.0053781586003166125 |
|
name: Dot Map@100 |
|
--- |
|
|
|
# SentenceTransformer based on intfloat/multilingual-e5-base |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) on the [rztk/rozetka_positive_pairs](https://huggingface.co/datasets/rztk/rozetka_positive_pairs) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) <!-- at revision d13f1b27baf31030b7fd040960d60d909913633f --> |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Output Dimensionality:** 768 tokens |
|
- **Similarity Function:** Cosine Similarity |
|
- **Training Dataset:** |
|
- [rztk/rozetka_positive_pairs](https://huggingface.co/datasets/rztk/rozetka_positive_pairs) |
|
<!-- - **Language:** Unknown --> |
|
<!-- - **License:** Unknown --> |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel |
|
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
(2): Normalize() |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("sentence_transformers_model_id") |
|
# Run inference |
|
sentences = [ |
|
'поилка для детей', |
|
"<category>Поїльники та непроливайки</category><brand>Nuk</brand><options><option_title>Стать дитини</option_title><option_value>Хлопчик</option_value><option_title>Стать дитини</option_title><option_value>Дівчинка</option_value><option_title>Кількість вантажних місць</option_title><option_value>1</option_value><option_title>Країна реєстрації бренда</option_title><option_value>Німеччина</option_value><option_title>Країна-виробник товару</option_title><option_value>Німеччина</option_value><option_title>Об'єм, мл</option_title><option_value>300</option_value><option_title>Матеріал</option_title><option_value>Пластик</option_value><option_title>Колір</option_title><option_value>Блакитний</option_value><option_title>Тип</option_title><option_value>Поїльник</option_value><option_title>Тип гарантійного талона</option_title><option_value>Гарантія по чеку</option_value><option_title>Доставка Premium</option_title></options>", |
|
'Шафа розпашній Fenster Оксфорд Лагуна', |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 768] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
* Dataset: `rusisms-uk-title` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:-----------------|:-----------| |
|
| dot_accuracy@1 | 0.5429 | |
|
| dot_accuracy@3 | 0.6889 | |
|
| dot_accuracy@5 | 0.7492 | |
|
| dot_accuracy@10 | 0.8 | |
|
| dot_precision@1 | 0.5429 | |
|
| dot_precision@3 | 0.5217 | |
|
| dot_precision@5 | 0.5035 | |
|
| dot_precision@10 | 0.4768 | |
|
| dot_recall@1 | 0.0092 | |
|
| dot_recall@3 | 0.0238 | |
|
| dot_recall@5 | 0.0351 | |
|
| dot_recall@10 | 0.0599 | |
|
| dot_ndcg@10 | 0.4937 | |
|
| dot_mrr@10 | 0.6287 | |
|
| **dot_map@100** | **0.1404** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `rusisms-uk-title--matryoshka_dim-768--` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:----------------|:-----------| |
|
| dot_accuracy@1 | 0.1619 | |
|
| dot_precision@1 | 0.1619 | |
|
| dot_recall@1 | 0.002 | |
|
| dot_ndcg@1 | 0.1619 | |
|
| dot_mrr@1 | 0.1619 | |
|
| **dot_map@100** | **0.0213** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `rusisms-uk-title--matryoshka_dim-512--` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:----------------|:-----------| |
|
| dot_accuracy@1 | 0.146 | |
|
| dot_precision@1 | 0.146 | |
|
| dot_recall@1 | 0.0017 | |
|
| dot_ndcg@1 | 0.146 | |
|
| dot_mrr@1 | 0.146 | |
|
| **dot_map@100** | **0.0152** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `rusisms-uk-title--matryoshka_dim-256--` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:----------------|:----------| |
|
| dot_accuracy@1 | 0.1016 | |
|
| dot_precision@1 | 0.1016 | |
|
| dot_recall@1 | 0.0013 | |
|
| dot_ndcg@1 | 0.1016 | |
|
| dot_mrr@1 | 0.1016 | |
|
| **dot_map@100** | **0.012** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `rusisms-uk-title--matryoshka_dim-128--` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:----------------|:-----------| |
|
| dot_accuracy@1 | 0.054 | |
|
| dot_precision@1 | 0.054 | |
|
| dot_recall@1 | 0.0007 | |
|
| dot_ndcg@1 | 0.054 | |
|
| dot_mrr@1 | 0.054 | |
|
| **dot_map@100** | **0.0054** | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### rztk/rozetka_positive_pairs |
|
|
|
* Dataset: [rztk/rozetka_positive_pairs](https://huggingface.co/datasets/rztk/rozetka_positive_pairs) |
|
* Size: 44,800 training samples |
|
* Columns: <code>query</code> and <code>text</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | query | text | |
|
|:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 3 tokens</li><li>mean: 7.18 tokens</li><li>max: 16 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 158.88 tokens</li><li>max: 512 tokens</li></ul> | |
|
* Samples: |
|
| query | text | |
|
|:-----------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code>p smart z</code> | <code>TPU чехол Ultrathin Series 0,33 mm для Huawei P Smart Z Безбарвний (прозорий)</code> | |
|
| <code>p smart z</code> | <code><category>Чохли для мобільних телефонів</category><options><option_title>Матеріал</option_title><option_value>Силікон</option_value><option_title>Колір</option_title><option_value>Transparent</option_value><option_title>Сумісна модель</option_title><option_value>P Smart Z</option_value></options></code> | |
|
| <code>p smart z</code> | <code>TPU чехол Ultrathin Series 0,33mm для Huawei P Smart Z Бесцветный (прозрачный)</code> | |
|
* Loss: <code>sentence_transformers_training.model.matryoshka2d_loss.RZTKMatryoshka2dLoss</code> with these parameters: |
|
```json |
|
{ |
|
"loss": "RZTKMultipleNegativesRankingLoss", |
|
"n_layers_per_step": 1, |
|
"last_layer_weight": 1.0, |
|
"prior_layers_weight": 1.0, |
|
"kl_div_weight": 1.0, |
|
"kl_temperature": 0.3, |
|
"matryoshka_dims": [ |
|
768, |
|
512, |
|
256, |
|
128 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": 1 |
|
} |
|
``` |
|
|
|
### Evaluation Dataset |
|
|
|
#### rztk/rozetka_positive_pairs |
|
|
|
* Dataset: [rztk/rozetka_positive_pairs](https://huggingface.co/datasets/rztk/rozetka_positive_pairs) |
|
* Size: 4,480 evaluation samples |
|
* Columns: <code>query</code> and <code>text</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | query | text | |
|
|:--------|:---------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 3 tokens</li><li>mean: 6.29 tokens</li><li>max: 11 tokens</li></ul> | <ul><li>min: 12 tokens</li><li>mean: 161.36 tokens</li><li>max: 512 tokens</li></ul> | |
|
* Samples: |
|
| query | text | |
|
|:------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code>кошелек женский</code> | <code>Портмоне BAELLERRY Forever N2345 Черный (020354)</code> | |
|
| <code>кошелек женский</code> | <code><category>Гаманці</category><brand>Baellerry</brand><options><option_title>Для кого</option_title><option_value>Для жінок</option_value><option_title>Вид</option_title><option_value>Портмоне</option_value><option_title>Матеріал</option_title><option_value>Штучна шкіра</option_value><option_title>Країна-виробник товару</option_title><option_value>Китай</option_value></options></code> | |
|
| <code>кошелек женский</code> | <code>Портмоне BAELLERRY Forever N2345 Черный (020354)</code> | |
|
* Loss: <code>sentence_transformers_training.model.matryoshka2d_loss.RZTKMatryoshka2dLoss</code> with these parameters: |
|
```json |
|
{ |
|
"loss": "RZTKMultipleNegativesRankingLoss", |
|
"n_layers_per_step": 1, |
|
"last_layer_weight": 1.0, |
|
"prior_layers_weight": 1.0, |
|
"kl_div_weight": 1.0, |
|
"kl_temperature": 0.3, |
|
"matryoshka_dims": [ |
|
768, |
|
512, |
|
256, |
|
128 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": 1 |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: steps |
|
- `per_device_train_batch_size`: 112 |
|
- `per_device_eval_batch_size`: 112 |
|
- `torch_empty_cache_steps`: 30 |
|
- `learning_rate`: 2e-05 |
|
- `num_train_epochs`: 1.0 |
|
- `warmup_ratio`: 0.1 |
|
- `bf16`: True |
|
- `bf16_full_eval`: True |
|
- `tf32`: True |
|
- `dataloader_num_workers`: 2 |
|
- `load_best_model_at_end`: True |
|
- `optim`: adafactor |
|
- `push_to_hub`: True |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: steps |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 112 |
|
- `per_device_eval_batch_size`: 112 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 1 |
|
- `eval_accumulation_steps`: None |
|
- `torch_empty_cache_steps`: 30 |
|
- `learning_rate`: 2e-05 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 1.0 |
|
- `num_train_epochs`: 1.0 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: linear |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.1 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: True |
|
- `fp16`: False |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: True |
|
- `fp16_full_eval`: False |
|
- `tf32`: True |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: True |
|
- `dataloader_num_workers`: 2 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: True |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adafactor |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: True |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: False |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `eval_on_start`: False |
|
- `use_liger_kernel`: False |
|
- `eval_use_gather_object`: False |
|
- `batch_sampler`: batch_sampler |
|
- `multi_dataset_batch_sampler`: proportional |
|
- `ddp_static_graph`: False |
|
- `ddp_comm_hook`: bf16 |
|
- `gradient_as_bucket_view`: False |
|
|
|
</details> |
|
|
|
### Training Logs |
|
| Epoch | Step | Training Loss | loss | rusisms-uk-title--matryoshka_dim-128--_dot_map@100 | rusisms-uk-title--matryoshka_dim-256--_dot_map@100 | rusisms-uk-title--matryoshka_dim-512--_dot_map@100 | rusisms-uk-title--matryoshka_dim-768--_dot_map@100 | rusisms-uk-title_dot_map@100 | |
|
|:-------:|:------:|:-------------:|:----------:|:--------------------------------------------------:|:--------------------------------------------------:|:--------------------------------------------------:|:--------------------------------------------------:|:----------------------------:| |
|
| 0.1 | 10 | 6.6103 | - | - | - | - | - | - | |
|
| 0.2 | 20 | 5.524 | - | - | - | - | - | - | |
|
| 0.3 | 30 | 4.759 | 3.6444 | - | - | - | - | - | |
|
| 0.4 | 40 | 4.5195 | - | - | - | - | - | - | |
|
| 0.5 | 50 | 3.6598 | - | - | - | - | - | - | |
|
| 0.6 | 60 | 3.7912 | 2.8962 | - | - | - | - | - | |
|
| 0.7 | 70 | 3.9935 | - | - | - | - | - | - | |
|
| 0.8 | 80 | 3.3929 | - | - | - | - | - | - | |
|
| **0.9** | **90** | **3.6101** | **2.6889** | **-** | **-** | **-** | **-** | **-** | |
|
| 1.0 | 100 | 3.8753 | - | 0.0054 | 0.0120 | 0.0152 | 0.0213 | 0.1404 | |
|
|
|
* The bold row denotes the saved checkpoint. |
|
|
|
### Framework Versions |
|
- Python: 3.12.6 |
|
- Sentence Transformers: 3.0.1 |
|
- Transformers: 4.45.1 |
|
- PyTorch: 2.4.1 |
|
- Accelerate: 0.34.2 |
|
- Datasets: 3.0.0 |
|
- Tokenizers: 0.20.0 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |