yahyaabd/allstats-search-miniLM-v1

SentenceTransformer based on sentence-transformers/paraphrase-MiniLM-L12-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-MiniLM-L12-v2 on the query-hard-pos-neg-doc-pairs-statictable dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-MiniLM-L12-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • query-hard-pos-neg-doc-pairs-statictable

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("yahyaabd/allstats-search-miniLM-v1")
# Run inference
sentences = [
    'Bagaimana perkembangan koperasi di Indonesia, khususnya sekitar tayun 2000?',
    'IHK dan Rata-rata Upah per Bulan Buruh Industri di Bawah Mandor (Supervisor), 1996-2014 (1996=100)',
    'Rata-Rata Harian Aliran Sungai, Tinggi Aliran, dan Volume Air di Beberapa Sungai yang Daerah Pengalirannya Lebih dari 1.000 km2, 2000-2011',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

• Datasets: allstats-semantic-mini-v1-eval and allstat-search-mini-v1-test
• Evaluated with EmbeddingSimilarityEvaluator

Metric	allstats-semantic-mini-v1-eval	allstat-search-mini-v1-test
pearson_cosine	0.848	0.8538
spearman_cosine	0.7746	0.7768

Training Details

Training Dataset

query-hard-pos-neg-doc-pairs-statictable

• Dataset: query-hard-pos-neg-doc-pairs-statictable at 25756d3
• Size: 25,551 training samples
• Columns: query, doc, and label
• Approximate statistics based on the first 1000 samples:

  	query	doc	label
  type	string	string	int
  details	min: 9 tokens mean: 28.64 tokens max: 53 tokens	min: 11 tokens mean: 36.67 tokens max: 70 tokens	0: ~65.80% 1: ~34.20%

• Samples:

  query	doc	label
  Gaji nominal, indeks upah: nominal & riil pekerja manufaktur non-mandor (2012=100), 2013-2014	Ringkasan Neraca Arus Dana, Triwulan I, 2007, (Miliar Rupiah)	0
  gaji nominal, indeks upah: nominal & riil pekerja manufaktur non-mandor (2012=100), 2013-2014	Ringkasan Neraca Arus Dana, Triwulan I, 2007, (Miliar Rupiah)	0
  GAJI NOMINAL, INDEKS UPAH: NOMINAL & RIIL PEKERJA MANUFAKTUR NON-MANDOR (2012=100), 2013-2014	Ringkasan Neraca Arus Dana, Triwulan I, 2007, (Miliar Rupiah)	0

• Loss: OnlineContrastiveLoss

Evaluation Dataset

query-hard-pos-neg-doc-pairs-statictable

• Dataset: query-hard-pos-neg-doc-pairs-statictable at 25756d3
• Size: 5,463 evaluation samples
• Columns: query, doc, and label
• Approximate statistics based on the first 1000 samples:

  	query	doc	label
  type	string	string	int
  details	min: 10 tokens mean: 29.3 tokens max: 62 tokens	min: 12 tokens mean: 37.1 tokens max: 69 tokens	0: ~73.20% 1: ~26.80%

• Samples:

  query	doc	label
  Bagaimana penghasilan wirausahawan di Indonesia bervariasi per provinsi dan jenis pekerjaan utama di tahun 2016?	Realisasi Penerimaan dan Pengeluaran Pemerintah Desa (Juta Rupiah) di Perkotaan menurut Provinsi, 2000-2012	0
  bagaimana penghasilan wirausahawan di indonesia bervariasi per provinsi dan jenis pekerjaan utama di tahun 2016?	Realisasi Penerimaan dan Pengeluaran Pemerintah Desa (Juta Rupiah) di Perkotaan menurut Provinsi, 2000-2012	0
  BAGAIMANA PENGHASILAN WIRAUSAHAWAN DI INDONESIA BERVARIASI PER PROVINSI DAN JENIS PEKERJAAN UTAMA DI TAHUN 2016?	Realisasi Penerimaan dan Pengeluaran Pemerintah Desa (Juta Rupiah) di Perkotaan menurut Provinsi, 2000-2012	0

• Loss: OnlineContrastiveLoss

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 4
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True
• eval_on_start: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	allstats-semantic-mini-v1-eval_spearman_cosine	allstat-search-mini-v1-test_spearman_cosine
0	0	-	1.0797	0.5314	-
0.0250	20	1.2823	0.9331	0.5510	-
0.0501	40	0.9562	0.6159	0.6492	-
0.0751	60	0.5872	0.4629	0.6913	-
0.1001	80	0.4101	0.3605	0.7221	-
0.1252	100	0.419	0.3919	0.7301	-
0.1502	120	0.1517	0.2565	0.7457	-
0.1752	140	0.2678	0.2503	0.7484	-
0.2003	160	0.225	0.2010	0.7546	-
0.2253	180	0.2846	0.3203	0.7420	-
0.2503	200	0.2086	0.1981	0.7589	-
0.2753	220	0.1255	0.1982	0.7610	-
0.3004	240	0.1182	0.2328	0.7583	-
0.3254	260	0.1328	0.2218	0.7561	-
0.3504	280	0.1228	0.4583	0.7343	-
0.3755	300	0.1394	0.1785	0.7705	-
0.4005	320	0.2577	0.1800	0.7650	-
0.4255	340	0.1903	0.2680	0.7557	-
0.4506	360	0.1164	0.1761	0.7616	-
0.4756	380	0.0779	0.3318	0.7453	-
0.5006	400	0.1563	0.2209	0.7582	-
0.5257	420	0.1835	0.1683	0.7662	-
0.5507	440	0.1171	0.1537	0.7658	-
0.5757	460	0.0973	0.1381	0.7710	-
0.6008	480	0.0578	0.2303	0.7618	-
0.6258	500	0.1343	0.1431	0.7710	-
0.6508	520	0.1274	0.1646	0.7695	-
0.6758	540	0.057	0.1775	0.7606	-
0.7009	560	0.0392	0.1425	0.7689	-
0.7259	580	0.0434	0.1399	0.7712	-
0.7509	600	0.1311	0.1747	0.7670	-
0.7760	620	0.0475	0.1375	0.7709	-
0.8010	640	0.0183	0.1465	0.7685	-
0.8260	660	0.024	0.1666	0.7669	-
0.8511	680	0.0249	0.1728	0.7656	-
0.8761	700	0.041	0.1624	0.7711	-
0.9011	720	0.0835	0.1397	0.7716	-
0.9262	740	0.0404	0.1507	0.7693	-
0.9512	760	0.0141	0.1369	0.7723	-
0.9762	780	0.0513	0.1555	0.7687	-
1.0013	800	0.0387	0.1306	0.7717	-
1.0263	820	0.0393	0.1420	0.7707	-
1.0513	840	0.0153	0.1656	0.7700	-
1.0763	860	0.0263	0.1525	0.7694	-
1.1014	880	0.0503	0.1947	0.7638	-
1.1264	900	0.0215	0.2202	0.7615	-
1.1514	920	0.0217	0.1542	0.7696	-
1.1765	940	0.007	0.1394	0.7713	-
1.2015	960	0.018	0.1573	0.7706	-
1.2265	980	0.0446	0.1504	0.7686	-
1.2516	1000	0.026	0.1573	0.7661	-
1.2766	1020	0.0098	0.1429	0.7683	-
1.3016	1040	0.0196	0.1374	0.7702	-
1.3267	1060	0.021	0.1594	0.7685	-
1.3517	1080	0.0499	0.1378	0.7724	-
1.3767	1100	0.0165	0.1335	0.7729	-
1.4018	1120	0.0294	0.1451	0.7713	-
1.4268	1140	0.0114	0.1338	0.7717	-
1.4518	1160	0.0192	0.1327	0.7719	-
1.4768	1180	0.0335	0.1618	0.7646	-
1.5019	1200	0.0546	0.1389	0.7711	-
1.5269	1220	0.0069	0.1239	0.7738	-
1.5519	1240	0.0094	0.1180	0.7739	-
1.5770	1260	0.0074	0.1238	0.7733	-
1.6020	1280	0.0557	0.1428	0.7720	-
1.6270	1300	0.056	0.1159	0.7751	-
1.6521	1320	0.0	0.1244	0.7758	-
1.6771	1340	0.0066	0.1185	0.7735	-
1.7021	1360	0.0178	0.1016	0.7757	-
1.7272	1380	0.0156	0.0939	0.7776	-
1.7522	1400	0.0	0.1138	0.7761	-
1.7772	1420	0.0436	0.0980	0.7775	-
1.8023	1440	0.0626	0.1096	0.7763	-
1.8273	1460	0.0222	0.0968	0.7774	-
1.8523	1480	0.0101	0.1021	0.7762	-
1.8773	1500	0.0171	0.1076	0.7754	-
1.9024	1520	0.0064	0.1279	0.7730	-
1.9274	1540	0.0068	0.1237	0.7729	-
1.9524	1560	0.0066	0.1229	0.7733	-
1.9775	1580	0.0	0.1263	0.7731	-
2.0025	1600	0.0065	0.1152	0.7746	-
2.0275	1620	0.0147	0.1021	0.7773	-
2.0526	1640	0.0	0.1021	0.7773	-
2.0776	1660	0.0209	0.1017	0.7774	-
2.1026	1680	0.0	0.0993	0.7773	-
2.1277	1700	0.0067	0.0922	0.7784	-
2.1527	1720	0.0333	0.1158	0.7749	-
2.1777	1740	0.0	0.1397	0.7721	-
2.2028	1760	0.0158	0.1248	0.7751	-
2.2278	1780	0.0201	0.1021	0.7767	-
2.2528	1800	0.0	0.1029	0.7768	-
2.2778	1820	0.0107	0.1007	0.7767	-
2.3029	1840	0.0156	0.0923	0.7767	-
2.3279	1860	0.0	0.1012	0.7754	-
2.3529	1880	0.0131	0.1184	0.7731	-
2.3780	1900	0.0072	0.1113	0.7752	-
2.4030	1920	0.0337	0.0952	0.7775	-
2.4280	1940	0.0068	0.1086	0.7754	-
2.4531	1960	0.0	0.1194	0.7740	-
2.4781	1980	0.0176	0.1184	0.7747	-
2.5031	2000	0.0188	0.1123	0.7745	-
2.5282	2020	0.0	0.1138	0.7742	-
2.5532	2040	0.0	0.1141	0.7742	-
2.5782	2060	0.0269	0.1126	0.7743	-
2.6033	2080	0.0193	0.1470	0.7707	-
2.6283	2100	0.0074	0.1333	0.7726	-
2.6533	2120	0.0253	0.1004	0.7756	-
2.6783	2140	0.0	0.0980	0.7758	-
2.7034	2160	0.0	0.0984	0.7758	-
2.7284	2180	0.0	0.0984	0.7758	-
2.7534	2200	0.0	0.0984	0.7758	-
2.7785	2220	0.007	0.0971	0.7766	-
2.8035	2240	0.0	0.0998	0.7766	-
2.8285	2260	0.015	0.0988	0.7760	-
2.8536	2280	0.0	0.1020	0.7757	-
2.8786	2300	0.0	0.1023	0.7756	-
2.9036	2320	0.0	0.1023	0.7756	-
2.9287	2340	0.0	0.1023	0.7756	-
2.9537	2360	0.0075	0.1043	0.7751	-
2.9787	2380	0.0067	0.1125	0.7749	-
3.0038	2400	0.0	0.1083	0.7752	-
3.0288	2420	0.0	0.1083	0.7752	-
3.0538	2440	0.0	0.1083	0.7752	-
3.0788	2460	0.0063	0.1018	0.7755	-
3.1039	2480	0.0	0.1012	0.7756	-
3.1289	2500	0.0162	0.092	0.7768	-
3.1539	2520	0.01	0.0941	0.7768	-
3.1790	2540	0.0069	0.0946	0.7761	-
3.2040	2560	0.0	0.0956	0.7759	-
3.2290	2580	0.0	0.0956	0.7758	-
3.2541	2600	0.0	0.0956	0.7758	-
3.2791	2620	0.0	0.0956	0.7758	-
3.3041	2640	0.0131	0.0981	0.7756	-
3.3292	2660	0.0195	0.1142	0.7748	-
3.3542	2680	0.0	0.1172	0.7746	-
3.3792	2700	0.0065	0.1186	0.7748	-
3.4043	2720	0.0169	0.1184	0.7750	-
3.4293	2740	0.0	0.1175	0.7749	-
3.4543	2760	0.0	0.1165	0.7748	-
3.4793	2780	0.0105	0.1173	0.7747	-
3.5044	2800	0.0066	0.1123	0.7751	-
3.5294	2820	0.0	0.1103	0.7753	-
3.5544	2840	0.0	0.1106	0.7753	-
3.5795	2860	0.0139	0.1158	0.7745	-
3.6045	2880	0.0	0.1183	0.7741	-
3.6295	2900	0.0	0.1181	0.7741	-
3.6546	2920	0.0	0.1179	0.7741	-
3.6796	2940	0.0	0.1179	0.7741	-
3.7046	2960	0.0119	0.1172	0.7742	-
3.7297	2980	0.0068	0.1183	0.7742	-
3.7547	3000	0.0	0.1193	0.7741	-
3.7797	3020	0.0	0.1193	0.7741	-
3.8048	3040	0.0	0.1193	0.7741	-
3.8298	3060	0.0	0.1191	0.7741	-
3.8548	3080	0.0	0.1193	0.7741	-
3.8798	3100	0.0	0.1193	0.7741	-
3.9049	3120	0.0131	0.1165	0.7745	-
3.9299	3140	0.0	0.1159	0.7745	-
3.9549	3160	0.0	0.1158	0.7746	-
3.9800	3180	0.0	0.1153	0.7746	-
-1	-1	-	-	-	0.7768

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.4.0
• Transformers: 4.48.1
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}