Shobhank-iiitdwd commited on
Commit
1d170b9
1 Parent(s): da09b64

Add new SentenceTransformer model.

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,524 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: []
3
+ library_name: sentence-transformers
4
+ tags:
5
+ - sentence-transformers
6
+ - sentence-similarity
7
+ - feature-extraction
8
+ - generated_from_trainer
9
+ - dataset_size:46453
10
+ - loss:MultipleNegativesRankingLoss
11
+ base_model: sentence-transformers/all-mpnet-base-v2
12
+ datasets: []
13
+ widget:
14
+ - source_sentence: clinician thinks the patient is homeless
15
+ sentences:
16
+ - '- Ms. ___ was homeless at the time of this admission.'
17
+ - This is ___ year old single homeless woman, previously diagnosed with borderline
18
+ personality disorder with chronic affective instability, reactive mood, impulsivity,
19
+ SIB (ingesting objects while hospitalized), recently discharged from ___ on ___,
20
+ ___ client, who presented to ___ on a ___ with worsening mood, threats of suicide
21
+ via cutting her legs off, as well as thoughts of wanting to hurt _
22
+ - Patient reports that her apartment is bugged, she has camera in her television,
23
+ and a helicopter is reading minds.
24
+ - source_sentence: assigned a case manager for housing
25
+ sentences:
26
+ - 'Home With Service Facility:'
27
+ - We consulted social work, psychiatry, and the case managers, who are working with
28
+ the hospital attorneys to acquire safer housing options with greater oversight
29
+ from health care professionals. .
30
+ - Has not established care with
31
+ - source_sentence: has been homeless
32
+ sentences:
33
+ - He reports being homeless, living in an empty garage near his sister.
34
+ - To complicate matters, patient's main support/roommate will be moving out of country
35
+ soon, so he will no longer be able to live in his apartment.
36
+ - 'Axis IV: homelessness'
37
+ - source_sentence: homelessness
38
+ sentences:
39
+ - Does not identify any acute stressors, but describes no longer being able to tolerate
40
+ being homeless (lack of food/clothing/showers).
41
+ - Unclear how reliable his group home is administering meds, notably nursing is
42
+ quite limited.
43
+ - Case management assisted in formulated a plan with ___ that would allow the patient's
44
+ ___ be the first responder when issues regarding her these two problems arise.
45
+ - source_sentence: assisted…housing benefits
46
+ sentences:
47
+ - As a result, patient is currently homeless.
48
+ - 'Home With Service Facility:'
49
+ - Patient with multiple admissions in the past several months, homeless.
50
+ pipeline_tag: sentence-similarity
51
+ ---
52
+
53
+ # SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
54
+
55
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
56
+
57
+ ## Model Details
58
+
59
+ ### Model Description
60
+ - **Model Type:** Sentence Transformer
61
+ - **Base model:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) <!-- at revision 84f2bcc00d77236f9e89c8a360a00fb1139bf47d -->
62
+ - **Maximum Sequence Length:** 384 tokens
63
+ - **Output Dimensionality:** 768 tokens
64
+ - **Similarity Function:** Cosine Similarity
65
+ <!-- - **Training Dataset:** Unknown -->
66
+ <!-- - **Language:** Unknown -->
67
+ <!-- - **License:** Unknown -->
68
+
69
+ ### Model Sources
70
+
71
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
72
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
73
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
74
+
75
+ ### Full Model Architecture
76
+
77
+ ```
78
+ SentenceTransformer(
79
+ (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel
80
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
81
+ (2): Normalize()
82
+ )
83
+ ```
84
+
85
+ ## Usage
86
+
87
+ ### Direct Usage (Sentence Transformers)
88
+
89
+ First install the Sentence Transformers library:
90
+
91
+ ```bash
92
+ pip install -U sentence-transformers
93
+ ```
94
+
95
+ Then you can load this model and run inference.
96
+ ```python
97
+ from sentence_transformers import SentenceTransformer
98
+
99
+ # Download from the 🤗 Hub
100
+ model = SentenceTransformer("Shobhank-iiitdwd/Clinical_sentence_transformers_mpnet_base_v2")
101
+ # Run inference
102
+ sentences = [
103
+ 'assisted…housing benefits',
104
+ 'Home With Service Facility:',
105
+ 'Patient with multiple admissions in the past several months, homeless.',
106
+ ]
107
+ embeddings = model.encode(sentences)
108
+ print(embeddings.shape)
109
+ # [3, 768]
110
+
111
+ # Get the similarity scores for the embeddings
112
+ similarities = model.similarity(embeddings, embeddings)
113
+ print(similarities.shape)
114
+ # [3, 3]
115
+ ```
116
+
117
+ <!--
118
+ ### Direct Usage (Transformers)
119
+
120
+ <details><summary>Click to see the direct usage in Transformers</summary>
121
+
122
+ </details>
123
+ -->
124
+
125
+ <!--
126
+ ### Downstream Usage (Sentence Transformers)
127
+
128
+ You can finetune this model on your own dataset.
129
+
130
+ <details><summary>Click to expand</summary>
131
+
132
+ </details>
133
+ -->
134
+
135
+ <!--
136
+ ### Out-of-Scope Use
137
+
138
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
139
+ -->
140
+
141
+ <!--
142
+ ## Bias, Risks and Limitations
143
+
144
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
145
+ -->
146
+
147
+ <!--
148
+ ### Recommendations
149
+
150
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
151
+ -->
152
+
153
+ ## Training Details
154
+
155
+ ### Training Dataset
156
+
157
+ #### Unnamed Dataset
158
+
159
+
160
+ * Size: 46,453 training samples
161
+ * Columns: <code>sentence_0</code> and <code>sentence_1</code>
162
+ * Approximate statistics based on the first 1000 samples:
163
+ | | sentence_0 | sentence_1 |
164
+ |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
165
+ | type | string | string |
166
+ | details | <ul><li>min: 3 tokens</li><li>mean: 6.64 tokens</li><li>max: 11 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 23.81 tokens</li><li>max: 384 tokens</li></ul> |
167
+ * Samples:
168
+ | sentence_0 | sentence_1 |
169
+ |:------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
170
+ | <code>has been homeless</code> | <code>He has a GED level education and previously held a stable job for a ___. However, mother reports he recently quit his job suddenly and is homeless right now after multiple family members kicked him out of their homes.</code> |
171
+ | <code>gave list of shelters</code> | <code>Home With Service Facility:</code> |
172
+ | <code>assessed housing needs</code> | <code>Patient with longstanding history of instrumental suicidal ideation and waxing and waning symptoms of depression and anxiety, SI when his needs, particularly regarding housing, are not being met with documented history of quick retraction of his</code> |
173
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
174
+ ```json
175
+ {
176
+ "scale": 20.0,
177
+ "similarity_fct": "cos_sim"
178
+ }
179
+ ```
180
+
181
+ ### Training Hyperparameters
182
+ #### Non-Default Hyperparameters
183
+
184
+ - `per_device_train_batch_size`: 64
185
+ - `per_device_eval_batch_size`: 64
186
+ - `num_train_epochs`: 100
187
+ - `multi_dataset_batch_sampler`: round_robin
188
+
189
+ #### All Hyperparameters
190
+ <details><summary>Click to expand</summary>
191
+
192
+ - `overwrite_output_dir`: False
193
+ - `do_predict`: False
194
+ - `eval_strategy`: no
195
+ - `prediction_loss_only`: True
196
+ - `per_device_train_batch_size`: 64
197
+ - `per_device_eval_batch_size`: 64
198
+ - `per_gpu_train_batch_size`: None
199
+ - `per_gpu_eval_batch_size`: None
200
+ - `gradient_accumulation_steps`: 1
201
+ - `eval_accumulation_steps`: None
202
+ - `learning_rate`: 5e-05
203
+ - `weight_decay`: 0.0
204
+ - `adam_beta1`: 0.9
205
+ - `adam_beta2`: 0.999
206
+ - `adam_epsilon`: 1e-08
207
+ - `max_grad_norm`: 1
208
+ - `num_train_epochs`: 100
209
+ - `max_steps`: -1
210
+ - `lr_scheduler_type`: linear
211
+ - `lr_scheduler_kwargs`: {}
212
+ - `warmup_ratio`: 0.0
213
+ - `warmup_steps`: 0
214
+ - `log_level`: passive
215
+ - `log_level_replica`: warning
216
+ - `log_on_each_node`: True
217
+ - `logging_nan_inf_filter`: True
218
+ - `save_safetensors`: True
219
+ - `save_on_each_node`: False
220
+ - `save_only_model`: False
221
+ - `restore_callback_states_from_checkpoint`: False
222
+ - `no_cuda`: False
223
+ - `use_cpu`: False
224
+ - `use_mps_device`: False
225
+ - `seed`: 42
226
+ - `data_seed`: None
227
+ - `jit_mode_eval`: False
228
+ - `use_ipex`: False
229
+ - `bf16`: False
230
+ - `fp16`: False
231
+ - `fp16_opt_level`: O1
232
+ - `half_precision_backend`: auto
233
+ - `bf16_full_eval`: False
234
+ - `fp16_full_eval`: False
235
+ - `tf32`: None
236
+ - `local_rank`: 0
237
+ - `ddp_backend`: None
238
+ - `tpu_num_cores`: None
239
+ - `tpu_metrics_debug`: False
240
+ - `debug`: []
241
+ - `dataloader_drop_last`: False
242
+ - `dataloader_num_workers`: 0
243
+ - `dataloader_prefetch_factor`: None
244
+ - `past_index`: -1
245
+ - `disable_tqdm`: False
246
+ - `remove_unused_columns`: True
247
+ - `label_names`: None
248
+ - `load_best_model_at_end`: False
249
+ - `ignore_data_skip`: False
250
+ - `fsdp`: []
251
+ - `fsdp_min_num_params`: 0
252
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
253
+ - `fsdp_transformer_layer_cls_to_wrap`: None
254
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
255
+ - `deepspeed`: None
256
+ - `label_smoothing_factor`: 0.0
257
+ - `optim`: adamw_torch
258
+ - `optim_args`: None
259
+ - `adafactor`: False
260
+ - `group_by_length`: False
261
+ - `length_column_name`: length
262
+ - `ddp_find_unused_parameters`: None
263
+ - `ddp_bucket_cap_mb`: None
264
+ - `ddp_broadcast_buffers`: False
265
+ - `dataloader_pin_memory`: True
266
+ - `dataloader_persistent_workers`: False
267
+ - `skip_memory_metrics`: True
268
+ - `use_legacy_prediction_loop`: False
269
+ - `push_to_hub`: False
270
+ - `resume_from_checkpoint`: None
271
+ - `hub_model_id`: None
272
+ - `hub_strategy`: every_save
273
+ - `hub_private_repo`: False
274
+ - `hub_always_push`: False
275
+ - `gradient_checkpointing`: False
276
+ - `gradient_checkpointing_kwargs`: None
277
+ - `include_inputs_for_metrics`: False
278
+ - `eval_do_concat_batches`: True
279
+ - `fp16_backend`: auto
280
+ - `push_to_hub_model_id`: None
281
+ - `push_to_hub_organization`: None
282
+ - `mp_parameters`:
283
+ - `auto_find_batch_size`: False
284
+ - `full_determinism`: False
285
+ - `torchdynamo`: None
286
+ - `ray_scope`: last
287
+ - `ddp_timeout`: 1800
288
+ - `torch_compile`: False
289
+ - `torch_compile_backend`: None
290
+ - `torch_compile_mode`: None
291
+ - `dispatch_batches`: None
292
+ - `split_batches`: None
293
+ - `include_tokens_per_second`: False
294
+ - `include_num_input_tokens_seen`: False
295
+ - `neftune_noise_alpha`: None
296
+ - `optim_target_modules`: None
297
+ - `batch_eval_metrics`: False
298
+ - `batch_sampler`: batch_sampler
299
+ - `multi_dataset_batch_sampler`: round_robin
300
+
301
+ </details>
302
+
303
+ ### Training Logs
304
+ <details><summary>Click to expand</summary>
305
+
306
+ | Epoch | Step | Training Loss |
307
+ |:-------:|:-----:|:-------------:|
308
+ | 0.6887 | 500 | 3.5133 |
309
+ | 1.3774 | 1000 | 3.2727 |
310
+ | 2.0661 | 1500 | 3.2238 |
311
+ | 2.7548 | 2000 | 3.1758 |
312
+ | 3.4435 | 2500 | 3.1582 |
313
+ | 4.1322 | 3000 | 3.1385 |
314
+ | 4.8209 | 3500 | 3.1155 |
315
+ | 5.5096 | 4000 | 3.1034 |
316
+ | 6.1983 | 4500 | 3.091 |
317
+ | 6.8871 | 5000 | 3.0768 |
318
+ | 7.5758 | 5500 | 3.065 |
319
+ | 8.2645 | 6000 | 3.0632 |
320
+ | 8.9532 | 6500 | 3.0566 |
321
+ | 9.6419 | 7000 | 3.0433 |
322
+ | 0.6887 | 500 | 3.0536 |
323
+ | 1.3774 | 1000 | 3.0608 |
324
+ | 2.0661 | 1500 | 3.0631 |
325
+ | 2.7548 | 2000 | 3.0644 |
326
+ | 3.4435 | 2500 | 3.0667 |
327
+ | 4.1322 | 3000 | 3.07 |
328
+ | 4.8209 | 3500 | 3.0682 |
329
+ | 5.5096 | 4000 | 3.0718 |
330
+ | 6.1983 | 4500 | 3.0719 |
331
+ | 6.8871 | 5000 | 3.0685 |
332
+ | 7.5758 | 5500 | 3.0723 |
333
+ | 8.2645 | 6000 | 3.0681 |
334
+ | 8.9532 | 6500 | 3.0633 |
335
+ | 9.6419 | 7000 | 3.0642 |
336
+ | 10.3306 | 7500 | 3.0511 |
337
+ | 11.0193 | 8000 | 3.0463 |
338
+ | 11.7080 | 8500 | 3.0301 |
339
+ | 12.3967 | 9000 | 3.0163 |
340
+ | 13.0854 | 9500 | 3.0059 |
341
+ | 13.7741 | 10000 | 2.9845 |
342
+ | 14.4628 | 10500 | 2.9705 |
343
+ | 15.1515 | 11000 | 2.9536 |
344
+ | 15.8402 | 11500 | 2.9263 |
345
+ | 16.5289 | 12000 | 2.9199 |
346
+ | 17.2176 | 12500 | 2.8989 |
347
+ | 17.9063 | 13000 | 2.8818 |
348
+ | 18.5950 | 13500 | 2.8735 |
349
+ | 19.2837 | 14000 | 2.852 |
350
+ | 19.9725 | 14500 | 2.8315 |
351
+ | 20.6612 | 15000 | 2.8095 |
352
+ | 21.3499 | 15500 | 2.7965 |
353
+ | 22.0386 | 16000 | 2.7802 |
354
+ | 22.7273 | 16500 | 2.7527 |
355
+ | 23.4160 | 17000 | 2.7547 |
356
+ | 24.1047 | 17500 | 2.7377 |
357
+ | 24.7934 | 18000 | 2.7035 |
358
+ | 25.4821 | 18500 | 2.7102 |
359
+ | 26.1708 | 19000 | 2.6997 |
360
+ | 26.8595 | 19500 | 2.6548 |
361
+ | 27.5482 | 20000 | 2.6704 |
362
+ | 28.2369 | 20500 | 2.6624 |
363
+ | 28.9256 | 21000 | 2.6306 |
364
+ | 29.6143 | 21500 | 2.6358 |
365
+ | 30.3030 | 22000 | 2.634 |
366
+ | 30.9917 | 22500 | 2.6089 |
367
+ | 31.6804 | 23000 | 2.607 |
368
+ | 32.3691 | 23500 | 2.6246 |
369
+ | 33.0579 | 24000 | 2.5947 |
370
+ | 33.7466 | 24500 | 2.5798 |
371
+ | 34.4353 | 25000 | 2.6025 |
372
+ | 35.1240 | 25500 | 2.5824 |
373
+ | 35.8127 | 26000 | 2.5698 |
374
+ | 36.5014 | 26500 | 2.5711 |
375
+ | 37.1901 | 27000 | 2.5636 |
376
+ | 37.8788 | 27500 | 2.5387 |
377
+ | 38.5675 | 28000 | 2.5472 |
378
+ | 39.2562 | 28500 | 2.5455 |
379
+ | 39.9449 | 29000 | 2.5204 |
380
+ | 40.6336 | 29500 | 2.524 |
381
+ | 41.3223 | 30000 | 2.5246 |
382
+ | 42.0110 | 30500 | 2.5125 |
383
+ | 42.6997 | 31000 | 2.5042 |
384
+ | 43.3884 | 31500 | 2.5165 |
385
+ | 44.0771 | 32000 | 2.5187 |
386
+ | 44.7658 | 32500 | 2.4975 |
387
+ | 45.4545 | 33000 | 2.5048 |
388
+ | 46.1433 | 33500 | 2.521 |
389
+ | 46.8320 | 34000 | 2.4825 |
390
+ | 47.5207 | 34500 | 2.5034 |
391
+ | 48.2094 | 35000 | 2.5049 |
392
+ | 48.8981 | 35500 | 2.4886 |
393
+ | 49.5868 | 36000 | 2.4992 |
394
+ | 50.2755 | 36500 | 2.5099 |
395
+ | 50.9642 | 37000 | 2.489 |
396
+ | 51.6529 | 37500 | 2.4825 |
397
+ | 52.3416 | 38000 | 2.4902 |
398
+ | 53.0303 | 38500 | 2.4815 |
399
+ | 53.7190 | 39000 | 2.4723 |
400
+ | 54.4077 | 39500 | 2.4921 |
401
+ | 55.0964 | 40000 | 2.4763 |
402
+ | 55.7851 | 40500 | 2.4692 |
403
+ | 56.4738 | 41000 | 2.4831 |
404
+ | 57.1625 | 41500 | 2.4705 |
405
+ | 57.8512 | 42000 | 2.4659 |
406
+ | 58.5399 | 42500 | 2.4804 |
407
+ | 59.2287 | 43000 | 2.4582 |
408
+ | 59.9174 | 43500 | 2.4544 |
409
+ | 60.6061 | 44000 | 2.4712 |
410
+ | 61.2948 | 44500 | 2.4478 |
411
+ | 61.9835 | 45000 | 2.4428 |
412
+ | 62.6722 | 45500 | 2.4558 |
413
+ | 63.3609 | 46000 | 2.4428 |
414
+ | 64.0496 | 46500 | 2.4399 |
415
+ | 64.7383 | 47000 | 2.4529 |
416
+ | 65.4270 | 47500 | 2.4374 |
417
+ | 66.1157 | 48000 | 2.4543 |
418
+ | 66.8044 | 48500 | 2.4576 |
419
+ | 67.4931 | 49000 | 2.4426 |
420
+ | 68.1818 | 49500 | 2.4698 |
421
+ | 68.8705 | 50000 | 2.4604 |
422
+ | 69.5592 | 50500 | 2.4515 |
423
+ | 70.2479 | 51000 | 2.4804 |
424
+ | 70.9366 | 51500 | 2.4545 |
425
+ | 71.6253 | 52000 | 2.4523 |
426
+ | 72.3140 | 52500 | 2.4756 |
427
+ | 73.0028 | 53000 | 2.4697 |
428
+ | 73.6915 | 53500 | 2.4536 |
429
+ | 74.3802 | 54000 | 2.4866 |
430
+ | 75.0689 | 54500 | 2.471 |
431
+ | 75.7576 | 55000 | 2.483 |
432
+ | 76.4463 | 55500 | 2.5002 |
433
+ | 77.1350 | 56000 | 2.4849 |
434
+ | 77.8237 | 56500 | 2.4848 |
435
+ | 78.5124 | 57000 | 2.5047 |
436
+ | 79.2011 | 57500 | 2.5143 |
437
+ | 79.8898 | 58000 | 2.4879 |
438
+ | 80.5785 | 58500 | 2.5093 |
439
+ | 81.2672 | 59000 | 2.5247 |
440
+ | 81.9559 | 59500 | 2.4915 |
441
+ | 82.6446 | 60000 | 2.5124 |
442
+ | 83.3333 | 60500 | 2.5056 |
443
+ | 84.0220 | 61000 | 2.4767 |
444
+ | 84.7107 | 61500 | 2.5068 |
445
+ | 85.3994 | 62000 | 2.5173 |
446
+ | 86.0882 | 62500 | 2.4911 |
447
+ | 86.7769 | 63000 | 2.526 |
448
+ | 87.4656 | 63500 | 2.5313 |
449
+ | 88.1543 | 64000 | 2.5312 |
450
+ | 88.8430 | 64500 | 2.5735 |
451
+ | 89.5317 | 65000 | 2.5873 |
452
+ | 90.2204 | 65500 | 2.6395 |
453
+ | 90.9091 | 66000 | 2.7914 |
454
+ | 91.5978 | 66500 | 2.6729 |
455
+ | 92.2865 | 67000 | 2.9846 |
456
+ | 92.9752 | 67500 | 2.9259 |
457
+ | 93.6639 | 68000 | 2.8845 |
458
+ | 94.3526 | 68500 | 2.9906 |
459
+ | 95.0413 | 69000 | 2.9534 |
460
+ | 95.7300 | 69500 | 2.9857 |
461
+ | 96.4187 | 70000 | 3.0559 |
462
+ | 97.1074 | 70500 | 2.9919 |
463
+ | 97.7961 | 71000 | 3.0435 |
464
+ | 98.4848 | 71500 | 3.0534 |
465
+ | 99.1736 | 72000 | 3.0169 |
466
+ | 99.8623 | 72500 | 3.0264 |
467
+
468
+ </details>
469
+
470
+ ### Framework Versions
471
+ - Python: 3.10.11
472
+ - Sentence Transformers: 3.0.1
473
+ - Transformers: 4.41.2
474
+ - PyTorch: 2.0.1
475
+ - Accelerate: 0.31.0
476
+ - Datasets: 2.19.1
477
+ - Tokenizers: 0.19.1
478
+
479
+ ## Citation
480
+
481
+ ### BibTeX
482
+
483
+ #### Sentence Transformers
484
+ ```bibtex
485
+ @inproceedings{reimers-2019-sentence-bert,
486
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
487
+ author = "Reimers, Nils and Gurevych, Iryna",
488
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
489
+ month = "11",
490
+ year = "2019",
491
+ publisher = "Association for Computational Linguistics",
492
+ url = "https://arxiv.org/abs/1908.10084",
493
+ }
494
+ ```
495
+
496
+ #### MultipleNegativesRankingLoss
497
+ ```bibtex
498
+ @misc{henderson2017efficient,
499
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
500
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
501
+ year={2017},
502
+ eprint={1705.00652},
503
+ archivePrefix={arXiv},
504
+ primaryClass={cs.CL}
505
+ }
506
+ ```
507
+
508
+ <!--
509
+ ## Glossary
510
+
511
+ *Clearly define terms in order to be accessible across audiences.*
512
+ -->
513
+
514
+ <!--
515
+ ## Model Card Authors
516
+
517
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
518
+ -->
519
+
520
+ <!--
521
+ ## Model Card Contact
522
+
523
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
524
+ -->
config.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "./Clincial_sentence-transformers/mpnet-base-v2",
3
+ "architectures": [
4
+ "MPNetModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-05,
15
+ "max_position_embeddings": 514,
16
+ "model_type": "mpnet",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 1,
20
+ "relative_attention_num_buckets": 32,
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.41.2",
23
+ "vocab_size": 30527
24
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.0.1",
4
+ "transformers": "4.41.2",
5
+ "pytorch": "2.0.1"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb1cbb0337340d47988a6d3bc4ea36ebdb6b7f920e90c393f75199842a8d12b0
3
+ size 437967672
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 384,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "[UNK]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "104": {
36
+ "content": "[UNK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ },
43
+ "30526": {
44
+ "content": "<mask>",
45
+ "lstrip": true,
46
+ "normalized": false,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": true
50
+ }
51
+ },
52
+ "bos_token": "<s>",
53
+ "clean_up_tokenization_spaces": true,
54
+ "cls_token": "<s>",
55
+ "do_lower_case": true,
56
+ "eos_token": "</s>",
57
+ "mask_token": "<mask>",
58
+ "max_length": 128,
59
+ "model_max_length": 384,
60
+ "pad_to_multiple_of": null,
61
+ "pad_token": "<pad>",
62
+ "pad_token_type_id": 0,
63
+ "padding_side": "right",
64
+ "sep_token": "</s>",
65
+ "stride": 0,
66
+ "strip_accents": null,
67
+ "tokenize_chinese_chars": true,
68
+ "tokenizer_class": "MPNetTokenizer",
69
+ "truncation_side": "right",
70
+ "truncation_strategy": "longest_first",
71
+ "unk_token": "[UNK]"
72
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff