---
base_model: sentence-transformers/all-MiniLM-L12-v2
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:2144
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: How do I find out when I should write my examinations?
sentences:
- Information relating to examination timetables is available from the Examination
Office and will be published on the official Institute Notice Board and the website.
- If you find an error on your academic record, you should contact the Registration
and Student Records Management Office immediately.
- To request accommodations for a disability, you must submit documentation of the
disability to the disability services office and meet with a disability services
coordinator.
- source_sentence: What is the language of instruction at the Harare Institute of
Technology?
sentences:
- English is the language of instruction.
- Tracking international events and conference and strategically link them to HIT,
internationalizing HIT programmes and activities, developing bouquet of events
and activities for international visitors, helping affiliate, accredit HIT, staff
and students to international bodies and associations, liaising with national
bodies and promote Zimbabwean culture and symbols, serving as a point of contact
for exchange students, staff and visitors, ensuring international programmes align
to national programmes and symbols, helping affiliate HIT ethos to national art
and culture, monitoring implementation of MoUs and MoAs, facilitation of international
travel and visits, providing Institute departments with consular advice, ensuring
HIT members get oriented to particular countries’ culture and services before
departure, driving recruitment of foreign students and exchange programmes.
- BFA 7206 is the course code for Financial Institutions Fraud, which is an elective
course in the second semester of the program.
- source_sentence: What is the process for collecting a certificate?
sentences:
- The programme is designed such that on completion, graduates should be able to
innovatively execute their professional role within prescribed and legislative
parameters, demonstrate a critical understanding and application of quality assurance
and radiation protection in Radiography, apply scientific knowledge and technical
skills to perform Radiography procedures, plan, develop and apply total quality
management appropriate to the Radiography context, apply management, entrepreneurial,
education and research skills independently and function in a supervisory clinical
governance and quality assurance capacity within the professional sector, demonstrate
the ability to reflect in clinical practice, critically evaluate and adjust to
current and new trends in Radiography, demonstrate capability to implement new
knowledge and solve problems in varying contexts, and engage life-long learning
and development in their profession.
- The process involves clearing any dues to the Institute and providing valid identification
documents.
- A student can apply for change of programme within two weeks after commencement
of lectures.
- source_sentence: How do I change my address or contact information?
sentences:
- Information Security & Assurance is a field that deals with the protection of
information and information systems from unauthorized access, use, disclosure,
disruption, modification, or destruction.
- The Information and Communications Technology Services (ICTS) Department at HIT
is responsible for providing and maintaining the Institute's IT infrastructure
and services.
- You can update your address or contact information through the online student
portal or by contacting the Academic Registry.
- source_sentence: What is the difference between Cloud Computing and Information
Security & Assurance?
sentences:
- The fourth semester focuses on courses such as Research Project, Clinical Practice
IV, and Seminar.
- Cloud Computing is focused on the design, implementation, and management of cloud
services, while Information Security & Assurance is focused on the protection
of information by mitigating information risks and ensuring availability, privacy,
and integrity of data.
- The Applied Research Methods course is designed to equip students with the skills
and knowledge necessary to conduct research in chemical engineering process and
plant design.
---
# SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) <!-- at revision a05860a77cef7b37e0048a7864658139bc18a854 -->
- **Maximum Sequence Length:** 128 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("Dex-X/finehit")
# Run inference
sentences = [
'What is the difference between Cloud Computing and Information Security & Assurance?',
'Cloud Computing is focused on the design, implementation, and management of cloud services, while Information Security & Assurance is focused on the protection of information by mitigating information risks and ensuring availability, privacy, and integrity of data.',
'The Applied Research Methods course is designed to equip students with the skills and knowledge necessary to conduct research in chemical engineering process and plant design.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 2,144 training samples
* Columns: <code>question</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
| | question | answer |
|:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 6 tokens</li><li>mean: 13.94 tokens</li><li>max: 31 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 30.7 tokens</li><li>max: 128 tokens</li></ul> |
* Samples:
| question | answer |
|:----------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>What is the role of the Dean of Students?</code> | <code>The Dean of Students oversees various aspects of student life, including student affairs, campus life and development, accommodation, wellness, and more.</code> |
| <code>What does the Student Affairs department do?</code> | <code>The Student Affairs department handles matters related to student life, conduct, and welfare.</code> |
| <code>What is the role of Campus Life and Student Development?</code> | <code>Campus Life and Student Development is responsible for fostering a positive campus environment and promoting student growth and development.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Evaluation Dataset
#### Unnamed Dataset
* Size: 214 evaluation samples
* Columns: <code>question</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
| | question | answer |
|:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 7 tokens</li><li>mean: 15.12 tokens</li><li>max: 31 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 31.14 tokens</li><li>max: 128 tokens</li></ul> |
* Samples:
| question | answer |
|:--------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------|
| <code>What is Student Accommodation and Catering?</code> | <code>Student Accommodation and Catering is a department that manages student housing and dining services.</code> |
| <code>What certification does Mr. Njonga have from the National Social Security Authority?</code> | <code>Safety and Health Advisor Certification</code> |
| <code>What is the duration of the B Tech (Hons) Computer Science programme?</code> | <code>The B Tech (Hons) Computer Science programme is a four-year full-time regular programme.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `num_train_epochs`: 1
- `warmup_ratio`: 0.1
- `fp16`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | Training Loss | loss |
|:------:|:----:|:-------------:|:------:|
| 0.7463 | 100 | 0.5551 | 0.0665 |
### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.3.0+cu121
- Accelerate: 0.32.1
- Datasets: 2.20.0
- Tokenizers: 0.19.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->