Subiendo modelo inicial

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 384,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,485 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:6552
+- loss:MultipleNegativesRankingLoss
+base_model: BAAI/bge-small-en-v1.5
+widget:
+- source_sentence: What property is denoted as the M→M property in the queueing network
+    literature?
+  sentences:
+  - 'The LOFAR system introduces two additional levels in the beam hierarchy: the
+    compound (tile) beam and the station beam.'
+  - The desired pseudonoise sequence in a CDMA system has the characteristics that
+    the fraction of 0's and 1's is almost half-and-half over the period, and the shifted
+    versions of the pseudonoise sequence are nearly orthogonal to each other. If the
+    shift of the pseudonoise sequence is randomized, it becomes a random process.
+  - The M→M property in the queueing network literature denotes the independence of
+    individual queues in the long term.
+- source_sentence: Which type of channel condition has better path loss exponent (PLE)
+    in terms of AA (air to air) and AG (air to ground) propagation channels?
+  sentences:
+  - The goal of the Fixed Access Information API is to provide access network related
+    information for the multitude of fixed access technologies.
+  - Error mitigation is a technique to reduce the impact of errors in near-term quantum
+    systems without requiring full fault-tolerant quantum codes.
+  - From the document, it is mentioned that the AA channel has better conditions than
+    the AG channel in terms of path loss exponent (PLE).
+- source_sentence: What is the goal of a functionality extraction attack?
+  sentences:
+  - Deep learning can automatically extract high-level features from data, reducing
+    the need for manual feature engineering.
+  - The goal of a functionality extraction attack is to create knock-off models that
+    mimic the behavior of an existing machine learning model.
+  - The main advantage of using wind turbine towers for communication is that they
+    already have a reliable power grid connection.
+- source_sentence: What is MTU?
+  sentences:
+  - The worst-case complexity of average consensus is exponential in the number of
+    nodes, but it can be reduced to linear if an upper bound on the total number of
+    nodes is known.
+  - In a normally clad fiber, at long wavelengths, the MFD is large compared to the
+    core diameter and the electric field extends far into the cladding region.
+  - MTU (Maximum Transmission Unit) represents the largest size of a data packet that
+    can be sent over a network without fragmentation.
+- source_sentence: What should the AP or PCP do if it is not decentralized AP or PCP
+    clustering capable or a decentralized AP or PCP cluster is not present?
+  sentences:
+  - When a Data, Management or Extension frame is received, a STA inserts it in an
+    appropriate cache.
+  - If the AP or PCP is not decentralized AP or PCP clustering capable or a decentralized
+    AP or PCP cluster is not present, it should set its Cluster Member Role to 0 (not
+    currently participating in a cluster) and remain unclustered.
+  - Analog beamforming based on slowly-varying second order statistics of the CSI
+    reduces the dimension of the effective instantaneous CSI for digital beamforming
+    within each coherent fading block, which helps to relieve the signaling overhead.
+datasets:
+- dinho1597/Telecom-QA-MultipleChoice
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_recall@1
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer based on BAAI/bge-small-en-v1.5
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: telecom ir eval
+      type: telecom-ir-eval
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.965675057208238
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.992372234935164
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.9931350114416476
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.9938977879481312
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.965675057208238
+      name: Cosine Precision@1
+    - type: cosine_recall@1
+      value: 0.965675057208238
+      name: Cosine Recall@1
+    - type: cosine_ndcg@10
+      value: 0.9824027787882591
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.9784334023464457
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.9786169716375667
+      name: Cosine Map@100
+---
+
+# SentenceTransformer based on BAAI/bge-small-en-v1.5
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) on the [telecom-qa-multiple_choice](https://huggingface.co/datasets/dinho1597/Telecom-QA-MultipleChoice) dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) <!-- at revision 5c38ec7c405ec4b44b94cc5a9bb96e735b38267a -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 384 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [telecom-qa-multiple_choice](https://huggingface.co/datasets/dinho1597/Telecom-QA-MultipleChoice)
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    'What should the AP or PCP do if it is not decentralized AP or PCP clustering capable or a decentralized AP or PCP cluster is not present?',
+    'If the AP or PCP is not decentralized AP or PCP clustering capable or a decentralized AP or PCP cluster is not present, it should set its Cluster Member Role to 0 (not currently participating in a cluster) and remain unclustered.',
+    'When a Data, Management or Extension frame is received, a STA inserts it in an appropriate cache.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 384]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Dataset: `telecom-ir-eval`
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric             | Value      |
+|:-------------------|:-----------|
+| cosine_accuracy@1  | 0.9657     |
+| cosine_accuracy@3  | 0.9924     |
+| cosine_accuracy@5  | 0.9931     |
+| cosine_accuracy@10 | 0.9939     |
+| cosine_precision@1 | 0.9657     |
+| cosine_recall@1    | 0.9657     |
+| **cosine_ndcg@10** | **0.9824** |
+| cosine_mrr@10      | 0.9784     |
+| cosine_map@100     | 0.9786     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### telecom-qa-multiple_choice
+
+* Dataset: [telecom-qa-multiple_choice](https://huggingface.co/datasets/dinho1597/Telecom-QA-MultipleChoice) at [73aebbb](https://huggingface.co/datasets/dinho1597/Telecom-QA-MultipleChoice/tree/73aebbb16651212e4b1947ac0d64fc80a6bc9398)
+* Size: 6,552 training samples
+* Columns: <code>anchor</code> and <code>positive</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                            | positive                                                                           |
+  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                             |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 18.95 tokens</li><li>max: 49 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 29.33 tokens</li><li>max: 112 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                        | positive                                                                                                                                                                                                                                                                                                                                    |
+  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>What is the goal of a jammer in a mobile edge caching system?</code>                                                                                                    | <code>The goal of a jammer in a mobile edge caching system is to interrupt ongoing radio transmissions of the edge node with cached chunks or caching users and prevent access to cached content. Additionally, jammers aim to deplete the resources of edge nodes, caching users, and sensors during failed communication attempts.</code> |
+  | <code>Which type of DRL uses DNNs (Deep Neural Networks) to fit action values and employs experience replay and target networks to ensure stable training convergence?</code> | <code>Value-based DRL, such as Deep Q-Learning (DQL), uses DNNs to fit action values and employs experience replay and target networks to ensure stable training convergence.</code>                                                                                                                                                        |
+  | <code>What is the relationship between the curvature of the decision boundary and the robustness of a network?</code>                                                         | <code>The lower the curvature of the decision boundaries, the more robust the network.</code>                                                                                                                                                                                                                                               |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### telecom-qa-multiple_choice
+
+* Dataset: [telecom-qa-multiple_choice](https://huggingface.co/datasets/dinho1597/Telecom-QA-MultipleChoice) at [73aebbb](https://huggingface.co/datasets/dinho1597/Telecom-QA-MultipleChoice/tree/73aebbb16651212e4b1947ac0d64fc80a6bc9398)
+* Size: 6,552 evaluation samples
+* Columns: <code>anchor</code> and <code>positive</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                            | positive                                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 18.87 tokens</li><li>max: 56 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 29.45 tokens</li><li>max: 91 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                   | positive                                                                                                                                                                                    |
+  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Which forward error correction (FEC) codes are available for the THz single carrier mode?</code>                                                                   | <code>The THz single carrier mode (THz-SC PHY) in the IEEE 802.15.3d standard supports two low-density parity-check (LDPC) codes: 14/15 LDPC (1440,1344) and 11/15 LDPC (1440,1056).</code> |
+  | <code>Which multiple access technique allows users to access the channel simultaneously using the same frequency and time resources, with different power levels?</code> | <code>Non-Orthogonal Multiple Access (NOMA) allows users to access the channel simultaneously using the same frequency and time resources, but with different power levels.</code>          |
+  | <code>What is the power gain when doubling the number of antennas?</code>                                                                                                | <code>Doubling the number of antennas yields a 3-dB power gain.</code>                                                                                                                      |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 512
+- `per_device_eval_batch_size`: 512
+- `weight_decay`: 0.01
+- `num_train_epochs`: 15
+- `lr_scheduler_type`: cosine_with_restarts
+- `warmup_ratio`: 0.1
+- `fp16`: True
+- `load_best_model_at_end`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 512
+- `per_device_eval_batch_size`: 512
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.01
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 15
+- `max_steps`: -1
+- `lr_scheduler_type`: cosine_with_restarts
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss | telecom-ir-eval_cosine_ndcg@10 |
+|:------:|:----:|:-------------:|:---------------:|:------------------------------:|
+| 1.2727 | 15   | 1.0332        | 0.0968          | 0.9725                         |
+| 2.5455 | 30   | 0.2091        | 0.0518          | 0.9808                         |
+| 3.8182 | 45   | 0.0997        | 0.0470          | 0.9824                         |
+
+
+### Framework Versions
+- Python: 3.10.12
+- Sentence Transformers: 3.3.1
+- Transformers: 4.47.1
+- PyTorch: 2.5.1+cu121
+- Accelerate: 1.2.1
+- Datasets: 3.2.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,31 @@
+{
+  "_name_or_path": "BAAI/bge-small-en-v1.5",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.47.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.47.1",
+    "pytorch": "2.5.1+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6a17ff064cf7dd59cd00dd009c029e97aa67812ae08b28555b86cae53de9346
+size 133462128

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b346d73e533eeed9898071d47878cf25c882b0a4865130ec649224ad1001cbc
+size 265862074

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b453c3c68d6d9de2e7549bd1ddfa70e40f298095123abbca773deb02b1581f6
+size 14244

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eeabf30df830deb924a29cbbe61710ac215a8a05461da8f799581c65f9d2e510
+size 1064

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": true
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

trainer_state.json

@@ -0,0 +1,105 @@
+{
+  "best_metric": 0.965675057208238,
+  "best_model_checkpoint": "/content/drive/MyDrive/Papers/RAG_3GPP/models/checkpoints/embedding/bge-small-telecom_15e_512bs/checkpoint-45",
+  "epoch": 3.8181818181818183,
+  "eval_steps": 15,
+  "global_step": 45,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 1.2727272727272727,
+      "grad_norm": 1.7355202436447144,
+      "learning_rate": 4.411764705882353e-05,
+      "loss": 1.0332,
+      "step": 15
+    },
+    {
+      "epoch": 1.2727272727272727,
+      "eval_loss": 0.0967816412448883,
+      "eval_runtime": 3.2637,
+      "eval_samples_per_second": 401.687,
+      "eval_steps_per_second": 0.919,
+      "eval_telecom-ir-eval_cosine_accuracy@1": 0.9450800915331807,
+      "eval_telecom-ir-eval_cosine_accuracy@10": 0.9931350114416476,
+      "eval_telecom-ir-eval_cosine_accuracy@3": 0.9870327993897788,
+      "eval_telecom-ir-eval_cosine_accuracy@5": 0.988558352402746,
+      "eval_telecom-ir-eval_cosine_map@100": 0.96574625373806,
+      "eval_telecom-ir-eval_cosine_mrr@10": 0.9655954499776009,
+      "eval_telecom-ir-eval_cosine_ndcg@10": 0.9725340084355174,
+      "eval_telecom-ir-eval_cosine_precision@1": 0.9450800915331807,
+      "eval_telecom-ir-eval_cosine_recall@1": 0.9450800915331807,
+      "step": 15
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "grad_norm": 0.8801536560058594,
+      "learning_rate": 4.9054165035221236e-05,
+      "loss": 0.2091,
+      "step": 30
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "eval_loss": 0.05175875499844551,
+      "eval_runtime": 3.3397,
+      "eval_samples_per_second": 392.552,
+      "eval_steps_per_second": 0.898,
+      "eval_telecom-ir-eval_cosine_accuracy@1": 0.96186117467582,
+      "eval_telecom-ir-eval_cosine_accuracy@10": 0.9938977879481312,
+      "eval_telecom-ir-eval_cosine_accuracy@3": 0.9908466819221968,
+      "eval_telecom-ir-eval_cosine_accuracy@5": 0.992372234935164,
+      "eval_telecom-ir-eval_cosine_map@100": 0.976456964386964,
+      "eval_telecom-ir-eval_cosine_mrr@10": 0.9762734130011017,
+      "eval_telecom-ir-eval_cosine_ndcg@10": 0.9807745601952235,
+      "eval_telecom-ir-eval_cosine_precision@1": 0.96186117467582,
+      "eval_telecom-ir-eval_cosine_recall@1": 0.96186117467582,
+      "step": 30
+    },
+    {
+      "epoch": 3.8181818181818183,
+      "grad_norm": 0.5518485307693481,
+      "learning_rate": 4.571274123109606e-05,
+      "loss": 0.0997,
+      "step": 45
+    },
+    {
+      "epoch": 3.8181818181818183,
+      "eval_loss": 0.046976786106824875,
+      "eval_runtime": 3.4455,
+      "eval_samples_per_second": 380.5,
+      "eval_steps_per_second": 0.871,
+      "eval_telecom-ir-eval_cosine_accuracy@1": 0.965675057208238,
+      "eval_telecom-ir-eval_cosine_accuracy@10": 0.9938977879481312,
+      "eval_telecom-ir-eval_cosine_accuracy@3": 0.992372234935164,
+      "eval_telecom-ir-eval_cosine_accuracy@5": 0.9931350114416476,
+      "eval_telecom-ir-eval_cosine_map@100": 0.9786169716375667,
+      "eval_telecom-ir-eval_cosine_mrr@10": 0.9784334023464457,
+      "eval_telecom-ir-eval_cosine_ndcg@10": 0.9824027787882591,
+      "eval_telecom-ir-eval_cosine_precision@1": 0.965675057208238,
+      "eval_telecom-ir-eval_cosine_recall@1": 0.965675057208238,
+      "step": 45
+    }
+  ],
+  "logging_steps": 15,
+  "max_steps": 165,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 15,
+  "save_steps": 15,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 512,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:917e4ea8b0b76de6ad48dbacba13d419d280dc7819d91087c6a3cc0040b413b7
+size 5816