---
license: apache-2.0
datasets:
- ArchitRastogi/Italian-BERT-FineTuning-Embeddings
language:
- it
metrics:
- Recall@1
- Recall@100
- Recall@1000
- Average Precision
- NDCG@10
- NDCG@100
- NDCG@1000
- MRR@10
- MRR@100
- MRR@1000
base_model:
- dbmdz/bert-base-italian-xxl-uncased
new_version: "true"
pipeline_tag: feature-extraction
library_name: transformers
tags:
- information-retrieval
- contrastive-learning
- embeddings
- italian
- fine-tuned
- bert
- retrieval-augmented-generation
model-index:
  - name: bert-base-italian-embeddings
    results:
      - task:
          type: information-retrieval
        dataset:
          name: mMARCO
          type: mMARCO
        metrics:
          - name: Recall@1000
            type: Recall
            value: 0.9719
          - name: NDCG@1000
            type: Normalized Discounted Cumulative Gain
            value: 0.4391
          - Average Precision: AP
            type: Precision
            value: 0.3173
          
        source:
          name: Fine-tuned Italian BERT Model Evaluation
          url: https://github.com/unicamp-dl/mMARCO
---


# bert-base-italian-embeddings: A Fine-Tuned Italian BERT Model for IR and RAG Applications

## Model Overview

This model is a fine-tuned version of [dbmdz/bert-base-italian-xxl-uncased](https://huggingface.co/dbmdz/bert-base-italian-xxl-uncased) tailored for Italian language Information Retrieval (IR) and Retrieval-Augmented Generation (RAG) tasks. It leverages contrastive learning to generate high-quality embeddings suitable for both industry and academic applications.

## Model Size

- **Size**: Approximately 450 MB

## Training Details

- **Base Model**: [dbmdz/bert-base-italian-xxl-uncased](https://huggingface.co/dbmdz/bert-base-italian-xxl-uncased)
- **Dataset**: [Italian-BERT-FineTuning-Embeddings](https://huggingface.co/datasets/ArchitRastogi/Italian-BERT-FineTuning-Embeddings)
  - Derived from the C4 dataset using sliding window segmentation and in-document sampling.
  - **Size**: ~5GB (4.5GB train, 0.5GB test)
- **Training Configuration**:
  - **Hardware**: NVIDIA A40 GPU
  - **Epochs**: 3
  - **Total Steps**: 922,958
  - **Training Time**: Approximately 5 days, 2 hours, and 23 minutes
- **Training Objective**: Contrastive Learning

## Evaluation Metrics

Evaluations were performed using the [mMARCO](https://github.com/unicamp-dl/mMARCO) dataset, a multilingual version of MS MARCO. The model was assessed on 6,980 queries.

### Results Comparison

| Metric              | Base Model (`dbmdz/bert-base-italian-xxl-uncased`) | `facebook/mcontriever-msmarco` | **Fine-Tuned Model** |
|---------------------|----------------------------------------------------|--------------------------------|----------------------|
| **Recall@1**        | 0.0026                                             | 0.0828                         | **0.2106**           |
| **Recall@100**      | 0.0417                                             | 0.5028                         | **0.8356**           |
| **Recall@1000**     | 0.2061                                             | 0.8049                         | **0.9719**           |
| **Average Precision** | 0.0050                                          | 0.1397                         | **0.3173**           |
| **NDCG@10**         | 0.0043                                             | 0.1591                         | **0.3601**           |
| **NDCG@100**        | 0.0108                                             | 0.2086                         | **0.4218**           |
| **NDCG@1000**       | 0.0299                                             | 0.2454                         | **0.4391**           |
| **MRR@10**          | 0.0036                                             | 0.1299                         | **0.3047**           |
| **MRR@100**         | 0.0045                                             | 0.1385                         | **0.3167**           |
| **MRR@1000**        | 0.0050                                             | 0.1397                         | **0.3173**           |

**Note**: The fine-tuned model significantly outperforms both the base model and `facebook/mcontriever-msmarco` across all metrics.

## Usage

You can load and use the model directly with the Hugging Face Transformers library:
```python
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("ArchitRastogi/bert-base-italian-embeddings")
model = AutoModelForMaskedLM.from_pretrained("ArchitRastogi/bert-base-italian-embeddings")

# Example usage
text = "Stanchi di non riuscire a trovare il partner perfetto?"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
```

## Intended Use
This model is intended for:

- Information Retrieval (IR): Enhancing search engines and retrieval systems in the Italian language.
- Retrieval-Augmented Generation (RAG): Improving the quality of generated content by providing relevant context.

Suitable for both industry applications and academic research.

## Limitations
- The model may inherit biases present in the C4 dataset.
- Performance is primarily evaluated on mMARCO; results may vary with other datasets.

---

## Contact

**Archit Rastogi**  
📧 architrastogi20@gmail.com