File size: 4,095 Bytes
a8027a9 e7a297e a8027a9 e7a297e a8027a9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 |
---
language: en
license: mit
tags:
- natural-language-inference
- sentence-transformers
- transformers
- nlp
- model-card
---
# bge-small-en-v1.5-nli
- **Base Model:** [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)
- **Task:** Natural Language Inference (NLI)
- **Framework:** Hugging Face Transformers, Sentence Transformers
bge-small-en-v1.5-nli is a fine-tuned NLI model that classifies the relationship between pairs of sentences into three categories: entailment, neutral, and contradiction. It enhances the capabilities of [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) for improved performance on NLI tasks.
## Intended Use
bge-small-en-v1.5-nli is ideal for applications requiring understanding of logical relationships between sentences, including:
- Semantic textual similarity
- Question answering
- Dialogue systems
- Content moderation
## Performance
bge-small-en-v1.5-nli was trained on the [sentence-transformers/all-nli](https://huggingface.co/datasets/sentence-transformers/all-nli) dataset, achieving competitive results in sentence pair classification.
Performance on the MNLI matched validation set:
- Accuracy: 0.7679
- Precision: 0.77
- Recall: 0.77
- F1-score: 0.77
## Training details
<details>
<summary><strong>Training Details</strong></summary>
- **Dataset:**
- Used [sentence-transformers/all-nli](https://huggingface.co/datasets/sentence-transformers/all-nli).
- **Sampling:**
- 100 000 training samples and 10 000 evaluation samples.
- **Fine-tuning Process:**
- Custom Python script with adaptive precision training (bfloat16).
- Early stopping based on evaluation loss.
- **Hyperparameters:**
- **Learning Rate:** 2e-5
- **Batch Size:** 64
- **Optimizer:** AdamW (weight decay: 0.01)
- **Training Duration:** Up to 10 epochs
</details>
<details>
<summary><strong>Reproducibility</strong></summary>
To ensure reproducibility:
- Fixed random seed: 42
- Environment:
- Python: 3.10.12
- PyTorch: 2.5.1
- Transformers: 4.44.2
</details>
## Usage Instructions
## Using Sentence Transformers
```python
from sentence_transformers import CrossEncoder
model_name = "agentlans/bge-small-en-v1.5-nli"
model = CrossEncoder(model_name)
scores = model.predict(
[
("A man is eating pizza", "A man eats something"),
(
"A black race car starts up in front of a crowd of people.",
"A man is driving down a lonely road.",
),
]
)
label_mapping = ["entailment", "neutral", "contradiction"]
labels = [label_mapping[score_max] for score_max in scores.argmax(axis=1)]
print(labels)
# Output: ['entailment', 'contradiction']
```
## Using Transformers Library
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_name = "agentlans/bge-small-en-v1.5-nli"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
features = tokenizer(
[
"A man is eating pizza",
"A black race car starts up in front of a crowd of people.",
],
["A man eats something", "A man is driving down a lonely road."],
padding=True,
truncation=True,
return_tensors="pt",
)
model.eval()
with torch.no_grad():
scores = model(**features).logits
label_mapping = ["entailment", "neutral", "contradiction"]
labels = [label_mapping[score_max] for score_max in scores.argmax(dim=1)]
print(labels)
# Output: ['entailment', 'contradiction']
```
## Limitations and Ethical Considerations
bge-small-en-v1.5-nli may reflect biases present in the training data. Users should evaluate its performance in specific contexts to ensure fairness and accuracy.
## Conclusion
bge-small-en-v1.5-nli offers a robust solution for NLI tasks, enhancing [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)'s capabilities with straightforward integration into existing frameworks. It aids developers in building intelligent applications that require nuanced language understanding.
|