README.md

---
inference: false
language:
- bg
license: mit
datasets:
- oscar
- chitanka
- wikipedia
tags:
- torch
---

# BERT BASE (cased) finetuned on Bulgarian natural-language-inference data

Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
[this paper](https://arxiv.org/abs/1810.04805) and first released in
[this repository](https://github.com/google-research/bert). This model is cased: it does make a difference
between bulgarian and Bulgarian. The training data is Bulgarian text from [OSCAR](https://oscar-corpus.com/post/oscar-2019/), [Chitanka](https://chitanka.info/) and [Wikipedia](https://bg.wikipedia.org/).

It was finetuned on private NLI Bulgarian data.

Then, it was compressed via [progressive module replacing](https://arxiv.org/abs/2002.02925).

### How to use

Here is how to use this model in PyTorch:

```python
>>> import torch
>>> from transformers import AutoModelForSequenceClassification, AutoTokenizer
>>> 
>>> model_id = 'rmihaylov/bert-base-nli-theseus-bg'
>>> model = AutoModelForSequenceClassification.from_pretrained(model_id)
>>> tokenizer = AutoTokenizer.from_pretrained(model_id)
>>>
>>> inputs = tokenizer.encode_plus(
>>>     'Няколко момчета играят футбол.', 
>>>     'Няколко момичета играят футбол.', 
>>>     return_tensors='pt')
>>>
>>> outputs = model(**inputs)
>>> contradiction, entailment, neutral = torch.softmax(outputs[0][0], dim=0).detach()
>>> contradiction, neutral, entailment

(tensor(0.9998), tensor(0.0001), tensor(5.9929e-05))
```