README.md

---
license: apache-2.0
---

This model is compiled for neuronx devices (eg. on inf2 instance).

This original checkpoint is [`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5).

## Export

Here below is the command used for exporting this model:

```bash
optimum-cli export neuron -m BAAI/bge-base-en-v1.5 --sequence_length 384 --batch_size 1 --task feature-extraction bge_emb/
```

## Usage

To use the compiled artifacts for inference, here is an example:

```python
from transformers import AutoTokenizer
from optimum.neuron import NeuronModelForSenetenceTransformers

emb_model = NeuronModelForSenetenceTransformers.from_pretrained("optimum/bge-base-en-v1.5-neuronx")
inputs = tokenizer("Hamilton is considered to be the best musical of human history.", return_tensors="pt")
emb = emb_model(**inputs)

# ["token_embeddings", "sentence_embedding"]
```