---
language:
- en
- de
- fr
- it
- pt
- hi
- es
- th
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- facebook
- meta
- pytorch
- llama
- llama-3
---

This repository is an pre-release checkpoint for Llama 3.2 90B Vision.

It contains two versions of the model, for use with `transformers` and with the original `llama3` codebase (under the `original` directory).


## Inference with transformers

Please, install the in-progress development wheel from https://huggingface.co/nltpt/transformers/tree/main.

This is an example inference snippet (API subject to change):

```python
import requests
import torch
from PIL import Image
from transformers import MllamaForConditionalGeneration, AutoProcessor

model_id = "nltpt/Llama-3.2-90B-Vision"
model = MllamaForConditionalGeneration.from_pretrained(model_id, device_map="auto", torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained(model_id)

prompt = "<|image|><|begin_of_text|>If I had to write a haiku for this one"
url = "https://llava-vl.github.io/static/images/view.jpg"
raw_image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(text=prompt, images=raw_image, return_tensors="pt").to(model.device)
output = model.generate(**inputs, do_sample=False, max_new_tokens=25)
print(processor.decode(output[0], skip_special_tokens=True))
```

Output:
```text
If I had to write a haiku for this one, it would be:.\nA dock on a lake.\nA mountain in the distance.\nA long exposure.\
```

## Running the original checkpoints
The package installed will provide three binaries:

1. example_chat_completion
2. example_text_completion
3. multimodal_example_chat_completion

You can invoke them via torchrun by doing the following:
```
CHECKPOINT_DIR=~/.llama/checkpoints/Llama-3.2-90B-Vision/

torchrun `which multimodal_example_chat_completion` "$CHECKPOINT_DIR"
```
You can study the code for the script by doing something like:
```
PACKAGE_DIR=$(pip show -f llama-models | grep Location | awk '{ print $2 }')

echo "Scripts are in the directory: $PACKAGE_DIR/llama-models/scripts/"
```