meta-llama

/

Llama-3.2-11B-Vision-Instruct

like 695

If I run the code like...

(...)
outputs = model.generate(**inputs, max_new_tokens=30)

Then this erorr occurs:

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Environments:

# python 3.10
# 6 x A6000 GPUs
transformers==4.45.2
torch==2.4.1
torchaudio==2.4.1
torchvision==0.19.1
accelerate==1.0.0

Question) What is the recommended CUDA version? I used CUDA 12.2 or 11.8.