Update pipeline example

README.md

@@ -32,11 +32,37 @@ other versions on a task that interests you.
 
 ### How to use
 
-Here's the prompt template for this model:
+Here's the prompt template for this model but we recomment to use the chat templates to format the prompt with `processor.apply_chat_template()`.
+That will apply the correct template for a given checkpoint for you.
+
 ```
 "[INST] <image>\nWhat is shown in this image? [/INST]"
 ```
-You can load and use the model like following:
+
+To run the model with the `pipeline`, see the below example:
+
+```python
+from transformers import pipeline
+
+pipe = pipeline("image-text-to-text", model="llava-hf/llava-v1.6-mistral-7b-hf")
+messages = [
+    {
+      "role": "user",
+      "content": [
+          {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"},
+          {"type": "text", "text": "What does the label 15 represent? (1) lava (2) core (3) tunnel (4) ash cloud"},
+        ],
+    },
+]
+
+out = pipe(text=messages, max_new_tokens=20)
+print(out)
+>>> [{'input_text': [{'role': 'user', 'content': [{'type': 'image', 'url': 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg'}, {'type': 'text', 'text': 'What does the label 15 represent? (1) lava (2) core (3) tunnel (4) ash cloud'}]}], 'generated_text': 'Lava'}]
+```
+
+
+You can also load and use the model like following:
+
 ```python
 from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration
 import torch