Inference Providers

Chat Completion

Generate a response given a list of messages in a conversational context, supporting both conversational Language Models (LLMs) and conversational Vision-Language Models (VLMs). This is a subtask of text-generation and image-text-to-text.

Recommended models

Conversational Large Language Models (LLMs)

• google/gemma-2-2b-it: A text-generation model trained to follow instructions.
• deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B: Smaller variant of one of the most powerful models.
• meta-llama/Meta-Llama-3.1-8B-Instruct: Very powerful text generation model trained to follow instructions.
• microsoft/phi-4: Powerful text generation model by Microsoft.
• Qwen/Qwen2.5-Coder-32B-Instruct: Text generation model used to write code.
• deepseek-ai/DeepSeek-R1: Powerful reasoning based open large language model.

Conversational Vision-Language Models (VLMs)

• Qwen/Qwen2.5-VL-7B-Instruct: Strong image-text-to-text model.

API Playground

For Chat Completion models, we provide an interactive UI Playground for easier testing:

• Quickly iterate on your prompts from the UI.
• Set and override system, assistant and user messages.
• Browse and select models currently available on the Inference API.
• Compare the output of two models side-by-side.
• Adjust requests parameters from the UI.
• Easily switch between UI view and code snippets.

Access the Inference UI Playground and start exploring: https://huggingface.co/playground

Using the API

The API supports:

• Using the chat completion API compatible with the OpenAI SDK.
• Using grammars, constraints, and tools.
• Streaming the output

Code snippet example for conversational LLMs

from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="cerebras",
    api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxx",
)

completion = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
    max_tokens=500,
)

print(completion.choices[0].message)

Code snippet example for conversational VLMs

from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="fireworks-ai",
    api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxx",
)

completion = client.chat.completions.create(
    model="meta-llama/Llama-3.2-11B-Vision-Instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Describe this image in one sentence."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
                    }
                }
            ]
        }
    ],
    max_tokens=500,
)

print(completion.choices[0].message)

API specification

Request

Response

Output type depends on the stream input parameter. If stream is false (default), the response will be a JSON object with the following fields:

If stream is true, generated tokens are returned as a stream, using Server-Sent Events (SSE). For more information about streaming, check out this guide.