Hugging Face's logo

Spaces:
Intel
/
intel-ai-enterprise-inference
Running

App Files Community
intel-ai-enterprise-inference / README.md
Benjamin Consolvo
license and readme
ec12af2
preview code
|
raw
history blame
3.5 kB
metadata
title: Intel® AI for Enterprise Inference
emoji: 📚
colorFrom: yellow
colorTo: purple
sdk: streamlit
sdk_version: 1.42.2
app_file: app.py
pinned: false
license: apache-2.0
short_description: LLM Chatbot on Denvr Dataworks and Intel Gaudi

LLM Chat App

This application provides a user-friendly interface to interact with various LLM models hosted on Denvr Dataworks, powered by Intel Gaudi accelerators. The chatbot supports streaming responses and offers a selection of different language models, including Llama models and DeepSeek models.

Features

  • Model Selection: Choose from multiple LLM models hosted on Intel Gaudi hardware
  • Chat Interface: Clean and intuitive Streamlit chat UI
  • Streaming Responses: Real-time streaming of AI-generated responses, including formatted code blocks if requested
  • Conversation History: Maintain context throughout your conversation
  • New Chat: Option to start a fresh conversation at any time

Installation

Prerequisites

  • Python 3.7+
  • Streamlit
  • OpenAI-compatible API key and endpoint

Setup

  1. Clone the repository:

git clone https://github.com/opea-project/Enterprise-Inference/ cd examples/chatapp


2. Install the required packages:
```bash
pip install -r requirements.txt

Configuration

Secrets Management

This application requires API credentials to be set up in Streamlit's secrets management:

  1. On Hugging Face Spaces:

    • Add your OpenAI-compatible API key under "Secrets" in the HF settings
    • Add the base URL for your model endpoint under "Variables" as base_url

  2. For local development:

    • Create a .streamlit/secrets.toml file with:

openai_apikey = "your-api-key-here" ```

  • Set the base_url environment variable to point to your model endpoint with hosted models.

Running the Application

On Hugging Face Spaces

You can create a new Hugging Face Space here, and then use git operations to clone, commit, and push your code changes directly to your Space. Here is the live link to the Space that you can replicate: https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference.

Local Development

Run the Streamlit application locally:

streamlit run app.py

Using the Chatbot

  1. Select your desired LLM model from the dropdown menu
  2. Type your message in the chat input field
  3. View the AI's response as it streams in real-time
  4. Continue the conversation or start a new chat using the "Start New Chat" button

Getting API Access

To use this application, you need an OpenAI-compatible API key from Denvr Dataworks:

  1. Visit Denvr Dataworks to check the latest available models
  2. Sign up for API access to receive your API key
  3. Configure the key in your Streamlit secrets

Troubleshooting

  • API Key Issues: Ensure your API key is correctly set in the Streamlit secrets
  • Model Unavailability: If a model is not responding, try selecting a different model
  • Error Messages: Check the error output for specific API or connection issues
  • Rate Limiting: You might encounter rate limits depending on your API plan

Community and Support

Join other AI developers on Intel's DevHub Discord server for discussions, support, and updates.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.