Add Azure OpenAI support

README.md

@@ -40,22 +40,23 @@ Clicking on the button will download the file.
 
 # Summary of the LLMs
 
-SlideDeck AI allows the use of different LLMs from four online providers—Hugging Face, Google, Cohere, and Together AI. These service providers—even the latter three—offer generous free usage of relevant LLMs without requiring any billing information.  
+SlideDeck AI allows the use of different LLMs from five online providers—Azure OpenAI, Hugging Face, Google, Cohere, and Together AI. The latter four service providers offer generous free usage of relevant LLMs without requiring any billing information.  
 
-Based on several experiments, SlideDeck AI generally recommends the use of Mistral NeMo and Gemini Flash to generate the slide decks.
+Based on several experiments, SlideDeck AI generally recommends the use of Mistral NeMo, Gemini Flash, and GPT-4o to generate the slide decks.
 
 The supported LLMs offer different styles of content generation. Use one of the following LLMs along with relevant API keys/access tokens, as appropriate, to create the content of the slide deck:
 
-| LLM                              | Provider (code) | Requires API key                                                                     | Characteristics          |
-|:---------------------------------| :------- |:-------------------------------------------------------------------------------------|:-------------------------|
-| Mistral 7B Instruct v0.2         | Hugging Face (`hf`) | Optional but strongly encouraged; [get here](https://huggingface.co/settings/tokens) | Faster, shorter content  |
-| Mistral NeMo Instruct 2407       | Hugging Face (`hf`) | Optional but strongly encouraged; [get here](https://huggingface.co/settings/tokens) | Slower, longer content   |
-| Gemini 1.5 Flash                 | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                            | Faster, longer content   |
-| Gemini 2.0 Flash                 | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                            | Faster, longer content   |
-| Gemini 2.0 Flash Lite            | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                            | Faster, longer content   |
-| Command R+                       | Cohere (`co`) | Mandatory; [get here](https://dashboard.cohere.com/api-keys)                         | Shorter, simpler content |
-| Llama 3.3 70B Instruct Turbo     | Together AI (`to`) | Mandatory; [get here](https://api.together.ai/settings/api-keys)                     | Detailed, slower         |
-| Llama 3.1 8B Instruct Turbo 128K | Together AI (`to`) | Mandatory; [get here](https://api.together.ai/settings/api-keys)                     | Shorter                  |
+| LLM                              | Provider (code) | Requires API key                                                                                                         | Characteristics          |
+|:---------------------------------| :------- |:-------------------------------------------------------------------------------------------------------------------------|:-------------------------|
+| Mistral 7B Instruct v0.2         | Hugging Face (`hf`) | Optional but strongly encouraged; [get here](https://huggingface.co/settings/tokens)                                     | Faster, shorter content  |
+| Mistral NeMo Instruct 2407       | Hugging Face (`hf`) | Optional but strongly encouraged; [get here](https://huggingface.co/settings/tokens)                                     | Slower, longer content   |
+| Gemini 1.5 Flash                 | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                                                                | Faster, longer content   |
+| Gemini 2.0 Flash                 | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                                                                | Faster, longer content   |
+| Gemini 2.0 Flash Lite            | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                                                                | Faster, longer content   |
+| GPT                              | Azure OpenAI (`az`)      | Mandatory; [get here](https://ai.azure.com/resource/playground)  NOTE: You need to have your subscription/billing set up | Faster, longer content   |
+| Command R+                       | Cohere (`co`) | Mandatory; [get here](https://dashboard.cohere.com/api-keys)                                                             | Shorter, simpler content |
+| Llama 3.3 70B Instruct Turbo     | Together AI (`to`) | Mandatory; [get here](https://api.together.ai/settings/api-keys)                                                         | Detailed, slower         |
+| Llama 3.1 8B Instruct Turbo 128K | Together AI (`to`) | Mandatory; [get here](https://api.together.ai/settings/api-keys)                                                         | Shorter                  |
 
 The Mistral models (via Hugging Face) do not mandatorily require an access token. In other words, you are always free to use these two LLMs, subject to Hugging Face's usage constrains. However, you are strongly encouraged to get and use your own Hugging Face access token.
 

app.py

@@ -66,6 +66,9 @@ def are_all_inputs_valid(
         selected_provider: str,
         selected_model: str,
         user_key: str,
+        azure_deployment_url: str = '',
+        azure_endpoint_name: str = '',
+        azure_api_version: str = '',
 ) -> bool:
     """
     Validate user input and LLM selection.
@@ -74,6 +77,9 @@ def are_all_inputs_valid(
     :param selected_provider: The LLM provider.
     :param selected_model: Name of the model.
     :param user_key: User-provided API key.
+    :param azure_deployment_url: Azure OpenAI deployment URL.
+    :param azure_endpoint_name: Azure OpenAI model endpoint.
+    :param azure_api_version: Azure OpenAI API version.
     :return: `True` if all inputs "look" OK; `False` otherwise.
     """
 
@@ -90,11 +96,16 @@ def are_all_inputs_valid(
         handle_error('No valid LLM provider and/or model name found!', False)
         return False
 
-    if not llm_helper.is_valid_llm_provider_model(selected_provider, selected_model, user_key):
+    if not llm_helper.is_valid_llm_provider_model(
+            selected_provider, selected_model, user_key,
+            azure_endpoint_name, azure_deployment_url, azure_api_version
+    ):
         handle_error(
             'The LLM settings do not look correct. Make sure that an API key/access token'
-            ' is provided if the selected LLM requires it. An API key should be 6-64 characters'
-            ' long, only containing alphanumeric characters, hyphens, and underscores.',
+            ' is provided if the selected LLM requires it. An API key should be 6-94 characters'
+            ' long, only containing alphanumeric characters, hyphens, and underscores.\n\n'
+            'If you are using Azure OpenAI, make sure that you have provided the additional and'
+            ' correct configurations.',
             False
         )
         return False
@@ -170,13 +181,35 @@ with st.sidebar:
         api_key_token = st.text_input(
             label=(
                 '3: Paste your API key/access token:\n\n'
-                '*Mandatory* for Cohere, Google Gemini, and Together AI providers.'
+                '*Mandatory* for Azure OpenAI, Cohere, Google Gemini, and Together AI providers.'
                 ' *Optional* for HF Mistral LLMs but still encouraged.\n\n'
             ),
             type='password',
             key='api_key_input'
         )
 
+        # Additional configs for Azure OpenAI
+        with st.expander('**Azure OpenAI-specific configurations**'):
+            azure_endpoint = st.text_input(
+                label=(
+                    '4: Azure endpoint URL, e.g., https://example.openai.azure.com/.\n\n'
+                    '*Mandatory* for Azure OpenAI (only).'
+                )
+            )
+            azure_deployment = st.text_input(
+                label=(
+                    '5: Deployment name on Azure OpenAI:\n\n'
+                    '*Mandatory* for Azure OpenAI (only).'
+                ),
+            )
+            api_version = st.text_input(
+                label=(
+                    '6: API version:\n\n'
+                    '*Mandatory* field. Change based on your deployment configurations.'
+                ),
+                value='2024-05-01-preview',
+            )
+
 
 def build_ui():
     """
@@ -238,7 +271,15 @@ def set_up_chat_ui():
             use_ollama=RUN_IN_OFFLINE_MODE
         )
 
-        if not are_all_inputs_valid(prompt, provider, llm_name, api_key_token):
+        user_key = api_key_token.strip()
+        az_deployment = azure_deployment.strip()
+        az_endpoint = azure_endpoint.strip()
+        api_ver = api_version.strip()
+
+        if not are_all_inputs_valid(
+                prompt, provider, llm_name, user_key,
+                az_deployment, az_endpoint, api_ver
+        ):
             return
 
         logger.info(
@@ -270,7 +311,10 @@ def set_up_chat_ui():
                 provider=provider,
                 model=llm_name,
                 max_new_tokens=gcfg.get_max_output_tokens(llm_provider_to_use),
-                api_key=api_key_token.strip(),
+                api_key=user_key,
+                azure_endpoint_url=az_endpoint,
+                azure_deployment_name=az_deployment,
+                azure_api_version=api_ver,
             )
 
             if not llm:
@@ -282,8 +326,11 @@ def set_up_chat_ui():
                 )
                 return
 
-            for _ in llm.stream(formatted_template):
-                response += _
+            for chunk in llm.stream(formatted_template):
+                if isinstance(chunk, str):
+                    response += chunk
+                else:
+                    response += chunk.content  # AIMessageChunk
 
                 # Update the progress bar with an approx progress percentage
                 progress_bar.progress(

global_config.py

@@ -22,14 +22,21 @@ class GlobalConfig:
     PROVIDER_HUGGING_FACE = 'hf'
     PROVIDER_OLLAMA = 'ol'
     PROVIDER_TOGETHER_AI = 'to'
+    PROVIDER_AZURE_OPENAI = 'az'
     VALID_PROVIDERS = {
         PROVIDER_COHERE,
         PROVIDER_GOOGLE_GEMINI,
         PROVIDER_HUGGING_FACE,
         PROVIDER_OLLAMA,
-        PROVIDER_TOGETHER_AI
+        PROVIDER_TOGETHER_AI,
+        PROVIDER_AZURE_OPENAI,
     }
     VALID_MODELS = {
+        '[az]azure/open-ai': {
+            'description': 'faster, detailed',
+            'max_new_tokens': 8192,
+            'paid': True,
+        },
         '[co]command-r-08-2024': {
             'description': 'simpler, slower',
             'max_new_tokens': 4096,
@@ -79,7 +86,7 @@ class GlobalConfig:
         '- **[to]**: Together AI\n\n'
         '[Find out more](https://github.com/barun-saha/slide-deck-ai?tab=readme-ov-file#summary-of-the-llms)'
     )
-    DEFAULT_MODEL_INDEX = 4
+    DEFAULT_MODEL_INDEX = 5
     LLM_MODEL_TEMPERATURE = 0.2
     LLM_MODEL_MIN_OUTPUT_LENGTH = 100
     LLM_MODEL_MAX_INPUT_LENGTH = 400  # characters
@@ -135,7 +142,7 @@ class GlobalConfig:
         'Remember, the conversational interface is meant to (and will) update yor *initial*'
         ' slide deck. If you want to create a new slide deck on a different topic,'
         ' start a new chat session by reloading this page.\n\n'
-        'Currently, eight *free-to-use* LLMs from four different providers are supported.'
+        'Currently, paid or *free-to-use* LLMs from five different providers are supported.'
         ' If one is not available, choose the other from the dropdown list. A [summary of'
         ' the supported LLMs]('
         'https://github.com/barun-saha/slide-deck-ai/blob/main/README.md#summary-of-the-llms)'

helpers/llm_helper.py

@@ -4,12 +4,14 @@ Helper functions to access LLMs.
 import logging
 import re
 import sys
+import urllib3
 from typing import Tuple, Union
 
 import requests
 from requests.adapters import HTTPAdapter
 from urllib3.util import Retry
-from langchain_core.language_models import BaseLLM
+from langchain_core.language_models import BaseLLM, BaseChatModel
+
 
 sys.path.append('..')
 
@@ -18,14 +20,16 @@ from global_config import GlobalConfig
 
 LLM_PROVIDER_MODEL_REGEX = re.compile(r'\[(.*?)\](.*)')
 OLLAMA_MODEL_REGEX = re.compile(r'[a-zA-Z0-9._:-]+$')
-# 6-64 characters long, only containing alphanumeric characters, hyphens, and underscores
-API_KEY_REGEX = re.compile(r'^[a-zA-Z0-9_-]{6,64}$')
+# 94 characters long, only containing alphanumeric characters, hyphens, and underscores
+API_KEY_REGEX = re.compile(r'^[a-zA-Z0-9_-]{6,94}$')
 HF_API_HEADERS = {'Authorization': f'Bearer {GlobalConfig.HUGGINGFACEHUB_API_TOKEN}'}
 REQUEST_TIMEOUT = 35
 
+
 logger = logging.getLogger(__name__)
 logging.getLogger('httpx').setLevel(logging.WARNING)
 logging.getLogger('httpcore').setLevel(logging.WARNING)
+logging.getLogger('openai').setLevel(logging.ERROR)
 
 retries = Retry(
     total=5,
@@ -66,7 +70,14 @@ def get_provider_model(provider_model: str, use_ollama: bool) -> Tuple[str, str]
     return '', ''
 
 
-def is_valid_llm_provider_model(provider: str, model: str, api_key: str) -> bool:
+def is_valid_llm_provider_model(
+        provider: str,
+        model: str,
+        api_key: str,
+        azure_endpoint_url: str = '',
+        azure_deployment_name: str = '',
+        azure_api_version: str = '',
+) -> bool:
     """
     Verify whether LLM settings are proper.
     This function does not verify whether `api_key` is correct. It only confirms that the key has
@@ -75,6 +86,9 @@ def is_valid_llm_provider_model(provider: str, model: str, api_key: str) -> bool
     :param provider: Name of the LLM provider.
     :param model: Name of the model.
     :param api_key: The API key or access token.
+    :param azure_endpoint_url: Azure OpenAI endpoint URL.
+    :param azure_deployment_name: Azure OpenAI deployment name.
+    :param azure_api_version: Azure OpenAI API version.
     :return: `True` if the settings "look" OK; `False` otherwise.
     """
 
@@ -85,11 +99,19 @@ def is_valid_llm_provider_model(provider: str, model: str, api_key: str) -> bool
         GlobalConfig.PROVIDER_GOOGLE_GEMINI,
         GlobalConfig.PROVIDER_COHERE,
         GlobalConfig.PROVIDER_TOGETHER_AI,
+        GlobalConfig.PROVIDER_AZURE_OPENAI,
     ] and not api_key:
         return False
 
-    if api_key:
-        return API_KEY_REGEX.match(api_key) is not None
+    if api_key and API_KEY_REGEX.match(api_key) is None:
+        return False
+
+    if provider == GlobalConfig.PROVIDER_AZURE_OPENAI:
+        valid_url = urllib3.util.parse_url(azure_endpoint_url)
+        all_status = all(
+            [azure_api_version, azure_deployment_name, str(valid_url)]
+        )
+        return all_status
 
     return True
 
@@ -98,8 +120,11 @@ def get_langchain_llm(
         provider: str,
         model: str,
         max_new_tokens: int,
-        api_key: str = ''
-) -> Union[BaseLLM, None]:
+        api_key: str = '',
+        azure_endpoint_url: str = '',
+        azure_deployment_name: str = '',
+        azure_api_version: str = '',
+) -> Union[BaseLLM, BaseChatModel, None]:
     """
     Get an LLM based on the provider and model specified.
 
@@ -107,7 +132,10 @@ def get_langchain_llm(
     :param model: The name of the LLM.
     :param max_new_tokens: The maximum number of tokens to generate.
     :param api_key: API key or access token to use.
-    :return: An instance of the LLM or `None` in case of any error.
+    :param azure_endpoint_url: Azure OpenAI endpoint URL.
+    :param azure_deployment_name: Azure OpenAI deployment name.
+    :param azure_api_version: Azure OpenAI API version.
+    :return: An instance of the LLM or Chat model; `None` in case of any error.
     """
 
     if provider == GlobalConfig.PROVIDER_HUGGING_FACE:
@@ -149,6 +177,23 @@ def get_langchain_llm(
             }
         )
 
+    if provider == GlobalConfig.PROVIDER_AZURE_OPENAI:
+        from langchain_openai import AzureChatOpenAI
+
+        logger.debug('Getting LLM via Azure OpenAI: %s', model)
+
+        # The `model` parameter is not used here; `azure_deployment` points to the desired name
+        return AzureChatOpenAI(
+            azure_deployment=azure_deployment_name,
+            api_version=azure_api_version,
+            azure_endpoint=azure_endpoint_url,
+            temperature=GlobalConfig.LLM_MODEL_TEMPERATURE,
+            max_tokens=max_new_tokens,
+            timeout=None,
+            max_retries=1,
+            api_key=api_key,
+        )
+
     if provider == GlobalConfig.PROVIDER_COHERE:
         from langchain_cohere.llms import Cohere
 

requirements.txt

@@ -14,6 +14,7 @@ langchain-google-genai==2.0.6
 langchain-cohere==0.3.3
 langchain-together==0.3.0
 langchain-ollama==0.2.1
+langchain-openai==0.3.3
 streamlit~=1.38.0
 
 python-pptx~=0.6.21