Pylint & CVE fix

.gitattributes

@@ -33,5 +33,3 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
-*.pptx filter=lfs diff=lfs merge=lfs -text
-pptx_templates/Minimalist_sales_pitch.pptx filter=lfs diff=lfs merge=lfs -text

.streamlit/config.toml

@@ -1,10 +0,0 @@
-[server]
-runOnSave = true
-headless = false
-maxUploadSize = 0
-
-[browser]
-gatherUsageStats = false
-
-[theme]
-base = "dark"

README.md

@@ -4,7 +4,7 @@ emoji: 🏢
 colorFrom: yellow
 colorTo: green
 sdk: streamlit
-sdk_version: 1.32.2
+sdk_version: 1.26.0
 app_file: app.py
 pinned: false
 license: mit
@@ -16,88 +16,36 @@ We spend a lot of time on creating the slides and organizing our thoughts for an
 With SlideDeck AI, co-create slide decks on any topic with Generative Artificial Intelligence.
 Describe your topic and let SlideDeck AI generate a PowerPoint slide deck for you—it's as simple as that!
 
+SlideDeck AI is powered by [Mistral 7B Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1).
+Originally, it was built using the Llama 2 API provided by Clarifai.
 
 # Process
 
 SlideDeck AI works in the following way:
 
-1. Given a topic description, it uses a Large Language Model (LLM) to generate the *initial* content of the slides. 
+1. Given a topic description, it uses Mistral 7B Instruct to generate the outline/contents of the slides. 
 The output is generated as structured JSON data based on a pre-defined schema.
-2. Next, it uses the keywords from the JSON output to search and download a few images with a certain probability.
-3. Subsequently, it uses the `python-pptx` library to generate the slides, 
+2. Subsequently, it uses the `python-pptx` library to generate the slides, 
 based on the JSON data from the previous step. 
-A user can choose from a set of pre-defined presentation templates.
-4. At this stage onward, a user can provide additional instructions to *refine* the content.
-For example, one can ask to add another slide or modify an existing slide.
-A history of instructions is maintained.
-5. Every time SlideDeck AI generates a PowerPoint presentation, a download button is provided.
-Clicking on the button will download the file.
+Here, a user can choose from a set of three pre-defined presentation templates.
+3. In addition, it uses Metaphor to fetch Web pages related to the topic.
 
-
-# Summary of the LLMs
-
-Different LLMs offer different styles of content generation. Use one of the following LLMs along with relevant API keys/access tokens, as appropriate, to create the content of the slide deck:
-
-| LLM | Provider (code) | Requires API key                                                            | Characteristics |
-| :-------- | :------- |:----------------------------------------------------------------------------| :------- |
-| Mistral 7B Instruct v0.2 | Hugging Face (`hf`) | Optional but encouraged; [get here](https://huggingface.co/settings/tokens) | Faster, shorter content |
-| Mistral Nemo Instruct 2407 | Hugging Face (`hf`) | Optional but encouraged; [get here](https://huggingface.co/settings/tokens) | Slower, longer content |
-| Gemini 1.5 Flash | Google Gemini API (`gg`) | Mandatory; [get here](https://aistudio.google.com/apikey)                   | Faster, longer content |
-| Command R+ | Cohere (`co`) | Mandatory; [get here](https://dashboard.cohere.com/api-keys)                | Shorter, simpler content |
-
-The Mistral models do not mandatorily require an access token. However, you are encouraged to get and use your own Hugging Face access token.
-
-In addition, offline LLMs provided by Ollama can be used. Read below to know more. 
-
-
-# Icons
-
-SlideDeck AI uses a subset of icons from [bootstrap-icons-1.11.3](https://github.com/twbs/icons)
- (MIT license) in the slides. A few icons from [SVG Repo](https://www.svgrepo.com/)
-(CC0, MIT, and Apache licenses) are also used. 
+4. ~~Finally, it uses Stable Diffusion 2 to generate an image, based on the title and each slide heading.~~
 
 
 # Local Development
 
-SlideDeck AI uses LLMs via different providers, such as Hugging Face, Google, and Gemini.
-To run this project by yourself, you need to provide the `HUGGINGFACEHUB_API_TOKEN` API key,
-for example, in a `.env` file. Alternatively, you can provide the access token in the app's user interface itself (UI). For other LLM providers, the API key can only be specified in the UI.  For image search, the `PEXEL_API_KEY` should be made available as an environment variable. 
-Visit the respective websites to obtain the API keys.
-
-## Offline LLMs Using Ollama
-
-SlideDeck AI allows the use of offline LLMs to generate the contents of the slide decks. This is typically suitable for individuals or organizations who would like to use self-hosted LLMs for privacy concerns, for example.
-
-Offline LLMs are made available via Ollama. Therefore, a pre-requisite here is to have [Ollama installed](https://ollama.com/download) on the system and the desired [LLM](https://ollama.com/search) pulled locally.
-
-In addition, the `RUN_IN_OFFLINE_MODE` environment variable needs to be set to `True` to enable the offline mode. This, for example, can be done using a `.env` file or from the terminal. The typical steps to use SlideDeck AI in offline mode (in a `bash` shell) are as follows:
-
-```bash
-ollama list  # View locally available LLMs
-export RUN_IN_OFFLINE_MODE=True  # Enable the offline mode to use Ollama
-git clone https://github.com/barun-saha/slide-deck-ai.git
-cd slide-deck-ai
-python -m venv venv  # Create a virtual environment
-source venv/bin/activate  # On a Linux system
-pip install -r requirements.txt
-streamlit run ./app.py  # Run the application
-```
-
-The `.env` file should be created inside the `slide-deck-ai` directory. 
-
-The UI is similar to the online mode. However, rather than selecting an LLM from a list, one has to write the name of the Ollama model to be used in a textbox. There is no API key asked here.
-
-The online and offline modes are mutually exclusive. So, setting `RUN_IN_OFFLINE_MODE` to `False` will make SlideDeck AI use the online LLMs (i.e., the "original mode."). By default, `RUN_IN_OFFLINE_MODE` is set to `False`.
-
-Finally, the focus is on using offline LLMs, not going completely offline. So, Internet connectivity would still be required to fetch the images from Pexels. 
+SlideDeck AI uses [Mistral 7B Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) 
+via the Hugging Face Inference API.
+To run this project by yourself, you need to provide the `HUGGINGFACEHUB_API_TOKEN` and `METAPHOR_API_KEY` API keys,
+for example, in a `.env` file. Visit the respective websites to obtain the keys.
 
 
 # Live Demo
 
-- [SlideDeck AI](https://huggingface.co/spaces/barunsaha/slide-deck-ai) on Hugging Face Spaces
-- [Demo video](https://youtu.be/QvAKzNKtk9k) of the chat interface on YouTube
+[SlideDeck AI](https://huggingface.co/spaces/barunsaha/slide-deck-ai)
 
 
 # Award
 
-SlideDeck AI has won the 3rd Place in the [Llama 2 Hackathon with Clarifai](https://lablab.ai/event/llama-2-hackathon-with-clarifai) in 2023.
+SlideDeck AI has won the 3rd Place in the [Llama 2 Hackathon with Clarifai](https://lablab.ai/event/llama-2-hackathon-with-clarifai).

app.py

@@ -1,493 +1,310 @@
-"""
-Streamlit app containing the UI and the application logic.
-"""
-import datetime
-import logging
-import os
 import pathlib
-import random
+import logging
 import tempfile
-from typing import List, Union
+from typing import List, Tuple
 
-import httpx
-import huggingface_hub
 import json5
-import ollama
-import requests
+import metaphor_python as metaphor
 import streamlit as st
-from dotenv import load_dotenv
-from langchain_community.chat_message_histories import StreamlitChatMessageHistory
-from langchain_core.messages import HumanMessage
-from langchain_core.prompts import ChatPromptTemplate
 
-import global_config as gcfg
+import llm_helper
+import pptx_helper
 from global_config import GlobalConfig
-from helpers import llm_helper, pptx_helper, text_helper
-
-
-load_dotenv()
-
 
-RUN_IN_OFFLINE_MODE = os.getenv('RUN_IN_OFFLINE_MODE', 'False').lower() == 'true'
 
+APP_TEXT = json5.loads(open(GlobalConfig.APP_STRINGS_FILE, 'r', encoding='utf-8').read())
+GB_CONVERTER = 2 ** 30
 
-@st.cache_data
-def _load_strings() -> dict:
-    """
-    Load various strings to be displayed in the app.
-    :return: The dictionary of strings.
-    """
 
-    with open(GlobalConfig.APP_STRINGS_FILE, 'r', encoding='utf-8') as in_file:
-        return json5.loads(in_file.read())
+logging.basicConfig(
+    level=GlobalConfig.LOG_LEVEL,
+    format='%(asctime)s - %(message)s',
+)
 
 
 @st.cache_data
-def _get_prompt_template(is_refinement: bool) -> str:
+def get_contents_wrapper(text: str) -> str:
     """
-    Return a prompt template.
+    Fetch and cache the slide deck contents on a topic by calling an external API.
 
-    :param is_refinement: Whether this is the initial or refinement prompt.
-    :return: The prompt template as f-string.
+    :param text: The presentation topic
+    :return: The slide deck contents or outline in JSON format
     """
 
-    if is_refinement:
-        with open(GlobalConfig.REFINEMENT_PROMPT_TEMPLATE, 'r', encoding='utf-8') as in_file:
-            template = in_file.read()
-    else:
-        with open(GlobalConfig.INITIAL_PROMPT_TEMPLATE, 'r', encoding='utf-8') as in_file:
-            template = in_file.read()
-
-    return template
+    logging.info('LLM call because of cache miss...')
+    return llm_helper.generate_slides_content(text).strip()
 
 
-def are_all_inputs_valid(
-        user_prompt: str,
-        selected_provider: str,
-        selected_model: str,
-        user_key: str,
-) -> bool:
+@st.cache_resource
+def get_metaphor_client_wrapper() -> metaphor.Metaphor:
     """
-    Validate user input and LLM selection.
+    Create a Metaphor client for semantic Web search.
 
-    :param user_prompt: The prompt.
-    :param selected_provider: The LLM provider.
-    :param selected_model: Name of the model.
-    :param user_key: User-provided API key.
-    :return: `True` if all inputs "look" OK; `False` otherwise.
+    :return: Metaphor instance
     """
 
-    if not text_helper.is_valid_prompt(user_prompt):
-        handle_error(
-            'Not enough information provided!'
-            ' Please be a little more descriptive and type a few words'
-            ' with a few characters :)',
-            False
-        )
-        return False
-
-    if not selected_provider or not selected_model:
-        handle_error('No valid LLM provider and/or model name found!', False)
-        return False
-
-    if not llm_helper.is_valid_llm_provider_model(selected_provider, selected_model, user_key):
-        handle_error(
-            'The LLM settings do not look correct. Make sure that an API key/access token'
-            ' is provided if the selected LLM requires it. An API key should be 6-64 characters'
-            ' long, only containing alphanumeric characters, hyphens, and underscores.',
-            False
-        )
-        return False
-
-    return True
+    return metaphor.Metaphor(api_key=GlobalConfig.METAPHOR_API_KEY)
 
 
-def handle_error(error_msg: str, should_log: bool):
-    """
-    Display an error message in the app.
-
-    :param error_msg: The error message to be displayed.
-    :param should_log: If `True`, log the message.
+@st.cache_data
+def get_web_search_results_wrapper(text: str) -> List[Tuple[str, str]]:
     """
+    Fetch and cache the Web search results on a given topic.
 
-    if should_log:
-        logger.error(error_msg)
-
-    st.error(error_msg)
-
-
-def reset_api_key():
-    """
-    Clear API key input when a different LLM is selected from the dropdown list.
+    :param text: The topic
+    :return: A list of (title, link) tuples
     """
 
-    st.session_state.api_key_input = ''
-
-
-APP_TEXT = _load_strings()
-
-# Session variables
-CHAT_MESSAGES = 'chat_messages'
-DOWNLOAD_FILE_KEY = 'download_file_name'
-IS_IT_REFINEMENT = 'is_it_refinement'
-
-
-logger = logging.getLogger(__name__)
-
-texts = list(GlobalConfig.PPTX_TEMPLATE_FILES.keys())
-captions = [GlobalConfig.PPTX_TEMPLATE_FILES[x]['caption'] for x in texts]
-
-with st.sidebar:
-    # The PPT templates
-    pptx_template = st.sidebar.radio(
-        '1: Select a presentation template:',
-        texts,
-        captions=captions,
-        horizontal=True
+    results = []
+    search_results = get_metaphor_client_wrapper().search(
+        text,
+        use_autoprompt=True,
+        num_results=5
     )
 
-    if RUN_IN_OFFLINE_MODE:
-        llm_provider_to_use = st.text_input(
-            label='2: Enter Ollama model name to use:',
-            help=(
-                'Specify a correct, locally available LLM, found by running `ollama list`, for'
-                ' example `mistral:v0.2` and `mistral-nemo:latest`. Having an Ollama-compatible'
-                ' and supported GPU is strongly recommended.'
-            )
-        )
-        api_key_token: str = ''
-    else:
-        # The LLMs
-        llm_provider_to_use = st.sidebar.selectbox(
-            label='2: Select an LLM to use:',
-            options=[f'{k} ({v["description"]})' for k, v in GlobalConfig.VALID_MODELS.items()],
-            index=GlobalConfig.DEFAULT_MODEL_INDEX,
-            help=GlobalConfig.LLM_PROVIDER_HELP,
-            on_change=reset_api_key
-        ).split(' ')[0]
-
-        # The API key/access token
-        api_key_token = st.text_input(
-            label=(
-                '3: Paste your API key/access token:\n\n'
-                '*Mandatory* for Cohere and Gemini LLMs.'
-                ' *Optional* for HF Mistral LLMs but still encouraged.\n\n'
-            ),
-            type='password',
-            key='api_key_input'
-        )
+    for a_result in search_results.results:
+        results.append((a_result.title, a_result.url))
+
+    return results
+
+
+# def get_disk_used_percentage() -> float:
+#     """
+#     Compute the disk usage.
+#
+#     :return: Percentage of the disk space currently used
+#     """
+#
+#     total, used, free = shutil.disk_usage(__file__)
+#     total = total // GB_CONVERTER
+#     used = used // GB_CONVERTER
+#     free = free // GB_CONVERTER
+#     used_perc = 100.0 * used / total
+#
+#     logging.debug(f'Total: {total} GB\n'
+#                   f'Used: {used} GB\n'
+#                   f'Free: {free} GB')
+#
+#     logging.debug('\n'.join(os.listdir()))
+#
+#     return used_perc
 
 
 def build_ui():
     """
-    Display the input elements for content generation.
+    Display the input elements for content generation. Only covers the first step.
     """
 
+    # get_disk_used_percentage()
+
     st.title(APP_TEXT['app_name'])
     st.subheader(APP_TEXT['caption'])
     st.markdown(
-        '![Visitors](https://api.visitorbadge.io/api/visitors?path=https%3A%2F%2Fhuggingface.co%2Fspaces%2Fbarunsaha%2Fslide-deck-ai&countColor=%23263759)'  # noqa: E501
+        'Powered by'
+        ' [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2).'
+    )
+    st.markdown(
+        '*If the JSON is generated or parsed incorrectly, try again later by making minor changes'
+        ' to the input text.*'
     )
 
-    today = datetime.date.today()
-    if today.month == 1 and 1 <= today.day <= 15:
-        st.success(
-            (
-                'Wishing you a happy and successful New Year!'
-                ' It is your appreciation that keeps SlideDeck AI going.'
-                f' May you make some great slide decks in {today.year} ✨'
-            ),
-            icon='🎆'
+    with st.form('my_form'):
+        # Topic input
+        try:
+            with open(GlobalConfig.PRELOAD_DATA_FILE, 'r', encoding='utf-8') as in_file:
+                preload_data = json5.loads(in_file.read())
+        except (FileExistsError, FileNotFoundError):
+            preload_data = {'topic': '', 'audience': ''}
+
+        topic = st.text_area(
+            APP_TEXT['input_labels'][0],
+            value=preload_data['topic']
         )
 
-    with st.expander('Usage Policies and Limitations'):
-        st.text(APP_TEXT['tos'] + '\n\n' + APP_TEXT['tos2'])
+        texts = list(GlobalConfig.PPTX_TEMPLATE_FILES.keys())
+        captions = [GlobalConfig.PPTX_TEMPLATE_FILES[x]['caption'] for x in texts]
 
-    set_up_chat_ui()
+        pptx_template = st.radio(
+            'Select a presentation template:',
+            texts,
+            captions=captions,
+            horizontal=True
+        )
 
+        st.divider()
+        submit = st.form_submit_button('Generate slide deck')
 
-def set_up_chat_ui():
-    """
-    Prepare the chat interface and related functionality.
-    """
+    if submit:
+        # st.write(f'Clicked {time.time()}')
+        st.session_state.submitted = True
 
-    with st.expander('Usage Instructions'):
-        st.markdown(GlobalConfig.CHAT_USAGE_INSTRUCTIONS)
+    # https://github.com/streamlit/streamlit/issues/3832#issuecomment-1138994421
+    if 'submitted' in st.session_state:
+        progress_text = 'Generating the slides...give it a moment'
+        progress_bar = st.progress(0, text=progress_text)
 
-    st.info(APP_TEXT['like_feedback'])
-    st.chat_message('ai').write(random.choice(APP_TEXT['ai_greetings']))
+        topic_txt = topic.strip()
+        generate_presentation(topic_txt, pptx_template, progress_bar)
 
-    history = StreamlitChatMessageHistory(key=CHAT_MESSAGES)
-    prompt_template = ChatPromptTemplate.from_template(
-        _get_prompt_template(
-            is_refinement=_is_it_refinement()
-        )
+    st.divider()
+    st.text(APP_TEXT['tos'])
+    st.text(APP_TEXT['tos2'])
+
+    st.markdown(
+        '![Visitors]'
+        '(https://api.visitorbadge.io/api/visitors?path=https%3A%2F%2Fhuggingface.co%2Fspaces%2Fbarunsaha%2Fslide-deck-ai&countColor=%23263759)'
     )
 
-    # Since Streamlit app reloads at every interaction, display the chat history
-    # from the save session state
-    for msg in history.messages:
-        st.chat_message(msg.type).code(msg.content, language='json')
-
-    if prompt := st.chat_input(
-        placeholder=APP_TEXT['chat_placeholder'],
-        max_chars=GlobalConfig.LLM_MODEL_MAX_INPUT_LENGTH
-    ):
-        provider, llm_name = llm_helper.get_provider_model(
-            llm_provider_to_use,
-            use_ollama=RUN_IN_OFFLINE_MODE
-        )
 
-        if not are_all_inputs_valid(prompt, provider, llm_name, api_key_token):
-            return
+def generate_presentation(topic: str, pptx_template: str, progress_bar):
+    """
+    Process the inputs to generate the slides.
 
-        logger.info(
-            'User input: %s | #characters: %d | LLM: %s',
-            prompt, len(prompt), llm_name
-        )
-        st.chat_message('user').write(prompt)
-
-        if _is_it_refinement():
-            user_messages = _get_user_messages()
-            user_messages.append(prompt)
-            list_of_msgs = [
-                f'{idx + 1}. {msg}' for idx, msg in enumerate(user_messages)
-            ]
-            formatted_template = prompt_template.format(
-                **{
-                    'instructions': '\n'.join(list_of_msgs),
-                    'previous_content': _get_last_response(),
-                }
-            )
-        else:
-            formatted_template = prompt_template.format(**{'question': prompt})
-
-        progress_bar = st.progress(0, 'Preparing to call LLM...')
-        response = ''
+    :param topic: The presentation topic based on which contents are to be generated
+    :param pptx_template: The PowerPoint template name to be used
+    :param progress_bar: Progress bar from the page
+    :return:
+    """
+
+    topic_length = len(topic)
+    logging.debug('Input length:: topic: %s', topic_length)
+
+    if topic_length >= 10:
+        logging.debug('Topic: %s', topic)
+        target_length = min(topic_length, GlobalConfig.LLM_MODEL_MAX_INPUT_LENGTH)
 
         try:
-            llm = llm_helper.get_langchain_llm(
-                provider=provider,
-                model=llm_name,
-                max_new_tokens=gcfg.get_max_output_tokens(llm_provider_to_use),
-                api_key=api_key_token.strip(),
-            )
-
-            if not llm:
-                handle_error(
-                    'Failed to create an LLM instance! Make sure that you have selected the'
-                    ' correct model from the dropdown list and have provided correct API key'
-                    ' or access token.',
-                    False
+            # Step 1: Generate the contents in JSON format using an LLM
+            json_str = process_slides_contents(topic[:target_length], progress_bar)
+            logging.debug('Truncated topic: %s', topic[:target_length])
+            logging.debug('Length of JSON: %d', len(json_str))
+
+            # Step 2: Generate the slide deck based on the template specified
+            if len(json_str) > 0:
+                st.info(
+                    'Tip: The generated content doesn\'t look so great?'
+                    ' Need alternatives? Just change your description text and try again.',
+                    icon="💡️"
+                )
+            else:
+                st.error(
+                    'Unfortunately, JSON generation failed, so the next steps would lead'
+                    ' to nowhere. Try again or come back later.'
                 )
                 return
 
-            for _ in llm.stream(formatted_template):
-                response += _
-
-                # Update the progress bar with an approx progress percentage
-                progress_bar.progress(
-                    min(
-                        len(response) / gcfg.get_max_output_tokens(llm_provider_to_use),
-                        0.95
-                    ),
-                    text='Streaming content...this might take a while...'
-                )
-        except (httpx.ConnectError, requests.exceptions.ConnectionError):
-            handle_error(
-                'A connection error occurred while streaming content from the LLM endpoint.'
-                ' Unfortunately, the slide deck cannot be generated. Please try again later.'
-                ' Alternatively, try selecting a different LLM from the dropdown list. If you are'
-                ' using Ollama, make sure that Ollama is already running on your system.',
-                True
-            )
-            return
-        except huggingface_hub.errors.ValidationError as ve:
-            handle_error(
-                f'An error occurred while trying to generate the content: {ve}'
-                '\nPlease try again with a significantly shorter input text.',
-                True
-            )
-            return
-        except ollama.ResponseError:
-            handle_error(
-                f'The model `{llm_name}` is unavailable with Ollama on your system.'
-                f' Make sure that you have provided the correct LLM name or pull it using'
-                f' `ollama pull {llm_name}`. View LLMs available locally by running `ollama list`.',
-                True
-            )
-            return
-        except Exception as ex:
-            handle_error(
-                f'An unexpected error occurred while generating the content: {ex}'
-                '\nPlease try again later, possibly with different inputs.'
-                ' Alternatively, try selecting a different LLM from the dropdown list.'
-                ' If you are using Cohere or Gemini models, make sure that you have provided'
-                ' a correct API key.',
-                True
-            )
-            return
-
-        history.add_user_message(prompt)
-        history.add_ai_message(response)
-
-        # The content has been generated as JSON
-        # There maybe trailing ``` at the end of the response -- remove them
-        # To be careful: ``` may be part of the content as well when code is generated
-        response = text_helper.get_clean_json(response)
-        logger.info(
-            'Cleaned JSON length: %d', len(response)
-        )
+            all_headers = generate_slide_deck(json_str, pptx_template, progress_bar)
 
-        # Now create the PPT file
-        progress_bar.progress(
-            GlobalConfig.LLM_PROGRESS_MAX,
-            text='Finding photos online and generating the slide deck...'
-        )
-        progress_bar.progress(1.0, text='Done!')
-        st.chat_message('ai').code(response, language='json')
+            # Step 3: Bonus stuff: Web references and AI art
+            show_bonus_stuff(all_headers)
 
-        if path := generate_slide_deck(response):
-            _display_download_button(path)
+        except ValueError as ve:
+            st.error(f'Unfortunately, an error occurred: {ve}! '
+                     f'Please change the text, try again later, or report it, sharing your inputs.')
 
-        logger.info(
-            '#messages in history / 2: %d',
-            len(st.session_state[CHAT_MESSAGES]) / 2
-        )
+    else:
+        st.error('Not enough information provided! Please be little more descriptive :)')
 
 
-def generate_slide_deck(json_str: str) -> Union[pathlib.Path, None]:
+def process_slides_contents(text: str, progress_bar: st.progress) -> str:
     """
-    Create a slide deck and return the file path. In case there is any error creating the slide
-    deck, the path may be to an empty file.
+    Convert given text into structured data and display. Update the UI.
 
-    :param json_str: The content in *valid* JSON format.
-    :return: The path to the .pptx file or `None` in case of error.
+    :param text: The topic description for the presentation
+    :param progress_bar: Progress bar for this step
+    :return: The contents as a JSON-formatted string
     """
 
-    try:
-        parsed_data = json5.loads(json_str)
-    except ValueError:
-        handle_error(
-            'Encountered error while parsing JSON...will fix it and retry',
-            True
-        )
-        try:
-            parsed_data = json5.loads(text_helper.fix_malformed_json(json_str))
-        except ValueError:
-            handle_error(
-                'Encountered an error again while fixing JSON...'
-                'the slide deck cannot be created, unfortunately ☹'
-                '\nPlease try again later.',
-                True
-            )
-            return None
-    except RecursionError:
-        handle_error(
-            'Encountered a recursion error while parsing JSON...'
-            'the slide deck cannot be created, unfortunately ☹'
-            '\nPlease try again later.',
-            True
-        )
-        return None
-    except Exception:
-        handle_error(
-            'Encountered an error while parsing JSON...'
-            'the slide deck cannot be created, unfortunately ☹'
-            '\nPlease try again later.',
-            True
-        )
-        return None
-
-    if DOWNLOAD_FILE_KEY in st.session_state:
-        path = pathlib.Path(st.session_state[DOWNLOAD_FILE_KEY])
-    else:
-        temp = tempfile.NamedTemporaryFile(delete=False, suffix='.pptx')
-        path = pathlib.Path(temp.name)
-        st.session_state[DOWNLOAD_FILE_KEY] = str(path)
-
-        if temp:
-            temp.close()
+    json_str = ''
 
     try:
-        logger.debug('Creating PPTX file: %s...', st.session_state[DOWNLOAD_FILE_KEY])
-        pptx_helper.generate_powerpoint_presentation(
-            parsed_data,
-            slides_template=pptx_template,
-            output_file_path=path
-        )
+        logging.info('Calling LLM for content generation on the topic: %s', text)
+        json_str = get_contents_wrapper(text)
     except Exception as ex:
-        st.error(APP_TEXT['content_generation_error'])
-        logger.error('Caught a generic exception: %s', str(ex))
-
-    return path
-
-
-def _is_it_refinement() -> bool:
-    """
-    Whether it is the initial prompt or a refinement.
-
-    :return: True if it is the initial prompt; False otherwise.
-    """
-
-    if IS_IT_REFINEMENT in st.session_state:
-        return True
+        st.error(
+            f'An exception occurred while trying to convert to JSON. It could be because of heavy'
+            f' traffic or something else. Try doing it again or try again later.'
+            f'\nError message: {ex}'
+        )
 
-    if len(st.session_state[CHAT_MESSAGES]) >= 2:
-        # Prepare for the next call
-        st.session_state[IS_IT_REFINEMENT] = True
-        return True
+    progress_bar.progress(50, text='Contents generated')
 
-    return False
+    with st.expander('The generated contents (in JSON format)'):
+        st.code(json_str, language='json')
 
+    return json_str
 
-def _get_user_messages() -> List[str]:
-    """
-    Get a list of user messages submitted until now from the session state.
 
-    :return: The list of user messages.
+def generate_slide_deck(json_str: str, pptx_template: str, progress_bar) -> List:
     """
+    Create a slide deck.
 
-    return [
-        msg.content for msg in st.session_state[CHAT_MESSAGES] if isinstance(msg, HumanMessage)
-    ]
-
-
-def _get_last_response() -> str:
+    :param json_str: The contents in JSON format
+    :param pptx_template: The PPTX template name
+    :param progress_bar: Progress bar
+    :return: A list of all slide headers and the title
     """
-    Get the last response generated by AI.
 
-    :return: The response text.
-    """
+    progress_text = 'Creating the slide deck...give it a moment'
+    progress_bar.progress(75, text=progress_text)
 
-    return st.session_state[CHAT_MESSAGES][-1].content
+    # # Get a unique name for the file to save -- use the session ID
+    # ctx = st_sr.get_script_run_ctx()
+    # session_id = ctx.session_id
+    # timestamp = time.time()
+    # output_file_name = f'{session_id}_{timestamp}.pptx'
 
+    temp = tempfile.NamedTemporaryFile(delete=False, suffix='.pptx')
+    path = pathlib.Path(temp.name)
 
-def _display_messages_history(view_messages: st.expander):
-    """
-    Display the history of messages.
+    logging.info('Creating PPTX file...')
+    all_headers = pptx_helper.generate_powerpoint_presentation(
+        json_str,
+        as_yaml=False,
+        slides_template=pptx_template,
+        output_file_path=path
+    )
+    progress_bar.progress(100, text='Done!')
 
-    :param view_messages: The list of AI and Human messages.
-    """
+    with open(path, 'rb') as f:
+        st.download_button('Download PPTX file', f, file_name='Presentation.pptx')
 
-    with view_messages:
-        view_messages.json(st.session_state[CHAT_MESSAGES])
+    return all_headers
 
 
-def _display_download_button(file_path: pathlib.Path):
+def show_bonus_stuff(ppt_headers: List[str]):
     """
-    Display a download button to download a slide deck.
+    Show bonus stuff for the presentation.
 
-    :param file_path: The path of the .pptx file.
+    :param ppt_headers: A list of the slide headings.
     """
 
-    with open(file_path, 'rb') as download_file:
-        st.download_button(
-            'Download PPTX file ⬇️',
-            data=download_file,
-            file_name='Presentation.pptx',
-            key=datetime.datetime.now()
-        )
+    # Use the presentation title and the slide headers to find relevant info online
+    logging.info('Calling Metaphor search...')
+    ppt_text = ' '.join(ppt_headers)
+    search_results = get_web_search_results_wrapper(ppt_text)
+    md_text_items = []
+
+    for (title, link) in search_results:
+        md_text_items.append(f'[{title}]({link})')
+
+    with st.expander('Related Web references'):
+        st.markdown('\n\n'.join(md_text_items))
+
+    logging.info('Done!')
+
+    # # Avoid image generation. It costs time and an API call, so just limit to the text generation.
+    # with st.expander('AI-generated image on the presentation topic'):
+    #     logging.info('Calling SDXL for image generation...')
+    #     # img_empty.write('')
+    #     # img_text.write(APP_TEXT['image_info'])
+    #     image = get_ai_image_wrapper(ppt_text)
+    #
+    #     if len(image) > 0:
+    #         image = base64.b64decode(image)
+    #         st.image(image, caption=ppt_text)
+    #         st.info('Tip: Right-click on the image to save it.', icon="💡️")
+    #         logging.info('Image added')
 
 
 def main():

clarifai_grpc_helper.py

@@ -0,0 +1,71 @@
+from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
+from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
+from clarifai_grpc.grpc.api.status import status_code_pb2
+
+from global_config import GlobalConfig
+
+
+CHANNEL = ClarifaiChannel.get_grpc_channel()
+STUB = service_pb2_grpc.V2Stub(CHANNEL)
+
+METADATA = (
+    ('authorization', 'Key ' + GlobalConfig.CLARIFAI_PAT),
+)
+
+USER_DATA_OBJECT = resources_pb2.UserAppIDSet(
+    user_id=GlobalConfig.CLARIFAI_USER_ID,
+    app_id=GlobalConfig.CLARIFAI_APP_ID
+)
+
+RAW_TEXT = '''You are a helpful, intelligent chatbot. Create the slides for a presentation on the given topic. Include main headings for each slide, detailed bullet points for each slide. Add relevant content to each slide. Do not output any blank line.
+
+Topic:
+Talk about AI, covering what it is and how it works. Add its pros, cons, and future prospects. Also, cover its job prospects.
+'''
+
+
+def get_text_from_llm(prompt: str) -> str:
+    post_model_outputs_response = STUB.PostModelOutputs(
+        service_pb2.PostModelOutputsRequest(
+            user_app_id=USER_DATA_OBJECT,  # The userDataObject is created in the overview and is required when using a PAT
+            model_id=GlobalConfig.CLARIFAI_MODEL_ID,
+            # version_id=MODEL_VERSION_ID,  # This is optional. Defaults to the latest model version
+            inputs=[
+                resources_pb2.Input(
+                    data=resources_pb2.Data(
+                        text=resources_pb2.Text(
+                            raw=prompt
+                        )
+                    )
+                )
+            ]
+        ),
+        metadata=METADATA
+    )
+
+    if post_model_outputs_response.status.code != status_code_pb2.SUCCESS:
+        print(post_model_outputs_response.status)
+        raise Exception(f"Post model outputs failed, status: {post_model_outputs_response.status.description}")
+
+    # Since we have one input, one output will exist here
+    output = post_model_outputs_response.outputs[0]
+
+    # print("Completion:\n")
+    # print(output.data.text.raw)
+
+    return output.data.text.raw
+
+
+if __name__ == '__main__':
+    topic = ('Talk about AI, covering what it is and how it works.'
+             ' Add its pros, cons, and future prospects.'
+             ' Also, cover its job prospects.'
+             )
+    print(topic)
+
+    with open(GlobalConfig.SLIDES_TEMPLATE_FILE, 'r') as in_file:
+        prompt_txt = in_file.read()
+        prompt_txt = prompt_txt.replace('{topic}', topic)
+        response_txt = get_text_from_llm(prompt_txt)
+
+        print('Output:\n', response_txt)

examples/example_04.json

@@ -1,3 +0,0 @@
-{
-    "topic": "12 slides on a basic tutorial on Python along with examples"
-}

file_embeddings/embeddings.npy

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:64a1ba79b20c81ba7ed6604468736f74ae89813fe378191af1d8574c008b3ab5
-size 326784

file_embeddings/icons.npy

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:ce5ce4c86bb213915606921084b3516464154edcae12f4bc708d62c6bd7acebb
-size 51168

global_config.py

@@ -1,7 +1,3 @@
-"""
-A set of configurations used by the app.
-"""
-import logging
 import os
 
 from dataclasses import dataclass
@@ -13,135 +9,32 @@ load_dotenv()
 
 @dataclass(frozen=True)
 class GlobalConfig:
-    """
-    A data class holding the configurations.
-    """
-
-    PROVIDER_COHERE = 'co'
-    PROVIDER_GOOGLE_GEMINI = 'gg'
-    PROVIDER_HUGGING_FACE = 'hf'
-    PROVIDER_OLLAMA = 'ol'
-    VALID_PROVIDERS = {
-        PROVIDER_COHERE,
-        PROVIDER_GOOGLE_GEMINI,
-        PROVIDER_HUGGING_FACE,
-        PROVIDER_OLLAMA
-    }
-    VALID_MODELS = {
-        '[co]command-r-08-2024': {
-            'description': 'simpler, slower',
-            'max_new_tokens': 4096,
-            'paid': True,
-        },
-        '[gg]gemini-1.5-flash-002': {
-            'description': 'faster response',
-            'max_new_tokens': 8192,
-            'paid': True,
-        },
-        '[hf]mistralai/Mistral-7B-Instruct-v0.2': {
-            'description': 'faster, shorter',
-            'max_new_tokens': 8192,
-            'paid': False,
-        },
-        '[hf]mistralai/Mistral-Nemo-Instruct-2407': {
-            'description': 'longer response',
-            'max_new_tokens': 10240,
-            'paid': False,
-        },
-    }
-    LLM_PROVIDER_HELP = (
-        'LLM provider codes:\n\n'
-        '- **[co]**: Cohere\n'
-        '- **[gg]**: Google Gemini API\n'
-        '- **[hf]**: Hugging Face Inference API\n'
-    )
-    DEFAULT_MODEL_INDEX = 2
-    LLM_MODEL_TEMPERATURE = 0.2
-    LLM_MODEL_MIN_OUTPUT_LENGTH = 100
-    LLM_MODEL_MAX_INPUT_LENGTH = 400  # characters
+    HF_LLM_MODEL_NAME = 'mistralai/Mistral-7B-Instruct-v0.2'
+    LLM_MODEL_TEMPERATURE: float = 0.2
+    LLM_MODEL_MIN_OUTPUT_LENGTH: int = 50
+    LLM_MODEL_MAX_OUTPUT_LENGTH: int = 2000
+    LLM_MODEL_MAX_INPUT_LENGTH: int = 300
 
     HUGGINGFACEHUB_API_TOKEN = os.environ.get('HUGGINGFACEHUB_API_TOKEN', '')
+    METAPHOR_API_KEY = os.environ.get('METAPHOR_API_KEY', '')
 
     LOG_LEVEL = 'DEBUG'
-    COUNT_TOKENS = False
     APP_STRINGS_FILE = 'strings.json'
     PRELOAD_DATA_FILE = 'examples/example_02.json'
     SLIDES_TEMPLATE_FILE = 'langchain_templates/template_combined.txt'
-    INITIAL_PROMPT_TEMPLATE = 'langchain_templates/chat_prompts/initial_template_v4_two_cols_img.txt'
-    REFINEMENT_PROMPT_TEMPLATE = 'langchain_templates/chat_prompts/refinement_template_v4_two_cols_img.txt'
-
-    LLM_PROGRESS_MAX = 90
-    ICONS_DIR = 'icons/png128/'
-    TINY_BERT_MODEL = 'gaunernst/bert-mini-uncased'
-    EMBEDDINGS_FILE_NAME = 'file_embeddings/embeddings.npy'
-    ICONS_FILE_NAME = 'file_embeddings/icons.npy'
+    JSON_TEMPLATE_FILE = 'langchain_templates/text_to_json_template_02.txt'
 
     PPTX_TEMPLATE_FILES = {
-        'Basic': {
+        'Blank': {
             'file': 'pptx_templates/Blank.pptx',
-            'caption': 'A good start (Uses [photos](https://unsplash.com/photos/AFZ-qBPEceA) by [cetteup](https://unsplash.com/@cetteup?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash) on [Unsplash](https://unsplash.com/photos/a-foggy-forest-filled-with-lots-of-trees-d3ci37Gcgxg?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)) 🟧'
-        },
-        'Minimalist Sales Pitch': {
-            'file': 'pptx_templates/Minimalist_sales_pitch.pptx',
-            'caption': 'In high contrast ⬛'
+            'caption': 'A good start'
         },
         'Ion Boardroom': {
             'file': 'pptx_templates/Ion_Boardroom.pptx',
-            'caption': 'Make some bold decisions 🟥'
+            'caption': 'Make some bold decisions'
         },
         'Urban Monochrome': {
             'file': 'pptx_templates/Urban_monochrome.pptx',
-            'caption': 'Marvel in a monochrome dream ⬜'
-        },
+            'caption': 'Marvel in a monochrome dream'
+        }
     }
-
-    # This is a long text, so not incorporated as a string in `strings.json`
-    CHAT_USAGE_INSTRUCTIONS = (
-        'Briefly describe your topic of presentation in the textbox provided below. For example:\n'
-        '- Make a slide deck on AI.'
-        '\n\n'
-        'Subsequently, you can add follow-up instructions, e.g.:\n'
-        '- Can you add a slide on GPUs?'
-        '\n\n'
-        ' You can also ask it to refine any particular slide, e.g.:\n'
-        '- Make the slide with title \'Examples of AI\' a bit more descriptive.'
-        '\n\n'
-        'Finally, click on the download button at the bottom to download the slide deck.'
-        ' See this [demo video](https://youtu.be/QvAKzNKtk9k) for a brief walkthrough.\n\n'
-        'Currently, three LLMs providers and four LLMs are supported:'
-        ' **Mistral 7B Instruct v0.2** and **Mistral Nemo Instruct 2407** via Hugging Face'
-        ' Inference Endpoint; **Gemini 1.5 Flash** via Gemini API; and **Command R+** via Cohere'
-        ' API. If one is not available, choose the other from the dropdown list. A [summary of'
-        ' the supported LLMs]('
-        'https://github.com/barun-saha/slide-deck-ai/blob/main/README.md#summary-of-the-llms)'
-        ' is available for reference.\n\n'
-        ' SlideDeck AI does not have access to the Web, apart for searching for images relevant'
-        ' to the slides. Photos are added probabilistically; transparency needs to be changed'
-        ' manually, if required.\n\n'
-        '[SlideDeck AI](https://github.com/barun-saha/slide-deck-ai) is an Open-Source project,'
-        ' released under the'
-        ' [MIT license](https://github.com/barun-saha/slide-deck-ai?tab=MIT-1-ov-file#readme).'
-        '\n\n---\n\n'
-        '© Copyright 2023-2024 Barun Saha.\n\n'
-    )
-
-
-logging.basicConfig(
-    level=GlobalConfig.LOG_LEVEL,
-    format='%(asctime)s - %(levelname)s - %(name)s - %(message)s',
-    datefmt='%Y-%m-%d %H:%M:%S'
-)
-
-
-def get_max_output_tokens(llm_name: str) -> int:
-    """
-    Get the max output tokens value configured for an LLM. Return a default value if not configured.
-
-    :param llm_name: The name of the LLM.
-    :return: Max output tokens or a default count.
-    """
-
-    try:
-        return GlobalConfig.VALID_MODELS[llm_name]['max_new_tokens']
-    except KeyError:
-        return 2048

helpers/icons_embeddings.py

@@ -1,166 +0,0 @@
-"""
-Generate and save the embeddings of a pre-defined list of icons.
-Compare them with keywords embeddings to find most relevant icons.
-"""
-import os
-import pathlib
-import sys
-from typing import List, Tuple
-
-import numpy as np
-from sklearn.metrics.pairwise import cosine_similarity
-from transformers import BertTokenizer, BertModel
-
-sys.path.append('..')
-sys.path.append('../..')
-
-from global_config import GlobalConfig
-
-
-tokenizer = BertTokenizer.from_pretrained(GlobalConfig.TINY_BERT_MODEL)
-model = BertModel.from_pretrained(GlobalConfig.TINY_BERT_MODEL)
-
-
-def get_icons_list() -> List[str]:
-    """
-    Get a list of available icons.
-
-    :return: The icons file names.
-    """
-
-    items = pathlib.Path('../' + GlobalConfig.ICONS_DIR).glob('*.png')
-    items = [
-        os.path.basename(str(item)).removesuffix('.png') for item in items
-    ]
-
-    return items
-
-
-def get_embeddings(texts) -> np.ndarray:
-    """
-    Generate embeddings for a list of texts using a pre-trained language model.
-
-    :param texts: A string or a list of strings to be converted into embeddings.
-    :type texts: Union[str, List[str]]
-    :return: A NumPy array containing the embeddings for the input texts.
-    :rtype: numpy.ndarray
-
-    :raises ValueError: If the input is not a string or a list of strings, or if any element
-    in the list is not a string.
-
-    Example usage:
-    >>> keyword = 'neural network'
-    >>> file_names = ['neural_network_icon.png', 'data_analysis_icon.png', 'machine_learning.png']
-    >>> keyword_embeddings = get_embeddings(keyword)
-    >>> file_name_embeddings = get_embeddings(file_names)
-    """
-
-    inputs = tokenizer(texts, return_tensors='pt', padding=True, max_length=128, truncation=True)
-    outputs = model(**inputs)
-
-    return outputs.last_hidden_state.mean(dim=1).detach().numpy()
-
-
-def save_icons_embeddings():
-    """
-    Generate and save the embeddings for the icon file names.
-    """
-
-    file_names = get_icons_list()
-    print(f'{len(file_names)} icon files available...')
-    file_name_embeddings = get_embeddings(file_names)
-    print(f'file_name_embeddings.shape: {file_name_embeddings.shape}')
-
-    # Save embeddings to a file
-    np.save(GlobalConfig.EMBEDDINGS_FILE_NAME, file_name_embeddings)
-    np.save(GlobalConfig.ICONS_FILE_NAME, file_names)  # Save file names for reference
-
-
-def load_saved_embeddings() -> Tuple[np.ndarray, np.ndarray]:
-    """
-    Load precomputed embeddings and icons file names.
-
-    :return: The embeddings and the icon file names.
-    """
-
-    file_name_embeddings = np.load(GlobalConfig.EMBEDDINGS_FILE_NAME)
-    file_names = np.load(GlobalConfig.ICONS_FILE_NAME)
-
-    return file_name_embeddings, file_names
-
-
-def find_icons(keywords: List[str]) -> List[str]:
-    """
-    Find relevant icon file names for a list of keywords.
-
-    :param keywords: The list of one or more keywords.
-    :return: A list of the file names relevant for each keyword.
-    """
-
-    keyword_embeddings = get_embeddings(keywords)
-    file_name_embeddings, file_names = load_saved_embeddings()
-
-    # Compute similarity
-    similarities = cosine_similarity(keyword_embeddings, file_name_embeddings)
-    icon_files = file_names[np.argmax(similarities, axis=-1)]
-
-    return icon_files
-
-
-def main():
-    """
-    Example usage.
-    """
-
-    # Run this again if icons are to be added/removed
-    save_icons_embeddings()
-
-    keywords = [
-        'deep learning',
-        '',
-        'recycling',
-        'handshake',
-        'Ferry',
-        'rain drop',
-        'speech bubble',
-        'mental resilience',
-        'turmeric',
-        'Art',
-        'price tag',
-        'Oxygen',
-        'oxygen',
-        'Social Connection',
-        'Accomplishment',
-        'Python',
-        'XML',
-        'Handshake',
-    ]
-    icon_files = find_icons(keywords)
-    print(
-        f'The relevant icon files are:\n'
-        f'{list(zip(keywords, icon_files))}'
-    )
-
-    # BERT tiny:
-    # [('deep learning', 'deep-learning'), ('', '123'), ('recycling', 'refinery'),
-    #  ('handshake', 'dash-circle'), ('Ferry', 'cart'), ('rain drop', 'bucket'),
-    #  ('speech bubble', 'globe'), ('mental resilience', 'exclamation-triangle'),
-    #  ('turmeric', 'kebab'), ('Art', 'display'), ('price tag', 'bug-fill'),
-    #  ('Oxygen', 'radioactive')]
-
-    # BERT mini
-    # [('deep learning', 'deep-learning'), ('', 'compass'), ('recycling', 'tools'),
-    #  ('handshake', 'bandaid'), ('Ferry', 'cart'), ('rain drop', 'trash'),
-    #  ('speech bubble', 'image'), ('mental resilience', 'recycle'), ('turmeric', 'linkedin'),
-    #  ('Art', 'book'), ('price tag', 'card-image'), ('Oxygen', 'radioactive')]
-
-    # BERT small
-    # [('deep learning', 'deep-learning'), ('', 'gem'), ('recycling', 'tools'),
-    #  ('handshake', 'handbag'), ('Ferry', 'truck'), ('rain drop', 'bucket'),
-    #  ('speech bubble', 'strategy'), ('mental resilience', 'deep-learning'),
-    #  ('turmeric', 'flower'),
-    #  ('Art', 'book'), ('price tag', 'hotdog'), ('Oxygen', 'radioactive')]
-
-
-if __name__ == '__main__':
-    main()

helpers/image_search.py

@@ -1,148 +0,0 @@
-"""
-Search photos using Pexels API.
-"""
-import logging
-import os
-import random
-from io import BytesIO
-from typing import Union, Tuple, Literal
-from urllib.parse import urlparse, parse_qs
-
-import requests
-from dotenv import load_dotenv
-
-
-load_dotenv()
-
-
-REQUEST_TIMEOUT = 12
-MAX_PHOTOS = 3
-
-
-# Only show errors
-logging.getLogger('urllib3').setLevel(logging.ERROR)
-# Disable all child loggers of urllib3, e.g. urllib3.connectionpool
-# logging.getLogger('urllib3').propagate = True
-
-
-
-def search_pexels(
-        query: str,
-        size: Literal['small', 'medium', 'large'] = 'medium',
-        per_page: int = MAX_PHOTOS
-) -> dict:
-    """
-    Searches for images on Pexels using the provided query.
-
-    This function sends a GET request to the Pexels API with the specified search query
-    and authorization header containing the API key. It returns the JSON response from the API.
-
-    [2024-08-31] Note:
-    `curl` succeeds but API call via Python `requests` fail. Apparently, this could be due to
-    Cloudflare (or others) blocking the requests, perhaps identifying as Web-scraping. So,
-    changing the user-agent to Firefox.
-    https://stackoverflow.com/a/74674276/147021
-    https://stackoverflow.com/a/51268523/147021
-    https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent/Firefox#linux
-
-    :param query: The search query for finding images.
-    :param size: The size of the images: small, medium, or large.
-    :param per_page: No. of results to be displayed per page.
-    :return: The JSON response from the Pexels API containing search results.
-    :raises requests.exceptions.RequestException: If the request to the Pexels API fails.
-    """
-
-    url = 'https://api.pexels.com/v1/search'
-    headers = {
-        'Authorization': os.getenv('PEXEL_API_KEY'),
-        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0',
-    }
-    params = {
-        'query': query,
-        'size': size,
-        'page': 1,
-        'per_page': per_page
-    }
-    response = requests.get(url, headers=headers, params=params, timeout=REQUEST_TIMEOUT)
-    response.raise_for_status()  # Ensure the request was successful
-
-    return response.json()
-
-
-def get_photo_url_from_api_response(
-        json_response: dict
-) -> Tuple[Union[str, None], Union[str, None]]:
-    """
-    Return a randomly chosen photo from a Pexels search API response. In addition, also return
-    the original URL of the page on Pexels.
-
-    :param json_response: The JSON response.
-    :return: The selected photo URL and page URL or `None`.
-    """
-
-    page_url = None
-    photo_url = None
-
-    if 'photos' in json_response:
-        photos = json_response['photos']
-
-        if photos:
-            photo_idx = random.choice(list(range(MAX_PHOTOS)))
-            photo = photos[photo_idx]
-
-            if 'url' in photo:
-                page_url = photo['url']
-
-            if 'src' in photo:
-                if 'large' in photo['src']:
-                    photo_url = photo['src']['large']
-                elif 'original' in photo['src']:
-                    photo_url = photo['src']['original']
-
-    return photo_url, page_url
-
-
-def get_image_from_url(url: str) -> BytesIO:
-    """
-    Fetches an image from the specified URL and returns it as a BytesIO object.
-
-    This function sends a GET request to the provided URL, retrieves the image data,
-    and wraps it in a BytesIO object, which can be used like a file.
-
-    :param url: The URL of the image to be fetched.
-    :return: A BytesIO object containing the image data.
-    :raises requests.exceptions.RequestException: If the request to the URL fails.
-    """
-
-    headers = {
-        'Authorization': os.getenv('PEXEL_API_KEY'),
-        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0',
-    }
-    response = requests.get(url, headers=headers, stream=True, timeout=REQUEST_TIMEOUT)
-    response.raise_for_status()
-    image_data = BytesIO(response.content)
-
-    return image_data
-
-
-def extract_dimensions(url: str) -> Tuple[int, int]:
-    """
-    Extracts the height and width from the URL parameters.
-
-    :param url: The URL containing the image dimensions.
-    :return: A tuple containing the width and height as integers.
-    """
-    parsed_url = urlparse(url)
-    query_params = parse_qs(parsed_url.query)
-    width = int(query_params.get('w', [0])[0])
-    height = int(query_params.get('h', [0])[0])
-
-    return width, height
-
-
-if __name__ == '__main__':
-    print(
-        search_pexels(
-            query='people'
-        )
-    )

helpers/llm_helper.py

@@ -1,187 +0,0 @@
-"""
-Helper functions to access LLMs.
-"""
-import logging
-import re
-import sys
-from typing import Tuple, Union
-
-import requests
-from requests.adapters import HTTPAdapter
-from urllib3.util import Retry
-from langchain_core.language_models import BaseLLM
-
-sys.path.append('..')
-
-from global_config import GlobalConfig
-
-
-LLM_PROVIDER_MODEL_REGEX = re.compile(r'\[(.*?)\](.*)')
-OLLAMA_MODEL_REGEX = re.compile(r'[a-zA-Z0-9._:-]+$')
-# 6-64 characters long, only containing alphanumeric characters, hyphens, and underscores
-API_KEY_REGEX = re.compile(r'^[a-zA-Z0-9_-]{6,64}$')
-HF_API_HEADERS = {'Authorization': f'Bearer {GlobalConfig.HUGGINGFACEHUB_API_TOKEN}'}
-REQUEST_TIMEOUT = 35
-
-logger = logging.getLogger(__name__)
-logging.getLogger('httpx').setLevel(logging.WARNING)
-logging.getLogger('httpcore').setLevel(logging.WARNING)
-
-retries = Retry(
-    total=5,
-    backoff_factor=0.25,
-    backoff_jitter=0.3,
-    status_forcelist=[502, 503, 504],
-    allowed_methods={'POST'},
-)
-adapter = HTTPAdapter(max_retries=retries)
-http_session = requests.Session()
-http_session.mount('https://', adapter)
-http_session.mount('http://', adapter)
-
-
-def get_provider_model(provider_model: str, use_ollama: bool) -> Tuple[str, str]:
-    """
-    Parse and get LLM provider and model name from strings like `[provider]model/name-version`.
-
-    :param provider_model: The provider, model name string from `GlobalConfig`.
-    :param use_ollama: Whether Ollama is used (i.e., running in offline mode).
-    :return: The provider and the model name; empty strings in case no matching pattern found.
-    """
-
-    provider_model = provider_model.strip()
-
-    if use_ollama:
-        match = OLLAMA_MODEL_REGEX.match(provider_model)
-        if match:
-            return GlobalConfig.PROVIDER_OLLAMA, match.group(0)
-    else:
-        match = LLM_PROVIDER_MODEL_REGEX.match(provider_model)
-
-        if match:
-            inside_brackets = match.group(1)
-            outside_brackets = match.group(2)
-            return inside_brackets, outside_brackets
-
-    return '', ''
-
-
-def is_valid_llm_provider_model(provider: str, model: str, api_key: str) -> bool:
-    """
-    Verify whether LLM settings are proper.
-    This function does not verify whether `api_key` is correct. It only confirms that the key has
-    at least five characters. Key verification is done when the LLM is created.
-
-    :param provider: Name of the LLM provider.
-    :param model: Name of the model.
-    :param api_key: The API key or access token.
-    :return: `True` if the settings "look" OK; `False` otherwise.
-    """
-
-    if not provider or not model or provider not in GlobalConfig.VALID_PROVIDERS:
-        return False
-
-    if provider in [
-        GlobalConfig.PROVIDER_GOOGLE_GEMINI,
-        GlobalConfig.PROVIDER_COHERE,
-    ] and not api_key:
-        return False
-
-    if api_key:
-        return API_KEY_REGEX.match(api_key) is not None
-
-    return True
-
-
-def get_langchain_llm(
-        provider: str,
-        model: str,
-        max_new_tokens: int,
-        api_key: str = ''
-) -> Union[BaseLLM, None]:
-    """
-    Get an LLM based on the provider and model specified.
-
-    :param provider: The LLM provider. Valid values are `hf` for Hugging Face.
-    :param model: The name of the LLM.
-    :param max_new_tokens: The maximum number of tokens to generate.
-    :param api_key: API key or access token to use.
-    :return: An instance of the LLM or `None` in case of any error.
-    """
-
-    if provider == GlobalConfig.PROVIDER_HUGGING_FACE:
-        from langchain_community.llms.huggingface_endpoint import HuggingFaceEndpoint
-
-        logger.debug('Getting LLM via HF endpoint: %s', model)
-        return HuggingFaceEndpoint(
-            repo_id=model,
-            max_new_tokens=max_new_tokens,
-            top_k=40,
-            top_p=0.95,
-            temperature=GlobalConfig.LLM_MODEL_TEMPERATURE,
-            repetition_penalty=1.03,
-            streaming=True,
-            huggingfacehub_api_token=api_key or GlobalConfig.HUGGINGFACEHUB_API_TOKEN,
-            return_full_text=False,
-            stop_sequences=['</s>'],
-        )
-
-    if provider == GlobalConfig.PROVIDER_GOOGLE_GEMINI:
-        from google.generativeai.types.safety_types import HarmBlockThreshold, HarmCategory
-        from langchain_google_genai import GoogleGenerativeAI
-
-        logger.debug('Getting LLM via Google Gemini: %s', model)
-        return GoogleGenerativeAI(
-            model=model,
-            temperature=GlobalConfig.LLM_MODEL_TEMPERATURE,
-            max_tokens=max_new_tokens,
-            timeout=None,
-            max_retries=2,
-            google_api_key=api_key,
-            safety_settings={
-                HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT:
-                    HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
-                HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
-                HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
-                HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT:
-                    HarmBlockThreshold.BLOCK_LOW_AND_ABOVE
-            }
-        )
-
-    if provider == GlobalConfig.PROVIDER_COHERE:
-        from langchain_cohere.llms import Cohere
-
-        logger.debug('Getting LLM via Cohere: %s', model)
-        return Cohere(
-            temperature=GlobalConfig.LLM_MODEL_TEMPERATURE,
-            max_tokens=max_new_tokens,
-            timeout_seconds=None,
-            max_retries=2,
-            cohere_api_key=api_key,
-            streaming=True,
-        )
-
-    if provider == GlobalConfig.PROVIDER_OLLAMA:
-        from langchain_ollama.llms import OllamaLLM
-
-        logger.debug('Getting LLM via Ollama: %s', model)
-        return OllamaLLM(
-            model=model,
-            temperature=GlobalConfig.LLM_MODEL_TEMPERATURE,
-            num_predict=max_new_tokens,
-            format='json',
-            streaming=True,
-        )
-
-    return None
-
-
-if __name__ == '__main__':
-    inputs = [
-        '[co]Cohere',
-        '[hf]mistralai/Mistral-7B-Instruct-v0.2',
-        '[gg]gemini-1.5-flash-002'
-    ]
-
-    for text in inputs:
-        print(get_provider_model(text, use_ollama=False))

helpers/pptx_helper.py

@@ -1,987 +0,0 @@
-"""
-A set of functions to create a PowerPoint slide deck.
-"""
-import logging
-import os
-import pathlib
-import random
-import re
-import sys
-import tempfile
-from typing import List, Tuple, Optional
-
-import json5
-import pptx
-from dotenv import load_dotenv
-from pptx.enum.shapes import MSO_AUTO_SHAPE_TYPE
-from pptx.shapes.placeholder import PicturePlaceholder, SlidePlaceholder
-
-sys.path.append('..')
-sys.path.append('../..')
-
-import helpers.icons_embeddings as ice
-import helpers.image_search as ims
-from global_config import GlobalConfig
-
-
-load_dotenv()
-
-
-# English Metric Unit (used by PowerPoint) to inches
-EMU_TO_INCH_SCALING_FACTOR = 1.0 / 914400
-INCHES_3 = pptx.util.Inches(3)
-INCHES_2 = pptx.util.Inches(2)
-INCHES_1_5 = pptx.util.Inches(1.5)
-INCHES_1 = pptx.util.Inches(1)
-INCHES_0_8 = pptx.util.Inches(0.8)
-INCHES_0_9 = pptx.util.Inches(0.9)
-INCHES_0_5 = pptx.util.Inches(0.5)
-INCHES_0_4 = pptx.util.Inches(0.4)
-INCHES_0_3 = pptx.util.Inches(0.3)
-INCHES_0_2 = pptx.util.Inches(0.2)
-
-STEP_BY_STEP_PROCESS_MARKER = '>> '
-ICON_BEGINNING_MARKER = '[['
-ICON_END_MARKER = ']]'
-
-ICON_SIZE = INCHES_0_8
-ICON_BG_SIZE = INCHES_1
-
-IMAGE_DISPLAY_PROBABILITY = 1 / 3.0
-FOREGROUND_IMAGE_PROBABILITY = 0.8
-
-SLIDE_NUMBER_REGEX = re.compile(r"^slide[ ]+\d+:", re.IGNORECASE)
-ICONS_REGEX = re.compile(r"\[\[(.*?)\]\]\s*(.*)")
-
-ICON_COLORS = [
-    pptx.dml.color.RGBColor.from_string('800000'),  # Maroon
-    pptx.dml.color.RGBColor.from_string('6A5ACD'),  # SlateBlue
-    pptx.dml.color.RGBColor.from_string('556B2F'),  # DarkOliveGreen
-    pptx.dml.color.RGBColor.from_string('2F4F4F'),  # DarkSlateGray
-    pptx.dml.color.RGBColor.from_string('4682B4'),  # SteelBlue
-    pptx.dml.color.RGBColor.from_string('5F9EA0'),  # CadetBlue
-]
-
-
-logger = logging.getLogger(__name__)
-logging.getLogger('PIL.PngImagePlugin').setLevel(logging.ERROR)
-
-
-def remove_slide_number_from_heading(header: str) -> str:
-    """
-    Remove the slide number from a given slide header.
-
-    :param header: The header of a slide.
-    :return: The header without slide number.
-    """
-
-    if SLIDE_NUMBER_REGEX.match(header):
-        idx = header.find(':')
-        header = header[idx + 1:]
-
-    return header
-
-
-def generate_powerpoint_presentation(
-        parsed_data: dict,
-        slides_template: str,
-        output_file_path: pathlib.Path
-) -> List:
-    """
-    Create and save a PowerPoint presentation file containing the content in JSON format.
-
-    :param parsed_data: The presentation content as parsed JSON data.
-    :param slides_template: The PPTX template to use.
-    :param output_file_path: The path of the PPTX file to save as.
-    :return: A list of presentation title and slides headers.
-    """
-
-    presentation = pptx.Presentation(GlobalConfig.PPTX_TEMPLATE_FILES[slides_template]['file'])
-    slide_width_inch, slide_height_inch = _get_slide_width_height_inches(presentation)
-
-    # The title slide
-    title_slide_layout = presentation.slide_layouts[0]
-    slide = presentation.slides.add_slide(title_slide_layout)
-    title = slide.shapes.title
-    subtitle = slide.placeholders[1]
-    title.text = parsed_data['title']
-    logger.info(
-        'PPT title: %s | #slides: %d | template: %s',
-        title.text, len(parsed_data['slides']),
-        GlobalConfig.PPTX_TEMPLATE_FILES[slides_template]['file']
-    )
-    subtitle.text = 'by Myself and SlideDeck AI :)'
-    all_headers = [title.text, ]
-
-    # Add content in a loop
-    for a_slide in parsed_data['slides']:
-        try:
-            is_processing_done = _handle_icons_ideas(
-                presentation=presentation,
-                slide_json=a_slide,
-                slide_width_inch=slide_width_inch,
-                slide_height_inch=slide_height_inch
-            )
-
-            if not is_processing_done:
-                is_processing_done = _handle_double_col_layout(
-                    presentation=presentation,
-                    slide_json=a_slide,
-                    slide_width_inch=slide_width_inch,
-                    slide_height_inch=slide_height_inch
-                )
-
-            if not is_processing_done:
-                is_processing_done = _handle_step_by_step_process(
-                    presentation=presentation,
-                    slide_json=a_slide,
-                    slide_width_inch=slide_width_inch,
-                    slide_height_inch=slide_height_inch
-                )
-
-            if not is_processing_done:
-                _handle_default_display(
-                    presentation=presentation,
-                    slide_json=a_slide,
-                    slide_width_inch=slide_width_inch,
-                    slide_height_inch=slide_height_inch
-            )
-
-        except Exception:
-            # In case of any unforeseen error, try to salvage what is available
-            continue
-
-    # The thank-you slide
-    last_slide_layout = presentation.slide_layouts[0]
-    slide = presentation.slides.add_slide(last_slide_layout)
-    title = slide.shapes.title
-    title.text = 'Thank you!'
-
-    presentation.save(output_file_path)
-
-    return all_headers
-
-
-def get_flat_list_of_contents(items: list, level: int) -> List[Tuple]:
-    """
-    Flatten a (hierarchical) list of bullet points to a single list containing each item and
-    its level.
-
-    :param items: A bullet point (string or list).
-    :param level: The current level of hierarchy.
-    :return: A list of (bullet item text, hierarchical level) tuples.
-    """
-
-    flat_list = []
-
-    for item in items:
-        if isinstance(item, str):
-            flat_list.append((item, level))
-        elif isinstance(item, list):
-            flat_list = flat_list + get_flat_list_of_contents(item, level + 1)
-
-    return flat_list
-
-
-def get_slide_placeholders(
-        slide: pptx.slide.Slide,
-        layout_number: int,
-        is_debug: bool = False
-) -> List[Tuple[int, str]]:
-    """
-    Return the index and name (lower case) of all placeholders present in a slide, except
-    the title placeholder.
-
-    A placeholder in a slide is a place to add content. Each placeholder has a name and an index.
-    This index is NOT a list index, rather a set of keys used to look up a dict. So, `idx` is
-    non-contiguous. Also, the title placeholder of a slide always has index 0. User-added
-    placeholder get indices assigned starting from 10.
-
-    With user-edited or added placeholders, their index may be difficult to track. This function
-    returns the placeholders name as well, which could be useful to distinguish between the
-    different placeholder.
-
-    :param slide: The slide.
-    :param layout_number: The layout number used by the slide.
-    :param is_debug: Whether to print debugging statements.
-    :return: A list containing placeholders (idx, name) tuples, except the title placeholder.
-    """
-
-    if is_debug:
-        print(
-            f'Slide layout #{layout_number}:'
-            f' # of placeholders: {len(slide.shapes.placeholders)} (including the title)'
-        )
-
-    placeholders = [
-        (shape.placeholder_format.idx, shape.name.lower()) for shape in slide.shapes.placeholders
-    ]
-    placeholders.pop(0)  # Remove the title placeholder
-
-    if is_debug:
-        print(placeholders)
-
-    return placeholders
-
-
-def _handle_default_display(
-        presentation: pptx.Presentation,
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-):
-    """
-    Display a list of text in a slide.
-
-    :param presentation: The presentation object.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    """
-
-    status = False
-
-    if 'img_keywords' in slide_json:
-        if random.random() < IMAGE_DISPLAY_PROBABILITY:
-            if random.random() < FOREGROUND_IMAGE_PROBABILITY:
-                status = _handle_display_image__in_foreground(
-                    presentation,
-                    slide_json,
-                    slide_width_inch,
-                    slide_height_inch
-                )
-            else:
-                status = _handle_display_image__in_background(
-                    presentation,
-                    slide_json,
-                    slide_width_inch,
-                    slide_height_inch
-                )
-
-    if status:
-        return
-
-    # Image display failed, so display only text
-    bullet_slide_layout = presentation.slide_layouts[1]
-    slide = presentation.slides.add_slide(bullet_slide_layout)
-
-    shapes = slide.shapes
-    title_shape = shapes.title
-
-    try:
-        body_shape = shapes.placeholders[1]
-    except KeyError:
-        placeholders = get_slide_placeholders(slide, layout_number=1)
-        body_shape = shapes.placeholders[placeholders[0][0]]
-
-    title_shape.text = remove_slide_number_from_heading(slide_json['heading'])
-    text_frame = body_shape.text_frame
-
-    # The bullet_points may contain a nested hierarchy of JSON arrays
-    # In some scenarios, it may contain objects (dictionaries) because the LLM generated so
-    #  ^ The second scenario is not covered
-
-    flat_items_list = get_flat_list_of_contents(slide_json['bullet_points'], level=0)
-
-    for idx, an_item in enumerate(flat_items_list):
-        if idx == 0:
-            text_frame.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-        else:
-            paragraph = text_frame.add_paragraph()
-            paragraph.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-            paragraph.level = an_item[1]
-
-    _handle_key_message(
-        the_slide=slide,
-        slide_json=slide_json,
-        slide_height_inch=slide_height_inch,
-        slide_width_inch=slide_width_inch
-    )
-
-
-def _handle_display_image__in_foreground(
-        presentation: pptx.Presentation(),
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-) -> bool:
-    """
-    Create a slide with text and image using a picture placeholder layout. If not image keyword is
-    available, it will add only text to the slide.
-
-    :param presentation: The presentation object.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    :return: True if the side has been processed.
-    """
-
-    img_keywords = slide_json['img_keywords'].strip()
-    slide = presentation.slide_layouts[8]  # Picture with Caption
-    slide = presentation.slides.add_slide(slide)
-    placeholders = None
-
-    title_placeholder = slide.shapes.title
-    title_placeholder.text = remove_slide_number_from_heading(slide_json['heading'])
-
-    try:
-        pic_col: PicturePlaceholder = slide.shapes.placeholders[1]
-    except KeyError:
-        placeholders = get_slide_placeholders(slide, layout_number=8)
-        pic_col = None
-        for idx, name in placeholders:
-            if 'picture' in name:
-                pic_col: PicturePlaceholder = slide.shapes.placeholders[idx]
-
-    try:
-        text_col: SlidePlaceholder = slide.shapes.placeholders[2]
-    except KeyError:
-        text_col = None
-        if not placeholders:
-            placeholders = get_slide_placeholders(slide, layout_number=8)
-
-        for idx, name in placeholders:
-            if 'content' in name:
-                text_col: SlidePlaceholder = slide.shapes.placeholders[idx]
-
-    flat_items_list = get_flat_list_of_contents(slide_json['bullet_points'], level=0)
-
-    for idx, an_item in enumerate(flat_items_list):
-        if idx == 0:
-            text_col.text_frame.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-        else:
-            paragraph = text_col.text_frame.add_paragraph()
-            paragraph.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-            paragraph.level = an_item[1]
-
-    if not img_keywords:
-        # No keywords, so no image search and addition
-        return True
-
-    try:
-        photo_url, page_url = ims.get_photo_url_from_api_response(
-            ims.search_pexels(query=img_keywords, size='medium')
-        )
-
-        if photo_url:
-            pic_col.insert_picture(
-                ims.get_image_from_url(photo_url)
-            )
-
-            _add_text_at_bottom(
-                slide=slide,
-                slide_width_inch=slide_width_inch,
-                slide_height_inch=slide_height_inch,
-                text='Photo provided by Pexels',
-                hyperlink=page_url
-            )
-    except Exception as ex:
-        logger.error(
-            '*** Error occurred while running adding image to slide: %s',
-            str(ex)
-        )
-
-    return True
-
-
-def _handle_display_image__in_background(
-        presentation: pptx.Presentation(),
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-) -> bool:
-    """
-    Add a slide with text and an image in the background. It works just like
-    `_handle_default_display()` but with a background image added. If not image keyword is
-    available, it will add only text to the slide.
-
-    :param presentation: The presentation object.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    :return: True if the slide has been processed.
-    """
-
-    img_keywords = slide_json['img_keywords'].strip()
-
-    # Add a photo in the background, text in the foreground
-    slide = presentation.slides.add_slide(presentation.slide_layouts[1])
-    title_shape = slide.shapes.title
-
-    try:
-        body_shape = slide.shapes.placeholders[1]
-    except KeyError:
-        placeholders = get_slide_placeholders(slide, layout_number=1)
-        # Layout 1 usually has two placeholders, including the title
-        body_shape = slide.shapes.placeholders[placeholders[0][0]]
-
-    title_shape.text = remove_slide_number_from_heading(slide_json['heading'])
-
-    flat_items_list = get_flat_list_of_contents(slide_json['bullet_points'], level=0)
-
-    for idx, an_item in enumerate(flat_items_list):
-        if idx == 0:
-            body_shape.text_frame.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-        else:
-            paragraph = body_shape.text_frame.add_paragraph()
-            paragraph.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-            paragraph.level = an_item[1]
-
-    if not img_keywords:
-        # No keywords, so no image search and addition
-        return True
-
-    try:
-        photo_url, page_url = ims.get_photo_url_from_api_response(
-            ims.search_pexels(query=img_keywords, size='large')
-        )
-
-        if photo_url:
-            picture = slide.shapes.add_picture(
-                image_file=ims.get_image_from_url(photo_url),
-                left=0,
-                top=0,
-                width=pptx.util.Inches(slide_width_inch),
-            )
-
-            _add_text_at_bottom(
-                slide=slide,
-                slide_width_inch=slide_width_inch,
-                slide_height_inch=slide_height_inch,
-                text='Photo provided by Pexels',
-                hyperlink=page_url
-            )
-
-            # Move picture to background
-            # https://github.com/scanny/python-pptx/issues/49#issuecomment-137172836
-            slide.shapes._spTree.remove(picture._element)
-            slide.shapes._spTree.insert(2, picture._element)
-    except Exception as ex:
-        logger.error(
-            '*** Error occurred while running adding image to the slide background: %s',
-            str(ex)
-        )
-
-    return True
-
-
-def _handle_icons_ideas(
-        presentation: pptx.Presentation(),
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-):
-    """
-    Add a slide with some icons and text.
-    If no suitable icons are found, the step numbers are shown.
-
-    :param presentation: The presentation object.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    :return: True if the slide has been processed.
-    """
-
-    if 'bullet_points' in slide_json and slide_json['bullet_points']:
-        items = slide_json['bullet_points']
-
-        # Ensure that it is a single list of strings without any sub-list
-        for step in items:
-            if not isinstance(step, str) or not step.startswith(ICON_BEGINNING_MARKER):
-                return False
-
-        slide_layout = presentation.slide_layouts[5]
-        slide = presentation.slides.add_slide(slide_layout)
-        slide.shapes.title.text = remove_slide_number_from_heading(slide_json['heading'])
-
-        n_items = len(items)
-        text_box_size = INCHES_2
-
-        # Calculate the total width of all pictures and the spacing
-        total_width = n_items * ICON_SIZE
-        spacing = (pptx.util.Inches(slide_width_inch) - total_width) / (n_items + 1)
-        top = INCHES_3
-
-        icons_texts = [
-            (match.group(1), match.group(2)) for match in [
-                ICONS_REGEX.search(item) for item in items
-            ]
-        ]
-        fallback_icon_files = ice.find_icons([item[0] for item in icons_texts])
-
-        for idx, item in enumerate(icons_texts):
-            icon, accompanying_text = item
-            icon_path = f'{GlobalConfig.ICONS_DIR}/{icon}.png'
-
-            if not os.path.exists(icon_path):
-                logger.warning(
-                    'Icon not found: %s...using fallback icon: %s',
-                    icon, fallback_icon_files[idx]
-                )
-                icon_path = f'{GlobalConfig.ICONS_DIR}/{fallback_icon_files[idx]}.png'
-
-            left = spacing + idx * (ICON_SIZE + spacing)
-            # Calculate the center position for alignment
-            center = left + ICON_SIZE / 2
-
-            # Add a rectangle shape with a fill color (background)
-            # The size of the shape is slightly bigger than the icon, so align the icon position
-            shape = slide.shapes.add_shape(
-                MSO_AUTO_SHAPE_TYPE.ROUNDED_RECTANGLE,
-                center - INCHES_0_5,
-                top - (ICON_BG_SIZE - ICON_SIZE) / 2,
-                INCHES_1, INCHES_1
-            )
-            shape.fill.solid()
-            shape.shadow.inherit = False
-
-            # Set the icon's background shape color
-            shape.fill.fore_color.rgb = shape.line.color.rgb = random.choice(ICON_COLORS)
-
-            # Add the icon image on top of the colored shape
-            slide.shapes.add_picture(icon_path, left, top, height=ICON_SIZE)
-
-            # Add a text box below the shape
-            text_box = slide.shapes.add_shape(
-                MSO_AUTO_SHAPE_TYPE.ROUNDED_RECTANGLE,
-                left=center - text_box_size / 2,  # Center the text box horizontally
-                top=top + ICON_SIZE + INCHES_0_2,
-                width=text_box_size,
-                height=text_box_size
-            )
-            text_frame = text_box.text_frame
-            text_frame.text = accompanying_text
-            text_frame.word_wrap = True
-            text_frame.paragraphs[0].alignment = pptx.enum.text.PP_ALIGN.CENTER
-
-            # Center the text vertically
-            text_frame.vertical_anchor = pptx.enum.text.MSO_ANCHOR.MIDDLE
-            text_box.fill.background()  # No fill
-            text_box.line.fill.background()  # No line
-            text_box.shadow.inherit = False
-
-            # Set the font color based on the theme
-            for paragraph in text_frame.paragraphs:
-                for run in paragraph.runs:
-                    run.font.color.theme_color = pptx.enum.dml.MSO_THEME_COLOR.TEXT_2
-
-            _add_text_at_bottom(
-                slide=slide,
-                slide_width_inch=slide_width_inch,
-                slide_height_inch=slide_height_inch,
-                text='More icons available in the SlideDeck AI repository',
-                hyperlink='https://github.com/barun-saha/slide-deck-ai/tree/main/icons/png128'
-            )
-
-        return True
-
-    return False
-
-
-def _add_text_at_bottom(
-        slide: pptx.slide.Slide,
-        slide_width_inch: float,
-        slide_height_inch: float,
-        text: str,
-        hyperlink: Optional[str] = None,
-        target_height: Optional[float] = 0.5
-):
-    """
-    Add arbitrary text to a textbox positioned near the lower left side of a slide.
-
-    :param slide: The slide.
-    :param slide_width_inch: The width of the slide.
-    :param slide_height_inch: The height of the slide.
-    :param target_height: the target height of the box in inches (optional).
-    :param text: The text to be added
-    :param hyperlink: The hyperlink to be added to the text (optional).
-    """
-
-    footer = slide.shapes.add_textbox(
-        left=INCHES_1,
-        top=pptx.util.Inches(slide_height_inch - target_height),
-        width=pptx.util.Inches(slide_width_inch),
-        height=pptx.util.Inches(target_height)
-    )
-
-    paragraph = footer.text_frame.paragraphs[0]
-    run = paragraph.add_run()
-    run.text = text
-    run.font.size = pptx.util.Pt(10)
-    run.font.underline = False
-
-    if hyperlink:
-        run.hyperlink.address = hyperlink
-
-
-def _handle_double_col_layout(
-        presentation: pptx.Presentation(),
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-) -> bool:
-    """
-    Add a slide with a double column layout for comparison.
-
-    :param presentation: The presentation object.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    :return: True if double col layout has been added; False otherwise.
-    """
-
-    if 'bullet_points' in slide_json and slide_json['bullet_points']:
-        double_col_content = slide_json['bullet_points']
-
-        if double_col_content and (
-                len(double_col_content) == 2
-        ) and isinstance(double_col_content[0], dict) and isinstance(double_col_content[1], dict):
-            slide = presentation.slide_layouts[4]
-            slide = presentation.slides.add_slide(slide)
-            placeholders = None
-
-            shapes = slide.shapes
-            title_placeholder = shapes.title
-            title_placeholder.text = remove_slide_number_from_heading(slide_json['heading'])
-
-            try:
-                left_heading, right_heading = shapes.placeholders[1], shapes.placeholders[3]
-            except KeyError:
-                # For manually edited/added master slides, the placeholder idx numbers in the dict
-                # will be different (>= 10)
-                left_heading, right_heading = None, None
-                placeholders = get_slide_placeholders(slide, layout_number=4)
-
-                for idx, name in placeholders:
-                    if 'text placeholder' in name:
-                        if not left_heading:
-                            left_heading = shapes.placeholders[idx]
-                        elif not right_heading:
-                            right_heading = shapes.placeholders[idx]
-
-            try:
-                left_col, right_col = shapes.placeholders[2], shapes.placeholders[4]
-            except KeyError:
-                left_col, right_col = None, None
-                if not placeholders:
-                    placeholders = get_slide_placeholders(slide, layout_number=4)
-
-                for idx, name in placeholders:
-                    if 'content placeholder' in name:
-                        if not left_col:
-                            left_col = shapes.placeholders[idx]
-                        elif not right_col:
-                            right_col = shapes.placeholders[idx]
-
-            left_col_frame, right_col_frame = left_col.text_frame, right_col.text_frame
-
-            if 'heading' in double_col_content[0] and left_heading:
-                left_heading.text = double_col_content[0]['heading']
-            if 'bullet_points' in double_col_content[0]:
-                flat_items_list = get_flat_list_of_contents(
-                    double_col_content[0]['bullet_points'], level=0
-                )
-
-                if not left_heading:
-                    left_col_frame.text = double_col_content[0]['heading']
-
-                for idx, an_item in enumerate(flat_items_list):
-                    if left_heading and idx == 0:
-                        left_col_frame.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-                    else:
-                        paragraph = left_col_frame.add_paragraph()
-                        paragraph.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-                        paragraph.level = an_item[1]
-
-            if 'heading' in double_col_content[1] and right_heading:
-                right_heading.text = double_col_content[1]['heading']
-            if 'bullet_points' in double_col_content[1]:
-                flat_items_list = get_flat_list_of_contents(
-                    double_col_content[1]['bullet_points'], level=0
-                )
-
-                if not right_heading:
-                    right_col_frame.text = double_col_content[1]['heading']
-
-                for idx, an_item in enumerate(flat_items_list):
-                    if right_col_frame and idx == 0:
-                        right_col_frame.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-                    else:
-                        paragraph = right_col_frame.add_paragraph()
-                        paragraph.text = an_item[0].removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-                        paragraph.level = an_item[1]
-
-            _handle_key_message(
-                the_slide=slide,
-                slide_json=slide_json,
-                slide_height_inch=slide_height_inch,
-                slide_width_inch=slide_width_inch
-            )
-
-            return True
-
-    return False
-
-
-def _handle_step_by_step_process(
-        presentation: pptx.Presentation,
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-) -> bool:
-    """
-    Add shapes to display a step-by-step process in the slide, if available.
-
-    :param presentation: The presentation object.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    :return True if this slide has a step-by-step process depiction added; False otherwise.
-    """
-
-    if 'bullet_points' in slide_json and slide_json['bullet_points']:
-        steps = slide_json['bullet_points']
-
-        no_marker_count = 0.0
-        n_steps = len(steps)
-
-        # Ensure that it is a single list of strings without any sub-list
-        for step in steps:
-            if not isinstance(step, str):
-                return False
-
-            # In some cases, one or two steps may not begin with >>, e.g.:
-            # {
-            #     "heading": "Step-by-Step Process: Creating a Legacy",
-            #     "bullet_points": [
-            #         "Identify your unique talents and passions",
-            #         ">> Develop your skills and knowledge",
-            #         ">> Create meaningful work",
-            #         ">> Share your work with the world",
-            #         ">> Continuously learn and adapt"
-            #     ],
-            #     "key_message": ""
-            # },
-            #
-            # Use a threshold, e.g., at most 20%
-            if not step.startswith(STEP_BY_STEP_PROCESS_MARKER):
-                no_marker_count += 1
-
-        slide_header = slide_json['heading'].lower()
-        if (no_marker_count / n_steps > 0.25) and not (
-                ('step-by-step' in slide_header) or ('step by step' in slide_header)
-        ):
-            return False
-
-        if n_steps < 3 or n_steps > 6:
-            # Two steps -- probably not a process
-            # More than 5--6 steps -- would likely cause a visual clutter
-            return False
-
-        bullet_slide_layout = presentation.slide_layouts[1]
-        slide = presentation.slides.add_slide(bullet_slide_layout)
-        shapes = slide.shapes
-        shapes.title.text = remove_slide_number_from_heading(slide_json['heading'])
-
-        if 3 <= n_steps <= 4:
-            # Horizontal display
-            height = INCHES_1_5
-            width = pptx.util.Inches(slide_width_inch / n_steps - 0.01)
-            top = pptx.util.Inches(slide_height_inch / 2)
-            left = pptx.util.Inches((slide_width_inch - width.inches * n_steps) / 2 + 0.05)
-
-            for step in steps:
-                shape = shapes.add_shape(MSO_AUTO_SHAPE_TYPE.CHEVRON, left, top, width, height)
-                shape.text = step.removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-                left += width - INCHES_0_4
-        elif 4 < n_steps <= 6:
-            # Vertical display
-            height = pptx.util.Inches(0.65)
-            top = pptx.util.Inches(slide_height_inch / 4)
-            left = INCHES_1  # slide_width_inch - width.inches)
-
-            # Find the close to median width, based on the length of each text, to be set
-            # for the shapes
-            width = pptx.util.Inches(slide_width_inch * 2 / 3)
-            lengths = [len(step) for step in steps]
-            font_size_20pt = pptx.util.Pt(20)
-            widths = sorted(
-                [
-                    min(
-                        pptx.util.Inches(font_size_20pt.inches * a_len),
-                        width
-                    ) for a_len in lengths
-                ]
-            )
-            width = widths[len(widths) // 2]
-
-            for step in steps:
-                shape = shapes.add_shape(MSO_AUTO_SHAPE_TYPE.PENTAGON, left, top, width, height)
-                shape.text = step.removeprefix(STEP_BY_STEP_PROCESS_MARKER)
-                top += height + INCHES_0_3
-                left += INCHES_0_5
-
-    return True
-
-
-def _handle_key_message(
-        the_slide: pptx.slide.Slide,
-        slide_json: dict,
-        slide_width_inch: float,
-        slide_height_inch: float
-):
-    """
-    Add a shape to display the key message in the slide, if available.
-
-    :param the_slide: The slide to be processed.
-    :param slide_json: The content of the slide as JSON data.
-    :param slide_width_inch: The width of the slide in inches.
-    :param slide_height_inch: The height of the slide in inches.
-    """
-
-    if 'key_message' in slide_json and slide_json['key_message']:
-        height = pptx.util.Inches(1.6)
-        width = pptx.util.Inches(slide_width_inch / 2.3)
-        top = pptx.util.Inches(slide_height_inch - height.inches - 0.1)
-        left = pptx.util.Inches((slide_width_inch - width.inches) / 2)
-        shape = the_slide.shapes.add_shape(
-            MSO_AUTO_SHAPE_TYPE.ROUNDED_RECTANGLE,
-            left=left,
-            top=top,
-            width=width,
-            height=height
-        )
-        shape.text = slide_json['key_message']
-
-
-def _get_slide_width_height_inches(presentation: pptx.Presentation) -> Tuple[float, float]:
-    """
-    Get the dimensions of a slide in inches.
-
-    :param presentation: The presentation object.
-    :return: The width and the height.
-    """
-
-    slide_width_inch = EMU_TO_INCH_SCALING_FACTOR * presentation.slide_width
-    slide_height_inch = EMU_TO_INCH_SCALING_FACTOR * presentation.slide_height
-    # logger.debug('Slide width: %f, height: %f', slide_width_inch, slide_height_inch)
-
-    return slide_width_inch, slide_height_inch
-
-
-if __name__ == '__main__':
-    _JSON_DATA = '''
-    {
-  "title": "AI Applications: Transforming Industries",
-  "slides": [
-    {
-      "heading": "Introduction to AI Applications",
-      "bullet_points": [
-        "Artificial Intelligence (AI) is transforming various industries",
-        "AI applications range from simple decision-making tools to complex systems",
-        "AI can be categorized into types: Rule-based, Instance-based, and Model-based"
-      ],
-      "key_message": "AI is a broad field with diverse applications and categories",
-      "img_keywords": "AI, transformation, industries, decision-making, categories"
-    },
-    {
-      "heading": "AI in Everyday Life",
-      "bullet_points": [
-        "Virtual assistants like Siri, Alexa, and Google Assistant",
-        "Recommender systems in Netflix, Amazon, and Spotify",
-        "Fraud detection in banking and credit card transactions"
-      ],
-      "key_message": "AI is integrated into our daily lives through various services",
-      "img_keywords": "virtual assistants, recommender systems, fraud detection"
-    },
-    {
-      "heading": "AI in Healthcare",
-      "bullet_points": [
-        "Disease diagnosis and prediction using machine learning algorithms",
-        "Personalized medicine and drug discovery",
-        "AI-powered robotic surgeries and remote patient monitoring"
-      ],
-      "key_message": "AI is revolutionizing healthcare with improved diagnostics and patient care",
-      "img_keywords": "healthcare, disease diagnosis, personalized medicine, robotic surgeries"
-    },
-    {
-      "heading": "AI in Key Industries",
-      "bullet_points": [
-        {
-          "heading": "Retail",
-          "bullet_points": [
-            "Inventory management and demand forecasting",
-            "Customer segmentation and targeted marketing",
-            "AI-driven chatbots for customer service"
-          ]
-        },
-        {
-          "heading": "Finance",
-          "bullet_points": [
-            "Credit scoring and risk assessment",
-            "Algorithmic trading and portfolio management",
-            "AI for detecting money laundering and cyber fraud"
-          ]
-        }
-      ],
-      "key_message": "AI is transforming retail and finance with improved operations and decision-making",
-      "img_keywords": "retail, finance, inventory management, credit scoring, algorithmic trading"
-    },
-    {
-      "heading": "AI in Education",
-      "bullet_points": [
-        "Personalized learning paths and adaptive testing",
-        "Intelligent tutoring systems for skill development",
-        "AI for predicting student performance and dropout rates"
-      ],
-      "key_message": "AI is personalizing education and improving student outcomes",
-    },
-    {
-      "heading": "Step-by-Step: AI Development Process",
-      "bullet_points": [
-        ">> Define the problem and objectives",
-        ">> Collect and preprocess data",
-        ">> Select and train the AI model",
-        ">> Evaluate and optimize the model",
-        ">> Deploy and monitor the AI system"
-      ],
-      "key_message": "Developing AI involves a structured process from problem definition to deployment",
-      "img_keywords": ""
-    },
-    {
-      "heading": "AI Icons: Key Aspects",
-      "bullet_points": [
-        "[[brain]] Human-like intelligence and decision-making",
-        "[[robot]] Automation and physical tasks",
-        "[[]] Data processing and cloud computing",
-        "[[lightbulb]] Insights and predictions",
-        "[[globe2]] Global connectivity and impact"
-      ],
-      "key_message": "AI encompasses various aspects, from human-like intelligence to global impact",
-      "img_keywords": "AI aspects, intelligence, automation, data processing, global impact"
-    },
-    {
-      "heading": "Conclusion: Embracing AI's Potential",
-      "bullet_points": [
-        "AI is transforming industries and improving lives",
-        "Ethical considerations are crucial for responsible AI development",
-        "Invest in AI education and workforce development",
-        "Call to action: Explore AI applications and contribute to shaping its future"
-      ],
-      "key_message": "AI offers immense potential, and we must embrace it responsibly",
-      "img_keywords": "AI transformation, ethical considerations, AI education, future of AI"
-    }
-  ]
-}'''
-
-    temp = tempfile.NamedTemporaryFile(delete=False, suffix='.pptx')
-    path = pathlib.Path(temp.name)
-
-    generate_powerpoint_presentation(
-        json5.loads(_JSON_DATA),
-        output_file_path=path,
-        slides_template='Basic'
-    )
-    print(f'File path: {path}')
-
-    temp.close()

helpers/text_helper.py

@@ -1,83 +0,0 @@
-"""
-Utility functions to help with text processing.
-"""
-import json_repair as jr
-
-
-def is_valid_prompt(prompt: str) -> bool:
-    """
-    Verify whether user input satisfies the concerned constraints.
-
-    :param prompt: The user input text.
-    :return: True if all criteria are satisfied; False otherwise.
-    """
-
-    if len(prompt) < 7 or ' ' not in prompt:
-        return False
-
-    return True
-
-
-def get_clean_json(json_str: str) -> str:
-    """
-    Attempt to clean a JSON response string from the LLM by removing ```json at the beginning and
-    trailing ``` and any text beyond that.
-    CAUTION: May not be always accurate.
-
-    :param json_str: The input string in JSON format.
-    :return: The "cleaned" JSON string.
-    """
-
-    response_cleaned = json_str
-
-    if json_str.startswith('```json'):
-        json_str = json_str[7:]
-
-    while True:
-        idx = json_str.rfind('```')  # -1 on failure
-
-        if idx <= 0:
-            break
-
-        # In the ideal scenario, the character before the last ``` should be
-        # a new line or a closing bracket
-        prev_char = json_str[idx - 1]
-
-        if (prev_char == '}') or (prev_char == '\n' and json_str[idx - 2] == '}'):
-            response_cleaned = json_str[:idx]
-
-        json_str = json_str[:idx]
-
-    return response_cleaned
-
-
-def fix_malformed_json(json_str: str) -> str:
-    """
-    Try and fix the syntax error(s) in a JSON string.
-
-    :param json_str: The input JSON string.
-    :return: The fixed JSOn string.
-    """
-
-    return jr.repair_json(json_str, skip_json_loads=True)
-
-
-if __name__ == '__main__':
-    JSON1 = '''{
-    "key": "value"
-    }
-    '''
-    JSON2 = '''["Reason": "Regular updates help protect against known vulnerabilities."]'''
-    JSON3 = '''["Reason" Regular updates help protect against known vulnerabilities."]'''
-    JSON4 = '''
-    {"bullet_points": [
-        ">> Write without stopping or editing",
-        >> Set daily writing goals and stick to them,
-        ">> Allow yourself to make mistakes"
-    ],}
-    '''
-
-    print(fix_malformed_json(JSON1))
-    print(fix_malformed_json(JSON2))
-    print(fix_malformed_json(JSON3))
-    print(fix_malformed_json(JSON4))