ai: Release J.A.R.V.I.S. Spaces Next-Gen!

* Implemented Audio Generation capabilities

* Added Image Generation features

* Upgrade Deep Search version 2.0 with no limit access

* Redesigned User Interface for enhanced usability

* Integrated with Gradio Chat Interface for seamless interaction

* Refactored core logic for improved performance and maintainability

README.md

@@ -4,11 +4,47 @@ license: apache-2.0
 license_link: https://huggingface.co/hadadrjt/JARVIS/blob/main/LICENSE
 colorFrom: yellow
 colorTo: purple
+emoji: 🌍
 sdk: gradio
-sdk_version: 5.34.0
+sdk_version: 5.34.2
 app_file: app.py
 pinned: true
 short_description: Just a Rather Very Intelligent System
 models:
-  - hadadrjt/JARVIS
+- hadadrjt/JARVIS
+- agentica-org/DeepCoder-14B-Preview
+- deepseek-ai/DeepSeek-V3-0324
+- deepseek-ai/DeepSeek-R1
+- deepseek-ai/DeepSeek-R1-0528
+- deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
+- deepseek-ai/DeepSeek-R1-Distill-Llama-70B
+- google/gemma-3-1b-it
+- google/gemma-3-4b-it
+- google/gemma-3-27b-it
+- meta-llama/Llama-3.1-8B-Instruct
+- meta-llama/Llama-3.2-3B-Instruct
+- meta-llama/Llama-3.3-70B-Instruct
+- meta-llama/Llama-4-Maverick-17B-128E-Instruct
+- meta-llama/Llama-4-Scout-17B-16E-Instruct
+- Qwen/Qwen2.5-VL-3B-Instruct
+- Qwen/Qwen2.5-VL-32B-Instruct
+- Qwen/Qwen2.5-VL-72B-Instruct
+- Qwen/QwQ-32B
+- Qwen/Qwen3-235B-A22B
+- mistralai/Devstral-Small-2505
+- google/gemma-3n-E4B-it-litert-preview
 ---
+
+## Credits
+
+This project expresses sincere gratitude to [Pollinations AI](https://pollinations.ai) for providing audio and image generation services that support the open source community.
+
+Thanks are extended to [SearXNG](https://paulgo.io), [Baidu](https://www.baidu.com), and [Jina AI](https://r.jina.ai) as valuable sources for data retrieval and processing, which contribute to the deep search functionality developed independently.
+
+The latest version of Deep Search is entirely inspired by the [OpenWebUI](https://openwebui.com/t/cooksleep/infinite_search) tools script.
+
+Special appreciation is given to [Hugging Face](https://huggingface.co) for hosting this Space as the primary deployment platform.
+
+## API
+
+Efforts are underway to restore API and multi-platform support at the earliest opportunity.

app.py

@@ -1,12 +1,14 @@
-#
-# SPDX-FileCopyrightText: Hadad <hadad@linuxmail.org>
-# SPDX-License-Identifier: Apache-2.0
-#
+# Import the 'ui' class or function from the 'interface' module located inside the 'src.ui' package.
+# This import statement allows us to use the user interface component defined in that module.
+from src.ui.interface import ui
 
-from src.main.gradio import launch_ui  # Import the function responsible for starting the graphical user interface
-
-# The following condition checks if this script is being run as the main program.
-# If true, it calls the launch_ui function to start the user interface.
-# This ensures that the UI only launches when this file is executed directly, not when imported as a module.
+# This conditional statement checks whether the current script is being run directly (not imported as a module).
+# If this script is executed as the main program, the code inside this block will run.
 if __name__ == "__main__":
-    launch_ui()  # Start the graphical user interface for the application
+    # Create an instance of the 'ui' class or call the 'ui' function to initialize the user interface application.
+    # This object 'app' will represent the running UI application.
+    app = ui()
+
+    # Call the 'launch' method on the 'app' object to start the user interface.
+    # This typically opens the UI window or begins the event loop, making the application interactive.
+    app.queue(default_concurrency_limit=2).launch(show_api=False)

assets/bin/ai

@@ -1,125 +0,0 @@
-#!/usr/bin/env python3
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-import sys
-import re
-
-from gradio_client import Client
-from rich.console import Console, Group
-from rich.markdown import Markdown
-from rich.syntax import Syntax
-from rich.live import Live
-
-# Prepares the display screen to show the final output
-console = Console()
-
-# Creates a connection to the server
-jarvis = Client("hadadrjt/ai")
-
-# Defines the specific AI model to use for responding
-# Change to the model you want, see:
-# https://huggingface.co/spaces/hadadrjt/ai/blob/main/docs/API.md#multi-platform
-model = "JARVIS: 2.1.3"
-
-# Reads user-provided input from the command line, if available
-args = sys.argv[1:]
-
-# Keeps track of whether deep search mode is activated
-deep_search = False
-
-# Checks if the input includes "-d", which turns on deep search
-if "-d" in args:
-    deep_search = True
-    args.remove("-d")
-
-# Combines the rest of the input into one complete message
-# If nothing was typed, it defaults to a basic greeting
-input = " ".join(args) if args else "Hi!"
-
-# Sets the AI to the desired version before sending any messages
-jarvis.predict(new=model, api_name="/change_model")
-
-# Ensures deep search is only enabled for the correct AI model
-if deep_search and model != "JARVIS: 2.1.3":
-    deep_search = False
-
-# Prepares and structures the AI’s response for display
-def layout(text):
-    # Searches for blocks of code within the full response text
-    code_blocks = list(re.finditer(r"\n\n```(.*?)\n\n(.*?)\n\n```\n\n\n", text, re.DOTALL))
-    segments = []  # Stores parts of the final display, including both text and code
-    last_end = 0  # Tracks where the previous segment ended
-
-    # Loops through each code block found in the AI's response
-    for block in code_blocks:
-        # Collects any normal text before the current code block
-        pre_text = text[last_end:block.start()]
-        if pre_text.strip():
-            # Converts plain text into styled text for easier reading
-            segments.append(Markdown(prepare_markdown(pre_text.strip())))
-
-        # Identifies the programming language used in the code block, if available
-        lang = block.group(1).strip() or "text"
-
-        # Extracts the actual code content
-        code = block.group(2).rstrip()
-
-        # Formats the code with syntax highlighting for clear presentation
-        segments.append(Syntax(code, lang, theme="monokai", line_numbers=False, word_wrap=True))
-
-        # Updates the position tracker to move past the current code block
-        last_end = block.end()
-
-    # Checks for any remaining text after the last code block
-    remaining = text[last_end:]
-    if remaining.strip():
-        # Formats and adds this final portion of text to the display
-        segments.append(Markdown(prepare_markdown(remaining.strip())))
-
-    # Returns a complete set of styled segments ready to be shown
-    return Group(*segments)
-
-# Adjusts special characters in the text to ensure they display correctly
-def prepare_markdown(text):
-    return text.replace("•", "*")
-
-# Displays the AI's response in real time as it’s being received
-def response(jarvis):
-    buffer = ""  # Holds the entire reply as it builds up
-    with Live(console=console, transient=False) as live:
-        # Continuously receives and processes parts of the reply
-        for partial in jarvis:
-            # Extracts the latest version of the message from the AI
-            text = partial[0][0][1]
-
-            # Determines what has changed since the last update
-            if text.startswith(buffer):
-                delta = text[len(buffer):]
-            else:
-                delta = text
-
-            # Updates the full message with any new content
-            buffer = text
-
-            # Refreshes the screen with the most recent version of the reply
-            live.update(layout(buffer))
-
-    # Ensures the final reply is printed once it’s complete
-    console.print()
-
-# Sends the user's input to the AI and selects the appropriate method
-if deep_search:
-    # Uses the deeper, slower method for more thoughtful responses
-    jarvis = jarvis.submit(multi={"text": input}, deep_search=True, api_name="/respond_async")
-else:
-    # Uses the standard method for quicker, more direct replies
-    jarvis = jarvis.submit(multi={"text": input}, api_name="/api")
-
-# Create a line break before the main AI's response
-print("")
-
-# Begins the process of showing the AI's response on the screen
-response(jarvis)

assets/bin/install.sh

@@ -1,36 +0,0 @@
-#!/bin/sh
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-echo "Installing required Python packages..."
-pip install gradio_client rich --upgrade
-echo "Installation complete."
-echo ""
-echo ""
-echo "Downloading the J.A.R.V.I.S. script..."
-wget https://huggingface.co/spaces/hadadrjt/ai/raw/main/assets/bin/ai
-echo "Download complete."
-echo ""
-echo ""
-echo "Setting executable permission..."
-chmod a+x ai
-echo "Permission set."
-echo ""
-echo ""
-echo "Removing installer script..."
-rm install.sh
-echo "Done."
-echo ""
-echo ""
-echo "To send a regular message:"
-echo "./ai Your message here"
-echo ""
-echo "To use Deep Search mode:"
-echo "./ai -d Your message here"
-echo ""
-echo ""
-echo "For more details and advanced options, visit:"
-echo "https://huggingface.co/spaces/hadadrjt/ai/blob/main/docs/API.md#installations"
-echo ""

config.py

@@ -3,78 +3,22 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
-import os  # Import os module to access environment variables and interact with the operating system
-import json  # Import json module to parse JSON strings into Python objects
+import os  # Import os module to interact with environment variables
+import json  # Import json module to parse JSON-formatted strings
 
-# Load initial welcome messages for the system from the environment variable "HELLO"
-# If "HELLO" is not set, default to an empty JSON array represented as "[]"
-# This variable typically contains a list of greeting messages or initialization instructions for the AI
-JARVIS_INIT = json.loads(os.getenv("HELLO", "[]"))
+# Load the 'auth' configuration from an environment variable named "auth"
+# This variable is expected to contain a JSON-formatted string representing authentication details
+auth = json.loads(os.getenv("auth"))
 
-# Deep Search service configuration variables loaded from environment variables
-# DEEP_SEARCH_PROVIDER_HOST holds the URL or IP address of the deep search service provider
-DEEP_SEARCH_PROVIDER_HOST = os.getenv("DEEP_SEARCH_PROVIDER_HOST")
-# DEEP_SEARCH_PROVIDER_KEY contains the API key or authentication token required to access the deep search provider
-DEEP_SEARCH_PROVIDER_KEY = os.getenv('DEEP_SEARCH_PROVIDER_KEY')
-# DEEP_SEARCH_INSTRUCTIONS may include specific instructions or parameters guiding how deep search queries should be handled
-DEEP_SEARCH_INSTRUCTIONS = os.getenv("DEEP_SEARCH_INSTRUCTIONS")
+# Load the 'restrictions' configuration from an environment variable named "restrictions"
+# This variable is expected to contain a plain string defining usage restrictions or guidelines
+restrictions = os.getenv("restrictions")
 
-# Internal AI server configuration and system instructions
-# INTERNAL_AI_GET_SERVER stores the endpoint URL or IP address for internal AI GET requests
-INTERNAL_AI_GET_SERVER = os.getenv("INTERNAL_AI_GET_SERVER")
-# INTERNAL_AI_INSTRUCTIONS contains system instructions used to guide the AI behavior
-INTERNAL_AI_INSTRUCTIONS = os.getenv("INTERNAL_TRAINING_DATA")
-
-# System instructions mappings and default instructions loaded from environment variables
-# SYSTEM_PROMPT_MAPPING is a dictionary mapping instructions keys to their corresponding instructions texts, parsed from JSON
-SYSTEM_PROMPT_MAPPING = json.loads(os.getenv("SYSTEM_PROMPT_MAPPING", "{}"))
-# SYSTEM_PROMPT_DEFAULT is the fallback instructions text used when no specific instructions mapping is found
-SYSTEM_PROMPT_DEFAULT = os.getenv("DEFAULT_SYSTEM")
-
-# List of available server hosts for connections or operations
-# This list is parsed from a JSON array string and filtered to exclude any empty or invalid entries
-LINUX_SERVER_HOSTS = [h for h in json.loads(os.getenv("LINUX_SERVER_HOST", "[]")) if h]
-
-# List of provider keys associated with servers, used for authentication
-# The list is parsed from JSON and filtered to remove empty strings
-LINUX_SERVER_PROVIDER_KEYS = [k for k in json.loads(os.getenv("LINUX_SERVER_PROVIDER_KEY", "[]")) if k]
-# Set to keep track of provider keys that have been marked or flagged during runtime
-LINUX_SERVER_PROVIDER_KEYS_MARKED = set()
-# Dictionary to record the number of attempts made with each provider key
-LINUX_SERVER_PROVIDER_KEYS_ATTEMPTS = {}
-
-# Set of server error codes that the system recognizes as critical or requiring special handling
-# The error codes are read from a comma-separated string, filtered to remove empty entries, converted to integers, and stored in a set
-LINUX_SERVER_ERRORS = set(map(int, filter(None, os.getenv("LINUX_SERVER_ERROR", "").split(","))))
-
-# Human-friendly AI types and response messages loaded from environment variables
-# AI_TYPES maps keys like "AI_TYPE_1" to descriptive names or categories of AI models or behaviors
-AI_TYPES = {f"AI_TYPE_{i}": os.getenv(f"AI_TYPE_{i}") for i in range(1, 10)}
-# RESPONSES maps keys like "RESPONSE_1" to predefined response templates or messages used by the AI system
-RESPONSES = {f"RESPONSE_{i}": os.getenv(f"RESPONSE_{i}") for i in range(1, 11)}
-
-# Model-related configurations loaded from environment variables
-# MODEL_MAPPING is a dictionary mapping model keys to their corresponding model names or identifiers, parsed from JSON
-MODEL_MAPPING = json.loads(os.getenv("MODEL_MAPPING", "{}"))
-# MODEL_CONFIG contains detailed configuration settings for each model, such as parameters or options, parsed from JSON
-MODEL_CONFIG = json.loads(os.getenv("MODEL_CONFIG", "{}"))
-# MODEL_CHOICES is a list of available model names extracted from the values of MODEL_MAPPING, useful for selection menus or validation
-MODEL_CHOICES = list(MODEL_MAPPING.values())
-
-# Default model configuration and key used as fallback if no specific model is selected
-# DEFAULT_CONFIG contains default parameters or settings for the AI model, parsed from JSON
-DEFAULT_CONFIG = json.loads(os.getenv("DEFAULT_CONFIG", "{}"))
-# DEFAULT_MODEL_KEY is set to the first key found in MODEL_MAPPING if available, otherwise None
-DEFAULT_MODEL_KEY = list(MODEL_MAPPING.keys())[0] if MODEL_MAPPING else None
+# Load the 'model' configuration from an environment variable named "model"
+# This variable is expected to contain a JSON-formatted string mapping model labels to model names or configurations
+model = json.loads(os.getenv("model"))
 
 # HTML meta tags for SEO and other purposes, loaded as a raw string from environment variables
 # These tags are intended to be inserted into the <head> section of generated HTML pages
-META_TAGS = os.getenv("META_TAGS")
-
-# List of allowed file extensions for upload or processing, parsed from a JSON array string
-# This list helps enforce file type restrictions within the system
-ALLOWED_EXTENSIONS = json.loads(os.getenv("ALLOWED_EXTENSIONS", "[]"))
-
-# Notices or announcements that may be displayed to users or logged by the system
-# The content is loaded as a raw string from the environment variable "NOTICES"
-NOTICES = os.getenv('NOTICES')
+# Used in https://hadadrjt-ai.hf.space
+meta_tags = os.getenv("META_TAGS")

docs/API.md

@@ -1,50 +0,0 @@
-#### Installations
-
-```bash
-# Linux/Android (Termux)/MacOS/Windows.
-# Make sure you have "wget", "python3" and "pip" installed.
-# This package have very small size.
-wget https://huggingface.co/spaces/hadadrjt/ai/raw/main/assets/bin/install.sh && chmod a+x install.sh && ./install.sh
-```
-
-#### Run J.A.R.V.I.S. in your terminal
-
-```bash
-# Example normal usage.
-./ai Your message here.
-
-# Example with Deep Search.
-./ai -d Your message here.
-```
-
-#### Linux user's
-
-```bash
-# Bonus for more flexible.
-sudo mv ai /bin/
-
-# Now you can run with simple command.
-ai Your message here.
-```
-
-### OpenAI Style (developers only)
-
-If you are using the OpenAI style, there is no need to install all the processes mentioned above.
-
-```
-curl https://hadadrjt-api.hf.space/v1/responses \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "JARVIS: 2.1.3",
-    "input": "Write a one-sentence bedtime story about a unicorn.",
-    "stream": true
-  }'
-```
-
-This is a powerful solution for integration with various systems and software, including building your own chatbot. No API key is required.
-
-```
-# Endpoint
-# See at https://huggingface.co/spaces/hadadrjt/api
-https://hadadrjt-api.hf.space/v1
-```

requirements.txt

@@ -1,9 +0,0 @@
-httpx
-openpyxl
-pandas
-pdfplumber
-pillow
-python-docx
-python-pptx
-pytesseract
-requests

src/client/chat_handler.py

@@ -0,0 +1,292 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import json  # Import JSON module for encoding and decoding JSON data
+import uuid  # Import UUID module to generate unique session identifiers
+from typing import Any, List  # Import typing annotations for type hinting
+from config import model  # Import model configuration dictionary from config module
+from src.core.server import jarvis  # Import the async function to interact with AI backend
+from src.core.parameter import parameters  # Import parameters (not used directly here but imported for completeness)
+from src.core.session import session  # Import session dictionary to store conversation histories
+from src.tools.audio import AudioGeneration  # Import AudioGeneration class to handle audio creation
+from src.tools.image import ImageGeneration  # Import ImageGeneration class to handle image creation
+from src.tools.deep_search import SearchTools  # Import SearchTools class for deep search functionality
+import gradio as gr  # Import Gradio library for UI and request handling
+
+# Define an asynchronous function 'respond' to process user messages and generate AI responses
+# This version uses the "messages" style for chat history, where history is a list of dicts with "role" and "content" keys,
+# supporting content as strings, dicts with "path" keys, or Gradio components.
+async def respond(
+    message,  # Incoming user message, can be a string or a dictionary containing text and files
+    history: List[Any],  # List containing conversation history as pairs of user and assistant messages (tuples style)
+    model_label,  # Label/key to select the AI model from the available models
+    temperature,  # Sampling temperature controlling randomness of AI response generation
+    top_k,  # Number of highest probability tokens to keep for sampling
+    min_p,  # Minimum probability threshold for token sampling
+    top_p,  # Cumulative probability threshold for nucleus sampling
+    repetition_penalty,  # Penalty factor to reduce repetitive tokens in generated text
+    thinking,  # Boolean flag indicating if AI should operate in "thinking" mode
+    image_gen,  # Boolean flag to enable image generation commands
+    audio_gen,  # Boolean flag to enable audio generation commands
+    search_gen,  # Boolean flag to enable deep search commands
+    request: gr.Request  # Gradio request object to access session information such as session hash
+):
+    # Select the AI model based on the provided label, if label not found, fallback to the first model in the config
+    selected_model = model.get(model_label, list(model.values())[0])
+    
+    # Instantiate SearchTools to enable deep search capabilities if requested
+    search_tools = SearchTools()
+    
+    # Retrieve session ID from the Gradio request's session hash, generate a new UUID if none exists
+    session_id = request.session_hash or str(uuid.uuid4())
+
+    # Initialize an empty conversation history for this session if it does not already exist
+    if session_id not in session:
+        session[session_id] = []
+
+    # Determine the mode string based on the 'thinking' flag, affects AI response generation behavior
+    mode = "/think" if thinking else "/no_think"
+
+    # Initialize variables for user input text and any attached files
+    input = ""
+    files = None
+
+    # Check if the incoming message is a dictionary (which may contain text and files)
+    if isinstance(message, dict):
+        # Extract the text content from the message dictionary, default to empty string if missing
+        input = message.get("text", "")
+        # Extract the first file from the files list if present, otherwise, set files to None
+        files = message.get("files")[0] if message.get("files") else None
+    else:
+        # If the message is a simple string, assign it directly to input
+        input = message
+
+    # Strip leading and trailing whitespace from the input for clean processing
+    stripped_input = input.strip()
+    # Convert the stripped input to lowercase for case-insensitive command detection
+    lowered_input = stripped_input.lower()
+
+    # If the input is empty after stripping, yield an empty list and exit the function early
+    if not stripped_input:
+        yield []
+        return
+
+    # If the input is exactly one of the command keywords without parameters, yield empty and exit early
+    if lowered_input in ["/audio", "/image", "/dp"]:
+        yield []
+        return
+
+    # Prepare a new conversation history list formatted with roles and content for AI model consumption
+    # Here we convert the old "tuples" style history (list of [user_msg, assistant_msg]) into "messages" style:
+    # a flat list of dicts with "role" and "content" keys.
+    new_history = []
+    for entry in history:
+        # Ensure the entry is a list with exactly two elements: user message and assistant message
+        if isinstance(entry, list) and len(entry) == 2:
+            user_msg, assistant_msg = entry
+            # Append the user message with role 'user' to the new history if not None
+            if user_msg is not None:
+                new_history.append({"role": "user", "content": user_msg})
+            # Append the assistant message with role 'assistant' if it exists and is not None
+            if assistant_msg is not None:
+                new_history.append({"role": "assistant", "content": assistant_msg})
+
+    # Update the global session dictionary with the newly formatted conversation history for this session
+    session[session_id] = new_history
+
+    # Handle audio generation command if enabled and input starts with '/audio'
+    if audio_gen and lowered_input.startswith("/audio"):
+        # Extract the audio instruction text after the '/audio' command prefix and strip whitespace
+        audio_instruction = input[6:].strip()
+        # If no instruction text is provided, yield empty and exit early
+        if not audio_instruction:
+            yield []
+            return
+        try:
+            # Asynchronously create audio content based on the instruction using AudioGeneration class
+            audio = await AudioGeneration.create_audio(audio_instruction)
+            # Serialize the audio data and instruction into a JSON formatted string
+            audio_generation_content = json.dumps({
+                "audio": audio,
+                "audio_instruction": audio_instruction
+            })
+            # Construct the conversation history including the audio generation result and detailed instructions
+            audio_generation_result = (
+                new_history
+                + [
+                    {
+                        "role": "system",
+                        "content": (
+                            f"Audio generation result:\n\n{audio_generation_content}\n\n\n"
+                             "Show the audio using the following HTML audio tag format, where '{audio_link}' is the URL of the generated audio:\n\n"
+                             "<audio controls src='{audio_link}' style='width:100%; max-width:100%;'></audio>\n\n"
+                             "Please replace '{audio_link}' with the actual audio URL provided in the context.\n\n"
+                             "Then, describe the generated audio based on the above information.\n\n\n"
+                             "Use the same language as the previous user input or user request.\n"
+                             "For example, if the previous user input or user request is in Indonesian, explain in Indonesian.\n"
+                             "If it is in English, explain in English. This also applies to other languages.\n\n\n"
+                        )
+                    }
+                ]
+            )
+
+            # Use async generator to get descriptive text about the generated audio
+            async for audio_description in jarvis(
+                session_id=session_id,
+                model=selected_model,
+                history=audio_generation_result,
+                user_message=input,
+                mode="/no_think",  # Use no_think mode to avoid extra processing
+                temperature=0.7,  # Fixed temperature for audio description generation
+                top_k=20,  # Limit token sampling to top 20 tokens
+                min_p=0,  # Minimum probability threshold
+                top_p=0.8,  # Nucleus sampling threshold
+                repetition_penalty=1.0  # No repetition penalty for this step
+            ):
+                # Yield the audio description wrapped in a tool role for UI display
+                yield [{"role": "tool", "content": f'{audio_description}'}]
+            return
+        except Exception:
+            # If audio generation fails, yield an error message and exit
+            yield [{"role": "tool", "content": "Audio generation failed. Please wait 15 seconds before trying again."}]
+            return
+
+    # Handle image generation command if enabled and input starts with '/image'
+    if image_gen and lowered_input.startswith("/image"):
+        # Extract the image generation instruction after the '/image' command prefix and strip whitespace
+        generate_image_instruction = input[6:].strip()
+        # If no instruction text is provided, yield empty and exit early
+        if not generate_image_instruction:
+            yield []
+            return
+        try:
+            # Asynchronously create image content based on the instruction using ImageGeneration class
+            image = await ImageGeneration.create_image(generate_image_instruction)
+
+            # Serialize the image data and instruction into a JSON formatted string
+            image_generation_content = json.dumps({
+                "image": image,
+                "generate_image_instruction": generate_image_instruction
+            })
+
+            # Construct the conversation history including the image generation result and detailed instructions
+            image_generation_result = (
+                new_history
+                + [
+                    {
+                        "role": "system",
+                        "content": (
+                            f"Image generation result:\n\n{image_generation_content}\n\n\n"
+                             "Show the generated image using the following markdown syntax format, where '{image_link}' is the URL of the image:\n\n"
+                             "![Generated Image]({image_link})\n\n"
+                             "Please replace '{image_link}' with the actual image URL provided in the context.\n\n"
+                             "Then, describe the generated image based on the above information.\n\n\n"
+                             "Use the same language as the previous user input or user request.\n"
+                             "For example, if the previous user input or user request is in Indonesian, explain in Indonesian.\n"
+                             "If it is in English, explain in English. This also applies to other languages.\n\n\n"
+                        )
+                    }
+                ]
+            )
+
+            # Use async generator to get descriptive text about the generated image
+            async for image_description in jarvis(
+                session_id=session_id,
+                model=selected_model,
+                history=image_generation_result,
+                user_message=input,
+                mode="/no_think",  # Use no_think mode to avoid extra processing
+                temperature=0.7,  # Fixed temperature for image description generation
+                top_k=20,  # Limit token sampling to top 20 tokens
+                min_p=0,  # Minimum probability threshold
+                top_p=0.8,  # Nucleus sampling threshold
+                repetition_penalty=1.0  # No repetition penalty for this step
+            ):
+                # Yield the image description wrapped in a tool role for UI display
+                yield [{"role": "tool", "content": f"{image_description}"}]
+            return
+        except Exception:
+            # If image generation fails, yield an error message and exit
+            yield [{"role": "tool", "content": "Image generation failed. Please wait 15 seconds before trying again."}]
+            return
+
+    # Handle deep search command if enabled and input starts with '/dp'
+    if search_gen and lowered_input.startswith("/dp"):
+        # Extract the search query after the '/dp' command prefix and strip whitespace
+        search_query = input[3:].strip()
+        # If no search query is provided, yield empty and exit early
+        if not search_query:
+            yield []
+            return
+        
+        try:
+            # Perform an asynchronous deep search using SearchTools with the given query
+            search_results = await search_tools.search(search_query)
+
+            # Serialize the search query and results (limited to first 5000 characters) into JSON string
+            search_content = json.dumps({
+                "query": search_query,
+                "search_results": search_results[:5000]
+            })
+            
+            # Construct conversation history including deep search results and detailed instructions for summarization
+            search_instructions = (
+                new_history
+                + [
+                    {
+                        "role": "system",
+                        "content": (
+                            f"Deep search results for query: '{search_query}':\n\n{search_content}\n\n\n"
+                             "Please analyze these search results and provide a comprehensive summary of the information.\n"
+                             "Identify the most relevant information related to the query.\n"
+                             "Format your response in a clear, structured way with appropriate headings and bullet points if needed.\n"
+                             "If the search results don't provide sufficient information, acknowledge this limitation.\n"
+                             "Please provide links or URLs from each of your search results.\n\n"
+                             "Use the same language as the previous user input or user request.\n"
+                             "For example, if the previous user input or user request is in Indonesian, explain in Indonesian.\n"
+                             "If it is in English, explain in English. This also applies to other languages.\n\n\n"
+                        )
+                    }
+                ]
+            )
+
+            # Use async generator to process the deep search results and generate a summary response
+            async for search_response in jarvis(
+                session_id=session_id,
+                model=selected_model,
+                history=search_instructions,
+                user_message=input,
+                mode=mode,  # Use the mode determined by the thinking flag
+                temperature=temperature,
+                top_k=top_k,
+                min_p=min_p,
+                top_p=top_p,
+                repetition_penalty=repetition_penalty
+            ):
+                # Yield the search summary wrapped in a tool role for UI display
+                yield [{"role": "tool", "content": f"{search_response}"}]
+            return
+            
+        except Exception as e:
+            # If deep search fails, yield an error message and exit
+            yield [{"role": "tool", "content": "Search failed, please try again later."}]
+            return
+
+    # For all other inputs that do not match special commands, use the jarvis function to generate a response
+    async for response in jarvis(
+        session_id=session_id,
+        model=selected_model,
+        history=new_history,  # Pass the conversation history in "messages" style format
+        user_message=input,
+        mode=mode,  # Use the mode determined by the thinking flag
+        files=files,  # Pass any attached files along with the message
+        temperature=temperature,
+        top_k=top_k,
+        min_p=min_p,
+        top_p=top_p,
+        repetition_penalty=repetition_penalty
+    ):
+        # Yield each chunk of the response as it is generated
+        yield response

src/core/parameter.py

@@ -0,0 +1,53 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+# Model parameters
+def parameters(reasoning: bool):
+    """
+    Determine and return a set of model generation parameters based on whether reasoning mode is enabled.
+
+    Args:
+        reasoning (bool): A flag indicating if reasoning mode is active. 
+                          When True, parameters favor more controlled and focused generation suitable for reasoning tasks.
+                          When False, parameters allow for more creative or diverse outputs.
+
+    Returns:
+        tuple: A tuple containing five parameters used for text generation:
+            - temperature (float): Controls randomness in generation. Lower values make output more deterministic.
+            - top_k (int): Limits sampling to the top_k most likely next tokens.
+            - min_p (float): Minimum probability threshold for token inclusion (0 means no minimum).
+            - top_p (float): Nucleus sampling threshold, cumulative probability cutoff for token selection.
+            - repetition_penalty (float): Penalizes repeated tokens to reduce repetition in generated text.
+    """
+    # Reasoning
+    if reasoning:
+        # Parameters tuned for reasoning tasks:
+        # Lower temperature (0.6) to reduce randomness and improve logical consistency.
+        # top_k set to 20 to limit choices to the 20 most probable tokens, focusing generation.
+        # min_p is 0, meaning no minimum probability cutoff is enforced.
+        # top_p is 0.95, allowing nucleus sampling to consider tokens covering 95% cumulative probability.
+        # repetition_penalty is 1.0, meaning no penalty applied for token repetition.
+        return (
+            0.6,  # temperature: less randomness for focused reasoning
+            20,   # top_k: restrict to top 20 tokens for more precise output
+            0,    # min_p: no minimum probability threshold
+            0.95, # top_p: nucleus sampling cutoff to include tokens up to 95% cumulative probability
+            1.0   # repetition_penalty: no penalty on repeated tokens
+        )
+    # Non-reasoning
+    else:
+        # Parameters tuned for non-reasoning or more creative generation:
+        # Slightly higher temperature (0.7) to allow more diversity and creativity.
+        # top_k remains 20 to keep some restriction on token selection.
+        # min_p is 0.0, no minimum probability cutoff.
+        # top_p is lower at 0.8, narrowing nucleus sampling to more probable tokens for balanced creativity.
+        # repetition_penalty remains 1.0, no penalty on repeated tokens.
+        return (
+            0.7,  # temperature: more randomness for creative outputs
+            20,   # top_k: restrict to top 20 tokens to maintain some control
+            0.0,  # min_p: no minimum probability threshold
+            0.8,  # top_p: nucleus sampling cutoff at 80% cumulative probability
+            1.0   # repetition_penalty: no penalty on repeated tokens
+        )

src/core/server.py

@@ -0,0 +1,188 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import json  # Import JSON module to parse and handle JSON data
+import uuid  # Import UUID module to generate unique identifiers for sessions
+from typing import List, Dict, Any  # Import type hints for lists, dictionaries, and generic types
+from datetime import datetime  # Import datetime to get and format current date/time
+from config import *  # Import all configuration variables including 'auth' and 'restrictions'
+from src.utils.session_mapping import get_host  # Import function to get server info by session ID
+from src.utils.ip_generator import generate_ip  # Import function to generate random IP for headers
+from src.utils.helper import mark  # Import function to mark a server as busy/unavailable
+from src.ui.reasoning import styles  # Import function to apply CSS styling to reasoning output
+import httpx  # Import httpx for async HTTP requests with streaming support
+
+async def jarvis(
+    session_id: str,  # Unique session identifier to maintain consistent server assignment
+    model: str,  # AI model name to specify which model to use
+    history: List[Dict[str, str]],  # List of previous conversation messages with roles and content
+    user_message: str,  # Latest user input message to send to the AI model
+    mode: str,  # Mode string to guide AI behavior, e.g., '/think' or '/no_think'
+    files=None,  # Optional files or attachments to include with the user message
+    temperature: float = 0.6,  # Sampling temperature controlling randomness in token generation
+    top_k: int = 20,  # Limit token selection to top_k probable tokens
+    min_p: float = 0,  # Minimum probability threshold for token selection
+    top_p: float = 0.95,  # Nucleus sampling cumulative probability threshold
+    repetition_penalty: float = 1.0,  # Penalty factor to reduce token repetition
+):
+    """
+    Asynchronously send a chat request to a Jarvis AI server and handle streaming response incrementally.
+
+    This function manages server selection based on the session ID, retries requests on specific error codes,
+    and yields incremental parts of the AI-generated response as they arrive. It integrates CSS styling into
+    the reasoning output only if the mode is not '/no_think', preserving the behavior where reasoning is streamed
+    first inside a styled HTML block, followed by the main content streamed normally.
+
+    Args:
+        session_id (str): Identifier for the user session to maintain consistent server assignment.
+        model (str): Name of the AI model to use for generating the response.
+        history (List[Dict[str, str]]): List of previous messages in the conversation.
+        user_message (str): The current message from the user to send to the AI model.
+        mode (str): Contextual instructions to guide the AI model's response style.
+        files (optional): Additional files or attachments to include with the user message.
+        temperature (float): Controls randomness in token generation.
+        top_k (int): Limits token selection to top_k probable tokens.
+        min_p (float): Minimum probability threshold for token selection.
+        top_p (float): Nucleus sampling cumulative probability threshold.
+        repetition_penalty (float): Factor to reduce token repetition.
+
+    Yields:
+        str: Incremental strings of AI-generated response streamed from the server.
+             Reasoning is wrapped in a styled HTML details block and streamed incrementally only if mode is not '/no_think'.
+             After reasoning finishes, the main content is streamed normally.
+
+    Notes:
+        The function attempts to send the request to a server assigned for the session.
+        If the server returns a specific error code indicating it is busy, it retries with another server.
+        If all servers are busy or fail, it yields a message indicating the server is busy.
+    """
+    tried = set()  # Track servers already tried to avoid repeated retries
+
+    # Loop until all available servers have been tried without success
+    while len(tried) < len(auth):
+        # Get server setup info assigned for this session, including endpoint, token, and error code
+        setup = get_host(session_id)
+        server = setup["jarvis"]  # Server identifier
+        host = setup["endpoint"]  # API endpoint URL
+        token = setup["token"]  # Authorization token
+        error = setup["error"]  # HTTP error code triggering retry
+        tried.add(server)  # Mark this server as tried
+
+        # Format current date/time string for system instructions
+        date = datetime.now().strftime("%A, %B %d, %Y, %I:%M %p %Z")
+
+        # Combine mode instructions, usage restrictions, and date into system instructions string
+        instructions = f"{mode}\n\n\n{restrictions}\n\n\nToday: {date}\n\n\n"
+
+        # Copy conversation history to avoid mutating original
+        messages = history.copy()
+        # Insert system instructions as first message
+        messages.insert(0, {"role": "system", "content": instructions})
+
+        # Prepare user message dict, include files if provided
+        msg = {"role": "user", "content": user_message}
+        if files:
+            msg["files"] = files
+        messages.append(msg)  # Append user message to conversation
+
+        # Prepare HTTP headers with authorization and randomized client IP
+        headers = {
+            "Authorization": f"Bearer {token}",  # Bearer token for API access
+            "Content-Type": "application/json",  # JSON content type
+            "X-Forwarded-For": generate_ip()  # Random IP to simulate different client origins
+        }
+
+        # Prepare JSON payload with model parameters and conversation messages
+        payload = {
+            "model": model,
+            "messages": messages,
+            "stream": True,
+            "temperature": temperature,
+            "top_k": top_k,
+            "min_p": min_p,
+            "top_p": top_p,
+            "repetition_penalty": repetition_penalty,
+        }
+
+        # Initialize accumulators and flags for streamed response parts
+        reasoning = ""  # Accumulate reasoning text
+        reasoning_check = None  # Flag to detect presence of reasoning in response
+        reasoning_done = False  # Flag marking reasoning completion
+        content = ""  # Accumulate main content text
+
+        try:
+            # Create async HTTP client with no timeout for long streaming
+            async with httpx.AsyncClient(timeout=None) as client:
+                # Open async streaming POST request to Jarvis server
+                async with client.stream("POST", host, headers=headers, json=payload) as response:
+                    # Iterate asynchronously over each line of streaming response
+                    async for chunk in response.aiter_lines():
+                        # Skip lines not starting with "data:"
+                        if not chunk.strip().startswith("data:"):
+                            continue
+                        try:
+                            # Parse JSON data after "data:" prefix
+                            data = json.loads(chunk[5:])
+                            # Extract incremental delta message from first choice
+                            choice = data["choices"][0]["delta"]
+
+                            # On first delta received, detect if 'reasoning' field is present and non-empty
+                            if reasoning_check is None:
+                                # Initialize reasoning_check to empty string if reasoning exists and is non-empty, else None
+                                reasoning_check = "" if ("reasoning" in choice and choice["reasoning"]) else None
+
+                            # If reasoning is present and mode is not '/no_think' and reasoning not done
+                            if (
+                                reasoning_check == ""  # Reasoning detected in response
+                                and mode != "/no_think"  # Mode allows reasoning output
+                                and not reasoning_done  # Reasoning phase not finished yet
+                                and "reasoning" in choice  # Current delta includes reasoning part
+                                and choice["reasoning"]  # Reasoning content is not empty
+                            ):
+                                reasoning += choice["reasoning"]  # Append incremental reasoning text
+                                # Yield reasoning wrapped in styled HTML block with details expanded
+                                yield styles(reasoning=reasoning, content="", expanded=True)
+                                continue  # Continue streaming reasoning increments
+
+                            # When reasoning ends and content starts, mark reasoning done, yield empty string, then content
+                            if (
+                                reasoning_check == ""  # Reasoning was detected previously
+                                and mode != "/no_think"  # Mode allows reasoning output
+                                and not reasoning_done  # Reasoning phase not finished yet
+                                and "content" in choice  # Current delta includes content part
+                                and choice["content"]  # Content is not empty
+                            ):
+                                reasoning_done = True  # Mark reasoning phase complete
+                                yield ""  # Yield empty string to signal end of reasoning block
+                                content += choice["content"]  # Start accumulating content text
+                                yield content  # Yield first part of content
+                                continue  # Continue streaming content increments
+
+                            # If no reasoning present or reasoning done, accumulate content and yield incrementally
+                            if (
+                                (reasoning_check is None or reasoning_done or mode == "/no_think")  # No reasoning or reasoning finished or mode disables reasoning
+                                and "content" in choice  # Current delta includes content part
+                                and choice["content"]  # Content is not empty
+                            ):
+                                content += choice["content"]  # Append incremental content text
+                                yield content  # Yield updated content string
+                        except Exception:
+                            # Ignore exceptions during JSON parsing or key access and continue streaming
+                            continue
+            return  # Exit function after successful streaming completion
+        except httpx.HTTPStatusError as e:
+            # If server returns specific error code indicating busy, retry with another server
+            if e.response.status_code == error:
+                continue  # Try next available server
+            else:
+                # For other HTTP errors, mark this server as busy
+                mark(server)
+        except Exception:
+            # For other exceptions (network errors, timeouts), mark server as busy
+            mark(server)
+
+    # If all servers tried and none succeeded, yield busy message
+    yield "The server is currently busy. Please wait a moment or try again later."
+    return  # End of function

src/core/session.py

@@ -0,0 +1,12 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+from typing import Dict, List  # Import type hinting tools 'Dict' and 'List' from the typing module to specify complex data structures
+
+# Initialize an empty dictionary named 'session' to store session-related data.
+# The dictionary keys are strings, which could represent session IDs or user identifiers.
+# Each key maps to a list of dictionaries, where each dictionary contains string keys and string values.
+# This structure allows storing multiple records per session, with each record represented as a dictionary of string-to-string pairs.
+session: Dict[str, List[Dict[str, str]]] = {}  # Empty dictionary ready to hold session data structured as described

src/cores/client.py

@@ -1,161 +0,0 @@
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-import asyncio  # Import asyncio for asynchronous programming capabilities
-import httpx  # Import httpx to perform asynchronous HTTP requests
-import json  # Import json to handle JSON encoding and decoding
-import random  # Import random to shuffle lists for load balancing
-import uuid  # Import uuid to generate unique session identifiers
-
-from config import *  # Import all configuration constants and variables from config module
-from src.cores.server import fetch_response_stream_async  # Import async function to fetch streamed AI responses
-from src.cores.session import ensure_stop_event, get_model_key  # Import session helper functions
-from datetime import datetime  # Import datetime to get current date and time information
-
-async def chat_with_model_async(history, user_input, model_display, sess, custom_prompt, deep_search):
-    """
-    Asynchronous function to handle interaction with an AI model and stream its responses.
-
-    Parameters:
-    - history: List of tuples containing previous conversation messages (user and assistant)
-    - user_input: The current input string from the user
-    - model_display: The display name of the AI model to use
-    - sess: Session object containing session state, stop event, and cancellation token
-    - custom_prompt: Optional custom system instructions to override default instructions
-    - deep_search: Boolean flag indicating whether to integrate deep search results into the instructions
-
-    This function prepares the message history and system instructions, optionally enriches the instructions
-    with deep search results if enabled, and attempts to fetch streamed responses from multiple backend
-    providers with fallback. It yields chunks of the response asynchronously for real-time UI updates.
-    """
-
-    # Ensure the session has a stop event initialized to control streaming cancellation
-    ensure_stop_event(sess)
-
-    # Clear any previous stop event state to allow new streaming session
-    sess.stop_event.clear()
-
-    # Reset the cancellation token to indicate the session is active and not cancelled
-    sess.cancel_token["cancelled"] = False
-
-    # Check if provider keys and hosts are configured; if not, yield a predefined error response and exit
-    if not LINUX_SERVER_PROVIDER_KEYS or not LINUX_SERVER_HOSTS:
-        yield ("content", RESPONSES["RESPONSE_3"])  # Inform user no backend providers are available
-        return
-
-    # Assign a unique session ID if not already present to track conversation context
-    if not hasattr(sess, "session_id") or not sess.session_id:
-        sess.session_id = str(uuid.uuid4())
-
-    # Determine the internal model key based on the display name, falling back to default if not found
-    model_key = get_model_key(model_display, MODEL_MAPPING, DEFAULT_MODEL_KEY)
-
-    # Retrieve model-specific configuration parameters or use default configuration
-    cfg = MODEL_CONFIG.get(model_key, DEFAULT_CONFIG)
-
-    # Initialize a list to hold the messages that will be sent to the AI model
-    msgs = []
-
-    # Obtain the current date and time formatted as a readable string for context in instructions
-    current_date = datetime.now().strftime("%A, %B %d, %Y, %I:%M %p %Z")
-
-    # Combine internal AI instructions with the current date to form a comprehensive system instructions
-    COMBINED_AI_INSTRUCTIONS = (
-        INTERNAL_AI_INSTRUCTIONS
-        + "\n\n\n"
-        + f"Today is: {current_date}"
-        + "\n\n\n"
-    )
-
-    # If deep search is enabled and the primary model is selected, prepend deep search instructions and results
-    if deep_search and model_display == MODEL_CHOICES[0]:
-        # Add deep search instructions as a system message to guide the AI
-        msgs.append({"role": "system", "content": DEEP_SEARCH_INSTRUCTIONS})
-        try:
-            # Create an asynchronous HTTP client session for making the deep search request
-            async with httpx.AsyncClient() as client:
-                # Define the payload with parameters for the deep search query
-                payload = {
-                    "query": user_input,
-                    "topic": "general",
-                    "search_depth": "basic",
-                    "chunks_per_source": 5,
-                    "max_results": 5,
-                    "time_range": None,
-                    "days": 7,
-                    "include_answer": True,
-                    "include_raw_content": False,
-                    "include_images": False,
-                    "include_image_descriptions": False,
-                    "include_domains": [],
-                    "exclude_domains": []
-                }
-                # Send a POST request to the deep search provider with authorization header and JSON payload
-                r = await client.post(
-                    DEEP_SEARCH_PROVIDER_HOST,
-                    headers={"Authorization": f"Bearer {DEEP_SEARCH_PROVIDER_KEY}"},
-                    json=payload
-                )
-                # Parse the JSON response from the deep search provider
-                sr_json = r.json()
-                # Append the deep search results as a system message in JSON string format
-                msgs.append({"role": "system", "content": json.dumps(sr_json)})
-        except Exception:
-            # If any error occurs during deep search, fail silently without interrupting the chat flow
-            pass
-        # Append the combined AI instructions after the deep search content to maintain context
-        msgs.append({"role": "system", "content": COMBINED_AI_INSTRUCTIONS})
-
-    # If deep search is not enabled but the primary model is selected, use only the combined AI instructions
-    elif model_display == MODEL_CHOICES[0]:
-        msgs.append({"role": "system", "content": COMBINED_AI_INSTRUCTIONS})
-
-    # For other models, use a custom instructions if provided, otherwise default to the system instructions mapping or default instructions
-    else:
-        msgs.append({"role": "system", "content": custom_prompt or SYSTEM_PROMPT_MAPPING.get(model_key, SYSTEM_PROMPT_DEFAULT)})
-
-    # Append the conversation history to the message list, alternating user and assistant messages
-    # First add all user messages from history
-    msgs.extend([{"role": "user", "content": u} for u, _ in history])
-    # Then add all assistant messages from history that are not empty
-    msgs.extend([{"role": "assistant", "content": a} for _, a in history if a])
-
-    # Append the current user input as the latest user message
-    msgs.append({"role": "user", "content": user_input})
-
-    # Create a list of all possible combinations of backend hosts and provider keys for load balancing and fallback
-    candidates = [(h, k) for h in LINUX_SERVER_HOSTS for k in LINUX_SERVER_PROVIDER_KEYS]
-
-    # Randomly shuffle the list of host-key pairs to distribute load evenly and avoid bias
-    random.shuffle(candidates)
-
-    # Iterate over each host and key pair to attempt fetching a streamed response
-    for h, k in candidates:
-        # Call the async generator function to fetch streamed response chunks from the backend
-        stream_gen = fetch_response_stream_async(
-            h, k, model_key, msgs, cfg, sess.session_id, sess.stop_event, sess.cancel_token
-        )
-
-        # Flag to track if any response chunks were received from this provider
-        got_responses = False
-
-        # Asynchronously iterate over each chunk yielded by the streaming generator
-        async for chunk in stream_gen:
-            # If the stop event is set or cancellation requested, terminate streaming immediately
-            if sess.stop_event.is_set() or sess.cancel_token["cancelled"]:
-                return
-
-            # Mark that at least one response chunk has been received
-            got_responses = True
-
-            # Yield the current chunk to the caller for incremental UI update or processing
-            yield chunk
-
-        # If any responses were received from this host-key pair, stop trying others and return
-        if got_responses:
-            return
-
-    # If no responses were received from any provider, yield a fallback message indicating failure
-    yield ("content", RESPONSES["RESPONSE_2"])

src/cores/server.py

@@ -1,101 +0,0 @@
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-import codecs  # Import codecs module for encoding and decoding operations, useful for handling text data
-import httpx  # Import httpx for making asynchronous HTTP requests to external servers or APIs
-import json  # Import json module to parse JSON formatted strings into Python objects and vice versa
-
-from src.cores.session import marked_item  # Import marked_item function to track and mark keys that fail repeatedly, helping to avoid using problematic keys
-from config import LINUX_SERVER_ERRORS, LINUX_SERVER_PROVIDER_KEYS_MARKED, LINUX_SERVER_PROVIDER_KEYS_ATTEMPTS, RESPONSES  # Import various constants used for error handling, key marking, retry attempts, and predefined responses
-
-async def fetch_response_stream_async(host, key, model, msgs, cfg, sid, stop_event, cancel_token):
-    """
-    Asynchronous generator function that streams AI-generated responses from a backend server endpoint.
-
-    Parameters:
-    - host: The URL of the backend server to send the request to.
-    - key: Authorization token (API key) used in the request header for authentication.
-    - model: The AI model identifier to be used for generating responses.
-    - msgs: The list of messages forming the conversation or prompt to send to the AI.
-    - cfg: Configuration dictionary containing additional parameters for the request.
-    - sid: Session ID string to associate the request with a particular session.
-    - stop_event: An asynchronous event object that signals when to stop streaming responses.
-    - cancel_token: A dictionary containing a 'cancelled' boolean flag to abort the streaming operation.
-
-    This function attempts to connect to the backend server twice with different timeout values (5 and 10 seconds).
-    It sends a POST request with JSON payload that includes model, messages, session ID, stream flag, and configuration.
-    The function streams the response line-by-line, parsing JSON data chunks as they arrive.
-
-    The streamed data contains two types of text parts:
-    - 'reasoning': Additional reasoning text that can be displayed separately in the UI for richer user experience.
-    - 'content': The main content text generated by the AI.
-
-    The function yields tuples of the form ('reasoning', text) or ('content', text) to the caller asynchronously.
-
-    If the server returns an error status code listed in LINUX_SERVER_ERRORS, the key is marked as problematic to avoid future use.
-    The function also respects stop_event and cancel_token to allow graceful cancellation of the streaming process.
-
-    If the response signals completion with a specific message defined in RESPONSES["RESPONSE_10"], the function ends the stream.
-
-    The function handles exceptions gracefully, including network errors and JSON parsing issues, retrying or marking keys as needed.
-    """
-    # Loop over two timeout values to attempt the request with increasing timeout durations for robustness
-    for timeout in [5, 10]:
-        try:
-            # Create an asynchronous HTTP client with the specified timeout for the request
-            async with httpx.AsyncClient(timeout=timeout) as client:
-                # Open a streaming POST request to the backend server with JSON payload and authorization header
-                async with client.stream(
-                    "POST",
-                    host,
-                    # Combine fixed parameters with additional configuration into the JSON body
-                    json={**{"model": model, "messages": msgs, "session_id": sid, "stream": True}, **cfg},
-                    headers={"Authorization": f"Bearer {key}"}  # Use Bearer token authentication
-                ) as response:
-                    # Check if the response status code indicates a server error that should mark the key
-                    if response.status_code in LINUX_SERVER_ERRORS:
-                        # Mark the key as problematic with the provided tracking function and exit the generator
-                        marked_item(key, LINUX_SERVER_PROVIDER_KEYS_MARKED, LINUX_SERVER_PROVIDER_KEYS_ATTEMPTS)
-                        return
-
-                    # Iterate asynchronously over each line of the streamed response content
-                    async for line in response.aiter_lines():
-                        # If the stop event is set or cancellation is requested, stop streaming and exit
-                        if stop_event.is_set() or cancel_token["cancelled"]:
-                            return
-                        # Skip empty lines to avoid unnecessary processing
-                        if not line:
-                            continue
-                        # Process lines that start with the prefix 'data: ' which contain JSON payloads
-                        if line.startswith("data: "):
-                            data = line[6:]  # Extract the JSON string after 'data: '
-                            # If the data matches the predefined end-of-response message, stop streaming
-                            if data.strip() == RESPONSES["RESPONSE_10"]:
-                                return
-                            try:
-                                # Attempt to parse the JSON data string into a Python dictionary
-                                j = json.loads(data)
-                                # Check if the parsed object is a dictionary containing 'choices' key
-                                if isinstance(j, dict) and j.get("choices"):
-                                    # Iterate over each choice in the response to extract text deltas
-                                    for ch in j["choices"]:
-                                        delta = ch.get("delta", {})  # Get the incremental update part
-                                        # If 'reasoning' text is present in the delta, decode unicode escapes and yield it
-                                        if "reasoning" in delta and delta["reasoning"]:
-                                            decoded = delta["reasoning"].encode('utf-8').decode('unicode_escape')
-                                            yield ("reasoning", decoded)  # Yield reasoning text for UI display
-                                        # If main 'content' text is present in the delta, yield it directly
-                                        if "content" in delta and delta["content"]:
-                                            yield ("content", delta["content"])  # Yield main content text
-                            except Exception:
-                                # Ignore exceptions from malformed JSON or unexpected data formats and continue streaming
-                                continue
-        except Exception:
-            # Catch network errors, timeouts, or other exceptions and try the next timeout or retry
-            continue
-        # If all attempts fail, mark the key as problematic to avoid future use
-        marked_item(key, LINUX_SERVER_PROVIDER_KEYS_MARKED, LINUX_SERVER_PROVIDER_KEYS_ATTEMPTS)
-    # Return None explicitly when streaming ends or fails after retries
-    return

src/cores/session.py

@@ -1,93 +0,0 @@
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-import asyncio  # Import the asyncio library to handle asynchronous operations and events
-import requests  # Import the requests library for HTTP requests and session management
-import uuid  # Import the uuid library to generate unique identifiers
-import threading  # Import threading to run background timers for delayed operations
-
-from config import LINUX_SERVER_PROVIDER_KEYS_MARKED, LINUX_SERVER_PROVIDER_KEYS_ATTEMPTS  # Import configuration variables that track marked provider keys and their failure attempts
-
-class SessionWithID(requests.Session):
-    """
-    Custom session class extending requests.Session to add unique session identification 
-    and asynchronous cancellation control. This allows tracking individual user sessions 
-    and managing cancellation of ongoing HTTP requests asynchronously.
-    """
-    def __init__(self):
-        super().__init__()  # Initialize the base requests.Session class
-        self.session_id = str(uuid.uuid4())  
-        # Generate and assign a unique string ID for this session instance to identify it uniquely
-        self.stop_event = asyncio.Event()    
-        # Create an asyncio Event object used to signal when the session should stop or cancel operations
-        self.cancel_token = {"cancelled": False}  
-        # Dictionary flag to indicate if the current session's operations have been cancelled
-
-def create_session():
-    """
-    Factory function to create and return a new SessionWithID instance.
-    This should be called whenever a new user session starts or a chat session is reset,
-    ensuring each session has its own unique ID and cancellation controls.
-    """
-    return SessionWithID()
-
-def ensure_stop_event(sess):
-    """
-    Utility function to verify that a given session object has the required asynchronous 
-    control attributes: stop_event and cancel_token. If they are missing (e.g., when restoring 
-    sessions from storage), this function adds them to maintain consistent session behavior.
-    
-    Parameters:
-    - sess: The session object to check and update.
-    """
-    if not hasattr(sess, "stop_event"):
-        sess.stop_event = asyncio.Event()  
-        # Add an asyncio Event to signal stop requests if missing
-    if not hasattr(sess, "cancel_token"):
-        sess.cancel_token = {"cancelled": False}  
-        # Add a cancellation flag dictionary if missing
-
-def marked_item(item, marked, attempts):
-    """
-    Mark a provider key or host as temporarily problematic after repeated failures to prevent 
-    using unreliable providers continuously. This function adds the item to a 'marked' set 
-    and increments its failure attempt count. If the failure count reaches 3 or more, a timer 
-    is started to automatically unmark the item after 5 minutes (300 seconds), allowing retries.
-    
-    Parameters:
-    - item: The provider key or host identifier to mark as problematic.
-    - marked: A set containing currently marked items.
-    - attempts: A dictionary tracking the number of failure attempts per item.
-    """
-    marked.add(item)  
-    # Add the item to the set of marked problematic providers
-    attempts[item] = attempts.get(item, 0) + 1  
-    # Increment the failure attempt count for this item, initializing if necessary
-    if attempts[item] >= 3:
-        # If the item has failed 3 or more times, schedule removal from marked after 5 minutes
-        def remove():
-            marked.discard(item)  
-            # Remove the item from the marked set to allow retrying
-            attempts.pop(item, None)  
-            # Remove the attempt count entry for this item to reset its failure state
-        threading.Timer(300, remove).start()  
-        # Start a background timer that will call remove() after 300 seconds (5 minutes)
-
-def get_model_key(display, MODEL_MAPPING, DEFAULT_MODEL_KEY):
-    """
-    Translate a human-readable model display name into its internal model key identifier.
-    Searches the MODEL_MAPPING dictionary for the key whose value matches the display name.
-    Returns the DEFAULT_MODEL_KEY if no matching display name is found.
-    
-    Parameters:
-    - display: The display name of the model as a string.
-    - MODEL_MAPPING: Dictionary mapping internal model keys to display names.
-    - DEFAULT_MODEL_KEY: The fallback model key to return if no match is found.
-    
-    Returns:
-    - The internal model key string corresponding to the display name.
-    """
-    # Iterate through the MODEL_MAPPING dictionary items and return the key where the value matches the display name
-    return next((k for k, v in MODEL_MAPPING.items() if v == display), DEFAULT_MODEL_KEY)

src/main/file_extractors.py

@@ -1,393 +0,0 @@
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-import pdfplumber  # Library to extract text and tables from PDF files
-import pytesseract  # OCR tool to extract text from images
-import docx  # Library to read Microsoft Word (.docx) files
-import zipfile  # To handle zipped archives, used here to access embedded images in Word files
-import io  # Provides tools for handling byte streams, used to open images from bytes
-import pandas as pd  # Data analysis library, used here to handle tables from Excel and other files
-import warnings  # Used to suppress warnings during Excel file reading
-import re  # Regular expressions for text cleaning
-
-from openpyxl import load_workbook  # Excel file reading library, used for .xlsx files
-from pptx import Presentation  # Library to read Microsoft PowerPoint files
-from PIL import Image, ImageEnhance, ImageFilter  # Image processing libraries for OCR preprocessing
-from pathlib import Path  # Object-oriented filesystem paths
-
-def clean_text(text):
-    """
-    Clean and normalize extracted text to improve readability and remove noise.
-    
-    This function performs several cleaning steps:
-    - Removes characters that are not letters, digits, spaces, or common punctuation.
-    - Removes isolated single letters which are often OCR errors or noise.
-    - Strips whitespace from each line and removes empty lines.
-    - Joins cleaned lines back into a single string separated by newlines.
-    
-    Args:
-        text (str): Raw extracted text from any source.
-    
-    Returns:
-        str: Cleaned and normalized text ready for display or further processing.
-    """
-    # Remove all characters except letters, digits, spaces, and common punctuation marks
-    text = re.sub(r'[^a-zA-Z0-9\s.,?!():;\'"-]', '', text)
-    # Remove single isolated letters which are likely errors or noise from OCR
-    text = re.sub(r'\b[a-zA-Z]\b', '', text)
-    # Split text into lines, strip whitespace, and remove empty lines
-    lines = [line.strip() for line in text.splitlines() if line.strip()]
-    # Join cleaned lines with newline characters
-    return "\n".join(lines)
-
-def format_table(df, max_rows=10):
-    """
-    Convert a pandas DataFrame into a clean, readable string representation of a table.
-    
-    This function:
-    - Removes rows and columns that are completely empty to reduce clutter.
-    - Replaces any NaN values with empty strings for cleaner output.
-    - Limits the output to a maximum number of rows for brevity.
-    - Adds a note if there are more rows than displayed.
-    
-    Args:
-        df (pandas.DataFrame): The table data to format.
-        max_rows (int): Maximum number of rows to display from the table.
-    
-    Returns:
-        str: Formatted string representation of the table or empty string if no data.
-    """
-    if df.empty:
-        return ""
-    # Remove rows and columns where all values are NaN to clean the table
-    df_clean = df.dropna(axis=0, how='all').dropna(axis=1, how='all')
-    # Replace remaining NaN values with empty strings for better readability
-    df_clean = df_clean.fillna('')
-    if df_clean.empty:
-        return ""
-    # Select only the first max_rows rows for display
-    display_df = df_clean.head(max_rows)
-    # Convert DataFrame to string without row indices
-    table_str = display_df.to_string(index=False)
-    # Append a message if there are more rows than displayed
-    if len(df_clean) > max_rows:
-        table_str += f"\n... ({len(df_clean) - max_rows} more rows)"
-    return table_str
-
-def preprocess_image(img):
-    """
-    Enhance an image to improve OCR accuracy by applying several preprocessing steps.
-    
-    The preprocessing includes:
-    - Converting the image to grayscale to simplify colors.
-    - Increasing contrast to make text stand out more.
-    - Applying a median filter to reduce noise.
-    - Binarizing the image by thresholding to black and white.
-    
-    Args:
-        img (PIL.Image.Image): The original image to preprocess.
-    
-    Returns:
-        PIL.Image.Image: The processed image ready for OCR.
-        If an error occurs during processing, returns the original image.
-    """
-    try:
-        # Convert image to grayscale mode
-        img = img.convert("L")
-        # Enhance contrast by a factor of 2 to make text clearer
-        enhancer = ImageEnhance.Contrast(img)
-        img = enhancer.enhance(2)
-        # Apply median filter to reduce noise and smooth the image
-        img = img.filter(ImageFilter.MedianFilter())
-        # Convert image to black and white using a threshold of 140
-        img = img.point(lambda x: 0 if x < 140 else 255, '1')
-        return img
-    except Exception:
-        # In case of any error, return the original image without changes
-        return img
-
-def ocr_image(img):
-    """
-    Extract text from an image using OCR after preprocessing to improve results.
-    
-    This function:
-    - Preprocesses the image to enhance text visibility.
-    - Uses pytesseract with page segmentation mode 6 (assumes a single uniform block of text).
-    - Cleans the extracted text using the clean_text function.
-    
-    Args:
-        img (PIL.Image.Image): The image from which to extract text.
-    
-    Returns:
-        str: The cleaned OCR-extracted text. Returns empty string if OCR fails.
-    """
-    try:
-        # Preprocess image to improve OCR quality
-        img = preprocess_image(img)
-        # Perform OCR using pytesseract with English language and specified config
-        text = pytesseract.image_to_string(img, lang='eng', config='--psm 6')
-        # Clean the OCR output to remove noise and normalize text
-        text = clean_text(text)
-        return text
-    except Exception:
-        # Return empty string if OCR fails for any reason
-        return ""
-
-def extract_pdf_content(fp):
-    """
-    Extract text and tables from a PDF file, including OCR on embedded images.
-    
-    This function:
-    - Opens the PDF file and iterates through each page.
-    - Extracts and cleans text from each page.
-    - Performs OCR on images embedded in pages to extract any text within images.
-    - Extracts tables from pages and formats them as readable text.
-    - Handles exceptions by appending error messages to the content.
-    
-    Args:
-        fp (str or Path): File path to the PDF document.
-    
-    Returns:
-        str: Combined extracted text, OCR results, and formatted tables from the PDF.
-    """
-    content = ""
-    try:
-        with pdfplumber.open(fp) as pdf:
-            for i, page in enumerate(pdf.pages, 1):
-                # Extract text from the current page, defaulting to empty string if None
-                text = page.extract_text() or ""
-                # Clean extracted text and add page header
-                content += f"Page {i} Text:\n{clean_text(text)}\n\n"
-                # If there are images on the page, perform OCR on each
-                if page.images:
-                    # Create an image object of the page with 300 dpi resolution for cropping
-                    img_obj = page.to_image(resolution=300)
-                    for img in page.images:
-                        # Define bounding box coordinates for the image on the page
-                        bbox = (img["x0"], img["top"], img["x1"], img["bottom"])
-                        # Crop the image from the page image
-                        cropped = img_obj.original.crop(bbox)
-                        # Perform OCR on the cropped image
-                        ocr_text = ocr_image(cropped)
-                        if ocr_text:
-                            # Append OCR text with page and image reference
-                            content += f"[OCR Text from image on page {i}]:\n{ocr_text}\n\n"
-                # Extract tables from the page
-                tables = page.extract_tables()
-                for idx, table in enumerate(tables, 1):
-                    if table:
-                        # Convert table list to DataFrame using first row as header
-                        df = pd.DataFrame(table[1:], columns=table[0])
-                        # Format and append the table text
-                        content += f"Table {idx} on page {i}:\n{format_table(df)}\n\n"
-    except Exception as e:
-        # Append error message if PDF reading fails
-        content += f"\n[Error reading PDF {fp}: {e}]"
-    # Return the combined content with whitespace trimmed
-    return content.strip()
-
-def extract_docx_content(fp):
-    """
-    Extract text, tables, and OCR text from images embedded in a Microsoft Word (.docx) file.
-    
-    This function:
-    - Reads paragraphs and tables from the document.
-    - Cleans and formats extracted text and tables.
-    - Opens the .docx file as a zip archive to extract embedded images.
-    - Performs OCR on embedded images to extract any text they contain.
-    - Handles exceptions and appends error messages if reading fails.
-    
-    Args:
-        fp (str or Path): File path to the Word document.
-    
-    Returns:
-        str: Combined extracted paragraphs, tables, and OCR text from embedded images.
-    """
-    content = ""
-    try:
-        # Load the Word document
-        doc = docx.Document(fp)
-        # Extract and clean all non-empty paragraphs
-        paragraphs = [para.text.strip() for para in doc.paragraphs if para.text.strip()]
-        if paragraphs:
-            content += "Paragraphs:\n" + "\n".join(paragraphs) + "\n\n"
-        # Extract tables from the document
-        tables = []
-        for table in doc.tables:
-            rows = []
-            for row in table.rows:
-                # Extract and clean text from each cell in the row
-                cells = [cell.text.strip() for cell in row.cells]
-                rows.append(cells)
-            if rows:
-                # Convert rows to DataFrame using first row as header
-                df = pd.DataFrame(rows[1:], columns=rows[0])
-                tables.append(df)
-        # Format and append each extracted table
-        for i, df in enumerate(tables, 1):
-            content += f"Table {i}:\n{format_table(df)}\n\n"
-        # Open the .docx file as a zip archive to access embedded media files
-        with zipfile.ZipFile(fp) as z:
-            for file in z.namelist():
-                # Look for images inside the word/media directory
-                if file.startswith("word/media/"):
-                    data = z.read(file)
-                    try:
-                        # Open image from bytes
-                        img = Image.open(io.BytesIO(data))
-                        # Perform OCR on the image
-                        ocr_text = ocr_image(img)
-                        if ocr_text:
-                            # Append OCR text extracted from embedded image
-                            content += f"[OCR Text from embedded image]:\n{ocr_text}\n\n"
-                    except Exception:
-                        # Ignore errors in image processing to continue extraction
-                        pass
-    except Exception as e:
-        # Append error message if Word document reading fails
-        content += f"\n[Error reading Microsoft Word {fp}: {e}]"
-    # Return combined content trimmed of extra whitespace
-    return content.strip()
-
-def extract_excel_content(fp):
-    """
-    Extract readable table content from Microsoft Excel files (.xlsx, .xls).
-    
-    This function:
-    - Reads all sheets in the Excel file.
-    - Converts each sheet to a formatted table string.
-    - Suppresses warnings during reading to avoid clutter.
-    - Does not attempt to extract images to avoid errors.
-    - Handles exceptions by appending error messages.
-    
-    Args:
-        fp (str or Path): File path to the Excel workbook.
-    
-    Returns:
-        str: Combined formatted tables from all sheets in the workbook.
-    """
-    content = ""
-    try:
-        # Suppress warnings such as openpyxl deprecation or data type warnings
-        with warnings.catch_warnings():
-            warnings.simplefilter("ignore")
-            # Read all sheets into a dictionary of DataFrames using openpyxl engine
-            sheets = pd.read_excel(fp, sheet_name=None, engine='openpyxl')
-        # Iterate over each sheet and format its content
-        for sheet_name, df in sheets.items():
-            content += f"Sheet: {sheet_name}\n"
-            content += format_table(df) + "\n\n"
-    except Exception as e:
-        # Append error message if Excel reading fails
-        content += f"\n[Error reading Microsoft Excel {fp}: {e}]"
-    # Return combined sheet contents trimmed of whitespace
-    return content.strip()
-
-def extract_pptx_content(fp):
-    """
-    Extract text, tables, and OCR text from images in Microsoft PowerPoint (.pptx) files.
-    
-    This function:
-    - Reads each slide in the presentation.
-    - Extracts text from shapes and tables on each slide.
-    - Performs OCR on images embedded in shapes.
-    - Handles exceptions and appends error messages if reading fails.
-    
-    Args:
-        fp (str or Path): File path to the PowerPoint presentation.
-    
-    Returns:
-        str: Combined extracted text, tables, and OCR results from all slides.
-    """
-    content = ""
-    try:
-        # Load the PowerPoint presentation
-        prs = Presentation(fp)
-        # Iterate through each slide by index starting at 1
-        for i, slide in enumerate(prs.slides, 1):
-            slide_texts = []
-            # Iterate through all shapes on the slide
-            for shape in slide.shapes:
-                # Extract and clean text from shapes that have text attribute
-                if hasattr(shape, "text") and shape.text.strip():
-                    slide_texts.append(shape.text.strip())
-                # Check if the shape is a picture (shape_type 13) with an image
-                if shape.shape_type == 13 and hasattr(shape, "image") and shape.image:
-                    try:
-                        # Open image from the shape's binary blob data
-                        img = Image.open(io.BytesIO(shape.image.blob))
-                        # Perform OCR on the image
-                        ocr_text = ocr_image(img)
-                        if ocr_text:
-                            # Append OCR text extracted from the image
-                            slide_texts.append(f"[OCR Text from image]:\n{ocr_text}")
-                    except Exception:
-                        # Ignore errors in image OCR to continue processing
-                        pass
-            # Add slide text or note if no text found
-            if slide_texts:
-                content += f"Slide {i} Text:\n" + "\n".join(slide_texts) + "\n\n"
-            else:
-                content += f"Slide {i} Text:\nNo text found on this slide.\n\n"
-            # Extract tables from shapes that have tables
-            for shape in slide.shapes:
-                if shape.has_table:
-                    rows = []
-                    table = shape.table
-                    # Extract text from each cell in the table rows
-                    for row in table.rows:
-                        cells = [cell.text.strip() for cell in row.cells]
-                        rows.append(cells)
-                    if rows:
-                        # Convert rows to DataFrame using first row as header
-                        df = pd.DataFrame(rows[1:], columns=rows[0])
-                        # Format and append the table text
-                        content += f"Table on slide {i}:\n{format_table(df)}\n\n"
-    except Exception as e:
-        # Append error message if PowerPoint reading fails
-        content += f"\n[Error reading Microsoft PowerPoint {fp}: {e}]"
-    # Return combined slide content trimmed of whitespace
-    return content.strip()
-
-def extract_file_content(fp):
-    """
-    Determine the file type based on its extension and extract text content accordingly.
-    
-    This function supports:
-    - PDF files with text, tables, and OCR on images.
-    - Microsoft Word documents with paragraphs, tables, and OCR on embedded images.
-    - Microsoft Excel workbooks with formatted sheet tables.
-    - Microsoft PowerPoint presentations with slide text, tables, and OCR on images.
-    - Other file types are attempted to be read as plain UTF-8 text.
-    
-    Args:
-        fp (str or Path): File path to the document to extract content from.
-    
-    Returns:
-        str: Extracted and cleaned text content from the file, or an error message.
-    """
-    # Get the file extension in lowercase to identify file type
-    ext = Path(fp).suffix.lower()
-    if ext == ".pdf":
-        # Extract content from PDF files
-        return extract_pdf_content(fp)
-    elif ext in [".doc", ".docx"]:
-        # Extract content from Word documents
-        return extract_docx_content(fp)
-    elif ext in [".xlsx", ".xls"]:
-        # Extract content from Excel workbooks
-        return extract_excel_content(fp)
-    elif ext in [".ppt", ".pptx"]:
-        # Extract content from PowerPoint presentations
-        return extract_pptx_content(fp)
-    else:
-        try:
-            # Attempt to read unknown file types as plain UTF-8 text
-            text = Path(fp).read_text(encoding="utf-8")
-            # Clean the extracted text before returning
-            return clean_text(text)
-        except Exception as e:
-            # Return error message if reading fails
-            return f"\n[Error reading file {fp}: {e}]"

src/main/gradio.py

@@ -1,332 +0,0 @@
-#
-# SPDX-FileCopyrightText: Hadad <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
-#
-
-import gradio as gr  # Import Gradio library for building the web UI
-import asyncio  # Import asyncio for asynchronous programming
-
-from pathlib import Path  # Import Path for filesystem path manipulations
-from config import *  # Import all configuration constants and variables
-from src.cores.session import create_session, ensure_stop_event, get_model_key  # Import session management utilities
-from src.main.file_extractors import extract_file_content  # Import function to extract content from uploaded files
-from src.cores.client import chat_with_model_async  # Import async chat function with AI model
-
-async def respond_async(multi, history, model_display, sess, custom_prompt, deep_search):
-    """
-    Asynchronous handler for processing user input submissions.
-    Supports multi-modal input including text and file uploads.
-    Extracts content from uploaded files and appends it to user text input.
-    Streams AI-generated responses back to the UI, updating chat history live.
-    Allows graceful stopping of response generation upon user request.
-    
-    Parameters:
-    - multi: dict containing user text input and uploaded files
-    - history: list of previous chat messages (user and AI)
-    - model_display: selected AI model identifier
-    - sess: current session object managing state and cancellation
-    - custom_prompt: user-defined system instructions
-    - deep_search: boolean flag to enable extended search capabilities
-    
-    Yields:
-    - Updated chat history and UI state for real-time interaction
-    """
-    ensure_stop_event(sess)  # Ensure the session has a stop event initialized
-    sess.stop_event.clear()  # Clear any previous stop signals
-    sess.cancel_token["cancelled"] = False  # Reset cancellation flag
-    
-    # Extract text and files from multimodal input dictionary
-    msg_input = {"text": multi.get("text", "").strip(), "files": multi.get("files", [])}
-    
-    # If no input text or files, reset UI input and return early
-    if not msg_input["text"] and not msg_input["files"]:
-        yield history, gr.update(value="", interactive=True, submit_btn=True, stop_btn=False), sess
-        return
-    
-    # Initialize combined input string with extracted file contents
-    inp = ""
-    for f in msg_input["files"]:
-        # Support both dict format or direct file path string
-        fp = f.get("data", f.get("name", "")) if isinstance(f, dict) else f
-        # Append extracted file content with spacing
-        inp += f"```\n{extract_file_content(fp)}\n``` \n\n\n"
-    
-    # Append user text input if present
-    if msg_input["text"]:
-        inp += msg_input["text"]
-    
-    # Append user input to chat history
-    history.append([inp, ""])  # placeholder
-    
-    # Yield updated history and disable input while AI is responding
-    yield history, gr.update(interactive=False, submit_btn=False, stop_btn=True), sess
-
-    # Create queue for streaming AI response chunks
-    queue = asyncio.Queue()
-
-    async def background():
-        """
-        This coroutine handles streaming responses from an AI model asynchronously.
-        It processes two types of streamed data separately: 'reasoning' chunks and 'content' chunks.
-        The function supports graceful cancellation if a stop event or cancel token is triggered in the session.
-
-        Reasoning text is accumulated until content streaming starts, after which reasoning is ignored.
-        Special tags <think> and </think> are managed to mark reasoning sections for UI display.
-        Content chunks are streamed and accumulated separately, with incremental UI updates.
-
-        When streaming ends, any open reasoning tags are closed properly.
-        Finally, the function signals completion by putting None into the queue and returns the full content response.
-        """
-        reasoning = ""  # String to accumulate reasoning text chunks
-        responses = ""  # String to accumulate content text chunks
-        content_started = False  # Flag to indicate if content streaming has begun
-        ignore_reasoning = False  # Flag to ignore reasoning after content starts streaming
-        think_opened = False  # Flag to track if reasoning <think> tag has been sent
-
-        # Asynchronously iterate over streamed response chunks from the AI model
-        async for typ, chunk in chat_with_model_async(history, inp, model_display, sess, custom_prompt, deep_search):
-            # Break the loop if user requested stop or cancellation is flagged
-            if sess.stop_event.is_set() or sess.cancel_token["cancelled"]:
-                break
-
-            if typ == "reasoning":
-                # Append reasoning chunk unless ignoring reasoning after content started
-                if ignore_reasoning:
-                    continue
-                # Handle opening <think> tag for reasoning
-                if chunk.strip() == "<think>":
-                    if not think_opened:
-                        think_opened = True  # Mark that reasoning tag has been opened
-                    continue  # Skip sending the tag itself to UI
-                if not think_opened:
-                    # If reasoning tag not yet opened, prepend it and mark as opened
-                    reasoning += "<think>\n" + chunk
-                    think_opened = True
-                else:
-                    # Append reasoning chunk normally
-                    reasoning += chunk
-                # Send current reasoning content to queue for UI update (without sending tag again)
-                await queue.put(("reasoning", reasoning))
-
-            elif typ == "content":
-                if not content_started:
-                    # On first content chunk, mark content started and ignore further reasoning
-                    content_started = True
-                    ignore_reasoning = True
-                    if think_opened:
-                        # Close reasoning tag before sending content
-                        reasoning += "\n</think>\n\n"
-                        await queue.put(("reasoning", reasoning))  # Update UI with closed reasoning
-                    else:
-                        # No reasoning was sent, clear reasoning display in UI
-                        await queue.put(("reasoning", ""))
-                    # Start accumulating content and send initial content to UI replacing placeholder
-                    responses = chunk
-                    await queue.put(("replace", responses))
-                else:
-                    # Append subsequent content chunks and update UI incrementally
-                    responses += chunk
-                    await queue.put(("append", responses))
-
-        # If stream ends without content, close reasoning tag if it was opened
-        if think_opened and not content_started:
-            reasoning += "\n</think>\n\n"
-            await queue.put(("reasoning", reasoning))
-
-        # Signal completion of streaming by putting None into the queue
-        await queue.put(None)
-        # Return the full accumulated content response
-        return responses
-    
-    bg_task = asyncio.create_task(background())  # Start background streaming task
-    stop_task = asyncio.create_task(sess.stop_event.wait())  # Task to wait for stop event
-    pending_tasks = {bg_task, stop_task}  # Track pending async tasks
-    
-    try:
-        while True:
-            queue_task = asyncio.create_task(queue.get())  # Task to get next queued update
-            pending_tasks.add(queue_task)
-            
-            # Wait for either stop event or new queue item
-            done, _ = await asyncio.wait({stop_task, queue_task}, return_when=asyncio.FIRST_COMPLETED)
-            
-            for task in done:
-                pending_tasks.discard(task)
-                
-                if task is stop_task:
-                    # User requested stop, cancel background task and update UI accordingly
-                    sess.cancel_token["cancelled"] = True
-                    bg_task.cancel()
-                    try:
-                        await bg_task
-                    except asyncio.CancelledError:
-                        pass
-                    # Update last message with cancellation notice
-                    history[-1][1] = RESPONSES["RESPONSE_1"]
-                    yield history, gr.update(value="", interactive=True, submit_btn=True, stop_btn=False), sess
-                    return
-                
-                result = task.result()
-                if result is None:
-                    # Streaming finished, stop iteration
-                    raise StopAsyncIteration
-                
-                action, text = result
-                # Update last message content in history with streamed text chunk
-                history[-1][1] = text
-                # Yield updated history and UI state to refresh chat display
-                yield history, gr.update(interactive=False, submit_btn=False, stop_btn=True), sess
-    
-    except StopAsyncIteration:
-        # Normal completion of streaming
-        pass
-    
-    finally:
-        # Cancel any remaining pending tasks to clean up
-        for task in pending_tasks:
-            task.cancel()
-        await asyncio.gather(*pending_tasks, return_exceptions=True)
-    
-    # After completion, reset UI input to ready state
-    yield history, gr.update(value="", interactive=True, submit_btn=True, stop_btn=False), sess
-
-def toggle_deep_search(deep_search_value, history, sess, prompt, model):
-    """
-    Toggle the deep search checkbox state.
-    Maintains current chat history and session for production use.
-    
-    Parameters:
-    - deep_search_value: new checkbox boolean value
-    - history: current chat history
-    - sess: current session object
-    - prompt: current system instructions
-    - model: currently selected model
-    
-    Returns:
-    - Unchanged history, session, prompt, model
-    - Updated deep search checkbox UI state
-    """
-    return history, sess, prompt, model, gr.update(value=deep_search_value)
-
-def change_model(new):
-    """
-    Handler to change the selected AI model.
-    Resets chat history and creates a new session.
-    Updates system instructions and deep search checkbox visibility.
-    Deep search is only enabled for the default model.
-    
-    Parameters:
-    - new: newly selected model identifier
-    
-    Returns:
-    - Empty chat history list
-    - New session object
-    - New model identifier
-    - Corresponding system instructions string
-    - Deep search checkbox reset to False
-    - UI update for deep search checkbox visibility
-    """
-    visible = new == MODEL_CHOICES[0]  # Deep search visible only for default model
-    
-    # Get system instructions for new model or fallback to default instructions
-    default_prompt = SYSTEM_PROMPT_MAPPING.get(get_model_key(new, MODEL_MAPPING, DEFAULT_MODEL_KEY), SYSTEM_PROMPT_DEFAULT)
-    
-    # Clear chat, create new session, reset deep search, update UI visibility
-    return [], create_session(), new, default_prompt, False, gr.update(visible=visible)
-
-def stop_response(history, sess):
-    """
-    Handler to stop ongoing AI response generation.
-    Sets cancellation flags and updates the last message to a cancellation notice.
-    
-    Parameters:
-    - history: current chat history list
-    - sess: current session object
-    
-    Returns:
-    - Updated chat history with cancellation message
-    - None for input box reset
-    - New session object for fresh state
-    """
-    ensure_stop_event(sess)  # Ensure stop event exists in session
-    sess.stop_event.set()  # Signal stop event to cancel ongoing tasks
-    sess.cancel_token["cancelled"] = True  # Mark cancellation flag
-    
-    if history:
-        # Replace last AI response with cancellation message
-        history[-1][1] = RESPONSES["RESPONSE_1"]
-    
-    return history, None, create_session()
-
-def launch_ui():
-    """
-    Launch the Gradio UI for the chatbot application.
-    Sets up the UI components, event handlers, and starts the server.
-    Installs required OCR dependencies for file content extraction.
-    """
-    # ============================
-    # System Setup
-    # ============================
-    
-    # Install Tesseract OCR and dependencies for extracting text from images
-    import os
-    os.system("apt-get update -q -y && \
-               apt-get install -q -y tesseract-ocr \
-               tesseract-ocr-eng tesseract-ocr-ind \
-               libleptonica-dev libtesseract-dev"
-    )
-    
-    # Create Gradio Blocks container for full UI layout
-    with gr.Blocks(fill_height=True, fill_width=True, title=AI_TYPES["AI_TYPE_4"], head=META_TAGS) as jarvis:
-        # State variables to hold chat history, session, selected model, and instructions
-        user_history = gr.State([])
-        user_session = gr.State(create_session())
-        selected_model = gr.State(MODEL_CHOICES[0] if MODEL_CHOICES else "")
-        J_A_R_V_I_S = gr.State("")
-        
-        # Chatbot UI
-        with gr.Column():
-            chatbot = gr.Chatbot(label=AI_TYPES["AI_TYPE_1"], show_copy_button=True, scale=1, elem_id=AI_TYPES["AI_TYPE_2"], examples=JARVIS_INIT, allow_tags=["think", "thinking"])
-        
-        # User input
-        msg = gr.MultimodalTextbox(show_label=False, placeholder=RESPONSES["RESPONSE_5"], interactive=True, file_count=None, file_types=None, sources=[])
-        
-        # Sidebar on left for model selection and deep search toggle
-        with gr.Sidebar(open=False):
-            deep_search = gr.Checkbox(label=AI_TYPES["AI_TYPE_8"], value=False, info=AI_TYPES["AI_TYPE_9"], visible=True)
-            # When deep search checkbox changes, call toggle_deep_search handler
-            deep_search.change(fn=toggle_deep_search, inputs=[deep_search, user_history, user_session, J_A_R_V_I_S, selected_model], outputs=[chatbot, user_session, J_A_R_V_I_S, selected_model, deep_search])
-            gr.Markdown()  # Add spacing line
-            model_radio = gr.Radio(show_label=False, choices=MODEL_CHOICES, value=MODEL_CHOICES[0])
-        
-        # Sidebar on right for notices and additional information
-        with gr.Sidebar(open=False, position="right"):
-            gr.Markdown(NOTICES)
-        
-        # When model selection changes, call change_model handler
-        model_radio.change(fn=change_model, inputs=[model_radio], outputs=[user_history, user_session, selected_model, J_A_R_V_I_S, deep_search, deep_search])
-        
-        # Event handler for selecting example messages in chatbot UI
-        def on_example_select(evt: gr.SelectData):
-            return evt.value
-        
-        chatbot.example_select(fn=on_example_select, inputs=[], outputs=[msg]).then(
-            fn=respond_async,
-            inputs=[msg, user_history, selected_model, user_session, J_A_R_V_I_S, deep_search],
-            outputs=[chatbot, msg, user_session]
-        )
-        
-        # Clear chat button handler resets chat, session, instructions, model, and history
-        def clear_chat(history, sess, prompt, model):
-            return [], create_session(), prompt, model, []
-        
-        chatbot.clear(fn=clear_chat, inputs=[user_history, user_session, J_A_R_V_I_S, selected_model], outputs=[chatbot, user_session, J_A_R_V_I_S, selected_model, user_history])
-        
-        # Submit user message triggers respond_async to generate AI response
-        msg.submit(fn=respond_async, inputs=[msg, user_history, selected_model, user_session, J_A_R_V_I_S, deep_search], outputs=[chatbot, msg, user_session], api_name=INTERNAL_AI_GET_SERVER)
-        
-        # Stop button triggers stop_response handler to cancel ongoing AI generation
-        msg.stop(fn=stop_response, inputs=[user_history, user_session], outputs=[chatbot, msg, user_session])
-    
-    # Launch
-    jarvis.queue(default_concurrency_limit=2).launch(max_file_size="1mb", mcp_server=True)

src/tools/audio.py

@@ -0,0 +1,68 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import asyncio  # Import asyncio to enable asynchronous waiting and retries
+import httpx  # Import the httpx library to perform asynchronous HTTP requests efficiently
+from urllib.parse import quote  # Import the quote function to safely encode strings for use in URLs
+from src.utils.ip_generator import generate_ip  # Import a custom utility function to generate random IP addresses
+from config import auth  # Import authentication configuration or credentials from the config module
+from src.utils.tools import initialize_tools  # Import a utility function to initialize and retrieve tool endpoints or resources
+
+# Define a class named AudioGeneration to encapsulate functionalities related to generating audio content
+class AudioGeneration:
+    # This class provides methods to create audio files based on text instructions and voice parameters
+
+    @staticmethod  # Decorator indicating that the following method does not depend on instance state and can be called on the class itself
+    # Define an asynchronous method to create audio from a text instruction, optionally specifying a voice style
+    async def create_audio(generate_audio_instruction: str, voice: str = "echo") -> str:
+        """
+        Generate an audio file URL by sending a request to an audio generation service.
+        This method will keep retrying until a successful response with status code 200 and audio content is received.
+
+        Args:
+            generate_audio_instruction (str): The textual instruction or content to convert into audio.
+            voice (str, optional): The voice style or effect to apply on the generated audio. Defaults to "echo".
+
+        Returns:
+            str: The URL to the generated audio file if successful.
+
+        Raises:
+            Exception: If the audio generation continuously fails after retries (optional, currently infinite retry).
+        """
+        # Encode the text instruction to make it safe for inclusion in a URL path segment
+        generate_audio_instruct = quote(generate_audio_instruction)
+
+        # Initialize tools and retrieve the audio generation service endpoint from the returned tuple
+        _, _, audio_tool = initialize_tools()
+
+        # Construct the full URL by appending the encoded instruction to the audio tool's base URL
+        url = f"{audio_tool}/{generate_audio_instruct}"
+
+        # Define query parameters for the HTTP request specifying the model and voice to use for audio generation
+        params = {
+            "model": "openai-audio",  # Specify the audio generation model to be used by the service
+            "voice": voice            # Specify the desired voice style or effect
+        }
+
+        # Create an asynchronous HTTP client with no timeout limit to perform the request
+        async with httpx.AsyncClient(timeout=None) as client:
+            # Enter an infinite loop to keep retrying the request until success criteria are met
+            while True:
+                # Define HTTP headers for the request, including random IP address to simulate different client origins
+                headers = {
+                    "X-Forwarded-For": generate_ip()  # Generate and set a random IP address for the request header
+                }
+
+                # Send a GET request to the audio generation service with specified URL, parameters, and headers
+                resp = await client.get(url, params=params, headers=headers)
+
+                # Check if the response status code indicates success and the content type is an audio MPEG stream
+                if resp.status_code == 200 and 'audio/mpeg' in resp.headers.get('Content-Type', ''):
+                    # Return the final URL of the generated audio resource as a string
+                    return str(resp.url)
+                else:
+                    # If the response is not successful, wait for a short delay before retrying to avoid hammering the server
+                    await asyncio.sleep(15)  # Pause for 15 second before retrying the request
+                    # The loop will continue and try again without closing the connection prematurely

src/tools/deep_search.py

@@ -0,0 +1,98 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import requests  # Import the requests library to perform HTTP requests synchronously
+from src.utils.ip_generator import generate_ip  # Import function to generate random IP addresses for request headers
+
+# Define a class named SearchTools to encapsulate functionalities related to deep search
+class SearchTools:
+    # This class provides methods to connect to the web
+
+    """
+    A class providing tools to perform web searches and read content from URLs using various search engines
+    and a reader API service.
+
+    Attributes:
+        searxng_url (str): Base URL for the SearXNG search proxy service.
+        baidu_url (str): Base URL for Baidu search engine.
+        timeout (int): Timeout duration in seconds for HTTP requests.
+        reader_api (str): Base URL for the reader API service used to extract content from URLs.
+
+    Methods:
+        read_url(url): Asynchronously reads and returns the textual content of the specified URL using the reader API.
+        search(query, engine): Asynchronously performs a web search with the given query on the specified search engine,
+                               returning the raw HTML response text.
+    """
+
+    def __init__(self):
+        """
+        Initialize the SearchTools instance with predefined URLs and timeout settings.
+        """
+        self.searxng_url = "https://paulgo.io/search"  # URL for the SearXNG search proxy service
+        self.baidu_url = "https://www.baidu.com/s"  # URL for Baidu search engine
+        self.timeout = 30  # Timeout in seconds for HTTP requests to avoid long hanging connections
+        self.reader_api = "https://r.jina.ai/"  # Reader API endpoint to extract readable content from URLs
+
+    async def read_url(self, url: str) -> str:
+        """
+        Asynchronously read and retrieve the textual content of a given URL using the reader API.
+
+        Args:
+            url (str): The URL of the webpage to read content from.
+
+        Returns:
+            str: The textual content extracted from the URL if successful.
+            None: If the request fails or an exception occurs.
+        """
+        try:
+            data = {"url": url}  # Prepare POST data with the target URL
+            # Send a synchronous POST request to the reader API with the URL data and timeout
+            response = requests.post(self.reader_api, data=data, timeout=self.timeout)
+            response.raise_for_status()  # Raise an exception if the response status is an HTTP error
+            return response.text  # Return the textual content of the response
+        except Exception:
+            # Return None if any error occurs during the request or response processing
+            return None
+
+    async def search(self, query: str, engine: str = "google") -> str:
+        """
+        Asynchronously perform a web search for the given query using the specified search engine.
+
+        Args:
+            query (str): The search query string.
+            engine (str, optional): The search engine to use. Supported values are "google" and "baidu".
+                                    Defaults to "google".
+
+        Returns:
+            str: The raw HTML content of the search results page if successful.
+            None: If the request fails or an exception occurs.
+        """
+        try:
+            if engine == "baidu":
+                # Construct the URL for Baidu search by appending the query parameter 'wd' with the search term
+                url = f"{self.reader_api}{self.baidu_url}?wd={query}"
+                # Set the HTTP header to target the main content container of Baidu search results
+                headers = {
+                    "X-Target-Selector": "#content_left",
+                    "X-Forwarded-For": generate_ip()  # Random IP address to simulate different client origins
+                }
+            else:
+                # For Google or other engines, define a prefix for the search command (!go for Google, !bi for Bing)
+                prefix = "!go" if engine == "google" else "!bi"
+                # Construct the URL for SearXNG search proxy with the prefixed query
+                url = f"{self.reader_api}{self.searxng_url}?q={prefix} {query}"
+                # Set the HTTP header to target the URLs container in the search results
+                headers = {
+                    "X-Target-Selector": "#urls",
+                    "X-Forwarded-For": generate_ip()  # Random IP address to simulate different client origins
+                }
+
+            # Send a synchronous GET request to the constructed URL with headers and timeout
+            response = requests.get(url, headers=headers, timeout=self.timeout)
+            response.raise_for_status()  # Raise an exception if the response status is an HTTP error
+            return response.text  # Return the raw HTML content of the search results
+        except Exception:
+            # Return None if any error occurs during the request or response processing
+            return None

src/tools/image.py

@@ -0,0 +1,116 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import httpx  # Import httpx library for performing asynchronous HTTP requests efficiently
+from urllib.parse import quote  # Import quote function to safely encode strings for use in URLs
+from typing import Optional  # Import Optional type hint for parameters that can be None
+from src.utils.ip_generator import generate_ip  # Import custom utility to generate random IP addresses for request headers
+from src.utils.tools import initialize_tools  # Import utility function to initialize and retrieve tool endpoints
+
+# Define a class named ImageGeneration to encapsulate functionalities related to generating image content
+class ImageGeneration:
+    # This class provides methods to create image files based on text instructions
+
+    """
+    A class to handle image generation requests to an external image generation service.
+
+    Attributes:
+        FORMATS (dict): A dictionary mapping image format names to their (width, height) dimensions.
+
+    Methods:
+        create_image: Asynchronously generates an image based on a textual instruction and parameters,
+                      returning the URL of the generated image.
+    """
+
+    # Image formats
+    FORMATS = {
+        "default": (1024, 1024),  # Default square image size (width x height)
+        "square": (1024, 1024),  # Square image format with equal width and height
+        "landscape": (1024, 768),  # Landscape format with wider width than height
+        "landscape_large": (1440, 1024),  # Larger landscape format with increased resolution
+        "portrait": (768, 1024),  # Portrait format with taller height than width
+        "portrait_large": (1024, 1440),  # Larger portrait format with increased resolution
+    }
+
+    @staticmethod  # Decorator indicating that the following method does not depend on instance state and can be called on the class itself
+    # Define an asynchronous method to create image from a text instruction
+    async def create_image(
+        generate_image_instruction: str,  # Text instruction describing the image to generate
+        image_format: str = "default",  # Desired image format key from FORMATS dictionary
+        model: Optional[str] = "flux-realism",  # Optional model name for image generation; defaults to 'flux-realism'
+        seed: Optional[int] = None,  # Optional seed value for randomization control in image generation
+        nologo: bool = True,  # Whether to generate image without logo watermark; defaults to True
+        private: bool = True,  # Whether the generated image should be private; defaults to True
+        enhance: bool = True,  # Whether to apply enhancement filters to the generated image; defaults to True
+    ) -> str:
+        """
+        Asynchronously generate an image URL by sending a request to the image generation service.
+        This method will keep retrying until a successful response with status code 200 is received.
+
+        Args:
+            generate_image_instruction (str): The textual instruction or description for the desired image.
+            image_format (str, optional): The format key specifying image dimensions. Defaults to "default".
+            model (Optional[str], optional): The image generation model to use. Defaults to "flux-realism".
+            seed (Optional[int], optional): Seed for randomization to reproduce images. Defaults to None.
+            nologo (bool, optional): Flag to exclude logo watermark. Defaults to True.
+            private (bool, optional): Flag to mark image as private. Defaults to True.
+            enhance (bool, optional): Flag to apply image enhancement. Defaults to True.
+
+        Returns:
+            str: The URL of the generated image if the request is successful.
+
+        Raises:
+            ValueError: If the specified image_format is not supported.
+            Exception: If the image generation continuously fails (currently infinite retry).
+        """
+        # Validate that the requested image format exists in the FORMATS dictionary
+        if image_format not in ImageGeneration.FORMATS:
+            raise ValueError("Invalid image format.")
+
+        # Retrieve width and height based on the requested image format
+        width, height = ImageGeneration.FORMATS[image_format]
+
+        # Initialize tools and retrieve the image generation service endpoint
+        _, image_tool, _ = initialize_tools()
+
+        # Encode the image instruction to safely include it in the URL path
+        generate_image_instruct = quote(generate_image_instruction)
+
+        # Construct the full URL for the image generation request by appending the encoded instruction
+        url = f"{image_tool}{generate_image_instruct}"  # Full endpoint URL for image generation
+
+        # Prepare query parameters including image dimensions, model, and flags converted to string "true"/"false"
+        params = {
+            "width": width,  # Image width parameter
+            "height": height,  # Image height parameter
+            "model": model,  # Model name for image generation
+            "nologo": "true" if nologo else "false",  # Flag to exclude logo watermark as string
+            "private": "true" if private else "false",  # Flag to mark image as private as string
+            "enhance": "true" if enhance else "false"  # Flag to apply enhancement as string
+        }
+
+        # Include seed parameter if provided to control randomness in image generation
+        if seed is not None:
+            params["seed"] = seed  # Add seed to parameters to reproduce images
+
+        # Prepare HTTP headers with a generated random IP to simulate different client origins
+        headers = {
+            "X-Forwarded-For": generate_ip()  # Random IP address for request header to simulate client origin
+        }
+
+        # Create an asynchronous HTTP client with no timeout limit to perform the request
+        async with httpx.AsyncClient(timeout=None) as client:
+            # Keep retrying the request until a successful response with status 200 is received
+            while True:
+                # Send a GET request to the image generation service with URL, parameters, and headers
+                resp = await client.get(url, params=params, headers=headers)
+
+                # Check if the response status code indicates success
+                if resp.status_code == 200:
+                    # Return the URL of the generated image as a string
+                    return str(resp.url)
+                else:
+                    # Wait briefly before retrying to avoid overwhelming the server
+                    await asyncio.sleep(15)  # Pause 15 second before retrying

src/ui/interface.py

@@ -0,0 +1,184 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import gradio as gr  # Import the Gradio library to build interactive web interfaces for machine learning applications
+from src.core.parameter import parameters  # Import the 'parameters' function from the core parameter module, which returns model parameter settings based on reasoning mode
+from src.client.chat_handler import respond  # Import the 'respond' function from the chat handler module, responsible for generating AI assistant responses
+from config import model, meta_tags  # Import 'model' dictionary containing available model precision options and their details, and 'meta_tags' containing HTML meta tag data
+
+# Gradio
+def ui():
+    """
+    Constructs the Gradio user interface for the J.A.R.V.I.S. AI assistant application.
+
+    This function sets up a web app with a sidebar for configuring model parameters and a main chat interface
+    for user interaction. It returns the Gradio Blocks object representing the entire app.
+    """
+    # Create a Gradio Blocks container that fills the entire available height and width of the browser window
+    with gr.Blocks(fill_height=True, fill_width=True, head=meta_tags) as app:
+        # Create a sidebar panel on the left side, initially closed, to hold model configuration controls
+        with gr.Sidebar(open=False):
+            # Dropdown menu for selecting the model precision from the keys of the 'model' dictionary
+            model_precision = gr.Dropdown(
+                choices=list(model.keys()),  # List of available model precision options, e.g., "F16", "F32"
+                label="Model Precision",  # Label displayed above the dropdown menu
+                info=(
+                    # Tooltip explaining the tradeoff between speed and accuracy based on precision choice
+                    "The smaller the value, the faster the response but less accurate. "
+                    "Conversely, the larger the value, the response is slower but more accurate."
+                ),
+                value="F16"  # Default selected precision value
+            )
+
+            # Checkbox to enable or disable reasoning mode, which toggles the AI's "thinking" capability
+            reasoning = gr.Checkbox(
+                label="Reasoning",  # Label shown next to the checkbox
+                info="Switching between thinking and non-thinking mode.",  # Tooltip describing the feature
+                value=True  # Default state is enabled (checked)
+            )
+
+            # Slider controlling the 'Temperature' parameter, affecting randomness in AI responses, initially non-interactive
+            temperature = gr.Slider(
+                minimum=0.0,  # Minimum slider value
+                maximum=2.0,  # Maximum slider value
+                step=0.01,  # Increment step size
+                label="Temperature",  # Label for the slider
+                interactive=False  # User cannot directly adjust this slider, updated dynamically
+            )
+
+            # Slider controlling the 'Top K' parameter, which limits the number of highest probability tokens considered, non-interactive initially
+            top_k = gr.Slider(
+                minimum=0,
+                maximum=100,
+                step=1,
+                label="Top K",
+                interactive=False
+            )
+
+            # Slider for 'Min P' parameter, representing minimum cumulative probability threshold, non-interactive initially
+            min_p = gr.Slider(
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                label="Min P",
+                interactive=False
+            )
+
+            # Slider for 'Top P' parameter, controlling nucleus sampling probability, non-interactive initially
+            top_p = gr.Slider(
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                label="Top P",
+                interactive=False
+            )
+
+            # Slider for 'Repetition Penalty' parameter to reduce repetitive text generation, non-interactive initially
+            repetition_penalty = gr.Slider(
+                minimum=0.1,
+                maximum=2.0,
+                step=0.01,
+                label="Repetition Penalty",
+                interactive=False
+            )
+
+            # Define a function to update the model parameter sliders based on the reasoning checkbox state
+            def update_parameters(switching):
+                """
+                Retrieve updated model parameter values based on reasoning mode.
+
+                Args:
+                    switching (bool): Current state of the reasoning checkbox.
+
+                Returns:
+                    tuple: Updated values for temperature, top_k, min_p, top_p, and repetition_penalty sliders.
+                """
+                # Call the 'parameters' function passing the reasoning state to get new parameter values
+                return parameters(switching)
+
+            # Set up an event listener to update parameter sliders when the reasoning checkbox state changes
+            reasoning.change(
+                fn=update_parameters,  # Function to call on checkbox state change
+                inputs=[reasoning],  # Input is the reasoning checkbox's current value
+                outputs=[temperature, top_k, min_p, top_p, repetition_penalty]  # Update these sliders with new values
+            )
+
+            # Initialize the parameter sliders with values corresponding to the default reasoning checkbox state
+            values = parameters(reasoning.value)
+            temperature.value, top_k.value, min_p.value, top_p.value, repetition_penalty.value = values
+
+            # Checkbox to enable or disable the image generation feature in the chat interface
+            image_generation = gr.Checkbox(
+                label="Image Generation",  # Label displayed next to the checkbox
+                info=(
+                    # Tooltip explaining how to trigger image generation via chat commands
+                    "Type <i><b>/image</b></i> followed by the instructions to start generating an image."
+                ),
+                value=True  # Enabled by default
+            )
+
+            # Checkbox to enable or disable the audio generation feature in the chat interface
+            audio_generation = gr.Checkbox(
+                label="Audio Generation",
+                info=(
+                    "Type <i><b>/audio</b></i> followed by the instructions to start generating audio."
+                ),
+                value=True
+            )
+
+            # Checkbox to enable or disable the deep web search feature in the chat interface
+            search_generation = gr.Checkbox(
+                label="Deep Search",
+                info=(
+                    "Type <i><b>/dp</b></i> followed by the instructions to search the web."
+                ),
+                value=True
+            )
+
+        # Create the main chat interface where users interact with the AI assistant
+        gr.ChatInterface(
+            fn=respond,  # Function called to generate responses to user inputs
+            additional_inputs=[
+                # Pass the current states of all configuration controls as additional inputs to the respond function
+                model_precision,
+                temperature,
+                top_k,
+                min_p,
+                top_p,
+                repetition_penalty,
+                reasoning,
+                image_generation,
+                audio_generation,
+                search_generation
+            ],
+            type='tuples', #  The format of the messages
+            chatbot=gr.Chatbot(
+                label="J.A.R.V.I.S.",   # Title label displayed above the chat window
+                show_copy_button=True,  # Show a button allowing users to copy chat messages
+                scale=1,  # Scale factor for the chatbot UI size
+                type='tuples' # Duplicate form Chat Interface to Chatbot
+            ),
+            examples=[
+                # Predefined example inputs to help users quickly test the assistant's features
+                ["Please introduce yourself."],
+                ["/audio Could you explain what Artificial Intelligence (AI) is?"],
+                ["/audio What is Hugging Face?"],
+                ["/dp Please search for the J.A.R.V.I.S. AI model on Hugging Face."],
+                ["/dp What is the capital city of Indonesia?"],
+                ["/image Create an image of a futuristic city."],
+                ["/image Create a cartoon-style image of a man."],
+                ["What day is it today, what's the date, and what time is it?"],
+                ['/audio Say "I am J.A.R.V.I.S.".'],
+                ["Please generate a highly complex code snippet on any topic."],
+                ["Explain about quantum computers."]
+            ],
+            cache_examples=False,  # Disable caching of example outputs to always generate fresh responses
+            multimodal=False,      # Disable support for multimodal inputs such as images or audio files
+            fill_height=True,      # Duplicate from Blocks to Chat Interface
+            fill_width=True,       # Duplicate from Blocks to Chat Interface
+            head=meta_tags         # Duplicate from Blocks to Chat Interface
+        )
+    # Return the complete Gradio app object for launching or embedding
+    return app

src/ui/reasoning.py

@@ -0,0 +1,75 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+def styles(reasoning: str, content: str, expanded: bool = False) -> str:
+    """
+    Create a visually captivating and interactive HTML <details> block that elegantly presents reasoning text 
+    inside a beautifully styled collapsible container with enhanced user experience features.
+
+    This function generates a sophisticated collapsible section using HTML and inline CSS, designed to grab 
+    attention with a modern, polished look. It leverages subtle shadows, smooth transitions, and vibrant styling 
+    to make the reasoning content stand out while maintaining excellent readability on dark backgrounds. 
+    The container uses the default background color to blend seamlessly with its surroundings, ensuring 
+    versatility in different UI contexts. The summary header includes an engaging emoji and changes color on hover 
+    to invite user interaction. The reasoning text is carefully spaced and styled with a clean font and 
+    crisp white color for maximum clarity. The collapsible block can start expanded or collapsed based on the 
+    'expanded' parameter. The 'content' parameter remains unused here to keep the function signature consistent 
+    with similar functions.
+
+    Args:
+        reasoning (str): The main explanation or reasoning text to be displayed inside the collapsible block. 
+                         This content is wrapped in a styled <div> for clear presentation.
+        content (str): An unused parameter retained for compatibility with other functions sharing this signature.
+        expanded (bool): Determines if the collapsible block is initially open (True) or closed (False) when rendered.
+
+    Returns:
+        str: A complete HTML snippet string containing a <details> element with inline CSS that styles it as 
+             a sleek, interactive collapsible container. The styling includes padding, rounded corners, 
+             a subtle but dynamic shadow, smooth hover effects on the summary header, and carefully sized fonts 
+             with white text for optimal contrast and readability.
+    """
+    # Conditionally add the 'open' attribute to the <details> element if expanded is True, 
+    # so the block starts expanded when rendered in the browser.
+    open_attr = "open" if expanded else ""
+
+    # Return the full HTML string with inline CSS styles applied to create an eye-catching, user-friendly collapsible block.
+    # The <details> element acts as a toggleable container with smooth rounded corners and a dynamic shadow that subtly intensifies on hover.
+    # The <summary> element serves as the clickable header with a brain emoji to visually represent reasoning, 
+    # featuring a color transition on hover to encourage user interaction.
+    # The reasoning text is enclosed in a <div> with generous spacing, a delicate top border, and crisp white text for excellent readability.
+    # The entire block uses a clean, modern sans-serif font and avoids any background color override to maintain design flexibility.
+    return f"""
+<details {open_attr} style="
+    padding: 16px;                              /* Comfortable inner spacing for a spacious feel */
+    border-radius: 12px;                       /* Smoothly rounded corners for a modern, friendly appearance */
+    margin: 12px 0;                            /* Vertical margin to separate from other page elements */
+    box-shadow: 0 4px 12px rgba(0,0,0,0.35);  /* Deeper, softly diffused shadow to create a subtle floating effect */
+    font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; /* Crisp, modern font stack for excellent readability */
+    color: white;                              /* Bright white text to stand out clearly on dark or varied backgrounds */
+    transition: box-shadow 0.3s ease-in-out;  /* Smooth shadow transition for dynamic visual feedback on hover */
+">
+  <summary style="
+    font-weight: 700;                         /* Bold font weight to make the summary header prominent */
+    color: white;                             /* White text color for consistent contrast */
+    font-size: 14px !important;               /* Slightly larger font size for better emphasis */
+    cursor: pointer;                          /* Pointer cursor to indicate the summary is clickable */
+    user-select: none;                        /* Prevent text selection on click for cleaner interaction */
+    transition: color 0.25s ease-in-out;     /* Smooth color transition when hovering */
+  " onmouseover="this.style.color='#FFD700';" onmouseout="this.style.color='white';">
+    🧠 Reasoning
+  </summary>
+  <div style="
+    margin-top: 12px;                         /* Clear separation between summary and content */
+    padding-top: 8px;                         /* Additional padding for comfortable reading space */
+    border-top: 1.5px solid rgba(255, 255, 255, 0.25); /* Elegant translucent top border to visually separate content */
+    font-size: 11px !important;               /* Slightly larger font size for improved readability */
+    line-height: 1.7;                         /* Increased line height for comfortable text flow */
+    color: white;                             /* Maintain white text color for clarity */
+    letter-spacing: 0.02em;                   /* Slight letter spacing to enhance text legibility */
+  ">
+    {reasoning}                              <!-- Reasoning -->
+  </div>
+</details>
+"""

src/utils/helper.py

@@ -0,0 +1,24 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+from datetime import datetime, timedelta  # Import datetime and timedelta classes to work with dates and time durations
+
+# Dictionary to track busy status of servers with their busy expiration timestamps
+busy = {}
+
+def mark(server: str):
+    """
+    Mark a server as busy by setting its busy expiration time to one hour from the current UTC time.
+
+    Args:
+        server (str): The identifier or name of the server to mark as busy.
+
+    Explanation:
+        This function updates the 'busy' dictionary by associating the given server
+        with a timestamp representing one hour from the current UTC time.
+        This indicates that the server is considered busy until that future time.
+    """
+    # Set the busy expiration time for the specified server to current UTC time plus one hour
+    busy[server] = datetime.utcnow() + timedelta(hours=1)

src/utils/ip_generator.py

@@ -0,0 +1,24 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import random  # Import the random module to generate random numbers
+
+def generate_ip() -> str:
+    """
+    Generate a random IPv4 address as a string.
+
+    Returns:
+        str: A randomly generated IPv4 address in dotted decimal notation,
+             where each octet is a number between 1 and 254 inclusive.
+
+    Explanation:
+        This function creates an IP address by generating four random integers,
+        each representing one octet of the IP address.
+        The range 1 to 254 is used to avoid special addresses like 0 (network) and 255 (broadcast).
+        The four octets are then joined together with dots to form a standard IPv4 address string.
+    """
+    # Generate four random integers between 1 and 254 inclusive, convert each to string,
+    # then join them with '.' to form a valid IPv4 address string
+    return ".".join(str(random.randint(1, 254)) for _ in range(4))

src/utils/session_mapping.py

@@ -0,0 +1,63 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import random  # Import random module to enable random selection from a list
+from datetime import datetime  # Import datetime class to work with current UTC time
+from typing import Dict, List  # Import type hints for dictionaries and lists (not explicitly used here but imported)
+from config import auth  # Import authentication configuration, likely a list of host dictionaries with credentials
+from src.utils.helper import busy, mark  # Import 'busy' dictionary and 'mark' function to track and update host busy status
+
+# Dictionary to map session IDs to their assigned host information
+mapping = {}
+
+def get_host(session_id: str):
+    """
+    Retrieve or assign a host for the given session ID.
+
+    Args:
+        session_id (str): A unique identifier for the current session.
+
+    Returns:
+        dict: The selected host dictionary from the auth configuration.
+
+    Raises:
+        Exception: If no available hosts are found to assign.
+
+    Explanation:
+        This function manages host assignment per session. If the session ID already has a host assigned,
+        it returns that host immediately. Otherwise, it filters the list of hosts from 'auth' to find those
+        that are currently not busy or whose busy period has expired (based on the 'busy' dictionary).
+        From the available hosts, it randomly selects one, records the assignment in 'mapping',
+        marks the selected host as busy for one hour, and returns the selected host.
+    """
+    # Check if the session ID already has an assigned host in the mapping dictionary
+    if session_id in mapping:
+        # Return the previously assigned host for this session
+        return mapping[session_id]
+
+    # Get the current UTC time to compare against busy timestamps
+    now = datetime.utcnow()
+
+    # Filter hosts from auth that are either not marked busy or whose busy period has expired
+    connect = [
+        h for h in auth
+        if h["jarvis"] not in busy or busy[h["jarvis"]] <= now
+    ]
+
+    # If no hosts are available after filtering, raise an exception to indicate no hosts can be assigned
+    if not connect:
+        raise Exception("No available hosts to assign.")
+
+    # Randomly select one host from the list of available hosts
+    selected = random.choice(connect)
+
+    # Map the session ID to the selected host for future reference
+    mapping[session_id] = selected
+
+    # Mark the selected host as busy for the next hour to prevent immediate reassignment
+    mark(selected["jarvis"])
+
+    # Return the selected host dictionary
+    return selected

src/utils/tools.py

@@ -0,0 +1,63 @@
+#
+# SPDX-FileCopyrightText: Hadad <[email protected]>
+# SPDX-License-Identifier: Apache-2.0
+#
+
+import random  # Import random module to enable random selection from a list
+from datetime import datetime  # Import datetime class to work with current UTC time
+from config import auth  # Import authentication configuration, likely a list of host dictionaries with credentials
+from src.utils.helper import busy, mark  # Import 'busy' dictionary and 'mark' function to track and update host busy status
+
+def initialize_tools():
+    """
+    Initialize and select available tools (endpoints) from the configured hosts.
+
+    Returns:
+        tuple: A tuple containing three elements:
+            - tool_setup (str): The endpoint or configuration for the main tool setup.
+            - image_tool (str): The endpoint URL for image generation services.
+            - audio_tool (str): The endpoint URL for audio generation services.
+
+    Raises:
+        Exception: If no available hosts are found or if any required tool endpoint is missing.
+
+    Explanation:
+        This function filters the list of hosts from 'auth' to find those not currently busy or whose busy period has expired.
+        It randomly selects one available host, marks it as busy for one hour,
+        and retrieves the required tool endpoints ('done', 'image', 'audio') from the selected host's configuration.
+        If any of these endpoints are missing, it raises an exception.
+        Finally, it returns the three tool endpoints as a tuple.
+    """
+    # Get the current UTC time for busy status comparison
+    now = datetime.utcnow()
+
+    # Filter hosts that are either not marked busy or whose busy period has expired
+    available = [
+        item for item in auth
+        if item["jarvis"] not in busy or busy[item["jarvis"]] <= now
+    ]
+
+    # Raise an exception if no hosts are currently available
+    if not available:
+        raise Exception("No available hosts to initialize tools.")
+
+    # Randomly select one host from the available list
+    selected = random.choice(available)
+
+    # Mark the selected host as busy for the next hour to prevent immediate reassignment
+    mark(selected["jarvis"])
+
+    # Retrieve the tool endpoints from the selected host's configuration dictionary
+    tool_setup = selected.get("done")  # Main tool setup endpoint or configuration
+    image_tool = selected.get("image")  # Image generation service endpoint
+    audio_tool = selected.get("audio")  # Audio generation service endpoint
+
+    # Verify that all required tool endpoints are present, raise exception if any is missing
+    if not tool_setup or not image_tool or not audio_tool:
+        raise Exception("Selected host is missing required tool endpoints.")
+
+    # Return the three tool endpoints as a tuple
+    return tool_setup, image_tool, audio_tool
+
+# Initialize the tools by selecting an available host and retrieving its endpoints
+tool_setup, image_tool, audio_tool = initialize_tools()