architecture_plan.md

# AI Agent Architecture Plan

## 1. Frameworks and Tools
- **Python**: The primary programming language.
- **Manim**: For creating mathematical animations.
- **OpenAI API**: For generating Manim code from text prompts.
- **pydantic-ai**: For structured AI agent creation, function calling, and workflow management.
- **Gradio**: For creating a user interface to input prompts and display generated videos.
- **dotenv**: For managing environment variables.
- **Logging**: For logging information and errors.

## 2. Agent Architecture
- **Input Handling**: Use Gradio to create a user interface where users can input text prompts.
- **Agent Structure**: Leverage pydantic-ai to define the agent's schema, capabilities, and functions.
- **System Prompts**: Use both static and dynamic system prompts to guide the agent's behavior.
  - Static prompts: Define the agent's role and general capabilities
  - Dynamic prompts: Adjust behavior based on complexity settings and current context
- **Keyword Identification**: Use pydantic-ai with OpenAI API to identify keywords and generate Manim code.
- **Scenario Creation**: Define structured schemas in pydantic-ai to guide the generation of scenarios.
- **Function Search**: Use pydantic-ai's function calling capabilities to organize and call Manim functions.
- **Code Generation and Testing**: The generated code will be tested by rendering the video using Manim.

## 3. Workflow
1. **User Input**: The user inputs a text prompt describing a mathematical or physics concept.
2. **Agent Processing**: The pydantic-ai agent processes the input through defined schemas and tools.
   - System prompts dynamically adjust based on user requirements
   - Tools are applied in sequence using the agent's capabilities
3. **Keyword Identification and Scenario Creation**: The agent uses OpenAI API to analyze the prompt and generate a structured scenario.
4. **Code Generation**: The agent transforms the structured scenario into Manim code using defined tools.
5. **Video Rendering**: The code is executed using Manim to render the video.
6. **Output**: The generated video is displayed to the user.

## 4. Detailed Steps
1. **Setup Environment**:
   - Ensure all required packages are installed (`gradio`, `openai`, `pydantic-ai`, `dotenv`, `manim`, etc.).
   - Set up environment variables in `.env` file (e.g., `TOGETHER_API_KEY`).
   - Configure pydantic-ai with appropriate model settings.

2. **Create Agent Structure**:
   - Define pydantic models for input prompts, scenario descriptions, and animation parameters.
   - Create static and dynamic system prompts to guide agent behavior:
     - Static: Define the agent's role and general capabilities
     - Dynamic: Adjust behavior based on request complexity and context
   - Create tool functions for scenario extraction, code generation, and rendering.
   - Configure the agent with appropriate tools and models.

3. **Create User Interface**:
   - Use Gradio to create a web interface for inputting prompts and displaying results.
   - Add complexity selection controls to customize animation generation.
   - Connect the UI to the pydantic-ai agent.

4. **Generate Manim Code**:
   - Implement functions using pydantic-ai tools to transform user prompts into structured scenarios.
   - Convert structured scenarios into Manim code templates.
   - Fill templates with specifics from the scenario.

5. **Render Video**:
   - Implement a function to render the generated Manim code into a video.
   - Add error handling and validation using pydantic models.

6. **Display Results**:
   - Display the generated video and code in the Gradio interface.
   - Provide feedback and explanations based on the agent's processing steps.