Spaces:
Running
Data Flow Breakdown in learning_path_orchestrator We follow a structured data pipeline where each step modifies and passes data to the next stage.
1οΈβ£ Define Custom Data Structures
- Topic (BaseModel) β Represents a single topic with name and description.
- Topics (BaseModel) β A wrapper around multiple Topic objects (essentially a list of topics).
- State (TypedDict) β Holds global state, including user input, generated topics, and completed topics.
- WorkerState (TypedDict) β Holds individual topic assignments for processing.
2οΈβ£ Step-by-Step Data Flow
Step 1: Orchestrator Generates Topics Input: user_skills and user_goals Process: Calls planner.invoke(), which uses an LLM (Groq API) to generate topics. Output: A structured Topics object (a list of Topic objects). Storage: The topics list is saved inside State. Returns: {"topics": study_plan.topics} π Key Detail: The Orchestrator only generates topics and doesnβt process them. It assigns each topic to workers.
Step 2: Assign Workers to Each Topic Function: assign_workers(state: State) Process: Iterates over state["topics"] and assigns each topic to a worker (i.e., llm_call). Returns: A list of dispatch instructions, sending each topic to the llm_call function. Key Mechanism: Uses Send("llm_call", {"topic": t}), which maps each topic to WorkerState. π Key Detail: This step distributes work in parallel across multiple workers, each handling a single topic.
Step 3: LLM Call Generates Topic Summaries Function: llm_call(state: WorkerState) Input: A single topic object (from WorkerState). Process: Calls the LLM (llm.invoke) with the topic's name and description. Generates a summary + resources in markdown format. Output: {"completed_topics": [topic_summary.content]} Storage: The summaries are stored inside completed_topics in State. π Key Detail: Each worker only receives one topic at a time. The WorkerState helps isolate one topic per call instead of processing everything at once.
Step 4: Synthesizer Combines Summaries into a Learning Roadmap Function: synthesizer(state: State) Input: completed_topics list (all processed topics). Process: Joins all summaries together into a structured format. Output: {"learning_roadmap": learning_roadmap} Final Storage: The roadmap is stored inside State. π Key Detail: This step aggregates all topic summaries into a final, structured learning plan.
3οΈβ£ Where Does the Data Go? Step Function Input Output Where the Data Goes 1 orchestrator(state) User skills & goals topics list Stored in State["topics"] 2 assign_workers(state) Topics list Send("llm_call", {"topic": t}) Sends each topic to llm_call 3 llm_call(state) A single topic {"completed_topics": [summary]} Appends to State["completed_topics"] 4 synthesizer(state) completed_topics list learning_roadmap Stores final roadmap in State["learning_roadmap"]
π Key Takeaways
- Orchestrator generates the topics based on user_skills and user_goals.
- Workers process each topic separately (using llm_call).
- WorkerState ensures only one topic is processed per worker to avoid mixing topics.
- The synthesizer combines all results into a final structured roadmap.
- Data flows in a structured manner through State and WorkerState, ensuring modular and parallel execution.