app.py

import os
import json
import re
from huggingface_hub import InferenceClient
import gradio as gr
from pydantic import BaseModel, Field
from typing import Optional, Literal

# Define the metaprompts
metaprompt1 = """
As an AI Prompt Enhancement Specialist, your task is to improve the given prompt using advanced prompting techniques while emphasizing the use of previously generated context. Analyze the input prompt and apply the following steps to enhance it:

1. Role Prompting: Assume the role of an "Expert Prompt Analyst and Optimizer" with deep knowledge in the subject matter of the input prompt.

2. Structured Output Generation: Design a prompt structure that builds upon previous sections:
   a) Introduction
   b) Key Points (referencing the introduction)
   c) Detailed Analysis (expanding on key points and referencing previous sections)
   d) Conclusion (summarizing and synthesizing all previous content)
   e) Final Review and Refinement

3. Few-Shot Prompting: Provide 2-3 relevant examples demonstrating the desired output format, reasoning, and effective use of previous context.

4. Chain-of-Thought (CoT) and Zero-Shot CoT: Incorporate step-by-step reasoning in your instructions, explaining the thought process for each enhancement and how to handle unfamiliar tasks.

5. Self-Consistency: Generate multiple reasoning paths and select the most consistent one for the final improved prompt.

6. Least-to-Most Prompting: Break down complex aspects of the prompt into smaller, manageable sub-tasks that build upon each other.

7. Tree-of-Thought: Create a branching structure for exploring multiple improvement paths simultaneously, considering different aspects of the prompt.

8. Prompt Paraphrasing: Rephrase key instructions in multiple ways to ensure clarity and comprehension.

9. Self-Calibration: Include steps for the AI to assess its own confidence in the generated output and adjustments.

10. Self-Refine: Implement an iterative process for the AI to improve its initial response to the prompt enhancement task.

11. Self-Verification and Chain-of-Verification (COVE): Add verification steps to check the logical consistency, effectiveness, and fulfillment of all requirements in the improved prompt.

12. Cumulative Reasoning: Build upon previous enhancements, accumulating improvements throughout the process.

13. Style Prompting: Adjust the language style of the prompt to match the intended audience or purpose.

14. Emotion Prompting: Incorporate language that emphasizes the importance and impact of the task described in the prompt.

15. System 2 Attention (S2A): Guide the AI to focus on specific critical aspects of the prompt that require careful consideration.

16. Rephrase and Respond (RaR): After each major enhancement, instruct the AI to rephrase the prompt and respond to it to test its effectiveness.

17. Re-reading (RE2): Instruct the AI to re-read the original and enhanced prompts multiple times to catch nuances and potential improvements.

18. Self-Ask: Encourage the AI to ask itself questions about the prompt's clarity, effectiveness, and potential weaknesses, especially in relation to using previous context.

19. Automatic Chain-of-Thought (Auto-CoT): Generate intermediate reasoning steps automatically to support the prompt enhancement process.

20. Program-of-Thoughts: Structure the prompt improvement process as a series of logical operations or a program-like sequence.

21. Skeleton-of-Thought: Create a basic structure or skeleton for the enhanced prompt, then iteratively fill in the details.

22. Mixture of Reasoning Experts (MoRE): Combine insights from multiple expert perspectives (e.g., subject matter expert, language specialist, task optimization expert) to enhance the prompt.

23. Max Mutual Information Method: Optimize the prompt to maximize the mutual information between the input and desired output.

Now, apply these techniques to improve the following prompt:

Initial Prompt: [Insert initial prompt here]

Follow these steps to generate an enhanced version of the prompt:

1. Analyze the original prompt, identifying its main objectives, strengths, and weaknesses.
2. Apply the role of an Expert Prompt Analyst and Optimizer, considering the subject matter.
3. Design a structured output format that encourages use of previous context.
4. Use the Tree-of-Thought technique to explore multiple enhancement paths.
5. For each path, apply Chain-of-Thought reasoning to explain the enhancements.
6. Incorporate Few-Shot examples to illustrate the desired outcome and effective use of previous context.
7. Break down complex aspects using Least-to-Most Prompting.
8. Rephrase key instructions using Prompt Paraphrasing.
9. Adjust the language style and incorporate emotional elements as appropriate.
10. Implement Self-Verification and Chain-of-Verification steps.
11. Use Self-Ask to critically evaluate the enhancements, especially regarding context usage.
12. Apply the Skeleton-of-Thought technique to structure the improved prompt.
13. Utilize the Mixture of Reasoning Experts approach to refine the prompt from multiple perspectives.
14. Employ the Max Mutual Information Method to optimize the prompt's effectiveness.
15. Use Self-Calibration to assess the confidence in the final enhanced prompt.
16. Apply Self-Refine to iteratively improve the enhanced prompt.

Present the final enhanced prompt, along with a brief explanation of the key improvements and techniques used. Ensure that the enhanced prompt:
1. Maintains the original intent
2. Significantly improves effectiveness and clarity
3. Effectively leverages previously generated context
4. Includes explicit instructions for the AI to refer back to and build upon its own previous outputs
5. Creates a cohesive and interconnected result

Your enhanced prompt should guide the AI to generate a response that not only addresses the original query but also demonstrates a clear progression of thought and utilization of previously generated information throughout the response.

Only provide the output in the following JSON format enclosed in <json> tags:
<json>
{
"initial_prompt_evaluation": "Your evaluation of the initial prompt with Strengths and Weaknesses in a string as bullet points format",
"refined_prompt": "Your refined prompt",
"explanation_of_refinements": "Explain techniques used and improvements made, including the extract of final prompt where it used. Answer in a string "
}
</json>
"""


loic_metaprompt = """
Given an initial prompt:

Initial Prompt: [Insert initial prompt here]

Understand the Task: Grasp the main objective, goals, requirements, constraints, and expected output.
- Minimal Changes: If an existing prompt is provided, improve it only if it's simple. For complex prompts, enhance clarity and add missing elements without altering the original structure.
- Leverage Autoregressive Nature: Strategically order information to prime the model for better results. Generate key concepts, context, or vocabulary early in the prompt to inform later reasoning and outputs.
- Reasoning Before Conclusions: Encourage reasoning steps before any conclusions are reached. If user examples show reasoning afterward, reverse the order. Never start examples with conclusions.
    - Reasoning Order: Identify reasoning and conclusion parts. Determine their order and reverse if needed. Conclusions, classifications, or results should always appear last.
- Examples: Include high-quality examples if helpful, using placeholders [in brackets] for complex elements.
   - Assess example needs, quantity, and complexity for potential placeholder use.
- Clarity and Conciseness: Use clear, specific language. Avoid unnecessary instructions or bland statements.
- Formatting: Use markdown features for readability. Avoid code blocks unless requested.
- Preserve User Content: Retain extensive guidelines or examples from input. Break down vague instructions into sub-steps. Maintain user-provided details, guidelines, examples, variables, or placeholders.
- Constants: Include constants (guides, rubrics, examples) as they resist prompt injection.
- Output Format: Specify appropriate output format in detail, including length and syntax.
    - For structured data, prefer JSON output (unwrapped unless requested).

The final prompt should follow this structure, with no additional commentary:
Leverage Autoregressive Nature: Strategically order information to prime the model for better results. Generate key concepts, context, or vocabulary early in the prompt to inform later reasoning and outputs.

[Concise task instruction - first line, no header]

[Additional details as needed]

[Optional sections with headings or bullet points for detailed steps]

# Context Generation

[Instructions for generating key concepts, vocabulary, or context to inform later reasoning]

# Steps [optional]

[Detailed task breakdown]

# Reasoning Process

[Specific instructions for the reasoning process, ensuring it precedes conclusions]

# Output Format

[Detailed output format specifications]

# Examples [optional]

[1-3 well-defined examples with placeholders if necessary. Mark input/output clearly. Use placeholders and indicate if real examples differ in length/complexity]

# Notes [optional]

[Edge cases, important considerations, repeated key points]

Only provide the output in the following JSON format enclosed in <json> tags:

<json>
{
"initial_prompt_evaluation": "Your evaluation of the initial prompt with Strengths and Weaknesses in a string",
"refined_prompt": "Your refined prompt into quote",
"explanation_of_refinements": "Explanation of the techniques used and improvements made, also include the extract of final prompt where it made.Answer in bullet points if accurate"
}
</json>
"""

openai_metaprompt='''
Given a task description or existing prompt, produce a detailed system prompt to guide a language model in completing the task effectively.

# Guidelines

- Understand the Task: Grasp the main objective, goals, requirements, constraints, and expected output.
- Minimal Changes: If an existing prompt is provided, improve it only if it's simple. For complex prompts, enhance clarity and add missing elements without altering the original structure.
- Reasoning Before Conclusions**: Encourage reasoning steps before any conclusions are reached. ATTENTION! If the user provides examples where the reasoning happens afterward, REVERSE the order! NEVER START EXAMPLES WITH CONCLUSIONS!
    - Reasoning Order: Call out reasoning portions of the prompt and conclusion parts (specific fields by name). For each, determine the ORDER in which this is done, and whether it needs to be reversed.
    - Conclusion, classifications, or results should ALWAYS appear last.
- Examples: Include high-quality examples if helpful, using placeholders [in brackets] for complex elements.
   - What kinds of examples may need to be included, how many, and whether they are complex enough to benefit from placeholders.
- Clarity and Conciseness: Use clear, specific language. Avoid unnecessary instructions or bland statements.
- Formatting: Use markdown features for readability. DO NOT USE ``` CODE BLOCKS UNLESS SPECIFICALLY REQUESTED.
- Preserve User Content: If the input task or prompt includes extensive guidelines or examples, preserve them entirely, or as closely as possible. If they are vague, consider breaking down into sub-steps. Keep any details, guidelines, examples, variables, or placeholders provided by the user.
- Constants: DO include constants in the prompt, as they are not susceptible to prompt injection. Such as guides, rubrics, and examples.
- Output Format: Explicitly the most appropriate output format, in detail. This should include length and syntax (e.g. short sentence, paragraph, JSON, etc.)
    - For tasks outputting well-defined or structured data (classification, JSON, etc.) bias toward outputting a JSON.
    - JSON should never be wrapped in code blocks (```) unless explicitly requested.

The final prompt you output should adhere to the following structure below. Do not include any additional commentary, only output the completed system prompt. SPECIFICALLY, do not include any additional messages at the start or end of the prompt. (e.g. no "---")

[Concise instruction describing the task - this should be the first line in the prompt, no section header]

[Additional details as needed.]

[Optional sections with headings or bullet points for detailed steps.]

# Steps [optional]

[optional: a detailed breakdown of the steps necessary to accomplish the task]

# Output Format

[Specifically call out how the output should be formatted, be it response length, structure e.g. JSON, markdown, etc]

# Examples [optional]

[Optional: 1-3 well-defined examples with placeholders if necessary. Clearly mark where examples start and end, and what the input and output are. User placeholders as necessary.]
[If the examples are shorter than what a realistic example is expected to be, make a reference with () explaining how real examples should be longer / shorter / different. AND USE PLACEHOLDERS! ]

# Notes [optional]

[optional: edge cases, details, and an area to call or repeat out specific important considerations]

Only provide the output in the following JSON format enclosed in <json> tags:

<json>
{
"initial_prompt_evaluation": "Your evaluation of the initial prompt with Strengths and Weaknesses in a string on markdown bullet points format",
"refined_prompt": "Your refined prompt",
"explanation_of_refinements": "Explanation of the techniques used and improvements made, also include the extract of final prompt where it made. Answer in bullet points if accurate"
}
</json>
'''
original_meta_prompt = '''Create an effective prompt from this one:

Initial Prompt: [Insert initial prompt here]

Use these simple steps:

1. Look at the task closely, focusing on goals, needs, and limits.
2. Make prompts better by making language clear without changing the main idea.
3. Keep a logical flow by putting reasoning steps first.
4. Use good examples with [placeholders] for tricky parts.
5. Use simple words; remove unneeded instructions.
6. Use markdown for easy reading; skip code blocks unless needed.
7. Keep user content like guides and examples.
8. Add constants like guides and rubrics.
9. Clearly state output format including structure and syntax.

Only provide the output in the following JSON format enclosed in <json> tags:

<json>
{
"initial_prompt_evaluation": "Your evaluation of the initial prompt with Strengths and Weaknesses in a string as bullet points format",
"refined_prompt": "Your refined prompt",
"explanation_of_refinements": "Explanation of the techniques used and improvements made, also include the extract of final prompt where it made. Answer in bullet points if accurate"
}
</json>
'''

new_meta_prompt = '''Given an initial prompt:

Initial Prompt: [Insert initial prompt here]

follow these steps to refine the prompt:

1. Analyze the prompt:
   - Identify the main topic and key concepts
   - Determine the type of task (e.g., math problem, logical reasoning, factual question)
   - Assess the complexity and any specific requirements

2. Evaluate the initial prompt:
   - Determine its strengths and weaknesses in relation to the query
   - Identify any missing elements or irrelevant information

3. Refine the prompt:
   - Retain relevant parts of the initial prompt
   - Add query-specific context or information
   - Incorporate appropriate reasoning frameworks (e.g., step-by-step, chain-of-thought)
   - Ensure clear instructions for the desired output format

4. Optimize for information flow:
   - Ensure the refined prompt allows for direct use of query information
   - Enable indirect information aggregation through prompt-guided reasoning

5. Enhance versatility:
   - Add elements that encourage flexible thinking and problem-solving
   - Include self-evaluation or error-checking instructions if appropriate

6. Formulate the refined prompt:
   - Combine all elements into a clear, concise, and effective prompt
   - Ensure the language and structure are appropriate for the task

7. Output the refined prompt:
   - Present the new prompt, highlighting key changes or additions

Only provide the output in the following JSON format enclosed in <json> tags:

<json>
{
"initial_prompt_evaluation": "Your evaluation of the initial prompt with Strengths and Weaknesses in a string as bullet points format",
"refined_prompt": "Your refined prompt",
"explanation_of_refinements": "Explanation of the techniques used and improvements made, also include the extract of final prompt where it made. Answer in bullet points if accurate"
}
</json>
'''

advanced_meta_prompt = '''
You are an expert in natural language processing and state-of-the-art prompt engineering. Your task is to refine and improve the given prompt using advanced prompt engineering techniques.

Initial Prompt: [Insert initial prompt here]

Analyze the initial prompt and generate an improved version. Apply various prompt engineering techniques to enhance clarity, specificity, and effectiveness. Consider techniques such as:

1. Clarity and Specificity
2. Structural Improvement
3. Conciseness
4. Ambiguity Resolution
5. Instruction Refinement
6. Query Focusing
7. Task Decomposition
8. Role-Playing
9. Chain-of-Thought
10. Few-Shot Learning
11. Constrained Generation
12. Metacognitive Prompting
13. Contextual Priming
14. Negative Prompting
15. Output Format Control
16. Emotional Tone Setting
17. Iterative Refinement
18. Analogical Reasoning
19. Perspective Shifting

Only provide the output in the following JSON format enclosed in <json> tags:

<json>
{
"initial_prompt_evaluation": "Your evaluation of the initial prompt with Strengths and Weaknesses in a string as bullet points format",
"refined_prompt": "Your refined prompt",
"explanation_of_refinements": "Explanation of the techniques used and improvements made, also include the extract of final prompt where it made. Answer in bullet points if accurate"
}
</json>
'''

import json
import re
import json
import re
from huggingface_hub import InferenceClient
from pydantic import BaseModel, Field
from typing import Optional, Literal

class PromptInput(BaseModel):
    text: str = Field(..., description="The initial prompt text")
    meta_prompt_choice: Literal["physics","morphosis", "verse", "phor","bolism"] = Field(..., description="Choice of meta prompt strategy")

class RefinementOutput(BaseModel):
    query_analysis: Optional[str] = None
    initial_prompt_evaluation: Optional[str] = None
    refined_prompt: Optional[str] = None
    explanation_of_refinements: Optional[str] = None
    raw_content: Optional[str] = None

class PromptRefiner:
    def __init__(self, api_token: str):
        self.client = InferenceClient(token=api_token)

    def refine_prompt(self, prompt_input: PromptInput) -> RefinementOutput:
        if prompt_input.meta_prompt_choice == "morphosis":
            selected_meta_prompt = original_meta_prompt
        elif prompt_input.meta_prompt_choice == "verse":
            selected_meta_prompt = new_meta_prompt
        elif prompt_input.meta_prompt_choice == "physics":
            selected_meta_prompt = metaprompt1
        elif prompt_input.meta_prompt_choice == "bolism":
            selected_meta_prompt = loic_metaprompt
        else:
            selected_meta_prompt = advanced_meta_prompt
    
        messages = [
            {"role": "system", "content": 'You are an expert at refining and extending prompts. Given a basic prompt, provide a more detailed.'},
            {"role": "user", "content": selected_meta_prompt.replace("[Insert initial prompt here]", prompt_input.text)}
        ]
        response = self.client.chat_completion(
            model="meta-llama/Meta-Llama-3-70B-Instruct",
            messages=messages,
            max_tokens=3000,
            temperature=0.3
        )
        response_content = response.choices[0].message.content.strip()
        try:
            # Extract JSON from between <json> tags
            json_match = re.search(r'<json>\s*(.*?)\s*</json>', response_content, re.DOTALL)
            if json_match:
                json_str = json_match.group(1)
                # Remove newlines and escape quotes within the JSON string
                json_str = re.sub(r'\n\s*', ' ', json_str)
                json_str = json_str.replace('"', '\\"')
                # Wrap the entire string in quotes and parse it
                json_output = json.loads(f'"{json_str}"')
                # Ensure json_output is a dictionary
                if isinstance(json_output, str):
                    json_output = json.loads(json_output)
                # Unescape the parsed JSON
                for key, value in json_output.items():
                    if isinstance(value, str):
                        json_output[key] = value.replace('\\"', '"')
                return RefinementOutput(**json_output, raw_content=response_content)
            else:
                raise ValueError("No JSON found in the response")
        except (json.JSONDecodeError, ValueError) as e:
            print(f"Error parsing JSON: {e}")
            print(f"Raw content: {response_content}")
            # If JSON parsing fails, attempt to extract the content manually
            output = {}
            for key in ["initial_prompt_evaluation", "refined_prompt", "explanation_of_refinements"]:
                pattern = rf'"{key}":\s*"(.*?)"(?:,|\}})'
                match = re.search(pattern, response_content, re.DOTALL)
                if match:
                    output[key] = match.group(1).replace('\\n', '\n').replace('\\"', '"')
                else:
                    output[key] = "" # Set empty string if content not found
            return RefinementOutput(**output, raw_content=response_content)
                
    def apply_prompt(self, prompt: str) -> str:
        try:
            messages = [
                {"role": "system", "content": "You are a helpful assistant. Answer in stylized version with latex format or markdown if relevant. Separate your answer into logical sections using level 2 headers (##) for sections and bolding (**) for subsections.Incorporate a variety of lists, headers, and text to make the answer visually appealing"},
                {"role": "user", "content": prompt}
            ]

            response = self.client.chat_completion(
                model="meta-llama/Meta-Llama-3-70B-Instruct",
                messages=messages,
                max_tokens=3000, # Increased token limit
                temperature=0.8
            )

            output = response.choices[0].message.content.strip()
            # Basic post-processing
            output = output.replace('\n\n', '\n').strip()
            return output
        except Exception as e:
            return f"Error: {str(e)}"

class GradioInterface:
    def __init__(self, prompt_refiner: PromptRefiner):
        self.prompt_refiner = prompt_refiner

        with gr.Blocks() as self.interface:
            gr.Markdown("# PROMPT++")
            gr.Markdown("### Refine your Prompts: Learn how to generate an improved version of your prompts")
            gr.Markdown("Enter a main idea for a prompt, choose a meta prompt, and the model will attempt to generate an improved version.")

            with gr.Row():
                prompt_text = gr.Textbox(label="Type the prompt here")
            with gr.Row():
                meta_prompt_choice = gr.Radio(["physics","morphosis", "verse", "phor","bolism"], label="Choose Meta Prompt", value="morphosis")
                refine_button = gr.Button("Refine Prompt")
            with gr.Row():
                gr.Markdown("### Initial prompt analysis")
            with gr.Column():
                analysis_evaluation = gr.Markdown(label="Analysis and Evaluation")
                gr.Markdown("### Refined Prompt")
                refined_prompt = gr.Textbox(label="Refined Prompt")
                gr.Markdown("### Explanation of Refinements")
                explanation_of_refinements = gr.Markdown(label="Explanation of Refinements")

            with gr.Accordion("Full Response JSON", open=False):
                full_response_json = gr.JSON()

            refine_button.click(
                fn=self.refine_prompt,
                inputs=[prompt_text, meta_prompt_choice],
                outputs=[analysis_evaluation, refined_prompt, explanation_of_refinements, full_response_json]
            )
            with gr.Row():
            #    apply_model=gr.Dropdown(["meta-llama/Llama-3.1-70B-Instruct",'Qwen/Qwen2.5-72B-Instruct'], value="meta-llama/Llama-3.1-70B-Instruct", label="Model"),
                apply_button = gr.Button("Apply Prompts")

            with gr.Row():
                with gr.Column():
                    gr.Markdown("### Original Prompt Output")
                    original_output = gr.Markdown(label="Original Prompt Output")
                with gr.Column():
                    gr.Markdown("### Refined Prompt Output")
                    refined_output = gr.Markdown(label="Refined Prompt Output")

            apply_button.click(
                fn=self.apply_prompts,
                inputs=[prompt_text, refined_prompt],
                outputs=[original_output, refined_output]
            )

            gr.Examples(
                examples=[
                    ["Tell me about that guy who invented the light bulb", "physics"],
                    ["Explain the universe.", "bolism"],
                    ["What's the population of New York City and how tall is the Empire State Building and who was the first mayor?", "morphosis"],
                    ["List American presidents.", "verse"],
                    ["Write a story.", "bolism"],
                    ["Explain why the experiment failed.", "morphosis"],
                    ["Is nuclear energy good?", "verse"],
                    ["How does a computer work?", "phor"],
                    ["How to make money fast?", "morphosis"],
                    ["how can you  prove IT0's lemma in stochastic calculus ?", "verse"],                    
                ],
                inputs=[prompt_text, meta_prompt_choice]
            )

                
    def refine_prompt(self, prompt: str, meta_prompt_choice: str) -> tuple:
        input_data = PromptInput(text=prompt, meta_prompt_choice=meta_prompt_choice)
        result = self.prompt_refiner.refine_prompt(input_data)
        analysis_evaluation = f"\n\n{result.initial_prompt_evaluation}"
        return (
            analysis_evaluation,
            result.refined_prompt,
            result.explanation_of_refinements,
            result.dict()
        )

    def apply_prompts(self,original_prompt: str, refined_prompt: str):
        original_output = self.prompt_refiner.apply_prompt(original_prompt)
        refined_output = self.prompt_refiner.apply_prompt(refined_prompt)
        return original_output, refined_output

    def launch(self):
        self.interface.launch()

# Main code to run the application
if __name__ == '__main__':
    api_token = os.getenv('HF_API_TOKEN')
    if not api_token:
        raise ValueError("HF_API_TOKEN not found in environment variables")

    prompt_refiner = PromptRefiner(api_token)
    gradio_interface = GradioInterface(prompt_refiner)
    gradio_interface.launch()