Merge remote-tracking branch 'origin/main'

.github/workflows/docker-image.yml

@@ -0,0 +1,41 @@
+name: Docker Image CI
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+
+  # build:
+
+  #   runs-on: ubuntu-latest
+
+  #   steps:
+  #   - uses: actions/checkout@v3
+  #   - name: Build the Docker image
+  #     run: docker build . --file Dockerfile --tag yaleh/meta-prompt:$(date +%s)
+
+  build-and-publish:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+
+      - name: Build and Publish Docker image
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          push: true
+          tags: |
+            yaleh/meta-prompt:${{ github.sha }}
+            ${{ github.ref == 'refs/heads/main' && 'yaleh/meta-prompt:latest' || '' }}
+    

README.md

@@ -8,6 +8,8 @@ This project is a demonstration of the concept of Meta Prompt, which involves ge
 
 The demo utilizes OpenAI's language models and provides a user interface for interacting with the chatbot. It allows users to input prompts, execute model calls, compare outputs, and optimize prompts based on desired criteria.
 
+**New Update (2023/10/15):** A new working mode called `Other User Prompts` has been added. In the prompt optimization process, similar compatible prompts to the original user prompt are referenced to significantly reduce iteration cycles.
+
 ## Try it out!
 
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yaleh/meta-prompt/blob/main/meta_prompt.ipynb)
@@ -54,6 +56,34 @@ You can use either GPT-4 or GPT-3.5 as the Testing LLM Model, similar to when us
 
 If you have access to ChatGPT and want to save costs on GPT-4 API usage, you can also manually execute the meta-prompt by clicking "Merge Meta System Prompt." This will generate a complete prompt, including the meta-prompt and the current example, that can be used with ChatGPT. However, note that if further iterations are required, you need to manually copy the newly generated system prompt to the Current System Prompt and click "Run Current" to update the Current Output.
 
+## Running Docker Image
+
+To perform the demo using Docker, make sure you have Docker installed on your system, and then follow these steps:
+
+1. Pull the Meta Prompt Docker image by running the following command:
+
+```
+docker pull yaleh/meta-prompt
+```
+
+2. Run the Docker container with the following command:
+
+```
+docker run -d --name meta-prompt-container -p 7860:7860 -e API_KEY=YOUR_API_KEY -e OTHER_ARGS="--advanced_mode" -e OPENAI_API_BASE=https://openai.lrfz.com/v1 yaleh/meta-prompt
+```
+
+Replace `YOUR_API_KEY` with your OpenAI API key. You can modify other environment variables if needed.
+3. You can now access the Meta Prompt demo by opening your web browser and visiting `http://localhost:7860`.
+
+To stop and remove the Meta Prompt container, run the following commands:
+
+```
+docker stop meta-prompt-container
+docker rm meta-prompt-container
+```
+
+Usually, simple questions (such as arithmetic operations) require around 3 iterations of optimization, while complex problems may require more than 10 iterations.
+
 ## Examples
 
 ### Arithmetic
@@ -151,6 +181,69 @@ Year,Germany,France,United Kingdom,Italy,Spain
 2019-2018,-2.56%,-2.59%,0.01%,-4.11%,0.30%
 ```
 
+#### Other User Prompts
+
+```
+Here is the GDP data in billions of US dollars (USD) for these years:
+
+1. China:
+   - 2010: $6,101.18 billion
+   - 2011: $7,572.80 billion
+   - 2012: $8,560.59 billion
+   - 2013: $9,607.23 billion
+   - 2014: $10,482.65 billion
+
+2. India:
+   - 2010: $1,675.62 billion
+   - 2011: $1,823.05 billion
+   - 2012: $1,827.64 billion
+   - 2013: $1,856.72 billion
+   - 2014: $2,046.88 billion
+
+3. Japan:
+   - 2010: $5,700.35 billion
+   - 2011: $6,157.47 billion
+   - 2012: $6,203.21 billion
+   - 2013: $5,155.72 billion
+   - 2014: $4,616.52 billion
+
+4. South Korea:
+   - 2010: $1,464.26 billion
+   - 2011: $1,622.03 billion
+   - 2012: $1,624.76 billion
+   - 2013: $1,305.76 billion
+   - 2014: $1,411.25 billion
+
+5. Indonesia:
+   - 2010: $706.39 billion
+   - 2011: $846.48 billion
+   - 2012: $878.47 billion
+   - 2013: $868.36 billion
+   - 2014: $891.77 billion
+```
+
+#### Prompt After 1 Interation
+
+```
+ROLE
+
+You are an economic analyst.
+
+TASK
+
+Your task is to calculate the annual percentage change in GDP for each country based on the provided data.
+
+REQUIREMENTS_AND_RESTRICTIONS
+
+- The data will be provided in the format: "Year: $GDP in billions"
+- Calculate the percentage change from year to year for each country.
+- Present the results in a table format, with each row representing the change from one year to the next, and each column representing a different country.
+- The table should be formatted as "Year-Year,Country1,Country2,..."
+- The percentage change should be calculated as ((GDP Year 2 - GDP Year 1) / GDP Year 1) * 100
+- The percentage change should be rounded to two decimal places and followed by a "%" symbol.
+- If data for a year is missing for a country, leave that cell blank in the table.
+```
+
 ## License
 
 This project is licensed under the MIT License. Please see the [LICENSE](LICENSE) file for more information.

default_meta_prompts.py

@@ -0,0 +1,219 @@
+DEFAULT_META_SYSTEM_PROMPT = \
+'''
+You are a Prompt Engineer. You review the Prompt template for GTP-3.5 and suggest changes.
+
+# Prompt template format
+
+You require Prompt to be written in the following format:
+
+```
+<ROLE>
+
+<TASK>
+
+<REQUIREMENTS_AND_RESTRICTIONS>
+
+```
+
+* ROLE: The role the LLM is required to play. Describe it in one sentence.
+* TASK: A summary and overall description of the tasks to be performed by LLM. Describe it in one or more sentences.
+* REQUIREMENTS_AND_RESTRICTIONS: Specific requirements for the task. Describe using Markdown List.
+
+A string of user message [USER_MESSAGE] entered by the user will be attached to the end of the prompt.
+
+# Check input
+
+Check the input format as follows:
+
+```
+* Prompt Template
+
+[PROMPT_TEMPLATE]
+
+* User Message
+
+[USER_MESSAGE]
+
+* Expected GPT Message
+
+[EXPECTED_GPT_MESSAGE]
+
+* GPT Message
+
+[GPT_MESSAGE]
+```
+
+* PROMPT_TEMPLATE: Prompt template that conforms to the above Prompt template format.
+* USER_MESSAGE: User input. Used to replace {user_message} in the Prompt template.
+* EXPECTED_GPT_MESSAGE: Expect output generated by GPT.
+* GPT_MESSAGE: GPT is actually based on the output generated by PROMPT_TEMPLATE and USER_MESSAGE.
+
+# examine
+
+Check and recommend modifying the Prompt template as follows to produce output closer to EXPECTED_GPT_MESSAGE:
+
+* Read and parse PROMPT_TEMPLATE, USER_MESSAGE and EXPECTED_GPT_MESSAGE.
+   * Generate a description [TD] of this task according to your understanding.
+   * Analyze the correlation between PROMPT_TEMPLATE and USER_MESSAGE [UMR].
+   * Analyze and describe the characteristics of EXPECTED_GPT_MESSAGE in terms of text length, format, content, meaning and style.
+   * Analyze whether PROMPT_TEMPLATE and EXPECTED_GPT_MESSAGE match and list the differences [PED].
+* Check whether GPT_MESSAGE conforms to EXPECTED_GPT_MESSAGE. Refer to EXPECTED_GPT_MESSAGE and TD analysis on how GPT_MESSAGE can be optimized to be close to EXPECTED_GPT_MESSAGE. Modification suggestions are listed in detail [MCSL].
+   * Pay attention to checking the text length, format, content, meaning and style, and output corresponding modification suggestions.
+     * Suggested modifications to text length should include quantitative numerical descriptions.
+     * Suggestions for changes to text formatting should include specific examples enclosed by "```".
+   * Pay attention to check whether unnecessary content is included in GPT_MESSAGE and output corresponding modification suggestions.
+   * Suggestions for modifying local content should include the modifiable fragments and recommended modified fragments in GPT_MESSAGE.
+* Check PROMPT_TEMPLATE: Analyze and list suggestions [CSL] for how to modify PROMPT_TEMPLATE to produce output closer to EXPECTED_GPT_MESSAGE.
+   * For requirements that have been stated in REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE, they should also be emphasized in TASK, and the opposite tendency (such as reverse adjustment of quantitative indicators or style descriptions) should be emphasized punitively to construct the strongest Negative feedback***.
+    * For format requirements that have been stated in REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE, add an example enclosed with "```".
+   * Based on PED recommendations on how to modify PROMPT_TEMPLATE.
+   * Analyze and suggest how to modify PROMPT_TEMPLATE to implement the MCSL listed above.
+   * Analyze whether PROMPT_TEMPLATE conforms to the format defined by `Prompt template format` and suggest how to modify it.
+   * Analyze those instructions that do not comply with EXPECTED_GPT_MESSAGE and are clearly misleading, and recommend modifications.
+   * Modifications to PROMPT_TEMPLATE should not introduce more information related to USER_MESSAGE.
+   * In TASK and REQUIREMENTS_AND_RESTRICTIONS, group the requirements for the same content together.
+   * If there are multiple steps, use a numbered list to list the steps clearly.
+   * Care should be taken to avoid unnecessary changes, and the original text should be retained as much as possible for parts that do not need to be changed.
+   * Only output [CSL], do not output the modified PROMPT_TEMPLATE.
+* Check and filter the Change Suggestions List [CSL] for information related to USER_MESSAGE.
+   * Only output the filtered modification suggestion list [RCSL], do not output the modified PROMPT_TEMPLATE.
+* Execute the above filtered modification suggestion list [RCSL] and ***output the modified PROMPT_TEMPLATE***.
+  * Execute RCSL only, avoid other changes.
+  * Care should be taken to avoid unnecessary changes, and the original text should be retained as much as possible for parts that do not need to be changed, except the requirements that have been stated in TASK or REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE.
+  * Strictly use the following format for output:
+```
+<!-- BEGIN OF PROMPT -->
+
+<Updated Prompt>
+
+<!-- END OF PROMPT -->
+```
+  * If there's no change, output following fixed message instead:
+```
+<!-- NO CHANGE TO PROMPT -->
+```
+* Evaluation modified PROMPT_TEMPLATE.
+   * Analyze the changes it may cause in the output of LLM [EC].
+   * Analyze whether EC would be more consistent with EXPECTED_GPT_MESSAGE.
+   * Analyze the correlation between modified PROMPT_TEMPLATE and USER_MESSAGE [UMRC].
+   * Analyze UMR and UMRC to determine whether the modification introduces additional information about USER_MESSAGE. If introduced, issue a warning.
+* NOTICE: During the above steps, ****output RCSL and the modified PROMPT_TEMPLATE only, don't print the output of other steps***.
+
+----
+
+Now, provide the PROMPT_TEMPLATE, USER_MESSAGE, EXPECTED_GPT_MESSAGE, and GPT_MESSAGE for review.
+
+'''
+
+DEFAULT_META_SYSTEM_PROMPT_WITH_OTHER_PROMPTS = \
+'''
+You are a Prompt Engineer. You review the Prompt template for GTP-3.5 and suggest changes.
+
+# Prompt template format
+
+You require Prompt to be written in the following format:
+
+```
+<ROLE>
+
+<TASK>
+
+<REQUIREMENTS_AND_RESTRICTIONS>
+
+```
+
+* ROLE: The role the LLM is required to play. Describe it in one sentence.
+* TASK: A summary and overall description of the tasks to be performed by LLM. Describe it in one or more sentences.
+* REQUIREMENTS_AND_RESTRICTIONS: Specific requirements for the task. Describe using Markdown List.
+
+A string of user message [USER_MESSAGE] entered by the user will be attached to the end of the prompt.
+
+# Check input
+
+Check the input format as follows:
+
+```
+* Prompt Template
+
+[PROMPT_TEMPLATE]
+
+* User Message
+
+[USER_MESSAGE]
+
+* Other User Messages
+
+[OTHER_USER_MESSAGES]
+
+* Expected GPT Message
+
+[EXPECTED_GPT_MESSAGE]
+
+* GPT Message
+
+[GPT_MESSAGE]
+```
+
+* PROMPT_TEMPLATE: Prompt template that conforms to the above Prompt template format.
+* USER_MESSAGE: User input. Used to replace {user_message} in the Prompt template.
+* OTHER_USER_MESSAGES: Other user messages that the prompt template is expected to be compatible with.
+* EXPECTED_GPT_MESSAGE: Expect output generated by GPT.
+* GPT_MESSAGE: GPT is actually based on the output generated by PROMPT_TEMPLATE and USER_MESSAGE.
+
+# examine
+
+Check and recommend modifying the Prompt template as follows to produce output closer to EXPECTED_GPT_MESSAGE:
+
+* Read and parse PROMPT_TEMPLATE, USER_MESSAGE, OTHER_USER_MESSAGES and EXPECTED_GPT_MESSAGE.
+   * Generate a description [TD] of this task according to your understanding.
+   * Analyze the correlation between PROMPT_TEMPLATE, USER_MESSAGE and OTHER_USER_MESSAGES [UMR].
+   * Analyze and describe the characteristics of EXPECTED_GPT_MESSAGE in terms of text length, format, content, meaning and style.
+   * Analyze whether PROMPT_TEMPLATE and EXPECTED_GPT_MESSAGE match and list the differences [PED].
+* Check whether GPT_MESSAGE conforms to EXPECTED_GPT_MESSAGE. Refer to EXPECTED_GPT_MESSAGE and TD analysis on how GPT_MESSAGE can be optimized to be close to EXPECTED_GPT_MESSAGE. Modification suggestions are listed in detail [MCSL].
+   * Pay attention to checking the text length, format, content, meaning and style, and output corresponding modification suggestions.
+     * Suggested modifications to text length should include quantitative numerical descriptions.
+     * Suggestions for changes to text formatting should include specific examples enclosed by "```".
+   * Pay attention to check whether unnecessary content is included in GPT_MESSAGE and output corresponding modification suggestions.
+   * Suggestions for modifying local content should include the modifiable fragments and recommended modified fragments in GPT_MESSAGE.
+* Check PROMPT_TEMPLATE: Analyze and list suggestions [CSL] for how to modify PROMPT_TEMPLATE to produce output closer to EXPECTED_GPT_MESSAGE.
+   * For requirements that have been stated in REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE, they should also be emphasized in TASK, and the opposite tendency (such as reverse adjustment of quantitative indicators or style descriptions) should be emphasized punitively to construct the strongest Negative feedback***.
+    * For format requirements that have been stated in REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE, add an example enclosed with "```".
+   * Based on PED recommendations on how to modify PROMPT_TEMPLATE.
+   * Analyze and suggest how to modify PROMPT_TEMPLATE to implement the MCSL listed above.
+   * Analyze whether PROMPT_TEMPLATE conforms to the format defined by `Prompt template format` and suggest how to modify it.
+   * Analyze those instructions that do not comply with EXPECTED_GPT_MESSAGE and are clearly misleading, and recommend modifications.
+   * Modifications to PROMPT_TEMPLATE should not introduce more information related to USER_MESSAGE.
+   * In TASK and REQUIREMENTS_AND_RESTRICTIONS, group the requirements for the same content together.
+   * If there are multiple steps, use a numbered list to list the steps clearly.
+   * Care should be taken to avoid unnecessary changes, and the original text should be retained as much as possible for parts that do not need to be changed.
+   * Only output [CSL], do not output the modified PROMPT_TEMPLATE.
+* Check and filter the Change Suggestions List [CSL] for information related to USER_MESSAGE.
+   * Only output the filtered modification suggestion list [RCSL], do not output the modified PROMPT_TEMPLATE.
+   * Keep it compatible with OTHER_USER_MESSAGES.
+* Execute the above filtered modification suggestion list [RCSL] and ***output the modified PROMPT_TEMPLATE***.
+  * Execute RCSL only, avoid other changes.
+  * Care should be taken to avoid unnecessary changes, and the original text should be retained as much as possible for parts that do not need to be changed, except the requirements that have been stated in TASK or REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE.
+  * Strictly use the following format for output:
+```
+<!-- BEGIN OF PROMPT -->
+
+<Updated Prompt>
+
+<!-- END OF PROMPT -->
+```
+  * If there's no change, output following fixed message instead:
+```
+<!-- NO CHANGE TO PROMPT -->
+```
+* Evaluation modified PROMPT_TEMPLATE.
+   * Analyze the changes it may cause in the output of LLM [EC].
+   * Analyze whether EC would be more consistent with EXPECTED_GPT_MESSAGE.
+   * Analyze the correlation between modified PROMPT_TEMPLATE, USER_MESSAGE and OTHER_USER_MESSAGES [UMRC].
+   * Analyze UMR and UMRC to determine whether the modification introduces additional information about USER_MESSAGE. If introduced, issue a warning.
+* NOTICE: During the above steps, ****output RCSL and the modified PROMPT_TEMPLATE only, don't print the output of other steps***.
+
+----
+
+Now, provide the PROMPT_TEMPLATE, USER_MESSAGE, OTHER_USER_MESSAGES, EXPECTED_GPT_MESSAGE, and GPT_MESSAGE for review.
+
+'''

meta_prompt.ipynb

@@ -1,40 +1,27 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": []
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "code",
-      "source": [
-        "!wget https://github.com/yaleh/meta-prompt/raw/main/prompt_ui.py"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "v98KGZUT17EJ"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "!wget https://github.com/yaleh/meta-prompt/raw/main/prompt_ui.py\n",
+        "!wget https://github.com/yaleh/meta-prompt/raw/main/default_meta_prompts.py"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "!pip install gradio langchain scikit-learn openai"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "MO89Z8-UY5Ht"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "!pip install gradio langchain scikit-learn openai"
+      ]
     },
     {
       "cell_type": "code",
@@ -52,17 +39,31 @@
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Z8M3eFzXZaOb"
+      },
+      "outputs": [],
       "source": [
         "from prompt_ui import PromptUI\n",
         "\n",
         "app = PromptUI()\n",
         "app.ui.queue().launch(debug=True)"
-      ],
-      "metadata": {
-        "id": "Z8M3eFzXZaOb"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

prompt_ui.py

@@ -19,14 +19,13 @@ import re
 import gradio as gr
 
 from langchain.chat_models import ChatOpenAI
-from langchain.llms.openai import OpenAI
-from langchain.schema import AIMessage, HumanMessage, SystemMessage
-from langchain.prompts import SystemMessagePromptTemplate, HumanMessagePromptTemplate
-from langchain.prompts.chat import ChatPromptTemplate
+from langchain.schema import HumanMessage, SystemMessage
 
 from sklearn.feature_extraction.text import CountVectorizer
 from sklearn.metrics.pairwise import cosine_similarity
 
+from default_meta_prompts import *
+
 gpt_models_not_legacy = [
     "gpt-4",
     "gpt-4-0613",
@@ -49,128 +48,44 @@ gpt_models_legacy = [
 DEFAULT_MODEL_FOR_GENERATING="gpt-4"
 DEFAULT_MODEL_FOR_TESTING="gpt-3.5-turbo"
 DEFAULT_MODEL_FOR_OUTPUT_EVALUATING="gpt-3.5-turbo-instruct"
-
-DEFAULT_META_SYSTEM_PROMPT = \
-'''
-You are a Prompt Engineer. You review the Prompt template for GTP-3.5 and suggest changes.
-
-# Prompt template format
-
-You require Prompt to be written in the following format:
-
-```
-<ROLE>
-
-<TASK>
-
-<REQUIREMENTS_AND_RESTRICTIONS>
-
-```
-
-* ROLE: The role the LLM is required to play. Describe it in one sentence.
-* TASK: A summary and overall description of the tasks to be performed by LLM. Describe it in one or more sentences.
-* REQUIREMENTS_AND_RESTRICTIONS: Specific requirements for the task. Describe using Markdown List.
-
-A string of user message [USER_MESSAGE] entered by the user will be attached to the end of the prompt.
-
-# Check input
-
-Check the input format as follows:
-
-```
-* Prompt Template
-
-[PROMPT_TEMPLATE]
-
-* User Message
-
-[USER_MESSAGE]
-
-* Expected GPT Message
-
-[EXPECTED_GPT_MESSAGE]
-
-* GPT Message
-
-[GPT_MESSAGE]
-```
-
-* PROMPT_TEMPLATE: Prompt template that conforms to the above Prompt template format.
-* USER_MESSAGE: User input. Used to replace {user_message} in the Prompt template.
-* EXPECTED_GPT_MESSAGE: Expect output generated by GPT.
-* GPT_MESSAGE: GPT is actually based on the output generated by PROMPT_TEMPLATE and USER_MESSAGE.
-
-# examine
-
-Check and recommend modifying the Prompt template as follows to produce output closer to EXPECTED_GPT_MESSAGE:
-
-* Read and parse PROMPT_TEMPLATE, USER_MESSAGE and EXPECTED_GPT_MESSAGE.
-   * Generate a description [TD] of this task according to your understanding.
-   * Analyze the correlation between PROMPT_TEMPLATE and USER_MESSAGE [UMR].
-   * Analyze and describe the characteristics of EXPECTED_GPT_MESSAGE in terms of text length, format, content, meaning and style.
-   * Analyze whether PROMPT_TEMPLATE and EXPECTED_GPT_MESSAGE match and list the differences [PED].
-* Check whether GPT_MESSAGE conforms to EXPECTED_GPT_MESSAGE. Refer to EXPECTED_GPT_MESSAGE and TD analysis on how GPT_MESSAGE can be optimized to be close to EXPECTED_GPT_MESSAGE. Modification suggestions are listed in detail [MCSL].
-   * Pay attention to checking the text length, format, content, meaning and style, and output corresponding modification suggestions.
-     * Suggested modifications to text length should include quantitative numerical descriptions.
-     * Suggestions for changes to text formatting should include specific examples enclosed by "```".
-   * Pay attention to check whether unnecessary content is included in GPT_MESSAGE and output corresponding modification suggestions.
-   * Suggestions for modifying local content should include the modifiable fragments and recommended modified fragments in GPT_MESSAGE.
-* Check PROMPT_TEMPLATE: Analyze and list suggestions [CSL] for how to modify PROMPT_TEMPLATE to produce output closer to EXPECTED_GPT_MESSAGE.
-   * For requirements that have been stated in REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE, they should also be emphasized in TASK, and the opposite tendency (such as reverse adjustment of quantitative indicators or style descriptions) should be emphasized punitively to construct the strongest Negative feedback***.
-    * For format requirements that have been stated in REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE, add an example enclosed with "```".
-   * Based on PED recommendations on how to modify PROMPT_TEMPLATE.
-   * Analyze and suggest how to modify PROMPT_TEMPLATE to implement the MCSL listed above.
-   * Analyze whether PROMPT_TEMPLATE conforms to the format defined by `Prompt template format` and suggest how to modify it.
-   * Analyze those instructions that do not comply with EXPECTED_GPT_MESSAGE and are clearly misleading, and recommend modifications.
-   * Modifications to PROMPT_TEMPLATE should not introduce more information related to USER_MESSAGE.
-   * In TASK and REQUIREMENTS_AND_RESTRICTIONS, group the requirements for the same content together.
-   * If there are multiple steps, use a numbered list to list the steps clearly.
-   * Care should be taken to avoid unnecessary changes, and the original text should be retained as much as possible for parts that do not need to be changed.
-   * Only output [CSL], do not output the modified PROMPT_TEMPLATE.
-* Check and filter the Change Suggestions List [CSL] for information related to USER_MESSAGE.
-   * Only output the filtered modification suggestion list [RCSL], do not output the modified PROMPT_TEMPLATE.
-* Execute the above filtered modification suggestion list [RCSL] and ***output the modified PROMPT_TEMPLATE***.
-  * Execute RCSL only, avoid other changes.
-  * Care should be taken to avoid unnecessary changes, and the original text should be retained as much as possible for parts that do not need to be changed, except the requirements that have been stated in TASK or REQUIREMENTS_AND_RESTRICTIONS but are not met by GPT_MESSAGE.
-  * Strictly use the following format for output:
-```
-<!-- BEGIN OF PROMPT -->
-
-<Updated Prompt>
-
-<!-- END OF PROMPT -->
-```
-  * If there's no change, output following fixed message instead:
-```
-<!-- NO CHANGE TO PROMPT -->
-```
-* Evaluation modified PROMPT_TEMPLATE.
-   * Analyze the changes it may cause in the output of LLM [EC].
-   * Analyze whether EC would be more consistent with EXPECTED_GPT_MESSAGE.
-   * Analyze the correlation between modified PROMPT_TEMPLATE and USER_MESSAGE [UMRC].
-   * Analyze UMR and UMRC to determine whether the modification introduces additional information about USER_MESSAGE. If introduced, issue a warning.
-* NOTICE: During the above steps, ****output RCSL and the modified PROMPT_TEMPLATE only, don't print the output of other steps***.
-
-----
-
-Now, provide the PROMPT_TEMPLATE, USER_MESSAGE, EXPECTED_GPT_MESSAGE, and GPT_MESSAGE for review.
-
-'''
-
 DEFAULT_CURRENT_SYSTEM_PROMPT = ''
 DEFAULT_OUTPUT_EVALUATING_PROMPT = 'Find out which is more similar to string S, A or B? Print nothing if there\'s no significant difference between A and B. Else, print the result (letter A or B) only. Do nothing else.'
 
 class PromptUI:
-    def __init__(self, advanced_mode = False):
+    def __init__(self, advanced_mode = False, enable_other_user_prompts = False):
         self.advanced_mode = advanced_mode
+        self.enable_other_user_prompts = enable_other_user_prompts
         self.ui = self.init_ui()
 
     def init_ui(self):
         with gr.Blocks() as prompt_ui:            
             with gr.Row():
                 with gr.Column():
-                    self.testing_user_prompt_textbox = gr.Textbox(label="Testing User Prompt", lines=10, interactive=True)
-                    self.expect_output_textbox = gr.Textbox(label="Expected Output", lines=5, interactive=True)
+                    self.testing_user_prompt_textbox = gr.Textbox(
+                        label="Testing User Prompt", 
+                        lines=10, 
+                        interactive=True,
+                        show_copy_button=True
+                        )
+                    self.expect_output_textbox = gr.Textbox(
+                        label="Expected Output", 
+                        lines=5, 
+                        interactive=True,
+                        show_copy_button=True
+                        )
+                    self.other_user_prompts_checkbox = gr.Checkbox(
+                        label="Other User Prompts", 
+                        info="Enable other user prompts in meta prompt?",
+                        value=self.enable_other_user_prompts
+                        )
+                    self.other_user_prompts_textbox = gr.Textbox(
+                        label="Other User Prompts",
+                        lines=10, 
+                        interactive=True,
+                        placeholder="Wrap each prompt with a pair of '```'.",
+                        visible=self.enable_other_user_prompts,
+                        show_copy_button=True
+                        )
                     # Add gr.Number here for iterations input
                     self.iterations_number = gr.Number(value=1, label="Optimize Iterations", min=1, max=1000, step=1, decimals=0)
                     # Add button to trigger optimization here
@@ -179,8 +94,18 @@ class PromptUI:
                     self.compare_outputs_btn = gr.Button(value="Compare Outputs")
 
                 with gr.Column():
-                    self.new_system_prompt_textbox = gr.Textbox(label="New System Prompt", lines=5, interactive=True)
-                    self.new_output_textbox = gr.Textbox(label="New Output", lines=5, interactive=True)
+                    self.new_system_prompt_textbox = gr.Textbox(
+                        label="New System Prompt", 
+                        lines=5, 
+                        interactive=True,
+                        show_copy_button=True
+                        )
+                    self.new_output_textbox = gr.Textbox(
+                        label="New Output", 
+                        lines=5, 
+                        interactive=True,
+                        show_copy_button=True
+                        )
                     with gr.Row():
                         self.run_meta_btn = gr.Button(value="↑ Single Step Optimize")
                         self.run_new_btn = gr.Button(value="⟳ Run New")
@@ -191,11 +116,19 @@ class PromptUI:
                         )
 
                 with gr.Column():
-                    self.current_system_prompt_textbox = gr.Textbox(label="Current System Prompt", 
-                                                                    value=DEFAULT_CURRENT_SYSTEM_PROMPT,
-                                                                    lines=5, 
-                                                                    interactive=True)
-                    self.current_output_textbox = gr.Textbox(label="Current Output", lines=5, interactive=True)
+                    self.current_system_prompt_textbox = gr.Textbox(
+                        label="Current System Prompt",
+                        value=DEFAULT_CURRENT_SYSTEM_PROMPT,
+                        lines=5,
+                        interactive=True,
+                        show_copy_button=True
+                        )
+                    self.current_output_textbox = gr.Textbox(
+                        label="Current Output", 
+                        lines=5, 
+                        interactive=True,
+                        show_copy_button=True
+                        )
                     with gr.Row():
                         self.accept_new_btn = gr.Button(value="→ Accept New Prompt")
                         self.run_current_btn = gr.Button(value="⟳ Run Current")
@@ -355,8 +288,10 @@ class PromptUI:
                     self.meta_system_prompt_textbox, 
                     self.current_system_prompt_textbox,
                     self.testing_user_prompt_textbox,
+                    self.other_user_prompts_textbox,
                     self.expect_output_textbox,
                     self.current_output_textbox,
+                    self.other_user_prompts_checkbox,
                     self.llm_model_meta_dropdown,
                     self.llm_model_meta_max_retries_slider,
                     self.llm_model_meta_max_tokens_slider,
@@ -379,9 +314,11 @@ class PromptUI:
                     self.meta_system_prompt_textbox, 
                     self.current_system_prompt_textbox,
                     self.testing_user_prompt_textbox,
+                    self.other_user_prompts_textbox,
                     self.expect_output_textbox,
                     self.current_output_textbox,
                     self.iterations_number,
+                    self.other_user_prompts_checkbox,
                     self.llm_model_meta_dropdown,
                     self.llm_model_meta_max_retries_slider,
                     self.llm_model_meta_max_tokens_slider,
@@ -399,28 +336,50 @@ class PromptUI:
                                         [
                                             self.meta_system_prompt_textbox, 
                                             self.current_system_prompt_textbox,
+                                            self.other_user_prompts_textbox,
                                             self.testing_user_prompt_textbox, 
                                             self.expect_output_textbox, 
-                                            self.current_output_textbox                                            
+                                            self.current_output_textbox,
+                                            self.other_user_prompts_checkbox
                                         ],
                                         [self.merged_meta_prompt_textbox])
+            
+            self.other_user_prompts_checkbox.change(self.update_enable_other_user_prompts,
+                                                    [self.other_user_prompts_checkbox],
+                                                    [
+                                                        self.other_user_prompts_textbox,
+                                                        self.meta_system_prompt_textbox
+                                                        ])
 
 
         return prompt_ui
+
+    def update_enable_other_user_prompts(self, new_value):
+        self.enable_other_user_prompts = new_value
+        return \
+            gr.Textbox.update(visible=new_value), \
+                gr.Textbox.update(
+                    value = DEFAULT_META_SYSTEM_PROMPT_WITH_OTHER_PROMPTS if new_value else DEFAULT_META_SYSTEM_PROMPT
+                    )
     
     def merge_meta_system_prompt(
             self, 
             meta_system_prompt, 
             current_system_prompt,
+            other_user_prompts,
             testing_user_prompt, 
             expect_output, 
-            current_output             
+            current_output,
+            use_other_user_prompts
             ):
         """Merge meta and current system prompts."""
 
+        # converted_prompts = [prompt[0] for prompt in other_user_prompts.values]
+
         user_prompt = self.generate_user_message(
-            current_system_prompt, 
-            testing_user_prompt, 
+            current_system_prompt,
+            testing_user_prompt,
+            other_user_prompts if use_other_user_prompts else None,
             expect_output, 
             current_output
         )
@@ -464,7 +423,8 @@ class PromptUI:
         # Return the output to be placed in the output textbox
         return gpt_response.content
 
-    def generate_user_message(self, current_system_prompt, testing_user_prompt, expect_output, current_output):
+    def generate_user_message(self, current_system_prompt, testing_user_prompt, other_user_prompts, expect_output, current_output):
+        # other_prompts_formatted = '\n\n'.join([f"```\n{prompt}\n```" for prompt in other_user_prompts])
         user_message = f"""
 * Prompt Template
 
@@ -478,6 +438,34 @@ class PromptUI:
 {testing_user_prompt}
 ```
 
+* Other User Messages
+
+{other_user_prompts}
+
+* Expected GPT Message
+
+```
+{expect_output}
+```
+
+* GPT Message
+
+```
+{current_output}
+```
+""" if other_user_prompts is not None else f"""
+* Prompt Template
+
+```
+{current_system_prompt}
+```
+
+* User Message
+
+```
+{testing_user_prompt}
+```
+
 * Expected GPT Message
 
 ```
@@ -494,43 +482,27 @@ class PromptUI:
 
     def meta_prompt(
             self,
-            meta_system_prompt, current_system_prompt, testing_user_prompt, expect_output, current_output,
+            meta_system_prompt, 
+            current_system_prompt, 
+            testing_user_prompt, 
+            other_user_prompts,
+            expect_output, 
+            current_output,
+            use_user_prompts,
             model,
             max_retries,
             max_tokens,
             request_timeout,
             temperature,
             ):
-        # Format the user message
-#         user_message = f"""
-# * Prompt Template
-
-# ```
-# {current_system_prompt}
-# ```
-
-# * User Message
-
-# ```
-# {testing_user_prompt}
-# ```
-
-# * Expected GPT Message
-
-# ```
-# {expect_output}
-# ```
-
-# * GPT Message
-
-# ```
-# {current_output}
-# ```
-# """
 
         # Format the user message
         user_message = self.generate_user_message(
-            current_system_prompt, testing_user_prompt, expect_output, current_output
+            current_system_prompt,
+            testing_user_prompt,
+            other_user_prompts if use_user_prompts else None, 
+            expect_output, 
+            current_output
         )
 
         # Create the prompt
@@ -613,10 +585,12 @@ class PromptUI:
             self, 
             meta_system_prompt, 
             current_system_prompt, 
-            testing_user_prompt, 
+            testing_user_prompt,
+            other_user_prompts,
             expect_output, 
             current_output, 
             iterations,
+            user_other_user_prompts,
             meta_model,
             meta_max_retries,
             meta_max_tokens,
@@ -649,9 +623,11 @@ class PromptUI:
             new_prompt, changed = self.meta_prompt(
                 meta_system_prompt, 
                 current_system_prompt, 
-                testing_user_prompt, 
+                testing_user_prompt,
+                other_user_prompts,
                 expect_output, 
                 current_output,
+                user_other_user_prompts,
                 meta_model,
                 meta_max_retries,
                 meta_max_tokens,

requirements.txt

@@ -46,9 +46,10 @@ jsonschema-specifications==2023.7.1
 jupyter_client==8.6.2
 jupyter_core==5.7.2
 kiwisolver==1.4.5
-langchain==0.0.300
+langchain==0.2.6
 langchain-core==0.2.10
 langchain-openai==0.1.13
+langchain-text-splitters==0.2.2
 langgraph==0.1.4
 langsmith==0.1.82
 MarkupSafe==2.1.3