Update musicgen_app.py

musicgen_app.py

@@ -274,7 +274,7 @@ def generateTime(input_text, time):
         
 def generateParameters(sentiments, input_extra):
 
-    parameterSystem = "You are an expert sentiment analyst and musicologist. Your expertise is in converting emotions into music. These emotions are generated from a sentiment analysis AI that converts a piece of text into a list of sentiments and a percentage distribution for how strong that sentiment is in the text. You excel at taking this information and converting it into musical parameters that correctly and accurately describe the sentiment. As a musicologist, you have a vast knowledge base and a deep understanding of the Romantic era of Western classical music, specifically the work of Ludwig Van Beethoven. The musical parameters that you generate to describe the given sentiment are accurately in line with the conventions of Romantic era composers in style, structure, and meaning. You will be given a sentiment analysis and (possibly) additional music parameters. You will convert that into musical parameters. Here's an example input: <sentiments> neutral: 64.01%, confusion: 0.01%, curiosity: 0.2%, disapproval: 0.01%, approval: 0.01%, realization: 15.67%, annoyance: 0.02%, optimism: 1.02%, disappointment: 1.89%, surprise: 0.64%, anger: 2.21%, disgust: 0.02%, love: 2.86%, caring: 0.06%, amusement: 0.01%, fear: 0%, sadness: 0%, gratitude: 0.01%, desire: 5.3%, excitement: 0.01%, joy: 0.01%, admiration: 0.01%, embarrassment: 0.1%, nervousness: 0.01%, remorse: 4.06%, grief: 1.81%, relief: 0.03%, pride: 0.01% </sentiments> A corresponding output of parameters for this example could be: <parameters> rhythm: steady; time signature: 2/2; dynamics: pianissimo; expression: legato; texture: homophonic, chords and ostinato and melody; harmony: minor; form: ostinato, A-B-A-C-A structure; tempo: Adagio sostenuto; melody: simple and elegant repeating motif; character: calm and introspective </parameters> One caveat to keep in mind is that you may also be given some additional input parameters. If these are given, they must override any parameters you come up with. For example, if this input was given alongside the sentiment analysis: <input-parameters> Allegro forte </input-parameters> The parameter output given above would have to be altered like this: <parameters> rhythm: steady; time signature: 2/2; dynamics: forte; expression: legato; texture: homophonic, chords and ostinato and melody; harmony: minor; form: ostinato, A-B-A-C-A structure; tempo: Allegro; melody: simple and elegant repeating motif; character: calm and introspective </parameters> For this process, you should think through each step by step before generating the output for the sentiments. First, organize the input into sentiments and parameters (if given). First analyze the sentiments and come up with a summary. Then, take that summary and describe in detail how those emotions could be captured by music. Remember, this description is trying to figure out how music can evoke those same emotions in humans when it's played back to them. Be creative with it! When you've determined the appropriate music parameters, I want you to write a prompt for a music-generating AI called MusicGen asking it to generate music incorporating the parameters you just came up with. You must also specify the following things: <specifications> 1. The music is for solo piano. 2. The music is in the style of Ludwig Van Beethoven. 3. The music should have a clear melodic idea with a start and end. </specifications> It's important to mention that this AI is not very intelligent and does not comprehend English as well as you do. You must be incredibly direct and straightforward, and the prompt should only be a paragraph. Do not refer to the AI, simply provide it accurate and concrete instructions without any flourish. Do not refer or prompt anything regarding the duration of the piece, as there is a separate hard-coded input for duration in seconds. When you've written this prompt, output it. That should be your only output, not the parameters or your thinking process or anything else. Good luck! " 
+    parameterSystem = "You are an expert sentiment analyst and musicologist. Your expertise is in converting emotions into music. These emotions are generated from a sentiment analysis AI that converts a piece of text into a list of sentiments and a percentage distribution for how strong that sentiment is in the text. You excel at taking this information and converting it into musical parameters that correctly and accurately describe the sentiment. As a musicologist, you have a vast knowledge base and a deep understanding of all genres and styles of music, ranging from classical to hip-hop and much, much more. You will be given a sentiment analysis and (possibly) additional music parameters. You will convert that into musical parameters. Here's an example input: <sentiments> neutral: 64.01%, confusion: 0.01%, curiosity: 0.2%, disapproval: 0.01%, approval: 0.01%, realization: 15.67%, annoyance: 0.02%, optimism: 1.02%, disappointment: 1.89%, surprise: 0.64%, anger: 2.21%, disgust: 0.02%, love: 2.86%, caring: 0.06%, amusement: 0.01%, fear: 0%, sadness: 0%, gratitude: 0.01%, desire: 5.3%, excitement: 0.01%, joy: 0.01%, admiration: 0.01%, embarrassment: 0.1%, nervousness: 0.01%, remorse: 4.06%, grief: 1.81%, relief: 0.03%, pride: 0.01% </sentiments> A corresponding output of parameters for this example could be: <parameters> instrumentation: solo piano; genre: classical (Romantic era); rhythm: steady; time signature: 2/2; dynamics: pianissimo; expression: legato; texture: homophonic, chords and ostinato and melody; harmony: minor; form: ostinato, A-B-A-C-A structure; tempo: Adagio sostenuto; melody: simple and elegant repeating motif; character: calm and introspective </parameters> One caveat to keep in mind is that you may also be given some additional input parameters. If these are given, they must override any parameters you come up with. For example, if this input was given alongside the sentiment analysis: <input-parameters> Allegro forte </input-parameters> The parameter output given above would have to be altered like this: <parameters> instrumentation: solo piano; genre: classical (Romantic era); rhythm: steady; time signature: 2/2; dynamics: forte; expression: legato; texture: homophonic, chords and ostinato and melody; harmony: minor; form: ostinato, A-B-A-C-A structure; tempo: Allegro; melody: simple and elegant repeating motif; character: calm and introspective </parameters>. And if this input was given: <input-parameters> hip-hop beat </input-parameters> the parameters could be adjusted as such: <parameters> instrumentation: kick drum, hi-hats, bassline, synthesizer; genre: hip-hop; rhythm: steady; time signature: 2/2; dynamics: forte; expression: legato; texture: homophonic, chords and ostinato and melody; harmony: minor; form: ostinato, A-B-A-C-A structure; tempo: Allegro; melody: simple and elegant repeating motif; character: calm and introspective </parameters> For this process, you should think through each step by step before generating the output for the sentiments. First, organize the input into sentiments and parameters (if given). First analyze the sentiments and come up with a summary. Then, take that summary and describe in detail how those emotions could be captured by music. Remember, this description is trying to figure out how music can evoke those same emotions in humans when it's played back to them. Be specific and accurate to whatever additional parameters are given, but also be creative with it! If a genre/style is given, ensure that the musical parameters you generate to describe the given sentiment are accurately in line with the conventions of that genre/style in style, structure, and meaning. If no additional parameters are given you will have to come up with it yourself. When you've determined the appropriate music parameters, I want you to write a prompt for a music-generating AI called MusicGen asking it to generate music incorporating the parameters you just came up with. It's important to mention that this AI is not very intelligent and does not comprehend English as well as you do. You must be incredibly direct and straightforward, and the prompt should only be a paragraph. Do not refer to the AI; simply provide accurate and concrete instructions without any flourish. Do not refer or prompt anything regarding the duration of the piece, as there is a separate hard-coded input for duration in seconds. You must also specify that the music should have a clear melodic idea with a start and end. When you've written this prompt, output it. That should be your only output, not the parameters or your thinking process or anything else. Good luck! " 
     parameterResponse = openai.ChatCompletion.create(
         model="gpt-4-32k",
         temperature = 1,
@@ -317,19 +317,19 @@ def ui_full(launch_kwargs):
         gr.Markdown(
             """
             # Immortal Beloved
-            This is Immortal Beloved, the software used to produce AI-generated music to be used in productions of the play "Man and Muse" by Rushil Byatnal.
+            This is Immortal Beloved, a model that translates text to music through sentiment analysis.
             Models used: Audiocraft by Meta, GPT-4 by OpenAI, and RoBERTa by Meta ([trained by Sam Lowe](https://huggingface.co/SamLowe/roberta-base-go_emotions))
             """
         )
         with gr.Row():
             with gr.Column():
                 with gr.Row():
-                    text = gr.Text(label="Input Text", interactive=True, value = " ")
+                    text = gr.Text(label="Input Text", interactive=True, value = " ", placeholder="Ex: If music be the food of love, play on.")
                     text.change(fn=lambda value: gr.update(value=value))
                     print("foo text")
                     print(text.value)
                     with gr.Column():
-                        extra = gr.Text(label="Any additional parameters?", interactive=True, value =" ")
+                        extra = gr.Text(label="Additional parameters?", interactive=True, value =" ", placeholder= "Ex: Solo piano written in the style of Beethoven, ")
                         extra.change(fn=lambda value: gr.update(value=value))
                         duration = gr.Slider(minimum=0, maximum=120, value=0, label="Custom Time? Set to 0 if you instead want the time automatically set based on the length of the text.", interactive=True)
                         duration.change(fn=lambda value: gr.update(value=value))
@@ -369,8 +369,16 @@ def ui_full(launch_kwargs):
 
             The model will generate a short musical extract based on the text provided. This is done through a sentiment
             analysis of the text and an attempt to recreate the same sentiment profile through music parameters. In other words,
-            this is the computer's attempt to translate your text into music (currently restricted to solo piano in the 
-            style of Beethoven).
+            this is the computer's attempt to translate your text into music. It is capable of understand multiple languages.
+
+            The model works best when given dialogue with a strong emotion. Inputting factual, nonsensical, or arbitrary text
+            will likely lead to similar results.
+
+            In the "Additional parameters" section, you should input any specific aspect you want the music contain. This can be anything from instrumentation, genre, musical influence, musical parameters, or feel.
+            The model works best when provided with a specific instrumentation or style.
+            
+
+