[UI] - Title of conversations with gibberish or...

#685
by robb-0 - opened

Regardless of model chosen, before the end of their output, all models named the conversations with what looks like gibberish:

I used the same prompt as a test. I did NOT change the system prompt tab, models which have it blank remained blank. And Qwen models which come with their default system prompt, were kept untouched.

"I have a 6- and a 12-liter jug. I want to measure exactly 6 liters."

"💬 副 conver terms angry listeningС финансов brig fix ideologies成 coefficients upro mainstream medium" - Model: QwQ 32B - Has system prompt by default

"💬 Although finds McK finds McK finds McK tends CMP McK finds McK finds McK finds" - Llama 3.3 70B

"💬 It looks like you've forgotten that you have a working solution that depends on" - Qwen 2.5 72B

"💬 diagigosJobs diagigosJobs diagigos diagigos diagigos diagigos diag" - Command R Plus 08-2024

"💬 Here is the complete correct response:

(```) note: brackets by me to avoid formatting this post.
//Summary actionshere$" - NVidia Nemotron 70B

Plus, which is the agent which handles the websearch if not the model we're interacting with? As the agent refuses to tell about "New Things", including Llama - which is not their common behaviour. Also turning the websearch on enhances the gibberish placed in the title during the first iteration, and sometimes it does look like a chat bleeding from other users, as it returns sentences which are well structured on their own, while not having any connection with the current interaction.

Websearch on

"💬 You can help me summarize the error. The error is a client error." - Nvidia Nemotron 70B

"Summary: Overview of function call" - Qwen 72B

While maitaining same conditions as above, such as websearch on, switching prompt to

"How do I make a delicious lemon cheesecake?"

".*
)
.*
)
.*
)
.*
).
)
.*
)
.*
)." - Mistral-Nemo 2407

"💬 You want to summarize the prompt in the format { name: value }, correct" - Hermes-3

"💬 (waiting for the cute animal prompt to answer, not the food prompt)" - DS-R1-Distilled-Qwen-32B

"💬 возможноinosaur sexleму념ентов.
сили вообще.
возможно сторонбу" - Phi

Note: I copied and pasted here the 'sentence' in the titles. Surprisingly enough, many have extra formatting or breaks. The only one I had to format though is that one from Nemotron which opens a box with (```) All the others were unmodified.

This comment has been hidden
Hugging Chat org

hey thanks for the report, I can reproduce this will look into it

Hugging Chat org

This should be fixed! Let me know if it occurs again btw

This comment has been hidden
robb-0 changed discussion status to closed
robb-0 changed discussion status to open

@nsarrazin
The generated titles are not very good. For example if my first message is "How are you?" the title becomes "Doing well thank you!" instead of something like "Checking Your Status"
Sometimes it is just a single emoji with no words.

Maybe you could use a prompt like this ?
Give an emoji and a 2-5 word title for the following message. Reply only with the emoji and title:
user message here

Hugging Chat org

@miraclemind we already have the following prompt You are a summarization AI. Summarize the user's request into a single short sentence of four words or less. Do not try to answer it, only summarize the user's query. Always start your answer with an emoji relevant to the summary the problem is that we use a small model for this task and sometimes it hallucinates 😅

Sign up or log in to comment