bartowski

/

RekaAI_reka-flash-3-GGUF

like 28

that's probably just my script parsing, I would assume you should use what they use, I'll edit my prompt :)

the main thing is they didn't specify one way or the other in their chat template

The template is actually quite odd, it's meant to stop when it sees <sep> but it's not marked as a special token or as an EOS token as far as I can tell?

That is kinda weird, I was wondering if my eyes were deceiving me but you're right:

"chat_template": "{% if messages[0]['role'] == 'system' %}{% set merged_content = messages[0]['content'] + ' ' + messages[1]['content'] %}{% set merged_messages = [{'role': messages[1]['role'], 'content': merged_content}] + messages[2:] %}{% else %}{% set merged_messages = messages %}{% endif %}{% for message in merged_messages %}{{message['role'] + ': ' + (message['content'].split('<reasoning>')[0] + message['content'].split('</reasoning>')[1] if message['role'] == 'assistant' and '</reasoning>' in message['content'] else message['content'])}}{% if (loop.last and add_generation_prompt and merged_messages[-1]['role'] != 'assistant') or not loop.last %}{{ ' <sep> ' }}{% endif %}{% endfor %}{% if add_generation_prompt and merged_messages[-1]['role'] != 'assistant' %}{{ 'assistant:' }}{% endif %}",

Where the heck is the user/human!

How are we in March 2025 and prompt templates are still not solved? It shouldn't be incomplete, missing, or as confusing as it still is. Now I'm wondering if "human" is a place-holder and can be swapped out for whatever, or this is exactly how it was fine-tuned.

Also, notice that round 1 is a bit different from round 2. For round 2 you would put the <sep> tag before human: if am reading their model card right.

This might make it tricky to set-up correctly for some of the GUI's.

And yeah you're right sep is not a special token, they mention it in the model card too:

Generation should stop on seeing the string <sep> or seeing the special token <|endoftext|>.

They're silly gooses.

I've been messing around with the model in ST, changing the human: out for hotdog: the model seems to see it as a name

It output this in the reasoning section
{{char}} has caught the user (hotdog)

@YearZero yeah the role is being defined here: {'role': messages[1]['role']

USUALLY it looks for the role "user" and then assigns it to whatever they actually want it to be, that's the accepted practice.. in this case, it's just going with whatever you set your role to, which is.. a decision

it makes it "more flexible" but imo likely at the cost of accuracy 🤷‍♂️

the bigger issue still is the lack of <sep> being a special token, I imagine a lot of UIs will struggle to stop on it since it will have to be manually set as a stop string, otherwise it'll endlessly generate

The template is actually quite odd, it's meant to stop when it sees but it's not marked as special token or as an EOS token as far as I can tell?

For a moment there, I thought I'm reading QwQ-32B reasoning... 🤣

LOL I didn't notice that when you write <sep> without wrapping it in ` it just removes it entirely from the text.. oh markdown, a gift and a curse

My prompt template seems broken. I get the following error: Failed to parse Jinja template: Expected closing parenthesis, got OpenSquareBracket instead
On this prompt template:

{% if messages[0]['role'] == 'system' %}{% set merged_content = messages[0]['content'] + ' ' + messages[1]['content'] %}{% set merged_messages = [{'role': messages[1]['role'], 'content': merged_content}] + messages[2:] %}{% else %}{% set merged_messages = messages %}{% endif %}{% for message in merged_messages %}{{message['role'] + ': ' + (message['content'].split('<reasoning>')[0] + message['content'].split('</reasoning>')[1] if message['role'] == 'assistant' and '</reasoning>' in message['content'] else message['content'])}}{% if (loop.last and add_generation_prompt and merged_messages[-1]['role'] != 'assistant') or not loop.last %}{{ ' <sep> ' }}{% endif %}{% endfor %}{% if add_generation_prompt and merged_messages[-1]['role'] != 'assistant' %}{{ 'assistant:' }}{% endif %}

My prompt template seems broken. I get the following error: Failed to parse Jinja template: Expected closing parenthesis, got OpenSquareBracket instead
On this prompt template:

{% if messages[0]['role'] == 'system' %}{% set merged_content = messages[0]['content'] + ' ' + messages[1]['content'] %}{% set merged_messages = [{'role': messages[1]['role'], 'content': merged_content}] + messages[2:] %}{% else %}{% set merged_messages = messages %}{% endif %}{% for message in merged_messages %}{{message['role'] + ': ' + (message['content'].split('<reasoning>')[0] + message['content'].split('</reasoning>')[1] if message['role'] == 'assistant' and '</reasoning>' in message['content'] else message['content'])}}{% if (loop.last and add_generation_prompt and merged_messages[-1]['role'] != 'assistant') or not loop.last %}{{ ' <sep> ' }}{% endif %}{% endfor %}{% if add_generation_prompt and merged_messages[-1]['role'] != 'assistant' %}{{ 'assistant:' }}{% endif %}

It IS broken. I copy and pasted the original template from json file and that's broken too! I asked grok 3 to fix it for me and it suggested this:

{# Ensure messages is not empty #}
{% if messages %}
  {# Merge system message with the first user message if applicable #}
  {% if messages|length > 1 and messages[0]['role'] == 'system' %}
    {% set merged_content = messages[0]['content'] + ' ' + messages[1]['content'] %}
    {% set merged_messages = [{'role': messages[1]['role'], 'content': merged_content}] + messages[2:] %}
  {% else %}
    {% set merged_messages = messages %}
  {% endif %}

  {# Process each message #}
  {% for message in merged_messages %}
    {# Handle assistant messages with <reasoning> tags #}
    {% if message['role'] == 'assistant' and '<reasoning>' in message['content'] and '</reasoning>' in message['content'] %}
      {% set content_parts = message['content'].split('<reasoning>') %}
      {% set content_after_reasoning = content_parts[1].split('</reasoning>') %}
      {{ message['role'] + ': ' + content_parts[0] + content_after_reasoning[1] }}
    {% else %}
      {{ message['role'] + ': ' + message['content'] }}
    {% endif %}
    {# Add separator if not the last message or if generation prompt is needed #}
    {% if (loop.last and add_generation_prompt and merged_messages[-1]['role'] != 'assistant') or not loop.last %}
      {{ ' <sep> ' }}
    {% endif %}
  {% endfor %}

  {# Add assistant prompt if needed #}
  {% if add_generation_prompt and merged_messages[-1]['role'] != 'assistant' %}
    {{ 'assistant:' }}
  {% endif %}
{% else %}
  {# Handle empty messages case #}
  {{ 'No messages provided.' }}
{% endif %}

You have to kinda take it with grain of salt. I mean, LM Studio doesn't complain anymore, but there's no guarantee if it actually works correctly for actual generation... 🤷‍♂️

LM Studio doesn't like [-1] for indexing in Jinja as seen with QwQ, but not sure if that's the only issue

LM Studio doesn't like [-1] for indexing in Jinja as seen with QwQ, but not sure if that's the only issue

Which is weird considering it doesn't complain about the template suggested by grok 3 and that also uses -1. 🤷‍♂️

They just fixed the prompt template on base (still has issues on <sep> in LMStudio for me but you can add it manually at least).
https://huggingface.co/RekaAI/reka-flash-3/commit/69cea64942e4db4809b757ae2b0d312b4b610263 .