Pinkstack/PARM-V2-QwQ-Qwen-2.5-o1-3B-GGUF

We are proud to announce, our new high quality flagship model series - PARM2, Very high quality reasoning, math and coding abilities for a small size, that anyone can run on their device for free.

🧀 Which quant is right for you?

• Q4: This model should be used on edge devices like high end phones or laptops due to its very compact size, quality is okay but fully usable.
• Q8: This model should be used on most high end modern devices like rtx 3080, Responses are very high quality, but its slightly slower than Q4.

This Parm v2 is based on Qwen 2.5 3B which has gotten many extra reasoning training parameters so it would have similar outputs to qwen QwQ / O.1 mini (only much, smaller.). We've trained it using the datasets here if you benchmarked this model let me know

⚠️ it may think it's name is Claude if you use our prompt format, due to the training data. we are sorry for this issue but is shouldn't effect the quality of the responses. If you use a chat ml prompt the quality of the responses would be lower but it won't have this issue.

This is a pretty lite model which can be run on high end phones pretty quickly using the q4 quant.

Passes "strawberry" test! (Q8 w/ msty & rtx 3080 10gb) ✅

To use this model, you must use a service which supports the GGUF file format. Additionaly, this is the Prompt Template: it uses the qwen2 template.

<thinking>

{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|>
{{- else if .Messages }}
{{- if or .System .Tools }}<|im_start|>system
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
# Tools
You may call one or more functions to assist with the user query.
You are provided with function signatures within <tools></tools> XML tags:
<tools>
{{- range .Tools }}
{"type": "function", "function": {{ .Function }}}
{{- end }}
</tools>
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>
{{- end }}<|im_end|>
{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
{{- if eq .Role "user" }}<|im_start|>user
{{ .Content }}<|im_end|>
{{- else if eq .Role "assistant" }}<|im_start|>assistant
{{ if .Content }}{{ .Content }}
{{- else if .ToolCalls }}<tool_call>
{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}</tool_call>
{{- end }}{{ if not $last }}<|im_end|>
{{ end }}
{{- else if eq .Role "tool" }}<|im_start|>user
<tool_response>
{{ .Content }}
</tool_response><|im_end|>
{{ end }}
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
{{ end }}
{{- end }}
{{- else }}
{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}

Or if you are using an anti prompt: <|end|><|assistant|>

Highly recommended to use with a system prompt. eg; You are a helpful assistant named Parm2 by Pinkstack. think step-by-step for complex stuff, use COT if neeed.

Uploaded model

• Developed by: Pinkstack
• License: apache-2.0
• Finetuned from model : Pinkstack/PARM-V1.5-QwQ-Qwen-2.5-o1-3B-VLLM

This AI model was trained with Unsloth and Huggingface's TRL library.