|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
- zh |
|
base_model: |
|
- mistralai/Mistral-Small-24B-Base-2501 |
|
tags: |
|
- axolotl |
|
datasets: |
|
- allenai/tulu-3-sft-personas-instruction-following |
|
- simplescaling/s1K-1.1 |
|
- simplescaling/s1K-claude-3-7-sonnet |
|
- reedmayhew/medical-o1-reasoning-SFT-jsonl |
|
- OpenCoder-LLM/opc-sft-stage1 |
|
- PocketDoc/Dans-Kinomaxx-VanillaBackrooms |
|
- cognitivecomputations/SystemChat-2.0 |
|
- anthracite-org/kalo-opus-instruct-22k-no-refusal |
|
- allura-org/scienceqa_sharegpt |
|
--- |
|
# Sisyphus 24b |
|
Hundreds of dollars later. |
|
Dozens of failed finetunes. |
|
Sisyphus has balanced his rock on the summit. |
|
One must have imagined him happy while pushing. Now, he is ecstatic. |
|
|
|
 |
|
|
|
## About |
|
|
|
This is a pretty generic finetune of the 24b base model for multiturn ~~instruct~~ RP. It's pretty coherent across a range of temps, assuming you use something like min-p or top-p. It also supports reasoning blocks. |
|
|
|
## System Prompts |
|
|
|
I tested with the following Claude-like system prompts, however they were not trained in and any similar prompts can likely be used: |
|
|
|
### Non-Reasoning |
|
``` |
|
You are Claude, a helpful and harmless AI assistant created by Anthropic. |
|
``` |
|
|
|
### Reasoning |
|
``` |
|
You are Claude, a helpful and harmless AI assistant created by Anthropic. Please contain all your thoughts in <think> </think> tags, and your final response right after the closing </think> tag. |
|
``` |
|
|
|
For reasoning, it's recommended to force the thinking (by prefilling `<think>\n` on the newest assistant response), as well as not including previous thought blocks in new requests. |
|
|
|
## Instruct Template |
|
|
|
v7-Tekken, same as the original instruct model. |
|
|
|
## Dataset |
|
|
|
~~This model was trained on [allura-org/inkstructmix-v0.1](https://hf.co/datasets/allura-org/inkstructmix-v0.1).~~ |
|
|
|
Okay. So. |
|
|
|
It was *supposed* to be trained on the above dataset. |
|
|
|
However, *my* dumb ass. Put. The roleplay data. Into the config. And didn't notice |
|
|
|
 |
|
|
|
So. Um. I guess this is supposed to be a roleplaying model? It sure doesn't act the part. |
|
|
