File size: 2,255 Bytes
fa6d905 e70ea18 fa6d905 c38364d fa6d905 c38364d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
license: apache-2.0
language:
- en
- zh
base_model:
- mistralai/Mistral-Small-24B-Base-2501
tags:
- axolotl
datasets:
- allenai/tulu-3-sft-personas-instruction-following
- simplescaling/s1K-1.1
- simplescaling/s1K-claude-3-7-sonnet
- reedmayhew/medical-o1-reasoning-SFT-jsonl
- OpenCoder-LLM/opc-sft-stage1
- PocketDoc/Dans-Kinomaxx-VanillaBackrooms
- cognitivecomputations/SystemChat-2.0
- anthracite-org/kalo-opus-instruct-22k-no-refusal
- allura-org/scienceqa_sharegpt
---
# Sisyphus 24b
Hundreds of dollars later.
Dozens of failed finetunes.
Sisyphus has balanced his rock on the summit.
One must have imagined him happy while pushing. Now, he is ecstatic.
## About
This is a pretty generic finetune of the 24b base model for multiturn ~~instruct~~ RP. It's pretty coherent across a range of temps, assuming you use something like min-p or top-p. It also supports reasoning blocks.
## System Prompts
I tested with the following Claude-like system prompts, however they were not trained in and any similar prompts can likely be used:
### Non-Reasoning
```
You are Claude, a helpful and harmless AI assistant created by Anthropic.
```
### Reasoning
```
You are Claude, a helpful and harmless AI assistant created by Anthropic. Please contain all your thoughts in <think> </think> tags, and your final response right after the closing </think> tag.
```
For reasoning, it's recommended to force the thinking (by prefilling `<think>\n` on the newest assistant response), as well as not including previous thought blocks in new requests.
## Instruct Template
v7-Tekken, same as the original instruct model.
## Dataset
~~This model was trained on [allura-org/inkstructmix-v0.1](https://hf.co/datasets/allura-org/inkstructmix-v0.1).~~
Okay. So.
It was *supposed* to be trained on the above dataset.
However, *my* dumb ass. Put. The roleplay data. Into the config. And didn't notice
So. Um. I guess this is supposed to be a roleplaying model? It sure doesn't act the part.
|