|
--- |
|
thumbnail: "https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/jg2NWmCUfPyzizm2USjMt.jpeg" |
|
datasets: |
|
- NewEden/Orion-LIT |
|
- NewEden/Orion-Asstr-Stories-16K |
|
- Mielikki/Erebus-87k |
|
- NewEden/RP-logs-V2-Experimental-prefixed |
|
- NewEden/Creative_Writing-Complexity |
|
- NewEden/Discord-Filtered |
|
- NewEden/DeepseekRP-Filtered |
|
- NewEden/Storium-Prefixed-Clean |
|
- NewEden/Basket-Weaving-Filtered |
|
- NewEden/LIMARP-Complexity |
|
- NewEden/Misc-Data-Sharegpt-Prefixed |
|
- NewEden/BlueSky-10K-Complexity |
|
- NewEden/OpenCAI-ShareGPT |
|
- NewEden/Basket-Weaving-Filtered |
|
- PocketDoc/Dans-Personamaxx-VN |
|
- PocketDoc/Dans-Kinomaxx-VanillaBackrooms |
|
|
|
base_model: |
|
- Delta-Vector/Hamanasu-QwQ-V1.5-Instruct |
|
tags: |
|
- qwen |
|
- roleplay |
|
- finetune |
|
- storywriting |
|
--- |
|
<!DOCTYPE html> |
|
<style> |
|
html, body { |
|
background: black; |
|
color: #c9d1d9 !important; |
|
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; |
|
margin: 0; |
|
padding: 0; |
|
min-height: 100vh; |
|
} |
|
.markdown-body { |
|
color: white; |
|
margin: 40px auto; |
|
padding: 40px; |
|
border-radius: 12px; |
|
position: relative; |
|
overflow: hidden; |
|
} |
|
|
|
.markdown-body::after { |
|
content: ''; |
|
position: absolute; |
|
top: 0; |
|
left: 0; |
|
width: 100%; |
|
height: 100%; |
|
background: #0c0f18; /* background color */ |
|
pointer-events: none; |
|
z-index: -999; |
|
} |
|
|
|
h1, h2, h3 { |
|
background: linear-gradient(45deg, #6e00ff, #00ffff); |
|
-webkit-background-clip: text; |
|
-webkit-text-fill-color: transparent; |
|
border-bottom: 1px solid #333; |
|
padding-bottom: 0.3em; |
|
} |
|
|
|
div[style*="border:2px solid #333"], |
|
div[style*="border: 2px solid #333"], |
|
div[style*="border:1px solid #333"], |
|
div[style*="border: 1px solid #333"] { |
|
background: rgba(22, 27, 34, 0.8) !important; |
|
border: 2px solid #6e00ff !important; |
|
box-shadow: 0 0 15px rgba(110, 0, 255, 0.5); |
|
border-radius: 10px; |
|
padding: 20px; |
|
margin: 20px 0; |
|
} |
|
|
|
code { |
|
background-color: #1a1a1a !important; |
|
border-radius: 4px; |
|
padding: 0.2em 0.4em; |
|
color: #00ffff; |
|
} |
|
|
|
pre { |
|
background-color: #1a1a1a !important; |
|
border: 1px solid #333; |
|
border-radius: 8px; |
|
padding: 16px; |
|
} |
|
|
|
table { |
|
width: 100%; |
|
border-collapse: collapse; |
|
margin: 20px 0; |
|
background: rgba(0,0,0,0.2); |
|
table-layout: fixed; |
|
color: white; |
|
} |
|
|
|
th, td { |
|
border: 1px solid #333; |
|
padding: 12px; |
|
text-align: center; |
|
color: white; |
|
} |
|
|
|
th { |
|
background: rgba(110, 0, 255, 0.1); |
|
} |
|
|
|
td:nth-child(1) { |
|
width: 1%; |
|
white-space: nowrap; |
|
} |
|
|
|
td:nth-child(2) { |
|
width: 100%; |
|
} |
|
|
|
td > span { |
|
display: block; |
|
padding: 4px 8px; |
|
background: rgba(110, 0, 255, 0.1); |
|
border-radius: 4px; |
|
transition: all 0.3s ease; |
|
} |
|
|
|
td > span:hover { |
|
background: rgba(110, 0, 255, 0.2); |
|
transform: translateY(-1px); |
|
} |
|
|
|
a { |
|
color: #00ffff; |
|
text-decoration: none; |
|
transition: all 0.3s ease; |
|
} |
|
|
|
a:hover { |
|
color: #6e00ff; |
|
text-decoration: none; |
|
} |
|
|
|
hr { |
|
border: 0; |
|
height: 1px; |
|
background: linear-gradient(90deg, transparent, #333, transparent); |
|
margin: 40px 0; |
|
} |
|
|
|
img { |
|
max-width: 100%; |
|
border-radius: 10px; |
|
} |
|
|
|
details summary:hover { |
|
color: #00ffff; |
|
} |
|
|
|
* { |
|
color-scheme: dark !important; |
|
} |
|
|
|
.prose, .max-w-none, .px-4 { |
|
background-color: transparent !important; |
|
color: #c9d1d9 !important; |
|
} |
|
</style> |
|
<body> |
|
<div class="markdown-body"> |
|
<div align="center"> |
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/o5WjJKA9f95ri9UzRxZQE.png" alt="Model Visualization" width="500px" style="border: 3px solid #333; box-shadow: 0 0 15px rgba(66, 0, 131, 0.5);" /> |
|
|
|
<br> |
|
<br> |
|
|
|
<div style="font-size:1.5em; font-weight:bold; background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent;"> |
|
Hamanasu 32B |
|
</div> |
|
|
|
</div> |
|
|
|
<div style="border:1px solid #333; border-radius:10px; padding:20px; margin:20px 0; background: rgba(0,0,0,0.4);"> |
|
|
|
|
|
## 🌌 Overview |
|
|
|
<i>This model is the Chat tune of the Instruct model, More accurately it is the "brainrotted" version, Finetuned with Bsky, 4chan and Discord logs, Its... really something beautiful.</i> |
|
<i>The model is suited best towards being a highly dumb chat partner rather then regular RP, All thanks to Ruka-Hamanasu for funding the train.</i> |
|
|
|
</div> |
|
|
|
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 20px; margin: 20px 0;"> |
|
|
|
<div style="border:2px solid #333; border-radius:10px; padding:20px; background: rgba(0,0,0,0.2);"> |
|
|
|
### 📈 Quantizations |
|
| Type | Link | |
|
|:---:|:---:| |
|
| `GGUF` | <span style="display:block; width:100%">https://huggingface.co/mradermacher/Hamanasu-QwQ-V2-RP-GGUF</span> | |
|
| `EXL2` | <span style="display:block; width:100%">https://huggingface.co/Delta-Vector/Hamanasu-QwQ-V2-RP-EXL2/</span> | |
|
|
|
</div> |
|
|
|
<div style="border:2px solid #333; border-radius:10px; padding:20px; background: rgba(0,0,0,0.2);"> |
|
|
|
### ⚔️ Hardware |
|
- 8x H100s |
|
- Epochs: 4 |
|
- Base: `Delta-Vector/Hamanasu-QwQ-V1.5-Instruct` |
|
- Amount of Tokens: 10M |
|
</div> |
|
|
|
</div> |
|
|
|
|
|
<div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
|
## 💰 Prompting |
|
|
|
|
|
<i>This model uses ChatML formatting</i> |
|
```python |
|
<|im_start|>system |
|
You are an uncensored AI, your job is to fulfill thy will of thy user.<|im_end|> |
|
<|im_start|>User request |
|
Take off your helmet.<|im_end|> |
|
<|im_start|>No i shall not. This is the way. |
|
``` |
|
</div> |
|
|
|
<div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
|
## 🎲 Recommended Sampler Preset |
|
|
|
```python |
|
temperature: 1.8 |
|
min_p: 0.1 |
|
System_Prompt: Keep blank for best chat experience. |
|
``` |
|
</div> |
|
|
|
<div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
|
## Axolotl Config ꒰(˶• ᴗ •˶)꒱ |
|
|
|
<details> |
|
|
|
```yaml |
|
base_model: NewEden/32B-inst |
|
model_type: AutoModelForCausalLM |
|
tokenizer_type: AutoTokenizer |
|
|
|
hub_model_id: NewEden/32b-rp |
|
hub_strategy: "all_checkpoints" |
|
push_dataset_to_hub: |
|
hf_use_auth_token: true |
|
|
|
plugins: |
|
- axolotl.integrations.liger.LigerPlugin |
|
- axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin |
|
liger_rope: true |
|
liger_rms_norm: true |
|
liger_layer_norm: true |
|
liger_glu_activation: true |
|
liger_fused_linear_cross_entropy: false |
|
cut_cross_entropy: true |
|
|
|
load_in_8bit: false |
|
load_in_4bit: false |
|
strict: false |
|
|
|
datasets: |
|
- path: NewEden/RP-logs-V2-Experimental-prefixed |
|
type: dan-chat-advanced |
|
- path: NewEden/Creative_Writing-Complexity |
|
type: dan-chat-advanced |
|
- path: NewEden/Discord-Filtered |
|
type: dan-chat-advanced |
|
- path: NewEden/DeepseekRP-Filtered |
|
type: dan-chat-advanced |
|
- path: NewEden/Storium-Prefixed-Clean |
|
type: dan-chat-advanced |
|
- path: NewEden/Basket-Weaving-Filtered |
|
type: dan-chat-advanced |
|
- path: NewEden/LIMARP-Complexity |
|
type: dan-chat-advanced |
|
- path: NewEden/Misc-Data-Sharegpt-Prefixed |
|
type: dan-chat-advanced |
|
- path: NewEden/BlueSky-10K-Complexity |
|
type: dan-chat-advanced |
|
- path: NewEden/OpenCAI-ShareGPT |
|
type: dan-chat-advanced |
|
- path: NewEden/Basket-Weaving-Filtered |
|
type: dan-chat-advanced |
|
- path: PocketDoc/Dans-Personamaxx-VN |
|
type: dan-chat-advanced |
|
- path: PocketDoc/Dans-Kinomaxx-VanillaBackrooms |
|
type: dan-chat-advanced |
|
dataset_prepared_path: prepared_data |
|
val_set_size: 0.0 |
|
output_dir: ./qwq-inst |
|
|
|
sequence_len: 32768 |
|
sample_packing: true |
|
pad_to_sequence_len: true |
|
|
|
# adapter: lora |
|
# lora_model_dir: |
|
# lora_r: 128 |
|
# lora_alpha: 16 |
|
# lora_dropout: 0.05 |
|
# lora_target_modules: |
|
# - gate_proj |
|
# - down_proj |
|
# - up_proj |
|
# - q_proj |
|
# - v_proj |
|
# - k_proj |
|
# - o_proj |
|
|
|
wandb_project: qwq |
|
wandb_entity: |
|
wandb_watch: |
|
wandb_name: rp-attempt-03 |
|
wandb_log_model: |
|
|
|
gradient_accumulation_steps: 2 |
|
micro_batch_size: 2 |
|
num_epochs: 4 |
|
optimizer: adamw_bnb_8bit |
|
lr_scheduler: cosine |
|
learning_rate: 2.5e-5 |
|
max_grad_norm: 1.0 |
|
|
|
train_on_inputs: false |
|
group_by_length: false |
|
bf16: auto |
|
fp16: |
|
tf32: false |
|
|
|
gradient_checkpointing: unsloth |
|
early_stopping_patience: |
|
resume_from_checkpoint: |
|
local_rank: |
|
logging_steps: 1 |
|
xformers_attention: |
|
flash_attention: true |
|
|
|
warmup_steps: 40 |
|
saves_per_epoch: 2 |
|
debug: |
|
deepspeed: deepspeed_configs/zero3_bf16.json |
|
weight_decay: 0.02 |
|
fsdp: |
|
fsdp_config: |
|
special_tokens: |
|
``` |
|
|
|
</details> |
|
</div> |
|
|
|
<div align="center"> |
|
|
|
<div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
|
## ⚡ Credits |
|
<div style="display: flex; justify-content: center;"> |
|
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 10px; margin: 20px 0; max-width: 600px;"> |
|
|
|
<div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
|
<a href="https://huggingface.co/lucyknada"> |
|
<img src="https://img.shields.io/badge/%F0%9F%8C%9F-Lucy_Knada-blueviolet" alt="Lucy Knada"> |
|
</a> |
|
</div> |
|
|
|
<div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
|
<a href="https://huggingface.co/hamanasu"> |
|
<img src="https://img.shields.io/badge/%E2%9A%94%EF%B8%8F-jeiku-blueviolet" alt="Ruka"> |
|
</a> |
|
</div> |
|
|
|
<div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
|
<a href="https://huggingface.co/intervitens"> |
|
<img src="https://img.shields.io/badge/%F0%9F%9B%A1%EF%B8%8F-Intervitens-blueviolet" alt="Intervitens"> |
|
</a> |
|
</div> |
|
|
|
<div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
|
<a href="https://huggingface.co/kalomaze"> |
|
<img src="https://img.shields.io/badge/%F0%9F%94%AE-Kalomaze-blueviolet" alt="Kalomaze"> |
|
</a> |
|
</div> |
|
|
|
<div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
|
<a href="https://huggingface.co/kubernetes-bad"> |
|
<img src="https://img.shields.io/badge/%E2%9A%A1-Kubernetes_Bad-blueviolet" alt="Kubernetes Bad"> |
|
</a> |
|
</div> |
|
|
|
<div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
|
<a href="https://huggingface.co/anthracite-org"> |
|
<img src="https://img.shields.io/badge/%F0%9F%8C%91-Anthracite-blueviolet" alt="Anthracite"> |
|
</a> |
|
</div> |
|
</div> |
|
</div> |
|
</div> |
|
|
|
--- |
|
|
|
<div align="center"> |
|
<div style="font-size:0.8em; opacity:0.8;">Made by</div> |
|
<div style="font-size:1.2em; font-weight:bold; background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent;">Delta-Vector</div> |
|
</div> |
|
|
|
</div> |
|
</body> |
|
</html> |
|
|
