Update README.md

README.md

@@ -7,4 +7,38 @@ datasets:
 - allura-org/fujin-cleaned-stage-2
 base_model:
 - internlm/internlm3-8b-instruct
----
+---
+
+# Ruby-Music-8B
+
+*Note that this model is based on InternLM3, **not** LLaMA 3.*
+
+A roleplaying/creative-writing fine tune of [internlm/internlm3-8b-instruct](https://huggingface.co/internlm/internlm3-8b-instruct), provided as an alternative to L3 8B for folks with 8GB VRAM.
+
+This was trained on a mix of private instruct (~1k samples) and roleplaying (~2.5k human and ~1k synthetic samples), along with the following public datasets:
+- allenai/tulu-3-sft-personas-instruction-following (~500 samples)
+- PocketDoc/Dans-Prosemaxx-Gutenberg (all samples)
+- ToastyPigeon/SpringDragon-Instruct (~500 samples)
+- allura-org/fujin-cleaned-stage-2 (~500 samples)
+
+The instruct format is standard ChatML:
+```
+<|im_start|>system
+{system prompt}<|im_end|>
+<|im_start|>user
+{user message}<|im_end|>
+<|im_start|>assistant
+{assistant response}<|im_end|>
+```
+
+## Recommended sampler settings:
+- temp 1
+- smoothing factor 0.5, smoothing curve 1
+- DRY 0.5/1.75/5/1024
+
+There may be better sampler settings, but this at least has proven stable in my testing. InternLM3 requires a high amount of tail filtering (high min-p, top-a, or something similar) to avoid making strange typos and spelling mistakes. *Note: this might be a current issue with llama.cpp and the GGUF versions I tested.*
+
+## Notes:
+I noticed this model has trouble outputting the EOS token sometimes (despite confirming that `<|im_end|>` appears at the end of every turn in the training data). This can cause it to ramble at the end of a message instead of ending its turn. 
+
+You can either cut the end out of the messages until it picks up the right response length, or use logit bias. I've had success getting right-sized turns setting logit bias for `<|im_end|>` to 2.