Prompt + Instruct + Samplers + Feedback Please!
Hey guys, would appreciate it if you could give us some feedback on this merge! So far, I'm digging it, but I'm extremely biased, lmao.
Here are my recommended settings for the model (ST):
Story String: https://files.catbox.moe/b970l6.json
Instruct: https://files.catbox.moe/ihel9x.json
Samplers: https://files.catbox.moe/t7o14a.json
OR
Samplers: https://files.catbox.moe/iw54oy.json
(Both work, not sure which are better.)
And here are Parasitic's (ST):
Story String: https://files.catbox.moe/cu222x.json
Instruct: https://files.catbox.moe/m6dhvu.json
Samplers: https://files.catbox.moe/1kwaw6.json
Let us know which ones work better and how the model fares overall, thanks!
GGUF?
GGUF?
Can't vouch on quality, but this guy has some.
https://huggingface.co/mradermacher/RP-Stew-v4.0-34B-i1-GGUF
exl2 4.00? and lower for longer context size
exl2 4.00? and lower for longer context size
I have a 4.25 already made that I was using privately, but never uploaded since I was trying out Nemo during the same time. That should get you around 72k context (no idea if it's good at that level, since I never tested it that high) if you have a 24GB card. I can post that for now if you want.
2 months later and still the best local model for RP
thank you!
2 months later and still the best local model for RP
thank you!
You are welcome!
I have a 4.25 already made that I was using privately, but never uploaded since I was trying out Nemo during the same time. That should get you around 72k context (no idea if it's good at that level, since I never tested it that high) if you have a 24GB card. I can post that for now if you want.
For me, recalling small nuanced details in only a 7k context can be challenging, but it's also expected given the mixture. As a suggestion for the next version, consider merging magnum-v3-34b and possibly dolphin-2_2-yi-34b using one of the new mergekit methods.
For me, recalling small nuanced details in only a 7k context can be challenging, but it's also expected given the mixture. As a suggestion for the next version, consider merging magnum-v3-34b and possibly dolphin-2_2-yi-34b using one of the new mergekit methods.
All of my 12B Nemo merges have been using the new Della method, so I already have a decent grasp on how it functions for merging. I didn't really test much with 34B models after it came out, since Nemo felt a bit better in most cases, but I don't mind doing another simple merge with Yi using Della just to test it out and compare for those few still interested. I'll see if I can do anything this weekend with Dolphin + Magnum, but no guarantees.
@Nekochu
I tried merging the 34B Magnum and Dolphin models together, and... I'm not sure if you'd really like it in all honesty. Even the worse versions of RP-Stew feel better then it from what I remember compared to Yi-1.5, and this is at low context too. You still want me to upload it?
Just magnum-v3-34b and a zest Dolphin? I was suggesting them in addition to all the current RP-Stew-v4.0-34B:Nous-Capybara-34B,Tess-2.0-Yi-34B-200K,bagel-dpo-34b-v0.5,PiVoT-SUS-RP,NyakuraV2-34B-Yi-Llama,CausalLM-RP-34B,Yi-34b-200K-AEZAKMI-RAW-TOXIC-2702.
Just magnum-v3-34b and a zest Dolphin? I was suggesting them in addition to all the current
RP-Stew-v4.0-34B:Nous-Capybara-34B,Tess-2.0-Yi-34B-200K,bagel-dpo-34b-v0.5,PiVoT-SUS-RP,NyakuraV2-34B-Yi-Llama,CausalLM-RP-34B,Yi-34b-200K-AEZAKMI-RAW-TOXIC-2702.
@Nekochu , old Yi 200k and the new Yi 1.5 32k are completely different models with different architectures. You cannot add them to the merge.
My mistake, no, we probably don’t need it.