README.md

metadata

license: apache-2.0

Hello darkness, my old friend I've come to talk with you again.

---

Use the '--control-vector-scaled' option like this:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-chaos.gguf 0.5 [the rest of your CLI arguments...]

or:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-chaos.gguf 0.0 [the rest of your CLI arguments...]

or:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-chaos.gguf 0.25 [the rest of your CLI arguments...]

NOTE:

• Use positive scale factors to make the model "more dark" or "more chaotic".
• I forgot to change the default scale factor for these to 1.0 (to match the old creative-writing-control-vectors-BETA-v0.1 files... oops) - these have a default of 0.5!
• I suggest you use --control-vector-scaled 0.5 for both to start off and then test the effect.
• The "chaos" control vectors generally seems less effective than the "dark" control vectors.
• Some models like command-r:35b and command-r-plus:104b need lower scale factors, whereas miqu-1:70b seems to need (much) higher scale factors to stamp out pesky redemption arcs.
• You can use one control vector file alone if you want, or alternatively set the scale factor to 0.0 for traits you don't want to use.
• You can use the same "--control-vector-scaled" command line arguments for "llama-server" as in the above "llama-cli" examples.
• Make sure you have a version of llama.cpp from after 27/06/24 - the PR to use multiple control vectors together was only added on that date.