README.md

metadata

license: apache-2.0

Hello darkness, my old friend I've come to talk with you again.

---

Use the '--control-vector-scaled' option like this:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-chaos.gguf 0.5 [the rest of your CLI arguments...]

or:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-chaos.gguf 0.0 [the rest of your CLI arguments...]

or:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-chaos.gguf 0.25 [the rest of your CLI arguments...]

NOTE:

• Use positive scale factors to make the model "more dark" or "more chaotic".
• I forgot to change the default scale factor for these to 1.0 (to match the old creative-writing-control-vectors-BETA-v0.1 files... oops) - these have a default of 0.5!
• I suggest you use --control-vector-scaled 0.5 for both to start off and then test the effect.
• The "chaos" control vectors generally seems less effective than the "dark" control vectors.
• Some models like command-r:35b and command-r-plus:104b need lower scale factors, whereas miqu-1:70b seems to need (much) higher scale factors to stamp out pesky redemption arcs.
• You can use one control vector file alone if you want, or alternatively set the scale factor to 0.0 for traits you don't want to use.
• You can use the same "--control-vector-scaled" command line arguments for "llama-server" as in the above "llama-cli" examples.
• Make sure you have a version of llama.cpp from after 27/06/24 - the PR to use multiple control vectors together was only added on that date. Older versions of llama.cpp will just silently load the first control vector (and none of the others) if the layer index of the final direction in each file does not match!!!