|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
 |
|
|
|
> ***Hello darkness, my old friend I've come to talk with you again.*** |
|
|
|
--- |
|
|
|
### Use the `'--control-vector-scaled'` option like this: |
|
|
|
```sh |
|
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-chaos.gguf 0.5 [the rest of your CLI arguments...] |
|
``` |
|
or: |
|
```sh |
|
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-chaos.gguf 0.0 [the rest of your CLI arguments...] |
|
``` |
|
or: |
|
```sh |
|
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-chaos.gguf 0.25 [the rest of your CLI arguments...] |
|
``` |
|
|
|
### NOTE: |
|
|
|
- Use ***positive scale factors*** to make the model "***more dark***" or "***more chaotic***". |
|
- I forgot to change the default scale factor for these to `1.0` (to match the old `creative-writing-control-vectors-BETA-v0.1` files... oops) - these have a default of `0.5`! |
|
- I suggest you use `--control-vector-scaled 0.5` for both to start off and then test the effect. |
|
- The "chaos" control vectors generally seems less effective than the "dark" control vectors. |
|
- Some models like `command-r:35b` and `command-r-plus:104b` need lower scale factors, whereas `miqu-1:70b` seems to need (much) higher scale factors to stamp out pesky redemption arcs. |
|
- You can use one control vector file alone if you want, or alternatively set the scale factor to `0.0` for traits you don't want to use. |
|
- You can use the same "`--control-vector-scaled`" command line arguments for "`llama-server`" as in the above "`llama-cli`" examples. |
|
- Make sure you have a version of `llama.cpp` from after `27/06/24` - the [PR to use multiple control vectors together](https://github.com/ggerganov/llama.cpp/commit/97877eb10bd8e7f8023420b5b5300bcbdadd62dc) was only added on that date. |
