Update README.md

README.md

@@ -7,6 +7,10 @@ tags:
 
 **NOTE**: See [creative-writing-control-vectors-v2.1](https://huggingface.co/jukofyork/creative-writing-control-vectors-v2.1) for the current main control-vector repo.
 
+*(I will also add `WizardLM-2-8x22B`, `c4ai-command-r-v01` and `gemma-2-27b-it` versions of these control vectors in the next few days...)*
+
+## Details
+
 The control-vectors in this repo were created as an experiment by increasing the triplets in `system_messages_outlook_extended.json` by 4x.
 
 This means each models' cross-covariance matrix will be the result of `120,000` hidden state samples and this in turn will mean each uses ~10x the hidden state dimension for the largest models (click to expand):
@@ -146,6 +150,8 @@ This means each models' cross-covariance matrix will be the result of `120,000`
 
 </details>
 
+## Regularisation
+
 I also include 3 different values for the `--regularisation_factor` option; `1.0` (the default), `0.5` and `0.0`:
 
 - [regularisation_factor = 1.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v2.1.2-EXPERIMENTAL/tree/main/regularisation_factor%20%3D%201.0)
@@ -154,6 +160,21 @@ I also include 3 different values for the `--regularisation_factor` option; `1.0
 
 Try to use the largest `regularisation_factor` that has the desired effect - this has the least chance of damaging the models' outputs.
 
----
+## Prompting format for `Mistral-Large-Instruct-2407` and `WizardLM-2-8x22B`:
+
+I have found by testing that `Mistral-Large-Instruct-2407` seems to work much better for creative writing if you use the following 'Vicuna' prompt template:
+
+```
+USER: {prompt}
+ASSISTANT:
+```
+
+so I altered the 'Jinja2' `chat_template` in the `tokenizer_config.json` for both `Mistral-Large-Instruct-2407` and `WizardLM-2-8x22B` to this for the training of these control vectors:
+
+```json
+{
+  "chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{{ messages[0]['content'] | trim + '\n\n' }}{% else %}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ 'USER: ' + message['content'] | trim + '\n' }}{% elif message['role'] == 'assistant' %}{{ 'ASSISTANT: ' + message['content'] | trim + eos_token + '\n' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ 'ASSISTANT:' }}{% endif %}"
+}
+```
 
-*I will also add `WizardLM-2-8x22B`, `c4ai-command-r-v01` and `gemma-2-27b-it` versions of these control vectors in the next few days.*
+The other 3 models still use their default templates though.