--- license: cc-by-nc-4.0 datasets: - danlou/based-chat-v0.1-Mistral-Nemo-Base-2407 base_model: - danlou/relay-v0.1-Mistral-Nemo-2407 pipeline_tag: text-generation tags: - axolotl - lmstudio - gguf --- # 📟 Relay v0.1 (Mistral Nemo 2407) This model page includes GGUF versions of [relay-v0.1-Mistral-Nemo-2407](https://huggingface.co/danlou/relay-v0.1-Mistral-Nemo-2407). For more details about this model, please see that model page. **Note**: If you have access to a CUDA GPU, it's highly recommended you use the [main version (HF)](https://huggingface.co/danlou/relay-v0.1-Mistral-Nemo-2407) of the model with the [relaylm.py](https://github.com/danlou/relay/blob/main/relaylm.py) script, which supports better use of commands (e.g., system messages). The `relaylm.py` script also supports [4bit](https://huggingface.co/danlou/relay-v0.1-Mistral-Nemo-2407-4bit) and [8bit](https://huggingface.co/danlou/relay-v0.1-Mistral-Nemo-2407-8bit) bitsandbytes quants. ## Custom Preset for LM Studio To use these GGUF files with LM Studio, you should use this [preset configuration](https://huggingface.co/danlou/relay-v0.1-Mistral-Nemo-2407-GGUF/resolve/main/lm_studio_preset.json). Relay models use ChatML, but not standard roles and system prompts. After you select and download the GGUF version you want to use: - Go to the `My Models` tab. - Click the button with the template name for the model (e.g., ChatML). - Click `Import Preset from file...`, and select the file. - Confirm that the model is set to use the relay preset (see screenshot below): ![image/png](https://cdn-uploads.huggingface.co/production/uploads/60f808c5c1adf9100f1f263c/IvHK1J5zLq1uM1c_Nug3a.png)