Request: Sao10K/MN-12B-Lyra-v4
[Required] Model name:
Sao10K/MN-12B-Lyra-v4
[Required] Model link:
https://huggingface.co/Sao10K/MN-12B-Lyra-v4
[Required] Brief description:
A finetune of Mistral Nemo by Sao10k.
[Required] An image/direct image link to represent the model (square shaped):
Will upload as soon as possible. Will be on the way soon[tm].
It's up:
https://huggingface.co/Lewdiculous/MN-12B-Lyra-v4-GGUF-IQ-Imatrix
I'll upload IQ2_S-imat (~2.06 bpw) and IQ2_XXS-imat (~2.5 bpw) too later as an experiment for lowest realistically usable quants.
Explain to me for heaven's sake.. How to use models on third-party sites, such as, for example,janitor.ai? Where to get, link, where is the model and where is the reverse API.. please explain
Run the model using KoboldCpp and enable the Remote Tunnel, you can use that remote cloudflare link that will be generated to connect to the API on remote services.
You can find the models in the Files tab here. For machines with 12GB of VRAM if you want balance in quality and speed get the Q4_K_M option. For less you can try smaller sizes (look at the file sizes) until you find the biggest one that runs fast enough for your needs on your machine.
Better:
No point in using stuff like Janitor if you can run your models locally in my opinion, use SillyTavern, since this is what the presets are for and the intended use is from me.
If none of this makes sense I'd recommend looking for how to get started with LLMs locally on Reddit or related Discord communities.
Thank you very much for the explanations! Good luck to you!