GGUF quants of TeeZee/Kyllene-57B-v1.0, remeber to set your max context length to proper length for your hardware, 4096 is fine. Default context length is 200k so it will eat RAM or VRAM like crazy if let unchecked.
GGUF quants of TeeZee/Kyllene-57B-v1.0, remeber to set your max context length to proper length for your hardware, 4096 is fine. Default context length is 200k so it will eat RAM or VRAM like crazy if let unchecked.