quant versions
#2
by
prudant
- opened
yes, that would be nice.
Alternatively, I'm working on distilling the model down into 2-3bn param range.
TheBloke/Sensei-7B-V1-AWQ ๐๐๐
it would be very nice have a distiled version, can you notify your progress please!
@emrgnt-cmplxty very interesting model! I made a new GGUF quantized based on your new models: https://huggingface.co/MaziyarPanahi/Sensei-7B-V1-GGUF