Usage in ooba
#10
by
goldendase
- opened
Is it possible to load these in ooba? Can you load these just as if they were a LoRA?
Is it possible to load these in ooba? Can you load these just as if they were a LoRA?
They aren't the same as LoRAs: they are a vector of values that get added to the hidden state between each transformer layer, whereas LoRAs get added to the actual weight matrices.
I don't use ooba, but it looks like there isn't any way to use control-vectors yet:
https://github.com/oobabooga/text-generation-webui/issues/5712
If ooba can use OpenAI compatabile API then it would be possible to use then via llama-server but I'm not sure how much of ooba would be missing via this method.