PEFT Implementation
It seems like the PEFT Implementation is not done yet. When trying to train a LORA following error occurs:
TypeError: MPTForCausalLM.forward() got an unexpected keyword argument 'inputs_embeds'
Any Information if or when this will be implemented?
have been looking at this. It looks like this might be a patch:
https://github.com/iwalton3/mpt-lora-patch
However, I didn't really look at the code nor am I sure if there is a related pull request or something else in the works.
The author said it worked but then doesn't seem sure. From the patch author: "The model code specifically says that left padding is not supported, but I forcibly did so and got decent results."
Works perfectly thanks :)
@SebastianBodza
where to use the patched version.
How to run it. which file to run.
Can you help me