Can you publish a javascript version?
We didn't train a JavaScript version, we might train slightly larger models which support the same programming languages as StarCoder in the future.
I see, problem of bigger multilanguage models is that it consume too many resources to run locally. Is there any chance you can train a JavaScript model just like this one?
We didn't train a JavaScript version, we might train slightly larger models which support the same programming languages as StarCoder in the future.
That would be really nice! A good balance between multiple language support/performance and model size is really important for democratic AI!
If I have to train a Javascript model for tiny_starcoder, is there any code that I can take a look at?
Do you use the same MegatronLM repo for training tiny_starcoder too with a smaller GPT2 model as base?