MoLora-v2
Collection
First Prototype of the second iteration of MoLora utilizing mixture of expert techniques applied to the Llama2 model.
•
5 items
•
Updated
•
2
YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
this is the initial model warmstarted from openllama 3b v2 with the expert layers copied from models trained with lora on only the mlp layers, if you can find the problem with the modeling code i will send you a picture of my cat
- Downloads last month
- 35
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The HF Inference API does not support model that require custom code execution.