llama3 variant for 22 Indian languages:-
- Tamil
- Telugu
- Assamese
- Kashmiri
- Punjabi
- Bengali
- Sanskrit
- Malyalam
- Sindhi
- Marathi
- Gujarati
- Kannada
- Odia
- Maithili
- Urdu
- Nepali
- Manipuri
- Dogri
- English
- Arabic
- Santali
- Bodo
We first pre-trained the model on 100 million plus Indic language tokens. Then, it was finetuned on close sourced GenZ_Vikas datasets consisting 7.5 million SFT pairs, including 5.5 million Hindi SFT pairs. Finally it underwent DPO training to align it with human preferences.
The model has been benchmarked on Indic LLM leaderboard where it outperforms our AryaBhatta series on Hindi evals. And llama3 base model on all Indian languages.
Training happened on 2*A100 for 24 days.
Link: https://huggingface.co/spaces/Cognitive-Lab/indic_llm_leaderboard
Release link: https://www.linkedin.com/feed/update/urn:li:activity:7199506579828662272
- Downloads last month
- 27
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.