manangarg
/

ind-llm-tokenizer

Model card Files Files and versions Community

ind-llm-tokenizer / README.md

manangarg's picture

Update README.md

5d0daa4 verified 11 months ago

|

history blame contribute delete

142 Bytes

	---
	license: apache-2.0
	---
	• Indic Language LLM Tokenizer
	- This is an indic language NLP tokenizer which is merged with LLaMA 2 tokenizer.