Hugging Face's logo

manangarg
/
ind-llm-tokenizer

Model card Files
xet
Community
manangarg commited on
Commit
1cbd965
·
verified ·
1 Parent(s): 4201009

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ • Indic Language LLM Tokenizer
2
+ - This is an indic language NLP tokenizer which is merged with LLaMA 2 tokenizer.