question about tokenizer
#7
by
surya-narayanan
- opened
Hi,
do the tiny, small, medium and mini models all use the same tokenizer?
Yes
prajjwal1
changed discussion status to
closed
Hi,
do the tiny, small, medium and mini models all use the same tokenizer?
Yes