Changes for fast tokenizer

#7

this seems ready no?

jonatanklosko changed pull request status to open

@ArthurZ yeah, I forgot to undraft this one!

ArthurZ changed pull request status to merged
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment