Remove tokenizer.json and replace tokenizer.py with correct version. ebac1e8 verified pragaash commited on Feb 26
Update byte tokenizer to be compatible with auto tokenizer and clean-up. 158e9d1 verified pragaash commited on Feb 26