helbert-prunned / added_tokens.json
vic35get's picture
Upload tokenizer
39fb4e5 verified
raw
history blame contribute delete
191 Bytes
{
"0": 33001,
"1": 33002,
"2": 33003,
"3": 33004,
"4": 33005,
"5": 33006,
"6": 33007,
"7": 33008,
"8": 33009,
"9": 33010,
"–": 33000,
"’": 33012,
"fi": 33011
}