TinyStoriesTokenizerSpecial / added_tokens.json
Qilex's picture
Upload tokenizer
3ce7268
raw
history blame contribute delete
137 Bytes
{
"<B>": 10003,
"<E>": 10005,
"<NOISE>": 10006,
"<PAD>": 10007,
"<S>": 10004,
"[R]": 10000,
"[S]": 10002,
"[X]": 10001
}