misti-specter2 / added_tokens.json
pawin205's picture
Upload tokenizer
0db74f2 verified
raw
history blame
161 Bytes
{
"[END-CONCEPT]": 31093,
"[END-SECTION]": 31095,
"[END-TITLE]": 31091,
"[START-CONCEPT]": 31092,
"[START-SECTION]": 31094,
"[START-TITLE]": 31090
}