VN-News-GPT2 / added_tokens.json
tuanle's picture
add tokenizer
8133c6b
raw
history blame
102 Bytes
{"<|content|>": 50259, "<|startoftext|>": 50261, "<PAD>": 50260, "<|headline|>": 50258, "\n\n": 50257}