cr4 / special_tokens_map.json
whoisltd's picture
Upload 9 files
11058aa
raw
history blame
163 Bytes
{"bos_token": "<bos>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>", "pad_token": "<pad>", "additional_special_tokens": ["<speaker1>", "<speaker2>"]}