storIA / tokenizer_config.json
Felipehonorato's picture
TOkenizador
37e4997
raw
history blame
125 Bytes
{"bos_token": "<|startoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "max_len": 1024, "init_inputs": []}