bert-tiny-char-ctc-bak-denoise / tokenizer_config.json
cointegrated's picture
Add model and tokenizer files
5b483a1
raw
history blame
241 Bytes
{
"bos_token": "[bos]",
"cls_token": "[cls]",
"do_lower_case": false,
"eos_token": "[eos]",
"mask_token": "[mask]",
"model_max_length": 1024,
"pad_token": "[pad]",
"tokenizer_class": "CharTokenizer",
"unk_token": "[unk]"
}