xlm-roberta-flash-implementation / tokenizer_config.json
bwang0911's picture
Update tokenizer_config.json (#14)
c8da5f5 verified
raw
history blame
75 Bytes
{
"model_max_length": 8194,
"tokenizer_class": "XLMRobertaTokenizer"
}