japanese-av-title-classification / tokenizer_config.json
wycstar's picture
[ADD]add tokenizer_config.json
52659b5
raw
history blame
103 Bytes
{
"do_lower_case": false,
"subword_tokenizer_type": "wordpiece",
"word_tokenizer_type": "mecab"
}