phillnet / tokenizer.json
ayjays132's picture
Update tokenizer.json
05d1bdc
raw
history blame
99 Bytes
{
"type": "basic_english",
"params": {
"lowercase": true,
"preserve_case": false
}
}