jupyterjazz commited on
Commit
3b9e730
1 Parent(s): 447c0db

chore: no need for tokenizer config as well

Browse files

Signed-off-by: jupyterjazz <[email protected]>

Files changed (1) hide show
  1. tokenizer_config.json +0 -4
tokenizer_config.json DELETED
@@ -1,4 +0,0 @@
1
- {
2
- "model_max_length": 8194,
3
- "tokenizer_class": "XLMRobertaTokenizer"
4
- }