update tokenizer for compatibility with new transformers (#64)
Browse files- update tokenizer for compatibility with new transformers (bbb5eacef5efb6b1d0b75c89296f576417bed4e1)
Co-authored-by: Ekaterina Aidova <[email protected]>
- update tokenizer for compatibility with new transformers (bbb5eacef5efb6b1d0b75c89296f576417bed4e1)
Co-authored-by: Ekaterina Aidova <[email protected]>