5fa1a76
1
This tokenizer type is significantly faster - especially during batch tokenization - due to its Rust implementation.
