TokenizerFast with tokenizer.json

by vubiqus - opened Nov 8, 2023

Nov 8, 2023

Hello,

Will it be possible to release a tokenizer.json config for TokenizerFast like in whisper-large-v2 ?

Thanks

lmz

Nov 8, 2023

Also interested by a tokenizer.json config, this would make it easy to use the model with candle.

Nov 16, 2023

Closed by #16

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment