可以增加tokenizer.json吗？

by navy985 - opened Nov 22, 2024

Nov 22, 2024

•

可以增加tokenizer.model对应的tokenzier.json文件吗？或者如何生成？

navy985 changed discussion title from 可以增加tokenzier.json吗？ to 可以增加tokenizer.json吗？ Nov 22, 2024

czczup

OpenGVLab org Nov 22, 2024

可以啊，这个是新版的transformers会生成的吗，我看看怎么弄

Nov 25, 2024

尝试生成tokenizer.json，报错：

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment