Special token

#12

by Yingshu - opened May 13, 2024

May 13, 2024

I found that there is an token added to the tokenizer. I just wonder what is the token used for?

Yingshu changed discussion status to closed May 13, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment