Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Qwen
/
Qwen-tokenizer
like
15
Follow
Qwen
7.94k
License:
tongyi-qianwen-license
Model card
Files
Files and versions
Community
1
main
Qwen-tokenizer
/
added_tokens.json
JustinLin610
Upload 6 files
5909c82
verified
12 months ago
raw
Copy download link
history
blame
contribute
delete
Safe
80 Bytes
{
"<|endoftext|>"
:
151643
,
"<|im_end|>"
:
151645
,
"<|im_start|>"
:
151644
}