Llama3-ChatQA-1.5-70B / tokenizer.json

Commit History

The tokenizer adds a special token '<|im_end|>' to solve the problem of non-stop generation when encountering <|im_end|>.
26140d9
verified

zjyhf commited on

Upload 6 files
0f20cba
verified

zihanliu commited on