internlm
/

internlm2-base-20b

Text Generation

Model card Files Files and versions Community

internlm2-base-20b / tokenization_internlm2_fast.py

Commit History

fix no white space when using stream_chat with fast tokenizer

edf2879

x54-729 commited on Feb 28, 2024

fast tokenizer and stream_chat fix (#3)

ca3f2c7
verified

x54-729 commited on Jan 24, 2024