OpenNLPLab's picture
Upload codebase
bc6e7dd
raw
history blame
181 Bytes
{
"model_max_length": 65536,
"tokenizer_class": "GPT4Tokenizer",
"auto_map": {
"AutoTokenizer": [
"tokenization_transnormerllm.GPT4Tokenizer",
null
]
}
}