THUDM

/

glm-4-9b-chat

like 605

错误和这个一样：https://github.com/vllm-project/vllm/issues/3778

错误在 https://github.com/vllm-project/vllm/blob/9606c7197df073e373ab9e716a62dd4c35398865/vllm/model_executor/guided_decoding/outlines_logits_processors.py#L189
如果像下面把 token 先转成str，会有新的错误： 'utf-8' codec can't decode byte 0xa1 in position 0: invalid start byte

def convert_token_to_string(token: str) -> str:
        from transformers.file_utils import SPIECE_UNDERLINE

        if isinstance(token, bytes):
            token = token.decode("utf-8")

        string = tokenizer.convert_tokens_to_string([token])

        print("=" * 16 + f"token, string : {token}, {string}, {SPIECE_UNDERLINE}")

        # A hack to handle missing spaces to HF's Llama tokenizers
        if token.startswith(SPIECE_UNDERLINE) or token == "<0x20>":
            return " " + string

        return string