eos token id mismatch

#12

by awni - opened 21 days ago

Discussion

awni

21 days ago

The eos_token in the config.json does not match the one in the tokenizer_config.json.

See here and here respectively.

From the model behavior it looks like the latter is the correct one. It would be great if that was reflected in the config.json. Would you be willing to make that change?

redmoe-ai-v1

rednote-hilab org 20 days ago

Thanks for pointing this out. We'll get it corrected shortly.

redmoe-ai-v1 changed discussion status to closed 16 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment