Extend eos_token_id
#68
by
Wovchena
- opened
Extend eos_token_id
with pad token because that's the token stored in processor.tokenizer.eos_token_id
. That also simplifies the sample from the model card from
generate_ids = model.generate(**inputs, eos_token_id=processor.tokenizer.eos_token_id, **generation_args)
to
generate_ids = model.generate(**inputs, **generation_args)