Extend eos_token_id

#68
by Wovchena - opened

Extend eos_token_id with pad token because that's the token stored in processor.tokenizer.eos_token_id. That also simplifies the sample from the model card from

generate_ids = model.generate(**inputs, eos_token_id=processor.tokenizer.eos_token_id, **generation_args)

to

generate_ids = model.generate(**inputs, **generation_args)
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment