casperhansen
/

mpt-7b-8k-chat-gptq

Text Generation

text-generation-inference

Model card Files Files and versions Community

casperhansen commited on Jul 27, 2023

Commit

344ed4c

·

1 Parent(s): 6c03407

Update README.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -61,6 +61,5 @@ print('generating...')
 # or you can also use pipeline
 streamer = TextStreamer(tokenizer, skip_special_tokens=True)
 tokens = tokenizer(prompt, return_tensors="pt").to(model.device)
-output = model.generate(**tokens, max_length=512, streamer=streamer)
 ```

 # or you can also use pipeline
 streamer = TextStreamer(tokenizer, skip_special_tokens=True)
 tokens = tokenizer(prompt, return_tensors="pt").to(model.device)
+output = model.generate(**tokens, max_length=512, streamer=streamer, eos_token_id=tokenizer.eos_token_id)
 ```