qwenstar
Collection
3 items
•
Updated
Example:
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("qwen-1_8b-bun-chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("qwen-1_8b-bun-chat", device_map="auto", trust_remote_code=True).eval()
response, _ = model.chat(tokenizer, "Hi, how are you?", history=None)
print(response)
#response, history = model.chat(tokenizer, "I'm so happy with my new wardrobe! The clothes I bought fit me perfectly and they look great.", history=history)
#print(response)
- Downloads last month
- 15
Inference API (serverless) does not yet support model repos that contain custom code.