使用transformers推理:

  from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
  from peft import PeftModel
  
  
  tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/baichuan-7B", trust_remote_code=True)
  model = AutoModelForCausalLM.from_pretrained("baichuan-inc/baichuan-7B", device_map="auto", trust_remote_code=True)
  model = PeftModel.from_pretrained(model, "Laurie/baichuan-7b-lora-alpaca-cn")
  streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
  
  query = "天空为什么是蓝色的?"
  
  inputs = tokenizer(["<human>:{}\n<bot>:".format(query)], return_tensors="pt")
  inputs = inputs.to("cuda")
  generate_ids = model.generate(**inputs, max_new_tokens=256, streamer=streamer)
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Dataset used to train Laurie/baichuan-7b-lora-alpaca-cn