Spaces:

nafisneehal
/

chanbot

Sleeping

nafisneehal commited on Nov 14, 2024

Commit

9d8332a

verified ·

1 Parent(s): ab87d76

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -30,6 +30,7 @@ model = AutoPeftModelForCausalLM.from_pretrained(
     load_in_4bit=load_in_4bit,
     device_map="auto" if device == "cuda" else None  # Automatic GPU mapping
 )
 tokenizer = AutoTokenizer.from_pretrained(model_name)
 # Define prompt structure (update if necessary for your model)

     load_in_4bit=load_in_4bit,
     device_map="auto" if device == "cuda" else None  # Automatic GPU mapping
 )
+model.to(device)
 tokenizer = AutoTokenizer.from_pretrained(model_name)
 # Define prompt structure (update if necessary for your model)