Spaces:

mikeee
/

qwen-7b-chat

Running on T4

ffreemt commited on Oct 8, 2023

Commit

b0c90a1

1 Parent(s): efd09eb

Update device_map={'': 0} rich, check nvidia-smi with subproces.run

Files changed (1) hide show

app.py CHANGED Viewed

@@ -83,7 +83,8 @@ def gen_model(model_name: str):
     model = AutoModelForCausalLM.from_pretrained(
         model_name,
         trust_remote_code=True,
-        device_map="auto",
         # load_in_4bit=True,
         load_in_8bit=True,
         max_memory=max_memory,

     model = AutoModelForCausalLM.from_pretrained(
         model_name,
         trust_remote_code=True,
+        # device_map="auto",
+        device_map={""： 0},
         # load_in_4bit=True,
         load_in_8bit=True,
         max_memory=max_memory,