TheBloke-starchat-beta-GPTQ

Runtime error

binqiangliu commited on Oct 24, 2023

Commit

42d6a0b

1 Parent(s): baadc89

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -18,7 +18,7 @@ tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
 model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
         use_safetensors=True,
         #device="cuda:0",
-        device_map="auto"
         use_triton=use_triton,
         quantize_config=None)

 model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
         use_safetensors=True,
         #device="cuda:0",
+        device_map="auto",
         use_triton=use_triton,
         quantize_config=None)