Spaces:

chansung
/

LLM-As-Chatbot

Runtime error

chansung commited on Jun 14, 2023

Commit

3b1f7c1

1 Parent(s): ece3ddf

Update models/guanaco.py

Files changed (1) hide show

models/guanaco.py CHANGED Viewed

@@ -67,7 +67,7 @@ def load_model(
         model = AutoModelForCausalLM.from_pretrained(
             base,
             load_in_8bit=mode_8bit,
-            load_in_4bit=mode_4bit,
             torch_dtype=torch.bfloat16,
             device_map="auto",
             use_safetensors=False

         model = AutoModelForCausalLM.from_pretrained(
             base,
             load_in_8bit=mode_8bit,
+            # load_in_4bit=mode_4bit,
             torch_dtype=torch.bfloat16,
             device_map="auto",
             use_safetensors=False