Spaces:

patpizio
/

llm-token-probs

Sleeping

patpizio commited on Sep 26, 2023

Commit

47fd2b6

1 Parent(s): 36d45d5

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -61,7 +61,8 @@ tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
 model = T5ForConditionalGeneration.from_pretrained(
     model_checkpoint,
     load_in_8bit=False,
-    device_map="auto"
 )

 model = T5ForConditionalGeneration.from_pretrained(
     model_checkpoint,
     load_in_8bit=False,
+    device_map="auto",
+    offload_folder="offload"
 )