MediaTek-Research
/

Breeze-7B-Base-v0_1

Text Generation

text-generation-inference

Model card Files Files and versions Community

Splend1dchan commited on Jan 17, 2024

Commit

f8b2f22

·

verified ·

1 Parent(s): 425ec5a

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -179,7 +179,7 @@ model = AutoModelForCausalLM.from_pretrained(
     model="MediaTek-Research/Breeze-7B-Base-v0.1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
-    use_flash_attn_2=True # optional
 )
 ```

     model="MediaTek-Research/Breeze-7B-Base-v0.1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
+    attn_implementation="flash_attention_2" # optional
 )
 ```