imranali291
/

gpt-base-prompt-generator

Model card Files Files and versions Community

imranali291 commited on Jan 24

Commit

7677540

·

verified ·

1 Parent(s): 27a166f

Update README.md

Files changed (1) hide show

README.md +3 -0

README.md CHANGED Viewed

@@ -7,6 +7,9 @@ tags:
 # Streaming Inference
 ```python
 from unsloth import FastLanguageModel
 model, tokenizer = FastLanguageModel.from_pretrained(
     model_name = "imranali291/gpt-base-prompt-generator",
     max_seq_length = max_seq_length,

 # Streaming Inference
 ```python
 from unsloth import FastLanguageModel
+max_seq_length = 2048
+dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
+load_in_4bit = True
 model, tokenizer = FastLanguageModel.from_pretrained(
     model_name = "imranali291/gpt-base-prompt-generator",
     max_seq_length = max_seq_length,