Update README.md

README.md

@@ -30,7 +30,7 @@ model = AutoModelForCausalLM.from_pretrained(
 )
 ```
 
-To run this model, you may need to load it in a lower precision in order for it to fit onto your GPU. We found for a T4 GPU, it requires loading the model in 8-bit precision. To load the model in 8-bit, please install the following pip packages:
+To run this model, you may need to load it in a lower precision in order for it to fit onto your GPU. We found for a T4 GPU, it requires loading the model in 8-bit precision. To load the model in 8-bit and 4-bit, please install the following pip packages:
 
 ```bash
 pip install bitsandbytes accelerate
@@ -55,6 +55,23 @@ model = AutoModelForCausalLM.from_pretrained(
 )
 ```
 
+</details><details>
+<summary><b>Code to load the model in 4 bit</b></summary>
+
+
+
+```python
+from transformers import AutoModelForCausalLM
+
+model_name = "lightblue/japanese-mpt-7b"
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype='auto',
+    load_in_4bit=True,
+    trust_remote_code=True
+)
+```
+
 </details>