Update README.md

README.md

@@ -6,6 +6,23 @@ datasets:
 
 This repo contains a low-rank adapter for LLaMA-13b fit on the Stanford Alpaca dataset.
 
-It doesn't contain the foundation model itself, so it's MIT licensed!
+### How to use (8-bit)
 
-Instructions for running it can be found at https://github.com/tloen/alpaca-lora.
+This model can be easily loaded using the `AutoModelForCausalLM` functionality:
+
+```python
+from peft import PeftModel
+from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
+
+tokenizer = LlamaTokenizer.from_pretrained("decapoda-research/llama-13b-hf")
+model = LlamaForCausalLM.from_pretrained(
+        "decapoda-research/llama-13b-hf",
+        load_in_8bit=True,
+        torch_dtype=torch.float16,
+        device_map="auto",
+)
+model = PeftModel.from_pretrained(
+    model, "baruga/alpaca-lora-13b",
+    torch_dtype=torch.float16
+)
+```