huu-ontocord commited on
Commit
805a077
1 Parent(s): a3afe7b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -7,7 +7,8 @@ license: mit
7
  The Phi-3-22b is a depth upsampled version of the 14b [Phi-3-medium-128k-instruct](https://huggingface.co/microsoft/Phi-3-medium-128k-instruct). We removed the bottom 8 layers of one copy of the 14b and the top 8 layers of another copy of the 14b model and stacked them. We plan to do continued pretraining to improve performance.
8
  Since this model has not been continued pretrained, the quality may vary.
9
  ```
10
- !pip install transformers accelerate
 
11
  from transformers import AutoTokenizer, AutoModelForCausalLM
12
  import torch
13
  tokenizer = AutoTokenizer.from_pretrained("ontocord/phi-3-22b", trust_remote_code=True)
@@ -27,7 +28,7 @@ Imagine, if you will, a vast kingdom stretching beyond the horizon, where countl
27
  In this kingdom, information flows like a mighty river,...
28
  ```
29
 
30
- In 4-bit
31
  ```
32
  from transformers import AutoTokenizer, AutoModelForCausalLM
33
  import torch
 
7
  The Phi-3-22b is a depth upsampled version of the 14b [Phi-3-medium-128k-instruct](https://huggingface.co/microsoft/Phi-3-medium-128k-instruct). We removed the bottom 8 layers of one copy of the 14b and the top 8 layers of another copy of the 14b model and stacked them. We plan to do continued pretraining to improve performance.
8
  Since this model has not been continued pretrained, the quality may vary.
9
  ```
10
+ !pip install flash-attn --no-build-isolation
11
+ !pip install peft bitsandbytes accelerate transformers
12
  from transformers import AutoTokenizer, AutoModelForCausalLM
13
  import torch
14
  tokenizer = AutoTokenizer.from_pretrained("ontocord/phi-3-22b", trust_remote_code=True)
 
28
  In this kingdom, information flows like a mighty river,...
29
  ```
30
 
31
+ To run on a Colab T4, try 4-bit
32
  ```
33
  from transformers import AutoTokenizer, AutoModelForCausalLM
34
  import torch