huu-ontocord
commited on
Commit
•
805a077
1
Parent(s):
a3afe7b
Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,8 @@ license: mit
|
|
7 |
The Phi-3-22b is a depth upsampled version of the 14b [Phi-3-medium-128k-instruct](https://huggingface.co/microsoft/Phi-3-medium-128k-instruct). We removed the bottom 8 layers of one copy of the 14b and the top 8 layers of another copy of the 14b model and stacked them. We plan to do continued pretraining to improve performance.
|
8 |
Since this model has not been continued pretrained, the quality may vary.
|
9 |
```
|
10 |
-
!pip install
|
|
|
11 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
12 |
import torch
|
13 |
tokenizer = AutoTokenizer.from_pretrained("ontocord/phi-3-22b", trust_remote_code=True)
|
@@ -27,7 +28,7 @@ Imagine, if you will, a vast kingdom stretching beyond the horizon, where countl
|
|
27 |
In this kingdom, information flows like a mighty river,...
|
28 |
```
|
29 |
|
30 |
-
|
31 |
```
|
32 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
33 |
import torch
|
|
|
7 |
The Phi-3-22b is a depth upsampled version of the 14b [Phi-3-medium-128k-instruct](https://huggingface.co/microsoft/Phi-3-medium-128k-instruct). We removed the bottom 8 layers of one copy of the 14b and the top 8 layers of another copy of the 14b model and stacked them. We plan to do continued pretraining to improve performance.
|
8 |
Since this model has not been continued pretrained, the quality may vary.
|
9 |
```
|
10 |
+
!pip install flash-attn --no-build-isolation
|
11 |
+
!pip install peft bitsandbytes accelerate transformers
|
12 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
13 |
import torch
|
14 |
tokenizer = AutoTokenizer.from_pretrained("ontocord/phi-3-22b", trust_remote_code=True)
|
|
|
28 |
In this kingdom, information flows like a mighty river,...
|
29 |
```
|
30 |
|
31 |
+
To run on a Colab T4, try 4-bit
|
32 |
```
|
33 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
34 |
import torch
|