Update README.md
Browse files
README.md
CHANGED
@@ -52,7 +52,7 @@ pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.1
|
|
52 |
```
|
53 |
|
54 |
The instruction following pipeline can be loaded using the `pipeline` function as shown below. This loads a custom `InstructionTextGenerationPipeline`
|
55 |
-
found in the model repo [here](https://huggingface.co/aisquared/dlite-v1-
|
56 |
Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
|
57 |
It is also fine to remove it if there is sufficient memory.
|
58 |
|
@@ -60,7 +60,7 @@ It is also fine to remove it if there is sufficient memory.
|
|
60 |
from transformers import pipeline
|
61 |
import torch
|
62 |
|
63 |
-
generate_text = pipeline(model="aisquared/dlite-v1-
|
64 |
```
|
65 |
|
66 |
You can then use the pipeline to answer instructions:
|
@@ -70,7 +70,7 @@ res = generate_text("Who was George Washington?")
|
|
70 |
print(res[0]["generated_text"])
|
71 |
```
|
72 |
|
73 |
-
Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/aisquared/dlite-v1-
|
74 |
store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
|
75 |
|
76 |
```python
|
@@ -78,8 +78,8 @@ from instruct_pipeline import InstructionTextGenerationPipeline
|
|
78 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
79 |
import torch
|
80 |
|
81 |
-
tokenizer = AutoTokenizer.from_pretrained("aisquared/dlite-v1-
|
82 |
-
model = AutoModelForCausalLM.from_pretrained("aisquared/dlite-v1-
|
83 |
|
84 |
generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
|
85 |
```
|
|
|
52 |
```
|
53 |
|
54 |
The instruction following pipeline can be loaded using the `pipeline` function as shown below. This loads a custom `InstructionTextGenerationPipeline`
|
55 |
+
found in the model repo [here](https://huggingface.co/aisquared/dlite-v1-1_5b/blob/main/instruct_pipeline.py), which is why `trust_remote_code=True` is required.
|
56 |
Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
|
57 |
It is also fine to remove it if there is sufficient memory.
|
58 |
|
|
|
60 |
from transformers import pipeline
|
61 |
import torch
|
62 |
|
63 |
+
generate_text = pipeline(model="aisquared/dlite-v1-1_5b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
|
64 |
```
|
65 |
|
66 |
You can then use the pipeline to answer instructions:
|
|
|
70 |
print(res[0]["generated_text"])
|
71 |
```
|
72 |
|
73 |
+
Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/aisquared/dlite-v1-1_5b/blob/main/instruct_pipeline.py),
|
74 |
store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
|
75 |
|
76 |
```python
|
|
|
78 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
79 |
import torch
|
80 |
|
81 |
+
tokenizer = AutoTokenizer.from_pretrained("aisquared/dlite-v1-1_5b", padding_side="left")
|
82 |
+
model = AutoModelForCausalLM.from_pretrained("aisquared/dlite-v1-1_5b", device_map="auto", torch_dtype=torch.bfloat16)
|
83 |
|
84 |
generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
|
85 |
```
|