iansotnek commited on
Commit
6bbba54
1 Parent(s): 26a336f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -52,7 +52,7 @@ pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.1
52
  ```
53
 
54
  The instruction following pipeline can be loaded using the `pipeline` function as shown below. This loads a custom `InstructionTextGenerationPipeline`
55
- found in the model repo [here](https://huggingface.co/aisquared/dlite-v1-1.5b/blob/main/instruct_pipeline.py), which is why `trust_remote_code=True` is required.
56
  Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
57
  It is also fine to remove it if there is sufficient memory.
58
 
@@ -60,7 +60,7 @@ It is also fine to remove it if there is sufficient memory.
60
  from transformers import pipeline
61
  import torch
62
 
63
- generate_text = pipeline(model="aisquared/dlite-v1-1.5b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
64
  ```
65
 
66
  You can then use the pipeline to answer instructions:
@@ -70,7 +70,7 @@ res = generate_text("Who was George Washington?")
70
  print(res[0]["generated_text"])
71
  ```
72
 
73
- Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/aisquared/dlite-v1-1.5b/blob/main/instruct_pipeline.py),
74
  store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
75
 
76
  ```python
@@ -78,8 +78,8 @@ from instruct_pipeline import InstructionTextGenerationPipeline
78
  from transformers import AutoModelForCausalLM, AutoTokenizer
79
  import torch
80
 
81
- tokenizer = AutoTokenizer.from_pretrained("aisquared/dlite-v1-1.5b", padding_side="left")
82
- model = AutoModelForCausalLM.from_pretrained("aisquared/dlite-v1-1.5b", device_map="auto", torch_dtype=torch.bfloat16)
83
 
84
  generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
85
  ```
 
52
  ```
53
 
54
  The instruction following pipeline can be loaded using the `pipeline` function as shown below. This loads a custom `InstructionTextGenerationPipeline`
55
+ found in the model repo [here](https://huggingface.co/aisquared/dlite-v1-1_5b/blob/main/instruct_pipeline.py), which is why `trust_remote_code=True` is required.
56
  Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
57
  It is also fine to remove it if there is sufficient memory.
58
 
 
60
  from transformers import pipeline
61
  import torch
62
 
63
+ generate_text = pipeline(model="aisquared/dlite-v1-1_5b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
64
  ```
65
 
66
  You can then use the pipeline to answer instructions:
 
70
  print(res[0]["generated_text"])
71
  ```
72
 
73
+ Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/aisquared/dlite-v1-1_5b/blob/main/instruct_pipeline.py),
74
  store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
75
 
76
  ```python
 
78
  from transformers import AutoModelForCausalLM, AutoTokenizer
79
  import torch
80
 
81
+ tokenizer = AutoTokenizer.from_pretrained("aisquared/dlite-v1-1_5b", padding_side="left")
82
+ model = AutoModelForCausalLM.from_pretrained("aisquared/dlite-v1-1_5b", device_map="auto", torch_dtype=torch.bfloat16)
83
 
84
  generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
85
  ```