Update README.md
Browse files
README.md
CHANGED
@@ -38,7 +38,6 @@ These files were quantised using hardware kindly provided by [Latitude.sh](https
|
|
38 |
|
39 |
## Prompt template: custom
|
40 |
|
41 |
-
```
|
42 |
Please install OpenAI `tiktoken` for the tokenizer.
|
43 |
|
44 |
```bash
|
@@ -59,8 +58,6 @@ generated_ids = model.generate(input_ids, max_length=128)
|
|
59 |
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
|
60 |
```
|
61 |
|
62 |
-
```
|
63 |
-
|
64 |
## Provided files
|
65 |
|
66 |
Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements.
|
@@ -110,6 +107,10 @@ First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) instal
|
|
110 |
|
111 |
`GITHUB_ACTIONS=true pip install auto-gptq`
|
112 |
|
|
|
|
|
|
|
|
|
113 |
Then try the following example code:
|
114 |
|
115 |
```python
|
@@ -143,28 +144,8 @@ model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
|
|
143 |
quantize_config=None)
|
144 |
"""
|
145 |
|
146 |
-
prompt = "
|
147 |
-
prompt_template=f'''
|
148 |
-
|
149 |
-
```bash
|
150 |
-
pip install tiktoken==0.4.0
|
151 |
-
```
|
152 |
-
|
153 |
-
### Causal sampling (code autocompletion)
|
154 |
-
|
155 |
-
For regular causal sampling, simply generate completions given the context:
|
156 |
-
```python
|
157 |
-
from transformers import AutoTokenizer, AutoModelForCausalLM
|
158 |
-
tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen25-7b-mono", trust_remote_code=True)
|
159 |
-
model = AutoModelForCausalLM.from_pretrained("Salesforce/codegen25-7b-mono")
|
160 |
-
|
161 |
-
text = "def hello_world():"
|
162 |
-
input_ids = tokenizer(text, return_tensors="pt").input_ids
|
163 |
-
generated_ids = model.generate(input_ids, max_length=128)
|
164 |
-
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
|
165 |
-
```
|
166 |
-
|
167 |
-
'''
|
168 |
|
169 |
print("\n\n*** Generate:")
|
170 |
|
|
|
38 |
|
39 |
## Prompt template: custom
|
40 |
|
|
|
41 |
Please install OpenAI `tiktoken` for the tokenizer.
|
42 |
|
43 |
```bash
|
|
|
58 |
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
|
59 |
```
|
60 |
|
|
|
|
|
61 |
## Provided files
|
62 |
|
63 |
Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements.
|
|
|
107 |
|
108 |
`GITHUB_ACTIONS=true pip install auto-gptq`
|
109 |
|
110 |
+
```bash
|
111 |
+
pip install tiktoken==0.4.0
|
112 |
+
```
|
113 |
+
|
114 |
Then try the following example code:
|
115 |
|
116 |
```python
|
|
|
144 |
quantize_config=None)
|
145 |
"""
|
146 |
|
147 |
+
prompt = "def hello_world()"
|
148 |
+
prompt_template=f'''{prompt}'''
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
149 |
|
150 |
print("\n\n*** Generate:")
|
151 |
|