TheBloke commited on
Commit
20c5cd7
1 Parent(s): 27ad339

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -25
README.md CHANGED
@@ -38,7 +38,6 @@ These files were quantised using hardware kindly provided by [Latitude.sh](https
38
 
39
  ## Prompt template: custom
40
 
41
- ```
42
  Please install OpenAI `tiktoken` for the tokenizer.
43
 
44
  ```bash
@@ -59,8 +58,6 @@ generated_ids = model.generate(input_ids, max_length=128)
59
  print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
60
  ```
61
 
62
- ```
63
-
64
  ## Provided files
65
 
66
  Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements.
@@ -110,6 +107,10 @@ First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) instal
110
 
111
  `GITHUB_ACTIONS=true pip install auto-gptq`
112
 
 
 
 
 
113
  Then try the following example code:
114
 
115
  ```python
@@ -143,28 +144,8 @@ model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
143
  quantize_config=None)
144
  """
145
 
146
- prompt = "Tell me about AI"
147
- prompt_template=f'''Please install OpenAI `tiktoken` for the tokenizer.
148
-
149
- ```bash
150
- pip install tiktoken==0.4.0
151
- ```
152
-
153
- ### Causal sampling (code autocompletion)
154
-
155
- For regular causal sampling, simply generate completions given the context:
156
- ```python
157
- from transformers import AutoTokenizer, AutoModelForCausalLM
158
- tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen25-7b-mono", trust_remote_code=True)
159
- model = AutoModelForCausalLM.from_pretrained("Salesforce/codegen25-7b-mono")
160
-
161
- text = "def hello_world():"
162
- input_ids = tokenizer(text, return_tensors="pt").input_ids
163
- generated_ids = model.generate(input_ids, max_length=128)
164
- print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
165
- ```
166
-
167
- '''
168
 
169
  print("\n\n*** Generate:")
170
 
 
38
 
39
  ## Prompt template: custom
40
 
 
41
  Please install OpenAI `tiktoken` for the tokenizer.
42
 
43
  ```bash
 
58
  print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
59
  ```
60
 
 
 
61
  ## Provided files
62
 
63
  Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements.
 
107
 
108
  `GITHUB_ACTIONS=true pip install auto-gptq`
109
 
110
+ ```bash
111
+ pip install tiktoken==0.4.0
112
+ ```
113
+
114
  Then try the following example code:
115
 
116
  ```python
 
144
  quantize_config=None)
145
  """
146
 
147
+ prompt = "def hello_world()"
148
+ prompt_template=f'''{prompt}'''
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
149
 
150
  print("\n\n*** Generate:")
151