seonglae commited on
Commit
e801f77
·
1 Parent(s): 7f8271d

docs: update get started and tags

Browse files
Files changed (1) hide show
  1. README.md +34 -2
README.md CHANGED
@@ -4,9 +4,41 @@ license: other
4
  datasets:
5
  - ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered
6
  tags:
7
- - wiwardlm
8
  - uncensored
9
  - gptq
 
10
  - 7b
11
  - llama
12
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  datasets:
5
  - ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered
6
  tags:
7
+ - wizardlm
8
  - uncensored
9
  - gptq
10
+ - auto-gptq
11
  - 7b
12
  - llama
13
+ ---
14
+
15
+ # Get Started
16
+
17
+
18
+ ```py
19
+ from transformers import AutoTokenizer, pipeline, AutoModelForCausalLM, LlamaForCausalLM, LlamaTokenizer, StoppingCriteria, PreTrainedTokenizerBase
20
+ from auto_gptq import AutoGPTQForCausalLM
21
+
22
+ model_id = 'seonglae/wizardlm-7b-uncensored-gptq'
23
+ tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
24
+ model = AutoGPTQForCausalLM.from_quantized(
25
+ model_id,
26
+ model_basename=model_basename,
27
+ trust_remote_code=True,
28
+ device='cuda:0',
29
+ use_triton=False,
30
+ use_safetensors=True,
31
+ )
32
+
33
+ pipe = pipeline(
34
+ "text-generation",
35
+ model=model,
36
+ tokenizer=tokenizer,
37
+ temperature=0.5,
38
+ top_p=0.95,
39
+ max_new_tokens=100,
40
+ repetition_penalty=1.15,
41
+ )
42
+ prompt = "USER: Are you AI?\nASSISTANT:"
43
+ pipe(prompt)
44
+ ```