84basi commited on
Commit
aa6f1ca
·
verified ·
1 Parent(s): b05b31e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +67 -0
README.md CHANGED
@@ -9,6 +9,8 @@ tags:
9
  license: apache-2.0
10
  language:
11
  - en
 
 
12
  ---
13
 
14
  # Uploaded model
@@ -20,3 +22,68 @@ language:
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  license: apache-2.0
10
  language:
11
  - en
12
+ datasets:
13
+ - elyza/ELYZA-tasks-100
14
  ---
15
 
16
  # Uploaded model
 
22
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
23
 
24
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
25
+
26
+
27
+ ## Readme
28
+
29
+ ```python
30
+ token = "" # token
31
+ model_id = "llm-jp-3-13b-it-4.20" # llm-jp-3-13b-it-4.17, gemma-2-27b-it-4.19
32
+ model_name = "84basi/" + model_id
33
+ answer_json_file = "./elyza-tasks-100-TV_0.jsonl"
34
+ output_json_file = "./" + model_id + "_output.jsonl"
35
+
36
+ %%capture
37
+ !pip install unsloth -q
38
+ !pip uninstall unsloth -y && pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git" -q
39
+
40
+ from unsloth import FastLanguageModel
41
+ from peft import PeftModel
42
+ import torch
43
+ import json
44
+
45
+ max_seq_length = 2048
46
+ dtype = None
47
+ load_in_4bit = True
48
+
49
+ model, tokenizer = FastLanguageModel.from_pretrained(
50
+ model_name = model_name,
51
+ max_seq_length = max_seq_length,
52
+ dtype = dtype,
53
+ load_in_4bit = load_in_4bit,
54
+ token = token,
55
+ trust_remote_code=True,
56
+ )
57
+
58
+ # 推論モードに切り替え
59
+ FastLanguageModel.for_inference(model)
60
+
61
+ # データセットの読み込み。
62
+ # omnicampusの開発環境では、左にタスクのjsonlをドラッグアンドドロップしてから実行。
63
+ datasets = []
64
+ with open(answer_json_file, "r") as f:
65
+ item = ""
66
+ for line in f:
67
+ line = line.strip()
68
+ item += line
69
+ if item.endswith("}"):
70
+ datasets.append(json.loads(item))
71
+ item = ""
72
+
73
+ # 推論
74
+ from tqdm import tqdm
75
+
76
+ results = []
77
+ for dt in tqdm(datasets):
78
+ input = dt["input"]
79
+ prompt = f"""### 指示\n{input}\n### 回答\n"""
80
+ inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
81
+ outputs = model.generate(**inputs, max_new_tokens=512, use_cache=True, do_sample=False, repetition_penalty=1.2)
82
+ prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]
83
+ results.append({"task_id": dt["task_id"], "input": input, "output": prediction})
84
+
85
+ with open(output_json_file, 'w', encoding='utf-8') as f:
86
+ for result in results:
87
+ json.dump(result, f, ensure_ascii=False)
88
+ f.write('\n')
89
+ ```