limecoding commited on
Commit
33fc9e4
ยท
verified ยท
1 Parent(s): 1caec0f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -17
README.md CHANGED
@@ -21,18 +21,17 @@ This gemma2 model was trained 2x faster with [Unsloth](https://github.com/unslot
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
23
 
24
- ## ๋ชจ๋ธ ์†Œ๊ฐœ
 
 
25
 
26
- ํ•ด๋‹น ๋ชจ๋ธ์€ ๋Œ€๋žต์ ์ธ ๋ฐœ๋ช…ํ’ˆ์˜ ์„ค๋ช…์„ ์ž…๋ ฅ์œผ๋กœ ๋ฐ›์•„ ํŠนํ—ˆ ๋ช…์„ธ์„œ ์ž‘์„ฑ์„ ๋„์™€์ฃผ๋Š” ํŒŒ์ธํŠœ๋‹๋œ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.
27
- ๋ฒ ์ด์Šค ๋ชจ๋ธ์€ unsloth/gemma-2-2b-it์ด๋ฉฐ, unsloth๋ฅผ ์ด์šฉํ•ด ํŒŒ์ธํŠœ๋‹๋œ adapter๋ฅผ ๋ณ‘ํ•ฉํ–ˆ์Šต๋‹ˆ๋‹ค.
 
 
28
 
29
-
30
- ## ๋ฐ์ดํ„ฐ์…‹
31
-
32
- ๋ฐ์ดํ„ฐ์…‹์€ AI-Hub์— ์žˆ๋Š” ๋…ผ๋ฌธ์ž๋ฃŒ ์š”์•ฝ ๋ฐ์ดํ„ฐ ์…‹๊ณผ ํ‚คํ”„๋ฆฌ์Šค์—์„œ ์ง์ ‘ ์ฒญ๊ตฌํ•ญ์„ ๊ฐ€์ ธ์™€ ์กฐํ•ฉํ•œ ๋ฐ์ดํ„ฐ์…‹์„ ์ด์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.
33
-
34
- ## ๋ชจ๋ธ ํ•™์Šต
35
- ๋ชจ๋ธ ํ•™์Šต์€ loRA๋ฅผ ์ด์šฉํ•˜์—ฌ ์ง„ํ–‰ํ•˜์˜€์œผ๋ฉฐ, ํ•™์Šต์— ์‚ฌ์šฉ๋œ ์ฝ”๋“œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
36
  ```
37
  model = FastLanguageModel.get_peft_model(
38
  model,
@@ -82,9 +81,9 @@ trainer = SFTTrainer(
82
  ```
83
 
84
 
85
- ## ๋ชจ๋ธ ์‚ฌ์šฉ๋ฒ•
86
 
87
- 1. unsloth๋ฅผ ์„ค์น˜ํ•ฉ๋‹ˆ๋‹ค
88
  ```
89
  %%capture
90
  !pip install unsloth
@@ -97,7 +96,7 @@ if torch.cuda.get_device_capability()[0] >= 8:
97
  !pip install --no-deps packaging ninja einops "flash-attn>=2.6.3"
98
  ```
99
 
100
- 2. ๋ชจ๋ธ์„ ๋ถˆ๋Ÿฌ์˜ต๋‹ˆ๋‹ค.
101
  ```
102
  from unsloth import FastLanguageModel
103
  import torch
@@ -114,7 +113,7 @@ model, tokenizer = FastLanguageModel.from_pretrained(
114
  token = token
115
  )
116
  ```
117
- 3. ํ”„๋กฌํ”„ํŠธ๋ฅผ ์ž‘์„ฑํ•˜์—ฌ ํ…์ŠคํŠธ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
118
  ```
119
  input = """
120
  ์ƒ์ˆ ํ•œ ๊ณผ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•˜์—ฌ, ๋ณธ ๊ณ ์•ˆ์€ ๋‚ด๋ถ€์— ๋ณด๊ด€ํ•  ๋ฌผ๊ฑด์„ ๋„ฃ์„ ์ˆ˜ ์žˆ๋Š” ๊ธฐ๋ณธ ๋‚ด์žฅ ๊ณต๊ฐ„๊ณผ ์ด๋ฅผ ๋‘˜๋Ÿฌ์‹ผ
@@ -163,9 +162,10 @@ _ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1000)
163
  ```
164
 
165
 
166
- ## ๋ชจ๋ธ ๊ฒฐ๊ณผ
167
- ํ•ด๋‹น ๋ชจ๋ธ๋กœ ์‹ค์ œ ํŠนํ—ˆ ๋ช…์„ธ์„œ์˜ ๊ณผ์ œ ํ•ด๊ฒฐ ์ˆ˜๋‹จ ํ•ญ๋ชฉ์„ ๊ฐ€์ง€๊ณ  ํ…Œ์ŠคํŠธํ–ˆ์œผ๋ฉฐ ์‹ค์ œ ๋ฌธ์„œ์™€ ๋น„๊ตํ–ˆ์„ ๋•Œ
168
- ๋น„๊ต์  ์œ ์‚ฌํ•œ ๋‚ด์šฉ์„ ์ƒ์„ฑํ–ˆ๋‹ค.
 
169
  ```
170
  [๋ฐœ๋ช…์˜ ๋ช…์นญ]
171
  ๊ฐ€๋ฐฉ
 
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
23
 
24
+ ## Model Overview
25
+ This model is fine-tuned to assist with drafting patent specifications based on a general description of an invention.
26
+ The base model is unsloth/gemma-2-2b-it, and I used unsloth to merge the fine-tuned adapter.
27
 
28
+ ## Dataset
29
+ The dataset used for fine-tuning includes a combination of research paper
30
+ summary datasets from AI-Hub and patent claims data directly retrieved from KIPRIS
31
+ (Korea Intellectual Property Rights Information Service).
32
 
33
+ Model Training
34
+ The model was trained using LoRA (Low-Rank Adaptation). The following code was used for training:
 
 
 
 
 
35
  ```
36
  model = FastLanguageModel.get_peft_model(
37
  model,
 
81
  ```
82
 
83
 
84
+ ## How to Use the Model
85
 
86
+ 1. Install unsloth:
87
  ```
88
  %%capture
89
  !pip install unsloth
 
96
  !pip install --no-deps packaging ninja einops "flash-attn>=2.6.3"
97
  ```
98
 
99
+ 2. Load the fine-tuned model and use it for inference:
100
  ```
101
  from unsloth import FastLanguageModel
102
  import torch
 
113
  token = token
114
  )
115
  ```
116
+ 3. Write a prompt and generate text:
117
  ```
118
  input = """
119
  ์ƒ์ˆ ํ•œ ๊ณผ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•˜์—ฌ, ๋ณธ ๊ณ ์•ˆ์€ ๋‚ด๋ถ€์— ๋ณด๊ด€ํ•  ๋ฌผ๊ฑด์„ ๋„ฃ์„ ์ˆ˜ ์žˆ๋Š” ๊ธฐ๋ณธ ๋‚ด์žฅ ๊ณต๊ฐ„๊ณผ ์ด๋ฅผ ๋‘˜๋Ÿฌ์‹ผ
 
162
  ```
163
 
164
 
165
+ ## Model Results
166
+ The model was tested using the "Means to Solve the Problem" section from actual patent specifications.
167
+ When compared with real patent documents, the model generated content that was relatively similar in
168
+ structure and meaning.
169
  ```
170
  [๋ฐœ๋ช…์˜ ๋ช…์นญ]
171
  ๊ฐ€๋ฐฉ