Update README.md
Browse files
README.md
CHANGED
@@ -21,18 +21,17 @@ This gemma2 model was trained 2x faster with [Unsloth](https://github.com/unslot
|
|
21 |
|
22 |
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
23 |
|
24 |
-
##
|
|
|
|
|
25 |
|
26 |
-
|
27 |
-
|
|
|
|
|
28 |
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
๋ฐ์ดํฐ์
์ AI-Hub์ ์๋ ๋
ผ๋ฌธ์๋ฃ ์์ฝ ๋ฐ์ดํฐ ์
๊ณผ ํคํ๋ฆฌ์ค์์ ์ง์ ์ฒญ๊ตฌํญ์ ๊ฐ์ ธ์ ์กฐํฉํ ๋ฐ์ดํฐ์
์ ์ด์ฉํ์ต๋๋ค.
|
33 |
-
|
34 |
-
## ๋ชจ๋ธ ํ์ต
|
35 |
-
๋ชจ๋ธ ํ์ต์ loRA๋ฅผ ์ด์ฉํ์ฌ ์งํํ์์ผ๋ฉฐ, ํ์ต์ ์ฌ์ฉ๋ ์ฝ๋๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
|
36 |
```
|
37 |
model = FastLanguageModel.get_peft_model(
|
38 |
model,
|
@@ -82,9 +81,9 @@ trainer = SFTTrainer(
|
|
82 |
```
|
83 |
|
84 |
|
85 |
-
##
|
86 |
|
87 |
-
1. unsloth
|
88 |
```
|
89 |
%%capture
|
90 |
!pip install unsloth
|
@@ -97,7 +96,7 @@ if torch.cuda.get_device_capability()[0] >= 8:
|
|
97 |
!pip install --no-deps packaging ninja einops "flash-attn>=2.6.3"
|
98 |
```
|
99 |
|
100 |
-
2.
|
101 |
```
|
102 |
from unsloth import FastLanguageModel
|
103 |
import torch
|
@@ -114,7 +113,7 @@ model, tokenizer = FastLanguageModel.from_pretrained(
|
|
114 |
token = token
|
115 |
)
|
116 |
```
|
117 |
-
3.
|
118 |
```
|
119 |
input = """
|
120 |
์์ ํ ๊ณผ์ ๋ฅผ ํด๊ฒฐํ๊ธฐ ์ํ์ฌ, ๋ณธ ๊ณ ์์ ๋ด๋ถ์ ๋ณด๊ดํ ๋ฌผ๊ฑด์ ๋ฃ์ ์ ์๋ ๊ธฐ๋ณธ ๋ด์ฅ ๊ณต๊ฐ๊ณผ ์ด๋ฅผ ๋๋ฌ์ผ
|
@@ -163,9 +162,10 @@ _ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1000)
|
|
163 |
```
|
164 |
|
165 |
|
166 |
-
##
|
167 |
-
|
168 |
-
|
|
|
169 |
```
|
170 |
[๋ฐ๋ช
์ ๋ช
์นญ]
|
171 |
๊ฐ๋ฐฉ
|
|
|
21 |
|
22 |
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
23 |
|
24 |
+
## Model Overview
|
25 |
+
This model is fine-tuned to assist with drafting patent specifications based on a general description of an invention.
|
26 |
+
The base model is unsloth/gemma-2-2b-it, and I used unsloth to merge the fine-tuned adapter.
|
27 |
|
28 |
+
## Dataset
|
29 |
+
The dataset used for fine-tuning includes a combination of research paper
|
30 |
+
summary datasets from AI-Hub and patent claims data directly retrieved from KIPRIS
|
31 |
+
(Korea Intellectual Property Rights Information Service).
|
32 |
|
33 |
+
Model Training
|
34 |
+
The model was trained using LoRA (Low-Rank Adaptation). The following code was used for training:
|
|
|
|
|
|
|
|
|
|
|
35 |
```
|
36 |
model = FastLanguageModel.get_peft_model(
|
37 |
model,
|
|
|
81 |
```
|
82 |
|
83 |
|
84 |
+
## How to Use the Model
|
85 |
|
86 |
+
1. Install unsloth:
|
87 |
```
|
88 |
%%capture
|
89 |
!pip install unsloth
|
|
|
96 |
!pip install --no-deps packaging ninja einops "flash-attn>=2.6.3"
|
97 |
```
|
98 |
|
99 |
+
2. Load the fine-tuned model and use it for inference:
|
100 |
```
|
101 |
from unsloth import FastLanguageModel
|
102 |
import torch
|
|
|
113 |
token = token
|
114 |
)
|
115 |
```
|
116 |
+
3. Write a prompt and generate text:
|
117 |
```
|
118 |
input = """
|
119 |
์์ ํ ๊ณผ์ ๋ฅผ ํด๊ฒฐํ๊ธฐ ์ํ์ฌ, ๋ณธ ๊ณ ์์ ๋ด๋ถ์ ๋ณด๊ดํ ๋ฌผ๊ฑด์ ๋ฃ์ ์ ์๋ ๊ธฐ๋ณธ ๋ด์ฅ ๊ณต๊ฐ๊ณผ ์ด๋ฅผ ๋๋ฌ์ผ
|
|
|
162 |
```
|
163 |
|
164 |
|
165 |
+
## Model Results
|
166 |
+
The model was tested using the "Means to Solve the Problem" section from actual patent specifications.
|
167 |
+
When compared with real patent documents, the model generated content that was relatively similar in
|
168 |
+
structure and meaning.
|
169 |
```
|
170 |
[๋ฐ๋ช
์ ๋ช
์นญ]
|
171 |
๊ฐ๋ฐฉ
|