Commit
·
5909726
1
Parent(s):
af85fa9
Update README.md
Browse files
README.md
CHANGED
@@ -6,4 +6,37 @@ library_name: transformers
|
|
6 |
pipeline_tag: text-generation
|
7 |
tags:
|
8 |
- gpt2
|
9 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
pipeline_tag: text-generation
|
7 |
tags:
|
8 |
- gpt2
|
9 |
+
---
|
10 |
+
# FineNeo: A simple way to finetune your very own GPT-Neo model.
|
11 |
+
> Created by Tekkonetes with debugging help from ChatGPT.
|
12 |
+
> @Tekkonetes (HuggingFace) / @pxlmastrXD (Replit)
|
13 |
+
|
14 |
+
So, you want to fine-tune a GPT-Neo model? Well, here's the simplest script you will get. It uses a text dataset to fine-tune, and it also will tune the model fairly quickly. Normally, it takes about 5 seconds for an epoch to finish, so here are some estimated times:
|
15 |
+
|Epochs|Time (Seconds)| Adjusted time
|
16 |
+
|--|--|--|
|
17 |
+
| 1 | 5 | |
|
18 |
+
| 10 | 50 |
|
19 |
+
| 50 | 250 | 4m 10s
|
20 |
+
| 100 | 500 | 8m 20s
|
21 |
+
|
22 |
+
Yes, it's fairly fast. However, it depends on which GPT-Neo model you're fine-tuning. For example, the chart above is the `EleutherAI/GPT-Neo-125M` model. If you use the `Gpt-Neo-1.3B` model, it will probably take longer.
|
23 |
+
|
24 |
+
## Using the script.
|
25 |
+
First, download the `tune.py` file to your computer. Then (optional) set up a virtual environment:
|
26 |
+
```bash
|
27 |
+
python -m venv venv
|
28 |
+
source venv/bin/activate
|
29 |
+
```
|
30 |
+
Now, install the needed packages:
|
31 |
+
```bash
|
32 |
+
pip install transformers torch
|
33 |
+
```
|
34 |
+
Finally, create your dataset, modify the file to use the dataset, and run the python script.
|
35 |
+
```bash
|
36 |
+
python tune.py
|
37 |
+
```
|
38 |
+
Your model and tokenizer will appear in the `fine-tuned-gpt-neo` directory. You can then use transformers to run the model, or upload the files to the HuggingFace hub.
|
39 |
+
|
40 |
+
Best of luck!
|
41 |
+
- Tekkonetes
|
42 |
+
- @pxlmastrXD (Replit)
|