tekkonetes commited on
Commit
5909726
·
1 Parent(s): af85fa9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -1
README.md CHANGED
@@ -6,4 +6,37 @@ library_name: transformers
6
  pipeline_tag: text-generation
7
  tags:
8
  - gpt2
9
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  pipeline_tag: text-generation
7
  tags:
8
  - gpt2
9
+ ---
10
+ # FineNeo: A simple way to finetune your very own GPT-Neo model.
11
+ > Created by Tekkonetes with debugging help from ChatGPT.
12
+ > @Tekkonetes (HuggingFace) / @pxlmastrXD (Replit)
13
+
14
+ So, you want to fine-tune a GPT-Neo model? Well, here's the simplest script you will get. It uses a text dataset to fine-tune, and it also will tune the model fairly quickly. Normally, it takes about 5 seconds for an epoch to finish, so here are some estimated times:
15
+ |Epochs|Time (Seconds)| Adjusted time
16
+ |--|--|--|
17
+ | 1 | 5 | |
18
+ | 10 | 50 |
19
+ | 50 | 250 | 4m 10s
20
+ | 100 | 500 | 8m 20s
21
+
22
+ Yes, it's fairly fast. However, it depends on which GPT-Neo model you're fine-tuning. For example, the chart above is the `EleutherAI/GPT-Neo-125M` model. If you use the `Gpt-Neo-1.3B` model, it will probably take longer.
23
+
24
+ ## Using the script.
25
+ First, download the `tune.py` file to your computer. Then (optional) set up a virtual environment:
26
+ ```bash
27
+ python -m venv venv
28
+ source venv/bin/activate
29
+ ```
30
+ Now, install the needed packages:
31
+ ```bash
32
+ pip install transformers torch
33
+ ```
34
+ Finally, create your dataset, modify the file to use the dataset, and run the python script.
35
+ ```bash
36
+ python tune.py
37
+ ```
38
+ Your model and tokenizer will appear in the `fine-tuned-gpt-neo` directory. You can then use transformers to run the model, or upload the files to the HuggingFace hub.
39
+
40
+ Best of luck!
41
+ - Tekkonetes
42
+ - @pxlmastrXD (Replit)