Update README.md
Browse files
README.md
CHANGED
@@ -15,9 +15,16 @@ pipeline_tag: text-generation
|
|
15 |
---
|
16 |
# OPT-125M finetuned Portuguese
|
17 |
|
18 |
-
Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of
|
19 |
|
20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
|
22 |
With an A100 with 40GB of RAM, the training took around 3 hours
|
23 |
|
|
|
15 |
---
|
16 |
# OPT-125M finetuned Portuguese
|
17 |
|
18 |
+
Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.
|
19 |
|
20 |
+
###### Hyper-parameters
|
21 |
+
|
22 |
+
|
23 |
+
- learning_rate = 5e-5
|
24 |
+
- batch_size = 32
|
25 |
+
- warmup = 500
|
26 |
+
- seq_length = 512
|
27 |
+
- num_train_epochs = 2.0
|
28 |
|
29 |
With an A100 with 40GB of RAM, the training took around 3 hours
|
30 |
|