Update README.md
Browse files
README.md
CHANGED
@@ -43,12 +43,13 @@ ReidLM, like all large language models, has inherent biases and limitations that
|
|
43 |
|
44 |
This section is meant to convey recommendations with respect to the bias, risk, and technical limitations.
|
45 |
|
46 |
-
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations
|
47 |
|
48 |
## Getting Started with the Model
|
49 |
|
50 |
Use the code below to get started with the model.
|
51 |
## Use with Transformers AutoModelForCausalLM
|
|
|
52 |
```
|
53 |
import transformers
|
54 |
import torch
|
@@ -69,11 +70,9 @@ generated_text = generate_text(prompt)
|
|
69 |
print(generated_text)
|
70 |
```
|
71 |
|
72 |
-
<br>
|
73 |
-
|
74 |
## Training Details
|
75 |
|
76 |
-
|
77 |
### Training Data
|
78 |
|
79 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
@@ -86,18 +85,18 @@ print(generated_text)
|
|
86 |
|
87 |
#### Training Hyperparameters
|
88 |
|
89 |
-
|
90 |
-
|
91 |
-
|
92 |
-
|
93 |
-
|
94 |
-
|
95 |
-
|
96 |
-
|
97 |
-
|
98 |
-
|
99 |
-
|
100 |
-
|
101 |
<!---#### Speeds, Sizes, Times [optional]
|
102 |
|
103 |
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
|
|
43 |
|
44 |
This section is meant to convey recommendations with respect to the bias, risk, and technical limitations.
|
45 |
|
46 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.-->
|
47 |
|
48 |
## Getting Started with the Model
|
49 |
|
50 |
Use the code below to get started with the model.
|
51 |
## Use with Transformers AutoModelForCausalLM
|
52 |
+
|
53 |
```
|
54 |
import transformers
|
55 |
import torch
|
|
|
70 |
print(generated_text)
|
71 |
```
|
72 |
|
|
|
|
|
73 |
## Training Details
|
74 |
|
75 |
+
<!-- -->
|
76 |
### Training Data
|
77 |
|
78 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
|
|
85 |
|
86 |
#### Training Hyperparameters
|
87 |
|
88 |
+
num_train_epochs=3, <br>
|
89 |
+
per_device_train_batch_size=4,<br>
|
90 |
+
gradient_accumulation_steps=2,<br>
|
91 |
+
optim="paged_adamw_8bit",<br>
|
92 |
+
save_steps=1000,<br>
|
93 |
+
logging_steps=30,<br>
|
94 |
+
learning_rate=2e-4,<br>
|
95 |
+
weight_decay=0.01,<br>
|
96 |
+
fp16=True,<br>
|
97 |
+
max_grad_norm=1.0,<br>
|
98 |
+
warmup_ratio=0.1<br>
|
99 |
+
<!-- -->
|
100 |
<!---#### Speeds, Sizes, Times [optional]
|
101 |
|
102 |
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|