ayjays132 commited on
Commit
3e52639
·
verified ·
1 Parent(s): b73df13

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -156,17 +156,17 @@ Here's a concise overview of the key hyperparameters used for training the model
156
  - `clip`: 5
157
  - `patience`: 7
158
  - `adaptation_rate`: 0.05
159
- - `sequence_length`: 200
160
- - `max_sequence_length`: 200
161
  - `weight_decay`: 0.005
162
- - `num_embeddings`: 25,000
163
- - `embedding_dim`: 768
164
  - `hidden_dim`: 2048
165
  - `learning_rate`: 1e-5
166
- - `some_intermediate_size`: 3072
167
 
168
  **Additional Parameters**
169
- - `input_dimension`: 768
170
  - `initial_neuron_count`: 5000
171
  - `some_adaptation_rate`: 0.05
172
  - `complexity_metric`: None
 
156
  - `clip`: 5
157
  - `patience`: 7
158
  - `adaptation_rate`: 0.05
159
+ - `sequence_length`: 2048
160
+ - `max_sequence_length`: 2048
161
  - `weight_decay`: 0.005
162
+ - `num_embeddings`: 100,000
163
+ - `embedding_dim`: 2048
164
  - `hidden_dim`: 2048
165
  - `learning_rate`: 1e-5
166
+ - `some_intermediate_size`: 2048
167
 
168
  **Additional Parameters**
169
+ - `input_dimension`: 2048
170
  - `initial_neuron_count`: 5000
171
  - `some_adaptation_rate`: 0.05
172
  - `complexity_metric`: None