ayjays132 commited on
Commit
b5d0f61
1 Parent(s): 473b792

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md CHANGED
@@ -93,6 +93,44 @@ Empowered by an Innovative Growth Network for dynamic adaptation:
93
  - **Initial Capacity:** 250000
94
  - **Input Size:** 2048
95
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  ## Seamless Integration with Hugging Face 🤗
97
 
98
  from transformers import AutoTokenizer, AutoModelForCausalLM
 
93
  - **Initial Capacity:** 250000
94
  - **Input Size:** 2048
95
 
96
+ ---
97
+
98
+ ## Hyperparameters Overview
99
+
100
+ Here's a concise overview of the key hyperparameters used for training your model:
101
+
102
+ **Training Parameters**
103
+ - `max_neurons`: 250,000
104
+ - `epochs`: 50
105
+ - `clip`: 5
106
+ - `patience`: 7
107
+ - `adaptation_rate`: 0.05
108
+ - `sequence_length`: 200
109
+ - `max_sequence_length`: 200
110
+ - `weight_decay`: 0.005
111
+ - `num_embeddings`: 25,000
112
+ - `embedding_dim`: 768
113
+ - `hidden_dim`: 2048
114
+ - `learning_rate`: 1e-5
115
+ - `some_intermediate_size`: 3072
116
+
117
+ **Additional Parameters**
118
+ - `input_dimension`: 768
119
+ - `initial_neuron_count`: 5000
120
+ - `some_adaptation_rate`: 0.05
121
+ - `complexity_metric`: None
122
+
123
+ **New Parameters**
124
+ - `growth_improvement_threshold`: 0.01
125
+ - `pruning_improvement_threshold`: 0.005
126
+ - `stability_threshold`: 0.02
127
+ - `max_complexity`: 10,000
128
+ - `low_stability_threshold`: 0.01
129
+ - `min_epochs_before_growth`: 5
130
+ - `desired_improvement_rate`: 0.02
131
+
132
+ ---
133
+
134
  ## Seamless Integration with Hugging Face 🤗
135
 
136
  from transformers import AutoTokenizer, AutoModelForCausalLM