update model to step 127090

.ipynb_checkpoints/README-checkpoint.md

@@ -19,19 +19,21 @@ More precise versions will be published shortly.<br/>
 <br/>
 Train on my server, i have studied and adapted the model starting from the repository https://github.com/karpathy/llama2.c<br/>
 <br/>
-# max_seq_len: (7b = 2048) The maximum sequence length for input data.<br/>
-# dim (7b= 4096) Represents the dimensionality of the model<br/>
-# n_layers: (7b = 32) The number of layers in the model<br/>
-# n_heads: (7b = 32) Determines the number of attention heads in the model<br/>
-# n_kv_heads: (7b = 32) The number of key and value heads<br/>
-# multiple_of: (7b = 256) A value used to make the SwiGLU hidden layer size a multiple of a large power of 2<br/>
+- LLama model parameter:
+  - max_seq_len: (7b = 2048) The maximum sequence length for input data.
+  - dim (7b= 4096) Represents the dimensionalityl
+  - n_layers: (7b = 32) The number of layers
+  - n_heads: (7b = 32) Determines the number of attention heads
+  - n_kv_heads: (7b = 32) The number of key and value heads
+  - multiple_of: (7b = 256) A value used to make the SwiGLU hidden layer size a multiple of a large power of 2
 <br/>
-max_seq_len = 1024<br/>
-dim         = 768<br/>
-n_layers    = 32<br/>
-n_heads     = 32<br/>
-n_kv_heads  = 32<br/>
-multiple_of = 32<br/>
+- Model parameter
+  - max_seq_len = 1024
+  - dim         = 768
+  - n_layers    = 32
+  - n_heads     = 32
+  - n_kv_heads  = 32
+  - multiple_of = 32
 <br/>
 num decayed parameter tensors: 225, with 251,068,416 parameters<br/>
 num non-decayed parameter tensors: 65, with 49,920 parameters<br/>

README.md

@@ -19,19 +19,21 @@ More precise versions will be published shortly.<br/>
 <br/>
 Train on my server, i have studied and adapted the model starting from the repository https://github.com/karpathy/llama2.c<br/>
 <br/>
-# max_seq_len: (7b = 2048) The maximum sequence length for input data.<br/>
-# dim (7b= 4096) Represents the dimensionality of the model<br/>
-# n_layers: (7b = 32) The number of layers in the model<br/>
-# n_heads: (7b = 32) Determines the number of attention heads in the model<br/>
-# n_kv_heads: (7b = 32) The number of key and value heads<br/>
-# multiple_of: (7b = 256) A value used to make the SwiGLU hidden layer size a multiple of a large power of 2<br/>
+- LLama model parameter:
+  - max_seq_len: (7b = 2048) The maximum sequence length for input data.
+  - dim (7b= 4096) Represents the dimensionalityl
+  - n_layers: (7b = 32) The number of layers
+  - n_heads: (7b = 32) Determines the number of attention heads
+  - n_kv_heads: (7b = 32) The number of key and value heads
+  - multiple_of: (7b = 256) A value used to make the SwiGLU hidden layer size a multiple of a large power of 2
 <br/>
-max_seq_len = 1024<br/>
-dim         = 768<br/>
-n_layers    = 32<br/>
-n_heads     = 32<br/>
-n_kv_heads  = 32<br/>
-multiple_of = 32<br/>
+- Model parameter
+  - max_seq_len = 1024
+  - dim         = 768
+  - n_layers    = 32
+  - n_heads     = 32
+  - n_kv_heads  = 32
+  - multiple_of = 32
 <br/>
 num decayed parameter tensors: 225, with 251,068,416 parameters<br/>
 num non-decayed parameter tensors: 65, with 49,920 parameters<br/>

config.json

@@ -20,7 +20,7 @@
   "rope_scaling": null,
   "rope_theta": 10000.0,
   "tie_word_embeddings": true,
-  "transformers_version": "4.37.1",
+  "transformers_version": "4.39.3",
   "use_cache": true,
   "vocab_size": 32000
 }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ab62b69b46b7f795f22d07447f33fa985864f7fdd281df9a3d26834a1750744f
+oid sha256:92be5db673d53c20628c7aab33078d4c7c4c44db04ddda6412317e58b14b7346
 size 1004567442