BigHuggyD commited on
Commit
4f508c8
1 Parent(s): 42a3a63

Upload model weights (#1)

Browse files

- 387c2de05c91a9de54feffd987fadca1ee847ae809f7a46929ff2fc9e747c37d (74ad396d4d4b1f20da5f92f4e49989d9aee28742)
- ed5f853644dc75dc2f6890c88c2e852148bdeec15d8c91f58f95bf95f5a45e47 (c0cbf847a2c57570e92ef4ea31aa9d4f157ac5dc)
- 706ad59cf2492aa30b80f074a7f8e61ac8ea78a9da79829c52feda8cec51a5be (803fc9477699f2ac12dd17d27f7432e1c7dea751)
- 8a6467baa79f2d90861d7f7f83ea7ef1b08a4c55dcf3f5211e099484448ff79d (0e96868173d40ea6a0a97ebd7ba69d5aca3b861d)
- 3fad1aebe7dd282015a22a71a7055a56d71241946a4e9d1e9d379a1012e9d78a (1fd87ee01044158feaec57f01694aa4fc308986e)
- 63785199d675f93f9afe9dba795424c88e92a42afea9fed8bf56c92d14c818f9 (99c0186d88f9ed30d978fc91c868688350b79c9a)
- e93e265d4f0c34b174d6601159f1873eb486df3de1beaeabae59645cf37a4b36 (b3cef1573743360ba37af9ac78dd674792555628)
- fed1c03f27aa3fdd3eea4e061fa304f7a192e8d083b8d49604e3225be8c75af6 (049ee7c5d2956c60d25fb8d7ddd96434bde01fad)

README.md ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - 152334H/miqu-1-70b-sf
4
+ - meta-llama/Llama-2-70b-hf
5
+ - sophosympatheia/Midnight-Rose-70B-v2.0.3
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+ license: other
11
+ ---
12
+
13
+ A creative writing model with 32k context, created by adding the essence of "what makes [Midnight-Rose-70B-v2.0.3](https://huggingface.co/sophosympatheia/Midnight-Rose-70B-v2.0.3) different to [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf)" onto [miqu-1-70b](https://huggingface.co/miqudev/miqu-1-70b).
14
+
15
+ # Prompting format
16
+
17
+ Vicuna format is preferred:
18
+
19
+ ```
20
+ USER: {prompt} ASSISTANT:
21
+ ```
22
+
23
+ Mistral and Alpaca formats are also supported:
24
+
25
+ ```
26
+ [INST] {prompt} [/INST]
27
+ ```
28
+
29
+ ```
30
+ ### Instruction:
31
+ {prompt}
32
+
33
+ ### Response:
34
+ ```
35
+
36
+ # Licence and usage restrictions
37
+
38
+ [miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf) is a dequantized version of the [miqu-1-70b](https://huggingface.co/miqudev/miqu-1-70b) model leaked from MistralAI. All miqu-derived models, including this merge, are suitable for non-commercial, personal use only.
39
+
40
+ # Mergekit configuration
41
+
42
+ The following YAML configuration was used to produce this model:
43
+
44
+ ```yaml
45
+ name: _miquplus-midnight-70b
46
+ merge_method: task_arithmetic
47
+ parameters:
48
+ normalize : false
49
+ weight: 1
50
+ models:
51
+ - model: meta-llama/Llama-2-70b-hf
52
+ - model: 152334H/miqu-1-70b-sf
53
+ - model: sophosympatheia/Midnight-Rose-70B-v2.0.3
54
+ base_model: meta-llama/Llama-2-70b-hf
55
+ dtype: float16
56
+ ---
57
+ name: miquplus-midnight-70b
58
+ merge_method: linear
59
+ models:
60
+ - model: 152334H/miqu-1-70b-sf
61
+ parameters:
62
+ weight:
63
+ - filter: v_proj
64
+ value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
65
+ - filter: o_proj
66
+ value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
67
+ - filter: up_proj
68
+ value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
69
+ - filter: gate_proj
70
+ value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
71
+ - filter: down_proj
72
+ value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
73
+ - value: 1
74
+ - model: _miquplus-midnight-70b
75
+ parameters:
76
+ weight:
77
+ - filter: v_proj
78
+ value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
79
+ - filter: o_proj
80
+ value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
81
+ - filter: up_proj
82
+ value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
83
+ - filter: gate_proj
84
+ value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
85
+ - filter: down_proj
86
+ value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
87
+ - value: 0
88
+ base_model: 152334H/miqu-1-70b-sf
89
+ tokenizer_source: base
90
+ dtype: float16
91
+ ```
92
+
93
+ **NOTE**: Run with `mergekit-mega` rather than `mergekit` as there are 2 documents in this file.
94
+
95
+ # Model background
96
+
97
+ Created using [Mergekit](https://github.com/arcee-ai/mergekit) in two stages:
98
+
99
+ - First, a (broken!) "donor" model was created by adding (`Midnight-Rose-70B-v2.0.3` - `Llama-2-70b-hf`) to `miqu-1-70b-sf` using `task_arithmetic`.
100
+ - In the second stage, the `v_proj`, `o_proj`, `up_proj`, `gate_proj` and `down_proj` tensors were merged back into `miqu-1-70b-sf`.
101
+
102
+ **NOTE**: After the second stage model is created the "donor" model can be deleted as it won't be needed again.
103
+
104
+ <details><summary>Click to see more details on the reasoning behind the merge</summary>
105
+
106
+ ## A. There are 3 very compelling reasons to keep the `q_proj` and `k_proj` matrices intact:
107
+
108
+ ### 1. These are the only matrices that have the RoPE rotations applied:
109
+
110
+ ![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/4j88A7gCkkifwxUX45gOQ.png)
111
+ see: https://adalkiran.github.io/llama-nuts-and-bolts/#model-diagram
112
+
113
+ and trying to blend the the `q_proj` and `k_proj` matrices of models with 100x different base RoPE frequencies is just going to contract and dilate their perceived "distance" / "time" in the text they write (so both are wrong). If you did want to insist on blending them, then the base RoPE frequency would need to be set to something in-between that is the "least wrong" for both base RoPE frequencies and the maximum context reduced accordingly...
114
+
115
+ ### 2. They don't actually appear to be responsible for domain-specific adaptation anyway:
116
+ ![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/B4GL59ViEA66gMsYKvbZH.png)
117
+
118
+ see: https://www.lesswrong.com/posts/j84JhErNezMxyK4dH/llm-modularity-the-separability-of-capabilities-in-large
119
+
120
+ ### 3: The top ~30% of embedding dimensions for the 10k base RoPE frequency models aren't being used properly:
121
+ > For LLaMA2(Touvron et al., 2023b), the critical dimension dextra is 92. This implies that only the first 92 dimensions of the qt, ks vectors of LLaMA2 have seen the complete positional information during the pre-training phase and are adequately trained. In other words, the last 36 dimensions lack sufficient training, contributing to the extrapolation challenges seen in RoPE-based LLMs
122
+
123
+ see: [Scaling Laws of RoPE-based Extrapolation](https://arxiv.org/abs/2310.05209)
124
+
125
+ So even if we could try to map the 10k RoPE models embedding vectors to match the 1M RoPE models embedding vectors post-dot-product, the 1M RoPE models embedding vectors are likely fine-tuned to use these top 36 dimensions properly and have thus changed very significantly during continued pre-training.
126
+
127
+ ## B. The norm vectors from the "donor" model can't be merged:
128
+
129
+ The `input_layernorm.weight`, `post_attention_layernorm.weight` and `norm.weight` vectors all use [LayerNorm](https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html), and:
130
+
131
+ > **weight** – the learnable weights of the module of shape normalized_shapenormalized_shape when elementwise_affine is set to True. The values are initialized to 1.
132
+
133
+ So to combine these we would need to use the [Geometric Mean](https://en.wikipedia.org/wiki/Geometric_mean) for the `task_arithmetic` operation (and hence why the "donor" model is broken).
134
+
135
+ ## C. The first and last 8 layers, the `embed_tokens.weight` and `lm_head.weight` tensors are not merged:
136
+
137
+ The model still functions if these are merged into the second stage model, but empirically it seems to work better if we don't do this as keep the original `miqu-1-70b-sf` weights.
138
+
139
+ </details>
config.json ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LlamaForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "silu",
10
+ "hidden_size": 8192,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 28672,
13
+ "max_position_embeddings": 32764,
14
+ "model_type": "llama",
15
+ "num_attention_heads": 64,
16
+ "num_hidden_layers": 80,
17
+ "num_key_value_heads": 8,
18
+ "pad_token_id": 0,
19
+ "pretraining_tp": 1,
20
+ "rms_norm_eps": 1e-05,
21
+ "rope_scaling": null,
22
+ "rope_theta": 1000000,
23
+ "tie_word_embeddings": false,
24
+ "torch_dtype": "float16",
25
+ "transformers_version": "4.39.3",
26
+ "use_cache": true,
27
+ "vocab_size": 32000,
28
+ "quantization_config": {
29
+ "quant_method": "exl2",
30
+ "version": "0.1.4",
31
+ "bits": 7.0,
32
+ "head_bits": 6,
33
+ "calibration": {
34
+ "rows": 100,
35
+ "length": 2048,
36
+ "dataset": "(default)"
37
+ }
38
+ }
39
+ }
huggingface-metadata.txt ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ url: https://huggingface.co/jukofyork/miquplus-midnight-70b
2
+ branch: main
3
+ download date: 2024-06-04 21:05:38
4
+ sha256sum:
5
+ 22ff297394158b9191939c4889c89b943472a27d0a2d218eec230059c21a9a92 model-00001-of-00015.safetensors
6
+ 0352518038e9bdd6c2d6903fd136012cf2656782fee0ac8d5b4887626d0aaa24 model-00002-of-00015.safetensors
7
+ 488aeeb4e584bd67a98f290adac025b3a7a5e3c01d2591d3ef29fc28f20fc19c model-00003-of-00015.safetensors
8
+ 72e13eecf27c62b663da21b217b0f43a205ff55512ee9f7d2460862a88fef557 model-00004-of-00015.safetensors
9
+ 0436255ea1570b74650035031e4a9aa28bfd8b0962ed077a7c520304f41c232f model-00005-of-00015.safetensors
10
+ b088a089b12d8026363bc26b736e5e2ae69ba23d0ee17a2baf7864c3d4dde49b model-00006-of-00015.safetensors
11
+ 40038dedff9272b6072cc04c29dbdf681adf025e3b0e458a8614377828177ac2 model-00007-of-00015.safetensors
12
+ 6bb7739b57b794edd366db37ad09eb6592d4d49eb466808c58a865cbfd5cec31 model-00008-of-00015.safetensors
13
+ 9a304949f23392c0e5c34a21b00a5339723dff86b76bbf8d89fb3e95323770a3 model-00009-of-00015.safetensors
14
+ 017e6a27acd52113266d9098c2fcc50a16fb7352c63bd2e461aa84860c8eca98 model-00010-of-00015.safetensors
15
+ 69509d433f250ac9ae7a53b16f7cbc430f63aa1e8db95b3d5199130c91cfcabb model-00011-of-00015.safetensors
16
+ 01e0e4fb5855c12fbd03d0a6a218dd2ea8f5d9ec6906889bc2be6e1ba1ca6dc2 model-00012-of-00015.safetensors
17
+ 2c81d5294f91cac5ca851b2005403afcd262d0a9409a424f6e26253f0e76458c model-00013-of-00015.safetensors
18
+ ea6705de902ea829bdc4b6516b5b26dfe4f89939a59bf908865e465a434e492c model-00014-of-00015.safetensors
19
+ 62ee6037a95c66861f7910a82168886a63e071750d4872ad43c980c724c68de2 model-00015-of-00015.safetensors
20
+ 9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347 tokenizer.model
model.safetensors.index.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"metadata": {"mergekit_version": "0.0.4.2", "total_size": 137953296384}, "weight_map": {"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00015.safetensors", "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00015.safetensors", "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.mlp.down_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.mlp.up_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.1.input_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.0.mlp.down_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.mlp.up_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.0.input_layernorm.weight": "model-00001-of-00015.safetensors", "model.embed_tokens.weight": "model-00001-of-00015.safetensors", "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00015.safetensors", "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.mlp.down_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.mlp.up_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.4.input_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.3.mlp.down_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.mlp.up_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.3.input_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.2.mlp.down_proj.weight": "model-00001-of-00015.safetensors", "model.layers.2.mlp.up_proj.weight": "model-00001-of-00015.safetensors", "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.2.input_layernorm.weight": "model-00001-of-00015.safetensors", "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00015.safetensors", "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00015.safetensors", "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00015.safetensors", "model.layers.7.mlp.down_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.mlp.up_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00015.safetensors", "model.layers.7.input_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.6.mlp.down_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.mlp.up_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00015.safetensors", "model.layers.6.input_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.5.mlp.down_proj.weight": "model-00002-of-00015.safetensors", "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00015.safetensors", "model.layers.5.mlp.up_proj.weight": "model-00002-of-00015.safetensors", "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.5.input_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.10.mlp.down_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.mlp.up_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00015.safetensors", "model.layers.10.input_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.9.mlp.down_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.mlp.up_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00015.safetensors", "model.layers.9.input_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.8.mlp.down_proj.weight": "model-00002-of-00015.safetensors", "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00015.safetensors", "model.layers.8.mlp.up_proj.weight": "model-00002-of-00015.safetensors", "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00015.safetensors", "model.layers.8.input_layernorm.weight": "model-00002-of-00015.safetensors", "model.layers.13.mlp.gate_proj.weight": "model-00003-of-00015.safetensors", "model.layers.13.mlp.up_proj.weight": "model-00003-of-00015.safetensors", "model.layers.13.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.13.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.13.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.13.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.mlp.down_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.mlp.gate_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.mlp.up_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.post_attention_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.12.input_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.11.mlp.down_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.mlp.gate_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.mlp.up_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.11.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.11.input_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00015.safetensors", "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.mlp.down_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.mlp.gate_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.mlp.up_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.15.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.15.input_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.14.mlp.down_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.mlp.gate_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.mlp.up_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.post_attention_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.14.input_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.13.mlp.down_proj.weight": "model-00003-of-00015.safetensors", "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.13.input_layernorm.weight": "model-00003-of-00015.safetensors", "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00015.safetensors", "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00015.safetensors", "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00015.safetensors", "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00015.safetensors", "model.layers.18.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.mlp.gate_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.mlp.up_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.post_attention_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.18.self_attn.o_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.self_attn.v_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.self_attn.k_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.self_attn.q_proj.weight": "model-00004-of-00015.safetensors", "model.layers.18.input_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.17.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.mlp.gate_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.mlp.up_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.post_attention_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.17.self_attn.o_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.self_attn.v_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.self_attn.k_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.self_attn.q_proj.weight": "model-00004-of-00015.safetensors", "model.layers.17.input_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.16.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.16.mlp.up_proj.weight": "model-00004-of-00015.safetensors", "model.layers.16.post_attention_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.16.input_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.22.self_attn.v_proj.weight": "model-00004-of-00015.safetensors", "model.layers.22.self_attn.k_proj.weight": "model-00004-of-00015.safetensors", "model.layers.22.self_attn.q_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.mlp.gate_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.mlp.up_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.post_attention_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.21.self_attn.o_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.self_attn.v_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.self_attn.k_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.self_attn.q_proj.weight": "model-00004-of-00015.safetensors", "model.layers.21.input_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.20.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.mlp.gate_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.mlp.up_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.post_attention_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.20.self_attn.o_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.self_attn.v_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.self_attn.k_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.self_attn.q_proj.weight": "model-00004-of-00015.safetensors", "model.layers.20.input_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.19.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.19.mlp.gate_proj.weight": "model-00004-of-00015.safetensors", "model.layers.19.mlp.up_proj.weight": "model-00004-of-00015.safetensors", "model.layers.19.post_attention_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.19.input_layernorm.weight": "model-00004-of-00015.safetensors", "model.layers.24.mlp.down_proj.weight": "model-00004-of-00015.safetensors", "model.layers.24.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.24.mlp.up_proj.weight": "model-00005-of-00015.safetensors", "model.layers.24.post_attention_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.24.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.24.self_attn.v_proj.weight": "model-00005-of-00015.safetensors", "model.layers.24.self_attn.k_proj.weight": "model-00005-of-00015.safetensors", "model.layers.24.self_attn.q_proj.weight": "model-00005-of-00015.safetensors", "model.layers.24.input_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.23.mlp.down_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.mlp.up_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.post_attention_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.23.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.self_attn.v_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.self_attn.k_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.self_attn.q_proj.weight": "model-00005-of-00015.safetensors", "model.layers.23.input_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.22.mlp.down_proj.weight": "model-00005-of-00015.safetensors", "model.layers.22.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.22.mlp.up_proj.weight": "model-00005-of-00015.safetensors", "model.layers.22.post_attention_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.22.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.22.input_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.27.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.27.mlp.up_proj.weight": "model-00005-of-00015.safetensors", "model.layers.27.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.27.self_attn.v_proj.weight": "model-00005-of-00015.safetensors", "model.layers.27.self_attn.k_proj.weight": "model-00005-of-00015.safetensors", "model.layers.27.self_attn.q_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.mlp.down_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.mlp.up_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.post_attention_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.26.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.self_attn.v_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.self_attn.k_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.self_attn.q_proj.weight": "model-00005-of-00015.safetensors", "model.layers.26.input_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.25.mlp.down_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.mlp.up_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.post_attention_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.25.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.self_attn.v_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.self_attn.k_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.self_attn.q_proj.weight": "model-00005-of-00015.safetensors", "model.layers.25.input_layernorm.weight": "model-00005-of-00015.safetensors", "model.layers.30.mlp.gate_proj.weight": "model-00005-of-00015.safetensors", "model.layers.30.self_attn.o_proj.weight": "model-00005-of-00015.safetensors", "model.layers.30.self_attn.v_proj.weight": "model-00005-of-00015.safetensors", "model.layers.30.self_attn.k_proj.weight": "model-00005-of-00015.safetensors", "model.layers.30.self_attn.q_proj.weight": "model-00005-of-00015.safetensors", "model.layers.29.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.mlp.gate_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.mlp.up_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.post_attention_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.29.self_attn.o_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.self_attn.v_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.self_attn.k_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.self_attn.q_proj.weight": "model-00006-of-00015.safetensors", "model.layers.29.input_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.28.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.mlp.gate_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.mlp.up_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.post_attention_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.28.self_attn.o_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.self_attn.v_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.self_attn.k_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.self_attn.q_proj.weight": "model-00006-of-00015.safetensors", "model.layers.28.input_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.27.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.27.post_attention_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.27.input_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.33.self_attn.o_proj.weight": "model-00006-of-00015.safetensors", "model.layers.33.self_attn.v_proj.weight": "model-00006-of-00015.safetensors", "model.layers.33.self_attn.k_proj.weight": "model-00006-of-00015.safetensors", "model.layers.33.self_attn.q_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.mlp.gate_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.mlp.up_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.post_attention_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.32.self_attn.o_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.self_attn.v_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.self_attn.k_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.self_attn.q_proj.weight": "model-00006-of-00015.safetensors", "model.layers.32.input_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.31.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.mlp.gate_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.mlp.up_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.post_attention_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.31.self_attn.o_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.self_attn.v_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.self_attn.k_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.self_attn.q_proj.weight": "model-00006-of-00015.safetensors", "model.layers.31.input_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.30.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.30.mlp.up_proj.weight": "model-00006-of-00015.safetensors", "model.layers.30.post_attention_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.30.input_layernorm.weight": "model-00006-of-00015.safetensors", "model.layers.36.self_attn.v_proj.weight": "model-00006-of-00015.safetensors", "model.layers.36.self_attn.k_proj.weight": "model-00006-of-00015.safetensors", "model.layers.36.self_attn.q_proj.weight": "model-00006-of-00015.safetensors", "model.layers.35.mlp.down_proj.weight": "model-00006-of-00015.safetensors", "model.layers.35.mlp.gate_proj.weight": "model-00006-of-00015.safetensors", "model.layers.35.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.35.post_attention_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.35.self_attn.o_proj.weight": "model-00007-of-00015.safetensors", "model.layers.35.self_attn.v_proj.weight": "model-00007-of-00015.safetensors", "model.layers.35.self_attn.k_proj.weight": "model-00007-of-00015.safetensors", "model.layers.35.self_attn.q_proj.weight": "model-00007-of-00015.safetensors", "model.layers.35.input_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.34.mlp.down_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.mlp.gate_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.post_attention_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.34.self_attn.o_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.self_attn.v_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.self_attn.k_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.self_attn.q_proj.weight": "model-00007-of-00015.safetensors", "model.layers.34.input_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.33.mlp.down_proj.weight": "model-00007-of-00015.safetensors", "model.layers.33.mlp.gate_proj.weight": "model-00007-of-00015.safetensors", "model.layers.33.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.33.post_attention_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.33.input_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.38.mlp.down_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.mlp.gate_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.post_attention_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.38.self_attn.o_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.self_attn.v_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.self_attn.k_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.self_attn.q_proj.weight": "model-00007-of-00015.safetensors", "model.layers.38.input_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.37.mlp.down_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.mlp.gate_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.post_attention_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.37.self_attn.o_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.self_attn.v_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.self_attn.k_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.self_attn.q_proj.weight": "model-00007-of-00015.safetensors", "model.layers.37.input_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.36.mlp.down_proj.weight": "model-00007-of-00015.safetensors", "model.layers.36.mlp.gate_proj.weight": "model-00007-of-00015.safetensors", "model.layers.36.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.36.post_attention_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.36.self_attn.o_proj.weight": "model-00007-of-00015.safetensors", "model.layers.36.input_layernorm.weight": "model-00007-of-00015.safetensors", "model.layers.41.mlp.gate_proj.weight": "model-00007-of-00015.safetensors", "model.layers.41.mlp.up_proj.weight": "model-00007-of-00015.safetensors", "model.layers.41.self_attn.o_proj.weight": "model-00007-of-00015.safetensors", "model.layers.41.self_attn.v_proj.weight": "model-00007-of-00015.safetensors", "model.layers.41.self_attn.k_proj.weight": "model-00007-of-00015.safetensors", "model.layers.41.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.mlp.down_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.mlp.gate_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.mlp.up_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.post_attention_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.40.self_attn.o_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.self_attn.v_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.self_attn.k_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.40.input_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.39.mlp.down_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.mlp.gate_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.mlp.up_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.post_attention_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.39.self_attn.o_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.self_attn.v_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.self_attn.k_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.39.input_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.44.mlp.gate_proj.weight": "model-00008-of-00015.safetensors", "model.layers.44.self_attn.o_proj.weight": "model-00008-of-00015.safetensors", "model.layers.44.self_attn.v_proj.weight": "model-00008-of-00015.safetensors", "model.layers.44.self_attn.k_proj.weight": "model-00008-of-00015.safetensors", "model.layers.44.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.mlp.down_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.mlp.gate_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.mlp.up_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.post_attention_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.43.self_attn.o_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.self_attn.v_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.self_attn.k_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.43.input_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.42.mlp.down_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.mlp.gate_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.mlp.up_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.post_attention_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.42.self_attn.o_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.self_attn.v_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.self_attn.k_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.42.input_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.41.mlp.down_proj.weight": "model-00008-of-00015.safetensors", "model.layers.41.post_attention_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.41.input_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.47.self_attn.o_proj.weight": "model-00008-of-00015.safetensors", "model.layers.47.self_attn.v_proj.weight": "model-00008-of-00015.safetensors", "model.layers.47.self_attn.k_proj.weight": "model-00008-of-00015.safetensors", "model.layers.47.self_attn.q_proj.weight": "model-00008-of-00015.safetensors", "model.layers.46.mlp.down_proj.weight": "model-00008-of-00015.safetensors", "model.layers.46.mlp.gate_proj.weight": "model-00008-of-00015.safetensors", "model.layers.46.mlp.up_proj.weight": "model-00008-of-00015.safetensors", "model.layers.46.post_attention_layernorm.weight": "model-00008-of-00015.safetensors", "model.layers.46.self_attn.o_proj.weight": "model-00009-of-00015.safetensors", "model.layers.46.self_attn.v_proj.weight": "model-00009-of-00015.safetensors", "model.layers.46.self_attn.k_proj.weight": "model-00009-of-00015.safetensors", "model.layers.46.self_attn.q_proj.weight": "model-00009-of-00015.safetensors", "model.layers.46.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.45.mlp.down_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.mlp.gate_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.mlp.up_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.post_attention_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.45.self_attn.o_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.self_attn.v_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.self_attn.k_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.self_attn.q_proj.weight": "model-00009-of-00015.safetensors", "model.layers.45.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.44.mlp.down_proj.weight": "model-00009-of-00015.safetensors", "model.layers.44.mlp.up_proj.weight": "model-00009-of-00015.safetensors", "model.layers.44.post_attention_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.44.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.50.self_attn.v_proj.weight": "model-00009-of-00015.safetensors", "model.layers.50.self_attn.k_proj.weight": "model-00009-of-00015.safetensors", "model.layers.50.self_attn.q_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.mlp.down_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.mlp.gate_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.mlp.up_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.post_attention_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.49.self_attn.o_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.self_attn.v_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.self_attn.k_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.self_attn.q_proj.weight": "model-00009-of-00015.safetensors", "model.layers.49.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.48.mlp.down_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.mlp.gate_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.mlp.up_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.post_attention_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.48.self_attn.o_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.self_attn.v_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.self_attn.k_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.self_attn.q_proj.weight": "model-00009-of-00015.safetensors", "model.layers.48.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.47.mlp.down_proj.weight": "model-00009-of-00015.safetensors", "model.layers.47.mlp.gate_proj.weight": "model-00009-of-00015.safetensors", "model.layers.47.mlp.up_proj.weight": "model-00009-of-00015.safetensors", "model.layers.47.post_attention_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.47.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.52.mlp.down_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.mlp.gate_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.mlp.up_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.post_attention_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.52.self_attn.o_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.self_attn.v_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.self_attn.k_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.self_attn.q_proj.weight": "model-00009-of-00015.safetensors", "model.layers.52.input_layernorm.weight": "model-00009-of-00015.safetensors", "model.layers.51.mlp.down_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.mlp.up_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.post_attention_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.51.self_attn.o_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.self_attn.v_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.self_attn.k_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.self_attn.q_proj.weight": "model-00010-of-00015.safetensors", "model.layers.51.input_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.50.mlp.down_proj.weight": "model-00010-of-00015.safetensors", "model.layers.50.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.50.mlp.up_proj.weight": "model-00010-of-00015.safetensors", "model.layers.50.post_attention_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.50.self_attn.o_proj.weight": "model-00010-of-00015.safetensors", "model.layers.50.input_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.55.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.55.mlp.up_proj.weight": "model-00010-of-00015.safetensors", "model.layers.55.self_attn.o_proj.weight": "model-00010-of-00015.safetensors", "model.layers.55.self_attn.v_proj.weight": "model-00010-of-00015.safetensors", "model.layers.55.self_attn.k_proj.weight": "model-00010-of-00015.safetensors", "model.layers.55.self_attn.q_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.mlp.down_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.mlp.up_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.post_attention_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.54.self_attn.o_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.self_attn.v_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.self_attn.k_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.self_attn.q_proj.weight": "model-00010-of-00015.safetensors", "model.layers.54.input_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.53.mlp.down_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.mlp.up_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.post_attention_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.53.self_attn.o_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.self_attn.v_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.self_attn.k_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.self_attn.q_proj.weight": "model-00010-of-00015.safetensors", "model.layers.53.input_layernorm.weight": "model-00010-of-00015.safetensors", "model.layers.58.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.58.self_attn.o_proj.weight": "model-00010-of-00015.safetensors", "model.layers.58.self_attn.v_proj.weight": "model-00010-of-00015.safetensors", "model.layers.58.self_attn.k_proj.weight": "model-00010-of-00015.safetensors", "model.layers.58.self_attn.q_proj.weight": "model-00010-of-00015.safetensors", "model.layers.57.mlp.down_proj.weight": "model-00010-of-00015.safetensors", "model.layers.57.mlp.gate_proj.weight": "model-00010-of-00015.safetensors", "model.layers.57.mlp.up_proj.weight": "model-00011-of-00015.safetensors", "model.layers.57.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.57.self_attn.o_proj.weight": "model-00011-of-00015.safetensors", "model.layers.57.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.57.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.57.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.57.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.56.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.mlp.gate_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.mlp.up_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.56.self_attn.o_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.56.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.55.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.55.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.55.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.61.self_attn.o_proj.weight": "model-00011-of-00015.safetensors", "model.layers.61.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.61.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.61.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.mlp.gate_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.mlp.up_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.60.self_attn.o_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.60.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.59.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.mlp.gate_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.mlp.up_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.59.self_attn.o_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.59.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.58.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.58.mlp.up_proj.weight": "model-00011-of-00015.safetensors", "model.layers.58.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.58.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.64.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.64.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.64.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.mlp.gate_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.mlp.up_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.post_attention_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.63.self_attn.o_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.self_attn.v_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.self_attn.k_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.self_attn.q_proj.weight": "model-00011-of-00015.safetensors", "model.layers.63.input_layernorm.weight": "model-00011-of-00015.safetensors", "model.layers.62.mlp.down_proj.weight": "model-00011-of-00015.safetensors", "model.layers.62.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.62.mlp.up_proj.weight": "model-00012-of-00015.safetensors", "model.layers.62.post_attention_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.62.self_attn.o_proj.weight": "model-00012-of-00015.safetensors", "model.layers.62.self_attn.v_proj.weight": "model-00012-of-00015.safetensors", "model.layers.62.self_attn.k_proj.weight": "model-00012-of-00015.safetensors", "model.layers.62.self_attn.q_proj.weight": "model-00012-of-00015.safetensors", "model.layers.62.input_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.61.mlp.down_proj.weight": "model-00012-of-00015.safetensors", "model.layers.61.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.61.mlp.up_proj.weight": "model-00012-of-00015.safetensors", "model.layers.61.post_attention_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.61.input_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.66.mlp.down_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.mlp.up_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.post_attention_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.66.self_attn.o_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.self_attn.v_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.self_attn.k_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.self_attn.q_proj.weight": "model-00012-of-00015.safetensors", "model.layers.66.input_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.65.mlp.down_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.mlp.up_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.post_attention_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.65.self_attn.o_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.self_attn.v_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.self_attn.k_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.self_attn.q_proj.weight": "model-00012-of-00015.safetensors", "model.layers.65.input_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.64.mlp.down_proj.weight": "model-00012-of-00015.safetensors", "model.layers.64.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.64.mlp.up_proj.weight": "model-00012-of-00015.safetensors", "model.layers.64.post_attention_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.64.self_attn.o_proj.weight": "model-00012-of-00015.safetensors", "model.layers.64.input_layernorm.weight": "model-00012-of-00015.safetensors", "model.layers.69.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.69.mlp.up_proj.weight": "model-00012-of-00015.safetensors", "model.layers.69.self_attn.o_proj.weight": "model-00012-of-00015.safetensors", "model.layers.69.self_attn.v_proj.weight": "model-00012-of-00015.safetensors", "model.layers.69.self_attn.k_proj.weight": "model-00012-of-00015.safetensors", "model.layers.69.self_attn.q_proj.weight": "model-00012-of-00015.safetensors", "model.layers.68.mlp.down_proj.weight": "model-00012-of-00015.safetensors", "model.layers.68.mlp.gate_proj.weight": "model-00012-of-00015.safetensors", "model.layers.68.mlp.up_proj.weight": "model-00013-of-00015.safetensors", "model.layers.68.post_attention_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.68.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.68.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.68.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.68.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.68.input_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.67.mlp.down_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.mlp.gate_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.mlp.up_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.post_attention_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.67.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.67.input_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.72.mlp.gate_proj.weight": "model-00013-of-00015.safetensors", "model.layers.72.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.72.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.72.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.72.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.mlp.down_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.mlp.gate_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.mlp.up_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.post_attention_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.71.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.71.input_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.70.mlp.down_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.mlp.gate_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.mlp.up_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.post_attention_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.70.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.70.input_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.69.mlp.down_proj.weight": "model-00013-of-00015.safetensors", "model.layers.69.post_attention_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.69.input_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.75.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.75.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.75.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.75.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.mlp.down_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.mlp.gate_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.mlp.up_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.post_attention_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.74.self_attn.o_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.self_attn.v_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.self_attn.k_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.self_attn.q_proj.weight": "model-00013-of-00015.safetensors", "model.layers.74.input_layernorm.weight": "model-00013-of-00015.safetensors", "model.layers.73.mlp.down_proj.weight": "model-00013-of-00015.safetensors", "model.layers.73.mlp.gate_proj.weight": "model-00014-of-00015.safetensors", "model.layers.73.mlp.up_proj.weight": "model-00014-of-00015.safetensors", "model.layers.73.post_attention_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.73.self_attn.o_proj.weight": "model-00014-of-00015.safetensors", "model.layers.73.self_attn.v_proj.weight": "model-00014-of-00015.safetensors", "model.layers.73.self_attn.k_proj.weight": "model-00014-of-00015.safetensors", "model.layers.73.self_attn.q_proj.weight": "model-00014-of-00015.safetensors", "model.layers.73.input_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.72.mlp.down_proj.weight": "model-00014-of-00015.safetensors", "model.layers.72.mlp.up_proj.weight": "model-00014-of-00015.safetensors", "model.layers.72.post_attention_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.72.input_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.78.self_attn.v_proj.weight": "model-00014-of-00015.safetensors", "model.layers.78.self_attn.k_proj.weight": "model-00014-of-00015.safetensors", "model.layers.78.self_attn.q_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.mlp.down_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.mlp.gate_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.mlp.up_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.post_attention_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.77.self_attn.o_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.self_attn.v_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.self_attn.k_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.self_attn.q_proj.weight": "model-00014-of-00015.safetensors", "model.layers.77.input_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.76.mlp.down_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.mlp.gate_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.mlp.up_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.post_attention_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.76.self_attn.o_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.self_attn.v_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.self_attn.k_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.self_attn.q_proj.weight": "model-00014-of-00015.safetensors", "model.layers.76.input_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.75.mlp.down_proj.weight": "model-00014-of-00015.safetensors", "model.layers.75.mlp.gate_proj.weight": "model-00014-of-00015.safetensors", "model.layers.75.mlp.up_proj.weight": "model-00014-of-00015.safetensors", "model.layers.75.post_attention_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.75.input_layernorm.weight": "model-00014-of-00015.safetensors", "lm_head.weight": "model-00014-of-00015.safetensors", "model.norm.weight": "model-00014-of-00015.safetensors", "model.layers.79.mlp.down_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.mlp.gate_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.mlp.up_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.post_attention_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.79.self_attn.o_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.self_attn.v_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.self_attn.k_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.self_attn.q_proj.weight": "model-00014-of-00015.safetensors", "model.layers.79.input_layernorm.weight": "model-00014-of-00015.safetensors", "model.layers.78.mlp.down_proj.weight": "model-00014-of-00015.safetensors", "model.layers.78.mlp.gate_proj.weight": "model-00015-of-00015.safetensors", "model.layers.78.mlp.up_proj.weight": "model-00015-of-00015.safetensors", "model.layers.78.post_attention_layernorm.weight": "model-00015-of-00015.safetensors", "model.layers.78.self_attn.o_proj.weight": "model-00015-of-00015.safetensors", "model.layers.78.input_layernorm.weight": "model-00015-of-00015.safetensors"}}
output-00001-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:245e5cfd71251807f8119ee12c8b1d55c00e5eea2a3075103a9766b13f37e85d
3
+ size 8449437676
output-00002-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8df86e8c1d4a3a17fe2571a0fee41a2aed26b96801511ba2801a80defa8eec93
3
+ size 8571049516
output-00003-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ff13aa68718ad17812a73cbd21d4bf45db4daff093bf0b799d670509d2d9171
3
+ size 8410115468
output-00004-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a60888a8d668bcbaaab7af93aa02018ca29a1eec38d6fc9a42a0655aa0ea6f9e
3
+ size 8509982764
output-00005-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:784b012e4bc2aac3ce5c69cbf9e22d5290d2eed2f25b577df8f4672f4275820a
3
+ size 8466730496
output-00006-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a60318450d5087f11be8235cdfd1af3f8bb59585cebaaa471d9872cc5f857658
3
+ size 8394015872
output-00007-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb77113dc73089c8475171c7e29f14e8c087a728e8dab92d02eb345b3ca0ee59
3
+ size 8468678296
output-00008-of-00008.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f731c6350b0e8a02988ce4c522eb44c106487d4aeb4ab3bfc3cd3a1d5d30e514
3
+ size 1147732656
special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<unk>",
18
+ "lstrip": false,
19
+ "normalized": true,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": true,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
tokenizer_config.json ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": true,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<unk>",
8
+ "lstrip": false,
9
+ "normalized": true,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<s>",
16
+ "lstrip": false,
17
+ "normalized": true,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "2": {
23
+ "content": "</s>",
24
+ "lstrip": false,
25
+ "normalized": true,
26
+ "rstrip": false,
27
+ "single_word": false,
28
+ "special": true
29
+ }
30
+ },
31
+ "bos_token": "<s>",
32
+ "clean_up_tokenization_spaces": false,
33
+ "eos_token": "</s>",
34
+ "legacy": false,
35
+ "model_max_length": 1000000000000000019884624838656,
36
+ "pad_token": "<unk>",
37
+ "sp_model_kwargs": {},
38
+ "spaces_between_special_tokens": false,
39
+ "tokenizer_class": "LlamaTokenizer",
40
+ "unk_token": "<unk>",
41
+ "use_default_system_prompt": false
42
+ }