Hugging Face's logo

mlc-ai
/
Mixtral-8x7B-Instruct-v0.1-q0f16-MLC

MLC-LLM
web-llm
Model card Files Community
Mixtral-8x7B-Instruct-v0.1-q0f16-MLC / logs.txt
riczhou's picture
riczhou
Initial commit
85516e0 verified
raw
history blame contribute delete
89.5 kB
/opt/conda/envs/py310/bin/python -m mlc_llm gen_config /models/Mixtral-8x7B-Instruct-v0.1 --quantization q0f16 --conv-template mistral_default --output /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC
[2024-06-06 22:24:15] INFO auto_config.py:116: Found model configuration: /models/Mixtral-8x7B-Instruct-v0.1/config.json
[2024-06-06 22:24:15] INFO auto_config.py:154: Found model type: mixtral. Use `--model-type` to override.
[2024-06-06 22:24:15] INFO llama_model.py:52: context_window_size not found in config.json. Falling back to max_position_embeddings (32768)
[2024-06-06 22:24:15] INFO llama_model.py:72: prefill_chunk_size defaults to 2048
[2024-06-06 22:24:15] INFO config.py:107: Overriding max_batch_size from 1 to 80
[2024-06-06 22:24:15] INFO gen_config.py:143: [generation_config.json] Setting bos_token_id: 1
[2024-06-06 22:24:15] INFO gen_config.py:143: [generation_config.json] Setting eos_token_id: 2
[2024-06-06 22:24:15] INFO gen_config.py:155: Found tokenizer config: /models/Mixtral-8x7B-Instruct-v0.1/tokenizer.model. Copying to /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC/tokenizer.model
[2024-06-06 22:24:15] INFO gen_config.py:155: Found tokenizer config: /models/Mixtral-8x7B-Instruct-v0.1/tokenizer.json. Copying to /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC/tokenizer.json
[2024-06-06 22:24:15] INFO gen_config.py:157: Not found tokenizer config: /models/Mixtral-8x7B-Instruct-v0.1/vocab.json
[2024-06-06 22:24:15] INFO gen_config.py:157: Not found tokenizer config: /models/Mixtral-8x7B-Instruct-v0.1/merges.txt
[2024-06-06 22:24:15] INFO gen_config.py:157: Not found tokenizer config: /models/Mixtral-8x7B-Instruct-v0.1/added_tokens.json
[2024-06-06 22:24:15] INFO gen_config.py:155: Found tokenizer config: /models/Mixtral-8x7B-Instruct-v0.1/tokenizer_config.json. Copying to /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC/tokenizer_config.json
[2024-06-06 22:24:15] INFO gen_config.py:216: Detected tokenizer info: {'token_postproc_method': 'byte_fallback', 'prepend_space_in_encode': True, 'strip_space_in_decode': True}
[2024-06-06 22:24:15] INFO gen_config.py:32: [System default] Setting pad_token_id: 0
[2024-06-06 22:24:15] INFO gen_config.py:32: [System default] Setting temperature: 1.0
[2024-06-06 22:24:15] INFO gen_config.py:32: [System default] Setting presence_penalty: 0.0
[2024-06-06 22:24:15] INFO gen_config.py:32: [System default] Setting frequency_penalty: 0.0
[2024-06-06 22:24:15] INFO gen_config.py:32: [System default] Setting repetition_penalty: 1.0
[2024-06-06 22:24:15] INFO gen_config.py:32: [System default] Setting top_p: 1.0
[2024-06-06 22:24:15] INFO gen_config.py:223: Dumping configuration file to: /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC/mlc-chat-config.json
/opt/conda/envs/py310/bin/python -m mlc_llm convert_weight /models/Mixtral-8x7B-Instruct-v0.1 --quantization q0f16 --output /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC
[2024-06-06 22:24:17] INFO auto_config.py:116: Found model configuration: /models/Mixtral-8x7B-Instruct-v0.1/config.json
[2024-06-06 22:24:18] INFO auto_device.py:79: Found device: cuda:0
[2024-06-06 22:24:20] INFO auto_device.py:88: Not found device: rocm:0
[2024-06-06 22:24:21] INFO auto_device.py:88: Not found device: metal:0
[2024-06-06 22:24:23] INFO auto_device.py:79: Found device: vulkan:0
[2024-06-06 22:24:23] INFO auto_device.py:79: Found device: vulkan:1
[2024-06-06 22:24:23] INFO auto_device.py:79: Found device: vulkan:2
[2024-06-06 22:24:23] INFO auto_device.py:79: Found device: vulkan:3
[2024-06-06 22:24:24] INFO auto_device.py:88: Not found device: opencl:0
[2024-06-06 22:24:24] INFO auto_device.py:35: Using device: cuda:0
[2024-06-06 22:24:24] INFO auto_weight.py:71: Finding weights in: /models/Mixtral-8x7B-Instruct-v0.1
[2024-06-06 22:24:24] INFO auto_weight.py:137: Not found Huggingface PyTorch
[2024-06-06 22:24:24] INFO auto_weight.py:144: Found source weight format: huggingface-safetensor. Source configuration: /models/Mixtral-8x7B-Instruct-v0.1/model.safetensors.index.json
[2024-06-06 22:24:24] INFO auto_weight.py:107: Using source weight configuration: /models/Mixtral-8x7B-Instruct-v0.1/model.safetensors.index.json. Use `--source` to override.
[2024-06-06 22:24:24] INFO auto_weight.py:111: Using source weight format: huggingface-safetensor. Use `--source-format` to override.
[2024-06-06 22:24:24] INFO auto_config.py:154: Found model type: mixtral. Use `--model-type` to override.
[2024-06-06 22:24:24] INFO llama_model.py:52: context_window_size not found in config.json. Falling back to max_position_embeddings (32768)
[2024-06-06 22:24:24] INFO llama_model.py:72: prefill_chunk_size defaults to 2048
Weight conversion with arguments:
--config /models/Mixtral-8x7B-Instruct-v0.1/config.json
--quantization NoQuantize(name='q0f16', kind='no-quant', model_dtype='float16')
--model-type mixtral
--device cuda:0
--source /models/Mixtral-8x7B-Instruct-v0.1/model.safetensors.index.json
--source-format huggingface-safetensor
--output /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC
Start storing to cache /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC
0%| | 0/227 [00:00<?, ?it/s] [2024-06-06 22:25:56] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00019-of-00019.safetensors
0%| | 0/227 [00:00<?, ?it/s] [2024-06-06 22:26:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "lm_head.weight", shape: (32000, 4096), dtype: float16
0%| | 0/227 [00:05<?, ?it/s] 0%| | 1/227 [00:05<22:09, 5.88s/it] [2024-06-06 22:26:02] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00018-of-00019.safetensors
0%| | 1/227 [00:05<22:09, 5.88s/it] [2024-06-06 22:26:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
0%| | 1/227 [00:19<22:09, 5.88s/it] 1%| | 2/227 [00:24<49:39, 13.24s/it] [2024-06-06 22:26:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
1%| | 2/227 [00:28<49:39, 13.24s/it] 1%|▏ | 3/227 [00:31<38:21, 10.27s/it] [2024-06-06 22:26:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.input_layernorm.weight", shape: (4096,), dtype: float16
1%|▏ | 3/227 [00:31<38:21, 10.27s/it] [2024-06-06 22:26:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.post_attention_layernorm.weight", shape: (4096,), dtype: float16
1%|▏ | 3/227 [00:31<38:21, 10.27s/it] [2024-06-06 22:26:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
1%|▏ | 3/227 [00:40<38:21, 10.27s/it] 3%|▎ | 6/227 [00:45<24:38, 6.69s/it] [2024-06-06 22:26:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
3%|▎ | 6/227 [00:51<24:38, 6.69s/it] 3%|▎ | 7/227 [00:53<25:41, 7.01s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.moe.gate.weight", shape: (8, 4096), dtype: float16
3%|▎ | 7/227 [00:53<25:41, 7.01s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.input_layernorm.weight", shape: (4096,), dtype: float16
3%|▎ | 7/227 [00:53<25:41, 7.01s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.post_attention_layernorm.weight", shape: (4096,), dtype: float16
3%|▎ | 7/227 [00:53<25:41, 7.01s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
3%|▎ | 7/227 [00:53<25:41, 7.01s/it] 5%|▍ | 11/227 [00:53<10:49, 3.01s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
5%|▍ | 11/227 [00:54<10:49, 3.01s/it] 5%|▌ | 12/227 [00:54<09:01, 2.52s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.norm.weight", shape: (4096,), dtype: float16
5%|▌ | 12/227 [00:54<09:01, 2.52s/it] [2024-06-06 22:26:50] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00019-of-00019.safetensors
5%|▌ | 12/227 [00:54<09:01, 2.52s/it] [2024-06-06 22:26:51] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00018-of-00019.safetensors
5%|▌ | 12/227 [00:54<09:01, 2.52s/it] [2024-06-06 22:26:51] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00001-of-00019.safetensors
5%|▌ | 12/227 [00:54<09:01, 2.52s/it] [2024-06-06 22:26:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.embed_tokens.weight", shape: (32000, 4096), dtype: float16
5%|▌ | 12/227 [00:59<09:01, 2.52s/it] 6%|▌ | 14/227 [01:00<09:31, 2.68s/it] [2024-06-06 22:27:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
6%|▌ | 14/227 [01:06<09:31, 2.68s/it] 7%|▋ | 15/227 [01:11<15:19, 4.34s/it] [2024-06-06 22:27:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
7%|▋ | 15/227 [01:13<15:19, 4.34s/it] 7%|▋ | 16/227 [01:15<15:20, 4.36s/it] [2024-06-06 22:27:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.moe.gate.weight", shape: (8, 4096), dtype: float16
7%|▋ | 16/227 [01:15<15:20, 4.36s/it] [2024-06-06 22:27:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.input_layernorm.weight", shape: (4096,), dtype: float16
7%|▋ | 16/227 [01:16<15:20, 4.36s/it] [2024-06-06 22:27:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.post_attention_layernorm.weight", shape: (4096,), dtype: float16
7%|▋ | 16/227 [01:16<15:20, 4.36s/it] [2024-06-06 22:27:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
7%|▋ | 16/227 [01:16<15:20, 4.36s/it] 9%|▉ | 20/227 [01:16<06:56, 2.01s/it] [2024-06-06 22:27:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
9%|▉ | 20/227 [01:16<06:56, 2.01s/it] 9%|▉ | 21/227 [01:16<05:51, 1.71s/it] [2024-06-06 22:27:12] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00002-of-00019.safetensors
9%|▉ | 21/227 [01:16<05:51, 1.71s/it] [2024-06-06 22:27:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
9%|▉ | 21/227 [01:37<05:51, 1.71s/it] 10%|▉ | 22/227 [01:42<21:45, 6.37s/it] [2024-06-06 22:27:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
10%|▉ | 22/227 [01:47<21:45, 6.37s/it] 10%|█ | 23/227 [01:50<22:49, 6.71s/it] [2024-06-06 22:27:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.moe.gate.weight", shape: (8, 4096), dtype: float16
10%|█ | 23/227 [01:50<22:49, 6.71s/it] 11%|█ | 24/227 [01:50<17:29, 5.17s/it] [2024-06-06 22:27:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
11%|█ | 24/227 [01:50<17:29, 5.17s/it] 11%|█ | 25/227 [01:51<13:16, 3.94s/it] [2024-06-06 22:27:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
11%|█ | 25/227 [01:51<13:16, 3.94s/it] 11%|█▏ | 26/227 [01:51<09:52, 2.95s/it] [2024-06-06 22:27:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.input_layernorm.weight", shape: (4096,), dtype: float16
11%|█▏ | 26/227 [01:51<09:52, 2.95s/it] [2024-06-06 22:27:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.post_attention_layernorm.weight", shape: (4096,), dtype: float16
11%|█▏ | 26/227 [01:51<09:52, 2.95s/it] [2024-06-06 22:27:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
11%|█▏ | 26/227 [02:01<09:52, 2.95s/it] 13%|█▎ | 29/227 [02:06<13:32, 4.10s/it] [2024-06-06 22:28:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
13%|█▎ | 29/227 [02:10<13:32, 4.10s/it] 13%|█▎ | 30/227 [02:13<15:09, 4.62s/it] [2024-06-06 22:28:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.moe.gate.weight", shape: (8, 4096), dtype: float16
13%|█▎ | 30/227 [02:13<15:09, 4.62s/it] [2024-06-06 22:28:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.input_layernorm.weight", shape: (4096,), dtype: float16
13%|█▎ | 30/227 [02:13<15:09, 4.62s/it] [2024-06-06 22:28:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.post_attention_layernorm.weight", shape: (4096,), dtype: float16
13%|█▎ | 30/227 [02:13<15:09, 4.62s/it] [2024-06-06 22:28:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
13%|█▎ | 30/227 [02:13<15:09, 4.62s/it] 15%|█▍ | 34/227 [02:13<07:02, 2.19s/it] [2024-06-06 22:28:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
15%|█▍ | 34/227 [02:13<07:02, 2.19s/it] 15%|█▌ | 35/227 [02:13<05:57, 1.86s/it] [2024-06-06 22:28:10] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00001-of-00019.safetensors
15%|█▌ | 35/227 [02:13<05:57, 1.86s/it] [2024-06-06 22:28:10] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00003-of-00019.safetensors
15%|█▌ | 35/227 [02:14<05:57, 1.86s/it] [2024-06-06 22:28:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
15%|█▌ | 35/227 [02:29<05:57, 1.86s/it] 16%|█▌ | 36/227 [02:34<17:13, 5.41s/it] [2024-06-06 22:28:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
16%|█▌ | 36/227 [02:40<17:13, 5.41s/it] 16%|█▋ | 37/227 [02:42<18:59, 6.00s/it] [2024-06-06 22:28:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.moe.gate.weight", shape: (8, 4096), dtype: float16
16%|█▋ | 37/227 [02:42<18:59, 6.00s/it] 17%|█▋ | 38/227 [02:42<14:36, 4.64s/it] [2024-06-06 22:28:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
17%|█▋ | 38/227 [02:43<14:36, 4.64s/it] 17%|█▋ | 39/227 [02:43<11:06, 3.54s/it] [2024-06-06 22:28:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
17%|█▋ | 39/227 [02:43<11:06, 3.54s/it] 18%|█▊ | 40/227 [02:43<08:17, 2.66s/it] [2024-06-06 22:28:39] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00002-of-00019.safetensors
18%|█▊ | 40/227 [02:43<08:17, 2.66s/it] [2024-06-06 22:28:40] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00003-of-00019.safetensors
18%|█▊ | 40/227 [02:43<08:17, 2.66s/it] [2024-06-06 22:28:40] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00006-of-00019.safetensors
18%|█▊ | 40/227 [02:44<08:17, 2.66s/it] [2024-06-06 22:28:44] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00007-of-00019.safetensors
18%|█▊ | 40/227 [02:48<08:17, 2.66s/it] [2024-06-06 22:28:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
18%|█▊ | 40/227 [03:01<08:17, 2.66s/it] 18%|█▊ | 41/227 [03:05<24:54, 8.03s/it] [2024-06-06 22:29:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
18%|█▊ | 41/227 [03:11<24:54, 8.03s/it] 19%|█▊ | 42/227 [03:13<24:48, 8.05s/it] [2024-06-06 22:29:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.moe.gate.weight", shape: (8, 4096), dtype: float16
19%|█▊ | 42/227 [03:13<24:48, 8.05s/it] [2024-06-06 22:29:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
19%|█▊ | 42/227 [03:14<24:48, 8.05s/it] 19%|█▉ | 44/227 [03:14<13:51, 4.54s/it] [2024-06-06 22:29:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
19%|█▉ | 44/227 [03:14<13:51, 4.54s/it] 20%|█▉ | 45/227 [03:14<10:34, 3.49s/it] [2024-06-06 22:29:10] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00007-of-00019.safetensors
20%|█▉ | 45/227 [03:14<10:34, 3.49s/it] [2024-06-06 22:29:11] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00005-of-00019.safetensors
20%|█▉ | 45/227 [03:14<10:34, 3.49s/it] [2024-06-06 22:29:16] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
20%|█▉ | 45/227 [03:20<10:34, 3.49s/it] 20%|██ | 46/227 [03:22<14:28, 4.80s/it] [2024-06-06 22:29:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
20%|██ | 46/227 [03:35<14:28, 4.80s/it] 21%|██ | 47/227 [03:40<24:25, 8.14s/it] [2024-06-06 22:29:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.input_layernorm.weight", shape: (4096,), dtype: float16
21%|██ | 47/227 [03:40<24:25, 8.14s/it] 21%|██ | 48/227 [03:40<17:42, 5.94s/it] [2024-06-06 22:29:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.post_attention_layernorm.weight", shape: (4096,), dtype: float16
21%|██ | 48/227 [03:40<17:42, 5.94s/it] [2024-06-06 22:29:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
21%|██ | 48/227 [03:48<17:42, 5.94s/it] 22%|██▏ | 50/227 [03:52<17:57, 6.09s/it] [2024-06-06 22:29:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
22%|██▏ | 50/227 [03:57<17:57, 6.09s/it] 22%|██▏ | 51/227 [03:59<18:27, 6.29s/it] [2024-06-06 22:29:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.moe.gate.weight", shape: (8, 4096), dtype: float16
22%|██▏ | 51/227 [03:59<18:27, 6.29s/it] [2024-06-06 22:29:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.input_layernorm.weight", shape: (4096,), dtype: float16
22%|██▏ | 51/227 [03:59<18:27, 6.29s/it] [2024-06-06 22:29:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.post_attention_layernorm.weight", shape: (4096,), dtype: float16
22%|██▏ | 51/227 [03:59<18:27, 6.29s/it] [2024-06-06 22:29:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
22%|██▏ | 51/227 [04:00<18:27, 6.29s/it] 24%|██▍ | 55/227 [04:00<07:45, 2.70s/it] [2024-06-06 22:29:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
24%|██▍ | 55/227 [04:00<07:45, 2.70s/it] 25%|██▍ | 56/227 [04:00<06:27, 2.27s/it] [2024-06-06 22:29:56] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00006-of-00019.safetensors
25%|██▍ | 56/227 [04:00<06:27, 2.27s/it] [2024-06-06 22:29:57] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00005-of-00019.safetensors
25%|██▍ | 56/227 [04:01<06:27, 2.27s/it] [2024-06-06 22:29:57] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00007-of-00019.safetensors
25%|██▍ | 56/227 [04:01<06:27, 2.27s/it] [2024-06-06 22:30:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.input_layernorm.weight", shape: (4096,), dtype: float16
25%|██▍ | 56/227 [04:05<06:27, 2.27s/it] 25%|██▌ | 57/227 [04:05<08:02, 2.84s/it] [2024-06-06 22:30:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.post_attention_layernorm.weight", shape: (4096,), dtype: float16
25%|██▌ | 57/227 [04:05<08:02, 2.84s/it] [2024-06-06 22:30:01] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00008-of-00019.safetensors
25%|██▌ | 57/227 [04:05<08:02, 2.84s/it] [2024-06-06 22:30:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
25%|██▌ | 57/227 [04:18<08:02, 2.84s/it] 26%|██▌ | 59/227 [04:22<13:51, 4.95s/it] [2024-06-06 22:30:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
26%|██▌ | 59/227 [04:27<13:51, 4.95s/it] 26%|██▋ | 60/227 [04:30<15:08, 5.44s/it] [2024-06-06 22:30:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.moe.gate.weight", shape: (8, 4096), dtype: float16
26%|██▋ | 60/227 [04:30<15:08, 5.44s/it] [2024-06-06 22:30:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
26%|██▋ | 60/227 [04:30<15:08, 5.44s/it] 27%|██▋ | 62/227 [04:30<09:36, 3.49s/it] [2024-06-06 22:30:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
27%|██▋ | 62/227 [04:30<09:36, 3.49s/it] 28%|██▊ | 63/227 [04:30<07:38, 2.80s/it] [2024-06-06 22:30:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.input_layernorm.weight", shape: (4096,), dtype: float16
28%|██▊ | 63/227 [04:30<07:38, 2.80s/it] [2024-06-06 22:30:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.post_attention_layernorm.weight", shape: (4096,), dtype: float16
28%|██▊ | 63/227 [04:30<07:38, 2.80s/it] [2024-06-06 22:30:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
28%|██▊ | 63/227 [04:41<07:38, 2.80s/it] 29%|██▉ | 66/227 [04:45<10:21, 3.86s/it] [2024-06-06 22:30:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
29%|██▉ | 66/227 [04:47<10:21, 3.86s/it] 30%|██▉ | 67/227 [04:50<10:37, 3.99s/it] [2024-06-06 22:30:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.moe.gate.weight", shape: (8, 4096), dtype: float16
30%|██▉ | 67/227 [04:50<10:37, 3.99s/it] 30%|██▉ | 68/227 [04:50<08:25, 3.18s/it] [2024-06-06 22:30:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.input_layernorm.weight", shape: (4096,), dtype: float16
30%|██▉ | 68/227 [04:50<08:25, 3.18s/it] [2024-06-06 22:30:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.post_attention_layernorm.weight", shape: (4096,), dtype: float16
30%|██▉ | 68/227 [04:50<08:25, 3.18s/it] [2024-06-06 22:30:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
30%|██▉ | 68/227 [04:50<08:25, 3.18s/it] 31%|███▏ | 71/227 [04:50<04:28, 1.72s/it] [2024-06-06 22:30:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
31%|███▏ | 71/227 [04:50<04:28, 1.72s/it] 32%|███▏ | 72/227 [04:51<03:42, 1.44s/it] [2024-06-06 22:30:47] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00007-of-00019.safetensors
32%|███▏ | 72/227 [04:51<03:42, 1.44s/it] [2024-06-06 22:30:48] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00009-of-00019.safetensors
32%|███▏ | 72/227 [04:51<03:42, 1.44s/it] [2024-06-06 22:30:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
32%|███▏ | 72/227 [05:01<03:42, 1.44s/it] 32%|███▏ | 73/227 [05:06<11:14, 4.38s/it] [2024-06-06 22:31:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
32%|███▏ | 73/227 [05:08<11:14, 4.38s/it] 33%|███▎ | 74/227 [05:11<11:14, 4.41s/it] [2024-06-06 22:31:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.moe.gate.weight", shape: (8, 4096), dtype: float16
33%|███▎ | 74/227 [05:11<11:14, 4.41s/it] [2024-06-06 22:31:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
33%|███▎ | 74/227 [05:11<11:14, 4.41s/it] 33%|███▎ | 76/227 [05:11<06:52, 2.73s/it] [2024-06-06 22:31:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
33%|███▎ | 76/227 [05:11<06:52, 2.73s/it] 34%|███▍ | 77/227 [05:11<05:24, 2.16s/it] [2024-06-06 22:31:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.input_layernorm.weight", shape: (4096,), dtype: float16
34%|███▍ | 77/227 [05:11<05:24, 2.16s/it] [2024-06-06 22:31:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.post_attention_layernorm.weight", shape: (4096,), dtype: float16
34%|███▍ | 77/227 [05:11<05:24, 2.16s/it] [2024-06-06 22:31:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
34%|███▍ | 77/227 [05:17<05:24, 2.16s/it] 35%|███▌ | 80/227 [05:22<07:03, 2.88s/it] [2024-06-06 22:31:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
35%|███▌ | 80/227 [05:24<07:03, 2.88s/it] 36%|███▌ | 81/227 [05:26<07:40, 3.16s/it] [2024-06-06 22:31:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.moe.gate.weight", shape: (8, 4096), dtype: float16
36%|███▌ | 81/227 [05:26<07:40, 3.16s/it] [2024-06-06 22:31:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.input_layernorm.weight", shape: (4096,), dtype: float16
36%|███▌ | 81/227 [05:26<07:40, 3.16s/it] [2024-06-06 22:31:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.post_attention_layernorm.weight", shape: (4096,), dtype: float16
36%|███▌ | 81/227 [05:26<07:40, 3.16s/it] [2024-06-06 22:31:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
36%|███▌ | 81/227 [05:27<07:40, 3.16s/it] 37%|███▋ | 85/227 [05:27<03:41, 1.56s/it] [2024-06-06 22:31:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
37%|███▋ | 85/227 [05:27<03:41, 1.56s/it] 38%|███▊ | 86/227 [05:27<03:09, 1.34s/it] [2024-06-06 22:31:23] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00008-of-00019.safetensors
38%|███▊ | 86/227 [05:27<03:09, 1.34s/it] [2024-06-06 22:31:24] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00010-of-00019.safetensors
38%|███▊ | 86/227 [05:27<03:09, 1.34s/it] [2024-06-06 22:31:34] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
38%|███▊ | 86/227 [05:38<03:09, 1.34s/it] 38%|███▊ | 87/227 [05:43<09:14, 3.96s/it] [2024-06-06 22:31:43] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
38%|███▊ | 87/227 [05:46<09:14, 3.96s/it] 39%|███▉ | 88/227 [05:49<10:20, 4.47s/it] [2024-06-06 22:31:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.moe.gate.weight", shape: (8, 4096), dtype: float16
39%|███▉ | 88/227 [05:49<10:20, 4.47s/it] [2024-06-06 22:31:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
39%|███▉ | 88/227 [05:49<10:20, 4.47s/it] 40%|███▉ | 90/227 [05:49<06:32, 2.87s/it] [2024-06-06 22:31:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
40%|███▉ | 90/227 [05:49<06:32, 2.87s/it] 40%|████ | 91/227 [05:49<05:12, 2.30s/it] [2024-06-06 22:31:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.input_layernorm.weight", shape: (4096,), dtype: float16
40%|████ | 91/227 [05:49<05:12, 2.30s/it] [2024-06-06 22:31:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.post_attention_layernorm.weight", shape: (4096,), dtype: float16
40%|████ | 91/227 [05:49<05:12, 2.30s/it] [2024-06-06 22:31:46] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00009-of-00019.safetensors
40%|████ | 91/227 [05:49<05:12, 2.30s/it] [2024-06-06 22:31:46] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00011-of-00019.safetensors
40%|████ | 91/227 [05:50<05:12, 2.30s/it] [2024-06-06 22:32:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
40%|████ | 91/227 [06:04<05:12, 2.30s/it] 41%|████▏ | 94/227 [06:09<09:24, 4.25s/it] [2024-06-06 22:32:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
41%|████▏ | 94/227 [06:14<09:24, 4.25s/it] 42%|████▏ | 95/227 [06:17<10:47, 4.91s/it] [2024-06-06 22:32:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.moe.gate.weight", shape: (8, 4096), dtype: float16
42%|████▏ | 95/227 [06:17<10:47, 4.91s/it] [2024-06-06 22:32:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
42%|████▏ | 95/227 [06:17<10:47, 4.91s/it] 43%|████▎ | 97/227 [06:17<07:05, 3.27s/it] [2024-06-06 22:32:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
43%|████▎ | 97/227 [06:17<07:05, 3.27s/it] 43%|████▎ | 98/227 [06:17<05:42, 2.66s/it] [2024-06-06 22:32:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.input_layernorm.weight", shape: (4096,), dtype: float16
43%|████▎ | 98/227 [06:17<05:42, 2.66s/it] [2024-06-06 22:32:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.post_attention_layernorm.weight", shape: (4096,), dtype: float16
43%|████▎ | 98/227 [06:17<05:42, 2.66s/it] [2024-06-06 22:32:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
43%|████▎ | 98/227 [06:26<05:42, 2.66s/it] 44%|████▍ | 101/227 [06:31<07:25, 3.54s/it] [2024-06-06 22:32:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
44%|████▍ | 101/227 [06:33<07:25, 3.54s/it] 45%|████▍ | 102/227 [06:36<07:51, 3.77s/it] [2024-06-06 22:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.moe.gate.weight", shape: (8, 4096), dtype: float16
45%|████▍ | 102/227 [06:36<07:51, 3.77s/it] [2024-06-06 22:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.input_layernorm.weight", shape: (4096,), dtype: float16
45%|████▍ | 102/227 [06:36<07:51, 3.77s/it] [2024-06-06 22:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.post_attention_layernorm.weight", shape: (4096,), dtype: float16
45%|████▍ | 102/227 [06:36<07:51, 3.77s/it] [2024-06-06 22:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
45%|████▍ | 102/227 [06:36<07:51, 3.77s/it] 47%|████▋ | 106/227 [06:36<03:50, 1.90s/it] [2024-06-06 22:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
47%|████▋ | 106/227 [06:36<03:50, 1.90s/it] 47%|████▋ | 107/227 [06:36<03:16, 1.64s/it] [2024-06-06 22:32:32] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00010-of-00019.safetensors
47%|████▋ | 107/227 [06:36<03:16, 1.64s/it] [2024-06-06 22:32:33] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00012-of-00019.safetensors
47%|████▋ | 107/227 [06:37<03:16, 1.64s/it] [2024-06-06 22:32:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
47%|████▋ | 107/227 [06:50<03:16, 1.64s/it] 48%|████▊ | 108/227 [06:55<09:20, 4.71s/it] [2024-06-06 22:32:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
48%|████▊ | 108/227 [07:00<09:20, 4.71s/it] 48%|████▊ | 109/227 [07:02<10:20, 5.26s/it] [2024-06-06 22:32:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.moe.gate.weight", shape: (8, 4096), dtype: float16
48%|████▊ | 109/227 [07:02<10:20, 5.26s/it] [2024-06-06 22:32:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
48%|████▊ | 109/227 [07:02<10:20, 5.26s/it] 49%|████▉ | 111/227 [07:03<06:32, 3.38s/it] [2024-06-06 22:32:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
49%|████▉ | 111/227 [07:03<06:32, 3.38s/it] 49%|████▉ | 112/227 [07:03<05:12, 2.71s/it] [2024-06-06 22:32:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.input_layernorm.weight", shape: (4096,), dtype: float16
49%|████▉ | 112/227 [07:03<05:12, 2.71s/it] [2024-06-06 22:32:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.post_attention_layernorm.weight", shape: (4096,), dtype: float16
49%|████▉ | 112/227 [07:03<05:12, 2.71s/it] [2024-06-06 22:33:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
49%|████▉ | 112/227 [07:20<05:12, 2.71s/it] 51%|█████ | 115/227 [07:25<09:20, 5.01s/it] [2024-06-06 22:33:28] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
51%|█████ | 115/227 [07:32<09:20, 5.01s/it] 51%|█████ | 116/227 [07:34<10:33, 5.71s/it] [2024-06-06 22:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.moe.gate.weight", shape: (8, 4096), dtype: float16
51%|█████ | 116/227 [07:34<10:33, 5.71s/it] 52%|█████▏ | 117/227 [07:34<08:20, 4.55s/it] [2024-06-06 22:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.input_layernorm.weight", shape: (4096,), dtype: float16
52%|█████▏ | 117/227 [07:34<08:20, 4.55s/it] [2024-06-06 22:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.post_attention_layernorm.weight", shape: (4096,), dtype: float16
52%|█████▏ | 117/227 [07:34<08:20, 4.55s/it] [2024-06-06 22:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
52%|█████▏ | 117/227 [07:35<08:20, 4.55s/it] 53%|█████▎ | 120/227 [07:35<04:21, 2.45s/it] [2024-06-06 22:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
53%|█████▎ | 120/227 [07:35<04:21, 2.45s/it] 53%|█████▎ | 121/227 [07:35<03:34, 2.03s/it] [2024-06-06 22:33:31] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00011-of-00019.safetensors
53%|█████▎ | 121/227 [07:35<03:34, 2.03s/it] [2024-06-06 22:33:32] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00013-of-00019.safetensors
53%|█████▎ | 121/227 [07:36<03:34, 2.03s/it] [2024-06-06 22:33:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
53%|█████▎ | 121/227 [07:56<03:34, 2.03s/it] 54%|█████▎ | 122/227 [08:02<12:30, 7.15s/it] [2024-06-06 22:34:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
54%|█████▎ | 122/227 [08:09<12:30, 7.15s/it] 54%|█████▍ | 123/227 [08:12<13:32, 7.81s/it] [2024-06-06 22:34:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.moe.gate.weight", shape: (8, 4096), dtype: float16
54%|█████▍ | 123/227 [08:12<13:32, 7.81s/it] 55%|█████▍ | 124/227 [08:12<10:09, 5.92s/it] [2024-06-06 22:34:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
55%|█████▍ | 124/227 [08:12<10:09, 5.92s/it] 55%|█████▌ | 125/227 [08:12<07:32, 4.44s/it] [2024-06-06 22:34:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
55%|█████▌ | 125/227 [08:12<07:32, 4.44s/it] 56%|█████▌ | 126/227 [08:12<05:31, 3.28s/it] [2024-06-06 22:34:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.input_layernorm.weight", shape: (4096,), dtype: float16
56%|█████▌ | 126/227 [08:12<05:31, 3.28s/it] [2024-06-06 22:34:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.post_attention_layernorm.weight", shape: (4096,), dtype: float16
56%|█████▌ | 126/227 [08:12<05:31, 3.28s/it] [2024-06-06 22:34:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
56%|█████▌ | 126/227 [08:24<05:31, 3.28s/it] 57%|█████▋ | 129/227 [08:28<07:14, 4.43s/it] [2024-06-06 22:34:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
57%|█████▋ | 129/227 [08:30<07:14, 4.43s/it] 57%|█████▋ | 130/227 [08:33<07:13, 4.46s/it] [2024-06-06 22:34:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.moe.gate.weight", shape: (8, 4096), dtype: float16
57%|█████▋ | 130/227 [08:33<07:13, 4.46s/it] [2024-06-06 22:34:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.input_layernorm.weight", shape: (4096,), dtype: float16
57%|█████▋ | 130/227 [08:33<07:13, 4.46s/it] [2024-06-06 22:34:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.post_attention_layernorm.weight", shape: (4096,), dtype: float16
57%|█████▋ | 130/227 [08:33<07:13, 4.46s/it] [2024-06-06 22:34:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
57%|█████▋ | 130/227 [08:33<07:13, 4.46s/it] 59%|█████▉ | 134/227 [08:33<03:15, 2.11s/it] [2024-06-06 22:34:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
59%|█████▉ | 134/227 [08:33<03:15, 2.11s/it] 59%|█████▉ | 135/227 [08:33<02:44, 1.79s/it] [2024-06-06 22:34:30] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00012-of-00019.safetensors
59%|█████▉ | 135/227 [08:33<02:44, 1.79s/it] [2024-06-06 22:34:30] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00014-of-00019.safetensors
59%|█████▉ | 135/227 [08:34<02:44, 1.79s/it] [2024-06-06 22:34:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
59%|█████▉ | 135/227 [08:44<02:44, 1.79s/it] 60%|█████▉ | 136/227 [08:49<06:38, 4.38s/it] [2024-06-06 22:34:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
60%|█████▉ | 136/227 [08:51<06:38, 4.38s/it] 60%|██████ | 137/227 [08:53<06:33, 4.37s/it] [2024-06-06 22:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.moe.gate.weight", shape: (8, 4096), dtype: float16
60%|██████ | 137/227 [08:53<06:33, 4.37s/it] [2024-06-06 22:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
60%|██████ | 137/227 [08:54<06:33, 4.37s/it] 61%|██████ | 139/227 [08:54<04:05, 2.78s/it] [2024-06-06 22:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
61%|██████ | 139/227 [08:54<04:05, 2.78s/it] 62%|██████▏ | 140/227 [08:54<03:13, 2.23s/it] [2024-06-06 22:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.input_layernorm.weight", shape: (4096,), dtype: float16
62%|██████▏ | 140/227 [08:54<03:13, 2.23s/it] [2024-06-06 22:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.post_attention_layernorm.weight", shape: (4096,), dtype: float16
62%|██████▏ | 140/227 [08:54<03:13, 2.23s/it] [2024-06-06 22:34:50] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00013-of-00019.safetensors
62%|██████▏ | 140/227 [08:54<03:13, 2.23s/it] [2024-06-06 22:34:51] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00015-of-00019.safetensors
62%|██████▏ | 140/227 [08:54<03:13, 2.23s/it] [2024-06-06 22:35:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
62%|██████▏ | 140/227 [09:05<03:13, 2.23s/it] 63%|██████▎ | 143/227 [09:10<05:11, 3.70s/it] [2024-06-06 22:35:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
63%|██████▎ | 143/227 [09:12<05:11, 3.70s/it] 63%|██████▎ | 144/227 [09:14<05:20, 3.87s/it] [2024-06-06 22:35:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.moe.gate.weight", shape: (8, 4096), dtype: float16
63%|██████▎ | 144/227 [09:14<05:20, 3.87s/it] 64%|██████▍ | 145/227 [09:15<04:12, 3.08s/it] [2024-06-06 22:35:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
64%|██████▍ | 145/227 [09:15<04:12, 3.08s/it] 64%|██████▍ | 146/227 [09:15<03:16, 2.43s/it] [2024-06-06 22:35:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
64%|██████▍ | 146/227 [09:15<03:16, 2.43s/it] 65%|██████▍ | 147/227 [09:15<02:29, 1.87s/it] [2024-06-06 22:35:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.input_layernorm.weight", shape: (4096,), dtype: float16
65%|██████▍ | 147/227 [09:15<02:29, 1.87s/it] [2024-06-06 22:35:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.post_attention_layernorm.weight", shape: (4096,), dtype: float16
65%|██████▍ | 147/227 [09:15<02:29, 1.87s/it] [2024-06-06 22:35:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
65%|██████▍ | 147/227 [09:21<02:29, 1.87s/it] 66%|██████▌ | 150/227 [09:26<03:37, 2.83s/it] [2024-06-06 22:35:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
66%|██████▌ | 150/227 [09:28<03:37, 2.83s/it] 67%|██████▋ | 151/227 [09:31<03:58, 3.14s/it] [2024-06-06 22:35:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.moe.gate.weight", shape: (8, 4096), dtype: float16
67%|██████▋ | 151/227 [09:31<03:58, 3.14s/it] [2024-06-06 22:35:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.input_layernorm.weight", shape: (4096,), dtype: float16
67%|██████▋ | 151/227 [09:31<03:58, 3.14s/it] [2024-06-06 22:35:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.post_attention_layernorm.weight", shape: (4096,), dtype: float16
67%|██████▋ | 151/227 [09:31<03:58, 3.14s/it] [2024-06-06 22:35:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
67%|██████▋ | 151/227 [09:31<03:58, 3.14s/it] 68%|██████▊ | 155/227 [09:31<01:49, 1.52s/it] [2024-06-06 22:35:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
68%|██████▊ | 155/227 [09:31<01:49, 1.52s/it] 69%|██████▊ | 156/227 [09:31<01:32, 1.30s/it] [2024-06-06 22:35:27] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00014-of-00019.safetensors
69%|██████▊ | 156/227 [09:31<01:32, 1.30s/it] [2024-06-06 22:35:28] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00016-of-00019.safetensors
69%|██████▊ | 156/227 [09:32<01:32, 1.30s/it] [2024-06-06 22:35:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
69%|██████▊ | 156/227 [09:43<01:32, 1.30s/it] 69%|██████▉ | 157/227 [09:48<04:51, 4.17s/it] [2024-06-06 22:35:49] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
69%|██████▉ | 157/227 [09:53<04:51, 4.17s/it] 70%|██████▉ | 158/227 [09:55<05:37, 4.90s/it] [2024-06-06 22:35:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.moe.gate.weight", shape: (8, 4096), dtype: float16
70%|██████▉ | 158/227 [09:56<05:37, 4.90s/it] 70%|███████ | 159/227 [09:56<04:18, 3.80s/it] [2024-06-06 22:35:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
70%|███████ | 159/227 [09:56<04:18, 3.80s/it] 70%|███████ | 160/227 [09:56<03:15, 2.92s/it] [2024-06-06 22:35:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
70%|███████ | 160/227 [09:56<03:15, 2.92s/it] 71%|███████ | 161/227 [09:56<02:25, 2.20s/it] [2024-06-06 22:35:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.input_layernorm.weight", shape: (4096,), dtype: float16
71%|███████ | 161/227 [09:56<02:25, 2.20s/it] [2024-06-06 22:35:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.post_attention_layernorm.weight", shape: (4096,), dtype: float16
71%|███████ | 161/227 [09:56<02:25, 2.20s/it] [2024-06-06 22:36:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
71%|███████ | 161/227 [10:11<02:25, 2.20s/it] 72%|███████▏ | 164/227 [10:16<04:45, 4.53s/it] [2024-06-06 22:36:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
72%|███████▏ | 164/227 [10:22<04:45, 4.53s/it] 73%|███████▎ | 165/227 [10:24<05:30, 5.34s/it] [2024-06-06 22:36:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.moe.gate.weight", shape: (8, 4096), dtype: float16
73%|███████▎ | 165/227 [10:24<05:30, 5.34s/it] 73%|███████▎ | 166/227 [10:25<04:14, 4.17s/it] [2024-06-06 22:36:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.input_layernorm.weight", shape: (4096,), dtype: float16
73%|███████▎ | 166/227 [10:25<04:14, 4.17s/it] [2024-06-06 22:36:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.post_attention_layernorm.weight", shape: (4096,), dtype: float16
73%|███████▎ | 166/227 [10:25<04:14, 4.17s/it] [2024-06-06 22:36:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
73%|███████▎ | 166/227 [10:25<04:14, 4.17s/it] 74%|███████▍ | 169/227 [10:25<02:06, 2.17s/it] [2024-06-06 22:36:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
74%|███████▍ | 169/227 [10:25<02:06, 2.17s/it] 75%|███████▍ | 170/227 [10:25<01:42, 1.79s/it] [2024-06-06 22:36:21] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00015-of-00019.safetensors
75%|███████▍ | 170/227 [10:25<01:42, 1.79s/it] [2024-06-06 22:36:22] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00017-of-00019.safetensors
75%|███████▍ | 170/227 [10:26<01:42, 1.79s/it] [2024-06-06 22:36:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
75%|███████▍ | 170/227 [10:45<01:42, 1.79s/it] 75%|███████▌ | 171/227 [10:51<06:23, 6.85s/it] [2024-06-06 22:36:49] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
75%|███████▌ | 171/227 [10:53<06:23, 6.85s/it] 76%|███████▌ | 172/227 [10:55<05:50, 6.38s/it] [2024-06-06 22:36:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.moe.gate.weight", shape: (8, 4096), dtype: float16
76%|███████▌ | 172/227 [10:55<05:50, 6.38s/it] 76%|███████▌ | 173/227 [10:55<04:19, 4.81s/it] [2024-06-06 22:36:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
76%|███████▌ | 173/227 [10:56<04:19, 4.81s/it] 77%|███████▋ | 174/227 [10:56<03:11, 3.61s/it] [2024-06-06 22:36:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
77%|███████▋ | 174/227 [10:56<03:11, 3.61s/it] 77%|███████▋ | 175/227 [10:56<02:18, 2.67s/it] [2024-06-06 22:36:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.input_layernorm.weight", shape: (4096,), dtype: float16
77%|███████▋ | 175/227 [10:56<02:18, 2.67s/it] [2024-06-06 22:36:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.post_attention_layernorm.weight", shape: (4096,), dtype: float16
77%|███████▋ | 175/227 [10:56<02:18, 2.67s/it] [2024-06-06 22:36:52] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00016-of-00019.safetensors
77%|███████▋ | 175/227 [10:56<02:18, 2.67s/it] [2024-06-06 22:36:53] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00018-of-00019.safetensors
77%|███████▋ | 175/227 [10:56<02:18, 2.67s/it] [2024-06-06 22:37:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
77%|███████▋ | 175/227 [11:09<02:18, 2.67s/it] 78%|███████▊ | 178/227 [11:14<03:40, 4.50s/it] [2024-06-06 22:37:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
78%|███████▊ | 178/227 [11:16<03:40, 4.50s/it] 79%|███████▉ | 179/227 [11:19<03:36, 4.51s/it] [2024-06-06 22:37:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.moe.gate.weight", shape: (8, 4096), dtype: float16
79%|███████▉ | 179/227 [11:19<03:36, 4.51s/it] [2024-06-06 22:37:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
79%|███████▉ | 179/227 [11:19<03:36, 4.51s/it] 80%|███████▉ | 181/227 [11:19<02:13, 2.91s/it] [2024-06-06 22:37:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
80%|███████▉ | 181/227 [11:19<02:13, 2.91s/it] 80%|████████ | 182/227 [11:19<01:45, 2.34s/it] [2024-06-06 22:37:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.input_layernorm.weight", shape: (4096,), dtype: float16
80%|████████ | 182/227 [11:19<01:45, 2.34s/it] [2024-06-06 22:37:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.post_attention_layernorm.weight", shape: (4096,), dtype: float16
80%|████████ | 182/227 [11:19<01:45, 2.34s/it] [2024-06-06 22:37:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
80%|████████ | 182/227 [11:24<01:45, 2.34s/it] 81%|████████▏ | 185/227 [11:28<01:52, 2.67s/it] [2024-06-06 22:37:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
81%|████████▏ | 185/227 [11:30<01:52, 2.67s/it] 82%|████████▏ | 186/227 [11:33<02:02, 2.98s/it] [2024-06-06 22:37:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.moe.gate.weight", shape: (8, 4096), dtype: float16
82%|████████▏ | 186/227 [11:33<02:02, 2.98s/it] [2024-06-06 22:37:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.input_layernorm.weight", shape: (4096,), dtype: float16
82%|████████▏ | 186/227 [11:33<02:02, 2.98s/it] [2024-06-06 22:37:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.post_attention_layernorm.weight", shape: (4096,), dtype: float16
82%|████████▏ | 186/227 [11:33<02:02, 2.98s/it] [2024-06-06 22:37:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
82%|████████▏ | 186/227 [11:33<02:02, 2.98s/it] 84%|████████▎ | 190/227 [11:33<00:55, 1.50s/it] [2024-06-06 22:37:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
84%|████████▎ | 190/227 [11:33<00:55, 1.50s/it] 84%|████████▍ | 191/227 [11:33<00:46, 1.29s/it] [2024-06-06 22:37:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.moe.gate.weight", shape: (8, 4096), dtype: float16
84%|████████▍ | 191/227 [11:33<00:46, 1.29s/it] [2024-06-06 22:37:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
84%|████████▍ | 191/227 [11:33<00:46, 1.29s/it] 85%|████████▌ | 193/227 [11:33<00:31, 1.07it/s] [2024-06-06 22:37:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
85%|████████▌ | 193/227 [11:33<00:31, 1.07it/s] 85%|████████▌ | 194/227 [11:33<00:26, 1.26it/s] [2024-06-06 22:37:30] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00017-of-00019.safetensors
85%|████████▌ | 194/227 [11:33<00:26, 1.26it/s] [2024-06-06 22:37:30] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00018-of-00019.safetensors
85%|████████▌ | 194/227 [11:34<00:26, 1.26it/s] [2024-06-06 22:37:31] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00003-of-00019.safetensors
85%|████████▌ | 194/227 [11:35<00:26, 1.26it/s] [2024-06-06 22:37:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.input_layernorm.weight", shape: (4096,), dtype: float16
85%|████████▌ | 194/227 [11:39<00:26, 1.26it/s] 86%|████████▌ | 195/227 [11:39<00:54, 1.71s/it] [2024-06-06 22:37:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.post_attention_layernorm.weight", shape: (4096,), dtype: float16
86%|████████▌ | 195/227 [11:39<00:54, 1.71s/it] [2024-06-06 22:37:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
86%|████████▌ | 195/227 [11:44<00:54, 1.71s/it] 87%|████████▋ | 197/227 [11:49<01:31, 3.04s/it] [2024-06-06 22:37:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
87%|████████▋ | 197/227 [11:51<01:31, 3.04s/it] 87%|████████▋ | 198/227 [11:54<01:37, 3.37s/it] [2024-06-06 22:37:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.moe.gate.weight", shape: (8, 4096), dtype: float16
87%|████████▋ | 198/227 [11:54<01:37, 3.37s/it] 88%|████████▊ | 199/227 [11:54<01:13, 2.62s/it] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.input_layernorm.weight", shape: (4096,), dtype: float16
88%|████████▊ | 199/227 [11:54<01:13, 2.62s/it] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.post_attention_layernorm.weight", shape: (4096,), dtype: float16
88%|████████▊ | 199/227 [11:54<01:13, 2.62s/it] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
88%|████████▊ | 199/227 [11:54<01:13, 2.62s/it] 89%|████████▉ | 202/227 [11:54<00:34, 1.36s/it] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
89%|████████▉ | 202/227 [11:54<00:34, 1.36s/it] 89%|████████▉ | 203/227 [11:55<00:27, 1.13s/it] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.moe.gate.weight", shape: (8, 4096), dtype: float16
89%|████████▉ | 203/227 [11:55<00:27, 1.13s/it] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
89%|████████▉ | 203/227 [11:55<00:27, 1.13s/it] 90%|█████████ | 205/227 [11:55<00:17, 1.29it/s] [2024-06-06 22:37:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
90%|█████████ | 205/227 [11:55<00:17, 1.29it/s] 91%|█████████ | 206/227 [11:55<00:13, 1.54it/s] [2024-06-06 22:37:51] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00003-of-00019.safetensors
91%|█████████ | 206/227 [11:55<00:13, 1.54it/s] [2024-06-06 22:37:52] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00004-of-00019.safetensors
91%|█████████ | 206/227 [11:56<00:13, 1.54it/s] [2024-06-06 22:38:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
91%|█████████ | 206/227 [12:04<00:13, 1.54it/s] 91%|█████████ | 207/227 [12:09<01:13, 3.65s/it] [2024-06-06 22:38:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
91%|█████████ | 207/227 [12:11<01:13, 3.65s/it] 92%|█████████▏| 208/227 [12:14<01:13, 3.88s/it] [2024-06-06 22:38:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.input_layernorm.weight", shape: (4096,), dtype: float16
92%|█████████▏| 208/227 [12:14<01:13, 3.88s/it] [2024-06-06 22:38:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.post_attention_layernorm.weight", shape: (4096,), dtype: float16
92%|█████████▏| 208/227 [12:14<01:13, 3.88s/it] [2024-06-06 22:38:10] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Mixtral-8x7B-Instruct-v0.1/model-00005-of-00019.safetensors
92%|█████████▏| 208/227 [12:14<01:13, 3.88s/it] [2024-06-06 22:38:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
92%|█████████▏| 208/227 [12:27<01:13, 3.88s/it] 93%|█████████▎| 211/227 [12:32<01:19, 4.98s/it] [2024-06-06 22:38:34] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
93%|█████████▎| 211/227 [12:37<01:19, 4.98s/it] 93%|█████████▎| 212/227 [12:40<01:24, 5.65s/it] [2024-06-06 22:38:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.moe.gate.weight", shape: (8, 4096), dtype: float16
93%|█████████▎| 212/227 [12:40<01:24, 5.65s/it] [2024-06-06 22:38:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
93%|█████████▎| 212/227 [12:40<01:24, 5.65s/it] 94%|█████████▍| 214/227 [12:41<00:48, 3.69s/it] [2024-06-06 22:38:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
94%|█████████▍| 214/227 [12:41<00:48, 3.69s/it] 95%|█████████▍| 215/227 [12:41<00:35, 2.97s/it] [2024-06-06 22:38:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.input_layernorm.weight", shape: (4096,), dtype: float16
95%|█████████▍| 215/227 [12:41<00:35, 2.97s/it] [2024-06-06 22:38:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.6.post_attention_layernorm.weight", shape: (4096,), dtype: float16
95%|█████████▍| 215/227 [12:41<00:35, 2.97s/it] [2024-06-06 22:38:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.moe.e1_e3.weight", shape: (8, 28672, 4096), dtype: float16
95%|█████████▍| 215/227 [12:54<00:35, 2.97s/it] 96%|█████████▌| 218/227 [12:59<00:40, 4.47s/it] [2024-06-06 22:39:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.moe.e2.weight", shape: (8, 4096, 14336), dtype: float16
96%|█████████▌| 218/227 [13:06<00:40, 4.47s/it] 96%|█████████▋| 219/227 [13:08<00:42, 5.32s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.moe.gate.weight", shape: (8, 4096), dtype: float16
96%|█████████▋| 219/227 [13:08<00:42, 5.32s/it] 97%|█████████▋| 220/227 [13:08<00:29, 4.25s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.input_layernorm.weight", shape: (4096,), dtype: float16
97%|█████████▋| 220/227 [13:08<00:29, 4.25s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.post_attention_layernorm.weight", shape: (4096,), dtype: float16
97%|█████████▋| 220/227 [13:08<00:29, 4.25s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
97%|█████████▋| 220/227 [13:09<00:29, 4.25s/it] 98%|█████████▊| 223/227 [13:09<00:09, 2.29s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.7.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
98%|█████████▊| 223/227 [13:09<00:09, 2.29s/it] 99%|█████████▊| 224/227 [13:09<00:05, 1.90s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.moe.gate.weight", shape: (8, 4096), dtype: float16
99%|█████████▊| 224/227 [13:09<00:05, 1.90s/it] [2024-06-06 22:39:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.self_attn.qkv_proj.weight", shape: (6144, 4096), dtype: float16
99%|█████████▊| 224/227 [13:09<00:05, 1.90s/it] 100%|█████████▉| 226/227 [13:09<00:01, 1.29s/it] [2024-06-06 22:39:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.8.self_attn.o_proj.weight", shape: (4096, 4096), dtype: float16
100%|█████████▉| 226/227 [13:09<00:01, 1.29s/it] 100%|██████████| 227/227 [13:09<00:00, 1.07s/it] 100%|██████████| 227/227 [13:09<00:00, 3.48s/it]
[2024-06-06 22:39:06] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00004-of-00019.safetensors
[2024-06-06 22:39:06] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Mixtral-8x7B-Instruct-v0.1/model-00005-of-00019.safetensors
[2024-06-06 22:39:07] INFO stats.py:77: Time usage: HF loading: 108.338 sec; Pre-quantization mapping: 421.469 sec; Quantization: 0.000 sec
[2024-06-06 22:39:07] INFO stats.py:91: RAM usage: Peak RAM: 18.563 GB. Total bytes loaded from disk: 210.951 GB
[2024-06-06 22:39:07] INFO convert_weight.py:155: Parameter size after quantization: 86.991 GB
[2024-06-06 22:39:07] INFO convert_weight.py:160: Total parameters: 56,626,802,688
[2024-06-06 22:39:07] INFO convert_weight.py:161: Bits per parameter: 13.196
[2024-06-06 22:39:07] INFO convert_weight.py:166: Saved to directory: /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC
All finished, 131 total shards committed, record saved to /models/mlc-delivery/hf/mlc-ai/Mixtral-8x7B-Instruct-v0.1-q0f16-MLC/ndarray-cache.json