/Users/cfruan/miniconda3/envs/mlc-chat-venv/bin/python -m mlc_llm gen_config /Users/Shared/models/Qwen2-Math-72B-Instruct --quantization q0f16 --conv-template chatml --output local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC
[2024-08-08 16:29:55] INFO auto_config.py:116: Found model configuration: /Users/Shared/models/Qwen2-Math-72B-Instruct/config.json
[2024-08-08 16:29:55] INFO auto_config.py:154: Found model type: qwen2. Use `--model-type` to override.
[2024-08-08 16:29:55] INFO qwen2_model.py:50: context_window_size not found in config.json. Falling back to max_position_embeddings (4096)
[2024-08-08 16:29:55] INFO qwen2_model.py:67: prefill_chunk_size defaults to 2048
[2024-08-08 16:29:55] INFO config.py:107: Overriding max_batch_size from 1 to 80
[2024-08-08 16:29:55] INFO gen_config.py:147: [generation_config.json] Setting bos_token_id: 151643
[2024-08-08 16:29:55] INFO gen_config.py:147: [generation_config.json] Setting pad_token_id: 151643
[2024-08-08 16:29:55] INFO gen_config.py:147: [generation_config.json] Setting eos_token_id: [151645, 151643]
[2024-08-08 16:29:55] INFO gen_config.py:147: [generation_config.json] Setting repetition_penalty: 1.05
[2024-08-08 16:29:55] INFO gen_config.py:147: [generation_config.json] Setting temperature: 0.7
[2024-08-08 16:29:55] INFO gen_config.py:147: [generation_config.json] Setting top_p: 0.8
[2024-08-08 16:29:55] INFO gen_config.py:161: Not found tokenizer config: /Users/Shared/models/Qwen2-Math-72B-Instruct/tokenizer.model
[2024-08-08 16:29:55] INFO gen_config.py:159: Found tokenizer config: /Users/Shared/models/Qwen2-Math-72B-Instruct/tokenizer.json. Copying to local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC/tokenizer.json
[2024-08-08 16:29:55] INFO gen_config.py:159: Found tokenizer config: /Users/Shared/models/Qwen2-Math-72B-Instruct/vocab.json. Copying to local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC/vocab.json
[2024-08-08 16:29:55] INFO gen_config.py:159: Found tokenizer config: /Users/Shared/models/Qwen2-Math-72B-Instruct/merges.txt. Copying to local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC/merges.txt
[2024-08-08 16:29:55] INFO gen_config.py:161: Not found tokenizer config: /Users/Shared/models/Qwen2-Math-72B-Instruct/added_tokens.json
[2024-08-08 16:29:55] INFO gen_config.py:159: Found tokenizer config: /Users/Shared/models/Qwen2-Math-72B-Instruct/tokenizer_config.json. Copying to local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC/tokenizer_config.json
[2024-08-08 16:29:55] INFO gen_config.py:220: Detected tokenizer info: {'token_postproc_method': 'byte_level', 'prepend_space_in_encode': False, 'strip_space_in_decode': False}
[2024-08-08 16:29:55] INFO gen_config.py:32: [System default] Setting presence_penalty: 0.0
[2024-08-08 16:29:55] INFO gen_config.py:32: [System default] Setting frequency_penalty: 0.0
[2024-08-08 16:29:55] INFO gen_config.py:248: Dumping configuration file to: local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC/mlc-chat-config.json
/Users/cfruan/miniconda3/envs/mlc-chat-venv/bin/python -m mlc_llm convert_weight /Users/Shared/models/Qwen2-Math-72B-Instruct --quantization q0f16 --output local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC
[2024-08-08 16:29:56] INFO auto_config.py:116: Found model configuration: /Users/Shared/models/Qwen2-Math-72B-Instruct/config.json
[2024-08-08 16:29:56] INFO auto_device.py:88: Not found device: cuda:0
[2024-08-08 16:29:57] INFO auto_device.py:88: Not found device: rocm:0
[2024-08-08 16:29:58] INFO auto_device.py:79: Found device: metal:0
[2024-08-08 16:29:59] INFO auto_device.py:88: Not found device: vulkan:0
[2024-08-08 16:29:59] INFO auto_device.py:88: Not found device: opencl:0
[2024-08-08 16:29:59] INFO auto_device.py:35: Using device: metal:0
[2024-08-08 16:29:59] INFO auto_weight.py:71: Finding weights in: /Users/Shared/models/Qwen2-Math-72B-Instruct
[2024-08-08 16:29:59] INFO auto_weight.py:137: Not found Huggingface PyTorch
[2024-08-08 16:29:59] INFO auto_weight.py:144: Found source weight format: huggingface-safetensor. Source configuration: /Users/Shared/models/Qwen2-Math-72B-Instruct/model.safetensors.index.json
[2024-08-08 16:29:59] INFO auto_weight.py:107: Using source weight configuration: /Users/Shared/models/Qwen2-Math-72B-Instruct/model.safetensors.index.json. Use `--source` to override.
[2024-08-08 16:29:59] INFO auto_weight.py:111: Using source weight format: huggingface-safetensor. Use `--source-format` to override.
[2024-08-08 16:29:59] INFO auto_config.py:154: Found model type: qwen2. Use `--model-type` to override.
[2024-08-08 16:29:59] INFO qwen2_model.py:50: context_window_size not found in config.json. Falling back to max_position_embeddings (4096)
[2024-08-08 16:29:59] INFO qwen2_model.py:67: prefill_chunk_size defaults to 2048
Weight conversion with arguments:
  --config          /Users/Shared/models/Qwen2-Math-72B-Instruct/config.json
  --quantization    NoQuantize(name='q0f16', kind='no-quant', model_dtype='float16')
  --model-type      qwen2
  --device          metal:0
  --source          /Users/Shared/models/Qwen2-Math-72B-Instruct/model.safetensors.index.json
  --source-format   huggingface-safetensor
  --output          local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC
Start storing to cache local_dir/Qwen2-Math-72B-Instruct-q0f16-MLC

  0%|          | 0/563 [00:00<?, ?it/s]
                                       
[2024-08-08 16:30:02] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00037-of-00037.safetensors

  0%|          | 0/563 [00:00<?, ?it/s]
                                       
[2024-08-08 16:30:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "lm_head.weight", shape: (152064, 8192), dtype: float16

  0%|          | 0/563 [00:05<?, ?it/s]
  0%|          | 1/563 [00:12<1:52:28, 12.01s/it]
                                                 
[2024-08-08 16:30:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.79.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  0%|          | 1/563 [00:12<1:52:28, 12.01s/it]
  0%|          | 2/563 [00:14<57:22,  6.14s/it]  
                                               
[2024-08-08 16:30:16] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00036-of-00037.safetensors

  0%|          | 2/563 [00:14<57:22,  6.14s/it]
                                               
[2024-08-08 16:30:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.79.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  0%|          | 2/563 [00:16<57:22,  6.14s/it]
  1%|          | 3/563 [00:19<52:55,  5.67s/it]
                                               
[2024-08-08 16:30:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.norm.weight", shape: (8192,), dtype: float16

  1%|          | 3/563 [00:19<52:55,  5.67s/it]
  1%|          | 4/563 [00:19<32:25,  3.48s/it]
                                               
[2024-08-08 16:30:21] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00036-of-00037.safetensors

  1%|          | 4/563 [00:19<32:25,  3.48s/it]
                                               
[2024-08-08 16:30:21] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00037-of-00037.safetensors

  1%|          | 4/563 [00:19<32:25,  3.48s/it]
                                               
[2024-08-08 16:30:22] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00001-of-00037.safetensors

  1%|          | 4/563 [00:19<32:25,  3.48s/it]
                                               
[2024-08-08 16:30:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.embed_tokens.weight", shape: (152064, 8192), dtype: float16

  1%|          | 4/563 [00:24<32:25,  3.48s/it]
  1%|          | 5/563 [00:30<56:53,  6.12s/it]
                                               
[2024-08-08 16:30:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.input_layernorm.weight", shape: (8192,), dtype: float16

  1%|          | 5/563 [00:30<56:53,  6.12s/it]
  1%|          | 6/563 [00:30<38:13,  4.12s/it]
                                               
[2024-08-08 16:30:34] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  1%|          | 6/563 [00:31<38:13,  4.12s/it]
  1%|          | 7/563 [00:34<37:23,  4.04s/it]
                                               
[2024-08-08 16:30:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  1%|          | 7/563 [00:34<37:23,  4.04s/it]
  1%|▏         | 8/563 [00:34<25:47,  2.79s/it]
                                               
[2024-08-08 16:30:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  1%|▏         | 8/563 [00:34<25:47,  2.79s/it]
                                               
[2024-08-08 16:30:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  1%|▏         | 8/563 [00:34<25:47,  2.79s/it]
  2%|▏         | 10/563 [00:34<14:46,  1.60s/it]
                                                
[2024-08-08 16:30:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

  2%|▏         | 10/563 [00:35<14:46,  1.60s/it]
  2%|▏         | 11/563 [00:35<12:04,  1.31s/it]
                                                
[2024-08-08 16:30:37] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00001-of-00037.safetensors

  2%|▏         | 11/563 [00:35<12:04,  1.31s/it]
                                                
[2024-08-08 16:30:37] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00002-of-00037.safetensors

  2%|▏         | 11/563 [00:35<12:04,  1.31s/it]
                                                
[2024-08-08 16:30:40] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.0.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  2%|▏         | 11/563 [00:37<12:04,  1.31s/it]
  2%|▏         | 12/563 [00:38<17:15,  1.88s/it]
                                                
[2024-08-08 16:30:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.input_layernorm.weight", shape: (8192,), dtype: float16

  2%|▏         | 12/563 [00:38<17:15,  1.88s/it]
                                                
[2024-08-08 16:30:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  2%|▏         | 12/563 [00:39<17:15,  1.88s/it]
  2%|▏         | 14/563 [00:40<13:07,  1.43s/it]
                                                
[2024-08-08 16:30:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  2%|▏         | 14/563 [00:42<13:07,  1.43s/it]
  3%|▎         | 15/563 [00:44<18:24,  2.02s/it]
                                                
[2024-08-08 16:30:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  3%|▎         | 15/563 [00:44<18:24,  2.02s/it]
  3%|▎         | 16/563 [00:44<14:01,  1.54s/it]
                                                
[2024-08-08 16:30:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  3%|▎         | 16/563 [00:44<14:01,  1.54s/it]
                                                
[2024-08-08 16:30:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  3%|▎         | 16/563 [00:44<14:01,  1.54s/it]
  3%|▎         | 18/563 [00:45<09:20,  1.03s/it]
                                                
[2024-08-08 16:30:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.1.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

  3%|▎         | 18/563 [00:45<09:20,  1.03s/it]
  3%|▎         | 19/563 [00:45<08:03,  1.12it/s]
                                                
[2024-08-08 16:30:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.input_layernorm.weight", shape: (8192,), dtype: float16

  3%|▎         | 19/563 [00:45<08:03,  1.12it/s]
                                                
[2024-08-08 16:30:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  3%|▎         | 19/563 [00:46<08:03,  1.12it/s]
  4%|▎         | 21/563 [00:47<07:49,  1.15it/s]
                                                
[2024-08-08 16:30:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  4%|▎         | 21/563 [00:48<07:49,  1.15it/s]
  4%|▍         | 22/563 [00:51<13:44,  1.52s/it]
                                                
[2024-08-08 16:30:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  4%|▍         | 22/563 [00:51<13:44,  1.52s/it]
  4%|▍         | 23/563 [00:51<10:40,  1.19s/it]
                                                
[2024-08-08 16:30:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  4%|▍         | 23/563 [00:51<10:40,  1.19s/it]
                                                
[2024-08-08 16:30:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  4%|▍         | 23/563 [00:51<10:40,  1.19s/it]
  4%|▍         | 25/563 [00:51<07:24,  1.21it/s]
                                                
[2024-08-08 16:30:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.2.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

  4%|▍         | 25/563 [00:51<07:24,  1.21it/s]
  5%|▍         | 26/563 [00:52<06:35,  1.36it/s]
                                                
[2024-08-08 16:30:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.input_layernorm.weight", shape: (8192,), dtype: float16

  5%|▍         | 26/563 [00:52<06:35,  1.36it/s]
                                                
[2024-08-08 16:30:54] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00002-of-00037.safetensors

  5%|▍         | 26/563 [00:52<06:35,  1.36it/s]
                                                
[2024-08-08 16:30:54] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00006-of-00037.safetensors

  5%|▍         | 26/563 [00:52<06:35,  1.36it/s]
                                                
[2024-08-08 16:30:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.input_layernorm.weight", shape: (8192,), dtype: float16

  5%|▍         | 26/563 [00:53<06:35,  1.36it/s]
  5%|▍         | 28/563 [00:53<06:37,  1.35it/s]
                                                
[2024-08-08 16:30:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  5%|▍         | 28/563 [00:54<06:37,  1.35it/s]
  5%|▌         | 29/563 [00:55<08:25,  1.06it/s]
                                                
[2024-08-08 16:30:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  5%|▌         | 29/563 [00:57<08:25,  1.06it/s]
  5%|▌         | 30/563 [00:59<14:56,  1.68s/it]
                                                
[2024-08-08 16:31:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  5%|▌         | 30/563 [00:59<14:56,  1.68s/it]
  6%|▌         | 31/563 [00:59<11:24,  1.29s/it]
                                                
[2024-08-08 16:31:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  6%|▌         | 31/563 [00:59<11:24,  1.29s/it]
                                                
[2024-08-08 16:31:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  6%|▌         | 31/563 [00:59<11:24,  1.29s/it]
  6%|▌         | 33/563 [01:00<07:40,  1.15it/s]
                                                
[2024-08-08 16:31:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.10.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

  6%|▌         | 33/563 [01:00<07:40,  1.15it/s]
  6%|▌         | 34/563 [01:00<06:43,  1.31it/s]
                                                
[2024-08-08 16:31:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.input_layernorm.weight", shape: (8192,), dtype: float16

  6%|▌         | 34/563 [01:00<06:43,  1.31it/s]
                                                
[2024-08-08 16:31:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  6%|▌         | 34/563 [01:01<06:43,  1.31it/s]
  6%|▋         | 36/563 [01:02<06:58,  1.26it/s]
                                                
[2024-08-08 16:31:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  6%|▋         | 36/563 [01:03<06:58,  1.26it/s]
  7%|▋         | 37/563 [01:06<12:59,  1.48s/it]
                                                
[2024-08-08 16:31:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  7%|▋         | 37/563 [01:06<12:59,  1.48s/it]
  7%|▋         | 38/563 [01:06<10:06,  1.15s/it]
                                                
[2024-08-08 16:31:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  7%|▋         | 38/563 [01:06<10:06,  1.15s/it]
                                                
[2024-08-08 16:31:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  7%|▋         | 38/563 [01:06<10:06,  1.15s/it]
  7%|▋         | 40/563 [01:06<06:59,  1.25it/s]
                                                
[2024-08-08 16:31:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.11.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

  7%|▋         | 40/563 [01:06<06:59,  1.25it/s]
  7%|▋         | 41/563 [01:07<06:15,  1.39it/s]
                                                
[2024-08-08 16:31:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.input_layernorm.weight", shape: (8192,), dtype: float16

  7%|▋         | 41/563 [01:07<06:15,  1.39it/s]
                                                
[2024-08-08 16:31:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.9.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  7%|▋         | 41/563 [01:07<06:15,  1.39it/s]
  8%|▊         | 43/563 [01:08<06:38,  1.30it/s]
                                                
[2024-08-08 16:31:11] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00006-of-00037.safetensors

  8%|▊         | 43/563 [01:08<06:38,  1.30it/s]
                                                
[2024-08-08 16:31:11] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00007-of-00037.safetensors

  8%|▊         | 43/563 [01:09<06:38,  1.30it/s]
                                                
[2024-08-08 16:31:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  8%|▊         | 43/563 [01:11<06:38,  1.30it/s]
  8%|▊         | 44/563 [01:12<12:23,  1.43s/it]
                                                
[2024-08-08 16:31:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  8%|▊         | 44/563 [01:14<12:23,  1.43s/it]
  8%|▊         | 45/563 [01:16<17:46,  2.06s/it]
                                                
[2024-08-08 16:31:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  8%|▊         | 45/563 [01:16<17:46,  2.06s/it]
  8%|▊         | 46/563 [01:16<13:29,  1.57s/it]
                                                
[2024-08-08 16:31:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  8%|▊         | 46/563 [01:16<13:29,  1.57s/it]
                                                
[2024-08-08 16:31:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  8%|▊         | 46/563 [01:17<13:29,  1.57s/it]
  9%|▊         | 48/563 [01:17<08:49,  1.03s/it]
                                                
[2024-08-08 16:31:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.12.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

  9%|▊         | 48/563 [01:17<08:49,  1.03s/it]
  9%|▊         | 49/563 [01:17<07:38,  1.12it/s]
                                                
[2024-08-08 16:31:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.input_layernorm.weight", shape: (8192,), dtype: float16

  9%|▊         | 49/563 [01:17<07:38,  1.12it/s]
                                                
[2024-08-08 16:31:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

  9%|▊         | 49/563 [01:18<07:38,  1.12it/s]
  9%|▉         | 51/563 [01:19<07:27,  1.15it/s]
                                                
[2024-08-08 16:31:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

  9%|▉         | 51/563 [01:21<07:27,  1.15it/s]
  9%|▉         | 52/563 [01:23<12:53,  1.51s/it]
                                                
[2024-08-08 16:31:25] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.post_attention_layernorm.weight", shape: (8192,), dtype: float16

  9%|▉         | 52/563 [01:23<12:53,  1.51s/it]
  9%|▉         | 53/563 [01:23<10:01,  1.18s/it]
                                                
[2024-08-08 16:31:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.self_attn.c_attn.bias", shape: (10240,), dtype: float16

  9%|▉         | 53/563 [01:23<10:01,  1.18s/it]
                                                
[2024-08-08 16:31:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

  9%|▉         | 53/563 [01:23<10:01,  1.18s/it]
 10%|▉         | 55/563 [01:24<06:54,  1.23it/s]
                                                
[2024-08-08 16:31:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.13.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 10%|▉         | 55/563 [01:24<06:54,  1.23it/s]
 10%|▉         | 56/563 [01:24<06:10,  1.37it/s]
                                                
[2024-08-08 16:31:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.input_layernorm.weight", shape: (8192,), dtype: float16

 10%|▉         | 56/563 [01:24<06:10,  1.37it/s]
                                                
[2024-08-08 16:31:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 10%|▉         | 56/563 [01:24<06:10,  1.37it/s]
                                                
[2024-08-08 16:31:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 10%|▉         | 56/563 [01:24<06:10,  1.37it/s]
                                                
[2024-08-08 16:31:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 10%|▉         | 56/563 [01:24<06:10,  1.37it/s]
 11%|█         | 60/563 [01:25<03:23,  2.48it/s]
                                                
[2024-08-08 16:31:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 11%|█         | 60/563 [01:25<03:23,  2.48it/s]
 11%|█         | 61/563 [01:25<03:26,  2.43it/s]
                                                
[2024-08-08 16:31:28] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00007-of-00037.safetensors

 11%|█         | 61/563 [01:25<03:26,  2.43it/s]
                                                
[2024-08-08 16:31:28] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00008-of-00037.safetensors

 11%|█         | 61/563 [01:25<03:26,  2.43it/s]
                                                
[2024-08-08 16:31:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 11%|█         | 61/563 [01:28<03:26,  2.43it/s]
 11%|█         | 62/563 [01:29<08:57,  1.07s/it]
                                                
[2024-08-08 16:31:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.14.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 11%|█         | 62/563 [01:30<08:57,  1.07s/it]
 11%|█         | 63/563 [01:33<13:32,  1.62s/it]
                                                
[2024-08-08 16:31:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.input_layernorm.weight", shape: (8192,), dtype: float16

 11%|█         | 63/563 [01:33<13:32,  1.62s/it]
 11%|█▏        | 64/563 [01:33<10:32,  1.27s/it]
                                                
[2024-08-08 16:31:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 11%|█▏        | 64/563 [01:33<10:32,  1.27s/it]
 12%|█▏        | 65/563 [01:34<11:16,  1.36s/it]
                                                
[2024-08-08 16:31:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 12%|█▏        | 65/563 [01:36<11:16,  1.36s/it]
 12%|█▏        | 66/563 [01:38<16:15,  1.96s/it]
                                                
[2024-08-08 16:31:40] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 12%|█▏        | 66/563 [01:38<16:15,  1.96s/it]
 12%|█▏        | 67/563 [01:38<12:02,  1.46s/it]
                                                
[2024-08-08 16:31:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 12%|█▏        | 67/563 [01:38<12:02,  1.46s/it]
                                                
[2024-08-08 16:31:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 12%|█▏        | 67/563 [01:38<12:02,  1.46s/it]
 12%|█▏        | 69/563 [01:39<07:46,  1.06it/s]
                                                
[2024-08-08 16:31:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.15.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 12%|█▏        | 69/563 [01:39<07:46,  1.06it/s]
 12%|█▏        | 70/563 [01:39<06:45,  1.21it/s]
                                                
[2024-08-08 16:31:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.input_layernorm.weight", shape: (8192,), dtype: float16

 12%|█▏        | 70/563 [01:39<06:45,  1.21it/s]
                                                
[2024-08-08 16:31:42] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00009-of-00037.safetensors

 12%|█▏        | 70/563 [01:39<06:45,  1.21it/s]
                                                
[2024-08-08 16:31:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 12%|█▏        | 70/563 [01:44<06:45,  1.21it/s]
 13%|█▎        | 72/563 [01:46<15:41,  1.92s/it]
                                                
[2024-08-08 16:31:49] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 13%|█▎        | 72/563 [01:46<15:41,  1.92s/it]
 13%|█▎        | 73/563 [01:46<12:21,  1.51s/it]
                                                
[2024-08-08 16:31:49] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 13%|█▎        | 73/563 [01:46<12:21,  1.51s/it]
                                                
[2024-08-08 16:31:49] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 13%|█▎        | 73/563 [01:46<12:21,  1.51s/it]
 13%|█▎        | 75/563 [01:47<08:24,  1.03s/it]
                                                
[2024-08-08 16:31:49] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 13%|█▎        | 75/563 [01:47<08:24,  1.03s/it]
 13%|█▎        | 76/563 [01:47<07:17,  1.11it/s]
                                                
[2024-08-08 16:31:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.16.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 13%|█▎        | 76/563 [01:48<07:17,  1.11it/s]
 14%|█▎        | 77/563 [01:49<08:39,  1.07s/it]
                                                
[2024-08-08 16:31:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.input_layernorm.weight", shape: (8192,), dtype: float16

 14%|█▎        | 77/563 [01:49<08:39,  1.07s/it]
                                                
[2024-08-08 16:31:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 14%|█▎        | 77/563 [01:49<08:39,  1.07s/it]
 14%|█▍        | 79/563 [01:50<07:46,  1.04it/s]
                                                
[2024-08-08 16:31:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 14%|█▍        | 79/563 [01:52<07:46,  1.04it/s]
 14%|█▍        | 80/563 [01:54<12:22,  1.54s/it]
                                                
[2024-08-08 16:31:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 14%|█▍        | 80/563 [01:54<12:22,  1.54s/it]
 14%|█▍        | 81/563 [01:54<09:33,  1.19s/it]
                                                
[2024-08-08 16:31:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 14%|█▍        | 81/563 [01:54<09:33,  1.19s/it]
                                                
[2024-08-08 16:31:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 14%|█▍        | 81/563 [01:54<09:33,  1.19s/it]
 15%|█▍        | 83/563 [01:55<06:33,  1.22it/s]
                                                
[2024-08-08 16:31:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.17.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 15%|█▍        | 83/563 [01:55<06:33,  1.22it/s]
 15%|█▍        | 84/563 [01:55<05:50,  1.37it/s]
                                                
[2024-08-08 16:31:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.input_layernorm.weight", shape: (8192,), dtype: float16

 15%|█▍        | 84/563 [01:55<05:50,  1.37it/s]
                                                
[2024-08-08 16:31:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 15%|█▍        | 84/563 [01:56<05:50,  1.37it/s]
 15%|█▌        | 86/563 [01:59<08:54,  1.12s/it]
                                                
[2024-08-08 16:32:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 15%|█▌        | 86/563 [01:59<08:54,  1.12s/it]
 15%|█▌        | 87/563 [01:59<07:08,  1.11it/s]
                                                
[2024-08-08 16:32:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 15%|█▌        | 87/563 [01:59<07:08,  1.11it/s]
                                                
[2024-08-08 16:32:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 15%|█▌        | 87/563 [01:59<07:08,  1.11it/s]
 16%|█▌        | 89/563 [01:59<05:13,  1.51it/s]
                                                
[2024-08-08 16:32:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 16%|█▌        | 89/563 [01:59<05:13,  1.51it/s]
 16%|█▌        | 90/563 [02:00<04:48,  1.64it/s]
                                                
[2024-08-08 16:32:02] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00009-of-00037.safetensors

 16%|█▌        | 90/563 [02:00<04:48,  1.64it/s]
                                                
[2024-08-08 16:32:02] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00008-of-00037.safetensors

 16%|█▌        | 90/563 [02:00<04:48,  1.64it/s]
                                                
[2024-08-08 16:32:02] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00010-of-00037.safetensors

 16%|█▌        | 90/563 [02:00<04:48,  1.64it/s]
                                                
[2024-08-08 16:32:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.18.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 16%|█▌        | 90/563 [02:02<04:48,  1.64it/s]
 16%|█▌        | 91/563 [02:03<10:13,  1.30s/it]
                                                
[2024-08-08 16:32:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.input_layernorm.weight", shape: (8192,), dtype: float16

 16%|█▌        | 91/563 [02:03<10:13,  1.30s/it]
                                                
[2024-08-08 16:32:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 16%|█▌        | 91/563 [02:04<10:13,  1.30s/it]
 17%|█▋        | 93/563 [02:05<08:36,  1.10s/it]
                                                
[2024-08-08 16:32:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 17%|█▋        | 93/563 [02:06<08:36,  1.10s/it]
 17%|█▋        | 94/563 [02:08<12:40,  1.62s/it]
                                                
[2024-08-08 16:32:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 17%|█▋        | 94/563 [02:08<12:40,  1.62s/it]
 17%|█▋        | 95/563 [02:08<09:48,  1.26s/it]
                                                
[2024-08-08 16:32:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 17%|█▋        | 95/563 [02:08<09:48,  1.26s/it]
                                                
[2024-08-08 16:32:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 17%|█▋        | 95/563 [02:09<09:48,  1.26s/it]
 17%|█▋        | 97/563 [02:09<06:41,  1.16it/s]
                                                
[2024-08-08 16:32:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.19.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 17%|█▋        | 97/563 [02:09<06:41,  1.16it/s]
 17%|█▋        | 98/563 [02:09<05:53,  1.31it/s]
                                                
[2024-08-08 16:32:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.input_layernorm.weight", shape: (8192,), dtype: float16

 17%|█▋        | 98/563 [02:09<05:53,  1.31it/s]
                                                
[2024-08-08 16:32:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 17%|█▋        | 98/563 [02:10<05:53,  1.31it/s]
 18%|█▊        | 100/563 [02:11<05:59,  1.29it/s]
                                                 
[2024-08-08 16:32:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 18%|█▊        | 100/563 [02:12<05:59,  1.29it/s]
 18%|█▊        | 101/563 [02:14<10:31,  1.37s/it]
                                                 
[2024-08-08 16:32:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 18%|█▊        | 101/563 [02:14<10:31,  1.37s/it]
 18%|█▊        | 102/563 [02:15<08:13,  1.07s/it]
                                                 
[2024-08-08 16:32:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 18%|█▊        | 102/563 [02:15<08:13,  1.07s/it]
                                                 
[2024-08-08 16:32:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 18%|█▊        | 102/563 [02:15<08:13,  1.07s/it]
 18%|█▊        | 104/563 [02:15<05:45,  1.33it/s]
                                                 
[2024-08-08 16:32:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.20.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 18%|█▊        | 104/563 [02:15<05:45,  1.33it/s]
 19%|█▊        | 105/563 [02:16<05:11,  1.47it/s]
                                                 
[2024-08-08 16:32:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.input_layernorm.weight", shape: (8192,), dtype: float16

 19%|█▊        | 105/563 [02:16<05:11,  1.47it/s]
                                                 
[2024-08-08 16:32:18] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00010-of-00037.safetensors

 19%|█▊        | 105/563 [02:16<05:11,  1.47it/s]
                                                 
[2024-08-08 16:32:18] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00011-of-00037.safetensors

 19%|█▊        | 105/563 [02:16<05:11,  1.47it/s]
                                                 
[2024-08-08 16:32:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 19%|█▊        | 105/563 [02:18<05:11,  1.47it/s]
 19%|█▉        | 107/563 [02:19<08:23,  1.10s/it]
                                                 
[2024-08-08 16:32:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 19%|█▉        | 107/563 [02:20<08:23,  1.10s/it]
 19%|█▉        | 108/563 [02:23<12:17,  1.62s/it]
                                                 
[2024-08-08 16:32:25] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 19%|█▉        | 108/563 [02:23<12:17,  1.62s/it]
 19%|█▉        | 109/563 [02:23<09:33,  1.26s/it]
                                                 
[2024-08-08 16:32:25] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 19%|█▉        | 109/563 [02:23<09:33,  1.26s/it]
                                                 
[2024-08-08 16:32:25] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 19%|█▉        | 109/563 [02:23<09:33,  1.26s/it]
 20%|█▉        | 111/563 [02:23<06:32,  1.15it/s]
                                                 
[2024-08-08 16:32:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.21.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 20%|█▉        | 111/563 [02:23<06:32,  1.15it/s]
 20%|█▉        | 112/563 [02:24<05:46,  1.30it/s]
                                                 
[2024-08-08 16:32:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.input_layernorm.weight", shape: (8192,), dtype: float16

 20%|█▉        | 112/563 [02:24<05:46,  1.30it/s]
                                                 
[2024-08-08 16:32:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 20%|█▉        | 112/563 [02:24<05:46,  1.30it/s]
 20%|██        | 114/563 [02:25<05:50,  1.28it/s]
                                                 
[2024-08-08 16:32:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 20%|██        | 114/563 [02:27<05:50,  1.28it/s]
 20%|██        | 115/563 [02:29<10:15,  1.37s/it]
                                                 
[2024-08-08 16:32:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 20%|██        | 115/563 [02:29<10:15,  1.37s/it]
 21%|██        | 116/563 [02:29<08:01,  1.08s/it]
                                                 
[2024-08-08 16:32:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 21%|██        | 116/563 [02:29<08:01,  1.08s/it]
                                                 
[2024-08-08 16:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 21%|██        | 116/563 [02:29<08:01,  1.08s/it]
 21%|██        | 118/563 [02:29<05:36,  1.32it/s]
                                                 
[2024-08-08 16:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.22.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 21%|██        | 118/563 [02:30<05:36,  1.32it/s]
 21%|██        | 119/563 [02:30<05:02,  1.47it/s]
                                                 
[2024-08-08 16:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.input_layernorm.weight", shape: (8192,), dtype: float16

 21%|██        | 119/563 [02:30<05:02,  1.47it/s]
                                                 
[2024-08-08 16:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 21%|██        | 119/563 [02:30<05:02,  1.47it/s]
                                                 
[2024-08-08 16:32:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 21%|██        | 119/563 [02:30<05:02,  1.47it/s]
                                                 
[2024-08-08 16:32:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 21%|██        | 119/563 [02:30<05:02,  1.47it/s]
 22%|██▏       | 123/563 [02:30<02:47,  2.63it/s]
                                                 
[2024-08-08 16:32:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 22%|██▏       | 123/563 [02:31<02:47,  2.63it/s]
 22%|██▏       | 124/563 [02:31<02:50,  2.58it/s]
                                                 
[2024-08-08 16:32:33] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00011-of-00037.safetensors

 22%|██▏       | 124/563 [02:31<02:50,  2.58it/s]
                                                 
[2024-08-08 16:32:33] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00012-of-00037.safetensors

 22%|██▏       | 124/563 [02:31<02:50,  2.58it/s]
                                                 
[2024-08-08 16:32:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 22%|██▏       | 124/563 [02:34<02:50,  2.58it/s]
 22%|██▏       | 125/563 [02:35<07:27,  1.02s/it]
                                                 
[2024-08-08 16:32:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.23.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 22%|██▏       | 125/563 [02:36<07:27,  1.02s/it]
 22%|██▏       | 126/563 [02:38<11:19,  1.56s/it]
                                                 
[2024-08-08 16:32:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.input_layernorm.weight", shape: (8192,), dtype: float16

 22%|██▏       | 126/563 [02:38<11:19,  1.56s/it]
 23%|██▎       | 127/563 [02:38<08:49,  1.22s/it]
                                                 
[2024-08-08 16:32:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 23%|██▎       | 127/563 [02:39<08:49,  1.22s/it]
 23%|██▎       | 128/563 [02:40<09:27,  1.30s/it]
                                                 
[2024-08-08 16:32:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 23%|██▎       | 128/563 [02:41<09:27,  1.30s/it]
 23%|██▎       | 129/563 [02:43<13:28,  1.86s/it]
                                                 
[2024-08-08 16:32:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 23%|██▎       | 129/563 [02:43<13:28,  1.86s/it]
 23%|██▎       | 130/563 [02:43<09:59,  1.39s/it]
                                                 
[2024-08-08 16:32:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 23%|██▎       | 130/563 [02:43<09:59,  1.39s/it]
                                                 
[2024-08-08 16:32:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 23%|██▎       | 130/563 [02:43<09:59,  1.39s/it]
 23%|██▎       | 132/563 [02:44<06:27,  1.11it/s]
                                                 
[2024-08-08 16:32:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.24.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 23%|██▎       | 132/563 [02:44<06:27,  1.11it/s]
 24%|██▎       | 133/563 [02:44<05:38,  1.27it/s]
                                                 
[2024-08-08 16:32:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.input_layernorm.weight", shape: (8192,), dtype: float16

 24%|██▎       | 133/563 [02:44<05:38,  1.27it/s]
                                                 
[2024-08-08 16:32:47] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00013-of-00037.safetensors

 24%|██▎       | 133/563 [02:44<05:38,  1.27it/s]
                                                 
[2024-08-08 16:32:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 24%|██▎       | 133/563 [02:49<05:38,  1.27it/s]
 24%|██▍       | 135/563 [02:51<13:49,  1.94s/it]
                                                 
[2024-08-08 16:32:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 24%|██▍       | 135/563 [02:52<13:49,  1.94s/it]
 24%|██▍       | 136/563 [02:52<10:52,  1.53s/it]
                                                 
[2024-08-08 16:32:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 24%|██▍       | 136/563 [02:52<10:52,  1.53s/it]
                                                 
[2024-08-08 16:32:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 24%|██▍       | 136/563 [02:52<10:52,  1.53s/it]
 25%|██▍       | 138/563 [02:52<07:23,  1.04s/it]
                                                 
[2024-08-08 16:32:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 25%|██▍       | 138/563 [02:52<07:23,  1.04s/it]
 25%|██▍       | 139/563 [02:53<06:25,  1.10it/s]
                                                 
[2024-08-08 16:32:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.25.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 25%|██▍       | 139/563 [02:53<06:25,  1.10it/s]
 25%|██▍       | 140/563 [02:54<07:30,  1.07s/it]
                                                 
[2024-08-08 16:32:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.input_layernorm.weight", shape: (8192,), dtype: float16

 25%|██▍       | 140/563 [02:54<07:30,  1.07s/it]
                                                 
[2024-08-08 16:32:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 25%|██▍       | 140/563 [02:55<07:30,  1.07s/it]
 25%|██▌       | 142/563 [02:56<06:43,  1.04it/s]
                                                 
[2024-08-08 16:33:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 25%|██▌       | 142/563 [02:57<06:43,  1.04it/s]
 25%|██▌       | 143/563 [02:59<10:42,  1.53s/it]
                                                 
[2024-08-08 16:33:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 25%|██▌       | 143/563 [02:59<10:42,  1.53s/it]
 26%|██▌       | 144/563 [02:59<08:17,  1.19s/it]
                                                 
[2024-08-08 16:33:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 26%|██▌       | 144/563 [02:59<08:17,  1.19s/it]
                                                 
[2024-08-08 16:33:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 26%|██▌       | 144/563 [03:00<08:17,  1.19s/it]
 26%|██▌       | 146/563 [03:00<05:47,  1.20it/s]
                                                 
[2024-08-08 16:33:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.26.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 26%|██▌       | 146/563 [03:00<05:47,  1.20it/s]
 26%|██▌       | 147/563 [03:01<05:13,  1.33it/s]
                                                 
[2024-08-08 16:33:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.input_layernorm.weight", shape: (8192,), dtype: float16

 26%|██▌       | 147/563 [03:01<05:13,  1.33it/s]
                                                 
[2024-08-08 16:33:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 26%|██▌       | 147/563 [03:02<05:13,  1.33it/s]
 26%|██▋       | 149/563 [03:04<07:55,  1.15s/it]
                                                 
[2024-08-08 16:33:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 26%|██▋       | 149/563 [03:04<07:55,  1.15s/it]
 27%|██▋       | 150/563 [03:04<06:21,  1.08it/s]
                                                 
[2024-08-08 16:33:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 27%|██▋       | 150/563 [03:04<06:21,  1.08it/s]
                                                 
[2024-08-08 16:33:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 27%|██▋       | 150/563 [03:04<06:21,  1.08it/s]
 27%|██▋       | 152/563 [03:05<04:37,  1.48it/s]
                                                 
[2024-08-08 16:33:07] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 27%|██▋       | 152/563 [03:05<04:37,  1.48it/s]
 27%|██▋       | 153/563 [03:05<04:14,  1.61it/s]
                                                 
[2024-08-08 16:33:08] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00012-of-00037.safetensors

 27%|██▋       | 153/563 [03:05<04:14,  1.61it/s]
                                                 
[2024-08-08 16:33:08] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00013-of-00037.safetensors

 27%|██▋       | 153/563 [03:05<04:14,  1.61it/s]
                                                 
[2024-08-08 16:33:08] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00014-of-00037.safetensors

 27%|██▋       | 153/563 [03:05<04:14,  1.61it/s]
                                                 
[2024-08-08 16:33:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.27.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 27%|██▋       | 153/563 [03:08<04:14,  1.61it/s]
 27%|██▋       | 154/563 [03:09<09:15,  1.36s/it]
                                                 
[2024-08-08 16:33:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.input_layernorm.weight", shape: (8192,), dtype: float16

 27%|██▋       | 154/563 [03:09<09:15,  1.36s/it]
                                                 
[2024-08-08 16:33:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 27%|██▋       | 154/563 [03:09<09:15,  1.36s/it]
 28%|██▊       | 156/563 [03:10<07:41,  1.13s/it]
                                                 
[2024-08-08 16:33:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 28%|██▊       | 156/563 [03:12<07:41,  1.13s/it]
 28%|██▊       | 157/563 [03:14<11:11,  1.65s/it]
                                                 
[2024-08-08 16:33:16] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 28%|██▊       | 157/563 [03:14<11:11,  1.65s/it]
 28%|██▊       | 158/563 [03:14<08:38,  1.28s/it]
                                                 
[2024-08-08 16:33:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 28%|██▊       | 158/563 [03:14<08:38,  1.28s/it]
                                                 
[2024-08-08 16:33:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 28%|██▊       | 158/563 [03:14<08:38,  1.28s/it]
 28%|██▊       | 160/563 [03:15<05:52,  1.14it/s]
                                                 
[2024-08-08 16:33:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.28.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 28%|██▊       | 160/563 [03:15<05:52,  1.14it/s]
 29%|██▊       | 161/563 [03:15<05:12,  1.29it/s]
                                                 
[2024-08-08 16:33:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.input_layernorm.weight", shape: (8192,), dtype: float16

 29%|██▊       | 161/563 [03:15<05:12,  1.29it/s]
                                                 
[2024-08-08 16:33:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 29%|██▊       | 161/563 [03:16<05:12,  1.29it/s]
 29%|██▉       | 163/563 [03:17<05:13,  1.27it/s]
                                                 
[2024-08-08 16:33:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 29%|██▉       | 163/563 [03:18<05:13,  1.27it/s]
 29%|██▉       | 164/563 [03:20<09:14,  1.39s/it]
                                                 
[2024-08-08 16:33:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 29%|██▉       | 164/563 [03:20<09:14,  1.39s/it]
 29%|██▉       | 165/563 [03:20<07:12,  1.09s/it]
                                                 
[2024-08-08 16:33:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 29%|██▉       | 165/563 [03:20<07:12,  1.09s/it]
                                                 
[2024-08-08 16:33:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 29%|██▉       | 165/563 [03:21<07:12,  1.09s/it]
 30%|██▉       | 167/563 [03:21<05:01,  1.31it/s]
                                                 
[2024-08-08 16:33:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.29.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 30%|██▉       | 167/563 [03:21<05:01,  1.31it/s]
 30%|██▉       | 168/563 [03:21<04:30,  1.46it/s]
                                                 
[2024-08-08 16:33:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.input_layernorm.weight", shape: (8192,), dtype: float16

 30%|██▉       | 168/563 [03:21<04:30,  1.46it/s]
                                                 
[2024-08-08 16:33:24] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00014-of-00037.safetensors

 30%|██▉       | 168/563 [03:21<04:30,  1.46it/s]
                                                 
[2024-08-08 16:33:24] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00003-of-00037.safetensors

 30%|██▉       | 168/563 [03:21<04:30,  1.46it/s]
                                                 
[2024-08-08 16:33:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 30%|██▉       | 168/563 [03:24<04:30,  1.46it/s]
 30%|███       | 170/563 [03:25<07:05,  1.08s/it]
                                                 
[2024-08-08 16:33:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 30%|███       | 170/563 [03:26<07:05,  1.08s/it]
 30%|███       | 171/563 [03:28<10:35,  1.62s/it]
                                                 
[2024-08-08 16:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 30%|███       | 171/563 [03:28<10:35,  1.62s/it]
 31%|███       | 172/563 [03:28<08:13,  1.26s/it]
                                                 
[2024-08-08 16:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 31%|███       | 172/563 [03:28<08:13,  1.26s/it]
                                                 
[2024-08-08 16:33:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 31%|███       | 172/563 [03:29<08:13,  1.26s/it]
 31%|███       | 174/563 [03:29<05:39,  1.15it/s]
                                                 
[2024-08-08 16:33:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.3.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 31%|███       | 174/563 [03:29<05:39,  1.15it/s]
 31%|███       | 175/563 [03:29<05:00,  1.29it/s]
                                                 
[2024-08-08 16:33:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.input_layernorm.weight", shape: (8192,), dtype: float16

 31%|███       | 175/563 [03:29<05:00,  1.29it/s]
                                                 
[2024-08-08 16:33:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 31%|███       | 175/563 [03:30<05:00,  1.29it/s]
 31%|███▏      | 177/563 [03:31<05:04,  1.27it/s]
                                                 
[2024-08-08 16:33:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 31%|███▏      | 177/563 [03:32<05:04,  1.27it/s]
 32%|███▏      | 178/563 [03:35<09:00,  1.40s/it]
                                                 
[2024-08-08 16:33:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 32%|███▏      | 178/563 [03:35<09:00,  1.40s/it]
 32%|███▏      | 179/563 [03:35<07:01,  1.10s/it]
                                                 
[2024-08-08 16:33:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 32%|███▏      | 179/563 [03:35<07:01,  1.10s/it]
                                                 
[2024-08-08 16:33:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 32%|███▏      | 179/563 [03:35<07:01,  1.10s/it]
 32%|███▏      | 181/563 [03:35<04:56,  1.29it/s]
                                                 
[2024-08-08 16:33:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.4.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 32%|███▏      | 181/563 [03:36<04:56,  1.29it/s]
 32%|███▏      | 182/563 [03:36<04:28,  1.42it/s]
                                                 
[2024-08-08 16:33:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.input_layernorm.weight", shape: (8192,), dtype: float16

 32%|███▏      | 182/563 [03:36<04:28,  1.42it/s]
                                                 
[2024-08-08 16:33:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 32%|███▏      | 182/563 [03:36<04:28,  1.42it/s]
                                                 
[2024-08-08 16:33:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 32%|███▏      | 182/563 [03:36<04:28,  1.42it/s]
                                                 
[2024-08-08 16:33:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 32%|███▏      | 182/563 [03:36<04:28,  1.42it/s]
 33%|███▎      | 186/563 [03:36<02:29,  2.53it/s]
                                                 
[2024-08-08 16:33:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.5.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 33%|███▎      | 186/563 [03:37<02:29,  2.53it/s]
 33%|███▎      | 187/563 [03:37<02:31,  2.48it/s]
                                                 
[2024-08-08 16:33:39] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00003-of-00037.safetensors

 33%|███▎      | 187/563 [03:37<02:31,  2.48it/s]
                                                 
[2024-08-08 16:33:40] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00015-of-00037.safetensors

 33%|███▎      | 187/563 [03:37<02:31,  2.48it/s]
                                                 
[2024-08-08 16:33:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 33%|███▎      | 187/563 [03:40<02:31,  2.48it/s]
 33%|███▎      | 188/563 [03:41<06:28,  1.04s/it]
                                                 
[2024-08-08 16:33:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 33%|███▎      | 188/563 [03:42<06:28,  1.04s/it]
 34%|███▎      | 189/563 [03:44<09:50,  1.58s/it]
                                                 
[2024-08-08 16:33:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 34%|███▎      | 189/563 [03:44<09:50,  1.58s/it]
 34%|███▎      | 190/563 [03:44<07:41,  1.24s/it]
                                                 
[2024-08-08 16:33:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 34%|███▎      | 190/563 [03:44<07:41,  1.24s/it]
                                                 
[2024-08-08 16:33:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 34%|███▎      | 190/563 [03:44<07:41,  1.24s/it]
 34%|███▍      | 192/563 [03:45<05:17,  1.17it/s]
                                                 
[2024-08-08 16:33:47] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.30.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 34%|███▍      | 192/563 [03:45<05:17,  1.17it/s]
 34%|███▍      | 193/563 [03:45<04:42,  1.31it/s]
                                                 
[2024-08-08 16:33:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.input_layernorm.weight", shape: (8192,), dtype: float16

 34%|███▍      | 193/563 [03:45<04:42,  1.31it/s]
                                                 
[2024-08-08 16:33:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 34%|███▍      | 193/563 [03:46<04:42,  1.31it/s]
 35%|███▍      | 195/563 [03:47<04:47,  1.28it/s]
                                                 
[2024-08-08 16:33:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 35%|███▍      | 195/563 [03:48<04:47,  1.28it/s]
 35%|███▍      | 196/563 [03:50<08:26,  1.38s/it]
                                                 
[2024-08-08 16:33:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 35%|███▍      | 196/563 [03:51<08:26,  1.38s/it]
 35%|███▍      | 197/563 [03:51<06:35,  1.08s/it]
                                                 
[2024-08-08 16:33:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 35%|███▍      | 197/563 [03:51<06:35,  1.08s/it]
                                                 
[2024-08-08 16:33:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 35%|███▍      | 197/563 [03:51<06:35,  1.08s/it]
 35%|███▌      | 199/563 [03:51<04:36,  1.32it/s]
                                                 
[2024-08-08 16:33:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.31.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 35%|███▌      | 199/563 [03:51<04:36,  1.32it/s]
 36%|███▌      | 200/563 [03:52<04:09,  1.45it/s]
                                                 
[2024-08-08 16:33:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.input_layernorm.weight", shape: (8192,), dtype: float16

 36%|███▌      | 200/563 [03:52<04:09,  1.45it/s]
                                                 
[2024-08-08 16:33:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 36%|███▌      | 200/563 [03:52<04:09,  1.45it/s]
                                                 
[2024-08-08 16:33:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 36%|███▌      | 200/563 [03:52<04:09,  1.45it/s]
                                                 
[2024-08-08 16:33:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 36%|███▌      | 200/563 [03:52<04:09,  1.45it/s]
 36%|███▌      | 204/563 [03:52<02:19,  2.58it/s]
                                                 
[2024-08-08 16:33:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 36%|███▌      | 204/563 [03:52<02:19,  2.58it/s]
 36%|███▋      | 205/563 [03:53<02:21,  2.53it/s]
                                                 
[2024-08-08 16:33:55] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00015-of-00037.safetensors

 36%|███▋      | 205/563 [03:53<02:21,  2.53it/s]
                                                 
[2024-08-08 16:33:55] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00016-of-00037.safetensors

 36%|███▋      | 205/563 [03:53<02:21,  2.53it/s]
                                                 
[2024-08-08 16:33:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 36%|███▋      | 205/563 [03:55<02:21,  2.53it/s]
 37%|███▋      | 206/563 [03:57<06:25,  1.08s/it]
                                                 
[2024-08-08 16:34:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.32.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 37%|███▋      | 206/563 [03:58<06:25,  1.08s/it]
 37%|███▋      | 207/563 [04:00<09:45,  1.64s/it]
                                                 
[2024-08-08 16:34:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.input_layernorm.weight", shape: (8192,), dtype: float16

 37%|███▋      | 207/563 [04:00<09:45,  1.64s/it]
 37%|███▋      | 208/563 [04:00<07:36,  1.29s/it]
                                                 
[2024-08-08 16:34:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 37%|███▋      | 208/563 [04:01<07:36,  1.29s/it]
 37%|███▋      | 209/563 [04:02<08:13,  1.39s/it]
                                                 
[2024-08-08 16:34:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 37%|███▋      | 209/563 [04:04<08:13,  1.39s/it]
 37%|███▋      | 210/563 [04:06<12:15,  2.08s/it]
                                                 
[2024-08-08 16:34:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 37%|███▋      | 210/563 [04:06<12:15,  2.08s/it]
 37%|███▋      | 211/563 [04:06<09:04,  1.55s/it]
                                                 
[2024-08-08 16:34:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 37%|███▋      | 211/563 [04:06<09:04,  1.55s/it]
                                                 
[2024-08-08 16:34:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 37%|███▋      | 211/563 [04:06<09:04,  1.55s/it]
 38%|███▊      | 213/563 [04:07<05:53,  1.01s/it]
                                                 
[2024-08-08 16:34:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.33.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 38%|███▊      | 213/563 [04:07<05:53,  1.01s/it]
 38%|███▊      | 214/563 [04:07<05:09,  1.13it/s]
                                                 
[2024-08-08 16:34:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.input_layernorm.weight", shape: (8192,), dtype: float16

 38%|███▊      | 214/563 [04:07<05:09,  1.13it/s]
                                                 
[2024-08-08 16:34:10] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00017-of-00037.safetensors

 38%|███▊      | 214/563 [04:07<05:09,  1.13it/s]
                                                 
[2024-08-08 16:34:16] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 38%|███▊      | 214/563 [04:13<05:09,  1.13it/s]
 38%|███▊      | 216/563 [04:15<12:28,  2.16s/it]
                                                 
[2024-08-08 16:34:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 38%|███▊      | 216/563 [04:15<12:28,  2.16s/it]
 39%|███▊      | 217/563 [04:16<09:48,  1.70s/it]
                                                 
[2024-08-08 16:34:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 39%|███▊      | 217/563 [04:16<09:48,  1.70s/it]
                                                 
[2024-08-08 16:34:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 39%|███▊      | 217/563 [04:16<09:48,  1.70s/it]
 39%|███▉      | 219/563 [04:16<06:36,  1.15s/it]
                                                 
[2024-08-08 16:34:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 39%|███▉      | 219/563 [04:16<06:36,  1.15s/it]
 39%|███▉      | 220/563 [04:17<05:43,  1.00s/it]
                                                 
[2024-08-08 16:34:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.34.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 39%|███▉      | 220/563 [04:17<05:43,  1.00s/it]
 39%|███▉      | 221/563 [04:18<06:36,  1.16s/it]
                                                 
[2024-08-08 16:34:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.input_layernorm.weight", shape: (8192,), dtype: float16

 39%|███▉      | 221/563 [04:18<06:36,  1.16s/it]
                                                 
[2024-08-08 16:34:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 39%|███▉      | 221/563 [04:19<06:36,  1.16s/it]
 40%|███▉      | 223/563 [04:20<06:21,  1.12s/it]
                                                 
[2024-08-08 16:34:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 40%|███▉      | 223/563 [04:22<06:21,  1.12s/it]
 40%|███▉      | 224/563 [04:24<09:29,  1.68s/it]
                                                 
[2024-08-08 16:34:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 40%|███▉      | 224/563 [04:24<09:29,  1.68s/it]
 40%|███▉      | 225/563 [04:24<07:19,  1.30s/it]
                                                 
[2024-08-08 16:34:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 40%|███▉      | 225/563 [04:24<07:19,  1.30s/it]
                                                 
[2024-08-08 16:34:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 40%|███▉      | 225/563 [04:24<07:19,  1.30s/it]
 40%|████      | 227/563 [04:25<04:59,  1.12it/s]
                                                 
[2024-08-08 16:34:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.35.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 40%|████      | 227/563 [04:25<04:59,  1.12it/s]
 40%|████      | 228/563 [04:25<04:23,  1.27it/s]
                                                 
[2024-08-08 16:34:28] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.input_layernorm.weight", shape: (8192,), dtype: float16

 40%|████      | 228/563 [04:25<04:23,  1.27it/s]
                                                 
[2024-08-08 16:34:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 40%|████      | 228/563 [04:27<04:23,  1.27it/s]
 41%|████      | 230/563 [04:29<06:34,  1.19s/it]
                                                 
[2024-08-08 16:34:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 41%|████      | 230/563 [04:29<06:34,  1.19s/it]
 41%|████      | 231/563 [04:29<05:16,  1.05it/s]
                                                 
[2024-08-08 16:34:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 41%|████      | 231/563 [04:29<05:16,  1.05it/s]
                                                 
[2024-08-08 16:34:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 41%|████      | 231/563 [04:29<05:16,  1.05it/s]
 41%|████▏     | 233/563 [04:29<03:50,  1.43it/s]
                                                 
[2024-08-08 16:34:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 41%|████▏     | 233/563 [04:30<03:50,  1.43it/s]
 42%|████▏     | 234/563 [04:30<03:33,  1.54it/s]
                                                 
[2024-08-08 16:34:32] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00016-of-00037.safetensors

 42%|████▏     | 234/563 [04:30<03:33,  1.54it/s]
                                                 
[2024-08-08 16:34:33] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00017-of-00037.safetensors

 42%|████▏     | 234/563 [04:30<03:33,  1.54it/s]
                                                 
[2024-08-08 16:34:33] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00018-of-00037.safetensors

 42%|████▏     | 234/563 [04:30<03:33,  1.54it/s]
                                                 
[2024-08-08 16:34:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.36.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 42%|████▏     | 234/563 [04:33<03:33,  1.54it/s]
 42%|████▏     | 235/563 [04:37<11:15,  2.06s/it]
                                                 
[2024-08-08 16:34:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.input_layernorm.weight", shape: (8192,), dtype: float16

 42%|████▏     | 235/563 [04:37<11:15,  2.06s/it]
                                                 
[2024-08-08 16:34:40] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 42%|████▏     | 235/563 [04:37<11:15,  2.06s/it]
 42%|████▏     | 237/563 [04:38<08:34,  1.58s/it]
                                                 
[2024-08-08 16:34:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 42%|████▏     | 237/563 [04:40<08:34,  1.58s/it]
 42%|████▏     | 238/563 [04:42<10:42,  1.98s/it]
                                                 
[2024-08-08 16:34:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 42%|████▏     | 238/563 [04:42<10:42,  1.98s/it]
 42%|████▏     | 239/563 [04:42<08:15,  1.53s/it]
                                                 
[2024-08-08 16:34:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 42%|████▏     | 239/563 [04:42<08:15,  1.53s/it]
                                                 
[2024-08-08 16:34:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 42%|████▏     | 239/563 [04:42<08:15,  1.53s/it]
 43%|████▎     | 241/563 [04:42<05:26,  1.01s/it]
                                                 
[2024-08-08 16:34:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.37.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 43%|████▎     | 241/563 [04:42<05:26,  1.01s/it]
 43%|████▎     | 242/563 [04:43<04:41,  1.14it/s]
                                                 
[2024-08-08 16:34:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.input_layernorm.weight", shape: (8192,), dtype: float16

 43%|████▎     | 242/563 [04:43<04:41,  1.14it/s]
                                                 
[2024-08-08 16:34:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 43%|████▎     | 242/563 [04:43<04:41,  1.14it/s]
 43%|████▎     | 244/563 [04:44<04:24,  1.20it/s]
                                                 
[2024-08-08 16:34:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 43%|████▎     | 244/563 [04:46<04:24,  1.20it/s]
 44%|████▎     | 245/563 [04:48<07:15,  1.37s/it]
                                                 
[2024-08-08 16:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 44%|████▎     | 245/563 [04:48<07:15,  1.37s/it]
 44%|████▎     | 246/563 [04:48<05:39,  1.07s/it]
                                                 
[2024-08-08 16:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 44%|████▎     | 246/563 [04:48<05:39,  1.07s/it]
                                                 
[2024-08-08 16:34:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 44%|████▎     | 246/563 [04:48<05:39,  1.07s/it]
 44%|████▍     | 248/563 [04:48<03:54,  1.34it/s]
                                                 
[2024-08-08 16:34:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.38.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 44%|████▍     | 248/563 [04:48<03:54,  1.34it/s]
 44%|████▍     | 249/563 [04:49<03:29,  1.50it/s]
                                                 
[2024-08-08 16:34:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.input_layernorm.weight", shape: (8192,), dtype: float16

 44%|████▍     | 249/563 [04:49<03:29,  1.50it/s]
                                                 
[2024-08-08 16:34:51] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00018-of-00037.safetensors

 44%|████▍     | 249/563 [04:49<03:29,  1.50it/s]
                                                 
[2024-08-08 16:34:51] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00019-of-00037.safetensors

 44%|████▍     | 249/563 [04:49<03:29,  1.50it/s]
                                                 
[2024-08-08 16:34:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 44%|████▍     | 249/563 [04:51<03:29,  1.50it/s]
 45%|████▍     | 251/563 [04:52<05:36,  1.08s/it]
                                                 
[2024-08-08 16:34:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 45%|████▍     | 251/563 [04:53<05:36,  1.08s/it]
 45%|████▍     | 252/563 [04:55<08:06,  1.56s/it]
                                                 
[2024-08-08 16:34:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 45%|████▍     | 252/563 [04:55<08:06,  1.56s/it]
 45%|████▍     | 253/563 [04:55<06:18,  1.22s/it]
                                                 
[2024-08-08 16:34:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 45%|████▍     | 253/563 [04:55<06:18,  1.22s/it]
                                                 
[2024-08-08 16:34:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 45%|████▍     | 253/563 [04:56<06:18,  1.22s/it]
 45%|████▌     | 255/563 [04:56<04:17,  1.20it/s]
                                                 
[2024-08-08 16:34:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.39.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 45%|████▌     | 255/563 [04:56<04:17,  1.20it/s]
 45%|████▌     | 256/563 [04:56<03:46,  1.36it/s]
                                                 
[2024-08-08 16:34:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.input_layernorm.weight", shape: (8192,), dtype: float16

 45%|████▌     | 256/563 [04:56<03:46,  1.36it/s]
                                                 
[2024-08-08 16:34:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 45%|████▌     | 256/563 [04:57<03:46,  1.36it/s]
 46%|████▌     | 258/563 [04:58<03:45,  1.35it/s]
                                                 
[2024-08-08 16:35:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 46%|████▌     | 258/563 [04:59<03:45,  1.35it/s]
 46%|████▌     | 259/563 [05:01<06:33,  1.30s/it]
                                                 
[2024-08-08 16:35:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 46%|████▌     | 259/563 [05:01<06:33,  1.30s/it]
 46%|████▌     | 260/563 [05:01<05:07,  1.01s/it]
                                                 
[2024-08-08 16:35:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 46%|████▌     | 260/563 [05:01<05:07,  1.01s/it]
                                                 
[2024-08-08 16:35:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 46%|████▌     | 260/563 [05:01<05:07,  1.01s/it]
 47%|████▋     | 262/563 [05:02<03:34,  1.40it/s]
                                                 
[2024-08-08 16:35:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.40.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 47%|████▋     | 262/563 [05:02<03:34,  1.40it/s]
 47%|████▋     | 263/563 [05:02<03:12,  1.56it/s]
                                                 
[2024-08-08 16:35:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.input_layernorm.weight", shape: (8192,), dtype: float16

 47%|████▋     | 263/563 [05:02<03:12,  1.56it/s]
                                                 
[2024-08-08 16:35:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 47%|████▋     | 263/563 [05:02<03:12,  1.56it/s]
                                                 
[2024-08-08 16:35:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 47%|████▋     | 263/563 [05:02<03:12,  1.56it/s]
                                                 
[2024-08-08 16:35:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 47%|████▋     | 263/563 [05:02<03:12,  1.56it/s]
 47%|████▋     | 267/563 [05:03<01:45,  2.80it/s]
                                                 
[2024-08-08 16:35:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 47%|████▋     | 267/563 [05:03<01:45,  2.80it/s]
 48%|████▊     | 268/563 [05:03<01:47,  2.74it/s]
                                                 
[2024-08-08 16:35:06] INFO huggingface_loader.py:197: Unloading HF weight file: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00019-of-00037.safetensors

 48%|████▊     | 268/563 [05:03<01:47,  2.74it/s]
                                                 
[2024-08-08 16:35:06] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00020-of-00037.safetensors

 48%|████▊     | 268/563 [05:03<01:47,  2.74it/s]
                                                 
[2024-08-08 16:35:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 48%|████▊     | 268/563 [05:06<01:47,  2.74it/s]
 48%|████▊     | 269/563 [05:07<05:20,  1.09s/it]
                                                 
[2024-08-08 16:35:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.41.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 48%|████▊     | 269/563 [05:09<05:20,  1.09s/it]
 48%|████▊     | 270/563 [05:11<07:39,  1.57s/it]
                                                 
[2024-08-08 16:35:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.input_layernorm.weight", shape: (8192,), dtype: float16

 48%|████▊     | 270/563 [05:11<07:39,  1.57s/it]
 48%|████▊     | 271/563 [05:11<05:57,  1.22s/it]
                                                 
[2024-08-08 16:35:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 48%|████▊     | 271/563 [05:11<05:57,  1.22s/it]
 48%|████▊     | 272/563 [05:12<06:14,  1.29s/it]
                                                 
[2024-08-08 16:35:16] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 48%|████▊     | 272/563 [05:14<06:14,  1.29s/it]
 48%|████▊     | 273/563 [05:16<08:50,  1.83s/it]
                                                 
[2024-08-08 16:35:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 48%|████▊     | 273/563 [05:16<08:50,  1.83s/it]
 49%|████▊     | 274/563 [05:16<06:33,  1.36s/it]
                                                 
[2024-08-08 16:35:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 49%|████▊     | 274/563 [05:16<06:33,  1.36s/it]
                                                 
[2024-08-08 16:35:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 49%|████▊     | 274/563 [05:16<06:33,  1.36s/it]
 49%|████▉     | 276/563 [05:16<04:13,  1.13it/s]
                                                 
[2024-08-08 16:35:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.42.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 49%|████▉     | 276/563 [05:16<04:13,  1.13it/s]
 49%|████▉     | 277/563 [05:17<03:39,  1.30it/s]
                                                 
[2024-08-08 16:35:19] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.input_layernorm.weight", shape: (8192,), dtype: float16

 49%|████▉     | 277/563 [05:17<03:39,  1.30it/s]
                                                 
[2024-08-08 16:35:19] INFO huggingface_loader.py:185: Loading HF parameters from: /Users/Shared/models/Qwen2-Math-72B-Instruct/model-00021-of-00037.safetensors

 49%|████▉     | 277/563 [05:17<03:39,  1.30it/s]
                                                 
[2024-08-08 16:35:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 49%|████▉     | 277/563 [05:22<03:39,  1.30it/s]
 50%|████▉     | 279/563 [05:24<08:52,  1.88s/it]
                                                 
[2024-08-08 16:35:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.post_attention_layernorm.weight", shape: (8192,), dtype: float16

 50%|████▉     | 279/563 [05:24<08:52,  1.88s/it]
 50%|████▉     | 280/563 [05:24<06:58,  1.48s/it]
                                                 
[2024-08-08 16:35:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.self_attn.c_attn.bias", shape: (10240,), dtype: float16

 50%|████▉     | 280/563 [05:24<06:58,  1.48s/it]
                                                 
[2024-08-08 16:35:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.self_attn.c_attn.weight", shape: (10240, 8192), dtype: float16

 50%|████▉     | 280/563 [05:24<06:58,  1.48s/it]
 50%|█████     | 282/563 [05:24<04:43,  1.01s/it]
                                                 
[2024-08-08 16:35:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.self_attn.o_proj.weight", shape: (8192, 8192), dtype: float16

 50%|█████     | 282/563 [05:24<04:43,  1.01s/it]
 50%|█████     | 283/563 [05:25<04:05,  1.14it/s]
                                                 
[2024-08-08 16:35:28] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.43.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 50%|█████     | 283/563 [05:25<04:05,  1.14it/s]
 50%|█████     | 284/563 [05:26<04:48,  1.03s/it]
                                                 
[2024-08-08 16:35:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.44.input_layernorm.weight", shape: (8192,), dtype: float16

 50%|█████     | 284/563 [05:26<04:48,  1.03s/it]
                                                 
[2024-08-08 16:35:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.44.mlp.down_proj.weight", shape: (8192, 29568), dtype: float16

 50%|█████     | 284/563 [05:27<04:48,  1.03s/it]
 51%|█████     | 286/563 [05:28<04:21,  1.06it/s]
                                                 
[2024-08-08 16:35:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "model.layers.44.mlp.gate_up_proj.weight", shape: (59136, 8192), dtype: float16

 51%|█████     | 286/563 [05:29<04:21,  1.06it/s]Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/cfruan/Documents/mlc-llm/python/mlc_llm/__main__.py", line 64, in <module>
    main()
  File "/Users/cfruan/Documents/mlc-llm/python/mlc_llm/__main__.py", line 37, in main
    cli.main(sys.argv[2:])
  File "/Users/cfruan/Documents/mlc-llm/python/mlc_llm/cli/convert_weight.py", line 88, in main
    convert_weight(
  File "/Users/cfruan/Documents/mlc-llm/python/mlc_llm/interface/convert_weight.py", line 181, in convert_weight
    _convert_args(args)
  File "/Users/cfruan/Documents/mlc-llm/python/mlc_llm/interface/convert_weight.py", line 145, in _convert_args
    tvmjs.dump_ndarray_cache(
  File "/Users/cfruan/Documents/tvm/python/tvm/contrib/tvmjs.py", line 296, in dump_ndarray_cache
    shard_manager.append_or_update(
  File "/Users/cfruan/Documents/tvm/python/tvm/contrib/tvmjs.py", line 143, in append_or_update
    self._commit_internal(data, [rec])
  File "/Users/cfruan/Documents/tvm/python/tvm/contrib/tvmjs.py", line 184, in _commit_internal
    outfile.write(data)
OSError: [Errno 28] No space left on device

 51%|█████     | 286/563 [05:30<05:19,  1.15s/it]