2024-05-16 07:25:42,957 INFO StreamThr :50 [internal.py:wandb_internal():86] W&B internal server running at pid: 50, started at: 2024-05-16 07:25:42.957042 2024-05-16 07:25:42,960 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status 2024-05-16 07:25:42,965 INFO WriterThread:50 [datastore.py:open_for_write():87] open: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/run-5ru1r69k.wandb 2024-05-16 07:25:42,965 DEBUG SenderThread:50 [sender.py:send():379] send: header 2024-05-16 07:25:42,968 DEBUG SenderThread:50 [sender.py:send():379] send: run 2024-05-16 07:25:43,130 INFO SenderThread:50 [dir_watcher.py:__init__():211] watching files in: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files 2024-05-16 07:25:43,130 INFO SenderThread:50 [sender.py:_start_run_threads():1124] run started: 5ru1r69k with start time 1715844342.958331 2024-05-16 07:25:43,133 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: check_version 2024-05-16 07:25:43,133 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: check_version 2024-05-16 07:25:43,226 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: run_start 2024-05-16 07:25:43,237 DEBUG HandlerThread:50 [system_info.py:__init__():26] System info init 2024-05-16 07:25:43,237 DEBUG HandlerThread:50 [system_info.py:__init__():41] System info init done 2024-05-16 07:25:43,237 INFO HandlerThread:50 [system_monitor.py:start():194] Starting system monitor 2024-05-16 07:25:43,237 INFO SystemMonitor:50 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-05-16 07:25:43,238 INFO HandlerThread:50 [system_monitor.py:probe():214] Collecting system info 2024-05-16 07:25:43,238 INFO SystemMonitor:50 [interfaces.py:start():190] Started cpu monitoring 2024-05-16 07:25:43,239 INFO SystemMonitor:50 [interfaces.py:start():190] Started disk monitoring 2024-05-16 07:25:43,241 INFO SystemMonitor:50 [interfaces.py:start():190] Started gpu monitoring 2024-05-16 07:25:43,242 INFO SystemMonitor:50 [interfaces.py:start():190] Started memory monitoring 2024-05-16 07:25:43,242 INFO SystemMonitor:50 [interfaces.py:start():190] Started network monitoring 2024-05-16 07:25:43,262 DEBUG HandlerThread:50 [system_info.py:probe():150] Probing system 2024-05-16 07:25:43,264 DEBUG HandlerThread:50 [gitlib.py:_init_repo():56] git repository is invalid 2024-05-16 07:25:43,264 DEBUG HandlerThread:50 [system_info.py:probe():198] Probing system done 2024-05-16 07:25:43,264 DEBUG HandlerThread:50 [system_monitor.py:probe():223] {'os': 'Linux-5.15.133+-x86_64-with-glibc2.31', 'python': '3.10.13', 'heartbeatAt': '2024-05-16T07:25:43.262193', 'startedAt': '2024-05-16T07:25:42.951047', 'docker': None, 'cuda': None, 'args': ('-f', '/tmp/tmpqsb6llcs.json', '--HistoryManager.hist_file=:memory:'), 'state': 'running', 'program': '', 'codePathLocal': None, 'host': '3064694ee8b3', 'username': 'root', 'executable': '/opt/conda/bin/python', 'cpu_count': 2, 'cpu_count_logical': 4, 'cpu_freq': {'current': 2000.142, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.142, 'min': 0.0, 'max': 0.0}, {'current': 2000.142, 'min': 0.0, 'max': 0.0}, {'current': 2000.142, 'min': 0.0, 'max': 0.0}, {'current': 2000.142, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 8062.387607574463, 'used': 5611.894878387451}}, 'gpu': 'Tesla T4', 'gpu_count': 2, 'gpu_devices': [{'name': 'Tesla T4', 'memory_total': 16106127360}, {'name': 'Tesla T4', 'memory_total': 16106127360}], 'memory': {'total': 31.357559204101562}} 2024-05-16 07:25:43,264 INFO HandlerThread:50 [system_monitor.py:probe():224] Finished collecting system info 2024-05-16 07:25:43,264 INFO HandlerThread:50 [system_monitor.py:probe():227] Publishing system info 2024-05-16 07:25:43,264 DEBUG HandlerThread:50 [system_info.py:_save_conda():207] Saving list of conda packages installed into the current environment 2024-05-16 07:25:44,132 INFO Thread-12 :50 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/conda-environment.yaml 2024-05-16 07:25:58,283 ERROR HandlerThread:50 [system_info.py:_save_conda():221] Error saving conda packages: Command '['conda', 'env', 'export']' timed out after 15 seconds Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/internal/system/system_info.py", line 214, in _save_conda subprocess.call( File "/opt/conda/lib/python3.10/subprocess.py", line 347, in call return p.wait(timeout=timeout) File "/opt/conda/lib/python3.10/subprocess.py", line 1209, in wait return self._wait(timeout=timeout) File "/opt/conda/lib/python3.10/subprocess.py", line 1951, in _wait raise TimeoutExpired(self.args, timeout) subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after 15 seconds 2024-05-16 07:25:58,287 DEBUG HandlerThread:50 [system_info.py:_save_conda():222] Saving conda packages done 2024-05-16 07:25:58,287 INFO HandlerThread:50 [system_monitor.py:probe():229] Finished publishing system info 2024-05-16 07:25:58,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:25:58,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: keepalive 2024-05-16 07:25:58,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:25:58,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: keepalive 2024-05-16 07:25:58,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:25:58,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: keepalive 2024-05-16 07:25:58,298 DEBUG SenderThread:50 [sender.py:send():379] send: files 2024-05-16 07:25:58,298 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-metadata.json with policy now 2024-05-16 07:25:58,505 INFO wandb-upload_0:50 [upload_job.py:push():131] Uploaded file /tmp/tmpod8889fgwandb/277ej48c-wandb-metadata.json 2024-05-16 07:25:59,135 INFO Thread-12 :50 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-metadata.json 2024-05-16 07:25:59,292 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: python_packages 2024-05-16 07:25:59,292 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: python_packages 2024-05-16 07:25:59,295 DEBUG SenderThread:50 [sender.py:send():379] send: telemetry 2024-05-16 07:25:59,307 DEBUG SenderThread:50 [sender.py:send():379] send: config 2024-05-16 07:25:59,309 DEBUG SenderThread:50 [sender.py:send():379] send: metric 2024-05-16 07:25:59,309 DEBUG SenderThread:50 [sender.py:send():379] send: telemetry 2024-05-16 07:25:59,309 DEBUG SenderThread:50 [sender.py:send():379] send: metric 2024-05-16 07:25:59,309 WARNING SenderThread:50 [sender.py:send_metric():1341] Seen metric with glob (shouldn't happen) 2024-05-16 07:25:59,309 DEBUG SenderThread:50 [sender.py:send():379] send: telemetry 2024-05-16 07:25:59,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:25:59,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:25:59,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:26:00,135 INFO Thread-12 :50 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/requirements.txt 2024-05-16 07:26:01,136 INFO Thread-12 :50 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/output.log 2024-05-16 07:26:03,136 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/output.log 2024-05-16 07:26:03,238 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:08,239 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:13,245 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:14,141 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/config.yaml 2024-05-16 07:26:14,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:26:14,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:26:14,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:26:18,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:23,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:28,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:29,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:26:29,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:26:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:26:33,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:38,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:43,243 DEBUG SystemMonitor:50 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-05-16 07:26:43,244 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:26:44,245 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:44,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:26:44,293 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:26:44,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:26:49,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:54,430 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:26:59,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:26:59,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:26:59,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:27:00,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:05,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:10,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:13,246 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:27:14,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:27:14,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:27:14,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:27:16,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:21,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:26,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:29,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:27:29,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:27:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:27:31,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:36,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:41,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:43,246 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:27:44,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:27:44,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:27:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:27:46,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:48,731 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:27:48,733 DEBUG SenderThread:50 [sender.py:send():379] send: metric 2024-05-16 07:27:48,733 DEBUG SenderThread:50 [sender.py:send():379] send: metric 2024-05-16 07:27:48,733 DEBUG SenderThread:50 [sender.py:send():379] send: metric 2024-05-16 07:27:48,734 DEBUG SenderThread:50 [sender.py:send():379] send: metric 2024-05-16 07:27:48,734 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:27:48,734 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:27:48,735 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:27:49,178 INFO Thread-12 :50 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:27:51,735 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:56,737 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:27:59,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:27:59,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:27:59,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:28:02,383 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:07,384 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:12,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:13,248 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:28:14,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:28:14,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:28:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:28:17,451 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:18,188 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/config.yaml 2024-05-16 07:28:22,564 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:27,565 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:29,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:28:29,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:28:29,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:28:33,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:38,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:43,249 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:28:44,249 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:44,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:28:44,294 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:28:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:28:49,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:54,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:28:59,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:28:59,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:28:59,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:28:59,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:04,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:09,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:13,250 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:29:14,294 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:29:14,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:29:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:29:15,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:20,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:25,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:29,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:29:29,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:29:29,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:29:30,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:35,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:40,380 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:43,251 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:29:44,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:29:44,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:29:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:29:45,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:45,937 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:29:45,938 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:29:45,938 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:29:45,939 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:29:46,223 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:29:50,940 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:55,941 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:29:59,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:29:59,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:29:59,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:30:01,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:06,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:11,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:13,251 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:30:14,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:30:14,296 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:30:14,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:30:16,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:21,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:26,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:29,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:30:29,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:30:29,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:30:32,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:37,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:42,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:43,252 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:30:44,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:30:44,296 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:30:44,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:30:48,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:53,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:58,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:30:59,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:30:59,296 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:30:59,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:31:03,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:08,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:13,253 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:31:14,254 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:14,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:31:14,296 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:31:14,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:31:19,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:24,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:29,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:31:29,296 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:31:29,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:31:29,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:34,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:39,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:41,486 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:31:41,487 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:31:41,487 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:31:41,488 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:31:42,271 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:31:43,254 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:31:44,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:31:44,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:31:44,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:31:45,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:50,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:55,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:31:59,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:31:59,296 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:31:59,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:32:01,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:06,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:11,364 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:13,255 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:32:14,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:32:14,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:32:14,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:32:16,382 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:21,383 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:26,384 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:29,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:32:29,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:32:29,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:32:32,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:37,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:42,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:43,256 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:32:44,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:32:44,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:32:44,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:32:47,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:52,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:57,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:32:59,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:32:59,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:32:59,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:33:02,383 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:07,384 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:12,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:13,257 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:33:14,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:33:14,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:33:14,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:33:17,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:22,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:27,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:29,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:33:29,297 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:33:29,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:33:33,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:38,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:33:38,309 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:33:38,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:33:38,309 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:33:38,315 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:33:39,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:43,258 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:33:44,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:33:44,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:33:44,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:33:44,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:49,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:54,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:33:59,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:33:59,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:33:59,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:34:01,257 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:06,257 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:11,258 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:13,259 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:34:14,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:34:14,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:34:14,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:34:16,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:21,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:26,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:29,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:34:29,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:34:29,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:34:32,353 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:37,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:42,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:43,260 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:34:44,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:34:44,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:34:44,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:34:47,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:52,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:57,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:34:59,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:34:59,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:34:59,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:35:03,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:08,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:13,261 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:35:14,262 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:14,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:35:14,298 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:35:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:35:19,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:24,389 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:29,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:35:29,299 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:35:29,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:35:29,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:34,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:38,766 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:35:38,768 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:35:38,768 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:35:38,768 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:35:39,360 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:35:39,769 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:43,262 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:35:44,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:35:44,299 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:35:44,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:35:45,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:50,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:55,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:35:59,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:35:59,299 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:35:59,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:36:01,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:06,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:11,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:13,263 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:36:14,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:36:14,299 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:36:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:36:16,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:21,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:26,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:29,298 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:36:29,299 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:36:29,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:36:32,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:37,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:42,373 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:43,264 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:36:44,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:36:44,299 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:36:44,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:36:47,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:52,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:57,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:36:59,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:36:59,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:36:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:37:02,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:07,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:12,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:13,265 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:37:14,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:37:14,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:37:14,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:37:18,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:23,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:28,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:29,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:37:29,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:37:29,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:37:33,447 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:34,876 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:37:34,877 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:37:34,878 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:37:34,878 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:37:35,402 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:37:38,879 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:43,266 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:37:44,267 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:44,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:37:44,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:37:44,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:37:49,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:54,359 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:37:59,299 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:37:59,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:37:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:37:59,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:04,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:09,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:13,272 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:38:14,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:38:14,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:38:14,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:38:15,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:20,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:25,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:29,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:38:29,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:38:29,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:38:31,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:36,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:41,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:43,268 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:38:44,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:38:44,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:38:44,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:38:47,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:52,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:57,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:38:59,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:38:59,301 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:38:59,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:39:02,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:07,414 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:12,415 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:13,271 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:39:14,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:39:14,300 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:39:14,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:39:18,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:23,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:28,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:29,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:39:29,301 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:39:29,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:39:33,241 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:39:33,243 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:39:33,243 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:39:33,243 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:39:33,445 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:39:34,244 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:39,245 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:43,271 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:39:44,271 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:44,300 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:39:44,301 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:39:44,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:39:49,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:54,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:39:59,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:39:59,301 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:39:59,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:40:00,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:05,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:10,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:13,271 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:40:14,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:40:14,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:40:14,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:40:15,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:20,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:25,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:29,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:40:29,301 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:40:29,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:40:30,430 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:35,431 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:40,431 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:43,273 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:40:44,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:40:44,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:40:44,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:40:46,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:51,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:56,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:40:59,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:40:59,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:40:59,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:41:01,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:06,430 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:11,431 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:13,274 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:41:14,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:41:14,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:41:14,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:41:17,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:22,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:27,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:28,229 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:41:28,230 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:41:28,230 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:41:28,231 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:41:28,491 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:41:29,301 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:41:29,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:41:29,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:41:32,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:37,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:42,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:43,274 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:41:44,302 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:41:44,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:41:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:41:48,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:53,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:58,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:41:59,302 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:41:59,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:41:59,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:42:03,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:08,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:13,275 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:42:14,277 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:14,302 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:42:14,302 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:42:14,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:42:19,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:24,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:29,302 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:42:29,303 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:42:29,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:42:29,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:34,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:39,411 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:43,276 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:42:44,302 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:42:44,303 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:42:44,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:42:44,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:49,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:54,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:42:59,302 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:42:59,303 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:42:59,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:43:00,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:05,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:10,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:13,277 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:43:14,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:43:14,303 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:43:14,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:43:16,375 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:21,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:24,818 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:43:24,819 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:43:24,820 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:43:24,820 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:43:25,538 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:43:26,821 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:29,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:43:29,303 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:43:29,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:43:32,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:37,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:42,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:43,278 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:43:44,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:43:44,303 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:43:44,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:43:47,365 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:52,366 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:57,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:43:59,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:43:59,304 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:43:59,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:44:02,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:07,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:12,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:13,279 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:44:14,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:44:14,304 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:44:14,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:44:17,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:22,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:27,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:29,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:44:29,304 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:44:29,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:44:33,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:38,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:43,279 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:44:44,280 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:44,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:44:44,304 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:44:44,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:44:49,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:54,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:44:59,303 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:44:59,304 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:44:59,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:44:59,441 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:04,442 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:09,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:13,280 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:45:14,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:45:14,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:45:14,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:45:15,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:20,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:23,853 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:45:23,854 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:45:23,854 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:45:23,854 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:45:24,584 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:45:25,855 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:29,304 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:45:29,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:45:29,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:45:31,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:36,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:41,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:43,281 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:45:44,304 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:45:44,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:45:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:45:47,357 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:52,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:57,359 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:45:59,304 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:45:59,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:45:59,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:46:02,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:07,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:12,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:13,282 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:46:14,304 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:46:14,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:46:14,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:46:18,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:23,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:28,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:29,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:46:29,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:46:29,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:46:33,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:38,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:43,283 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:46:44,284 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:44,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:46:44,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:46:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:46:49,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:54,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:46:59,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:46:59,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:46:59,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:46:59,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:04,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:09,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:13,284 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:47:14,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:47:14,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:47:14,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:47:14,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:18,205 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:47:18,206 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:47:18,206 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:47:18,207 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:47:18,632 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:47:20,208 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:25,209 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:29,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:47:29,305 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:47:29,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:47:30,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:35,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:40,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:43,285 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:47:44,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:47:44,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:47:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:47:45,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:50,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:55,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:47:59,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:47:59,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:47:59,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:48:01,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:06,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:11,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:13,286 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:48:14,305 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:48:14,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:48:14,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:48:17,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:22,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:27,356 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:29,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:48:29,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:48:29,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:48:32,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:37,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:42,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:43,287 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:48:44,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:48:44,306 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:48:44,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:48:47,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:52,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:57,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:48:59,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:48:59,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:48:59,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:49:03,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:08,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:13,288 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:49:14,289 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:14,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:49:14,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:49:14,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:49:15,601 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:49:15,603 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:49:15,603 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:49:15,603 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:49:15,679 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:49:19,604 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:24,605 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:29,306 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:49:29,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:49:29,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:49:30,375 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:35,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:40,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:43,288 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:49:44,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:49:44,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:49:44,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:49:46,365 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:51,366 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:56,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:49:59,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:49:59,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:49:59,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:50:02,360 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:07,361 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:12,361 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:13,289 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:50:14,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:50:14,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:50:14,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:50:17,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:22,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:27,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:29,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:50:29,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:50:29,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:50:33,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:38,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:43,291 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:50:44,291 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:44,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:50:44,307 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:50:44,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:50:49,357 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:54,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:50:59,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:50:59,308 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:50:59,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:50:59,364 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:04,365 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:09,366 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:09,825 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:51:09,830 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:51:09,830 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:51:09,831 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:51:10,726 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:51:13,291 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:51:14,307 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:51:14,308 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:51:14,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:51:14,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:19,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:24,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:29,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:51:29,308 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:51:29,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:51:29,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:34,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:39,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:43,292 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:51:44,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:51:44,308 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:51:44,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:51:45,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:50,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:55,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:51:59,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:51:59,308 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:51:59,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:52:00,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:05,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:10,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:13,293 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:52:14,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:52:14,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:52:14,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:52:15,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:20,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:25,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:29,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:52:29,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:52:29,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:52:31,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:36,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:41,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:43,294 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:52:44,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:52:44,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:52:44,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:52:47,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:52,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:57,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:52:59,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:52:59,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:52:59,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:53:03,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:08,067 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:53:08,068 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:53:08,068 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:53:08,069 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:53:08,767 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:53:09,069 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:13,295 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:53:14,295 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:14,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:53:14,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:53:14,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:53:19,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:24,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:29,309 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:53:29,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:53:29,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:53:30,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:35,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:40,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:43,295 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:53:44,309 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:53:44,309 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:53:44,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:53:46,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:51,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:56,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:53:59,309 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:53:59,310 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:53:59,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:54:01,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:06,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:11,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:13,296 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:54:14,309 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:54:14,310 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:54:14,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:54:16,442 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:21,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:26,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:29,309 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:54:29,310 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:54:29,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:54:32,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:37,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:42,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:43,297 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:54:44,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:54:44,310 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:54:44,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:54:48,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:53,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:58,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:54:59,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:54:59,310 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:54:59,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:55:03,418 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:05,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:55:05,378 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:55:05,378 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:55:05,379 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:55:05,809 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:55:09,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:13,298 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:55:14,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:55:14,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:55:14,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:55:14,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:19,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:24,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:29,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:55:29,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:55:29,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:55:29,431 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:34,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:39,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:43,299 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:55:44,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:55:44,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:55:44,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:55:45,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:50,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:55,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:55:59,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:55:59,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:55:59,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:56:01,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:06,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:11,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:13,304 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:56:14,310 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:56:14,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:56:14,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:56:16,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:21,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:26,364 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:29,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:56:29,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:56:29,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:56:32,353 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:37,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:42,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:43,300 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:56:44,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:56:44,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:56:44,351 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:56:47,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:52,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:57,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:56:59,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:56:59,311 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:56:59,352 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:57:01,616 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:57:01,618 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:57:01,618 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:57:01,620 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:57:01,853 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:57:02,621 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:07,622 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:12,623 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:13,301 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:57:14,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:57:14,312 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:57:14,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:57:18,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:23,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:28,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:29,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:57:29,312 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:57:29,352 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:57:34,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:39,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:43,304 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:57:44,311 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:57:44,312 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:57:44,352 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:57:44,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:49,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:54,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:57:59,312 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:57:59,312 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:57:59,352 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:58:00,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:05,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:10,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:13,304 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:58:14,312 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:58:14,312 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:58:14,352 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:58:16,361 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:21,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:26,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:29,312 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:58:29,312 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:58:29,353 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:58:31,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:36,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:41,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:43,305 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:58:44,312 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:58:44,313 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:58:44,353 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:58:46,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:51,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:56,414 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:58:56,975 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 07:58:56,976 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 07:58:56,976 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 07:58:56,977 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 07:58:57,899 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 07:58:59,312 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:58:59,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:58:59,316 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:59:02,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:07,380 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:12,381 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:13,306 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:59:14,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:59:14,313 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:59:14,353 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:59:17,442 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:22,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:27,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:29,312 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:59:29,313 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:59:29,353 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:59:33,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:38,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:43,307 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 07:59:44,308 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:44,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:59:44,313 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:59:44,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:59:49,357 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:54,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 07:59:59,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 07:59:59,313 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 07:59:59,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 07:59:59,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:04,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:09,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:13,308 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:00:14,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:00:14,313 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:00:14,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:00:15,361 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:20,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:25,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:29,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:00:29,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:00:29,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:00:30,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:35,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:40,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:43,309 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:00:44,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:00:44,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:00:44,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:00:46,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:51,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:52,497 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:00:52,499 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:00:52,499 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:00:52,499 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:00:52,938 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:00:56,500 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:00:59,313 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:00:59,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:00:59,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:01:02,451 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:07,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:12,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:13,310 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:01:14,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:01:14,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:01:14,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:01:18,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:23,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:28,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:29,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:01:29,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:01:29,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:01:34,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:39,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:43,311 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:01:44,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:01:44,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:01:44,354 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:01:45,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:50,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:55,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:01:59,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:01:59,314 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:01:59,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:02:00,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:05,423 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:10,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:13,312 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:02:14,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:02:14,315 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:02:14,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:02:16,419 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:21,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:26,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:29,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:02:29,315 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:02:29,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:02:31,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:36,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:41,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:43,313 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:02:44,314 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:02:44,315 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:02:44,355 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:02:45,822 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:02:45,823 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:02:45,824 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:02:45,824 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:02:45,979 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:02:46,825 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:51,826 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:56,826 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:02:59,315 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:02:59,315 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:02:59,316 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:03:02,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:07,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:12,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:13,314 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:03:14,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:03:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:03:14,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:03:18,360 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:23,361 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:28,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:29,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:03:29,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:03:29,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:03:33,445 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:38,446 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:43,315 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:03:44,316 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:44,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:03:44,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:03:44,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:03:49,390 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:54,391 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:03:59,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:03:59,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:03:59,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:03:59,450 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:04,451 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:09,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:13,316 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:04:14,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:04:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:04:14,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:04:14,462 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:19,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:24,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:29,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:04:29,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:04:29,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:04:30,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:35,359 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:40,360 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:41,445 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:04:41,446 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:04:41,447 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:04:41,447 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:04:42,019 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:04:43,317 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:04:44,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:04:44,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:04:44,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:04:45,374 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:50,375 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:55,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:04:59,319 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:04:59,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:04:59,320 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:05:01,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:06,359 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:11,360 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:13,318 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:05:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:05:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:05:14,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:05:17,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:22,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:27,350 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:29,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:05:29,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:05:29,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:05:32,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:37,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:42,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:43,319 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:05:44,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:05:44,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:05:44,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:05:47,431 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:52,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:57,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:05:59,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:05:59,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:05:59,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:06:03,383 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:08,384 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:13,320 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:06:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:06:14,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:06:14,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:06:19,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:24,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:29,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:06:29,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:06:29,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:06:29,441 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:34,442 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:34,521 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:06:34,522 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:06:34,522 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:06:34,523 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:06:35,063 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:06:39,524 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:43,321 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:06:44,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:06:44,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:06:44,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:06:45,414 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:50,415 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:55,416 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:06:59,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:06:59,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:06:59,321 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:07:00,417 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:05,418 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:10,419 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:13,322 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:07:14,320 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:07:14,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:07:14,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:07:16,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:21,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:26,373 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:29,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:07:29,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:07:29,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:07:31,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:36,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:41,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:43,322 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:07:44,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:07:44,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:07:44,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:07:46,416 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:51,417 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:56,418 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:07:59,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:07:59,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:07:59,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:08:01,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:06,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:11,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:13,324 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:08:14,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:08:14,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:08:14,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:08:17,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:22,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:27,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:29,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:08:29,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:08:29,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:08:32,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:36,293 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:08:36,295 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:08:36,295 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:08:36,295 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:08:37,111 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:08:38,296 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:43,297 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:43,325 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:08:44,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:08:44,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:08:44,322 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:08:48,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:53,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:58,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:08:59,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:08:59,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:08:59,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:09:04,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:09,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:13,326 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:09:14,321 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:09:14,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:09:14,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:09:15,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:20,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:25,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:29,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:09:29,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:09:29,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:09:30,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:35,459 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:40,460 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:43,328 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:09:44,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:09:44,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:09:44,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:09:46,356 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:51,357 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:56,358 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:09:59,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:09:59,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:09:59,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:10:01,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:06,454 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:11,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:13,329 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:10:14,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:10:14,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:10:14,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:10:17,374 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:22,375 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:27,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:29,322 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:10:29,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:10:29,323 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:10:29,926 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:10:29,927 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:10:29,927 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:10:29,928 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:10:30,154 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:10:32,929 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:37,930 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:42,931 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:43,330 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:10:44,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:10:44,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:10:44,324 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:10:48,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:53,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:58,441 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:10:59,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:10:59,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:10:59,324 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:11:03,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:08,445 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:13,330 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:11:14,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:14,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:11:14,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:11:14,324 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:11:19,391 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:24,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:29,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:11:29,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:11:29,324 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:11:29,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:34,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:39,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:43,332 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:11:44,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:11:44,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:11:44,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:11:44,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:49,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:54,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:11:59,323 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:11:59,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:11:59,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:11:59,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:04,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:09,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:13,333 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:12:14,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:12:14,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:12:14,324 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:12:15,374 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:20,375 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:25,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:26,829 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:12:26,831 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:12:26,831 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:12:26,832 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:12:27,199 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:12:29,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:12:29,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:12:29,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:12:30,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:35,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:40,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:43,334 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:12:44,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:12:44,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:12:44,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:12:45,459 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:50,460 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:55,461 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:12:59,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:12:59,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:12:59,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:13:01,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:06,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:11,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:13,335 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:13:14,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:13:14,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:13:14,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:13:16,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:21,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:26,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:29,324 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:13:29,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:13:29,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:13:31,447 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:36,447 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:41,448 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:43,340 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:13:44,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:13:44,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:13:44,325 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:13:47,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:52,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:57,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:13:59,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:13:59,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:13:59,326 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:14:02,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:07,468 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:12,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:13,337 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:14:14,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:14:14,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:14:14,326 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:14:18,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:23,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:24,278 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:14:24,280 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:14:24,280 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:14:24,282 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:14:25,242 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:14:29,284 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:29,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:14:29,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:14:29,326 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:14:34,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:39,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:43,338 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:14:44,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:14:44,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:14:44,326 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:14:44,383 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:49,384 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:54,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:14:59,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:14:59,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:14:59,326 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:14:59,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:04,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:09,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:13,340 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:15:14,325 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:15:14,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:15:14,326 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:15:14,445 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:19,446 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:24,447 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:29,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:15:29,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:15:29,327 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:15:29,468 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:34,469 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:39,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:43,340 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:15:44,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:15:44,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:15:44,327 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:15:45,361 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:50,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:55,363 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:15:59,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:15:59,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:15:59,327 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:16:00,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:05,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:10,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:13,340 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:16:14,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:16:14,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:16:14,327 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:16:15,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:19,490 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:16:19,491 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:16:19,492 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:16:19,492 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:16:20,286 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:16:20,493 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:25,494 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:29,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:16:29,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:16:29,327 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:16:31,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:36,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:41,414 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:43,341 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:16:44,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:16:44,328 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:16:44,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:16:47,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:52,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:57,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:16:59,326 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:16:59,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:16:59,327 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:17:03,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:08,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:13,342 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:17:14,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:14,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:17:14,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:17:14,328 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:17:19,454 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:24,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:29,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:17:29,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:17:29,328 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:17:29,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:34,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:39,465 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:43,343 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:17:44,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:17:44,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:17:44,328 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:17:45,366 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:50,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:55,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:17:59,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:17:59,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:17:59,328 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:18:00,451 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:05,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:10,454 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:13,345 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:18:14,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:18:14,328 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:18:14,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:18:16,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:19,275 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:18:19,277 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:18:19,277 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:18:19,277 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:18:19,339 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:18:22,278 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:27,279 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:29,327 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:18:29,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:18:29,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:18:32,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:37,380 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:42,381 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:43,346 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:18:44,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:18:44,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:18:44,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:18:47,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:52,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:57,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:18:59,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:18:59,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:18:59,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:19:02,466 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:07,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:12,468 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:13,348 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:19:14,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:19:14,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:19:14,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:19:18,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:23,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:28,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:29,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:19:29,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:19:29,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:19:33,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:38,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:43,348 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:19:44,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:44,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:19:44,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:19:44,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:19:49,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:54,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:19:59,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:19:59,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:19:59,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:20:00,385 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:05,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:10,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:13,349 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:20:14,010 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:20:14,011 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:20:14,011 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:20:14,012 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:20:14,328 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:20:14,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:20:14,329 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:20:14,391 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:20:15,465 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:20,466 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:25,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:29,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:20:29,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:20:29,330 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:20:31,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:36,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:41,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:43,351 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:20:44,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:20:44,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:20:44,330 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:20:47,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:52,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:57,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:20:59,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:20:59,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:20:59,330 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:21:02,466 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:07,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:12,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:13,352 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:21:14,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:21:14,330 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:21:14,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:21:18,419 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:23,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:28,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:29,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:21:29,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:21:29,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:21:33,469 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:38,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:43,354 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:21:44,329 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:44,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:21:44,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:21:44,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:21:49,394 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:54,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:21:59,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:21:59,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:21:59,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:22:00,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:05,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:09,267 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:22:09,268 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:22:09,269 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:22:09,269 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:22:09,431 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:22:11,270 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:13,355 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:22:14,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:22:14,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:22:14,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:22:16,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:21,415 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:26,416 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:29,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:22:29,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:22:29,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:22:31,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:36,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:41,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:43,355 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:22:44,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:22:44,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:22:44,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:22:47,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:52,389 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:57,390 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:22:59,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:22:59,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:22:59,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:23:02,391 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:07,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:12,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:13,356 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:23:14,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:23:14,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:23:14,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:23:17,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:22,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:27,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:29,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:23:29,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:23:29,331 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:23:33,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:38,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:43,358 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:23:44,330 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:44,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:23:44,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:23:44,332 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:23:49,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:54,380 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:23:59,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:23:59,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:23:59,332 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:23:59,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:04,441 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:06,737 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:24:06,738 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:24:06,739 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:24:06,739 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:24:07,472 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:24:09,740 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:13,358 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:24:14,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:24:14,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:24:14,332 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:24:15,360 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:20,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:25,362 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:29,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:24:29,332 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:24:29,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:24:30,457 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:35,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:40,459 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:43,360 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:24:44,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:24:44,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:24:44,332 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:24:46,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:51,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:56,430 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:24:59,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:24:59,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:24:59,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:25:01,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:06,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:11,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:13,361 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:25:14,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:25:14,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:25:14,332 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:25:16,468 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:21,469 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:26,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:29,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:25:29,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:25:29,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:25:32,449 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:37,450 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:42,451 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:43,361 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:25:44,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:25:44,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:25:44,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:25:48,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:53,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:58,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:25:59,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:25:59,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:25:59,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:26:03,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:04,945 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:26:04,946 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:26:04,947 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:26:04,947 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:26:05,526 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:26:08,948 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:13,362 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:26:14,331 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:26:14,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:26:14,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:26:14,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:19,381 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:24,382 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:29,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:26:29,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:26:29,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:26:29,395 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:34,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:39,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:43,363 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:26:44,332 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:26:44,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:26:44,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:26:44,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:49,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:54,414 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:26:59,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:26:59,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:26:59,333 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:27:00,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:05,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:10,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:13,365 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:27:14,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:27:14,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:27:14,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:27:15,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:20,456 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:25,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:29,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:27:29,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:27:29,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:27:31,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:36,456 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:41,457 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:43,366 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:27:44,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:27:44,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:27:44,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:27:47,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:52,456 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:57,457 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:27:58,197 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:27:58,198 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:27:58,199 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:27:58,199 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:27:58,571 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:27:59,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:27:59,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:27:59,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:28:03,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:08,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:13,367 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:28:14,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:14,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:28:14,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:28:14,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:28:19,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:24,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:29,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:28:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:28:29,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:28:29,477 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:34,478 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:39,479 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:43,368 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:28:44,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:28:44,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:28:44,334 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:28:45,448 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:50,449 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:55,450 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:28:59,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:28:59,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:28:59,335 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:29:01,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:06,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:11,389 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:13,369 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:29:14,333 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:29:14,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:29:14,335 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:29:16,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:21,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:26,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:29:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:29:29,335 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:29:32,390 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:37,391 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:42,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:43,370 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:29:44,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:29:44,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:29:44,335 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:29:47,396 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:52,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:52,946 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:29:52,948 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:29:52,948 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:29:52,948 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:29:53,616 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:29:57,949 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:29:59,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:29:59,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:29:59,335 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:30:03,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:08,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:13,371 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:30:14,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:30:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:30:14,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:30:19,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:24,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:30:29,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:30:29,335 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:30:29,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:34,459 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:39,460 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:43,371 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:30:44,334 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:30:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:30:44,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:30:44,469 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:49,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:54,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:30:59,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:30:59,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:30:59,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:31:00,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:05,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:10,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:13,373 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:31:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:31:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:31:14,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:31:15,469 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:20,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:25,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:29,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:31:29,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:31:29,376 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:31:31,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:36,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:41,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:43,377 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:31:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:31:44,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:31:44,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:31:46,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:47,627 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:31:47,628 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:31:47,628 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:31:47,628 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:31:47,660 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:31:51,629 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:56,630 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:31:59,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:31:59,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:31:59,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:32:02,427 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:07,428 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:12,429 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:13,374 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:32:14,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:32:14,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:32:14,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:32:18,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:23,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:28,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:29,335 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:32:29,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:32:29,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:32:34,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:39,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:43,375 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:32:44,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:32:44,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:32:44,336 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:32:44,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:49,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:54,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:32:59,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:32:59,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:32:59,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:32:59,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:04,459 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:09,460 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:13,376 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:33:14,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:33:14,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:33:14,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:33:15,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:20,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:25,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:29,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:33:29,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:33:29,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:33:30,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:35,456 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:40,457 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:43,377 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:33:44,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:33:44,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:33:44,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:33:44,544 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:33:44,545 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:33:44,545 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:33:44,546 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:33:44,702 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:33:45,547 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:50,547 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:55,548 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:33:59,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:33:59,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:33:59,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:34:01,457 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:06,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:11,459 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:13,378 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:34:14,336 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:34:14,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:34:14,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:34:16,462 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:21,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:26,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:29,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:34:29,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:34:29,338 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:34:32,431 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:37,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:42,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:43,379 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:34:44,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:34:44,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:34:44,337 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:34:47,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:52,472 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:57,473 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:34:59,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:34:59,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:34:59,338 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:35:03,375 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:08,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:13,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:13,379 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:35:14,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:35:14,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:35:14,338 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:35:18,435 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:23,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:28,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:29,337 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:35:29,338 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:35:29,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:35:34,368 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:39,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:43,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:35:43,381 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:35:43,381 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:35:43,384 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:35:43,384 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:35:43,743 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:35:44,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:35:44,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:35:44,339 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:35:44,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:49,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:54,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:35:59,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:35:59,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:35:59,380 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:35:59,477 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:04,478 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:09,479 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:13,382 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:36:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:36:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:36:14,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:36:15,366 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:20,366 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:25,367 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:29,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:36:29,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:36:29,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:36:30,473 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:35,474 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:40,475 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:43,383 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:36:44,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:36:44,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:36:44,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:36:46,461 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:51,462 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:56,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:36:59,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:36:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:36:59,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:37:02,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:07,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:12,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:13,384 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:37:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:37:14,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:37:14,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:37:17,461 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:22,462 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:27,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:29,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:37:29,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:37:29,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:37:33,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:38,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:37:38,423 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:37:38,423 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:37:38,424 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:37:38,789 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:37:39,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:43,385 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:37:44,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:37:44,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:37:44,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:37:45,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:50,425 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:55,426 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:37:59,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:37:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:37:59,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:38:01,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:06,405 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:11,406 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:13,386 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:38:14,338 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:38:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:38:14,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:38:16,414 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:21,415 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:26,416 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:29,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:38:29,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:38:29,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:38:32,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:37,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:42,409 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:43,387 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:38:44,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:38:44,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:38:44,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:38:47,451 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:52,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:57,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:38:59,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:38:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:38:59,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:39:02,488 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:07,490 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:12,491 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:13,388 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:39:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:39:14,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:39:14,340 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:39:18,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:23,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:28,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:29,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:39:29,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:39:29,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:39:33,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:38,315 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:39:38,316 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:39:38,316 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:39:38,317 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:39:38,834 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:39:39,317 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:43,389 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:39:44,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:39:44,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:44,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:39:44,342 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:39:49,407 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:54,408 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:39:59,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:39:59,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:39:59,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:40:00,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:05,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:10,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:13,391 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:40:14,339 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:40:14,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:40:14,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:40:15,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:20,436 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:25,437 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:29,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:40:29,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:40:29,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:40:30,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:35,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:40,454 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:43,391 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:40:44,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:40:44,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:40:44,341 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:40:46,369 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:51,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:56,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:40:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:40:59,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:40:59,342 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:41:01,410 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:06,411 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:11,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:13,392 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:41:14,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:41:14,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:41:14,342 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:41:16,465 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:21,466 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:26,467 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:29,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:41:29,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:41:29,342 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:41:32,424 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:34,771 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:41:34,773 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:41:34,773 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:41:34,773 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:41:34,874 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:41:37,774 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:42,775 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:43,394 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:41:44,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:41:44,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:41:44,342 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:41:48,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:53,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:58,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:41:59,340 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:41:59,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:41:59,342 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:42:03,460 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:08,461 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:13,395 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:42:14,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:42:14,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:14,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:42:14,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:42:19,477 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:24,478 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:29,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:42:29,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:42:29,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:42:30,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:35,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:40,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:43,395 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:42:44,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:42:44,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:42:44,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:42:45,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:50,472 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:55,473 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:42:59,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:42:59,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:42:59,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:43:00,475 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:05,475 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:10,476 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:13,396 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:43:14,341 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:43:14,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:43:14,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:43:16,417 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:21,418 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:26,419 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:29,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:43:29,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:43:29,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:43:32,415 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:35,822 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:43:35,823 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:43:35,824 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:43:35,824 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:43:35,917 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:43:37,825 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:42,826 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:43,397 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:43:44,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:43:44,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:43:44,343 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:43:48,442 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:53,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:58,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:43:59,342 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:43:59,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:43:59,344 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:44:04,397 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:09,398 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:13,398 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:44:14,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:44:14,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:44:14,344 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:44:14,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:19,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:24,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:29,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:44:29,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:44:29,344 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:44:29,449 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:34,450 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:39,450 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:43,399 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:44:44,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:44:44,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:44:44,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:44:45,390 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:50,391 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:55,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:44:59,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:44:59,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:44:59,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:45:00,454 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:05,455 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:10,456 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:13,400 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:45:14,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:45:14,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:45:14,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:45:16,380 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:21,381 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:26,382 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:29,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:45:29,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:45:29,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:45:31,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:36,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:37,035 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:45:37,036 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:45:37,037 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:45:37,037 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:45:37,966 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:45:42,038 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:43,401 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:45:44,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:45:44,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:45:44,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:45:47,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:52,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:57,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:45:59,343 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:45:59,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:45:59,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:46:03,388 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:08,389 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:13,390 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:13,402 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:46:14,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:46:14,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:46:14,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:46:18,465 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:23,465 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:28,466 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:29,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:46:29,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:46:29,345 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:46:33,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:38,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:43,403 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:46:44,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:46:44,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:46:44,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:46:44,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:49,457 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:54,458 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:46:59,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:46:59,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:46:59,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:47:00,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:05,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:10,379 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:13,404 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:47:14,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:47:14,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:47:14,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:47:15,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:20,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:25,441 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:29,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:47:29,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:47:29,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:47:30,468 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:35,469 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:39,863 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:47:39,865 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:47:39,865 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:47:39,865 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:47:40,012 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:47:40,866 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:43,406 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:47:44,344 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:47:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:47:44,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:47:46,370 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:51,371 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:56,372 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:47:59,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:47:59,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:47:59,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:48:01,438 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:06,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:11,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:13,407 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:48:14,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:48:14,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:48:14,346 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:48:16,452 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:21,453 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:26,454 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:29,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:48:29,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:48:29,347 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:48:32,439 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:37,440 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:42,441 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:43,408 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:48:44,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:48:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:48:44,347 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:48:48,411 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:53,412 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:58,413 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:48:59,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:48:59,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:48:59,347 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:49:03,460 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:08,461 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:13,408 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:49:14,345 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:49:14,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:14,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:49:14,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:49:19,483 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:24,484 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:29,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:49:29,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:49:29,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:49:30,421 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:35,422 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:37,888 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:49:37,889 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:49:37,889 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:49:37,890 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:49:38,053 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:49:40,890 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:43,418 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:49:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:49:44,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:49:44,347 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:49:46,442 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:51,443 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:56,444 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:49:59,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:49:59,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:49:59,347 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:50:02,399 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:07,400 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:12,401 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:13,410 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:50:14,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:50:14,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:50:14,347 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:50:18,391 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:23,392 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:28,393 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:29,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:50:29,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:50:29,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:50:33,481 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:38,482 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:43,413 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:50:44,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:50:44,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:50:44,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:50:44,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:49,386 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:54,387 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:50:59,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:50:59,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:50:59,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:50:59,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:04,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:09,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:13,413 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:51:14,346 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:51:14,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:51:14,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:51:15,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:20,465 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:25,466 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:29,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:51:29,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:51:29,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:51:31,463 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:36,464 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:37,183 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:51:37,184 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:51:37,184 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:51:37,185 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:51:38,097 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:51:42,186 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:43,414 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:51:44,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:51:44,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:51:44,348 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:51:47,377 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:52,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:57,378 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:51:59,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:51:59,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:51:59,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:52:02,419 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:07,419 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:12,420 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:13,415 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:52:14,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:52:14,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:52:14,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:52:17,432 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:22,433 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:27,434 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:29,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:52:29,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:52:29,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:52:33,402 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:38,403 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:43,404 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:43,416 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:52:44,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:52:44,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:52:44,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:52:49,381 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:54,382 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:52:59,347 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:52:59,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:52:59,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:52:59,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:04,472 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:09,473 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:13,417 DEBUG SenderThread:50 [sender.py:send():379] send: stats 2024-05-16 08:53:14,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:53:14,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:53:14,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:53:15,470 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:20,471 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:25,472 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:29,348 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: internal_messages 2024-05-16 08:53:29,349 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: stop_status 2024-05-16 08:53:29,349 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: stop_status 2024-05-16 08:53:30,491 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:53:30,492 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:53:30,493 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,493 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: status_report 2024-05-16 08:53:30,494 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:30,955 DEBUG SenderThread:50 [sender.py:send():379] send: telemetry 2024-05-16 08:53:30,955 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,956 DEBUG HandlerThread:50 [handler.py:handle_request():146] handle_request: partial_history 2024-05-16 08:53:30,957 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:30,958 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,958 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:30,958 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,959 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:30,959 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,959 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:30,960 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,960 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:30,960 DEBUG SenderThread:50 [sender.py:send():379] send: history 2024-05-16 08:53:30,961 DEBUG SenderThread:50 [sender.py:send_request():406] send_request: summary_record 2024-05-16 08:53:30,961 INFO SenderThread:50 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end 2024-05-16 08:53:31,141 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/wandb-summary.json 2024-05-16 08:53:33,142 INFO Thread-12 :50 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240516_072542-5ru1r69k/files/output.log