wandb/run-20230407_041839-fpz463uv/logs/debug-internal.log

2023-04-07 04:18:39,269 INFO    StreamThr :70092 [internal.py:wandb_internal():86] W&B internal server running at pid: 70092, started at: 2023-04-07 04:18:39.261242
2023-04-07 04:18:39,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status
2023-04-07 04:18:39,705 INFO    WriterThread:70092 [datastore.py:open_for_write():85] open: /content/wandb/run-20230407_041839-fpz463uv/run-fpz463uv.wandb
2023-04-07 04:18:39,706 DEBUG   SenderThread:70092 [sender.py:send():375] send: header
2023-04-07 04:18:39,791 DEBUG   SenderThread:70092 [sender.py:send():375] send: run
2023-04-07 04:18:40,014 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: check_version
2023-04-07 04:18:40,014 INFO    SenderThread:70092 [dir_watcher.py:__init__():219] watching files in: /content/wandb/run-20230407_041839-fpz463uv/files
2023-04-07 04:18:40,015 INFO    SenderThread:70092 [sender.py:_start_run_threads():1124] run started: fpz463uv with start time 1680841119.262089
2023-04-07 04:18:40,015 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:18:40,015 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:18:40,015 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: check_version
2023-04-07 04:18:40,098 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: run_start
2023-04-07 04:18:40,100 DEBUG   HandlerThread:70092 [system_info.py:__init__():31] System info init
2023-04-07 04:18:40,100 DEBUG   HandlerThread:70092 [system_info.py:__init__():46] System info init done
2023-04-07 04:18:40,100 INFO    HandlerThread:70092 [system_monitor.py:start():181] Starting system monitor
2023-04-07 04:18:40,100 INFO    SystemMonitor:70092 [system_monitor.py:_start():145] Starting system asset monitoring threads
2023-04-07 04:18:40,100 INFO    HandlerThread:70092 [system_monitor.py:probe():201] Collecting system info
2023-04-07 04:18:40,102 DEBUG   HandlerThread:70092 [system_info.py:probe():195] Probing system
2023-04-07 04:18:40,106 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started cpu monitoring
2023-04-07 04:18:40,108 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started disk monitoring
2023-04-07 04:18:40,108 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started memory monitoring
2023-04-07 04:18:40,110 DEBUG   HandlerThread:70092 [git.py:repo():40] git repository is invalid
2023-04-07 04:18:40,110 DEBUG   HandlerThread:70092 [system_info.py:probe():240] Probing system done
2023-04-07 04:18:40,110 DEBUG   HandlerThread:70092 [system_monitor.py:probe():210] {'os': 'Linux-5.10.147+-x86_64-with-glibc2.31', 'python': '3.9.16', 'heartbeatAt': '2023-04-07T04:18:40.102996', 'startedAt': '2023-04-07T04:18:39.243901', 'docker': None, 'cuda': None, 'args': None, 'state': 'running', 'program': 'AI_Camp_Updated_CV_Notebook.ipynb', 'colab': 'https://colab.research.google.com/notebook#fileId=1Kj9-q_BBYGnrn8n7OFYq1bUZOG8im0BO', 'host': '52ce220a2379', 'username': 'root', 'executable': '/usr/bin/python3', 'cpu_count': 1, 'cpu_count_logical': 2, 'cpu_freq': {'current': 2200.166, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2200.166, 'min': 0.0, 'max': 0.0}, {'current': 2200.166, 'min': 0.0, 'max': 0.0}], 'disk': {'total': 107.71508407592773, 'used': 23.694255828857422}, 'memory': {'total': 12.681198120117188}}
2023-04-07 04:18:40,110 INFO    HandlerThread:70092 [system_monitor.py:probe():211] Finished collecting system info
2023-04-07 04:18:40,110 INFO    HandlerThread:70092 [system_monitor.py:probe():214] Publishing system info
2023-04-07 04:18:40,110 DEBUG   HandlerThread:70092 [system_info.py:_save_pip():51] Saving list of pip packages installed into the current environment
2023-04-07 04:18:40,112 DEBUG   HandlerThread:70092 [system_info.py:_save_pip():67] Saving pip packages done
2023-04-07 04:18:40,112 INFO    HandlerThread:70092 [system_monitor.py:probe():216] Finished publishing system info
2023-04-07 04:18:40,113 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started network monitoring
2023-04-07 04:18:40,126 DEBUG   SenderThread:70092 [sender.py:send():375] send: files
2023-04-07 04:18:40,126 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-metadata.json with policy now
2023-04-07 04:18:40,165 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:18:40,166 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:18:40,285 DEBUG   SenderThread:70092 [sender.py:send():375] send: telemetry
2023-04-07 04:18:40,287 DEBUG   SenderThread:70092 [sender.py:send():375] send: config
2023-04-07 04:18:40,288 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:18:40,289 DEBUG   SenderThread:70092 [sender.py:send():375] send: telemetry
2023-04-07 04:18:40,289 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:18:40,289 WARNING SenderThread:70092 [sender.py:send_metric():1329] Seen metric with glob (shouldn't happen)
2023-04-07 04:18:40,724 INFO    wandb-upload_0:70092 [upload_job.py:push():137] Uploaded file /tmp/tmp2eeedqewwandb/nltukg0m-wandb-metadata.json
2023-04-07 04:18:41,017 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_created():278] file/dir created: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:18:41,017 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_created():278] file/dir created: /content/wandb/run-20230407_041839-fpz463uv/files/requirements.txt
2023-04-07 04:18:41,018 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_created():278] file/dir created: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-metadata.json
2023-04-07 04:18:44,290 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:18:49,291 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:18:54,292 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:18:55,166 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:18:55,166 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:19:00,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:05,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:10,166 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:19:10,167 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:19:11,236 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:12,027 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/config.yaml
2023-04-07 04:19:16,419 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:21,420 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:25,166 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:19:25,166 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:19:27,226 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:32,227 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:37,228 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:40,113 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():159] Starting system metrics aggregation loop
2023-04-07 04:19:40,115 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:19:40,166 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:19:40,167 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:19:42,228 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:47,230 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:52,231 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:19:55,170 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:19:55,170 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:19:58,230 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:03,231 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:08,232 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:10,115 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:20:10,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:20:10,171 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:20:14,231 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:19,232 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:24,233 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:25,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:20:25,171 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:20:29,236 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:34,237 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:39,238 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:40,119 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:20:40,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:20:40,171 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:20:44,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:49,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:54,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:20:55,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:20:55,171 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:21:00,241 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:05,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:10,120 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:21:10,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:21:10,171 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:21:11,231 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:16,232 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:21,233 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:25,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:21:25,172 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:21:26,235 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:31,236 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:36,237 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:40,121 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:21:40,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:21:40,171 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:21:41,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:46,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:51,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:21:55,172 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:21:55,172 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:21:57,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:02,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:07,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:10,121 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:22:10,171 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:22:10,172 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:22:13,241 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:18,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:23,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:25,172 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:22:25,172 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:22:29,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:34,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:39,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:40,122 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:22:40,172 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:22:40,172 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:22:45,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:50,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:22:55,172 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:22:55,172 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:22:56,235 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:01,236 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:06,238 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:10,123 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:23:10,172 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:23:10,173 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:23:12,232 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:17,233 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:22,235 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:25,172 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:23:25,173 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:23:27,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:32,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:37,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:40,123 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:23:40,173 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:23:40,173 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:23:43,233 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:48,203 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:23:48,206 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:23:48,206 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:23:48,207 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:23:48,207 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:23:48,212 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:23:48,213 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:23:49,145 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:23:49,213 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:54,221 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:23:55,147 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/config.yaml
2023-04-07 04:23:55,173 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:23:55,173 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:24:00,232 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:05,233 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:10,124 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:24:10,173 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:24:10,173 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:24:10,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:15,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:20,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:25,176 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:24:25,177 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:24:26,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:31,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:36,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:40,125 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:24:40,177 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:24:40,178 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:24:41,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:46,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:51,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:24:55,177 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:24:55,178 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:24:57,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:02,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:07,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:10,125 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:25:10,177 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:25:10,177 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:25:13,238 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:18,239 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:23,239 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:25,178 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:25:25,178 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:25:28,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:33,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:38,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:40,126 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:25:40,178 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:25:40,178 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:25:43,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:48,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:53,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:25:55,178 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:25:55,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:25:59,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:04,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:09,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:10,127 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:26:10,178 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:26:10,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:26:15,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:20,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:25,178 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:26:25,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:26:26,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:31,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:36,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:40,127 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:26:40,178 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:26:40,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:26:42,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:47,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:52,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:26:55,179 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:26:55,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:26:57,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:02,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:07,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:10,128 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:27:10,179 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:27:10,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:27:13,241 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:18,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:23,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:25,179 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:27:25,180 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:27:29,240 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:34,241 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:39,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:40,129 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:27:40,179 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:27:40,179 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:27:44,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:49,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:54,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:27:55,179 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:27:55,180 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:28:00,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:05,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:10,129 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:28:10,179 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:28:10,180 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:28:11,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:12,541 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:28:12,543 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:28:12,543 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:28:12,544 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:28:13,267 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:28:16,545 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:21,546 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:25,180 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:28:25,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:28:27,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:32,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:37,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:40,130 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:28:40,180 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:28:40,180 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:28:42,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:47,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:52,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:28:55,180 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:28:55,180 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:28:58,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:03,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:08,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:10,131 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:29:10,180 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:29:10,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:29:14,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:19,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:24,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:25,180 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:29:25,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:29:30,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:35,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:40,131 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:29:40,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:29:40,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:29:41,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:46,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:51,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:29:55,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:29:55,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:29:57,242 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:02,243 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:07,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:10,132 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:30:10,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:30:10,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:30:12,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:17,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:22,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:25,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:30:25,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:30:27,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:32,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:37,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:40,133 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:30:40,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:30:40,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:30:43,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:48,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:53,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:30:55,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:30:55,181 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:30:59,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:04,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:09,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:10,134 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:31:10,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:31:10,182 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:31:15,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:20,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:25,181 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:31:25,182 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:31:26,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:31,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:36,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:40,135 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:31:40,182 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:31:40,182 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:31:41,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:46,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:51,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:31:55,182 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:31:55,182 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:31:57,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:02,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:07,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:10,136 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:32:10,182 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:32:10,182 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:32:12,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:17,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:22,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:25,182 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:32:25,182 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:32:28,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:33,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:38,261 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:40,137 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:32:40,182 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:32:40,183 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:32:40,788 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:32:40,791 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:32:40,791 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:32:40,792 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:32:41,380 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:32:43,794 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:48,795 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:53,796 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:32:55,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:32:55,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:32:59,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:04,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:09,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:10,137 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:33:10,183 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:33:10,183 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:33:15,244 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:20,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:25,183 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:33:25,183 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:33:26,245 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:31,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:36,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:40,138 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:33:40,183 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:33:40,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:33:41,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:46,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:51,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:33:55,183 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:33:55,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:33:57,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:02,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:07,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:10,139 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:34:10,183 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:34:10,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:34:12,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:17,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:22,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:25,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:34:25,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:34:27,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:32,259 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:37,260 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:40,139 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:34:40,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:34:40,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:34:43,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:48,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:53,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:34:55,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:34:55,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:34:59,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:04,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:09,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:10,140 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:35:10,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:35:10,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:35:15,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:20,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:25,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:35:25,184 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:35:26,246 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:31,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:36,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:40,141 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:35:40,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:35:40,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:35:41,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:46,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:51,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:35:55,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:35:55,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:35:57,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:02,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:07,259 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:10,142 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:36:10,184 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:36:10,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:36:13,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:18,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:23,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:25,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:36:25,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:36:29,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:34,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:39,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:40,142 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:36:40,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:36:40,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:36:45,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:50,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:36:55,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:36:55,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:36:56,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:01,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:06,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:06,370 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:37:06,371 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:37:06,371 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:37:06,372 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:37:06,494 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:37:10,143 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:37:10,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:37:10,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:37:12,247 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:17,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:22,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:25,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:37:25,185 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:37:27,259 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:32,261 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:37,262 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:40,143 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:37:40,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:37:40,186 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:37:43,262 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:48,263 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:53,264 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:37:55,185 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:37:55,186 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:37:59,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:04,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:09,259 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:10,144 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:38:10,186 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:38:10,186 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:38:15,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:20,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:25,186 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:38:25,186 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:38:26,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:31,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:36,251 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:40,145 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:38:40,186 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:38:40,186 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:38:42,248 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:47,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:52,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:38:55,186 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:38:55,187 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:38:58,249 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:03,250 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:07,341 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:39:07,344 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:39:07,344 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:39:07,344 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:39:07,345 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:39:07,345 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:39:07,345 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:39:07,345 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:39:07,346 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:39:07,556 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:39:08,355 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:08,557 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/config.yaml
2023-04-07 04:39:10,170 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:39:10,207 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:39:10,207 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:39:14,275 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:19,276 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:24,278 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:25,207 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:39:25,208 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:39:30,272 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:35,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:40,168 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:39:40,207 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:39:40,207 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:39:40,280 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:45,281 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:50,283 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:39:55,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:39:55,208 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:39:56,279 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:01,280 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:06,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:10,169 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:40:10,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:40:10,208 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:40:12,269 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:17,270 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:22,271 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:25,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:40:25,208 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:40:27,288 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:32,288 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:37,300 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:40,169 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:40:40,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:40:40,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:40:43,287 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:48,288 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:53,296 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:40:55,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:40:55,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:40:59,272 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:04,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:09,274 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:10,170 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:41:10,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:41:10,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:41:14,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:19,288 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:24,289 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:25,208 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:41:25,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:41:30,278 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:35,278 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:40,171 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:41:40,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:41:40,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:41:40,281 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:45,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:50,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:41:55,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:41:55,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:41:56,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:01,283 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:06,284 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:10,172 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:42:10,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:42:10,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:42:12,279 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:17,280 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:22,281 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:25,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:42:25,209 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:42:28,272 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:33,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:38,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:40,173 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:42:40,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:42:40,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:42:43,280 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:48,282 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:53,283 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:42:55,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:42:55,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:42:59,283 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:04,291 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:09,292 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:10,173 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:43:10,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:43:10,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:43:15,284 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:20,285 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:25,209 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:43:25,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:43:26,271 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:31,272 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:36,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:40,174 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:43:40,210 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:43:40,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:43:41,277 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:46,278 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:51,279 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:43:55,210 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:43:55,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:43:57,274 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:02,277 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:05,382 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:44:05,383 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:44:05,384 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:44:05,385 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:44:05,734 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:44:07,386 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:10,175 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:44:10,210 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:44:10,210 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:44:13,272 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:18,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:23,274 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:25,211 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:44:25,211 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:44:29,272 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:34,273 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:39,274 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:40,042 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: partial_history
2023-04-07 04:44:40,045 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:44:40,045 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:44:40,045 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:44:40,045 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:44:40,045 DEBUG   SenderThread:70092 [sender.py:send():375] send: metric
2023-04-07 04:44:40,046 DEBUG   SenderThread:70092 [sender.py:send():375] send: history
2023-04-07 04:44:40,046 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: summary_record
2023-04-07 04:44:40,047 INFO    SenderThread:70092 [sender.py:_save_file():1378] saving file wandb-summary.json with policy end
2023-04-07 04:44:40,175 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:44:40,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:44:40,252 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:44:40,749 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/wandb-summary.json
2023-04-07 04:44:44,386 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:46,050 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: log_artifact
2023-04-07 04:44:46,051 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: log_artifact
2023-04-07 04:44:47,320 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: pause
2023-04-07 04:44:47,320 INFO    HandlerThread:70092 [handler.py:handle_request_pause():719] stopping system metrics thread or process
2023-04-07 04:44:47,320 INFO    HandlerThread:70092 [system_monitor.py:finish():190] Stopping system monitor
2023-04-07 04:44:47,321 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():166] Finished system metrics aggregation loop
2023-04-07 04:44:47,321 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():170] Publishing last batch of metrics
2023-04-07 04:44:47,322 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined cpu monitor
2023-04-07 04:44:47,322 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined disk monitor
2023-04-07 04:44:47,322 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined memory monitor
2023-04-07 04:44:47,323 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined network monitor
2023-04-07 04:44:47,392 INFO    wandb-upload_3:70092 [upload_job.py:push():95] Uploaded file /root/.local/share/wandb/artifacts/staging/tmp_1xwm1md
2023-04-07 04:44:47,396 INFO    wandb-upload_0:70092 [upload_job.py:push():95] Uploaded file /root/.local/share/wandb/artifacts/staging/tmpl_634fy7
2023-04-07 04:44:47,404 INFO    wandb-upload_1:70092 [upload_job.py:push():95] Uploaded file /root/.local/share/wandb/artifacts/staging/tmp9bnux6s5
2023-04-07 04:44:47,405 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: resume
2023-04-07 04:44:47,406 INFO    HandlerThread:70092 [handler.py:handle_request_resume():710] starting system metrics thread or process
2023-04-07 04:44:47,406 INFO    HandlerThread:70092 [system_monitor.py:start():181] Starting system monitor
2023-04-07 04:44:47,407 INFO    SystemMonitor:70092 [system_monitor.py:_start():145] Starting system asset monitoring threads
2023-04-07 04:44:47,411 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started cpu monitoring
2023-04-07 04:44:47,436 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started disk monitoring
2023-04-07 04:44:47,454 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started memory monitoring
2023-04-07 04:44:47,466 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started network monitoring
2023-04-07 04:44:47,483 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: pause
2023-04-07 04:44:47,483 INFO    HandlerThread:70092 [handler.py:handle_request_pause():719] stopping system metrics thread or process
2023-04-07 04:44:47,484 INFO    HandlerThread:70092 [system_monitor.py:finish():190] Stopping system monitor
2023-04-07 04:44:47,484 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():159] Starting system metrics aggregation loop
2023-04-07 04:44:47,484 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():166] Finished system metrics aggregation loop
2023-04-07 04:44:47,484 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():170] Publishing last batch of metrics
2023-04-07 04:44:47,490 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined cpu monitor
2023-04-07 04:44:47,491 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined disk monitor
2023-04-07 04:44:47,491 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined memory monitor
2023-04-07 04:44:47,491 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined network monitor
2023-04-07 04:44:51,999 INFO    wandb-upload_2:70092 [upload_job.py:push():95] Uploaded file /root/.local/share/wandb/artifacts/staging/tmpagp94iy5
2023-04-07 04:44:52,819 INFO    SenderThread:70092 [sender.py:send_request_log_artifact():1438] logged artifact model-fpz463uv - {'id': 'QXJ0aWZhY3Q6NDE2MzIwMzMy', 'digest': '2a0bf8fd2a54ce8c7a255743df6e1662', 'state': 'PENDING', 'aliases': [], 'artifactSequence': {'id': 'QXJ0aWZhY3RDb2xsZWN0aW9uOjYxMTYxNjc2', 'latestArtifact': None}, 'version': 'latest'}
2023-04-07 04:44:52,820 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:52,821 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:44:52,821 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:44:53,802 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_created():278] file/dir created: /content/wandb/run-20230407_041839-fpz463uv/files/output.log
2023-04-07 04:44:55,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:44:55,252 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:44:55,803 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/output.log
2023-04-07 04:44:58,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:44:58,804 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/config.yaml
2023-04-07 04:45:03,451 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:08,452 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:10,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:45:10,252 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:45:14,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:19,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:24,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:25,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:45:25,252 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:45:30,312 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:35,313 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:40,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:45:40,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:45:41,312 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:46,313 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:51,313 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:45:55,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:45:55,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:45:57,311 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:02,312 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:07,314 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:10,252 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:46:10,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:46:12,322 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:17,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:22,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:25,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:46:25,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:46:28,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:33,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:38,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:40,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:46:40,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:46:44,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:49,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:54,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:46:55,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:46:55,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:47:00,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:05,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:10,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:47:10,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:47:10,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:15,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:20,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:25,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:47:25,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:47:26,314 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:31,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:36,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:40,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:47:40,253 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:47:42,311 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:47,312 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:52,314 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:47:55,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:47:55,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:47:58,313 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:03,314 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:08,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:10,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:48:10,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:48:13,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:18,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:23,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:25,253 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:48:25,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:48:29,322 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:34,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:39,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:40,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:48:40,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:48:45,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:50,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:48:55,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:48:55,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:48:56,314 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:01,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:06,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:10,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:49:10,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:49:11,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:16,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:21,327 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:25,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:49:25,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:49:27,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:32,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:37,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:40,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:49:40,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:49:42,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:47,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:52,327 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:49:55,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:49:55,254 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:49:58,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:03,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:08,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:10,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:50:10,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:50:13,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:18,327 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:23,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:25,254 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:50:25,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:50:29,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:34,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:39,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:40,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:50:40,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:50:45,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:50,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:50:55,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:50:55,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:50:56,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:01,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:06,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:10,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:51:10,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:51:12,314 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:17,951 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/output.log
2023-04-07 04:51:18,073 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:19,952 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/output.log
2023-04-07 04:51:21,517 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: resume
2023-04-07 04:51:21,517 INFO    HandlerThread:70092 [handler.py:handle_request_resume():710] starting system metrics thread or process
2023-04-07 04:51:21,517 INFO    HandlerThread:70092 [system_monitor.py:start():181] Starting system monitor
2023-04-07 04:51:21,517 INFO    SystemMonitor:70092 [system_monitor.py:_start():145] Starting system asset monitoring threads
2023-04-07 04:51:21,518 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started cpu monitoring
2023-04-07 04:51:21,525 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started disk monitoring
2023-04-07 04:51:21,532 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started memory monitoring
2023-04-07 04:51:21,533 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started network monitoring
2023-04-07 04:51:21,586 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: pause
2023-04-07 04:51:21,586 INFO    HandlerThread:70092 [handler.py:handle_request_pause():719] stopping system metrics thread or process
2023-04-07 04:51:21,586 INFO    HandlerThread:70092 [system_monitor.py:finish():190] Stopping system monitor
2023-04-07 04:51:21,589 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined cpu monitor
2023-04-07 04:51:21,589 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined disk monitor
2023-04-07 04:51:21,590 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined memory monitor
2023-04-07 04:51:21,590 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined network monitor
2023-04-07 04:51:21,591 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():159] Starting system metrics aggregation loop
2023-04-07 04:51:21,591 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():166] Finished system metrics aggregation loop
2023-04-07 04:51:21,591 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():170] Publishing last batch of metrics
2023-04-07 04:51:21,593 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:51:23,593 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:25,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:51:25,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:51:29,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:34,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:39,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:40,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:51:40,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:51:44,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:49,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:54,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:51:55,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:51:55,255 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:52:00,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:05,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:10,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:52:10,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:52:11,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:16,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:21,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:25,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:52:25,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:52:26,324 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:31,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:36,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:40,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:52:40,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:52:41,327 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:46,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:51,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:52:55,255 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:52:55,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:52:57,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:02,315 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:07,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:10,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:53:10,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:53:13,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:18,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:23,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:25,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:53:25,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:53:28,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:33,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:38,320 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:40,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:53:40,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:53:43,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:48,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:53,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:53:55,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:53:55,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:53:59,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:04,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:09,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:10,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:54:10,256 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:54:14,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:19,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:24,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:25,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:54:25,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:54:30,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:35,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:40,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:54:40,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:54:41,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:46,316 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:51,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:54:55,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:54:55,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:54:56,325 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:01,326 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:06,327 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:07,749 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: resume
2023-04-07 04:55:07,749 INFO    HandlerThread:70092 [handler.py:handle_request_resume():710] starting system metrics thread or process
2023-04-07 04:55:07,749 INFO    HandlerThread:70092 [system_monitor.py:start():181] Starting system monitor
2023-04-07 04:55:07,755 INFO    SystemMonitor:70092 [system_monitor.py:_start():145] Starting system asset monitoring threads
2023-04-07 04:55:07,760 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started cpu monitoring
2023-04-07 04:55:07,776 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started disk monitoring
2023-04-07 04:55:07,781 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started memory monitoring
2023-04-07 04:55:07,789 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started network monitoring
2023-04-07 04:55:10,256 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:55:10,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:55:11,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:16,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:19,044 INFO    Thread-12 :70092 [dir_watcher.py:_on_file_modified():295] file/dir modified: /content/wandb/run-20230407_041839-fpz463uv/files/output.log
2023-04-07 04:55:21,813 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:25,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:55:25,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:55:27,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:32,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:37,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:40,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:55:40,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:55:43,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:48,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:53,332 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:55:55,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:55:55,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:55:59,322 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:04,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:07,790 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():159] Starting system metrics aggregation loop
2023-04-07 04:56:07,791 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:56:09,792 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:10,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:56:10,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:56:13,116 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: pause
2023-04-07 04:56:13,116 INFO    HandlerThread:70092 [handler.py:handle_request_pause():719] stopping system metrics thread or process
2023-04-07 04:56:13,116 INFO    HandlerThread:70092 [system_monitor.py:finish():190] Stopping system monitor
2023-04-07 04:56:13,116 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():166] Finished system metrics aggregation loop
2023-04-07 04:56:13,116 DEBUG   SystemMonitor:70092 [system_monitor.py:_start():170] Publishing last batch of metrics
2023-04-07 04:56:13,117 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined cpu monitor
2023-04-07 04:56:13,118 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined disk monitor
2023-04-07 04:56:13,118 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined memory monitor
2023-04-07 04:56:13,118 INFO    HandlerThread:70092 [interfaces.py:finish():202] Joined network monitor
2023-04-07 04:56:13,119 DEBUG   SenderThread:70092 [sender.py:send():375] send: stats
2023-04-07 04:56:15,119 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:20,120 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:25,121 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:25,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:56:25,257 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:56:30,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:35,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:40,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:56:40,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:56:41,317 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:46,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:51,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:56:55,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:56:55,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:56:56,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:01,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:06,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:10,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:57:10,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:57:12,320 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:17,322 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:22,323 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:25,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:57:25,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:57:27,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:32,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:37,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:40,257 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:57:40,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:57:43,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:48,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:53,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:57:55,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:57:55,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:57:59,318 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:04,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:09,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:10,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:58:10,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:58:14,328 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:19,329 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:24,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:25,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:58:25,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:58:29,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:34,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:39,332 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:40,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:58:40,259 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:58:45,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:46,060 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: resume
2023-04-07 04:58:46,061 INFO    HandlerThread:70092 [handler.py:handle_request_resume():710] starting system metrics thread or process
2023-04-07 04:58:46,061 INFO    HandlerThread:70092 [system_monitor.py:start():181] Starting system monitor
2023-04-07 04:58:46,061 INFO    SystemMonitor:70092 [system_monitor.py:_start():145] Starting system asset monitoring threads
2023-04-07 04:58:46,063 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started cpu monitoring
2023-04-07 04:58:46,072 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started disk monitoring
2023-04-07 04:58:46,076 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started memory monitoring
2023-04-07 04:58:46,079 INFO    SystemMonitor:70092 [interfaces.py:start():190] Started network monitoring
2023-04-07 04:58:50,319 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:58:55,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:58:55,258 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:58:55,335 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:00,336 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:05,337 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:10,259 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:59:10,260 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:59:10,348 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:15,349 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:20,349 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:25,259 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:59:25,259 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:59:26,330 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:31,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:36,331 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report
2023-04-07 04:59:40,258 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: stop_status
2023-04-07 04:59:40,259 DEBUG   SenderThread:70092 [sender.py:send_request():402] send_request: stop_status
2023-04-07 04:59:42,327 DEBUG   HandlerThread:70092 [handler.py:handle_request():144] handle_request: status_report