diff --git "a/sf_log.txt" "b/sf_log.txt" new file mode 100644--- /dev/null +++ "b/sf_log.txt" @@ -0,0 +1,1117 @@ +[2024-11-11 14:13:20,505][00562] Saving configuration to /content/train_dir/default_experiment/config.json... +[2024-11-11 14:13:20,509][00562] Rollout worker 0 uses device cpu +[2024-11-11 14:13:20,510][00562] Rollout worker 1 uses device cpu +[2024-11-11 14:13:20,511][00562] Rollout worker 2 uses device cpu +[2024-11-11 14:13:20,512][00562] Rollout worker 3 uses device cpu +[2024-11-11 14:13:20,513][00562] Rollout worker 4 uses device cpu +[2024-11-11 14:13:20,514][00562] Rollout worker 5 uses device cpu +[2024-11-11 14:13:20,515][00562] Rollout worker 6 uses device cpu +[2024-11-11 14:13:20,516][00562] Rollout worker 7 uses device cpu +[2024-11-11 14:13:20,669][00562] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-11-11 14:13:20,670][00562] InferenceWorker_p0-w0: min num requests: 2 +[2024-11-11 14:13:20,702][00562] Starting all processes... +[2024-11-11 14:13:20,703][00562] Starting process learner_proc0 +[2024-11-11 14:13:20,748][00562] Starting all processes... +[2024-11-11 14:13:20,754][00562] Starting process inference_proc0-0 +[2024-11-11 14:13:20,755][00562] Starting process rollout_proc0 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc1 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc2 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc3 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc4 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc5 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc6 +[2024-11-11 14:13:20,757][00562] Starting process rollout_proc7 +[2024-11-11 14:13:38,740][03143] Worker 7 uses CPU cores [1] +[2024-11-11 14:13:38,851][03135] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-11-11 14:13:38,858][03135] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2024-11-11 14:13:38,892][03141] Worker 5 uses CPU cores [1] +[2024-11-11 14:13:38,946][03137] Worker 1 uses CPU cores [1] +[2024-11-11 14:13:38,955][03135] Num visible devices: 1 +[2024-11-11 14:13:39,090][03142] Worker 6 uses CPU cores [0] +[2024-11-11 14:13:39,192][03138] Worker 2 uses CPU cores [0] +[2024-11-11 14:13:39,308][03122] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-11-11 14:13:39,309][03122] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2024-11-11 14:13:39,334][03122] Num visible devices: 1 +[2024-11-11 14:13:39,359][03122] Starting seed is not provided +[2024-11-11 14:13:39,360][03122] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-11-11 14:13:39,360][03122] Initializing actor-critic model on device cuda:0 +[2024-11-11 14:13:39,361][03122] RunningMeanStd input shape: (3, 72, 128) +[2024-11-11 14:13:39,364][03122] RunningMeanStd input shape: (1,) +[2024-11-11 14:13:39,401][03139] Worker 3 uses CPU cores [1] +[2024-11-11 14:13:39,400][03122] ConvEncoder: input_channels=3 +[2024-11-11 14:13:39,420][03136] Worker 0 uses CPU cores [0] +[2024-11-11 14:13:39,425][03140] Worker 4 uses CPU cores [0] +[2024-11-11 14:13:40,071][03122] Conv encoder output size: 512 +[2024-11-11 14:13:40,071][03122] Policy head output size: 512 +[2024-11-11 14:13:40,151][03122] Created Actor Critic model with architecture: +[2024-11-11 14:13:40,152][03122] ActorCriticSharedWeights( + (obs_normalizer): ObservationNormalizer( + (running_mean_std): RunningMeanStdDictInPlace( + (running_mean_std): ModuleDict( + (obs): RunningMeanStdInPlace() + ) + ) + ) + (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) + (encoder): VizdoomEncoder( + (basic_encoder): ConvEncoder( + (enc): RecursiveScriptModule( + original_name=ConvEncoderImpl + (conv_head): RecursiveScriptModule( + original_name=Sequential + (0): RecursiveScriptModule(original_name=Conv2d) + (1): RecursiveScriptModule(original_name=ELU) + (2): RecursiveScriptModule(original_name=Conv2d) + (3): RecursiveScriptModule(original_name=ELU) + (4): RecursiveScriptModule(original_name=Conv2d) + (5): RecursiveScriptModule(original_name=ELU) + ) + (mlp_layers): RecursiveScriptModule( + original_name=Sequential + (0): RecursiveScriptModule(original_name=Linear) + (1): RecursiveScriptModule(original_name=ELU) + ) + ) + ) + ) + (core): ModelCoreRNN( + (core): GRU(512, 512) + ) + (decoder): MlpDecoder( + (mlp): Identity() + ) + (critic_linear): Linear(in_features=512, out_features=1, bias=True) + (action_parameterization): ActionParameterizationDefault( + (distribution_linear): Linear(in_features=512, out_features=5, bias=True) + ) +) +[2024-11-11 14:13:40,593][03122] Using optimizer +[2024-11-11 14:13:40,671][00562] Heartbeat connected on InferenceWorker_p0-w0 +[2024-11-11 14:13:40,677][00562] Heartbeat connected on RolloutWorker_w0 +[2024-11-11 14:13:40,687][00562] Heartbeat connected on RolloutWorker_w1 +[2024-11-11 14:13:40,693][00562] Heartbeat connected on RolloutWorker_w2 +[2024-11-11 14:13:40,694][00562] Heartbeat connected on RolloutWorker_w3 +[2024-11-11 14:13:40,699][00562] Heartbeat connected on RolloutWorker_w4 +[2024-11-11 14:13:40,700][00562] Heartbeat connected on RolloutWorker_w5 +[2024-11-11 14:13:40,701][00562] Heartbeat connected on RolloutWorker_w6 +[2024-11-11 14:13:40,706][00562] Heartbeat connected on RolloutWorker_w7 +[2024-11-11 14:13:40,741][00562] Heartbeat connected on Batcher_0 +[2024-11-11 14:13:45,455][03122] No checkpoints found +[2024-11-11 14:13:45,456][03122] Did not load from checkpoint, starting from scratch! +[2024-11-11 14:13:45,457][03122] Initialized policy 0 weights for model version 0 +[2024-11-11 14:13:45,464][03122] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-11-11 14:13:45,477][03122] LearnerWorker_p0 finished initialization! +[2024-11-11 14:13:45,477][00562] Heartbeat connected on LearnerWorker_p0 +[2024-11-11 14:13:45,558][03135] RunningMeanStd input shape: (3, 72, 128) +[2024-11-11 14:13:45,560][03135] RunningMeanStd input shape: (1,) +[2024-11-11 14:13:45,571][03135] ConvEncoder: input_channels=3 +[2024-11-11 14:13:45,674][03135] Conv encoder output size: 512 +[2024-11-11 14:13:45,675][03135] Policy head output size: 512 +[2024-11-11 14:13:45,728][00562] Inference worker 0-0 is ready! +[2024-11-11 14:13:45,729][00562] All inference workers are ready! Signal rollout workers to start! +[2024-11-11 14:13:45,932][03143] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,933][03139] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,929][03136] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,936][03137] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,934][03138] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,936][03141] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,933][03140] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:45,935][03142] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:13:46,615][03143] Decorrelating experience for 0 frames... +[2024-11-11 14:13:47,230][03136] Decorrelating experience for 0 frames... +[2024-11-11 14:13:47,233][03142] Decorrelating experience for 0 frames... +[2024-11-11 14:13:47,238][03140] Decorrelating experience for 0 frames... +[2024-11-11 14:13:47,235][03138] Decorrelating experience for 0 frames... +[2024-11-11 14:13:48,364][03136] Decorrelating experience for 32 frames... +[2024-11-11 14:13:48,380][03138] Decorrelating experience for 32 frames... +[2024-11-11 14:13:48,385][03140] Decorrelating experience for 32 frames... +[2024-11-11 14:13:48,436][03139] Decorrelating experience for 0 frames... +[2024-11-11 14:13:48,441][03137] Decorrelating experience for 0 frames... +[2024-11-11 14:13:48,444][03141] Decorrelating experience for 0 frames... +[2024-11-11 14:13:49,388][03142] Decorrelating experience for 32 frames... +[2024-11-11 14:13:49,536][00562] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-11-11 14:13:49,634][03136] Decorrelating experience for 64 frames... +[2024-11-11 14:13:50,015][03139] Decorrelating experience for 32 frames... +[2024-11-11 14:13:50,010][03137] Decorrelating experience for 32 frames... +[2024-11-11 14:13:50,012][03141] Decorrelating experience for 32 frames... +[2024-11-11 14:13:50,018][03143] Decorrelating experience for 32 frames... +[2024-11-11 14:13:50,794][03138] Decorrelating experience for 64 frames... +[2024-11-11 14:13:50,831][03136] Decorrelating experience for 96 frames... +[2024-11-11 14:13:50,911][03139] Decorrelating experience for 64 frames... +[2024-11-11 14:13:51,535][03142] Decorrelating experience for 64 frames... +[2024-11-11 14:13:51,539][03140] Decorrelating experience for 64 frames... +[2024-11-11 14:13:52,033][03143] Decorrelating experience for 64 frames... +[2024-11-11 14:13:52,175][03139] Decorrelating experience for 96 frames... +[2024-11-11 14:13:52,407][03138] Decorrelating experience for 96 frames... +[2024-11-11 14:13:52,976][03140] Decorrelating experience for 96 frames... +[2024-11-11 14:13:53,081][03137] Decorrelating experience for 64 frames... +[2024-11-11 14:13:53,881][03141] Decorrelating experience for 64 frames... +[2024-11-11 14:13:54,126][03143] Decorrelating experience for 96 frames... +[2024-11-11 14:13:54,496][03142] Decorrelating experience for 96 frames... +[2024-11-11 14:13:54,536][00562] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 3.2. Samples: 16. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-11-11 14:13:54,538][00562] Avg episode reward: [(0, '1.240')] +[2024-11-11 14:13:56,816][03137] Decorrelating experience for 96 frames... +[2024-11-11 14:13:58,646][03141] Decorrelating experience for 96 frames... +[2024-11-11 14:13:59,536][00562] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 195.2. Samples: 1952. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-11-11 14:13:59,539][00562] Avg episode reward: [(0, '2.538')] +[2024-11-11 14:14:00,594][03122] Signal inference workers to stop experience collection... +[2024-11-11 14:14:00,626][03135] InferenceWorker_p0-w0: stopping experience collection +[2024-11-11 14:14:02,988][03122] Signal inference workers to resume experience collection... +[2024-11-11 14:14:02,990][03135] InferenceWorker_p0-w0: resuming experience collection +[2024-11-11 14:14:04,536][00562] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 12288. Throughput: 0: 224.7. Samples: 3370. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) +[2024-11-11 14:14:04,541][00562] Avg episode reward: [(0, '2.654')] +[2024-11-11 14:14:09,536][00562] Fps is (10 sec: 3276.8, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 32768. Throughput: 0: 330.9. Samples: 6618. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:14:09,540][00562] Avg episode reward: [(0, '3.712')] +[2024-11-11 14:14:10,740][03135] Updated weights for policy 0, policy_version 10 (0.0158) +[2024-11-11 14:14:14,539][00562] Fps is (10 sec: 3685.4, 60 sec: 1965.9, 300 sec: 1965.9). Total num frames: 49152. Throughput: 0: 479.4. Samples: 11986. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:14:14,541][00562] Avg episode reward: [(0, '4.213')] +[2024-11-11 14:14:19,536][00562] Fps is (10 sec: 2867.2, 60 sec: 2048.0, 300 sec: 2048.0). Total num frames: 61440. Throughput: 0: 522.7. Samples: 15682. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:14:19,540][00562] Avg episode reward: [(0, '4.425')] +[2024-11-11 14:14:23,894][03135] Updated weights for policy 0, policy_version 20 (0.0033) +[2024-11-11 14:14:24,536][00562] Fps is (10 sec: 3277.7, 60 sec: 2340.6, 300 sec: 2340.6). Total num frames: 81920. Throughput: 0: 536.7. Samples: 18784. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:14:24,542][00562] Avg episode reward: [(0, '4.474')] +[2024-11-11 14:14:29,537][00562] Fps is (10 sec: 4095.4, 60 sec: 2559.9, 300 sec: 2559.9). Total num frames: 102400. Throughput: 0: 631.5. Samples: 25262. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:14:29,543][00562] Avg episode reward: [(0, '4.385')] +[2024-11-11 14:14:29,546][03122] Saving new best policy, reward=4.385! +[2024-11-11 14:14:34,540][00562] Fps is (10 sec: 3275.4, 60 sec: 2548.4, 300 sec: 2548.4). Total num frames: 114688. Throughput: 0: 649.9. Samples: 29246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:14:34,543][00562] Avg episode reward: [(0, '4.387')] +[2024-11-11 14:14:34,555][03122] Saving new best policy, reward=4.387! +[2024-11-11 14:14:36,570][03135] Updated weights for policy 0, policy_version 30 (0.0035) +[2024-11-11 14:14:39,536][00562] Fps is (10 sec: 2867.6, 60 sec: 2621.4, 300 sec: 2621.4). Total num frames: 131072. Throughput: 0: 700.9. Samples: 31556. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:14:39,541][00562] Avg episode reward: [(0, '4.332')] +[2024-11-11 14:14:44,536][00562] Fps is (10 sec: 4097.7, 60 sec: 2830.0, 300 sec: 2830.0). Total num frames: 155648. Throughput: 0: 794.4. Samples: 37702. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:14:44,539][00562] Avg episode reward: [(0, '4.309')] +[2024-11-11 14:14:46,762][03135] Updated weights for policy 0, policy_version 40 (0.0032) +[2024-11-11 14:14:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 2798.9, 300 sec: 2798.9). Total num frames: 167936. Throughput: 0: 877.4. Samples: 42854. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:14:49,538][00562] Avg episode reward: [(0, '4.368')] +[2024-11-11 14:14:54,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3140.3, 300 sec: 2898.7). Total num frames: 188416. Throughput: 0: 850.6. Samples: 44896. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:14:54,542][00562] Avg episode reward: [(0, '4.408')] +[2024-11-11 14:14:54,551][03122] Saving new best policy, reward=4.408! +[2024-11-11 14:14:58,442][03135] Updated weights for policy 0, policy_version 50 (0.0018) +[2024-11-11 14:14:59,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 2984.2). Total num frames: 208896. Throughput: 0: 866.1. Samples: 50956. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:14:59,541][00562] Avg episode reward: [(0, '4.312')] +[2024-11-11 14:15:04,537][00562] Fps is (10 sec: 3686.1, 60 sec: 3549.8, 300 sec: 3003.7). Total num frames: 225280. Throughput: 0: 920.3. Samples: 57098. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:15:04,543][00562] Avg episode reward: [(0, '4.295')] +[2024-11-11 14:15:09,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3020.8). Total num frames: 241664. Throughput: 0: 895.1. Samples: 59062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:15:09,540][00562] Avg episode reward: [(0, '4.409')] +[2024-11-11 14:15:09,542][03122] Saving new best policy, reward=4.409! +[2024-11-11 14:15:10,575][03135] Updated weights for policy 0, policy_version 60 (0.0042) +[2024-11-11 14:15:14,536][00562] Fps is (10 sec: 3686.7, 60 sec: 3550.0, 300 sec: 3084.0). Total num frames: 262144. Throughput: 0: 866.4. Samples: 64250. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:15:14,543][00562] Avg episode reward: [(0, '4.556')] +[2024-11-11 14:15:14,550][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000064_262144.pth... +[2024-11-11 14:15:14,700][03122] Saving new best policy, reward=4.556! +[2024-11-11 14:15:19,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3140.3). Total num frames: 282624. Throughput: 0: 915.1. Samples: 70422. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:15:19,543][00562] Avg episode reward: [(0, '4.557')] +[2024-11-11 14:15:20,911][03135] Updated weights for policy 0, policy_version 70 (0.0019) +[2024-11-11 14:15:24,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3104.3). Total num frames: 294912. Throughput: 0: 916.0. Samples: 72776. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:15:24,542][00562] Avg episode reward: [(0, '4.631')] +[2024-11-11 14:15:24,551][03122] Saving new best policy, reward=4.631! +[2024-11-11 14:15:29,536][00562] Fps is (10 sec: 2867.2, 60 sec: 3481.7, 300 sec: 3113.0). Total num frames: 311296. Throughput: 0: 876.0. Samples: 77120. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:15:29,543][00562] Avg episode reward: [(0, '4.743')] +[2024-11-11 14:15:29,548][03122] Saving new best policy, reward=4.743! +[2024-11-11 14:15:33,030][03135] Updated weights for policy 0, policy_version 80 (0.0028) +[2024-11-11 14:15:34,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.4, 300 sec: 3159.8). Total num frames: 331776. Throughput: 0: 900.9. Samples: 83394. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:15:34,542][00562] Avg episode reward: [(0, '4.588')] +[2024-11-11 14:15:39,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3165.1). Total num frames: 348160. Throughput: 0: 926.0. Samples: 86564. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:15:39,543][00562] Avg episode reward: [(0, '4.451')] +[2024-11-11 14:15:44,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3169.9). Total num frames: 364544. Throughput: 0: 878.4. Samples: 90484. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:15:44,539][00562] Avg episode reward: [(0, '4.660')] +[2024-11-11 14:15:45,430][03135] Updated weights for policy 0, policy_version 90 (0.0023) +[2024-11-11 14:15:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3208.5). Total num frames: 385024. Throughput: 0: 872.9. Samples: 96376. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:15:49,538][00562] Avg episode reward: [(0, '4.720')] +[2024-11-11 14:15:54,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3244.0). Total num frames: 405504. Throughput: 0: 903.6. Samples: 99722. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:15:54,541][00562] Avg episode reward: [(0, '4.603')] +[2024-11-11 14:15:55,158][03135] Updated weights for policy 0, policy_version 100 (0.0013) +[2024-11-11 14:15:59,541][00562] Fps is (10 sec: 3275.1, 60 sec: 3481.3, 300 sec: 3213.7). Total num frames: 417792. Throughput: 0: 896.5. Samples: 104598. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:15:59,544][00562] Avg episode reward: [(0, '4.750')] +[2024-11-11 14:15:59,550][03122] Saving new best policy, reward=4.750! +[2024-11-11 14:16:04,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3246.5). Total num frames: 438272. Throughput: 0: 874.9. Samples: 109794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:16:04,539][00562] Avg episode reward: [(0, '4.947')] +[2024-11-11 14:16:04,547][03122] Saving new best policy, reward=4.947! +[2024-11-11 14:16:07,043][03135] Updated weights for policy 0, policy_version 110 (0.0031) +[2024-11-11 14:16:09,536][00562] Fps is (10 sec: 4098.1, 60 sec: 3618.1, 300 sec: 3276.8). Total num frames: 458752. Throughput: 0: 891.1. Samples: 112874. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:16:09,540][00562] Avg episode reward: [(0, '4.808')] +[2024-11-11 14:16:14,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3276.8). Total num frames: 475136. Throughput: 0: 918.0. Samples: 118428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:16:14,541][00562] Avg episode reward: [(0, '4.764')] +[2024-11-11 14:16:19,536][00562] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3249.5). Total num frames: 487424. Throughput: 0: 871.7. Samples: 122620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:16:19,543][00562] Avg episode reward: [(0, '4.668')] +[2024-11-11 14:16:19,670][03135] Updated weights for policy 0, policy_version 120 (0.0034) +[2024-11-11 14:16:24,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3303.2). Total num frames: 512000. Throughput: 0: 870.3. Samples: 125726. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:16:24,538][00562] Avg episode reward: [(0, '4.833')] +[2024-11-11 14:16:29,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3302.4). Total num frames: 528384. Throughput: 0: 927.5. Samples: 132222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:16:29,539][00562] Avg episode reward: [(0, '5.245')] +[2024-11-11 14:16:29,649][03135] Updated weights for policy 0, policy_version 130 (0.0021) +[2024-11-11 14:16:29,664][03122] Saving new best policy, reward=5.245! +[2024-11-11 14:16:34,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3301.6). Total num frames: 544768. Throughput: 0: 889.1. Samples: 136386. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:16:34,544][00562] Avg episode reward: [(0, '5.391')] +[2024-11-11 14:16:34,557][03122] Saving new best policy, reward=5.391! +[2024-11-11 14:16:39,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3325.0). Total num frames: 565248. Throughput: 0: 870.9. Samples: 138912. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:16:39,541][00562] Avg episode reward: [(0, '5.441')] +[2024-11-11 14:16:39,546][03122] Saving new best policy, reward=5.441! +[2024-11-11 14:16:41,376][03135] Updated weights for policy 0, policy_version 140 (0.0024) +[2024-11-11 14:16:44,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3347.0). Total num frames: 585728. Throughput: 0: 905.1. Samples: 145322. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:16:44,540][00562] Avg episode reward: [(0, '5.502')] +[2024-11-11 14:16:44,555][03122] Saving new best policy, reward=5.502! +[2024-11-11 14:16:49,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3322.3). Total num frames: 598016. Throughput: 0: 895.1. Samples: 150072. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:16:49,542][00562] Avg episode reward: [(0, '5.174')] +[2024-11-11 14:16:54,027][03135] Updated weights for policy 0, policy_version 150 (0.0026) +[2024-11-11 14:16:54,536][00562] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3321.1). Total num frames: 614400. Throughput: 0: 873.1. Samples: 152164. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:16:54,539][00562] Avg episode reward: [(0, '5.378')] +[2024-11-11 14:16:59,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.4, 300 sec: 3341.5). Total num frames: 634880. Throughput: 0: 884.6. Samples: 158236. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:16:59,545][00562] Avg episode reward: [(0, '5.539')] +[2024-11-11 14:16:59,620][03122] Saving new best policy, reward=5.539! +[2024-11-11 14:17:04,115][03135] Updated weights for policy 0, policy_version 160 (0.0030) +[2024-11-11 14:17:04,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3360.8). Total num frames: 655360. Throughput: 0: 921.9. Samples: 164106. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:17:04,540][00562] Avg episode reward: [(0, '5.798')] +[2024-11-11 14:17:04,552][03122] Saving new best policy, reward=5.798! +[2024-11-11 14:17:09,537][00562] Fps is (10 sec: 3276.5, 60 sec: 3481.6, 300 sec: 3338.2). Total num frames: 667648. Throughput: 0: 894.6. Samples: 165982. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:17:09,542][00562] Avg episode reward: [(0, '5.654')] +[2024-11-11 14:17:14,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3356.7). Total num frames: 688128. Throughput: 0: 870.3. Samples: 171386. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:17:14,542][00562] Avg episode reward: [(0, '5.711')] +[2024-11-11 14:17:14,552][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000168_688128.pth... +[2024-11-11 14:17:15,909][03135] Updated weights for policy 0, policy_version 170 (0.0018) +[2024-11-11 14:17:19,536][00562] Fps is (10 sec: 4096.3, 60 sec: 3686.4, 300 sec: 3374.3). Total num frames: 708608. Throughput: 0: 917.2. Samples: 177662. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:17:19,538][00562] Avg episode reward: [(0, '5.479')] +[2024-11-11 14:17:24,541][00562] Fps is (10 sec: 3684.5, 60 sec: 3549.6, 300 sec: 3372.0). Total num frames: 724992. Throughput: 0: 911.6. Samples: 179940. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:17:24,546][00562] Avg episode reward: [(0, '5.572')] +[2024-11-11 14:17:28,184][03135] Updated weights for policy 0, policy_version 180 (0.0019) +[2024-11-11 14:17:29,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3369.9). Total num frames: 741376. Throughput: 0: 866.4. Samples: 184310. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:17:29,541][00562] Avg episode reward: [(0, '5.582')] +[2024-11-11 14:17:34,536][00562] Fps is (10 sec: 3688.3, 60 sec: 3618.1, 300 sec: 3386.0). Total num frames: 761856. Throughput: 0: 903.1. Samples: 190712. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:17:34,540][00562] Avg episode reward: [(0, '5.400')] +[2024-11-11 14:17:38,535][03135] Updated weights for policy 0, policy_version 190 (0.0019) +[2024-11-11 14:17:39,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3383.7). Total num frames: 778240. Throughput: 0: 926.4. Samples: 193854. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:17:39,539][00562] Avg episode reward: [(0, '5.618')] +[2024-11-11 14:17:44,536][00562] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3363.9). Total num frames: 790528. Throughput: 0: 880.0. Samples: 197836. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:17:44,541][00562] Avg episode reward: [(0, '5.695')] +[2024-11-11 14:17:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3396.3). Total num frames: 815104. Throughput: 0: 879.5. Samples: 203684. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:17:49,539][00562] Avg episode reward: [(0, '6.146')] +[2024-11-11 14:17:49,544][03122] Saving new best policy, reward=6.146! +[2024-11-11 14:17:50,420][03135] Updated weights for policy 0, policy_version 200 (0.0033) +[2024-11-11 14:17:54,541][00562] Fps is (10 sec: 4503.4, 60 sec: 3686.1, 300 sec: 3410.5). Total num frames: 835584. Throughput: 0: 908.1. Samples: 206850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:17:54,544][00562] Avg episode reward: [(0, '6.938')] +[2024-11-11 14:17:54,556][03122] Saving new best policy, reward=6.938! +[2024-11-11 14:17:59,538][00562] Fps is (10 sec: 3276.1, 60 sec: 3549.7, 300 sec: 3391.5). Total num frames: 847872. Throughput: 0: 893.9. Samples: 211612. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:17:59,541][00562] Avg episode reward: [(0, '7.284')] +[2024-11-11 14:17:59,543][03122] Saving new best policy, reward=7.284! +[2024-11-11 14:18:02,770][03135] Updated weights for policy 0, policy_version 210 (0.0032) +[2024-11-11 14:18:04,537][00562] Fps is (10 sec: 2868.3, 60 sec: 3481.6, 300 sec: 3389.2). Total num frames: 864256. Throughput: 0: 866.4. Samples: 216650. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:18:04,539][00562] Avg episode reward: [(0, '7.289')] +[2024-11-11 14:18:04,553][03122] Saving new best policy, reward=7.289! +[2024-11-11 14:18:09,536][00562] Fps is (10 sec: 3687.0, 60 sec: 3618.2, 300 sec: 3402.8). Total num frames: 884736. Throughput: 0: 884.5. Samples: 219738. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:18:09,541][00562] Avg episode reward: [(0, '6.772')] +[2024-11-11 14:18:13,268][03135] Updated weights for policy 0, policy_version 220 (0.0029) +[2024-11-11 14:18:14,537][00562] Fps is (10 sec: 3686.5, 60 sec: 3549.8, 300 sec: 3400.4). Total num frames: 901120. Throughput: 0: 914.7. Samples: 225474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:18:14,544][00562] Avg episode reward: [(0, '6.888')] +[2024-11-11 14:18:19,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3398.2). Total num frames: 917504. Throughput: 0: 869.5. Samples: 229838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:18:19,542][00562] Avg episode reward: [(0, '6.874')] +[2024-11-11 14:18:24,536][00562] Fps is (10 sec: 3686.6, 60 sec: 3550.2, 300 sec: 3410.9). Total num frames: 937984. Throughput: 0: 871.2. Samples: 233058. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:18:24,543][00562] Avg episode reward: [(0, '7.313')] +[2024-11-11 14:18:24,564][03122] Saving new best policy, reward=7.313! +[2024-11-11 14:18:24,562][03135] Updated weights for policy 0, policy_version 230 (0.0019) +[2024-11-11 14:18:29,538][00562] Fps is (10 sec: 4095.4, 60 sec: 3618.0, 300 sec: 3423.1). Total num frames: 958464. Throughput: 0: 925.4. Samples: 239480. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:18:29,542][00562] Avg episode reward: [(0, '7.727')] +[2024-11-11 14:18:29,544][03122] Saving new best policy, reward=7.727! +[2024-11-11 14:18:34,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3406.1). Total num frames: 970752. Throughput: 0: 880.6. Samples: 243310. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:18:34,541][00562] Avg episode reward: [(0, '7.795')] +[2024-11-11 14:18:34,551][03122] Saving new best policy, reward=7.795! +[2024-11-11 14:18:37,011][03135] Updated weights for policy 0, policy_version 240 (0.0025) +[2024-11-11 14:18:39,536][00562] Fps is (10 sec: 3277.3, 60 sec: 3549.9, 300 sec: 3418.0). Total num frames: 991232. Throughput: 0: 873.4. Samples: 246148. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:18:39,541][00562] Avg episode reward: [(0, '8.393')] +[2024-11-11 14:18:39,544][03122] Saving new best policy, reward=8.393! +[2024-11-11 14:18:44,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3429.5). Total num frames: 1011712. Throughput: 0: 910.6. Samples: 252588. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:18:44,539][00562] Avg episode reward: [(0, '8.526')] +[2024-11-11 14:18:44,549][03122] Saving new best policy, reward=8.526! +[2024-11-11 14:18:48,068][03135] Updated weights for policy 0, policy_version 250 (0.0015) +[2024-11-11 14:18:49,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 1024000. Throughput: 0: 898.2. Samples: 257066. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:18:49,539][00562] Avg episode reward: [(0, '9.033')] +[2024-11-11 14:18:49,559][03122] Saving new best policy, reward=9.033! +[2024-11-11 14:18:54,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3481.9, 300 sec: 3540.6). Total num frames: 1044480. Throughput: 0: 874.0. Samples: 259066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:18:54,543][00562] Avg episode reward: [(0, '8.546')] +[2024-11-11 14:18:59,081][03135] Updated weights for policy 0, policy_version 260 (0.0025) +[2024-11-11 14:18:59,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3618.3, 300 sec: 3568.4). Total num frames: 1064960. Throughput: 0: 890.6. Samples: 265550. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:18:59,540][00562] Avg episode reward: [(0, '7.860')] +[2024-11-11 14:19:04,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3554.5). Total num frames: 1081344. Throughput: 0: 913.6. Samples: 270950. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:19:04,546][00562] Avg episode reward: [(0, '7.881')] +[2024-11-11 14:19:09,536][00562] Fps is (10 sec: 3276.7, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1097728. Throughput: 0: 887.1. Samples: 272976. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:19:09,538][00562] Avg episode reward: [(0, '8.809')] +[2024-11-11 14:19:11,099][03135] Updated weights for policy 0, policy_version 270 (0.0031) +[2024-11-11 14:19:14,536][00562] Fps is (10 sec: 3686.3, 60 sec: 3618.2, 300 sec: 3582.3). Total num frames: 1118208. Throughput: 0: 878.3. Samples: 279000. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:19:14,539][00562] Avg episode reward: [(0, '9.228')] +[2024-11-11 14:19:14,555][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000273_1118208.pth... +[2024-11-11 14:19:14,675][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000064_262144.pth +[2024-11-11 14:19:14,692][03122] Saving new best policy, reward=9.228! +[2024-11-11 14:19:19,536][00562] Fps is (10 sec: 3686.5, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 1134592. Throughput: 0: 920.3. Samples: 284724. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:19:19,540][00562] Avg episode reward: [(0, '9.538')] +[2024-11-11 14:19:19,604][03122] Saving new best policy, reward=9.538! +[2024-11-11 14:19:22,954][03135] Updated weights for policy 0, policy_version 280 (0.0018) +[2024-11-11 14:19:24,536][00562] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1150976. Throughput: 0: 898.6. Samples: 286586. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:19:24,540][00562] Avg episode reward: [(0, '9.325')] +[2024-11-11 14:19:29,536][00562] Fps is (10 sec: 3686.3, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 1171456. Throughput: 0: 873.1. Samples: 291876. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2024-11-11 14:19:29,538][00562] Avg episode reward: [(0, '9.553')] +[2024-11-11 14:19:29,542][03122] Saving new best policy, reward=9.553! +[2024-11-11 14:19:33,246][03135] Updated weights for policy 0, policy_version 290 (0.0019) +[2024-11-11 14:19:34,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1191936. Throughput: 0: 910.7. Samples: 298046. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:19:34,541][00562] Avg episode reward: [(0, '10.099')] +[2024-11-11 14:19:34,551][03122] Saving new best policy, reward=10.099! +[2024-11-11 14:19:39,541][00562] Fps is (10 sec: 3275.2, 60 sec: 3549.6, 300 sec: 3554.4). Total num frames: 1204224. Throughput: 0: 915.3. Samples: 300258. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:19:39,544][00562] Avg episode reward: [(0, '9.851')] +[2024-11-11 14:19:44,542][00562] Fps is (10 sec: 3274.9, 60 sec: 3549.5, 300 sec: 3582.2). Total num frames: 1224704. Throughput: 0: 873.6. Samples: 304868. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:19:44,544][00562] Avg episode reward: [(0, '9.566')] +[2024-11-11 14:19:45,424][03135] Updated weights for policy 0, policy_version 300 (0.0022) +[2024-11-11 14:19:49,536][00562] Fps is (10 sec: 4098.1, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 1245184. Throughput: 0: 899.8. Samples: 311440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:19:49,544][00562] Avg episode reward: [(0, '9.843')] +[2024-11-11 14:19:54,536][00562] Fps is (10 sec: 3688.6, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 1261568. Throughput: 0: 932.5. Samples: 314936. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:19:54,539][00562] Avg episode reward: [(0, '9.722')] +[2024-11-11 14:19:56,009][03135] Updated weights for policy 0, policy_version 310 (0.0016) +[2024-11-11 14:19:59,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 1277952. Throughput: 0: 893.7. Samples: 319216. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:19:59,538][00562] Avg episode reward: [(0, '9.716')] +[2024-11-11 14:20:04,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1302528. Throughput: 0: 914.5. Samples: 325876. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:20:04,541][00562] Avg episode reward: [(0, '10.107')] +[2024-11-11 14:20:04,556][03122] Saving new best policy, reward=10.107! +[2024-11-11 14:20:05,974][03135] Updated weights for policy 0, policy_version 320 (0.0017) +[2024-11-11 14:20:09,539][00562] Fps is (10 sec: 4504.2, 60 sec: 3754.5, 300 sec: 3596.1). Total num frames: 1323008. Throughput: 0: 949.0. Samples: 329296. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:20:09,541][00562] Avg episode reward: [(0, '11.050')] +[2024-11-11 14:20:09,552][03122] Saving new best policy, reward=11.050! +[2024-11-11 14:20:14,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 1335296. Throughput: 0: 941.1. Samples: 334226. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) +[2024-11-11 14:20:14,539][00562] Avg episode reward: [(0, '11.349')] +[2024-11-11 14:20:14,558][03122] Saving new best policy, reward=11.349! +[2024-11-11 14:20:17,364][03135] Updated weights for policy 0, policy_version 330 (0.0037) +[2024-11-11 14:20:19,536][00562] Fps is (10 sec: 3687.5, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 1359872. Throughput: 0: 932.2. Samples: 339996. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:20:19,538][00562] Avg episode reward: [(0, '11.071')] +[2024-11-11 14:20:24,536][00562] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3637.8). Total num frames: 1384448. Throughput: 0: 961.4. Samples: 343514. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:20:24,538][00562] Avg episode reward: [(0, '10.360')] +[2024-11-11 14:20:26,823][03135] Updated weights for policy 0, policy_version 340 (0.0031) +[2024-11-11 14:20:29,536][00562] Fps is (10 sec: 3686.3, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 1396736. Throughput: 0: 988.7. Samples: 349354. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:20:29,546][00562] Avg episode reward: [(0, '10.946')] +[2024-11-11 14:20:34,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3623.9). Total num frames: 1417216. Throughput: 0: 961.4. Samples: 354702. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:20:34,538][00562] Avg episode reward: [(0, '11.230')] +[2024-11-11 14:20:37,353][03135] Updated weights for policy 0, policy_version 350 (0.0020) +[2024-11-11 14:20:39,536][00562] Fps is (10 sec: 4505.7, 60 sec: 3959.8, 300 sec: 3651.7). Total num frames: 1441792. Throughput: 0: 960.0. Samples: 358138. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:20:39,538][00562] Avg episode reward: [(0, '13.045')] +[2024-11-11 14:20:39,542][03122] Saving new best policy, reward=13.045! +[2024-11-11 14:20:44,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3891.6, 300 sec: 3637.8). Total num frames: 1458176. Throughput: 0: 1009.6. Samples: 364648. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:20:44,538][00562] Avg episode reward: [(0, '14.305')] +[2024-11-11 14:20:44,555][03122] Saving new best policy, reward=14.305! +[2024-11-11 14:20:48,981][03135] Updated weights for policy 0, policy_version 360 (0.0027) +[2024-11-11 14:20:49,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3623.9). Total num frames: 1474560. Throughput: 0: 958.2. Samples: 368996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:20:49,538][00562] Avg episode reward: [(0, '14.481')] +[2024-11-11 14:20:49,540][03122] Saving new best policy, reward=14.481! +[2024-11-11 14:20:54,536][00562] Fps is (10 sec: 4095.9, 60 sec: 3959.4, 300 sec: 3665.6). Total num frames: 1499136. Throughput: 0: 958.9. Samples: 372442. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:20:54,543][00562] Avg episode reward: [(0, '14.633')] +[2024-11-11 14:20:54,551][03122] Saving new best policy, reward=14.633! +[2024-11-11 14:20:57,890][03135] Updated weights for policy 0, policy_version 370 (0.0025) +[2024-11-11 14:20:59,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3665.6). Total num frames: 1519616. Throughput: 0: 1002.1. Samples: 379322. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:20:59,542][00562] Avg episode reward: [(0, '13.241')] +[2024-11-11 14:21:04,536][00562] Fps is (10 sec: 3276.9, 60 sec: 3822.9, 300 sec: 3637.8). Total num frames: 1531904. Throughput: 0: 974.9. Samples: 383868. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:21:04,539][00562] Avg episode reward: [(0, '13.089')] +[2024-11-11 14:21:09,174][03135] Updated weights for policy 0, policy_version 380 (0.0021) +[2024-11-11 14:21:09,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.4, 300 sec: 3665.6). Total num frames: 1556480. Throughput: 0: 962.0. Samples: 386802. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:21:09,539][00562] Avg episode reward: [(0, '13.534')] +[2024-11-11 14:21:14,536][00562] Fps is (10 sec: 4915.2, 60 sec: 4096.0, 300 sec: 3707.2). Total num frames: 1581056. Throughput: 0: 989.8. Samples: 393894. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:21:14,538][00562] Avg episode reward: [(0, '14.577')] +[2024-11-11 14:21:14,546][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000386_1581056.pth... +[2024-11-11 14:21:14,667][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000168_688128.pth +[2024-11-11 14:21:19,540][00562] Fps is (10 sec: 3684.8, 60 sec: 3890.9, 300 sec: 3665.5). Total num frames: 1593344. Throughput: 0: 988.3. Samples: 399180. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:21:19,546][00562] Avg episode reward: [(0, '15.060')] +[2024-11-11 14:21:19,628][03135] Updated weights for policy 0, policy_version 390 (0.0020) +[2024-11-11 14:21:19,632][03122] Saving new best policy, reward=15.060! +[2024-11-11 14:21:24,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 1613824. Throughput: 0: 960.8. Samples: 401372. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:21:24,539][00562] Avg episode reward: [(0, '15.528')] +[2024-11-11 14:21:24,549][03122] Saving new best policy, reward=15.528! +[2024-11-11 14:21:29,315][03135] Updated weights for policy 0, policy_version 400 (0.0027) +[2024-11-11 14:21:29,536][00562] Fps is (10 sec: 4507.6, 60 sec: 4027.7, 300 sec: 3707.2). Total num frames: 1638400. Throughput: 0: 969.6. Samples: 408280. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:21:29,544][00562] Avg episode reward: [(0, '14.303')] +[2024-11-11 14:21:34,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3693.3). Total num frames: 1654784. Throughput: 0: 1011.9. Samples: 414530. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:21:34,542][00562] Avg episode reward: [(0, '12.693')] +[2024-11-11 14:21:39,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3679.5). Total num frames: 1671168. Throughput: 0: 983.8. Samples: 416712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:21:39,542][00562] Avg episode reward: [(0, '11.913')] +[2024-11-11 14:21:40,711][03135] Updated weights for policy 0, policy_version 410 (0.0020) +[2024-11-11 14:21:44,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3721.1). Total num frames: 1695744. Throughput: 0: 969.8. Samples: 422964. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:21:44,539][00562] Avg episode reward: [(0, '12.488')] +[2024-11-11 14:21:49,537][00562] Fps is (10 sec: 4505.1, 60 sec: 4027.7, 300 sec: 3735.0). Total num frames: 1716224. Throughput: 0: 1025.6. Samples: 430020. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:21:49,540][00562] Avg episode reward: [(0, '13.037')] +[2024-11-11 14:21:49,650][03135] Updated weights for policy 0, policy_version 420 (0.0014) +[2024-11-11 14:21:54,542][00562] Fps is (10 sec: 3684.2, 60 sec: 3890.8, 300 sec: 3721.0). Total num frames: 1732608. Throughput: 0: 1008.1. Samples: 432174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:21:54,551][00562] Avg episode reward: [(0, '13.699')] +[2024-11-11 14:21:59,536][00562] Fps is (10 sec: 3686.8, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 1753088. Throughput: 0: 967.2. Samples: 437418. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:21:59,539][00562] Avg episode reward: [(0, '15.479')] +[2024-11-11 14:22:01,010][03135] Updated weights for policy 0, policy_version 430 (0.0020) +[2024-11-11 14:22:04,537][00562] Fps is (10 sec: 4507.9, 60 sec: 4095.9, 300 sec: 3762.8). Total num frames: 1777664. Throughput: 0: 998.2. Samples: 444096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:22:04,544][00562] Avg episode reward: [(0, '15.164')] +[2024-11-11 14:22:09,537][00562] Fps is (10 sec: 3685.8, 60 sec: 3891.1, 300 sec: 3735.0). Total num frames: 1789952. Throughput: 0: 1012.5. Samples: 446936. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:22:09,543][00562] Avg episode reward: [(0, '15.847')] +[2024-11-11 14:22:09,548][03122] Saving new best policy, reward=15.847! +[2024-11-11 14:22:13,001][03135] Updated weights for policy 0, policy_version 440 (0.0038) +[2024-11-11 14:22:14,536][00562] Fps is (10 sec: 2867.5, 60 sec: 3754.7, 300 sec: 3721.1). Total num frames: 1806336. Throughput: 0: 948.5. Samples: 450962. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:22:14,539][00562] Avg episode reward: [(0, '15.502')] +[2024-11-11 14:22:19,536][00562] Fps is (10 sec: 4096.6, 60 sec: 3959.8, 300 sec: 3748.9). Total num frames: 1830912. Throughput: 0: 954.4. Samples: 457478. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:22:19,542][00562] Avg episode reward: [(0, '14.958')] +[2024-11-11 14:22:22,092][03135] Updated weights for policy 0, policy_version 450 (0.0028) +[2024-11-11 14:22:24,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 1847296. Throughput: 0: 982.1. Samples: 460906. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:22:24,540][00562] Avg episode reward: [(0, '15.776')] +[2024-11-11 14:22:29,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3735.0). Total num frames: 1863680. Throughput: 0: 939.7. Samples: 465250. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:22:29,538][00562] Avg episode reward: [(0, '16.064')] +[2024-11-11 14:22:29,540][03122] Saving new best policy, reward=16.064! +[2024-11-11 14:22:34,004][03135] Updated weights for policy 0, policy_version 460 (0.0025) +[2024-11-11 14:22:34,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1884160. Throughput: 0: 916.6. Samples: 471266. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:22:34,538][00562] Avg episode reward: [(0, '15.748')] +[2024-11-11 14:22:39,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 1904640. Throughput: 0: 943.1. Samples: 474608. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:22:39,541][00562] Avg episode reward: [(0, '15.982')] +[2024-11-11 14:22:44,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3748.9). Total num frames: 1921024. Throughput: 0: 939.2. Samples: 479680. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:22:44,541][00562] Avg episode reward: [(0, '14.914')] +[2024-11-11 14:22:45,999][03135] Updated weights for policy 0, policy_version 470 (0.0026) +[2024-11-11 14:22:49,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3686.5, 300 sec: 3735.1). Total num frames: 1937408. Throughput: 0: 906.5. Samples: 484888. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:22:49,539][00562] Avg episode reward: [(0, '14.322')] +[2024-11-11 14:22:54,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3823.3, 300 sec: 3776.7). Total num frames: 1961984. Throughput: 0: 916.4. Samples: 488174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:22:54,538][00562] Avg episode reward: [(0, '12.973')] +[2024-11-11 14:22:55,248][03135] Updated weights for policy 0, policy_version 480 (0.0021) +[2024-11-11 14:22:59,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 1978368. Throughput: 0: 959.3. Samples: 494130. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:22:59,539][00562] Avg episode reward: [(0, '13.627')] +[2024-11-11 14:23:04,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3762.8). Total num frames: 1994752. Throughput: 0: 918.2. Samples: 498798. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:23:04,543][00562] Avg episode reward: [(0, '15.122')] +[2024-11-11 14:23:06,911][03135] Updated weights for policy 0, policy_version 490 (0.0036) +[2024-11-11 14:23:09,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3754.8, 300 sec: 3776.7). Total num frames: 2015232. Throughput: 0: 917.5. Samples: 502194. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:23:09,542][00562] Avg episode reward: [(0, '16.168')] +[2024-11-11 14:23:09,550][03122] Saving new best policy, reward=16.168! +[2024-11-11 14:23:14,538][00562] Fps is (10 sec: 4095.1, 60 sec: 3822.8, 300 sec: 3790.5). Total num frames: 2035712. Throughput: 0: 969.3. Samples: 508872. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) +[2024-11-11 14:23:14,545][00562] Avg episode reward: [(0, '17.297')] +[2024-11-11 14:23:14,560][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000497_2035712.pth... +[2024-11-11 14:23:14,723][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000273_1118208.pth +[2024-11-11 14:23:14,744][03122] Saving new best policy, reward=17.297! +[2024-11-11 14:23:18,676][03135] Updated weights for policy 0, policy_version 500 (0.0024) +[2024-11-11 14:23:19,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 2048000. Throughput: 0: 918.7. Samples: 512608. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:23:19,539][00562] Avg episode reward: [(0, '18.508')] +[2024-11-11 14:23:19,553][03122] Saving new best policy, reward=18.508! +[2024-11-11 14:23:24,536][00562] Fps is (10 sec: 3687.2, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 2072576. Throughput: 0: 910.7. Samples: 515588. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:23:24,538][00562] Avg episode reward: [(0, '17.975')] +[2024-11-11 14:23:28,016][03135] Updated weights for policy 0, policy_version 510 (0.0013) +[2024-11-11 14:23:29,537][00562] Fps is (10 sec: 4505.0, 60 sec: 3822.8, 300 sec: 3804.4). Total num frames: 2093056. Throughput: 0: 950.5. Samples: 522452. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:23:29,545][00562] Avg episode reward: [(0, '17.632')] +[2024-11-11 14:23:34,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3776.7). Total num frames: 2105344. Throughput: 0: 944.7. Samples: 527398. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:23:34,543][00562] Avg episode reward: [(0, '18.036')] +[2024-11-11 14:23:39,536][00562] Fps is (10 sec: 3277.3, 60 sec: 3686.4, 300 sec: 3776.7). Total num frames: 2125824. Throughput: 0: 923.7. Samples: 529740. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:23:39,542][00562] Avg episode reward: [(0, '18.040')] +[2024-11-11 14:23:39,830][03135] Updated weights for policy 0, policy_version 520 (0.0023) +[2024-11-11 14:23:44,536][00562] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2150400. Throughput: 0: 939.6. Samples: 536414. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:23:44,542][00562] Avg episode reward: [(0, '18.160')] +[2024-11-11 14:23:49,536][00562] Fps is (10 sec: 4095.9, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 2166784. Throughput: 0: 958.8. Samples: 541944. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:23:49,542][00562] Avg episode reward: [(0, '19.216')] +[2024-11-11 14:23:49,543][03122] Saving new best policy, reward=19.216! +[2024-11-11 14:23:50,734][03135] Updated weights for policy 0, policy_version 530 (0.0014) +[2024-11-11 14:23:54,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3790.5). Total num frames: 2183168. Throughput: 0: 926.4. Samples: 543880. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:23:54,543][00562] Avg episode reward: [(0, '19.125')] +[2024-11-11 14:23:59,536][00562] Fps is (10 sec: 3686.5, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 2203648. Throughput: 0: 915.8. Samples: 550082. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:23:59,540][00562] Avg episode reward: [(0, '18.306')] +[2024-11-11 14:24:00,831][03135] Updated weights for policy 0, policy_version 540 (0.0028) +[2024-11-11 14:24:04,536][00562] Fps is (10 sec: 4095.9, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2224128. Throughput: 0: 982.9. Samples: 556838. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:24:04,538][00562] Avg episode reward: [(0, '17.906')] +[2024-11-11 14:24:09,536][00562] Fps is (10 sec: 3686.3, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 2240512. Throughput: 0: 962.2. Samples: 558888. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:24:09,541][00562] Avg episode reward: [(0, '17.494')] +[2024-11-11 14:24:12,425][03135] Updated weights for policy 0, policy_version 550 (0.0024) +[2024-11-11 14:24:14,536][00562] Fps is (10 sec: 3686.5, 60 sec: 3754.8, 300 sec: 3818.3). Total num frames: 2260992. Throughput: 0: 928.3. Samples: 564222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:24:14,543][00562] Avg episode reward: [(0, '17.795')] +[2024-11-11 14:24:19,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 2281472. Throughput: 0: 973.9. Samples: 571224. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:24:19,544][00562] Avg episode reward: [(0, '17.263')] +[2024-11-11 14:24:22,313][03135] Updated weights for policy 0, policy_version 560 (0.0018) +[2024-11-11 14:24:24,537][00562] Fps is (10 sec: 3686.1, 60 sec: 3754.6, 300 sec: 3818.3). Total num frames: 2297856. Throughput: 0: 978.9. Samples: 573792. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:24:24,539][00562] Avg episode reward: [(0, '17.846')] +[2024-11-11 14:24:29,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 2318336. Throughput: 0: 937.1. Samples: 578584. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:24:29,544][00562] Avg episode reward: [(0, '18.910')] +[2024-11-11 14:24:32,729][03135] Updated weights for policy 0, policy_version 570 (0.0031) +[2024-11-11 14:24:34,536][00562] Fps is (10 sec: 4506.0, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 2342912. Throughput: 0: 968.3. Samples: 585518. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:24:34,539][00562] Avg episode reward: [(0, '19.948')] +[2024-11-11 14:24:34,549][03122] Saving new best policy, reward=19.948! +[2024-11-11 14:24:39,536][00562] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3846.2). Total num frames: 2359296. Throughput: 0: 999.1. Samples: 588840. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:24:39,549][00562] Avg episode reward: [(0, '19.368')] +[2024-11-11 14:24:44,415][03135] Updated weights for policy 0, policy_version 580 (0.0024) +[2024-11-11 14:24:44,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3832.2). Total num frames: 2375680. Throughput: 0: 955.0. Samples: 593056. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:24:44,538][00562] Avg episode reward: [(0, '20.561')] +[2024-11-11 14:24:44,546][03122] Saving new best policy, reward=20.561! +[2024-11-11 14:24:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 2396160. Throughput: 0: 951.2. Samples: 599644. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:24:49,538][00562] Avg episode reward: [(0, '20.524')] +[2024-11-11 14:24:53,193][03135] Updated weights for policy 0, policy_version 590 (0.0025) +[2024-11-11 14:24:54,536][00562] Fps is (10 sec: 4505.5, 60 sec: 3959.4, 300 sec: 3873.8). Total num frames: 2420736. Throughput: 0: 980.6. Samples: 603014. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:24:54,540][00562] Avg episode reward: [(0, '19.929')] +[2024-11-11 14:24:59,542][00562] Fps is (10 sec: 3684.2, 60 sec: 3822.5, 300 sec: 3832.1). Total num frames: 2433024. Throughput: 0: 975.2. Samples: 608110. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:24:59,545][00562] Avg episode reward: [(0, '19.025')] +[2024-11-11 14:25:04,536][00562] Fps is (10 sec: 3276.9, 60 sec: 3822.9, 300 sec: 3832.2). Total num frames: 2453504. Throughput: 0: 950.8. Samples: 614012. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:25:04,538][00562] Avg episode reward: [(0, '18.559')] +[2024-11-11 14:25:04,664][03135] Updated weights for policy 0, policy_version 600 (0.0028) +[2024-11-11 14:25:09,536][00562] Fps is (10 sec: 4508.4, 60 sec: 3959.5, 300 sec: 3873.8). Total num frames: 2478080. Throughput: 0: 973.1. Samples: 617580. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:25:09,538][00562] Avg episode reward: [(0, '18.975')] +[2024-11-11 14:25:14,538][00562] Fps is (10 sec: 4095.3, 60 sec: 3891.1, 300 sec: 3846.1). Total num frames: 2494464. Throughput: 0: 995.4. Samples: 623378. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:25:14,543][00562] Avg episode reward: [(0, '19.347')] +[2024-11-11 14:25:14,559][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000609_2494464.pth... +[2024-11-11 14:25:14,724][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000386_1581056.pth +[2024-11-11 14:25:15,624][03135] Updated weights for policy 0, policy_version 610 (0.0032) +[2024-11-11 14:25:19,538][00562] Fps is (10 sec: 3276.2, 60 sec: 3822.8, 300 sec: 3818.3). Total num frames: 2510848. Throughput: 0: 952.2. Samples: 628368. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:25:19,542][00562] Avg episode reward: [(0, '18.871')] +[2024-11-11 14:25:24,536][00562] Fps is (10 sec: 4096.7, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 2535424. Throughput: 0: 947.9. Samples: 631496. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:25:24,542][00562] Avg episode reward: [(0, '19.056')] +[2024-11-11 14:25:24,980][03135] Updated weights for policy 0, policy_version 620 (0.0026) +[2024-11-11 14:25:29,538][00562] Fps is (10 sec: 4505.5, 60 sec: 3959.4, 300 sec: 3859.9). Total num frames: 2555904. Throughput: 0: 1006.5. Samples: 638350. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:25:29,540][00562] Avg episode reward: [(0, '18.963')] +[2024-11-11 14:25:34,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 2568192. Throughput: 0: 954.9. Samples: 642616. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:25:34,539][00562] Avg episode reward: [(0, '18.796')] +[2024-11-11 14:25:36,392][03135] Updated weights for policy 0, policy_version 630 (0.0023) +[2024-11-11 14:25:39,536][00562] Fps is (10 sec: 3687.1, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2592768. Throughput: 0: 958.7. Samples: 646154. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:25:39,538][00562] Avg episode reward: [(0, '19.304')] +[2024-11-11 14:25:44,536][00562] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3873.8). Total num frames: 2617344. Throughput: 0: 1002.5. Samples: 653216. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:25:44,539][00562] Avg episode reward: [(0, '19.259')] +[2024-11-11 14:25:45,777][03135] Updated weights for policy 0, policy_version 640 (0.0039) +[2024-11-11 14:25:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 2629632. Throughput: 0: 975.9. Samples: 657928. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:25:49,539][00562] Avg episode reward: [(0, '19.747')] +[2024-11-11 14:25:54,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3823.0, 300 sec: 3832.2). Total num frames: 2650112. Throughput: 0: 956.8. Samples: 660636. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:25:54,543][00562] Avg episode reward: [(0, '19.441')] +[2024-11-11 14:25:56,442][03135] Updated weights for policy 0, policy_version 650 (0.0032) +[2024-11-11 14:25:59,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4028.1, 300 sec: 3873.8). Total num frames: 2674688. Throughput: 0: 984.3. Samples: 667670. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:25:59,538][00562] Avg episode reward: [(0, '19.029')] +[2024-11-11 14:26:04,539][00562] Fps is (10 sec: 4094.8, 60 sec: 3959.3, 300 sec: 3846.0). Total num frames: 2691072. Throughput: 0: 996.0. Samples: 673188. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-11-11 14:26:04,542][00562] Avg episode reward: [(0, '19.359')] +[2024-11-11 14:26:07,949][03135] Updated weights for policy 0, policy_version 660 (0.0031) +[2024-11-11 14:26:09,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2707456. Throughput: 0: 973.6. Samples: 675308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:26:09,538][00562] Avg episode reward: [(0, '19.579')] +[2024-11-11 14:26:14,536][00562] Fps is (10 sec: 4097.2, 60 sec: 3959.6, 300 sec: 3860.0). Total num frames: 2732032. Throughput: 0: 973.4. Samples: 682150. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:26:14,546][00562] Avg episode reward: [(0, '18.167')] +[2024-11-11 14:26:16,721][03135] Updated weights for policy 0, policy_version 670 (0.0020) +[2024-11-11 14:26:19,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.9, 300 sec: 3860.0). Total num frames: 2752512. Throughput: 0: 1017.9. Samples: 688422. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:26:19,538][00562] Avg episode reward: [(0, '18.763')] +[2024-11-11 14:26:24,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 2768896. Throughput: 0: 986.2. Samples: 690532. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:26:24,538][00562] Avg episode reward: [(0, '18.673')] +[2024-11-11 14:26:27,909][03135] Updated weights for policy 0, policy_version 680 (0.0024) +[2024-11-11 14:26:29,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.3, 300 sec: 3846.1). Total num frames: 2789376. Throughput: 0: 963.3. Samples: 696564. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:26:29,544][00562] Avg episode reward: [(0, '17.968')] +[2024-11-11 14:26:34,537][00562] Fps is (10 sec: 4505.1, 60 sec: 4095.9, 300 sec: 3873.8). Total num frames: 2813952. Throughput: 0: 1016.3. Samples: 703662. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:26:34,540][00562] Avg episode reward: [(0, '18.548')] +[2024-11-11 14:26:38,381][03135] Updated weights for policy 0, policy_version 690 (0.0019) +[2024-11-11 14:26:39,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 2826240. Throughput: 0: 1006.8. Samples: 705944. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:26:39,541][00562] Avg episode reward: [(0, '18.296')] +[2024-11-11 14:26:44,536][00562] Fps is (10 sec: 3277.2, 60 sec: 3822.9, 300 sec: 3832.2). Total num frames: 2846720. Throughput: 0: 965.7. Samples: 711128. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:26:44,541][00562] Avg episode reward: [(0, '17.230')] +[2024-11-11 14:26:48,080][03135] Updated weights for policy 0, policy_version 700 (0.0031) +[2024-11-11 14:26:49,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 2871296. Throughput: 0: 1000.8. Samples: 718222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:26:49,545][00562] Avg episode reward: [(0, '17.083')] +[2024-11-11 14:26:54,537][00562] Fps is (10 sec: 4095.5, 60 sec: 3959.4, 300 sec: 3846.1). Total num frames: 2887680. Throughput: 0: 1019.0. Samples: 721164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:26:54,543][00562] Avg episode reward: [(0, '17.814')] +[2024-11-11 14:26:59,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2904064. Throughput: 0: 965.1. Samples: 725578. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:26:59,543][00562] Avg episode reward: [(0, '18.477')] +[2024-11-11 14:26:59,565][03135] Updated weights for policy 0, policy_version 710 (0.0029) +[2024-11-11 14:27:04,543][00562] Fps is (10 sec: 4093.5, 60 sec: 3959.2, 300 sec: 3859.9). Total num frames: 2928640. Throughput: 0: 983.2. Samples: 732674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:27:04,549][00562] Avg episode reward: [(0, '19.283')] +[2024-11-11 14:27:08,764][03135] Updated weights for policy 0, policy_version 720 (0.0028) +[2024-11-11 14:27:09,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3873.8). Total num frames: 2949120. Throughput: 0: 1014.0. Samples: 736160. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:27:09,540][00562] Avg episode reward: [(0, '20.200')] +[2024-11-11 14:27:14,536][00562] Fps is (10 sec: 3689.1, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2965504. Throughput: 0: 984.0. Samples: 740842. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:27:14,538][00562] Avg episode reward: [(0, '18.441')] +[2024-11-11 14:27:14,550][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000724_2965504.pth... +[2024-11-11 14:27:14,671][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000497_2035712.pth +[2024-11-11 14:27:19,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3860.0). Total num frames: 2985984. Throughput: 0: 966.7. Samples: 747164. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:27:19,538][00562] Avg episode reward: [(0, '18.515')] +[2024-11-11 14:27:19,552][03135] Updated weights for policy 0, policy_version 730 (0.0013) +[2024-11-11 14:27:24,537][00562] Fps is (10 sec: 4504.9, 60 sec: 4027.6, 300 sec: 3887.7). Total num frames: 3010560. Throughput: 0: 987.8. Samples: 750398. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:27:24,542][00562] Avg episode reward: [(0, '18.392')] +[2024-11-11 14:27:29,537][00562] Fps is (10 sec: 3686.0, 60 sec: 3891.1, 300 sec: 3859.9). Total num frames: 3022848. Throughput: 0: 991.1. Samples: 755730. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:27:29,539][00562] Avg episode reward: [(0, '17.663')] +[2024-11-11 14:27:31,246][03135] Updated weights for policy 0, policy_version 740 (0.0023) +[2024-11-11 14:27:34,536][00562] Fps is (10 sec: 3277.3, 60 sec: 3823.0, 300 sec: 3860.0). Total num frames: 3043328. Throughput: 0: 960.1. Samples: 761426. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:27:34,544][00562] Avg episode reward: [(0, '18.847')] +[2024-11-11 14:27:39,536][00562] Fps is (10 sec: 4506.0, 60 sec: 4027.7, 300 sec: 3887.7). Total num frames: 3067904. Throughput: 0: 973.3. Samples: 764960. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:27:39,543][00562] Avg episode reward: [(0, '18.547')] +[2024-11-11 14:27:39,839][03135] Updated weights for policy 0, policy_version 750 (0.0013) +[2024-11-11 14:27:44,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3887.7). Total num frames: 3084288. Throughput: 0: 1009.1. Samples: 770986. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:27:44,540][00562] Avg episode reward: [(0, '19.671')] +[2024-11-11 14:27:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3873.8). Total num frames: 3104768. Throughput: 0: 962.2. Samples: 775968. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:27:49,537][00562] Avg episode reward: [(0, '20.647')] +[2024-11-11 14:27:49,540][03122] Saving new best policy, reward=20.647! +[2024-11-11 14:27:51,277][03135] Updated weights for policy 0, policy_version 760 (0.0017) +[2024-11-11 14:27:54,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3887.7). Total num frames: 3125248. Throughput: 0: 958.9. Samples: 779310. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:27:54,543][00562] Avg episode reward: [(0, '20.763')] +[2024-11-11 14:27:54,555][03122] Saving new best policy, reward=20.763! +[2024-11-11 14:27:59,537][00562] Fps is (10 sec: 4095.6, 60 sec: 4027.7, 300 sec: 3901.6). Total num frames: 3145728. Throughput: 0: 1007.4. Samples: 786178. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:27:59,539][00562] Avg episode reward: [(0, '20.604')] +[2024-11-11 14:28:01,918][03135] Updated weights for policy 0, policy_version 770 (0.0014) +[2024-11-11 14:28:04,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.7, 300 sec: 3887.7). Total num frames: 3162112. Throughput: 0: 961.4. Samples: 790426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:28:04,539][00562] Avg episode reward: [(0, '19.923')] +[2024-11-11 14:28:09,536][00562] Fps is (10 sec: 3686.8, 60 sec: 3891.2, 300 sec: 3887.8). Total num frames: 3182592. Throughput: 0: 964.2. Samples: 793784. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:28:09,543][00562] Avg episode reward: [(0, '18.911')] +[2024-11-11 14:28:11,370][03135] Updated weights for policy 0, policy_version 780 (0.0016) +[2024-11-11 14:28:14,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3929.4). Total num frames: 3207168. Throughput: 0: 1001.4. Samples: 800794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:28:14,538][00562] Avg episode reward: [(0, '19.310')] +[2024-11-11 14:28:19,538][00562] Fps is (10 sec: 3685.7, 60 sec: 3891.1, 300 sec: 3887.7). Total num frames: 3219456. Throughput: 0: 982.7. Samples: 805648. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:28:19,543][00562] Avg episode reward: [(0, '19.464')] +[2024-11-11 14:28:23,046][03135] Updated weights for policy 0, policy_version 790 (0.0027) +[2024-11-11 14:28:24,536][00562] Fps is (10 sec: 3276.7, 60 sec: 3823.0, 300 sec: 3887.7). Total num frames: 3239936. Throughput: 0: 956.8. Samples: 808016. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:28:24,539][00562] Avg episode reward: [(0, '20.023')] +[2024-11-11 14:28:29,536][00562] Fps is (10 sec: 4506.4, 60 sec: 4027.8, 300 sec: 3929.4). Total num frames: 3264512. Throughput: 0: 978.3. Samples: 815008. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-11-11 14:28:29,538][00562] Avg episode reward: [(0, '20.631')] +[2024-11-11 14:28:32,363][03135] Updated weights for policy 0, policy_version 800 (0.0015) +[2024-11-11 14:28:34,536][00562] Fps is (10 sec: 4096.1, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 3280896. Throughput: 0: 995.2. Samples: 820750. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:28:34,542][00562] Avg episode reward: [(0, '21.092')] +[2024-11-11 14:28:34,561][03122] Saving new best policy, reward=21.092! +[2024-11-11 14:28:39,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3887.7). Total num frames: 3297280. Throughput: 0: 966.7. Samples: 822812. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:28:39,538][00562] Avg episode reward: [(0, '20.886')] +[2024-11-11 14:28:43,414][03135] Updated weights for policy 0, policy_version 810 (0.0022) +[2024-11-11 14:28:44,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 3321856. Throughput: 0: 957.8. Samples: 829276. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:28:44,542][00562] Avg episode reward: [(0, '19.328')] +[2024-11-11 14:28:49,536][00562] Fps is (10 sec: 4505.5, 60 sec: 3959.4, 300 sec: 3929.4). Total num frames: 3342336. Throughput: 0: 1011.5. Samples: 835946. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:28:49,540][00562] Avg episode reward: [(0, '20.569')] +[2024-11-11 14:28:54,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 3354624. Throughput: 0: 983.5. Samples: 838042. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:28:54,540][00562] Avg episode reward: [(0, '20.043')] +[2024-11-11 14:28:54,723][03135] Updated weights for policy 0, policy_version 820 (0.0017) +[2024-11-11 14:28:59,536][00562] Fps is (10 sec: 3686.5, 60 sec: 3891.3, 300 sec: 3915.5). Total num frames: 3379200. Throughput: 0: 953.2. Samples: 843686. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:28:59,542][00562] Avg episode reward: [(0, '18.447')] +[2024-11-11 14:29:03,547][03135] Updated weights for policy 0, policy_version 830 (0.0029) +[2024-11-11 14:29:04,536][00562] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 3403776. Throughput: 0: 1002.9. Samples: 850778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:29:04,541][00562] Avg episode reward: [(0, '18.528')] +[2024-11-11 14:29:09,536][00562] Fps is (10 sec: 3686.3, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 3416064. Throughput: 0: 1007.3. Samples: 853346. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:29:09,545][00562] Avg episode reward: [(0, '19.173')] +[2024-11-11 14:29:14,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 3436544. Throughput: 0: 962.7. Samples: 858330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:29:14,543][00562] Avg episode reward: [(0, '18.941')] +[2024-11-11 14:29:14,552][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000839_3436544.pth... +[2024-11-11 14:29:14,688][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000609_2494464.pth +[2024-11-11 14:29:14,788][03135] Updated weights for policy 0, policy_version 840 (0.0025) +[2024-11-11 14:29:19,536][00562] Fps is (10 sec: 4505.7, 60 sec: 4027.9, 300 sec: 3943.3). Total num frames: 3461120. Throughput: 0: 989.2. Samples: 865264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:29:19,542][00562] Avg episode reward: [(0, '20.902')] +[2024-11-11 14:29:24,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 3477504. Throughput: 0: 1016.5. Samples: 868554. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:29:24,542][00562] Avg episode reward: [(0, '21.155')] +[2024-11-11 14:29:24,553][03122] Saving new best policy, reward=21.155! +[2024-11-11 14:29:25,327][03135] Updated weights for policy 0, policy_version 850 (0.0025) +[2024-11-11 14:29:29,536][00562] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 3493888. Throughput: 0: 965.4. Samples: 872718. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:29:29,545][00562] Avg episode reward: [(0, '21.123')] +[2024-11-11 14:29:34,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 3518464. Throughput: 0: 969.6. Samples: 879576. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:29:34,541][00562] Avg episode reward: [(0, '20.929')] +[2024-11-11 14:29:35,193][03135] Updated weights for policy 0, policy_version 860 (0.0026) +[2024-11-11 14:29:39,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 3538944. Throughput: 0: 1002.3. Samples: 883144. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:29:39,538][00562] Avg episode reward: [(0, '20.835')] +[2024-11-11 14:29:44,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 3551232. Throughput: 0: 987.0. Samples: 888100. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:29:44,542][00562] Avg episode reward: [(0, '20.317')] +[2024-11-11 14:29:46,450][03135] Updated weights for policy 0, policy_version 870 (0.0030) +[2024-11-11 14:29:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 3575808. Throughput: 0: 966.3. Samples: 894262. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:29:49,538][00562] Avg episode reward: [(0, '21.075')] +[2024-11-11 14:29:54,536][00562] Fps is (10 sec: 4915.2, 60 sec: 4096.0, 300 sec: 3957.2). Total num frames: 3600384. Throughput: 0: 987.7. Samples: 897792. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:29:54,538][00562] Avg episode reward: [(0, '21.177')] +[2024-11-11 14:29:54,546][03122] Saving new best policy, reward=21.177! +[2024-11-11 14:29:55,461][03135] Updated weights for policy 0, policy_version 880 (0.0033) +[2024-11-11 14:29:59,538][00562] Fps is (10 sec: 3685.6, 60 sec: 3891.1, 300 sec: 3929.4). Total num frames: 3612672. Throughput: 0: 998.1. Samples: 903246. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-11-11 14:29:59,542][00562] Avg episode reward: [(0, '19.835')] +[2024-11-11 14:30:04,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 3633152. Throughput: 0: 956.1. Samples: 908288. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:30:04,542][00562] Avg episode reward: [(0, '20.637')] +[2024-11-11 14:30:06,895][03135] Updated weights for policy 0, policy_version 890 (0.0029) +[2024-11-11 14:30:09,536][00562] Fps is (10 sec: 4506.6, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 3657728. Throughput: 0: 960.6. Samples: 911780. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:30:09,539][00562] Avg episode reward: [(0, '20.359')] +[2024-11-11 14:30:14,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 3674112. Throughput: 0: 1016.9. Samples: 918478. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:30:14,542][00562] Avg episode reward: [(0, '19.161')] +[2024-11-11 14:30:18,074][03135] Updated weights for policy 0, policy_version 900 (0.0013) +[2024-11-11 14:30:19,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 3690496. Throughput: 0: 960.4. Samples: 922794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:30:19,540][00562] Avg episode reward: [(0, '19.307')] +[2024-11-11 14:30:24,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 3715072. Throughput: 0: 959.4. Samples: 926316. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:30:24,539][00562] Avg episode reward: [(0, '20.115')] +[2024-11-11 14:30:27,067][03135] Updated weights for policy 0, policy_version 910 (0.0042) +[2024-11-11 14:30:29,536][00562] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3957.2). Total num frames: 3735552. Throughput: 0: 1002.1. Samples: 933196. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:30:29,540][00562] Avg episode reward: [(0, '20.229')] +[2024-11-11 14:30:34,536][00562] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 3747840. Throughput: 0: 968.5. Samples: 937844. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:30:34,546][00562] Avg episode reward: [(0, '20.060')] +[2024-11-11 14:30:38,637][03135] Updated weights for policy 0, policy_version 920 (0.0019) +[2024-11-11 14:30:39,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 3772416. Throughput: 0: 953.4. Samples: 940696. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:30:39,543][00562] Avg episode reward: [(0, '21.008')] +[2024-11-11 14:30:44,538][00562] Fps is (10 sec: 4504.7, 60 sec: 4027.6, 300 sec: 3943.2). Total num frames: 3792896. Throughput: 0: 985.7. Samples: 947602. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-11-11 14:30:44,544][00562] Avg episode reward: [(0, '22.849')] +[2024-11-11 14:30:44,552][03122] Saving new best policy, reward=22.849! +[2024-11-11 14:30:48,980][03135] Updated weights for policy 0, policy_version 930 (0.0017) +[2024-11-11 14:30:49,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 3809280. Throughput: 0: 987.6. Samples: 952728. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:30:49,541][00562] Avg episode reward: [(0, '23.123')] +[2024-11-11 14:30:49,547][03122] Saving new best policy, reward=23.123! +[2024-11-11 14:30:54,536][00562] Fps is (10 sec: 3277.6, 60 sec: 3754.7, 300 sec: 3901.6). Total num frames: 3825664. Throughput: 0: 956.2. Samples: 954808. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:30:54,539][00562] Avg episode reward: [(0, '24.388')] +[2024-11-11 14:30:54,547][03122] Saving new best policy, reward=24.388! +[2024-11-11 14:30:59,079][03135] Updated weights for policy 0, policy_version 940 (0.0023) +[2024-11-11 14:30:59,536][00562] Fps is (10 sec: 4096.0, 60 sec: 3959.6, 300 sec: 3929.4). Total num frames: 3850240. Throughput: 0: 958.2. Samples: 961598. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:30:59,544][00562] Avg episode reward: [(0, '23.414')] +[2024-11-11 14:31:04,536][00562] Fps is (10 sec: 4095.8, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 3866624. Throughput: 0: 996.1. Samples: 967618. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:31:04,544][00562] Avg episode reward: [(0, '23.652')] +[2024-11-11 14:31:09,536][00562] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3901.6). Total num frames: 3883008. Throughput: 0: 966.4. Samples: 969804. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:31:09,538][00562] Avg episode reward: [(0, '24.082')] +[2024-11-11 14:31:10,466][03135] Updated weights for policy 0, policy_version 950 (0.0017) +[2024-11-11 14:31:14,536][00562] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 3907584. Throughput: 0: 950.8. Samples: 975984. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:31:14,538][00562] Avg episode reward: [(0, '21.893')] +[2024-11-11 14:31:14,547][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000954_3907584.pth... +[2024-11-11 14:31:14,665][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000724_2965504.pth +[2024-11-11 14:31:19,537][00562] Fps is (10 sec: 4505.2, 60 sec: 3959.4, 300 sec: 3929.4). Total num frames: 3928064. Throughput: 0: 999.0. Samples: 982798. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-11-11 14:31:19,539][00562] Avg episode reward: [(0, '20.911')] +[2024-11-11 14:31:19,945][03135] Updated weights for policy 0, policy_version 960 (0.0022) +[2024-11-11 14:31:24,536][00562] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 3944448. Throughput: 0: 981.1. Samples: 984846. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:31:24,543][00562] Avg episode reward: [(0, '22.390')] +[2024-11-11 14:31:29,536][00562] Fps is (10 sec: 3686.7, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 3964928. Throughput: 0: 944.8. Samples: 990116. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-11-11 14:31:29,540][00562] Avg episode reward: [(0, '21.640')] +[2024-11-11 14:31:31,036][03135] Updated weights for policy 0, policy_version 970 (0.0018) +[2024-11-11 14:31:34,536][00562] Fps is (10 sec: 4505.7, 60 sec: 4027.8, 300 sec: 3943.3). Total num frames: 3989504. Throughput: 0: 985.9. Samples: 997094. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-11-11 14:31:34,545][00562] Avg episode reward: [(0, '21.698')] +[2024-11-11 14:31:39,536][00562] Fps is (10 sec: 3686.5, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 4001792. Throughput: 0: 1002.7. Samples: 999930. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-11-11 14:31:39,546][00562] Avg episode reward: [(0, '21.392')] +[2024-11-11 14:31:39,657][03122] Stopping Batcher_0... +[2024-11-11 14:31:39,659][03122] Loop batcher_evt_loop terminating... +[2024-11-11 14:31:39,664][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-11-11 14:31:39,660][00562] Component Batcher_0 stopped! +[2024-11-11 14:31:39,766][03135] Weights refcount: 2 0 +[2024-11-11 14:31:39,772][00562] Component InferenceWorker_p0-w0 stopped! +[2024-11-11 14:31:39,778][03135] Stopping InferenceWorker_p0-w0... +[2024-11-11 14:31:39,778][03135] Loop inference_proc0-0_evt_loop terminating... +[2024-11-11 14:31:39,854][03122] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000839_3436544.pth +[2024-11-11 14:31:39,872][03122] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-11-11 14:31:40,081][00562] Component LearnerWorker_p0 stopped! +[2024-11-11 14:31:40,088][03122] Stopping LearnerWorker_p0... +[2024-11-11 14:31:40,088][03122] Loop learner_proc0_evt_loop terminating... +[2024-11-11 14:31:40,394][00562] Component RolloutWorker_w1 stopped! +[2024-11-11 14:31:40,405][03137] Stopping RolloutWorker_w1... +[2024-11-11 14:31:40,405][03137] Loop rollout_proc1_evt_loop terminating... +[2024-11-11 14:31:40,449][03143] Stopping RolloutWorker_w7... +[2024-11-11 14:31:40,451][00562] Component RolloutWorker_w7 stopped! +[2024-11-11 14:31:40,450][03143] Loop rollout_proc7_evt_loop terminating... +[2024-11-11 14:31:40,471][03141] Stopping RolloutWorker_w5... +[2024-11-11 14:31:40,472][03141] Loop rollout_proc5_evt_loop terminating... +[2024-11-11 14:31:40,473][00562] Component RolloutWorker_w5 stopped! +[2024-11-11 14:31:40,483][00562] Component RolloutWorker_w3 stopped! +[2024-11-11 14:31:40,490][03139] Stopping RolloutWorker_w3... +[2024-11-11 14:31:40,490][03139] Loop rollout_proc3_evt_loop terminating... +[2024-11-11 14:31:40,541][03138] Stopping RolloutWorker_w2... +[2024-11-11 14:31:40,540][00562] Component RolloutWorker_w2 stopped! +[2024-11-11 14:31:40,541][03138] Loop rollout_proc2_evt_loop terminating... +[2024-11-11 14:31:40,577][03140] Stopping RolloutWorker_w4... +[2024-11-11 14:31:40,577][03140] Loop rollout_proc4_evt_loop terminating... +[2024-11-11 14:31:40,577][00562] Component RolloutWorker_w4 stopped! +[2024-11-11 14:31:40,591][00562] Component RolloutWorker_w0 stopped! +[2024-11-11 14:31:40,590][03136] Stopping RolloutWorker_w0... +[2024-11-11 14:31:40,593][03136] Loop rollout_proc0_evt_loop terminating... +[2024-11-11 14:31:40,636][00562] Component RolloutWorker_w6 stopped! +[2024-11-11 14:31:40,638][00562] Waiting for process learner_proc0 to stop... +[2024-11-11 14:31:40,641][03142] Stopping RolloutWorker_w6... +[2024-11-11 14:31:40,641][03142] Loop rollout_proc6_evt_loop terminating... +[2024-11-11 14:31:41,983][00562] Waiting for process inference_proc0-0 to join... +[2024-11-11 14:31:41,987][00562] Waiting for process rollout_proc0 to join... +[2024-11-11 14:31:44,119][00562] Waiting for process rollout_proc1 to join... +[2024-11-11 14:31:44,143][00562] Waiting for process rollout_proc2 to join... +[2024-11-11 14:31:44,149][00562] Waiting for process rollout_proc3 to join... +[2024-11-11 14:31:44,154][00562] Waiting for process rollout_proc4 to join... +[2024-11-11 14:31:44,157][00562] Waiting for process rollout_proc5 to join... +[2024-11-11 14:31:44,161][00562] Waiting for process rollout_proc6 to join... +[2024-11-11 14:31:44,164][00562] Waiting for process rollout_proc7 to join... +[2024-11-11 14:31:44,167][00562] Batcher 0 profile tree view: +batching: 28.3437, releasing_batches: 0.0330 +[2024-11-11 14:31:44,170][00562] InferenceWorker_p0-w0 profile tree view: +wait_policy: 0.0025 + wait_policy_total: 436.3030 +update_model: 8.7112 + weight_update: 0.0019 +one_step: 0.0036 + handle_policy_step: 583.4895 + deserialize: 15.2080, stack: 3.1268, obs_to_device_normalize: 124.0045, forward: 294.0283, send_messages: 29.0137 + prepare_outputs: 88.5107 + to_cpu: 52.8766 +[2024-11-11 14:31:44,172][00562] Learner 0 profile tree view: +misc: 0.0051, prepare_batch: 13.6783 +train: 75.4392 + epoch_init: 0.0055, minibatch_init: 0.0159, losses_postprocess: 0.5740, kl_divergence: 0.5243, after_optimizer: 33.4716 + calculate_losses: 27.7655 + losses_init: 0.0037, forward_head: 1.4214, bptt_initial: 18.8187, tail: 1.2498, advantages_returns: 0.2608, losses: 3.7434 + bptt: 1.9478 + bptt_forward_core: 1.8665 + update: 12.4283 + clip: 1.0064 +[2024-11-11 14:31:44,174][00562] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 0.4060, enqueue_policy_requests: 106.8350, env_step: 839.8568, overhead: 14.1801, complete_rollouts: 7.3456 +save_policy_outputs: 22.4906 + split_output_tensors: 8.6830 +[2024-11-11 14:31:44,176][00562] RolloutWorker_w7 profile tree view: +wait_for_trajectories: 0.3708, enqueue_policy_requests: 109.4797, env_step: 834.1560, overhead: 14.2467, complete_rollouts: 7.0637 +save_policy_outputs: 21.3815 + split_output_tensors: 8.4240 +[2024-11-11 14:31:44,178][00562] Loop Runner_EvtLoop terminating... +[2024-11-11 14:31:44,180][00562] Runner profile tree view: +main_loop: 1103.4782 +[2024-11-11 14:31:44,181][00562] Collected {0: 4005888}, FPS: 3630.2 +[2024-11-11 14:31:56,379][00562] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2024-11-11 14:31:56,381][00562] Overriding arg 'num_workers' with value 1 passed from command line +[2024-11-11 14:31:56,383][00562] Adding new argument 'no_render'=True that is not in the saved config file! +[2024-11-11 14:31:56,385][00562] Adding new argument 'save_video'=True that is not in the saved config file! +[2024-11-11 14:31:56,387][00562] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2024-11-11 14:31:56,388][00562] Adding new argument 'video_name'=None that is not in the saved config file! +[2024-11-11 14:31:56,389][00562] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! +[2024-11-11 14:31:56,391][00562] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2024-11-11 14:31:56,392][00562] Adding new argument 'push_to_hub'=False that is not in the saved config file! +[2024-11-11 14:31:56,393][00562] Adding new argument 'hf_repository'=None that is not in the saved config file! +[2024-11-11 14:31:56,394][00562] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2024-11-11 14:31:56,395][00562] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2024-11-11 14:31:56,396][00562] Adding new argument 'train_script'=None that is not in the saved config file! +[2024-11-11 14:31:56,397][00562] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2024-11-11 14:31:56,398][00562] Using frameskip 1 and render_action_repeat=4 for evaluation +[2024-11-11 14:31:56,446][00562] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-11-11 14:31:56,450][00562] RunningMeanStd input shape: (3, 72, 128) +[2024-11-11 14:31:56,453][00562] RunningMeanStd input shape: (1,) +[2024-11-11 14:31:56,468][00562] ConvEncoder: input_channels=3 +[2024-11-11 14:31:56,569][00562] Conv encoder output size: 512 +[2024-11-11 14:31:56,571][00562] Policy head output size: 512 +[2024-11-11 14:31:56,748][00562] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-11-11 14:31:57,562][00562] Num frames 100... +[2024-11-11 14:31:57,684][00562] Num frames 200... +[2024-11-11 14:31:57,807][00562] Num frames 300... +[2024-11-11 14:31:57,936][00562] Num frames 400... +[2024-11-11 14:31:58,058][00562] Num frames 500... +[2024-11-11 14:31:58,185][00562] Num frames 600... +[2024-11-11 14:31:58,309][00562] Num frames 700... +[2024-11-11 14:31:58,449][00562] Avg episode rewards: #0: 18.690, true rewards: #0: 7.690 +[2024-11-11 14:31:58,451][00562] Avg episode reward: 18.690, avg true_objective: 7.690 +[2024-11-11 14:31:58,493][00562] Num frames 800... +[2024-11-11 14:31:58,614][00562] Num frames 900... +[2024-11-11 14:31:58,737][00562] Num frames 1000... +[2024-11-11 14:31:58,857][00562] Num frames 1100... +[2024-11-11 14:31:58,999][00562] Num frames 1200... +[2024-11-11 14:31:59,134][00562] Num frames 1300... +[2024-11-11 14:31:59,257][00562] Num frames 1400... +[2024-11-11 14:31:59,325][00562] Avg episode rewards: #0: 14.545, true rewards: #0: 7.045 +[2024-11-11 14:31:59,326][00562] Avg episode reward: 14.545, avg true_objective: 7.045 +[2024-11-11 14:31:59,436][00562] Num frames 1500... +[2024-11-11 14:31:59,558][00562] Num frames 1600... +[2024-11-11 14:31:59,682][00562] Num frames 1700... +[2024-11-11 14:31:59,804][00562] Num frames 1800... +[2024-11-11 14:31:59,925][00562] Num frames 1900... +[2024-11-11 14:32:00,054][00562] Num frames 2000... +[2024-11-11 14:32:00,184][00562] Num frames 2100... +[2024-11-11 14:32:00,310][00562] Num frames 2200... +[2024-11-11 14:32:00,433][00562] Num frames 2300... +[2024-11-11 14:32:00,554][00562] Num frames 2400... +[2024-11-11 14:32:00,676][00562] Num frames 2500... +[2024-11-11 14:32:00,797][00562] Num frames 2600... +[2024-11-11 14:32:00,918][00562] Num frames 2700... +[2024-11-11 14:32:01,046][00562] Num frames 2800... +[2024-11-11 14:32:01,177][00562] Num frames 2900... +[2024-11-11 14:32:01,303][00562] Num frames 3000... +[2024-11-11 14:32:01,425][00562] Num frames 3100... +[2024-11-11 14:32:01,546][00562] Num frames 3200... +[2024-11-11 14:32:01,658][00562] Avg episode rewards: #0: 23.487, true rewards: #0: 10.820 +[2024-11-11 14:32:01,659][00562] Avg episode reward: 23.487, avg true_objective: 10.820 +[2024-11-11 14:32:01,727][00562] Num frames 3300... +[2024-11-11 14:32:01,850][00562] Num frames 3400... +[2024-11-11 14:32:01,969][00562] Num frames 3500... +[2024-11-11 14:32:02,102][00562] Num frames 3600... +[2024-11-11 14:32:02,271][00562] Avg episode rewards: #0: 20.220, true rewards: #0: 9.220 +[2024-11-11 14:32:02,273][00562] Avg episode reward: 20.220, avg true_objective: 9.220 +[2024-11-11 14:32:02,299][00562] Num frames 3700... +[2024-11-11 14:32:02,467][00562] Num frames 3800... +[2024-11-11 14:32:02,632][00562] Num frames 3900... +[2024-11-11 14:32:02,795][00562] Num frames 4000... +[2024-11-11 14:32:02,960][00562] Num frames 4100... +[2024-11-11 14:32:03,131][00562] Num frames 4200... +[2024-11-11 14:32:03,295][00562] Num frames 4300... +[2024-11-11 14:32:03,462][00562] Num frames 4400... +[2024-11-11 14:32:03,635][00562] Num frames 4500... +[2024-11-11 14:32:03,813][00562] Num frames 4600... +[2024-11-11 14:32:03,987][00562] Num frames 4700... +[2024-11-11 14:32:04,175][00562] Num frames 4800... +[2024-11-11 14:32:04,247][00562] Avg episode rewards: #0: 20.816, true rewards: #0: 9.616 +[2024-11-11 14:32:04,249][00562] Avg episode reward: 20.816, avg true_objective: 9.616 +[2024-11-11 14:32:04,413][00562] Num frames 4900... +[2024-11-11 14:32:04,546][00562] Num frames 5000... +[2024-11-11 14:32:04,668][00562] Num frames 5100... +[2024-11-11 14:32:04,787][00562] Num frames 5200... +[2024-11-11 14:32:04,904][00562] Num frames 5300... +[2024-11-11 14:32:05,020][00562] Avg episode rewards: #0: 18.587, true rewards: #0: 8.920 +[2024-11-11 14:32:05,022][00562] Avg episode reward: 18.587, avg true_objective: 8.920 +[2024-11-11 14:32:05,084][00562] Num frames 5400... +[2024-11-11 14:32:05,221][00562] Num frames 5500... +[2024-11-11 14:32:05,342][00562] Num frames 5600... +[2024-11-11 14:32:05,465][00562] Num frames 5700... +[2024-11-11 14:32:05,583][00562] Num frames 5800... +[2024-11-11 14:32:05,703][00562] Num frames 5900... +[2024-11-11 14:32:05,823][00562] Num frames 6000... +[2024-11-11 14:32:05,954][00562] Num frames 6100... +[2024-11-11 14:32:06,132][00562] Avg episode rewards: #0: 18.704, true rewards: #0: 8.847 +[2024-11-11 14:32:06,133][00562] Avg episode reward: 18.704, avg true_objective: 8.847 +[2024-11-11 14:32:06,145][00562] Num frames 6200... +[2024-11-11 14:32:06,276][00562] Num frames 6300... +[2024-11-11 14:32:06,398][00562] Num frames 6400... +[2024-11-11 14:32:06,517][00562] Num frames 6500... +[2024-11-11 14:32:06,636][00562] Num frames 6600... +[2024-11-11 14:32:06,754][00562] Num frames 6700... +[2024-11-11 14:32:06,875][00562] Num frames 6800... +[2024-11-11 14:32:06,997][00562] Num frames 6900... +[2024-11-11 14:32:07,135][00562] Num frames 7000... +[2024-11-11 14:32:07,243][00562] Avg episode rewards: #0: 18.555, true rewards: #0: 8.805 +[2024-11-11 14:32:07,245][00562] Avg episode reward: 18.555, avg true_objective: 8.805 +[2024-11-11 14:32:07,315][00562] Num frames 7100... +[2024-11-11 14:32:07,436][00562] Num frames 7200... +[2024-11-11 14:32:07,556][00562] Num frames 7300... +[2024-11-11 14:32:07,674][00562] Num frames 7400... +[2024-11-11 14:32:07,796][00562] Num frames 7500... +[2024-11-11 14:32:07,915][00562] Num frames 7600... +[2024-11-11 14:32:08,037][00562] Num frames 7700... +[2024-11-11 14:32:08,167][00562] Num frames 7800... +[2024-11-11 14:32:08,295][00562] Num frames 7900... +[2024-11-11 14:32:08,416][00562] Num frames 8000... +[2024-11-11 14:32:08,535][00562] Num frames 8100... +[2024-11-11 14:32:08,656][00562] Num frames 8200... +[2024-11-11 14:32:08,777][00562] Num frames 8300... +[2024-11-11 14:32:08,900][00562] Num frames 8400... +[2024-11-11 14:32:09,020][00562] Num frames 8500... +[2024-11-11 14:32:09,147][00562] Num frames 8600... +[2024-11-11 14:32:09,275][00562] Num frames 8700... +[2024-11-11 14:32:09,398][00562] Num frames 8800... +[2024-11-11 14:32:09,519][00562] Num frames 8900... +[2024-11-11 14:32:09,641][00562] Num frames 9000... +[2024-11-11 14:32:09,763][00562] Num frames 9100... +[2024-11-11 14:32:09,871][00562] Avg episode rewards: #0: 22.715, true rewards: #0: 10.160 +[2024-11-11 14:32:09,872][00562] Avg episode reward: 22.715, avg true_objective: 10.160 +[2024-11-11 14:32:09,944][00562] Num frames 9200... +[2024-11-11 14:32:10,061][00562] Num frames 9300... +[2024-11-11 14:32:10,193][00562] Num frames 9400... +[2024-11-11 14:32:10,320][00562] Num frames 9500... +[2024-11-11 14:32:10,442][00562] Num frames 9600... +[2024-11-11 14:32:10,564][00562] Avg episode rewards: #0: 21.256, true rewards: #0: 9.656 +[2024-11-11 14:32:10,565][00562] Avg episode reward: 21.256, avg true_objective: 9.656 +[2024-11-11 14:33:08,223][00562] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2024-11-11 14:41:13,430][00562] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2024-11-11 14:41:13,432][00562] Overriding arg 'num_workers' with value 1 passed from command line +[2024-11-11 14:41:13,434][00562] Adding new argument 'no_render'=True that is not in the saved config file! +[2024-11-11 14:41:13,435][00562] Adding new argument 'save_video'=True that is not in the saved config file! +[2024-11-11 14:41:13,437][00562] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2024-11-11 14:41:13,439][00562] Adding new argument 'video_name'=None that is not in the saved config file! +[2024-11-11 14:41:13,441][00562] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! +[2024-11-11 14:41:13,442][00562] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2024-11-11 14:41:13,443][00562] Adding new argument 'push_to_hub'=True that is not in the saved config file! +[2024-11-11 14:41:13,444][00562] Adding new argument 'hf_repository'='usamabuttar/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! +[2024-11-11 14:41:13,445][00562] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2024-11-11 14:41:13,446][00562] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2024-11-11 14:41:13,447][00562] Adding new argument 'train_script'=None that is not in the saved config file! +[2024-11-11 14:41:13,448][00562] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2024-11-11 14:41:13,449][00562] Using frameskip 1 and render_action_repeat=4 for evaluation +[2024-11-11 14:41:13,478][00562] RunningMeanStd input shape: (3, 72, 128) +[2024-11-11 14:41:13,480][00562] RunningMeanStd input shape: (1,) +[2024-11-11 14:41:13,492][00562] ConvEncoder: input_channels=3 +[2024-11-11 14:41:13,529][00562] Conv encoder output size: 512 +[2024-11-11 14:41:13,530][00562] Policy head output size: 512 +[2024-11-11 14:41:13,551][00562] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-11-11 14:41:13,972][00562] Num frames 100... +[2024-11-11 14:41:14,107][00562] Num frames 200... +[2024-11-11 14:41:14,247][00562] Num frames 300... +[2024-11-11 14:41:14,369][00562] Num frames 400... +[2024-11-11 14:41:14,489][00562] Num frames 500... +[2024-11-11 14:41:14,614][00562] Num frames 600... +[2024-11-11 14:41:14,732][00562] Num frames 700... +[2024-11-11 14:41:14,866][00562] Num frames 800... +[2024-11-11 14:41:14,988][00562] Num frames 900... +[2024-11-11 14:41:15,110][00562] Num frames 1000... +[2024-11-11 14:41:15,240][00562] Num frames 1100... +[2024-11-11 14:41:15,363][00562] Num frames 1200... +[2024-11-11 14:41:15,482][00562] Num frames 1300... +[2024-11-11 14:41:15,603][00562] Num frames 1400... +[2024-11-11 14:41:15,727][00562] Num frames 1500... +[2024-11-11 14:41:15,847][00562] Num frames 1600... +[2024-11-11 14:41:15,969][00562] Num frames 1700... +[2024-11-11 14:41:16,089][00562] Num frames 1800... +[2024-11-11 14:41:16,214][00562] Num frames 1900... +[2024-11-11 14:41:16,343][00562] Num frames 2000... +[2024-11-11 14:41:16,469][00562] Num frames 2100... +[2024-11-11 14:41:16,521][00562] Avg episode rewards: #0: 59.999, true rewards: #0: 21.000 +[2024-11-11 14:41:16,524][00562] Avg episode reward: 59.999, avg true_objective: 21.000 +[2024-11-11 14:41:16,643][00562] Num frames 2200... +[2024-11-11 14:41:16,761][00562] Num frames 2300... +[2024-11-11 14:41:16,875][00562] Num frames 2400... +[2024-11-11 14:41:16,993][00562] Num frames 2500... +[2024-11-11 14:41:17,115][00562] Num frames 2600... +[2024-11-11 14:41:17,236][00562] Num frames 2700... +[2024-11-11 14:41:17,363][00562] Num frames 2800... +[2024-11-11 14:41:17,485][00562] Num frames 2900... +[2024-11-11 14:41:17,603][00562] Num frames 3000... +[2024-11-11 14:41:17,724][00562] Num frames 3100... +[2024-11-11 14:41:17,848][00562] Num frames 3200... +[2024-11-11 14:41:17,937][00562] Avg episode rewards: #0: 41.600, true rewards: #0: 16.100 +[2024-11-11 14:41:17,940][00562] Avg episode reward: 41.600, avg true_objective: 16.100 +[2024-11-11 14:41:18,074][00562] Num frames 3300... +[2024-11-11 14:41:18,236][00562] Num frames 3400... +[2024-11-11 14:41:18,407][00562] Num frames 3500... +[2024-11-11 14:41:18,570][00562] Num frames 3600... +[2024-11-11 14:41:18,738][00562] Num frames 3700... +[2024-11-11 14:41:18,899][00562] Num frames 3800... +[2024-11-11 14:41:19,060][00562] Num frames 3900... +[2024-11-11 14:41:19,233][00562] Num frames 4000... +[2024-11-11 14:41:19,416][00562] Num frames 4100... +[2024-11-11 14:41:19,591][00562] Num frames 4200... +[2024-11-11 14:41:19,766][00562] Num frames 4300... +[2024-11-11 14:41:19,941][00562] Num frames 4400... +[2024-11-11 14:41:20,124][00562] Num frames 4500... +[2024-11-11 14:41:20,297][00562] Num frames 4600... +[2024-11-11 14:41:20,452][00562] Num frames 4700... +[2024-11-11 14:41:20,577][00562] Num frames 4800... +[2024-11-11 14:41:20,701][00562] Num frames 4900... +[2024-11-11 14:41:20,822][00562] Num frames 5000... +[2024-11-11 14:41:20,946][00562] Num frames 5100... +[2024-11-11 14:41:21,066][00562] Num frames 5200... +[2024-11-11 14:41:21,195][00562] Num frames 5300... +[2024-11-11 14:41:21,276][00562] Avg episode rewards: #0: 45.733, true rewards: #0: 17.733 +[2024-11-11 14:41:21,278][00562] Avg episode reward: 45.733, avg true_objective: 17.733 +[2024-11-11 14:41:21,380][00562] Num frames 5400... +[2024-11-11 14:41:21,505][00562] Num frames 5500... +[2024-11-11 14:41:21,624][00562] Num frames 5600... +[2024-11-11 14:41:21,747][00562] Num frames 5700... +[2024-11-11 14:41:21,899][00562] Num frames 5800... +[2024-11-11 14:41:22,018][00562] Num frames 5900... +[2024-11-11 14:41:22,170][00562] Avg episode rewards: #0: 37.435, true rewards: #0: 14.935 +[2024-11-11 14:41:22,172][00562] Avg episode reward: 37.435, avg true_objective: 14.935 +[2024-11-11 14:41:22,206][00562] Num frames 6000... +[2024-11-11 14:41:22,325][00562] Num frames 6100... +[2024-11-11 14:41:22,454][00562] Num frames 6200... +[2024-11-11 14:41:22,607][00562] Num frames 6300... +[2024-11-11 14:41:22,728][00562] Num frames 6400... +[2024-11-11 14:41:22,845][00562] Num frames 6500... +[2024-11-11 14:41:22,965][00562] Num frames 6600... +[2024-11-11 14:41:23,085][00562] Num frames 6700... +[2024-11-11 14:41:23,213][00562] Num frames 6800... +[2024-11-11 14:41:23,334][00562] Num frames 6900... +[2024-11-11 14:41:23,466][00562] Num frames 7000... +[2024-11-11 14:41:23,598][00562] Num frames 7100... +[2024-11-11 14:41:23,720][00562] Num frames 7200... +[2024-11-11 14:41:23,843][00562] Num frames 7300... +[2024-11-11 14:41:23,963][00562] Num frames 7400... +[2024-11-11 14:41:24,073][00562] Avg episode rewards: #0: 36.892, true rewards: #0: 14.892 +[2024-11-11 14:41:24,075][00562] Avg episode reward: 36.892, avg true_objective: 14.892 +[2024-11-11 14:41:24,148][00562] Num frames 7500... +[2024-11-11 14:41:24,267][00562] Num frames 7600... +[2024-11-11 14:41:24,388][00562] Num frames 7700... +[2024-11-11 14:41:24,517][00562] Num frames 7800... +[2024-11-11 14:41:24,610][00562] Avg episode rewards: #0: 31.383, true rewards: #0: 13.050 +[2024-11-11 14:41:24,612][00562] Avg episode reward: 31.383, avg true_objective: 13.050 +[2024-11-11 14:41:24,699][00562] Num frames 7900... +[2024-11-11 14:41:24,817][00562] Num frames 8000... +[2024-11-11 14:41:24,937][00562] Num frames 8100... +[2024-11-11 14:41:25,058][00562] Num frames 8200... +[2024-11-11 14:41:25,185][00562] Num frames 8300... +[2024-11-11 14:41:25,311][00562] Num frames 8400... +[2024-11-11 14:41:25,431][00562] Num frames 8500... +[2024-11-11 14:41:25,559][00562] Num frames 8600... +[2024-11-11 14:41:25,690][00562] Num frames 8700... +[2024-11-11 14:41:25,809][00562] Num frames 8800... +[2024-11-11 14:41:25,930][00562] Num frames 8900... +[2024-11-11 14:41:26,084][00562] Avg episode rewards: #0: 30.403, true rewards: #0: 12.831 +[2024-11-11 14:41:26,086][00562] Avg episode reward: 30.403, avg true_objective: 12.831 +[2024-11-11 14:41:26,118][00562] Num frames 9000... +[2024-11-11 14:41:26,241][00562] Num frames 9100... +[2024-11-11 14:41:26,362][00562] Num frames 9200... +[2024-11-11 14:41:26,485][00562] Num frames 9300... +[2024-11-11 14:41:26,645][00562] Num frames 9400... +[2024-11-11 14:41:26,768][00562] Num frames 9500... +[2024-11-11 14:41:26,856][00562] Avg episode rewards: #0: 27.657, true rewards: #0: 11.908 +[2024-11-11 14:41:26,858][00562] Avg episode reward: 27.657, avg true_objective: 11.908 +[2024-11-11 14:41:26,948][00562] Num frames 9600... +[2024-11-11 14:41:27,067][00562] Num frames 9700... +[2024-11-11 14:41:27,195][00562] Num frames 9800... +[2024-11-11 14:41:27,316][00562] Num frames 9900... +[2024-11-11 14:41:27,440][00562] Num frames 10000... +[2024-11-11 14:41:27,562][00562] Num frames 10100... +[2024-11-11 14:41:27,693][00562] Num frames 10200... +[2024-11-11 14:41:27,861][00562] Avg episode rewards: #0: 25.993, true rewards: #0: 11.438 +[2024-11-11 14:41:27,863][00562] Avg episode reward: 25.993, avg true_objective: 11.438 +[2024-11-11 14:41:27,874][00562] Num frames 10300... +[2024-11-11 14:41:27,998][00562] Num frames 10400... +[2024-11-11 14:41:28,081][00562] Avg episode rewards: #0: 23.522, true rewards: #0: 10.422 +[2024-11-11 14:41:28,082][00562] Avg episode reward: 23.522, avg true_objective: 10.422 +[2024-11-11 14:42:30,928][00562] Replay video saved to /content/train_dir/default_experiment/replay.mp4!