Training in progress, step 4000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c1c1884504de849ed639a26627b5c0c92b49f2932b7c5a3c7b8ea9b6f31fe8e9
+oid sha256:656ad65a0efd016b92be9b27dc8dca4b6485147fc7a287642049ae5aaf68cffa
 size 967102601

training.log

@@ -1316,3 +1316,182 @@ Adding files tracked by Git LFS: ['wandb/run-20230509_115211-hq92t8sj/run-hq92t8
 [INFO|modeling_utils.py:1855] 2023-05-11 12:24:49,746 >> Model weights saved in ./checkpoint-3000/pytorch_model.bin
 [INFO|feature_extraction_utils.py:369] 2023-05-11 12:24:49,750 >> Feature extractor saved in ./checkpoint-3000/preprocessor_config.json
 [INFO|feature_extraction_utils.py:369] 2023-05-11 12:25:01,129 >> Feature extractor saved in ./preprocessor_config.json
+/home/local/QCRI/dizham/miniconda3/envs/whisper/lib/python3.9/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
+  warnings.warn('Was asked to gather along dimension 0, but all '
+
 60%|██████    | 3001/5000 [26:44:35<531:11:53, 956.64s/it]
 60%|██████    | 3002/5000 [26:45:04<376:32:57, 678.47s/it]
 60%|██████    | 3003/5000 [26:45:35<268:33:10, 484.12s/it]
 60%|██████    | 3004/5000 [26:46:03<192:29:24, 347.18s/it]
 60%|██████    | 3005/5000 [26:46:31<139:23:30, 251.53s/it]
 60%|██████    | 3006/5000 [26:46:59<102:13:21, 184.55s/it]
 60%|██████    | 3007/5000 [26:47:28<76:19:42, 137.87s/it] 
 60%|██████    | 3008/5000 [26:47:57<58:07:30, 105.05s/it]
 60%|██████    | 3009/5000 [26:48:25<45:24:30, 82.10s/it] 
 60%|██████    | 3010/5000 [26:48:53<36:22:14, 65.80s/it]
 60%|██████    | 3011/5000 [26:49:21<30:01:46, 54.35s/it]
 60%|██████    | 3012/5000 [26:49:49<25:43:04, 46.57s/it]
 60%|██████    | 3013/5000 [26:50:18<22:51:13, 41.41s/it]
 60%|██████    | 3014/5000 [26:50:46<20:30:09, 37.17s/it]
 60%|██████    | 3015/5000 [26:51:14<19:00:24, 34.47s/it]
 60%|██████    | 3016/5000 [26:51:42<17:54:07, 32.48s/it]
 60%|██████    | 3017/5000 [26:52:09<17:06:04, 31.05s/it]
 60%|██████    | 3018/5000 [26:52:37<16:33:03, 30.06s/it]
 60%|██████    | 3019/5000 [26:53:05<16:06:01, 29.26s/it]
 60%|██████    | 3020/5000 [26:53:33<15:52:28, 28.86s/it]
 60%|██████    | 3021/5000 [26:54:00<15:37:25, 28.42s/it]
 60%|██████    | 3022/5000 [26:54:28<15:34:47, 28.36s/it]
 60%|██████    | 3023/5000 [26:54:56<15:28:22, 28.18s/it]
 60%|██████    | 3024/5000 [26:55:24<15:22:43, 28.02s/it]
 60%|██████    | 3025/5000 [26:55:52<15:29:03, 28.22s/it]
                                                         
 60%|██████    | 3025/5000 [26:55:52<15:29:03, 28.22s/it]
 61%|██████    | 3026/5000 [26:56:21<15:34:03, 28.39s/it]
 61%|██████    | 3027/5000 [26:56:49<15:24:36, 28.12s/it]
 61%|██████    | 3028/5000 [26:57:16<15:21:07, 28.03s/it]
 61%|██████    | 3029/5000 [26:57:44<15:18:22, 27.96s/it]
 61%|██████    | 3030/5000 [26:58:12<15:15:36, 27.89s/it]
 61%|██████    | 3031/5000 [26:58:40<15:18:45, 28.00s/it]
 61%|██████    | 3032/5000 [26:59:08<15:17:31, 27.97s/it]
 61%|██████    | 3033/5000 [26:59:36<15:19:41, 28.05s/it]
 61%|██████    | 3034/5000 [27:00:03<15:06:50, 27.68s/it]
 61%|██████    | 3035/5000 [27:00:31<15:08:01, 27.73s/it]
 61%|██████    | 3036/5000 [27:00:58<15:01:05, 27.53s/it]
 61%|██████    | 3037/5000 [27:01:26<15:00:44, 27.53s/it]
 61%|██████    | 3038/5000 [27:01:54<15:14:22, 27.96s/it]
 61%|██████    | 3039/5000 [27:02:22<15:08:56, 27.81s/it]
 61%|██████    | 3040/5000 [27:02:50<15:11:30, 27.90s/it]
 61%|██████    | 3041/5000 [27:03:19<15:22:45, 28.26s/it]
 61%|██████    | 3042/5000 [27:03:47<15:18:32, 28.15s/it]
 61%|██████    | 3043/5000 [27:04:15<15:14:13, 28.03s/it]
 61%|██████    | 3044/5000 [27:04:42<15:10:27, 27.93s/it]
 61%|██████    | 3045/5000 [27:05:10<15:10:20, 27.94s/it]
 61%|██████    | 3046/5000 [27:05:39<15:12:01, 28.00s/it]
 61%|██████    | 3047/5000 [27:06:06<15:05:52, 27.83s/it]
 61%|██████    | 3048/5000 [27:06:35<15:15:17, 28.13s/it]
 61%|██████    | 3049/5000 [27:07:02<15:05:52, 27.86s/it]
 61%|██████    | 3050/5000 [27:07:30<15:03:59, 27.81s/it]
                                                         
 61%|██████    | 3050/5000 [27:07:30<15:03:59, 27.81s/it]
 61%|██████    | 3051/5000 [27:07:58<15:05:10, 27.87s/it]
 61%|██████    | 3052/5000 [27:08:25<15:01:36, 27.77s/it]
 61%|██████    | 3053/5000 [27:08:54<15:09:56, 28.04s/it]
 61%|██████    | 3054/5000 [27:09:21<14:57:14, 27.66s/it]
 61%|██████    | 3055/5000 [27:09:49<15:05:25, 27.93s/it]
 61%|██████    | 3056/5000 [27:10:17<15:01:45, 27.83s/it]
 61%|██████    | 3057/5000 [27:10:44<14:56:00, 27.67s/it]
 61%|██████    | 3058/5000 [27:11:12<14:57:05, 27.72s/it]
 61%|██████    | 3059/5000 [27:11:40<14:56:06, 27.70s/it]
 61%|██████    | 3060/5000 [27:12:07<14:53:36, 27.64s/it]
 61%|██████    | 3061/5000 [27:12:36<14:59:24, 27.83s/it]
 61%|██████    | 3062/5000 [27:13:03<14:54:50, 27.70s/it]
 61%|██████▏   | 3063/5000 [27:13:31<14:57:14, 27.79s/it]
 61%|██████▏   | 3064/5000 [27:13:58<14:54:32, 27.72s/it]
 61%|██████▏   | 3065/5000 [27:14:26<14:51:01, 27.63s/it]
 61%|██████▏   | 3066/5000 [27:14:54<14:51:19, 27.65s/it]
 61%|██████▏   | 3067/5000 [27:15:21<14:45:04, 27.47s/it]
 61%|██████▏   | 3068/5000 [27:15:49<14:56:13, 27.83s/it]
 61%|██████▏   | 3069/5000 [27:16:17<14:53:31, 27.76s/it]
 61%|██████▏   | 3070/5000 [27:16:44<14:50:13, 27.68s/it]
 61%|██████▏   | 3071/5000 [27:17:12<14:49:39, 27.67s/it]
 61%|██████▏   | 3072/5000 [27:17:40<14:47:21, 27.61s/it]
 61%|██████▏   | 3073/5000 [27:18:08<14:54:31, 27.85s/it]
 61%|██████▏   | 3074/5000 [27:18:36<14:55:09, 27.89s/it]
 62%|██████▏   | 3075/5000 [27:19:03<14:46:05, 27.62s/it]
                                                         
 62%|██████▏   | 3075/5000 [27:19:03<14:46:05, 27.62s/it]
 62%|██████▏   | 3076/5000 [27:19:31<14:51:12, 27.79s/it]
 62%|██████▏   | 3077/5000 [27:19:59<14:49:16, 27.75s/it]
 62%|██████▏   | 3078/5000 [27:20:27<14:49:06, 27.76s/it]
 62%|██████▏   | 3079/5000 [27:20:54<14:45:00, 27.64s/it]
 62%|██████▏   | 3080/5000 [27:21:21<14:41:19, 27.54s/it]
 62%|██████▏   | 3081/5000 [27:21:59<16:18:32, 30.60s/it]
 62%|██████▏   | 3082/5000 [27:22:26<15:43:37, 29.52s/it]
 62%|██████▏   | 3083/5000 [27:23:01<16:38:11, 31.24s/it]
 62%|██████▏   | 3084/5000 [27:23:28<15:57:17, 29.98s/it]
 62%|██████▏   | 3085/5000 [27:24:04<16:48:03, 31.58s/it]
 62%|██████▏   | 3086/5000 [27:24:33<16:29:01, 31.00s/it]
 62%|██████▏   | 3087/5000 [27:25:03<16:21:14, 30.78s/it]
 62%|██████▏   | 3088/5000 [27:25:33<16:13:21, 30.54s/it]
 62%|██████▏   | 3089/5000 [27:26:03<16:00:42, 30.16s/it]
 62%|██████▏   | 3090/5000 [27:26:35<16:19:34, 30.77s/it]
 62%|██████▏   | 3091/5000 [27:27:04<15:59:30, 30.16s/it]
 62%|██████▏   | 3092/5000 [27:27:33<15:55:24, 30.04s/it]
 62%|██████▏   | 3093/5000 [27:28:02<15:40:32, 29.59s/it]
 62%|██████▏   | 3094/5000 [27:28:30<15:29:35, 29.26s/it]
 62%|██████▏   | 3095/5000 [27:29:01<15:40:07, 29.61s/it]
 62%|██████▏   | 3096/5000 [27:29:34<16:15:40, 30.75s/it]
 62%|██████▏   | 3097/5000 [27:30:01<15:40:11, 29.64s/it]
 62%|██████▏   | 3098/5000 [27:30:33<15:57:31, 30.21s/it]
 62%|██████▏   | 3099/5000 [27:31:00<15:28:26, 29.30s/it]
 62%|██████▏   | 3100/5000 [27:31:33<15:58:08, 30.26s/it]
                                                         
 62%|██████▏   | 3100/5000 [27:31:33<15:58:08, 30.26s/it]
 62%|██████▏   | 3101/5000 [27:32:00<15:27:58, 29.32s/it]
 62%|██████▏   | 3102/5000 [27:32:33<16:02:32, 30.43s/it]
 62%|██████▏   | 3103/5000 [27:33:04<16:13:55, 30.80s/it]
 62%|██████▏   | 3104/5000 [27:33:34<16:00:36, 30.40s/it]
 62%|██████▏   | 3105/5000 [27:34:04<16:01:07, 30.43s/it]
 62%|██████▏   | 3106/5000 [27:34:36<16:09:36, 30.72s/it]
 62%|██████▏   | 3107/5000 [27:35:09<16:30:18, 31.39s/it]
 62%|██████▏   | 3108/5000 [27:35:44<17:04:44, 32.50s/it]
 62%|██████▏   | 3109/5000 [27:36:11<16:11:08, 30.81s/it]
 62%|██████▏   | 3110/5000 [27:36:46<16:50:50, 32.09s/it]
 62%|██████▏   | 3111/5000 [27:37:13<16:04:53, 30.65s/it]
 62%|██████▏   | 3112/5000 [27:37:48<16:44:28, 31.92s/it]
 62%|██████▏   | 3113/5000 [27:38:15<16:01:34, 30.57s/it]
 62%|██████▏   | 3114/5000 [27:38:51<16:52:05, 32.20s/it]
 62%|██████▏   | 3115/5000 [27:39:20<16:23:02, 31.29s/it]
 62%|██████▏   | 3116/5000 [27:39:50<16:06:36, 30.78s/it]
 62%|██████▏   | 3117/5000 [27:40:20<16:02:02, 30.65s/it]
 62%|██████▏   | 3118/5000 [27:40:54<16:25:00, 31.40s/it]
 62%|██████▏   | 3119/5000 [27:41:24<16:19:21, 31.24s/it]
 62%|██████▏   | 3120/5000 [27:41:57<16:33:47, 31.72s/it]
 62%|██████▏   | 3121/5000 [27:42:24<15:49:14, 30.31s/it]
 62%|██████▏   | 3122/5000 [27:43:01<16:45:21, 32.12s/it]
 62%|██████▏   | 3123/5000 [27:43:28<15:56:55, 30.59s/it]
 62%|██████▏   | 3124/5000 [27:44:17<18:49:57, 36.14s/it]
 62%|██████▎   | 3125/5000 [27:44:44<17:24:13, 33.42s/it]
                                                         
 62%|██████▎   | 3125/5000 [27:44:44<17:24:13, 33.42s/it]
 63%|██████▎   | 3126/5000 [27:45:17<17:17:25, 33.22s/it]
 63%|██████▎   | 3127/5000 [27:45:44<16:21:39, 31.45s/it]
 63%|██████▎   | 3128/5000 [27:46:20<17:05:58, 32.88s/it]
 63%|██████▎   | 3129/5000 [27:46:51<16:51:27, 32.44s/it]
 63%|██████▎   | 3130/5000 [27:47:26<17:05:52, 32.92s/it]
 63%|██████▎   | 3131/5000 [27:48:06<18:11:47, 35.05s/it]
 63%|██████▎   | 3132/5000 [27:48:33<16:56:32, 32.65s/it]
 63%|██████▎   | 3133/5000 [27:49:07<17:16:10, 33.30s/it]
 63%|██████▎   | 3134/5000 [27:49:38<16:50:22, 32.49s/it]
 63%|██████▎   | 3135/5000 [27:50:09<16:34:39, 32.00s/it]
 63%|██████▎   | 3136/5000 [27:50:39<16:14:39, 31.37s/it]
 63%|██████▎   | 3137/5000 [27:51:10<16:14:36, 31.39s/it]
 63%|██████▎   | 3138/5000 [27:51:50<17:31:52, 33.89s/it]
 63%|██████▎   | 3139/5000 [27:52:21<17:05:34, 33.07s/it]
 63%|██████▎   | 3140/5000 [27:52:59<17:50:37, 34.54s/it]
 63%|██████▎   | 3141/5000 [27:53:38<18:28:36, 35.78s/it]
 63%|██████▎   | 3142/5000 [27:54:05<17:07:57, 33.20s/it]
 63%|██████▎   | 3143/5000 [27:54:42<17:44:03, 34.38s/it]
 63%|██████▎   | 3144/5000 [27:55:09<16:36:46, 32.22s/it]
 63%|██████▎   | 3145/5000 [27:55:57<19:01:24, 36.92s/it]
 63%|██████▎   | 3146/5000 [27:56:24<17:30:23, 33.99s/it]
 63%|██████▎   | 3147/5000 [27:56:59<17:31:54, 34.06s/it]
 63%|██████▎   | 3148/5000 [27:57:37<18:15:11, 35.48s/it]
 63%|██████▎   | 3149/5000 [27:57:58<15:55:58, 30.99s/it]
 63%|██████▎   | 3150/5000 [27:58:08<12:45:17, 24.82s/it]
                                                         
 63%|██████▎   | 3150/5000 [27:58:08<12:45:17, 24.82s/it]
 63%|██████▎   | 3151/5000 [27:58:19<10:33:02, 20.54s/it]
 63%|██████▎   | 3152/5000 [27:58:29<8:59:11, 17.51s/it] {'eval_loss': 0.25769394636154175, 'eval_wer': 8.695623928070265, 'eval_runtime': 3033.1756, 'eval_samples_per_second': 5.117, 'eval_steps_per_second': 0.08, 'epoch': 12.02}
+{'loss': 0.0242, 'learning_rate': 4.391111111111112e-06, 'epoch': 12.02}
+{'loss': 0.0257, 'learning_rate': 4.3355555555555565e-06, 'epoch': 12.03}
+{'loss': 0.0264, 'learning_rate': 4.2800000000000005e-06, 'epoch': 12.03}
+{'loss': 0.0253, 'learning_rate': 4.2244444444444446e-06, 'epoch': 12.04}
+{'loss': 0.0274, 'learning_rate': 4.168888888888889e-06, 'epoch': 12.04}
+{'loss': 0.0295, 'learning_rate': 4.1133333333333335e-06, 'epoch': 12.05}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 63%|██████▎   | 3153/5000 [28:08:32<99:03:14, 193.07s/it]
 63%|██████▎   | 3154/5000 [28:09:13<75:36:02, 147.43s/it]
 63%|██████▎   | 3155/5000 [28:09:40<57:04:29, 111.37s/it]
 63%|██████▎   | 3156/5000 [28:10:14<45:10:00, 88.18s/it] 
 63%|██████▎   | 3157/5000 [28:10:52<37:22:08, 72.99s/it]
 63%|██████▎   | 3158/5000 [28:11:28<31:43:23, 62.00s/it]
 63%|██████▎   | 3159/5000 [28:11:58<26:45:44, 52.33s/it]
 63%|██████▎   | 3160/5000 [28:12:41<25:15:52, 49.43s/it]
 63%|██████▎   | 3161/5000 [28:13:23<24:08:04, 47.25s/it]
 63%|██████▎   | 3162/5000 [28:13:50<21:01:51, 41.19s/it]
 63%|██████▎   | 3163/5000 [28:14:58<25:13:37, 49.44s/it]
 63%|██████▎   | 3164/5000 [28:15:35<23:17:17, 45.66s/it]
 63%|██████▎   | 3165/5000 [28:16:03<20:28:31, 40.17s/it]
 63%|██████▎   | 3166/5000 [28:16:43<20:31:16, 40.28s/it]
 63%|██████▎   | 3167/5000 [28:17:13<18:54:48, 37.15s/it]
 63%|██████▎   | 3168/5000 [28:17:50<18:51:59, 37.07s/it]
 63%|██████▎   | 3169/5000 [28:18:22<18:10:27, 35.73s/it]
 63%|██████▎   | 3170/5000 [28:18:49<16:44:07, 32.92s/it]
 63%|██████▎   | 3171/5000 [28:19:23<16:54:58, 33.30s/it]
 63%|██████▎   | 3172/5000 [28:19:53<16:24:26, 32.31s/it]
 63%|██████▎   | 3173/5000 [28:20:26<16:33:37, 32.63s/it]
 63%|██████▎   | 3174/5000 [28:21:04<17:17:44, 34.10s/it]
 64%|██████▎   | 3175/5000 [28:21:31<16:14:53, 32.05s/it]
                                                         
 64%|██████▎   | 3175/5000 [28:21:31<16:14:53, 32.05s/it]
 64%|██████▎   | 3176/5000 [28:22:04<16:22:38, 32.32s/it]
 64%|██████▎   | 3177/5000 [28:22:34<16:03:07, 31.70s/it]
 64%|██████▎   | 3178/5000 [28:23:04<15:39:12, 30.93s/it]
 64%|██████▎   | 3179/5000 [28:23:33<15:25:16, 30.49s/it]
 64%|██████▎   | 3180/5000 [28:24:03<15:17:34, 30.25s/it]
 64%|██████▎   | 3181/5000 [28:24:31<15:00:51, 29.72s/it]
 64%|██████▎   | 3182/5000 [28:25:03<15:17:38, 30.29s/it]
 64%|██████▎   | 3183/5000 [28:25:36<15:39:50, 31.03s/it]
 64%|██████▎   | 3184/5000 [28:26:07<15:41:08, 31.10s/it]
 64%|██████▎   | 3185/5000 [28:26:36<15:20:28, 30.43s/it]
 64%|██████▎   | 3186/5000 [28:27:13<16:23:36, 32.53s/it]
 64%|██████▎   | 3187/5000 [28:27:50<17:01:29, 33.81s/it]
 64%|██████▍   | 3188/5000 [28:28:25<17:09:01, 34.07s/it]
 64%|██████▍   | 3189/5000 [28:29:05<18:05:35, 35.97s/it]
 64%|██████▍   | 3190/5000 [28:29:32<16:43:10, 33.25s/it]
 64%|██████▍   | 3191/5000 [28:30:01<16:06:45, 32.06s/it]
 64%|██████▍   | 3192/5000 [28:30:37<16:42:16, 33.26s/it]
 64%|██████▍   | 3193/5000 [28:31:04<15:45:09, 31.38s/it]
 64%|██████▍   | 3194/5000 [28:31:35<15:38:07, 31.17s/it]
 64%|██████▍   | 3195/5000 [28:32:02<15:02:16, 29.99s/it]
 64%|██████▍   | 3196/5000 [28:32:59<19:00:08, 37.92s/it]
 64%|██████▍   | 3197/5000 [28:33:29<17:56:18, 35.82s/it]
 64%|██████▍   | 3198/5000 [28:33:57<16:39:21, 33.28s/it]
 64%|██████▍   | 3199/5000 [28:34:39<17:58:16, 35.92s/it]
 64%|██████▍   | 3200/5000 [28:35:06<16:39:40, 33.32s/it]
                                                         
 64%|██████▍   | 3200/5000 [28:35:06<16:39:40, 33.32s/it]
 64%|██████▍   | 3201/5000 [28:36:01<19:50:28, 39.70s/it]
 64%|██████▍   | 3202/5000 [28:36:37<19:21:17, 38.75s/it]
 64%|██████▍   | 3203/5000 [28:37:08<18:09:49, 36.39s/it]
 64%|██████▍   | 3204/5000 [28:37:38<17:09:08, 34.38s/it]
 64%|██████▍   | 3205/5000 [28:38:10<16:46:07, 33.63s/it]
 64%|██████▍   | 3206/5000 [28:38:38<16:00:41, 32.13s/it]
 64%|██████▍   | 3207/5000 [28:39:08<15:37:49, 31.38s/it]
 64%|██████▍   | 3208/5000 [28:39:38<15:26:03, 31.01s/it]
 64%|██████▍   | 3209/5000 [28:40:09<15:21:52, 30.88s/it]
 64%|██████▍   | 3210/5000 [28:40:39<15:13:07, 30.61s/it]
 64%|██████▍   | 3211/5000 [28:41:06<14:43:55, 29.65s/it]
 64%|██████▍   | 3212/5000 [28:41:34<14:31:54, 29.26s/it]
 64%|██████▍   | 3213/5000 [28:42:03<14:22:36, 28.96s/it]
 64%|██████▍   | 3214/5000 [28:42:31<14:13:40, 28.68s/it]
 64%|██████▍   | 3215/5000 [28:43:00<14:17:47, 28.83s/it]
 64%|██████▍   | 3216/5000 [28:43:27<14:03:54, 28.38s/it]
 64%|██████▍   | 3217/5000 [28:43:57<14:15:10, 28.78s/it]
 64%|██████▍   | 3218/5000 [28:44:24<14:00:08, 28.29s/it]
 64%|██████▍   | 3219/5000 [28:44:53<14:05:08, 28.47s/it]
 64%|██████▍   | 3220/5000 [28:45:22<14:09:01, 28.62s/it]
 64%|██████▍   | 3221/5000 [28:45:53<14:33:15, 29.45s/it]
 64%|██████▍   | 3222/5000 [28:46:22<14:28:46, 29.32s/it]
 64%|██████▍   | 3223/5000 [28:46:54<14:52:17, 30.13s/it]
 64%|██████▍   | 3224/5000 [28:47:26<15:01:43, 30.46s/it]
 64%|██████▍   | 3225/5000 [28:47:56<14:57:40, 30.34s/it]
                                                         
 64%|██████▍   | 3225/5000 [28:47:56<14:57:40, 30.34s/it]
 65%|██████▍   | 3226/5000 [28:48:24<14:38:03, 29.70s/it]
 65%|██████▍   | 3227/5000 [28:48:53<14:29:59, 29.44s/it]
 65%|██████▍   | 3228/5000 [28:49:34<16:11:56, 32.91s/it]
 65%|██████▍   | 3229/5000 [28:50:05<15:52:42, 32.28s/it]
 65%|██████▍   | 3230/5000 [28:50:34<15:25:40, 31.38s/it]
 65%|██████▍   | 3231/5000 [28:51:02<14:59:14, 30.50s/it]
 65%|██████▍   | 3232/5000 [28:51:33<15:03:04, 30.65s/it]
 65%|██████▍   | 3233/5000 [28:52:00<14:31:55, 29.61s/it]
 65%|██████▍   | 3234/5000 [28:52:31<14:43:37, 30.02s/it]
 65%|██████▍   | 3235/5000 [28:52:58<14:13:01, 29.00s/it]
 65%|██████▍   | 3236/5000 [28:53:30<14:34:20, 29.74s/it]
 65%|██████▍   | 3237/5000 [28:54:00<14:43:39, 30.07s/it]
 65%|██████▍   | 3238/5000 [28:54:28<14:20:08, 29.29s/it]
 65%|██████▍   | 3239/5000 [28:55:01<14:51:57, 30.39s/it]
 65%|██████▍   | 3240/5000 [28:55:28<14:21:28, 29.37s/it]
 65%|██████▍   | 3241/5000 [28:56:08<15:58:59, 32.71s/it]
 65%|██████▍   | 3242/5000 [28:56:38<15:32:28, 31.82s/it]
 65%|██████▍   | 3243/5000 [28:57:05<14:51:02, 30.43s/it]
 65%|██████▍   | 3244/5000 [28:57:36<14:56:33, 30.63s/it]
 65%|██████▍   | 3245/5000 [28:58:04<14:26:03, 29.61s/it]
 65%|██████▍   | 3246/5000 [28:58:37<14:56:51, 30.68s/it]
 65%|██████▍   | 3247/5000 [28:59:05<14:36:32, 30.00s/it]
 65%|██████▍   | 3248/5000 [28:59:34<14:27:30, 29.71s/it]
 65%|██████▍   | 3249/5000 [29:00:06<14:41:41, 30.21s/it]
 65%|██████▌   | 3250/5000 [29:00:33<14:17:11, 29.39s/it]
                                                         
 65%|██████▌   | 3250/5000 [29:00:33<14:17:11, 29.39s/it]
 65%|██████▌   | 3251/5000 [29:01:04<14:31:08, 29.89s/it]
 65%|██████▌   | 3252/5000 [29:01:32<14:16:11, 29.39s/it]
 65%|██████▌   | 3253/5000 [29:02:02<14:20:44, 29.56s/it]
 65%|██████▌   | 3254/5000 [29:02:33<14:33:44, 30.03s/it]
 65%|██████▌   | 3255/5000 [29:03:03<14:28:31, 29.86s/it]
 65%|██████▌   | 3256/5000 [29:03:31<14:16:41, 29.47s/it]
 65%|██████▌   | 3257/5000 [29:04:01<14:13:47, 29.39s/it]
 65%|██████▌   | 3258/5000 [29:04:30<14:13:33, 29.40s/it]
 65%|██████▌   | 3259/5000 [29:04:59<14:06:12, 29.16s/it]
 65%|██████▌   | 3260/5000 [29:05:29<14:13:58, 29.45s/it]
 65%|██████▌   | 3261/5000 [29:05:57<14:04:21, 29.13s/it]
 65%|██████▌   | 3262/5000 [29:06:41<16:08:11, 33.42s/it]
 65%|██████▌   | 3263/5000 [29:07:08<15:13:20, 31.55s/it]
 65%|██████▌   | 3264/5000 [29:08:03<18:40:13, 38.72s/it]
 65%|██████▌   | 3265/5000 [29:08:30<16:56:12, 35.14s/it]
 65%|██████▌   | 3266/5000 [29:09:25<19:48:02, 41.11s/it]
 65%|██████▌   | 3267/5000 [29:09:59<18:47:09, 39.02s/it]
 65%|██████▌   | 3268/5000 [29:10:26<17:02:32, 35.42s/it]
 65%|██████▌   | 3269/5000 [29:10:56<16:11:37, 33.68s/it]
 65%|██████▌   | 3270/5000 [29:11:26<15:43:35, 32.73s/it]
 65%|██████▌   | 3271/5000 [29:12:00<15:48:36, 32.92s/it]
 65%|██████▌   | 3272/5000 [29:12:34<16:03:19, 33.45s/it]
 65%|██████▌   | 3273/5000 [29:13:01<15:06:36, 31.50s/it]
 65%|██████▌   | 3274/5000 [29:13:41<16:15:24, 33.91s/it]
 66%|██████▌   | 3275/5000 [29:14:09<15:26:39, 32.23s/it]
                                                         
 66%|██████▌   | 3275/5000 [29:14:09<15:26:39, 32.23s/it]
 66%|██████▌   | 3276/5000 [29:14:39<15:02:08, 31.40s/it]
 66%|██████▌   | 3277/5000 [29:15:17<15:57:26, 33.34s/it]
 66%|██████▌   | 3278/5000 [29:15:44<15:02:37, 31.45s/it]
 66%|██████▌   | 3279/5000 [29:16:21<15:53:45, 33.25s/it]
 66%|██████▌   | 3280/5000 [29:16:48<14:59:35, 31.38s/it]
 66%|██████▌   | 3281/5000 [29:17:43<18:19:33, 38.38s/it]
 66%|██████▌   | 3282/5000 [29:18:17<17:43:15, 37.13s/it]
 66%|██████▌   | 3283/5000 [29:18:54<17:40:21, 37.05s/it]
 66%|██████▌   | 3284/5000 [29:19:35<18:15:10, 38.29s/it]
 66%|██████▌   | 3285/5000 [29:20:17<18:47:31, 39.45s/it]
 66%|██████▌   | 3286/5000 [29:20:44<17:00:11, 35.71s/it]
 66%|██████▌   | 3287/5000 [29:21:24<17:31:42, 36.84s/it]
 66%|██████▌   | 3288/5000 [29:21:51<16:07:35, 33.91s/it]
 66%|██████▌   | 3289/5000 [29:22:32<17:10:26, 36.13s/it]
 66%|██████▌   | 3290/5000 [29:23:16<18:16:28, 38.47s/it]
 66%|██████▌   | 3291/5000 [29:23:49<17:30:28, 36.88s/it]
 66%|██████▌   | 3292/5000 [29:24:23<17:06:32, 36.06s/it]
 66%|██████▌   | 3293/5000 [29:24:54<16:16:34, 34.33s/it]
 66%|██████▌   | 3294/5000 [29:25:28<16:15:30, 34.31s/it]
 66%|██████▌   | 3295/5000 [29:25:57<15:33:27, 32.85s/it]
 66%|██████▌   | 3296/5000 [29:26:38<16:41:28, 35.26s/it]
 66%|██████▌   | 3297/5000 [29:27:16<17:02:25, 36.02s/it]
 66%|██████▌   | 3298/5000 [29:27:43<15:43:14, 33.25s/it]
 66%|██████▌   | 3299/5000 [29:28:18<16:00:08, 33.87s/it]
 66%|██████▌   | 3300/5000 [29:28:49<15:38:51, 33.14s/it]
                                                         
 66%|██████▌   | 3300/5000 [29:28:49<15:38:51, 33.14s/it]
 66%|██████▌   | 3301/5000 [29:29:16<14:46:14, 31.30s/it]
 66%|██████▌   | 3302/5000 [29:29:55<15:45:51, 33.42s/it]
 66%|██████▌   | 3303/5000 [29:30:22<14:51:35, 31.52s/it]
 66%|██████▌   | 3304/5000 [29:30:55<15:07:01, 32.09s/it]
 66%|██████▌   | 3305/5000 [29:31:25<14:48:41, 31.46s/it]
 66%|██████▌   | 3306/5000 [29:31:53<14:11:54, 30.17s/it]
 66%|██████▌   | 3307/5000 [29:32:28<14:58:15, 31.83s/it]
 66%|██████▌   | 3308/5000 [29:32:55<14:11:52, 30.21s/it]
 66%|██████▌   | 3309/5000 [29:33:24<14:03:19, 29.92s/it]
 66%|██████▌   | 3310/5000 [29:33:52<13:47:20, 29.37s/it]
 66%|██████▌   | 3311/5000 [29:34:19<13:28:52, 28.73s/it]
 66%|██████▌   | 3312/5000 [29:34:51<13:52:01, 29.57s/it]
 66%|██████▋   | 3313/5000 [29:35:18<13:30:08, 28.81s/it]
 66%|██████▋   | 3314/5000 [29:35:49<13:46:14, 29.40s/it]
 66%|██████▋   | 3315/5000 [29:36:19<13:51:12, 29.60s/it]
 66%|██████▋   | 3316/5000 [29:36:46<13:30:11, 28.87s/it]
 66%|██████▋   | 3317/5000 [29:37:15<13:28:28, 28.82s/it]
 66%|██████▋   | 3318/5000 [29:37:44<13:33:51, 29.03s/it]
 66%|██████▋   | 3319/5000 [29:38:13<13:33:47, 29.05s/it]
 66%|██████▋   | 3320/5000 [29:38:42<13:33:57, 29.07s/it]
 66%|██████▋   | 3321/5000 [29:39:12<13:36:27, 29.18s/it]
 66%|██████▋   | 3322/5000 [29:39:43<13:56:23, 29.91s/it]
 66%|██████▋   | 3323/5000 [29:40:10<13:31:08, 29.02s/it]
 66%|██████▋   | 3324/5000 [29:40:43<14:00:20, 30.08s/it]
 66%|██████▋   | 3325/5000 [29:41:14<14:07:26, 30.36s/it]
                                                         
 66%|██████▋   | 3325/5000 [29:41:14<14:07:26, 30.36s/it]
 67%|██████▋   | 3326/5000 [29:41:44<14:08:21, 30.41s/it]
 67%|██████▋   | 3327/5000 [29:42:14<13:59:43, 30.12s/it]
 67%|██████▋   | 3328/5000 [29:42:43<13:50:58, 29.82s/it]
 67%|██████▋   | 3329/5000 [29:43:12<13:44:55, 29.62s/it]
 67%|██████▋   | 3330/5000 [29:43:40<13:29:14, 29.07s/it]
 67%|██████▋   | 3331/5000 [29:44:08<13:25:06, 28.94s/it]
 67%|██████▋   | 3332/5000 [29:44:38<13:28:34, 29.09s/it]
 67%|██████▋   | 3333/5000 [29:45:04<13:06:05, 28.29s/it]
 67%|██████▋   | 3334/5000 [29:45:33<13:06:39, 28.33s/it]
 67%|██████▋   | 3335/5000 [29:46:03<13:23:36, 28.96s/it]
 67%|██████▋   | 3336/5000 [29:46:30<13:09:25, 28.46s/it]
 67%|██████▋   | 3337/5000 [29:47:01<13:29:31, 29.21s/it]
 67%|██████▋   | 3338/5000 [29:47:30<13:23:27, 29.01s/it]
 67%|██████▋   | 3339/5000 [29:47:58<13:14:51, 28.71s/it]
 67%|██████▋   | 3340/5000 [29:48:27<13:15:11, 28.74s/it]
 67%|██████▋   | 3341/5000 [29:48:57<13:24:47, 29.11s/it]
 67%|██████▋   | 3342/5000 [29:49:25<13:15:47, 28.80s/it]
 67%|██████▋   | 3343/5000 [29:49:54<13:15:26, 28.80s/it]
 67%|██████▋   | 3344/5000 [29:50:24<13:28:39, 29.30s/it]
 67%|██████▋   | 3345/5000 [29:50:53<13:25:03, 29.19s/it]
 67%|██████▋   | 3346/5000 [29:51:24<13:41:51, 29.81s/it]
 67%|██████▋   | 3347/5000 [29:52:01<14:35:03, 31.76s/it]
 67%|██████▋   | 3348/5000 [29:52:29<14:08:26, 30.82s/it]
 67%|██████▋   | 3349/5000 [29:52:59<13:59:41, 30.52s/it]
 67%|██████▋   | 3350/5000 [29:53:28<13:42:47, 29.92s/it]
                                                         
 67%|██████▋   | 3350/5000 [29:53:28<13:42:47, 29.92s/it]
 67%|██████▋   | 3351/5000 [29:53:58<13:43:15, 29.95s/it]
 67%|██████▋   | 3352/5000 [29:54:26<13:32:48, 29.59s/it]
 67%|██████▋   | 3353/5000 [29:54:56<13:35:11, 29.70s/it]
 67%|██████▋   | 3354/5000 [29:55:35<14:45:22, 32.27s/it]
 67%|██████▋   | 3355/5000 [29:56:02<14:04:09, 30.79s/it]
 67%|██████▋   | 3356/5000 [29:56:35<14:20:20, 31.40s/it]
 67%|██████▋   | 3357/5000 [29:57:02<13:45:27, 30.14s/it]
 67%|██████▋   | 3358/5000 [29:57:34<14:04:03, 30.84s/it]
 67%|██████▋   | 3359/5000 [29:58:05<14:01:15, 30.76s/it]
 67%|██████▋   | 3360/5000 [29:58:34<13:47:31, 30.28s/it]
 67%|██████▋   | 3361/5000 [29:59:04<13:43:35, 30.15s/it]
 67%|██████▋   | 3362/5000 [29:59:33<13:30:51, 29.70s/it]
 67%|██████▋   | 3363/5000 [30:00:05<13:53:13, 30.54s/it]
 67%|██████▋   | 3364/5000 [30:00:34<13:37:00, 29.96s/it]
 67%|██████▋   | 3365/5000 [30:01:04<13:35:10, 29.91s/it]
 67%|██████▋   | 3366/5000 [30:01:33<13:28:21, 29.68s/it]
 67%|██████▋   | 3367/5000 [30:02:02<13:27:37, 29.67s/it]
 67%|██████▋   | 3368/5000 [30:02:33<13:31:54, 29.85s/it]
 67%|██████▋   | 3369/5000 [30:03:01<13:22:10, 29.51s/it]
 67%|██████▋   | 3370/5000 [30:03:32<13:30:53, 29.85s/it]
 67%|██████▋   | 3371/5000 [30:04:00<13:18:48, 29.42s/it]
 67%|██████▋   | 3372/5000 [30:04:30<13:20:56, 29.52s/it]
 67%|██████▋   | 3373/5000 [30:04:58<13:10:33, 29.15s/it]
 67%|██████▋   | 3374/5000 [30:05:30<13:26:26, 29.76s/it]
 68%|██████▊   | 3375/5000 [30:05:59<13:26:22, 29.77s/it]
                                                         
 68%|██████▊   | 3375/5000 [30:05:59<13:26:22, 29.77s/it]
 68%|██████▊   | 3376/5000 [30:06:28<13:16:23, 29.42s/it]
 68%|██████▊   | 3377/5000 [30:06:59<13:26:12, 29.80s/it]
 68%|██████▊   | 3378/5000 [30:07:27<13:15:09, 29.41s/it]
 68%|██████▊   | 3379/5000 [30:08:02<14:00:01, 31.09s/it]
 68%|██████▊   | 3380/5000 [30:08:35<14:10:24, 31.50s/it]
 68%|██████▊   | 3381/5000 [30:09:02<13:33:26, 30.15s/it]
 68%|██████▊   | 3382/5000 [30:09:38<14:25:29, 32.09s/it]
 68%|██████▊   | 3383/5000 [30:10:07<13:55:49, 31.01s/it]
 68%|██████▊   | 3384/5000 [30:10:37<13:50:59, 30.85s/it]
 68%|██████▊   | 3385/5000 [30:11:04<13:20:27, 29.74s/it]
 68%|██████▊   | 3386/5000 [30:11:36<13:31:46, 30.18s/it]
 68%|██████▊   | 3387/5000 [30:12:05<13:28:51, 30.09s/it]
 68%|██████▊   | 3388/5000 [30:12:35<13:20:07, 29.78s/it]
 68%|██████▊   | 3389/5000 [30:13:05<13:23:02, 29.91s/it]
 68%|██████▊   | 3390/5000 [30:13:32<12:57:25, 28.97s/it]
 68%|██████▊   | 3391/5000 [30:14:03<13:18:12, 29.77s/it]
 68%|██████▊   | 3392/5000 [30:14:15<10:55:43, 24.47s/it]
 68%|██████▊   | 3393/5000 [30:14:26<9:02:21, 20.25s/it] 
 68%|██████▊   | 3394/5000 [30:14:36<7:43:14, 17.31s/it]
 68%|██████▊   | 3395/5000 [30:14:47<6:48:42, 15.28s/it]{'loss': 0.0433, 'learning_rate': 4.057777777777778e-06, 'epoch': 13.0}
+{'loss': 0.031, 'learning_rate': 4.002222222222222e-06, 'epoch': 13.01}
+{'loss': 0.0389, 'learning_rate': 3.946666666666667e-06, 'epoch': 13.01}
+{'loss': 0.0287, 'learning_rate': 3.891111111111111e-06, 'epoch': 13.02}
+{'loss': 0.0324, 'learning_rate': 3.835555555555555e-06, 'epoch': 13.02}
+{'loss': 0.0293, 'learning_rate': 3.7800000000000002e-06, 'epoch': 13.03}
+{'loss': 0.0281, 'learning_rate': 3.724444444444445e-06, 'epoch': 13.03}
+{'loss': 0.0218, 'learning_rate': 3.668888888888889e-06, 'epoch': 13.04}
+{'loss': 0.0249, 'learning_rate': 3.6133333333333336e-06, 'epoch': 13.04}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 68%|██████▊   | 3396/5000 [30:16:59<22:31:06, 50.54s/it]
 68%|██████▊   | 3397/5000 [30:17:28<19:36:52, 44.05s/it]
 68%|██████▊   | 3398/5000 [30:17:59<17:48:45, 40.03s/it]
 68%|██████▊   | 3399/5000 [30:18:26<16:00:47, 36.01s/it]
 68%|██████▊   | 3400/5000 [30:18:56<15:12:05, 34.20s/it]
                                                         
 68%|██████▊   | 3400/5000 [30:18:56<15:12:05, 34.20s/it]
 68%|██████▊   | 3401/5000 [30:19:23<14:18:24, 32.21s/it]
 68%|██████▊   | 3402/5000 [30:19:52<13:51:41, 31.23s/it]
 68%|██████▊   | 3403/5000 [30:20:21<13:32:51, 30.54s/it]
 68%|██████▊   | 3404/5000 [30:20:49<13:12:57, 29.81s/it]
 68%|██████▊   | 3405/5000 [30:21:19<13:14:51, 29.90s/it]
 68%|██████▊   | 3406/5000 [30:21:46<12:51:50, 29.05s/it]
 68%|██████▊   | 3407/5000 [30:22:16<12:59:07, 29.35s/it]
 68%|██████▊   | 3408/5000 [30:22:45<12:54:17, 29.18s/it]
 68%|██████▊   | 3409/5000 [30:23:13<12:42:09, 28.74s/it]
 68%|██████▊   | 3410/5000 [30:23:43<12:55:30, 29.26s/it]
 68%|██████▊   | 3411/5000 [30:24:10<12:36:43, 28.57s/it]
 68%|██████▊   | 3412/5000 [30:24:41<12:54:32, 29.26s/it]
 68%|██████▊   | 3413/5000 [30:25:10<12:48:31, 29.06s/it]
 68%|██████▊   | 3414/5000 [30:25:38<12:42:26, 28.84s/it]
 68%|██████▊   | 3415/5000 [30:26:06<12:33:31, 28.52s/it]
 68%|██████▊   | 3416/5000 [30:26:34<12:25:21, 28.23s/it]
 68%|██████▊   | 3417/5000 [30:27:02<12:29:23, 28.40s/it]
 68%|██████▊   | 3418/5000 [30:27:31<12:32:55, 28.56s/it]
 68%|██████▊   | 3419/5000 [30:27:59<12:28:15, 28.40s/it]
 68%|██████▊   | 3420/5000 [30:28:28<12:29:36, 28.47s/it]
 68%|██████▊   | 3421/5000 [30:28:56<12:23:45, 28.26s/it]
 68%|██████▊   | 3422/5000 [30:29:24<12:25:22, 28.34s/it]
 68%|██████▊   | 3423/5000 [30:29:53<12:32:22, 28.63s/it]
 68%|██████▊   | 3424/5000 [30:30:23<12:38:30, 28.88s/it]
 68%|██████▊   | 3425/5000 [30:30:51<12:32:17, 28.66s/it]
                                                         
 68%|██████▊   | 3425/5000 [30:30:51<12:32:17, 28.66s/it]
 69%|██████▊   | 3426/5000 [30:31:19<12:28:45, 28.54s/it]
 69%|██████▊   | 3427/5000 [30:31:48<12:30:28, 28.63s/it]
 69%|██████▊   | 3428/5000 [30:32:16<12:27:10, 28.52s/it]
 69%|██████▊   | 3429/5000 [30:32:45<12:24:13, 28.42s/it]
 69%|██████▊   | 3430/5000 [30:33:12<12:15:31, 28.11s/it]
 69%|██████▊   | 3431/5000 [30:33:40<12:11:09, 27.96s/it]
 69%|██████▊   | 3432/5000 [30:34:08<12:13:10, 28.06s/it]
 69%|██████▊   | 3433/5000 [30:34:38<12:25:19, 28.54s/it]
 69%|██████▊   | 3434/5000 [30:35:05<12:16:27, 28.22s/it]
 69%|██████▊   | 3435/5000 [30:35:33<12:17:35, 28.28s/it]
 69%|██████▊   | 3436/5000 [30:36:03<12:26:40, 28.64s/it]
 69%|██████▊   | 3437/5000 [30:36:30<12:14:24, 28.19s/it]
 69%|██████▉   | 3438/5000 [30:36:59<12:20:37, 28.45s/it]
 69%|██████▉   | 3439/5000 [30:37:27<12:16:28, 28.31s/it]
 69%|██████▉   | 3440/5000 [30:37:56<12:22:30, 28.56s/it]
 69%|██████▉   | 3441/5000 [30:38:26<12:33:33, 29.00s/it]
 69%|██████▉   | 3442/5000 [30:38:53<12:17:20, 28.40s/it]
 69%|██████▉   | 3443/5000 [30:39:21<12:11:56, 28.21s/it]
 69%|██████▉   | 3444/5000 [30:39:48<12:03:41, 27.91s/it]
 69%|██████▉   | 3445/5000 [30:40:17<12:06:52, 28.05s/it]
 69%|██████▉   | 3446/5000 [30:40:47<12:22:23, 28.66s/it]
 69%|██████▉   | 3447/5000 [30:41:14<12:11:16, 28.25s/it]
 69%|██████▉   | 3448/5000 [30:41:44<12:26:53, 28.87s/it]
 69%|██████▉   | 3449/5000 [30:42:12<12:14:00, 28.40s/it]
 69%|██████▉   | 3450/5000 [30:42:41<12:24:08, 28.81s/it]
                                                         
 69%|██████▉   | 3450/5000 [30:42:41<12:24:08, 28.81s/it]
 69%|██████▉   | 3451/5000 [30:43:10<12:19:44, 28.65s/it]
 69%|██████▉   | 3452/5000 [30:43:37<12:08:53, 28.25s/it]
 69%|██████▉   | 3453/5000 [30:44:06<12:17:30, 28.60s/it]
 69%|██████▉   | 3454/5000 [30:44:34<12:06:53, 28.21s/it]
 69%|██████▉   | 3455/5000 [30:45:03<12:10:50, 28.38s/it]
 69%|██████▉   | 3456/5000 [30:45:31<12:11:19, 28.42s/it]
 69%|██████▉   | 3457/5000 [30:45:59<12:10:50, 28.42s/it]
 69%|██████▉   | 3458/5000 [30:46:30<12:28:50, 29.14s/it]
 69%|██████▉   | 3459/5000 [30:46:57<12:12:32, 28.52s/it]
 69%|██████▉   | 3460/5000 [30:47:27<12:19:53, 28.83s/it]
 69%|██████▉   | 3461/5000 [30:47:55<12:14:21, 28.63s/it]
 69%|██████▉   | 3462/5000 [30:48:23<12:06:34, 28.35s/it]
 69%|██████▉   | 3463/5000 [30:48:52<12:10:32, 28.52s/it]
 69%|██████▉   | 3464/5000 [30:49:19<12:02:57, 28.24s/it]
 69%|██████▉   | 3465/5000 [30:49:48<12:09:36, 28.52s/it]
 69%|██████▉   | 3466/5000 [30:50:16<12:03:34, 28.30s/it]
 69%|██████▉   | 3467/5000 [30:50:44<11:58:00, 28.10s/it]
 69%|██████▉   | 3468/5000 [30:51:13<12:03:58, 28.35s/it]
 69%|██████▉   | 3469/5000 [30:51:41<12:03:30, 28.35s/it]
 69%|██████▉   | 3470/5000 [30:52:11<12:17:01, 28.90s/it]
 69%|██████▉   | 3471/5000 [30:52:38<12:02:22, 28.35s/it]
 69%|██████▉   | 3472/5000 [30:53:07<12:06:08, 28.51s/it]
 69%|██████▉   | 3473/5000 [30:53:36<12:07:22, 28.58s/it]
 69%|██████▉   | 3474/5000 [30:54:04<12:05:36, 28.53s/it]
 70%|██████▉   | 3475/5000 [30:54:34<12:12:33, 28.82s/it]
                                                         
 70%|██████▉   | 3475/5000 [30:54:34<12:12:33, 28.82s/it]
 70%|██████▉   | 3476/5000 [30:55:01<11:58:52, 28.30s/it]
 70%|██████▉   | 3477/5000 [30:55:32<12:17:52, 29.07s/it]
 70%|██████▉   | 3478/5000 [30:56:01<12:16:54, 29.05s/it]
 70%|██████▉   | 3479/5000 [30:56:29<12:10:32, 28.82s/it]
 70%|██████▉   | 3480/5000 [30:56:57<12:02:22, 28.51s/it]
 70%|██████▉   | 3481/5000 [30:57:25<11:56:43, 28.31s/it]
 70%|██████▉   | 3482/5000 [30:57:53<11:53:07, 28.19s/it]
 70%|██████▉   | 3483/5000 [30:58:21<11:49:53, 28.08s/it]
 70%|██████▉   | 3484/5000 [30:58:49<11:51:21, 28.15s/it]
 70%|██████▉   | 3485/5000 [30:59:17<11:53:40, 28.26s/it]
 70%|██████▉   | 3486/5000 [30:59:46<11:52:15, 28.23s/it]
 70%|██████▉   | 3487/5000 [31:00:14<11:55:47, 28.39s/it]
 70%|██████▉   | 3488/5000 [31:00:43<11:54:19, 28.35s/it]
 70%|██████▉   | 3489/5000 [31:01:10<11:50:16, 28.20s/it]
 70%|██████▉   | 3490/5000 [31:01:39<11:53:13, 28.34s/it]
 70%|██████▉   | 3491/5000 [31:02:06<11:42:45, 27.94s/it]
 70%|██████▉   | 3492/5000 [31:02:34<11:45:21, 28.06s/it]
 70%|██████▉   | 3493/5000 [31:03:05<12:00:54, 28.70s/it]
 70%|██████▉   | 3494/5000 [31:03:32<11:50:50, 28.32s/it]
 70%|██████▉   | 3495/5000 [31:04:02<12:04:19, 28.88s/it]
 70%|██████▉   | 3496/5000 [31:04:30<11:57:16, 28.61s/it]
 70%|██████▉   | 3497/5000 [31:04:58<11:50:12, 28.35s/it]
 70%|██████▉   | 3498/5000 [31:05:27<11:54:44, 28.55s/it]
 70%|██████▉   | 3499/5000 [31:05:54<11:41:43, 28.05s/it]
 70%|███████   | 3500/5000 [31:06:24<11:58:04, 28.72s/it]
                                                         
 70%|███████   | 3500/5000 [31:06:24<11:58:04, 28.72s/it]
 70%|███████   | 3501/5000 [31:06:53<11:56:59, 28.70s/it]
 70%|███████   | 3502/5000 [31:07:20<11:47:32, 28.34s/it]
 70%|███████   | 3503/5000 [31:07:49<11:51:13, 28.51s/it]
 70%|███████   | 3504/5000 [31:08:16<11:40:49, 28.11s/it]
 70%|███████   | 3505/5000 [31:08:47<12:00:39, 28.92s/it]
 70%|███████   | 3506/5000 [31:09:15<11:48:38, 28.46s/it]
 70%|███████   | 3507/5000 [31:09:44<11:58:05, 28.86s/it]
 70%|███████   | 3508/5000 [31:10:13<11:57:19, 28.85s/it]
 70%|███████   | 3509/5000 [31:10:41<11:51:09, 28.62s/it]
 70%|███████   | 3510/5000 [31:11:12<12:03:57, 29.15s/it]
 70%|███████   | 3511/5000 [31:11:40<11:55:04, 28.81s/it]
 70%|███████   | 3512/5000 [31:12:07<11:46:20, 28.48s/it]
 70%|███████   | 3513/5000 [31:12:36<11:45:49, 28.48s/it]
 70%|███████   | 3514/5000 [31:13:05<11:47:09, 28.55s/it]
 70%|███████   | 3515/5000 [31:13:34<11:50:10, 28.69s/it]
 70%|███████   | 3516/5000 [31:14:02<11:45:47, 28.54s/it]
 70%|███████   | 3517/5000 [31:14:30<11:46:42, 28.59s/it]
 70%|███████   | 3518/5000 [31:14:59<11:45:39, 28.57s/it]
 70%|███████   | 3519/5000 [31:15:27<11:42:50, 28.47s/it]
 70%|███████   | 3520/5000 [31:15:57<11:49:45, 28.77s/it]
 70%|███████   | 3521/5000 [31:16:26<11:55:29, 29.03s/it]
 70%|███████   | 3522/5000 [31:16:53<11:39:45, 28.41s/it]
 70%|███████   | 3523/5000 [31:17:22<11:41:38, 28.50s/it]
 70%|███████   | 3524/5000 [31:17:50<11:34:42, 28.24s/it]
 70%|███████   | 3525/5000 [31:18:18<11:36:55, 28.35s/it]
                                                         
 70%|███████   | 3525/5000 [31:18:18<11:36:55, 28.35s/it]
 71%|███████   | 3526/5000 [31:18:47<11:35:59, 28.33s/it]
 71%|███████   | 3527/5000 [31:19:14<11:26:30, 27.96s/it]
 71%|███████   | 3528/5000 [31:19:43<11:39:38, 28.52s/it]
 71%|███████   | 3529/5000 [31:20:11<11:33:41, 28.29s/it]
 71%|███████   | 3530/5000 [31:20:41<11:40:23, 28.59s/it]
 71%|███████   | 3531/5000 [31:21:10<11:44:03, 28.76s/it]
 71%|███████   | 3532/5000 [31:21:37<11:32:19, 28.30s/it]
 71%|███████   | 3533/5000 [31:22:05<11:30:31, 28.24s/it]
 71%|███████   | 3534/5000 [31:22:32<11:23:42, 27.98s/it]
 71%|███████   | 3535/5000 [31:23:02<11:32:09, 28.35s/it]
 71%|███████   | 3536/5000 [31:23:30<11:34:41, 28.47s/it]
 71%|███████   | 3537/5000 [31:23:58<11:25:05, 28.10s/it]
 71%|███████   | 3538/5000 [31:24:27<11:37:00, 28.60s/it]
 71%|███████   | 3539/5000 [31:24:54<11:25:05, 28.14s/it]
 71%|███████   | 3540/5000 [31:25:24<11:33:46, 28.51s/it]
 71%|███████   | 3541/5000 [31:25:52<11:30:53, 28.41s/it]
 71%|███████   | 3542/5000 [31:26:18<11:16:37, 27.84s/it]
 71%|███████   | 3543/5000 [31:26:47<11:23:53, 28.16s/it]
 71%|███████   | 3544/5000 [31:27:16<11:23:22, 28.16s/it]
 71%|███████   | 3545/5000 [31:27:45<11:30:55, 28.49s/it]
 71%|███████   | 3546/5000 [31:28:15<11:40:12, 28.89s/it]
 71%|███████   | 3547/5000 [31:28:43<11:33:24, 28.63s/it]
 71%|███████   | 3548/5000 [31:29:12<11:36:52, 28.80s/it]
 71%|███████   | 3549/5000 [31:29:41<11:39:07, 28.91s/it]
 71%|███████   | 3550/5000 [31:30:08<11:24:19, 28.32s/it]
                                                         
 71%|███████   | 3550/5000 [31:30:08<11:24:19, 28.32s/it]
 71%|███████   | 3551/5000 [31:30:37<11:32:15, 28.67s/it]
 71%|███████   | 3552/5000 [31:31:04<11:20:09, 28.18s/it]
 71%|███████   | 3553/5000 [31:31:33<11:23:21, 28.34s/it]
 71%|███████   | 3554/5000 [31:32:03<11:34:44, 28.83s/it]
 71%|███████   | 3555/5000 [31:32:30<11:22:30, 28.34s/it]
 71%|███████   | 3556/5000 [31:33:01<11:37:10, 28.97s/it]
 71%|███████   | 3557/5000 [31:33:28<11:25:54, 28.52s/it]
 71%|███████   | 3558/5000 [31:33:58<11:33:42, 28.86s/it]
 71%|███████   | 3559/5000 [31:34:28<11:42:05, 29.23s/it]
 71%|███████   | 3560/5000 [31:34:56<11:32:39, 28.86s/it]
 71%|███████   | 3561/5000 [31:35:24<11:28:53, 28.72s/it]
 71%|███████   | 3562/5000 [31:35:52<11:20:28, 28.39s/it]
 71%|███████▏  | 3563/5000 [31:36:21<11:22:43, 28.51s/it]
 71%|███████▏  | 3564/5000 [31:36:48<11:13:13, 28.13s/it]
 71%|███████▏  | 3565/5000 [31:37:18<11:23:28, 28.58s/it]
 71%|███████▏  | 3566/5000 [31:37:45<11:16:43, 28.31s/it]
 71%|███████▏  | 3567/5000 [31:38:13<11:11:36, 28.12s/it]
 71%|███████▏  | 3568/5000 [31:38:42<11:14:58, 28.28s/it]
 71%|███████▏  | 3569/5000 [31:39:10<11:15:41, 28.33s/it]
 71%|███████▏  | 3570/5000 [31:39:38<11:11:53, 28.19s/it]
 71%|███████▏  | 3571/5000 [31:40:06<11:12:39, 28.24s/it]
 71%|███████▏  | 3572/5000 [31:40:35<11:16:46, 28.44s/it]
 71%|███████▏  | 3573/5000 [31:41:05<11:22:37, 28.70s/it]
 71%|███████▏  | 3574/5000 [31:41:33<11:19:50, 28.60s/it]
 72%|███████▏  | 3575/5000 [31:42:02<11:24:03, 28.80s/it]
                                                         
 72%|███████▏  | 3575/5000 [31:42:02<11:24:03, 28.80s/it]
 72%|███████▏  | 3576/5000 [31:42:32<11:28:27, 29.01s/it]
 72%|███████▏  | 3577/5000 [31:42:59<11:18:09, 28.59s/it]
 72%|███████▏  | 3578/5000 [31:43:28<11:18:35, 28.63s/it]
 72%|███████▏  | 3579/5000 [31:43:56<11:11:43, 28.36s/it]
 72%|███████▏  | 3580/5000 [31:44:24<11:10:31, 28.33s/it]
 72%|███████▏  | 3581/5000 [31:44:54<11:22:54, 28.88s/it]
 72%|███████▏  | 3582/5000 [31:45:22<11:11:06, 28.40s/it]
 72%|███████▏  | 3583/5000 [31:45:52<11:25:43, 29.04s/it]
 72%|███████▏  | 3584/5000 [31:46:20<11:21:12, 28.86s/it]
 72%|███████▏  | 3585/5000 [31:46:49<11:21:16, 28.89s/it]
 72%|███████▏  | 3586/5000 [31:47:17<11:11:16, 28.48s/it]
 72%|███████▏  | 3587/5000 [31:47:44<11:03:48, 28.19s/it]
 72%|███████▏  | 3588/5000 [31:48:13<11:04:23, 28.23s/it]
 72%|███████▏  | 3589/5000 [31:48:40<10:56:03, 27.90s/it]
 72%|███████▏  | 3590/5000 [31:49:08<10:55:37, 27.90s/it]
 72%|███████▏  | 3591/5000 [31:49:35<10:50:08, 27.69s/it]
 72%|███████▏  | 3592/5000 [31:50:03<10:52:13, 27.79s/it]
 72%|███████▏  | 3593/5000 [31:50:33<11:04:15, 28.33s/it]
 72%|███████▏  | 3594/5000 [31:51:00<10:54:16, 27.92s/it]
 72%|███████▏  | 3595/5000 [31:51:28<10:59:10, 28.15s/it]
 72%|███████▏  | 3596/5000 [31:51:55<10:47:36, 27.68s/it]
 72%|███████▏  | 3597/5000 [31:52:23<10:50:12, 27.81s/it]
 72%|███████▏  | 3598/5000 [31:52:50<10:46:09, 27.65s/it]
 72%|███████▏  | 3599/5000 [31:53:19<10:50:25, 27.86s/it]
 72%|███████▏  | 3600/5000 [31:53:49<11:06:25, 28.56s/it]
                                                         
 72%|███████▏  | 3600/5000 [31:53:49<11:06:25, 28.56s/it]
 72%|███████▏  | 3601/5000 [31:54:16<10:57:30, 28.20s/it]
 72%|███████▏  | 3602/5000 [31:54:44<10:57:14, 28.21s/it]
 72%|███████▏  | 3603/5000 [31:55:12<10:50:16, 27.93s/it]
 72%|███████▏  | 3604/5000 [31:55:41<11:03:07, 28.50s/it]
 72%|███████▏  | 3605/5000 [31:56:09<10:55:21, 28.19s/it]
 72%|███████▏  | 3606/5000 [31:56:37<10:51:11, 28.03s/it]
 72%|███████▏  | 3607/5000 [31:57:05<10:50:46, 28.03s/it]
 72%|███████▏  | 3608/5000 [31:57:32<10:42:39, 27.70s/it]
 72%|███████▏  | 3609/5000 [31:58:00<10:46:12, 27.87s/it]
 72%|███████▏  | 3610/5000 [31:58:27<10:39:01, 27.58s/it]
 72%|███████▏  | 3611/5000 [31:58:55<10:39:56, 27.64s/it]
 72%|███████▏  | 3612/5000 [31:59:22<10:36:29, 27.51s/it]
 72%|███████▏  | 3613/5000 [31:59:51<10:49:26, 28.09s/it]
 72%|███████▏  | 3614/5000 [32:00:19<10:46:06, 27.97s/it]
 72%|███████▏  | 3615/5000 [32:00:46<10:39:38, 27.71s/it]
 72%|███████▏  | 3616/5000 [32:01:14<10:44:08, 27.93s/it]
 72%|███████▏  | 3617/5000 [32:01:42<10:37:52, 27.67s/it]
 72%|███████▏  | 3618/5000 [32:02:10<10:44:31, 27.98s/it]
 72%|███████▏  | 3619/5000 [32:02:39<10:51:29, 28.31s/it]
 72%|███████▏  | 3620/5000 [32:03:06<10:43:10, 27.96s/it]
 72%|███████▏  | 3621/5000 [32:03:35<10:44:41, 28.05s/it]
 72%|███████▏  | 3622/5000 [32:04:02<10:36:51, 27.73s/it]
 72%|███████▏  | 3623/5000 [32:04:30<10:40:44, 27.92s/it]
 72%|███████▏  | 3624/5000 [32:04:57<10:35:00, 27.69s/it]
 72%|███████▎  | 3625/5000 [32:05:26<10:42:06, 28.02s/it]
                                                         
 72%|███████▎  | 3625/5000 [32:05:26<10:42:06, 28.02s/it]
 73%|███████▎  | 3626/5000 [32:05:54<10:44:00, 28.12s/it]
 73%|███████▎  | 3627/5000 [32:06:22<10:43:18, 28.11s/it]
 73%|███████▎  | 3628/5000 [32:06:51<10:49:04, 28.38s/it]
 73%|███████▎  | 3629/5000 [32:07:19<10:40:54, 28.05s/it]
 73%|███████▎  | 3630/5000 [32:07:47<10:42:23, 28.13s/it]
 73%|███████▎  | 3631/5000 [32:08:15<10:37:26, 27.94s/it]
 73%|███████▎  | 3632/5000 [32:08:44<10:48:29, 28.44s/it]
 73%|███████▎  | 3633/5000 [32:09:14<10:57:50, 28.87s/it]
 73%|███████▎  | 3634/5000 [32:09:36<10:07:13, 26.67s/it]
 73%|███████▎  | 3635/5000 [32:09:46<8:15:21, 21.77s/it] 
 73%|███████▎  | 3636/5000 [32:09:56<6:57:33, 18.37s/it]
 73%|███████▎  | 3637/5000 [32:10:07<6:03:02, 15.98s/it]{'loss': 0.0314, 'learning_rate': 3.5577777777777785e-06, 'epoch': 14.0}
+{'loss': 0.0457, 'learning_rate': 3.5022222222222225e-06, 'epoch': 14.01}
+{'loss': 0.0354, 'learning_rate': 3.446666666666667e-06, 'epoch': 14.01}
+{'loss': 0.0405, 'learning_rate': 3.391111111111111e-06, 'epoch': 14.02}
+{'loss': 0.0354, 'learning_rate': 3.335555555555556e-06, 'epoch': 14.02}
+{'loss': 0.0395, 'learning_rate': 3.2800000000000004e-06, 'epoch': 14.03}
+{'loss': 0.0367, 'learning_rate': 3.2244444444444444e-06, 'epoch': 14.03}
+{'loss': 0.0389, 'learning_rate': 3.1688888888888893e-06, 'epoch': 14.04}
+{'loss': 0.037, 'learning_rate': 3.1133333333333337e-06, 'epoch': 14.04}
+{'loss': 0.0368, 'learning_rate': 3.0577777777777778e-06, 'epoch': 14.05}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 73%|███████▎  | 3638/5000 [32:12:08<18:02:37, 47.69s/it]
 73%|███████▎  | 3639/5000 [32:12:39<16:05:51, 42.58s/it]
 73%|███████▎  | 3640/5000 [32:13:08<14:34:40, 38.59s/it]
 73%|███████▎  | 3641/5000 [32:13:38<13:33:48, 35.93s/it]
 73%|███████▎  | 3642/5000 [32:14:08<12:53:51, 34.19s/it]
 73%|███████▎  | 3643/5000 [32:14:36<12:06:39, 32.13s/it]
 73%|███████▎  | 3644/5000 [32:15:10<12:25:09, 32.97s/it]
 73%|███████▎  | 3645/5000 [32:15:38<11:47:17, 31.32s/it]
 73%|███████▎  | 3646/5000 [32:16:07<11:34:49, 30.79s/it]
 73%|███████▎  | 3647/5000 [32:16:37<11:25:57, 30.42s/it]
 73%|███████▎  | 3648/5000 [32:17:07<11:23:18, 30.32s/it]
 73%|███████▎  | 3649/5000 [32:17:38<11:23:10, 30.34s/it]
 73%|███████▎  | 3650/5000 [32:18:05<11:02:03, 29.42s/it]
                                                         
 73%|███████▎  | 3650/5000 [32:18:05<11:02:03, 29.42s/it]
 73%|███████▎  | 3651/5000 [32:18:38<11:28:31, 30.62s/it]
 73%|███████▎  | 3652/5000 [32:19:05<11:03:12, 29.52s/it]
 73%|███████▎  | 3653/5000 [32:19:39<11:31:33, 30.80s/it]
 73%|███████▎  | 3654/5000 [32:20:10<11:34:08, 30.94s/it]
 73%|███████▎  | 3655/5000 [32:20:38<11:12:54, 30.02s/it]
 73%|███████▎  | 3656/5000 [32:21:09<11:18:54, 30.31s/it]
 73%|███████▎  | 3657/5000 [32:21:36<10:55:57, 29.31s/it]
 73%|███████▎  | 3658/5000 [32:22:05<10:50:03, 29.06s/it]
 73%|███████▎  | 3659/5000 [32:22:35<10:56:37, 29.38s/it]
 73%|███████▎  | 3660/5000 [32:23:04<10:58:54, 29.50s/it]
 73%|███████▎  | 3661/5000 [32:23:35<11:04:00, 29.75s/it]
 73%|███████▎  | 3662/5000 [32:24:08<11:23:50, 30.67s/it]
 73%|███████▎  | 3663/5000 [32:24:36<11:08:58, 30.02s/it]
 73%|███████▎  | 3664/5000 [32:25:07<11:12:26, 30.20s/it]
 73%|███████▎  | 3665/5000 [32:25:34<10:53:28, 29.37s/it]
 73%|███████▎  | 3666/5000 [32:26:04<10:57:23, 29.57s/it]
 73%|███████▎  | 3667/5000 [32:26:36<11:08:53, 30.11s/it]
 73%|███████▎  | 3668/5000 [32:27:03<10:48:47, 29.23s/it]
 73%|███████▎  | 3669/5000 [32:27:36<11:13:50, 30.38s/it]
 73%|███████▎  | 3670/5000 [32:28:05<11:02:40, 29.89s/it]
 73%|███████▎  | 3671/5000 [32:28:32<10:47:37, 29.24s/it]
 73%|███████▎  | 3672/5000 [32:29:02<10:52:00, 29.46s/it]
 73%|███████▎  | 3673/5000 [32:29:29<10:36:03, 28.76s/it]
 73%|███████▎  | 3674/5000 [32:30:03<11:11:03, 30.37s/it]
 74%|███████▎  | 3675/5000 [32:30:31<10:49:34, 29.41s/it]
                                                         
 74%|███████▎  | 3675/5000 [32:30:31<10:49:34, 29.41s/it]
 74%|███████▎  | 3676/5000 [32:31:02<11:04:02, 30.09s/it]
 74%|███████▎  | 3677/5000 [32:31:30<10:49:23, 29.45s/it]
 74%|███████▎  | 3678/5000 [32:31:58<10:37:48, 28.95s/it]
 74%|███████▎  | 3679/5000 [32:32:36<11:34:27, 31.54s/it]
 74%|███████▎  | 3680/5000 [32:33:04<11:12:53, 30.59s/it]
 74%|███████▎  | 3681/5000 [32:33:32<10:56:12, 29.85s/it]
 74%|███████▎  | 3682/5000 [32:34:11<11:52:31, 32.44s/it]
 74%|███████▎  | 3683/5000 [32:34:38<11:17:39, 30.87s/it]
 74%|███████▎  | 3684/5000 [32:35:16<12:01:57, 32.92s/it]
 74%|███████▎  | 3685/5000 [32:35:43<11:22:35, 31.15s/it]
 74%|███████▎  | 3686/5000 [32:36:13<11:14:37, 30.80s/it]
 74%|███████▎  | 3687/5000 [32:36:41<10:56:31, 30.00s/it]
 74%|███████▍  | 3688/5000 [32:37:08<10:41:27, 29.34s/it]
 74%|███████▍  | 3689/5000 [32:37:40<10:54:19, 29.95s/it]
 74%|███████▍  | 3690/5000 [32:38:09<10:51:26, 29.84s/it]
 74%|███████▍  | 3691/5000 [32:38:38<10:41:11, 29.39s/it]
 74%|███████▍  | 3692/5000 [32:39:08<10:44:33, 29.57s/it]
 74%|███████▍  | 3693/5000 [32:39:36<10:37:49, 29.28s/it]
 74%|███████▍  | 3694/5000 [32:40:04<10:29:27, 28.92s/it]
 74%|███████▍  | 3695/5000 [32:40:35<10:41:56, 29.51s/it]
 74%|███████▍  | 3696/5000 [32:41:03<10:26:16, 28.82s/it]
 74%|███████▍  | 3697/5000 [32:41:34<10:40:08, 29.48s/it]
 74%|███████▍  | 3698/5000 [32:42:01<10:25:18, 28.82s/it]
 74%|███████▍  | 3699/5000 [32:42:32<10:40:27, 29.54s/it]
 74%|███████▍  | 3700/5000 [32:43:02<10:45:35, 29.80s/it]
                                                         
 74%|███████▍  | 3700/5000 [32:43:02<10:45:35, 29.80s/it]
 74%|███████▍  | 3701/5000 [32:43:32<10:46:28, 29.86s/it]
 74%|███████▍  | 3702/5000 [32:44:03<10:48:40, 29.98s/it]
 74%|███████▍  | 3703/5000 [32:44:32<10:45:07, 29.84s/it]
 74%|███████▍  | 3704/5000 [32:45:02<10:43:42, 29.80s/it]
 74%|███████▍  | 3705/5000 [32:45:30<10:32:18, 29.30s/it]
 74%|███████▍  | 3706/5000 [32:46:01<10:39:39, 29.66s/it]
 74%|███████▍  | 3707/5000 [32:46:31<10:46:29, 30.00s/it]
 74%|███████▍  | 3708/5000 [32:47:02<10:48:44, 30.13s/it]
 74%|███████▍  | 3709/5000 [32:47:30<10:36:02, 29.56s/it]
 74%|███████▍  | 3710/5000 [32:47:58<10:27:15, 29.17s/it]
 74%|███████▍  | 3711/5000 [32:48:28<10:29:59, 29.32s/it]
 74%|███████▍  | 3712/5000 [32:48:56<10:18:30, 28.81s/it]
 74%|███████▍  | 3713/5000 [32:49:24<10:12:35, 28.56s/it]
 74%|███████▍  | 3714/5000 [32:49:53<10:15:10, 28.70s/it]
 74%|███████▍  | 3715/5000 [32:50:19<10:03:09, 28.16s/it]
 74%|███████▍  | 3716/5000 [32:50:51<10:24:57, 29.20s/it]
 74%|███████▍  | 3717/5000 [32:51:21<10:25:58, 29.27s/it]
 74%|███████▍  | 3718/5000 [32:51:49<10:20:34, 29.04s/it]
 74%|███████▍  | 3719/5000 [32:52:19<10:27:03, 29.37s/it]
 74%|███████▍  | 3720/5000 [32:52:47<10:18:37, 29.00s/it]
 74%|███████▍  | 3721/5000 [32:53:18<10:26:51, 29.41s/it]
 74%|███████▍  | 3722/5000 [32:53:49<10:39:19, 30.02s/it]
 74%|███████▍  | 3723/5000 [32:54:16<10:19:06, 29.09s/it]
 74%|███████▍  | 3724/5000 [32:54:44<10:14:21, 28.89s/it]
 74%|███████▍  | 3725/5000 [32:55:12<10:03:07, 28.38s/it]
                                                         
 74%|███████▍  | 3725/5000 [32:55:12<10:03:07, 28.38s/it]
 75%|███████▍  | 3726/5000 [32:55:40<9:59:16, 28.22s/it] 
 75%|███████▍  | 3727/5000 [32:56:17<10:58:19, 31.03s/it]
 75%|███████▍  | 3728/5000 [32:56:47<10:50:06, 30.67s/it]
 75%|███████▍  | 3729/5000 [32:57:17<10:46:04, 30.50s/it]
 75%|███████▍  | 3730/5000 [32:57:48<10:47:00, 30.57s/it]
 75%|███████▍  | 3731/5000 [32:58:18<10:46:06, 30.55s/it]
 75%|███████▍  | 3732/5000 [32:58:48<10:42:26, 30.40s/it]
 75%|███████▍  | 3733/5000 [32:59:16<10:22:02, 29.46s/it]
 75%|███████▍  | 3734/5000 [32:59:47<10:35:53, 30.14s/it]
 75%|███████▍  | 3735/5000 [33:00:14<10:16:39, 29.25s/it]
 75%|███████▍  | 3736/5000 [33:00:47<10:34:00, 30.10s/it]
 75%|███████▍  | 3737/5000 [33:01:18<10:40:59, 30.45s/it]
 75%|███████▍  | 3738/5000 [33:01:45<10:18:56, 29.43s/it]
 75%|███████▍  | 3739/5000 [33:02:14<10:17:33, 29.38s/it]
 75%|███████▍  | 3740/5000 [33:02:41<10:03:29, 28.74s/it]
 75%|███████▍  | 3741/5000 [33:03:12<10:12:24, 29.19s/it]
 75%|███████▍  | 3742/5000 [33:03:42<10:20:11, 29.58s/it]
 75%|███████▍  | 3743/5000 [33:04:11<10:13:55, 29.30s/it]
 75%|███████▍  | 3744/5000 [33:04:41<10:17:54, 29.52s/it]
 75%|███████▍  | 3745/5000 [33:05:11<10:23:38, 29.82s/it]
 75%|███████▍  | 3746/5000 [33:05:42<10:29:15, 30.11s/it]
 75%|███████▍  | 3747/5000 [33:06:13<10:31:40, 30.25s/it]
 75%|███████▍  | 3748/5000 [33:06:44<10:39:17, 30.64s/it]
 75%|███████▍  | 3749/5000 [33:07:16<10:43:26, 30.86s/it]
 75%|███████▌  | 3750/5000 [33:07:43<10:19:00, 29.71s/it]
                                                         
 75%|███████▌  | 3750/5000 [33:07:43<10:19:00, 29.71s/it]
 75%|███████▌  | 3751/5000 [33:08:12<10:18:56, 29.73s/it]
 75%|███████▌  | 3752/5000 [33:08:46<10:41:56, 30.86s/it]
 75%|███████▌  | 3753/5000 [33:09:16<10:35:58, 30.60s/it]
 75%|███████▌  | 3754/5000 [33:09:46<10:34:08, 30.54s/it]
 75%|███████▌  | 3755/5000 [33:10:13<10:12:04, 29.50s/it]
 75%|███████▌  | 3756/5000 [33:10:45<10:24:35, 30.12s/it]
 75%|███████▌  | 3757/5000 [33:11:16<10:28:24, 30.33s/it]
 75%|███████▌  | 3758/5000 [33:11:46<10:24:40, 30.18s/it]
 75%|███████▌  | 3759/5000 [33:12:16<10:25:22, 30.24s/it]
 75%|███████▌  | 3760/5000 [33:12:43<10:06:20, 29.34s/it]
 75%|███████▌  | 3761/5000 [33:13:14<10:16:25, 29.85s/it]
 75%|███████▌  | 3762/5000 [33:13:44<10:18:16, 29.96s/it]
 75%|███████▌  | 3763/5000 [33:14:13<10:06:30, 29.42s/it]
 75%|███████▌  | 3764/5000 [33:14:43<10:14:11, 29.82s/it]
 75%|███████▌  | 3765/5000 [33:15:11<10:00:13, 29.16s/it]
 75%|███████▌  | 3766/5000 [33:15:38<9:49:13, 28.65s/it] 
 75%|███████▌  | 3767/5000 [33:16:09<10:00:36, 29.23s/it]
 75%|███████▌  | 3768/5000 [33:16:36<9:47:39, 28.62s/it] 
 75%|███████▌  | 3769/5000 [33:17:05<9:49:41, 28.74s/it]
 75%|███████▌  | 3770/5000 [33:17:39<10:21:27, 30.32s/it]
 75%|███████▌  | 3771/5000 [33:18:07<10:08:18, 29.70s/it]
 75%|███████▌  | 3772/5000 [33:18:37<10:09:42, 29.79s/it]
 75%|███████▌  | 3773/5000 [33:19:07<10:09:00, 29.78s/it]
 75%|███████▌  | 3774/5000 [33:19:41<10:35:11, 31.09s/it]
 76%|███████▌  | 3775/5000 [33:20:10<10:17:34, 30.25s/it]
                                                         
 76%|███████▌  | 3775/5000 [33:20:10<10:17:34, 30.25s/it]
 76%|███████▌  | 3776/5000 [33:20:40<10:20:14, 30.40s/it]
 76%|███████▌  | 3777/5000 [33:21:10<10:14:33, 30.15s/it]
 76%|███████▌  | 3778/5000 [33:21:38<9:58:42, 29.40s/it] 
 76%|███████▌  | 3779/5000 [33:22:08<10:05:42, 29.76s/it]
 76%|███████▌  | 3780/5000 [33:22:36<9:55:42, 29.30s/it] 
 76%|███████▌  | 3781/5000 [33:23:07<10:03:30, 29.71s/it]
 76%|███████▌  | 3782/5000 [33:23:37<10:04:37, 29.78s/it]
 76%|███████▌  | 3783/5000 [33:24:04<9:48:11, 29.00s/it] 
 76%|███████▌  | 3784/5000 [33:24:35<9:55:47, 29.40s/it]
 76%|███████▌  | 3785/5000 [33:25:12<10:41:22, 31.67s/it]
 76%|███████▌  | 3786/5000 [33:25:39<10:17:13, 30.51s/it]
 76%|███████▌  | 3787/5000 [33:26:10<10:18:01, 30.57s/it]
 76%|███████▌  | 3788/5000 [33:26:40<10:11:05, 30.25s/it]
 76%|███████▌  | 3789/5000 [33:27:10<10:12:10, 30.33s/it]
 76%|███████▌  | 3790/5000 [33:27:41<10:18:06, 30.65s/it]
 76%|███████▌  | 3791/5000 [33:28:10<10:03:16, 29.94s/it]
 76%|███████▌  | 3792/5000 [33:28:40<10:06:11, 30.11s/it]
 76%|███████▌  | 3793/5000 [33:29:10<10:04:51, 30.07s/it]
 76%|███████▌  | 3794/5000 [33:29:40<10:05:03, 30.10s/it]
 76%|███████▌  | 3795/5000 [33:30:11<10:06:34, 30.20s/it]
 76%|███████▌  | 3796/5000 [33:30:38<9:48:25, 29.32s/it] 
 76%|███████▌  | 3797/5000 [33:31:09<9:57:18, 29.79s/it]
 76%|███████▌  | 3798/5000 [33:31:36<9:41:41, 29.04s/it]
 76%|███████▌  | 3799/5000 [33:32:08<9:58:45, 29.91s/it]
 76%|███████▌  | 3800/5000 [33:32:39<10:04:22, 30.22s/it]
                                                         
 76%|███████▌  | 3800/5000 [33:32:39<10:04:22, 30.22s/it]
 76%|███████▌  | 3801/5000 [33:33:08<9:52:39, 29.66s/it] 
 76%|███████▌  | 3802/5000 [33:33:38<9:56:42, 29.89s/it]
 76%|███████▌  | 3803/5000 [33:34:06<9:47:13, 29.43s/it]
 76%|███████▌  | 3804/5000 [33:34:36<9:46:34, 29.43s/it]
 76%|███████▌  | 3805/5000 [33:35:04<9:41:52, 29.22s/it]
 76%|███████▌  | 3806/5000 [33:35:35<9:50:21, 29.67s/it]
 76%|███████▌  | 3807/5000 [33:36:03<9:38:52, 29.11s/it]
 76%|███████▌  | 3808/5000 [33:36:31<9:34:24, 28.91s/it]
 76%|███████▌  | 3809/5000 [33:36:59<9:26:46, 28.55s/it]
 76%|███████▌  | 3810/5000 [33:37:30<9:42:36, 29.38s/it]
 76%|███████▌  | 3811/5000 [33:37:58<9:29:12, 28.72s/it]
 76%|███████▌  | 3812/5000 [33:38:29<9:42:44, 29.43s/it]
 76%|███████▋  | 3813/5000 [33:38:59<9:48:42, 29.76s/it]
 76%|███████▋  | 3814/5000 [33:39:27<9:38:43, 29.28s/it]
 76%|███████▋  | 3815/5000 [33:39:58<9:45:25, 29.64s/it]
 76%|███████▋  | 3816/5000 [33:40:25<9:30:05, 28.89s/it]
 76%|███████▋  | 3817/5000 [33:40:57<9:45:47, 29.71s/it]
 76%|███████▋  | 3818/5000 [33:41:24<9:30:47, 28.97s/it]
 76%|███████▋  | 3819/5000 [33:41:56<9:46:17, 29.79s/it]
 76%|███████▋  | 3820/5000 [33:42:29<10:06:09, 30.82s/it]
 76%|��██████▋  | 3821/5000 [33:42:56<9:43:19, 29.69s/it] 
 76%|███████▋  | 3822/5000 [33:43:24<9:35:01, 29.29s/it]
 76%|███████▋  | 3823/5000 [33:43:52<9:23:04, 28.70s/it]
 76%|███████▋  | 3824/5000 [33:44:29<10:14:31, 31.35s/it]
 76%|███████▋  | 3825/5000 [33:44:59<10:05:00, 30.89s/it]
                                                         
 76%|███████▋  | 3825/5000 [33:44:59<10:05:00, 30.89s/it]
 77%|███████▋  | 3826/5000 [33:45:29<10:02:13, 30.78s/it]
 77%|███████▋  | 3827/5000 [33:46:02<10:12:39, 31.34s/it]
 77%|███████▋  | 3828/5000 [33:46:29<9:47:01, 30.05s/it] 
 77%|███████▋  | 3829/5000 [33:47:00<9:48:49, 30.17s/it]
 77%|███████▋  | 3830/5000 [33:47:27<9:30:17, 29.25s/it]
 77%|███████▋  | 3831/5000 [33:47:55<9:25:05, 29.00s/it]
 77%|███████▋  | 3832/5000 [33:48:24<9:24:03, 28.98s/it]
 77%|███████▋  | 3833/5000 [33:48:52<9:14:51, 28.53s/it]
 77%|███████▋  | 3834/5000 [33:49:23<9:34:06, 29.54s/it]
 77%|███████▋  | 3835/5000 [33:49:53<9:35:34, 29.64s/it]
 77%|███████▋  | 3836/5000 [33:50:22<9:28:49, 29.32s/it]
 77%|███████▋  | 3837/5000 [33:50:50<9:19:56, 28.89s/it]
 77%|███████▋  | 3838/5000 [33:51:18<9:13:28, 28.58s/it]
 77%|███████▋  | 3839/5000 [33:51:49<9:29:21, 29.42s/it]
 77%|███████▋  | 3840/5000 [33:52:16<9:16:17, 28.77s/it]
 77%|███████▋  | 3841/5000 [33:52:44<9:12:10, 28.59s/it]
 77%|███████▋  | 3842/5000 [33:53:12<9:03:48, 28.18s/it]
 77%|███████▋  | 3843/5000 [33:53:40<9:05:06, 28.27s/it]
 77%|███████▋  | 3844/5000 [33:54:08<9:01:23, 28.10s/it]
 77%|███████▋  | 3845/5000 [33:54:36<9:03:48, 28.25s/it]
 77%|███████▋  | 3846/5000 [33:55:05<9:02:42, 28.22s/it]
 77%|███████▋  | 3847/5000 [33:55:33<9:03:44, 28.30s/it]
 77%|███████▋  | 3848/5000 [33:56:01<9:01:42, 28.21s/it]
 77%|███████▋  | 3849/5000 [33:56:29<8:58:01, 28.05s/it]
 77%|███████▋  | 3850/5000 [33:56:57<8:56:32, 27.99s/it]
                                                        
 77%|███████▋  | 3850/5000 [33:56:57<8:56:32, 27.99s/it]
 77%|███████▋  | 3851/5000 [33:57:28<9:13:04, 28.88s/it]
 77%|███████▋  | 3852/5000 [33:57:55<9:03:38, 28.41s/it]
 77%|███████▋  | 3853/5000 [33:58:24<9:05:14, 28.52s/it]
 77%|███████▋  | 3854/5000 [33:58:51<8:57:10, 28.12s/it]
 77%|███████▋  | 3855/5000 [33:59:20<9:05:18, 28.57s/it]
 77%|███████▋  | 3856/5000 [33:59:48<8:56:24, 28.13s/it]
 77%|███████▋  | 3857/5000 [34:00:19<9:16:52, 29.23s/it]
 77%|███████▋  | 3858/5000 [34:00:48<9:12:10, 29.01s/it]
 77%|███████▋  | 3859/5000 [34:01:16<9:07:36, 28.80s/it]
 77%|███████▋  | 3860/5000 [34:01:48<9:22:58, 29.63s/it]
 77%|███████▋  | 3861/5000 [34:02:15<9:09:40, 28.96s/it]
 77%|███████▋  | 3862/5000 [34:02:45<9:13:05, 29.16s/it]
 77%|███████▋  | 3863/5000 [34:03:13<9:09:34, 29.00s/it]
 77%|███████▋  | 3864/5000 [34:03:41<9:01:11, 28.58s/it]
 77%|███████▋  | 3865/5000 [34:04:09<8:59:54, 28.54s/it]
 77%|███████▋  | 3866/5000 [34:04:40<9:09:45, 29.09s/it]
 77%|███████▋  | 3867/5000 [34:05:10<9:13:49, 29.33s/it]
 77%|███████▋  | 3868/5000 [34:05:37<9:01:49, 28.72s/it]
 77%|███████▋  | 3869/5000 [34:06:06<9:01:52, 28.75s/it]
 77%|███████▋  | 3870/5000 [34:06:36<9:08:20, 29.12s/it]
 77%|███████▋  | 3871/5000 [34:07:03<8:59:40, 28.68s/it]
 77%|███████▋  | 3872/5000 [34:07:32<8:59:26, 28.69s/it]
 77%|███████▋  | 3873/5000 [34:08:00<8:51:31, 28.30s/it]
 77%|███████▋  | 3874/5000 [34:08:31<9:11:29, 29.39s/it]
 78%|███████▊  | 3875/5000 [34:08:59<8:58:43, 28.73s/it]
                                                        
 78%|███████▊  | 3875/5000 [34:08:59<8:58:43, 28.73s/it]
 78%|███████▊  | 3876/5000 [34:09:29<9:07:41, 29.24s/it]
 78%|███████▊  | 3877/5000 [34:09:41<7:31:09, 24.10s/it]
 78%|███████▊  | 3878/5000 [34:09:52<6:13:43, 19.99s/it]
 78%|███████▊  | 3879/5000 [34:10:02<5:20:35, 17.16s/it]
 78%|███████▊  | 3880/5000 [34:10:13<4:42:32, 15.14s/it]{'loss': 0.0517, 'learning_rate': 3.0022222222222227e-06, 'epoch': 15.0}
+{'loss': 0.0638, 'learning_rate': 2.946666666666667e-06, 'epoch': 15.01}
+{'loss': 0.0588, 'learning_rate': 2.891111111111111e-06, 'epoch': 15.01}
+{'loss': 0.0636, 'learning_rate': 2.835555555555556e-06, 'epoch': 15.02}
+{'loss': 0.062, 'learning_rate': 2.7800000000000005e-06, 'epoch': 15.02}
+{'loss': 0.0596, 'learning_rate': 2.7244444444444445e-06, 'epoch': 15.03}
+{'loss': 0.075, 'learning_rate': 2.6688888888888894e-06, 'epoch': 15.03}
+{'loss': 0.0694, 'learning_rate': 2.6133333333333334e-06, 'epoch': 15.04}
+{'loss': 0.0686, 'learning_rate': 2.557777777777778e-06, 'epoch': 15.04}
+{'loss': 0.0682, 'learning_rate': 2.5022222222222224e-06, 'epoch': 15.05}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 78%|███████▊  | 3881/5000 [34:12:11<14:21:30, 46.19s/it]
 78%|███████▊  | 3882/5000 [34:12:44<13:03:19, 42.04s/it]
 78%|███████▊  | 3883/5000 [34:13:12<11:47:09, 37.99s/it]
 78%|███████▊  | 3884/5000 [34:13:44<11:10:45, 36.06s/it]
 78%|███████▊  | 3885/5000 [34:14:12<10:24:44, 33.62s/it]
 78%|███████▊  | 3886/5000 [34:14:44<10:19:54, 33.39s/it]
 78%|███████▊  | 3887/5000 [34:15:13<9:52:06, 31.92s/it] 
 78%|███████▊  | 3888/5000 [34:15:44<9:46:55, 31.67s/it]
 78%|███████▊  | 3889/5000 [34:16:15<9:40:20, 31.34s/it]
 78%|███████▊  | 3890/5000 [34:16:42<9:19:52, 30.26s/it]
 78%|███████▊  | 3891/5000 [34:17:13<9:21:05, 30.36s/it]
 78%|███████▊  | 3892/5000 [34:17:41<9:08:52, 29.72s/it]
 78%|███████▊  | 3893/5000 [34:18:13<9:20:03, 30.36s/it]
 78%|███████▊  | 3894/5000 [34:18:40<9:01:43, 29.39s/it]
 78%|███████▊  | 3895/5000 [34:19:10<9:06:25, 29.67s/it]
 78%|███████▊  | 3896/5000 [34:19:41<9:09:56, 29.89s/it]
 78%|███████▊  | 3897/5000 [34:20:09<9:00:31, 29.40s/it]
 78%|███████▊  | 3898/5000 [34:20:40<9:08:48, 29.88s/it]
 78%|███████▊  | 3899/5000 [34:21:07<8:54:10, 29.11s/it]
 78%|███████▊  | 3900/5000 [34:21:39<9:07:53, 29.89s/it]
                                                        
 78%|███████▊  | 3900/5000 [34:21:39<9:07:53, 29.89s/it]
 78%|███████▊  | 3901/5000 [34:22:06<8:52:35, 29.08s/it]
 78%|███████▊  | 3902/5000 [34:22:49<10:07:58, 33.22s/it]
 78%|███████▊  | 3903/5000 [34:23:18<9:41:18, 31.79s/it] 
 78%|███████▊  | 3904/5000 [34:23:48<9:31:50, 31.31s/it]
 78%|███████▊  | 3905/5000 [34:24:16<9:13:25, 30.32s/it]
 78%|███████▊  | 3906/5000 [34:24:46<9:13:07, 30.34s/it]
 78%|███████▊  | 3907/5000 [34:25:19<9:26:32, 31.10s/it]
 78%|███████▊  | 3908/5000 [34:25:48<9:11:32, 30.30s/it]
 78%|███████▊  | 3909/5000 [34:26:18<9:09:08, 30.20s/it]
 78%|███████▊  | 3910/5000 [34:26:45<8:55:05, 29.45s/it]
 78%|███████▊  | 3911/5000 [34:27:16<9:00:53, 29.80s/it]
 78%|███████▊  | 3912/5000 [34:27:44<8:50:57, 29.28s/it]
 78%|███████▊  | 3913/5000 [34:28:15<9:01:57, 29.92s/it]
 78%|███████▊  | 3914/5000 [34:28:43<8:46:51, 29.11s/it]
 78%|███████▊  | 3915/5000 [34:29:13<8:54:37, 29.56s/it]
 78%|███████▊  | 3916/5000 [34:29:40<8:40:06, 28.79s/it]
 78%|███████▊  | 3917/5000 [34:30:11<8:50:22, 29.38s/it]
 78%|███████▊  | 3918/5000 [34:30:41<8:53:06, 29.56s/it]
 78%|███████▊  | 3919/5000 [34:31:08<8:40:14, 28.88s/it]
 78%|███████▊  | 3920/5000 [34:31:39<8:49:03, 29.39s/it]
 78%|███████▊  | 3921/5000 [34:32:07<8:42:29, 29.05s/it]
 78%|███████▊  | 3922/5000 [34:32:37<8:46:27, 29.30s/it]
 78%|███████▊  | 3923/5000 [34:33:05<8:40:31, 29.00s/it]
 78%|███████▊  | 3924/5000 [34:33:36<8:47:19, 29.41s/it]
 78%|███████▊  | 3925/5000 [34:34:03<8:35:36, 28.78s/it]
                                                        
 78%|███████▊  | 3925/5000 [34:34:03<8:35:36, 28.78s/it]
 79%|███████▊  | 3926/5000 [34:34:33<8:43:13, 29.23s/it]
 79%|███████▊  | 3927/5000 [34:35:00<8:31:54, 28.62s/it]
 79%|███████▊  | 3928/5000 [34:35:32<8:46:13, 29.45s/it]
 79%|███████▊  | 3929/5000 [34:36:03<8:53:21, 29.88s/it]
 79%|███████▊  | 3930/5000 [34:36:30<8:40:29, 29.19s/it]
 79%|███████▊  | 3931/5000 [34:37:02<8:52:57, 29.91s/it]
 79%|███████▊  | 3932/5000 [34:37:29<8:36:52, 29.04s/it]
 79%|███████▊  | 3933/5000 [34:38:00<8:49:36, 29.78s/it]
 79%|███████▊  | 3934/5000 [34:38:28<8:36:11, 29.05s/it]
 79%|███████▊  | 3935/5000 [34:38:58<8:43:30, 29.49s/it]
 79%|███████▊  | 3936/5000 [34:39:27<8:39:37, 29.30s/it]
 79%|███████▊  | 3937/5000 [34:39:58<8:46:25, 29.71s/it]
 79%|███████▉  | 3938/5000 [34:40:26<8:38:54, 29.32s/it]
 79%|███████▉  | 3939/5000 [34:40:57<8:46:33, 29.78s/it]
 79%|███████▉  | 3940/5000 [34:41:26<8:43:44, 29.65s/it]
 79%|███████▉  | 3941/5000 [34:41:58<8:52:23, 30.16s/it]
 79%|███████▉  | 3942/5000 [34:42:27<8:49:16, 30.02s/it]
 79%|███████▉  | 3943/5000 [34:42:56<8:40:26, 29.54s/it]
 79%|███████▉  | 3944/5000 [34:43:26<8:44:50, 29.82s/it]
 79%|███████▉  | 3945/5000 [34:43:54<8:33:42, 29.22s/it]
 79%|███████▉  | 3946/5000 [34:44:25<8:41:29, 29.69s/it]
 79%|███████▉  | 3947/5000 [34:44:52<8:27:40, 28.93s/it]
 79%|███████▉  | 3948/5000 [34:45:24<8:42:30, 29.80s/it]
 79%|███████▉  | 3949/5000 [34:45:51<8:27:26, 28.97s/it]
 79%|███████▉  | 3950/5000 [34:46:22<8:36:50, 29.53s/it]
                                                        
 79%|███████▉  | 3950/5000 [34:46:22<8:36:50, 29.53s/it]
 79%|███████▉  | 3951/5000 [34:46:49<8:25:56, 28.94s/it]
 79%|███████▉  | 3952/5000 [34:47:20<8:32:36, 29.35s/it]
 79%|███████▉  | 3953/5000 [34:47:48<8:25:04, 28.94s/it]
 79%|███████▉  | 3954/5000 [34:48:17<8:28:10, 29.15s/it]
 79%|███████▉  | 3955/5000 [34:48:45<8:21:44, 28.81s/it]
 79%|███████▉  | 3956/5000 [34:49:15<8:25:22, 29.04s/it]
 79%|███████▉  | 3957/5000 [34:49:43<8:20:11, 28.77s/it]
 79%|███████▉  | 3958/5000 [34:50:14<8:31:47, 29.47s/it]
 79%|███████▉  | 3959/5000 [34:50:41<8:17:14, 28.66s/it]
 79%|███████▉  | 3960/5000 [34:51:12<8:31:24, 29.50s/it]
 79%|███████▉  | 3961/5000 [34:51:40<8:19:17, 28.83s/it]
 79%|███████▉  | 3962/5000 [34:52:11<8:30:54, 29.53s/it]
 79%|███████▉  | 3963/5000 [34:52:42<8:38:51, 30.02s/it]
 79%|███████▉  | 3964/5000 [34:53:09<8:23:45, 29.18s/it]
 79%|███████▉  | 3965/5000 [34:53:41<8:39:59, 30.14s/it]
 79%|███████▉  | 3966/5000 [34:54:09<8:25:02, 29.31s/it]
 79%|███████▉  | 3967/5000 [34:54:40<8:32:06, 29.74s/it]
 79%|███████▉  | 3968/5000 [34:55:08<8:24:20, 29.32s/it]
 79%|███████▉  | 3969/5000 [34:55:38<8:29:54, 29.67s/it]
 79%|███████▉  | 3970/5000 [34:56:06<8:19:15, 29.08s/it]
 79%|███████▉  | 3971/5000 [34:56:37<8:29:21, 29.70s/it]
 79%|███████▉  | 3972/5000 [34:57:08<8:36:15, 30.13s/it]
 79%|███████▉  | 3973/5000 [34:57:36<8:22:31, 29.36s/it]
 79%|███████▉  | 3974/5000 [34:58:07<8:29:21, 29.79s/it]
 80%|███████▉  | 3975/5000 [34:58:35<8:21:20, 29.35s/it]
                                                        
 80%|███████▉  | 3975/5000 [34:58:35<8:21:20, 29.35s/it]
 80%|███████▉  | 3976/5000 [34:59:05<8:23:34, 29.51s/it]
 80%|███████▉  | 3977/5000 [34:59:33<8:14:20, 28.99s/it]
 80%|███████▉  | 3978/5000 [35:00:05<8:28:26, 29.85s/it]
 80%|███████▉  | 3979/5000 [35:00:32<8:14:46, 29.08s/it]
 80%|███████▉  | 3980/5000 [35:01:03<8:26:39, 29.80s/it]
 80%|███████▉  | 3981/5000 [35:01:33<8:27:25, 29.88s/it]
 80%|███████▉  | 3982/5000 [35:02:01<8:17:09, 29.30s/it]
 80%|███████▉  | 3983/5000 [35:02:32<8:24:09, 29.74s/it]
 80%|███████▉  | 3984/5000 [35:02:59<8:09:47, 28.92s/it]
 80%|███████▉  | 3985/5000 [35:03:30<8:16:42, 29.36s/it]
 80%|███████▉  | 3986/5000 [35:03:57<8:08:39, 28.91s/it]
 80%|███████▉  | 3987/5000 [35:04:27<8:12:35, 29.18s/it]
 80%|███��███▉  | 3988/5000 [35:04:59<8:23:23, 29.85s/it]
 80%|███████▉  | 3989/5000 [35:05:26<8:09:16, 29.04s/it]
 80%|███████▉  | 3990/5000 [35:05:58<8:26:50, 30.11s/it]
 80%|███████▉  | 3991/5000 [35:06:25<8:11:09, 29.21s/it]
 80%|███████▉  | 3992/5000 [35:06:56<8:18:55, 29.70s/it]
 80%|███████▉  | 3993/5000 [35:07:24<8:09:56, 29.19s/it]
 80%|███████▉  | 3994/5000 [35:07:55<8:15:45, 29.57s/it]
 80%|███████▉  | 3995/5000 [35:08:23<8:06:37, 29.05s/it]
 80%|███████▉  | 3996/5000 [35:08:53<8:14:56, 29.58s/it]
 80%|███████▉  | 3997/5000 [35:09:24<8:20:22, 29.93s/it]
 80%|███████▉  | 3998/5000 [35:09:52<8:08:36, 29.26s/it]
 80%|███████▉  | 3999/5000 [35:10:22<8:13:11, 29.56s/it]
 80%|████████  | 4000/5000 [35:10:50<8:05:10, 29.11s/it]
                                                        
 80%|████████  | 4000/5000 [35:10:50<8:05:10, 29.11s/it][INFO|trainer.py:3138] 2023-05-11 20:52:31,246 >> ***** Running Evaluation *****
+[INFO|trainer.py:3142] 2023-05-11 20:52:31,247 >>   Num examples: Unknown
+[INFO|trainer.py:3143] 2023-05-11 20:52:31,247 >>   Batch size = 64
+{'loss': 0.096, 'learning_rate': 2.446666666666667e-06, 'epoch': 16.0}
+{'loss': 0.1025, 'learning_rate': 2.3911111111111113e-06, 'epoch': 16.01}
+{'loss': 0.103, 'learning_rate': 2.3355555555555557e-06, 'epoch': 16.01}
+{'loss': 0.1102, 'learning_rate': 2.28e-06, 'epoch': 16.02}
+{'loss': 0.1083, 'learning_rate': 2.2244444444444447e-06, 'epoch': 16.02}
+
+
+
+[INFO|trainer_utils.py:693] 2023-05-11 20:52:40,903 >> The following columns in the evaluation set don't have a corresponding argument in `WhisperForConditionalGeneration.forward` and have been ignored: input_length. If input_length are not expected by `WhisperForConditionalGeneration.forward`,  you can safely ignore this message.
+
                                                        
 80%|████████  | 4000/5000 [36:03:32<8:05:10, 29.11s/it][INFO|trainer.py:2877] 2023-05-11 21:45:12,895 >> Saving model checkpoint to ./checkpoint-4000
+[INFO|configuration_utils.py:458] 2023-05-11 21:45:12,903 >> Configuration saved in ./checkpoint-4000/config.json
+[INFO|configuration_utils.py:364] 2023-05-11 21:45:12,909 >> Configuration saved in ./checkpoint-4000/generation_config.json
+[INFO|modeling_utils.py:1855] 2023-05-11 21:45:15,833 >> Model weights saved in ./checkpoint-4000/pytorch_model.bin
+[INFO|feature_extraction_utils.py:369] 2023-05-11 21:45:15,837 >> Feature extractor saved in ./checkpoint-4000/preprocessor_config.json
+[INFO|feature_extraction_utils.py:369] 2023-05-11 21:45:26,357 >> Feature extractor saved in ./preprocessor_config.json

wandb/run-20230510_094132-lvsln7ks/files/output.log

@@ -3268,3 +3268,1115 @@ Reading metadata...: 15520it [00:00, 24837.17it/s]
 {'eval_loss': 0.25769394636154175, 'eval_wer': 8.695623928070265, 'eval_runtime': 3033.1756, 'eval_samples_per_second': 5.117, 'eval_steps_per_second': 0.08, 'epoch': 12.02}
 [INFO|modeling_utils.py:1855] 2023-05-11 12:24:49,746 >> Model weights saved in ./checkpoint-3000/pytorch_model.bin
 [INFO|feature_extraction_utils.py:369] 2023-05-11 12:24:49,750 >> Feature extractor saved in ./checkpoint-3000/preprocessor_config.json
+[INFO|feature_extraction_utils.py:369] 2023-05-11 12:25:01,129 >> Feature extractor saved in ./preprocessor_config.json
+/home/local/QCRI/dizham/miniconda3/envs/whisper/lib/python3.9/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
+  warnings.warn('Was asked to gather along dimension 0, but all '
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 60%|██████    | 3025/5000 [26:55:52<15:29:03, 28.22s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 61%|██████    | 3050/5000 [27:07:30<15:03:59, 27.81s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 62%|██████▏   | 3075/5000 [27:19:03<14:46:05, 27.62s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 62%|██████▏   | 3100/5000 [27:31:33<15:58:08, 30.26s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 62%|██████▏   | 3124/5000 [27:44:17<18:49:57, 36.14s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 63%|██████▎   | 3150/5000 [27:58:08<12:45:17, 24.82s/it]
+
+
+ 63%|██████▎   | 3152/5000 [27:58:29<8:59:11, 17.51s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Reading metadata...: 218215it [05:38, 808.68it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 64%|██████▎   | 3175/5000 [28:21:31<16:14:53, 32.05s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 64%|██████▍   | 3200/5000 [28:35:06<16:39:40, 33.32s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 64%|██████▍   | 3225/5000 [28:47:56<14:57:40, 30.34s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 65%|██████▍   | 3249/5000 [29:00:06<14:41:41, 30.21s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 66%|██████▌   | 3275/5000 [29:14:09<15:26:39, 32.23s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 66%|██████▌   | 3300/5000 [29:28:49<15:38:51, 33.14s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 66%|██████▋   | 3324/5000 [29:40:43<14:00:20, 30.08s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 67%|██████▋   | 3350/5000 [29:53:28<13:42:47, 29.92s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 67%|██████▋   | 3374/5000 [30:05:30<13:26:26, 29.76s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 68%|██████▊   | 3395/5000 [30:14:47<6:48:42, 15.28s/it]
+
+
+
+
+
+
+
+
+
+Reading metadata...: 221914it [00:24, 10269.69it/s]
+
+
+
+
+
+ 68%|██████▊   | 3399/5000 [30:18:26<16:00:47, 36.01s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 68%|██████▊   | 3424/5000 [30:30:23<12:38:30, 28.88s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 69%|██████▉   | 3449/5000 [30:42:12<12:14:00, 28.40s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 69%|██████▉   | 3474/5000 [30:54:04<12:05:36, 28.53s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 70%|██████▉   | 3499/5000 [31:05:54<11:41:43, 28.05s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 70%|███████   | 3524/5000 [31:17:50<11:34:42, 28.24s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 71%|███████   | 3549/5000 [31:29:41<11:39:07, 28.91s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 71%|███████▏  | 3574/5000 [31:41:33<11:19:50, 28.60s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 72%|███████▏  | 3599/5000 [31:53:19<10:50:25, 27.86s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 72%|███████▏  | 3624/5000 [32:04:57<10:35:00, 27.69s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 73%|███████▎  | 3637/5000 [32:10:07<6:03:02, 15.98s/it]
+
+
+
+
+
+
+
+
+
+Reading metadata...: 15520it [00:00, 34489.56it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+ 73%|███████▎  | 3649/5000 [32:17:38<11:23:10, 30.34s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 74%|███████▎  | 3675/5000 [32:30:31<10:49:34, 29.41s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 74%|███████▍  | 3699/5000 [32:42:32<10:40:27, 29.54s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 74%|███████▍  | 3724/5000 [32:54:44<10:14:21, 28.89s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 75%|███████▍  | 3749/5000 [33:07:16<10:43:26, 30.86s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 75%|███████▌  | 3774/5000 [33:19:41<10:35:11, 31.09s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 76%|███████▌  | 3799/5000 [33:32:08<9:58:45, 29.91s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 76%|███████▋  | 3824/5000 [33:44:29<10:14:31, 31.35s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 77%|███████▋  | 3850/5000 [33:56:57<8:56:32, 27.99s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 77%|███████▋  | 3874/5000 [34:08:31<9:11:29, 29.39s/it]
+
+
+
+
+
+ 78%|███████▊  | 3880/5000 [34:10:13<4:42:32, 15.14s/it]
+
+
+
+
+
+
+Reading metadata...: 15520it [00:00, 32765.41it/s]]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 78%|███████▊  | 3899/5000 [34:21:07<8:54:10, 29.11s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 78%|███████▊  | 3924/5000 [34:33:36<8:47:19, 29.41s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 79%|███████▉  | 3949/5000 [34:45:51<8:27:26, 28.97s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 79%|███████▉  | 3974/5000 [34:58:07<8:29:21, 29.79s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 80%|███████▉  | 3999/5000 [35:10:22<8:13:11, 29.56s/it]
+ 80%|████████  | 4000/5000 [35:10:50<8:05:10, 29.11s/it][INFO|trainer.py:3138] 2023-05-11 20:52:31,246 >> ***** Running Evaluation *****
+[INFO|trainer.py:3142] 2023-05-11 20:52:31,247 >>   Num examples: Unknown
+[INFO|trainer.py:3143] 2023-05-11 20:52:31,247 >>   Batch size = 64
+Reading metadata...: 15520it [00:00, 34879.19it/s]
+
+[INFO|trainer_utils.py:693] 2023-05-11 20:52:40,903 >> The following columns in the evaluation set don't have a corresponding argument in `WhisperForConditionalGeneration.forward` and have been ignored: input_length. If input_length are not expected by `WhisperForConditionalGeneration.forward`,  you can safely ignore this message.
+ 80%|████████  | 4000/5000 [36:03:32<8:05:10, 29.11s/it][INFO|trainer.py:2877] 2023-05-11 21:45:12,895 >> Saving model checkpoint to ./checkpoint-4000
+[INFO|configuration_utils.py:458] 2023-05-11 21:45:12,903 >> Configuration saved in ./checkpoint-4000/config.json
+[INFO|configuration_utils.py:364] 2023-05-11 21:45:12,909 >> Configuration saved in ./checkpoint-4000/generation_config.json
+{'eval_loss': 0.22095343470573425, 'eval_wer': 8.212281066472636, 'eval_runtime': 3161.6276, 'eval_samples_per_second': 4.909, 'eval_steps_per_second': 0.077, 'epoch': 16.02}
+[INFO|modeling_utils.py:1855] 2023-05-11 21:45:15,833 >> Model weights saved in ./checkpoint-4000/pytorch_model.bin
+[INFO|feature_extraction_utils.py:369] 2023-05-11 21:45:15,837 >> Feature extractor saved in ./checkpoint-4000/preprocessor_config.json

wandb/run-20230510_094132-lvsln7ks/files/wandb-summary.json

@@ -1 +1 @@
-{"eval/loss": 0.25769394636154175, "eval/wer": 8.695623928070265, "eval/runtime": 3033.1756, "eval/samples_per_second": 5.117, "eval/steps_per_second": 0.08, "_timestamp": 1683797085.5536704, "_runtime": 96192.85912036896, "_step": 122}
+{"eval/loss": 0.22095343470573425, "eval/wer": 8.212281066472636, "eval/runtime": 3161.6276, "eval/samples_per_second": 4.909, "eval/steps_per_second": 0.077, "_timestamp": 1683830712.87446, "_runtime": 129820.17990994453, "_step": 163}

wandb/run-20230510_094132-lvsln7ks/run-lvsln7ks.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8bce697a33afcaf55f1622d49f50977aa97742c363791028b6dc74cb789befcf
-size 8913574
+oid sha256:8c468e9b711e38b3400f6ef0efe4b2d44110679aa553d9d74488445634973d2a
+size 12027825