Use bfloat16 for core module.
Use parallel attention and MLP in ViViT.

Epoch 001/400
Train		loss: 111647336.00	correlation: 0.0122
Validation	loss: 200300080.00	correlation: 0.0267
Elapse: 537.13s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 002/400
Train		loss: 97648256.00	correlation: 0.0367
Validation	loss: 199289632.00	correlation: 0.0355
Elapse: 539.72s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 003/400
Train		loss: 96321792.00	correlation: 0.0500
Validation	loss: 198299696.00	correlation: 0.0446
Elapse: 548.37s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 004/400
Train		loss: 95213544.00	correlation: 0.0617
Validation	loss: 196983504.00	correlation: 0.0527
Elapse: 552.04s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 005/400
Train		loss: 93915056.00	correlation: 0.0756
Validation	loss: 195022704.00	correlation: 0.0660
Elapse: 551.29s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 006/400
Train		loss: 92500376.00	correlation: 0.0909
Validation	loss: 193045536.00	correlation: 0.0815
Elapse: 549.45s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 007/400
Train		loss: 91511136.00	correlation: 0.1012
Validation	loss: 191545056.00	correlation: 0.0905
Elapse: 547.11s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 008/400
Train		loss: 90428944.00	correlation: 0.1113
Validation	loss: 189976016.00	correlation: 0.0998
Elapse: 544.76s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 009/400
Train		loss: 89301664.00	correlation: 0.1228
Validation	loss: 188514336.00	correlation: 0.1074
Elapse: 541.79s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 010/400
Train		loss: 88238616.00	correlation: 0.1335
Validation	loss: 186730592.00	correlation: 0.1194
Elapse: 540.44s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 011/400
Train		loss: 87239336.00	correlation: 0.1436
Validation	loss: 185287104.00	correlation: 0.1296
Elapse: 539.71s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 012/400
Train		loss: 86205256.00	correlation: 0.1540
Validation	loss: 183237792.00	correlation: 0.1419
Elapse: 540.02s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 013/400
Train		loss: 85068472.00	correlation: 0.1655
Validation	loss: 181721296.00	correlation: 0.1515
Elapse: 539.81s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 014/400
Train		loss: 84160560.00	correlation: 0.1745
Validation	loss: 180469664.00	correlation: 0.1595
Elapse: 540.34s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 015/400
Train		loss: 83386128.00	correlation: 0.1821
Validation	loss: 179517792.00	correlation: 0.1658
Elapse: 540.81s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 016/400
Train		loss: 82805200.00	correlation: 0.1877
Validation	loss: 178801584.00	correlation: 0.1698
Elapse: 540.70s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 017/400
Train		loss: 82275736.00	correlation: 0.1926
Validation	loss: 177842080.00	correlation: 0.1761
Elapse: 540.76s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 018/400
Train		loss: 81662840.00	correlation: 0.1987
Validation	loss: 176971248.00	correlation: 0.1820
Elapse: 541.07s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 019/400
Train		loss: 81147088.00	correlation: 0.2038
Validation	loss: 176423600.00	correlation: 0.1858
Elapse: 541.20s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 020/400
Train		loss: 80748784.00	correlation: 0.2079
Validation	loss: 175690976.00	correlation: 0.1908
Elapse: 541.22s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 021/400
Train		loss: 80313880.00	correlation: 0.2122
Validation	loss: 175389216.00	correlation: 0.1929
Elapse: 540.65s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 022/400
Train		loss: 79957744.00	correlation: 0.2153
Validation	loss: 174862864.00	correlation: 0.1968
Elapse: 540.96s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 023/400
Train		loss: 79621360.00	correlation: 0.2186
Validation	loss: 174437632.00	correlation: 0.1996
Elapse: 541.11s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 024/400
Train		loss: 79374368.00	correlation: 0.2211
Validation	loss: 173783968.00	correlation: 0.2031
Elapse: 542.03s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 025/400
Train		loss: 79085072.00	correlation: 0.2240
Validation	loss: 173357520.00	correlation: 0.2059
Elapse: 541.66s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 026/400
Train		loss: 78723728.00	correlation: 0.2273
Validation	loss: 173125120.00	correlation: 0.2087
Elapse: 542.13s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 027/400
Train		loss: 78456624.00	correlation: 0.2299
Validation	loss: 172740432.00	correlation: 0.2102
Elapse: 541.72s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 028/400
Train		loss: 78264240.00	correlation: 0.2316
Validation	loss: 172426416.00	correlation: 0.2125
Elapse: 541.90s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 029/400
Train		loss: 78095968.00	correlation: 0.2335
Validation	loss: 172338688.00	correlation: 0.2139
Elapse: 542.04s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 030/400
Train		loss: 77880056.00	correlation: 0.2357
Validation	loss: 171972944.00	correlation: 0.2151
Elapse: 541.96s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 031/400
Train		loss: 77782920.00	correlation: 0.2367
Validation	loss: 171662176.00	correlation: 0.2171
Elapse: 542.59s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 032/400
Train		loss: 77596272.00	correlation: 0.2382
Validation	loss: 171484624.00	correlation: 0.2187
Elapse: 543.51s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 033/400
Train		loss: 77378984.00	correlation: 0.2403
Validation	loss: 171180896.00	correlation: 0.2208
Elapse: 542.09s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 034/400
Train		loss: 77329272.00	correlation: 0.2411
Validation	loss: 171382000.00	correlation: 0.2191
Elapse: 541.49s

Epoch 035/400
Train		loss: 77173400.00	correlation: 0.2426
Validation	loss: 170990288.00	correlation: 0.2215
Elapse: 542.22s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 036/400
Train		loss: 77103352.00	correlation: 0.2430
Validation	loss: 171064224.00	correlation: 0.2212
Elapse: 542.85s

Epoch 037/400
Train		loss: 76961264.00	correlation: 0.2447
Validation	loss: 170693952.00	correlation: 0.2239
Elapse: 542.80s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 038/400
Train		loss: 76838424.00	correlation: 0.2456
Validation	loss: 170481408.00	correlation: 0.2246
Elapse: 542.40s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 039/400
Train		loss: 76726736.00	correlation: 0.2469
Validation	loss: 170405888.00	correlation: 0.2257
Elapse: 541.86s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 040/400
Train		loss: 76625736.00	correlation: 0.2481
Validation	loss: 170183216.00	correlation: 0.2271
Elapse: 542.67s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 041/400
Train		loss: 76633712.00	correlation: 0.2477
Validation	loss: 170194400.00	correlation: 0.2270
Elapse: 542.30s

Epoch 042/400
Train		loss: 76442136.00	correlation: 0.2493
Validation	loss: 170452384.00	correlation: 0.2263
Elapse: 542.26s

Epoch 043/400
Train		loss: 76492200.00	correlation: 0.2491
Validation	loss: 170249952.00	correlation: 0.2260
Elapse: 542.10s

Epoch 044/400
Train		loss: 76279480.00	correlation: 0.2510
Validation	loss: 169784496.00	correlation: 0.2295
Elapse: 541.77s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 045/400
Train		loss: 76193080.00	correlation: 0.2520
Validation	loss: 169944928.00	correlation: 0.2278
Elapse: 542.23s

Epoch 046/400
Train		loss: 76107936.00	correlation: 0.2526
Validation	loss: 169860016.00	correlation: 0.2283
Elapse: 542.90s

Epoch 047/400
Train		loss: 76111024.00	correlation: 0.2527
Validation	loss: 169548304.00	correlation: 0.2308
Elapse: 541.89s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 048/400
Train		loss: 76049672.00	correlation: 0.2534
Validation	loss: 169801264.00	correlation: 0.2296
Elapse: 543.95s

Epoch 049/400
Train		loss: 75997232.00	correlation: 0.2537
Validation	loss: 169721248.00	correlation: 0.2301
Elapse: 543.00s

Epoch 050/400
Train		loss: 75961488.00	correlation: 0.2545
Validation	loss: 169722912.00	correlation: 0.2293
Elapse: 542.91s

Epoch 051/400
Train		loss: 75908376.00	correlation: 0.2548
Validation	loss: 169474704.00	correlation: 0.2310
Elapse: 542.56s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 052/400
Train		loss: 75771856.00	correlation: 0.2561
Validation	loss: 170132720.00	correlation: 0.2272
Elapse: 542.05s

Epoch 053/400
Train		loss: 75913696.00	correlation: 0.2547
Validation	loss: 169491888.00	correlation: 0.2309
Elapse: 542.57s

Epoch 054/400
Train		loss: 75764272.00	correlation: 0.2560
Validation	loss: 169266704.00	correlation: 0.2330
Elapse: 542.91s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 055/400
Train		loss: 75670816.00	correlation: 0.2569
Validation	loss: 169200320.00	correlation: 0.2327
Elapse: 542.21s

Epoch 056/400
Train		loss: 75680240.00	correlation: 0.2570
Validation	loss: 169253344.00	correlation: 0.2322
Elapse: 542.73s

Epoch 057/400
Train		loss: 75623792.00	correlation: 0.2573
Validation	loss: 169158704.00	correlation: 0.2336
Elapse: 544.29s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 058/400
Train		loss: 75518968.00	correlation: 0.2584
Validation	loss: 169124832.00	correlation: 0.2339
Elapse: 542.64s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 059/400
Train		loss: 75452928.00	correlation: 0.2591
Validation	loss: 169483008.00	correlation: 0.2320
Elapse: 542.34s

Epoch 060/400
Train		loss: 75478992.00	correlation: 0.2590
Validation	loss: 169223728.00	correlation: 0.2337
Elapse: 543.01s

Epoch 061/400
Train		loss: 75414704.00	correlation: 0.2597
Validation	loss: 169588672.00	correlation: 0.2307
Elapse: 542.79s

Epoch 062/400
Train		loss: 75631880.00	correlation: 0.2578
Validation	loss: 169325440.00	correlation: 0.2326
Elapse: 543.75s

Epoch 063/400
Train		loss: 75359640.00	correlation: 0.2602
Validation	loss: 169091040.00	correlation: 0.2339
Elapse: 543.16s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 064/400
Train		loss: 75319080.00	correlation: 0.2605
Validation	loss: 169416560.00	correlation: 0.2321
Elapse: 543.52s

Epoch 065/400
Train		loss: 75306104.00	correlation: 0.2609
Validation	loss: 169176224.00	correlation: 0.2331
Elapse: 542.80s

Epoch 066/400
Train		loss: 75301592.00	correlation: 0.2609
Validation	loss: 169371616.00	correlation: 0.2327
Elapse: 543.10s

Epoch 067/400
Train		loss: 75389672.00	correlation: 0.2601
Validation	loss: 169169312.00	correlation: 0.2332
Elapse: 544.11s

Epoch 068/400
Train		loss: 75361008.00	correlation: 0.2604
Validation	loss: 169366960.00	correlation: 0.2315
Elapse: 543.42s

Loaded checkpoint from epoch 63 (correlation: 0.2339).

Reduce learning rate of core to 1.4400e-03 (num. reduce: 1).
Reduce learning rate of readouts to 1.0800e-03 (num. reduce: 1).
Reduce learning rate of shifters to 1.0800e-03 (num. reduce: 1).

Epoch 069/400
Train		loss: 73278464.00	correlation: 0.2784
Validation	loss: 167482848.00	correlation: 0.2447
Elapse: 544.59s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 070/400
Train		loss: 72562424.00	correlation: 0.2849
Validation	loss: 167277872.00	correlation: 0.2456
Elapse: 544.90s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 071/400
Train		loss: 72429480.00	correlation: 0.2862
Validation	loss: 167337568.00	correlation: 0.2452
Elapse: 544.84s

Epoch 072/400
Train		loss: 72339856.00	correlation: 0.2872
Validation	loss: 167472656.00	correlation: 0.2442
Elapse: 544.89s

Epoch 073/400
Train		loss: 72270912.00	correlation: 0.2877
Validation	loss: 167356592.00	correlation: 0.2443
Elapse: 545.34s

Epoch 074/400
Train		loss: 72207960.00	correlation: 0.2884
Validation	loss: 167467040.00	correlation: 0.2434
Elapse: 545.38s

Epoch 075/400
Train		loss: 72186752.00	correlation: 0.2885
Validation	loss: 167473600.00	correlation: 0.2434
Elapse: 545.13s

Loaded checkpoint from epoch 70 (correlation: 0.2456).

Reduce learning rate of core to 4.3200e-04 (num. reduce: 1).
Reduce learning rate of readouts to 3.2400e-04 (num. reduce: 1).
Reduce learning rate of shifters to 3.2400e-04 (num. reduce: 1).

Epoch 076/400
Train		loss: 71743648.00	correlation: 0.2919
Validation	loss: 166945040.00	correlation: 0.2479
Elapse: 544.86s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 077/400
Train		loss: 71453328.00	correlation: 0.2949
Validation	loss: 166933984.00	correlation: 0.2480
Elapse: 545.20s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 078/400
Train		loss: 71407184.00	correlation: 0.2951
Validation	loss: 166839616.00	correlation: 0.2480
Elapse: 545.38s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 079/400
Train		loss: 71308560.00	correlation: 0.2960
Validation	loss: 166820688.00	correlation: 0.2484
Elapse: 545.26s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 080/400
Train		loss: 71223520.00	correlation: 0.2968
Validation	loss: 166801616.00	correlation: 0.2487
Elapse: 545.81s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 081/400
Train		loss: 71151600.00	correlation: 0.2975
Validation	loss: 166777584.00	correlation: 0.2485
Elapse: 545.15s

Epoch 082/400
Train		loss: 71166128.00	correlation: 0.2973
Validation	loss: 166806736.00	correlation: 0.2484
Elapse: 545.51s

Epoch 083/400
Train		loss: 71003768.00	correlation: 0.2991
Validation	loss: 166794000.00	correlation: 0.2487
Elapse: 545.39s

Epoch 084/400
Train		loss: 71028680.00	correlation: 0.2987
Validation	loss: 166740976.00	correlation: 0.2486
Elapse: 545.50s

Epoch 085/400
Train		loss: 70987920.00	correlation: 0.2989
Validation	loss: 166771344.00	correlation: 0.2483
Elapse: 545.38s

Loaded checkpoint from epoch 80 (correlation: 0.2487).

Reduce learning rate of core to 1.2960e-04 (num. reduce: 1).
Reduce learning rate of readouts to 9.7200e-05 (num. reduce: 1).
Reduce learning rate of shifters to 9.7200e-05 (num. reduce: 1).

Epoch 086/400
Train		loss: 70886192.00	correlation: 0.2998
Validation	loss: 166696544.00	correlation: 0.2492
Elapse: 545.68s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 087/400
Train		loss: 70891072.00	correlation: 0.2998
Validation	loss: 166700640.00	correlation: 0.2493
Elapse: 545.36s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 088/400
Train		loss: 70786208.00	correlation: 0.3009
Validation	loss: 166720928.00	correlation: 0.2490
Elapse: 545.75s

Epoch 089/400
Train		loss: 70840864.00	correlation: 0.3002
Validation	loss: 166705888.00	correlation: 0.2491
Elapse: 546.01s

Epoch 090/400
Train		loss: 70781008.00	correlation: 0.3008
Validation	loss: 166757248.00	correlation: 0.2489
Elapse: 545.02s

Epoch 091/400
Train		loss: 70768960.00	correlation: 0.3009
Validation	loss: 166698640.00	correlation: 0.2489
Elapse: 545.07s

Epoch 092/400
Train		loss: 70700664.00	correlation: 0.3016
Validation	loss: 166705392.00	correlation: 0.2491
Elapse: 545.85s

Loaded checkpoint from epoch 87 (correlation: 0.2493).

Reduce learning rate of core to 3.8880e-05 (num. reduce: 1).
Reduce learning rate of readouts to 2.9160e-05 (num. reduce: 1).
Reduce learning rate of shifters to 2.9160e-05 (num. reduce: 1).

Epoch 093/400
Train		loss: 70743624.00	correlation: 0.3012
Validation	loss: 166685456.00	correlation: 0.2494
Elapse: 545.69s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 094/400
Train		loss: 70752552.00	correlation: 0.3012
Validation	loss: 166676048.00	correlation: 0.2493
Elapse: 545.53s

Epoch 095/400
Train		loss: 70729328.00	correlation: 0.3013
Validation	loss: 166677072.00	correlation: 0.2494
Elapse: 545.12s

Epoch 096/400
Train		loss: 70661304.00	correlation: 0.3018
Validation	loss: 166687824.00	correlation: 0.2493
Elapse: 545.30s

Epoch 097/400
Train		loss: 70768896.00	correlation: 0.3006
Validation	loss: 166658272.00	correlation: 0.2492
Elapse: 545.47s

Epoch 098/400
Train		loss: 70673976.00	correlation: 0.3017
Validation	loss: 166648288.00	correlation: 0.2495
Elapse: 545.22s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 099/400
Train		loss: 70635672.00	correlation: 0.3022
Validation	loss: 166648032.00	correlation: 0.2495
Elapse: 545.17s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 100/400
Train		loss: 70748008.00	correlation: 0.3010
Validation	loss: 166647312.00	correlation: 0.2495
Elapse: 545.61s

Epoch 101/400
Train		loss: 70682408.00	correlation: 0.3016
Validation	loss: 166662208.00	correlation: 0.2494
Elapse: 545.56s

Epoch 102/400
Train		loss: 70642688.00	correlation: 0.3020
Validation	loss: 166662624.00	correlation: 0.2495
Elapse: 545.17s

Epoch 103/400
Train		loss: 70638368.00	correlation: 0.3019
Validation	loss: 166645120.00	correlation: 0.2495
Elapse: 545.25s

Epoch 104/400
Train		loss: 70563872.00	correlation: 0.3027
Validation	loss: 166649696.00	correlation: 0.2495
Elapse: 545.70s

Loaded checkpoint from epoch 99 (correlation: 0.2495).

Reduce learning rate of core to 1.1664e-05 (num. reduce: 1).
Reduce learning rate of readouts to 8.7480e-06 (num. reduce: 1).
Reduce learning rate of shifters to 8.7480e-06 (num. reduce: 1).

Epoch 105/400
Train		loss: 70679696.00	correlation: 0.3019
Validation	loss: 166646944.00	correlation: 0.2495
Elapse: 545.44s

Epoch 106/400
Train		loss: 70623656.00	correlation: 0.3021
Validation	loss: 166651056.00	correlation: 0.2495
Elapse: 545.46s

Epoch 107/400
Train		loss: 70689912.00	correlation: 0.3013
Validation	loss: 166642080.00	correlation: 0.2495
Elapse: 545.17s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 108/400
Train		loss: 70636352.00	correlation: 0.3019
Validation	loss: 166643376.00	correlation: 0.2495
Elapse: 545.12s

Epoch 109/400
Train		loss: 70622056.00	correlation: 0.3021
Validation	loss: 166645664.00	correlation: 0.2495
Elapse: 545.91s

Epoch 110/400
Train		loss: 70689848.00	correlation: 0.3013
Validation	loss: 166646544.00	correlation: 0.2495
Elapse: 544.95s

Epoch 111/400
Train		loss: 70594792.00	correlation: 0.3025
Validation	loss: 166655344.00	correlation: 0.2494
Elapse: 545.66s

Epoch 112/400
Train		loss: 70636904.00	correlation: 0.3020
Validation	loss: 166638224.00	correlation: 0.2495
Elapse: 544.74s

Loaded checkpoint from epoch 107 (correlation: 0.2495).

Reduce learning rate of core to 3.4992e-06 (num. reduce: 1).
Reduce learning rate of readouts to 2.6244e-06 (num. reduce: 1).
Reduce learning rate of shifters to 2.6244e-06 (num. reduce: 1).

Epoch 113/400
Train		loss: 70679760.00	correlation: 0.3016
Validation	loss: 166645488.00	correlation: 0.2495
Elapse: 543.62s

Epoch 114/400
Train		loss: 70609840.00	correlation: 0.3026
Validation	loss: 166641568.00	correlation: 0.2495
Elapse: 543.57s

Checkpoint saved to /home/storage/runs/vivit_ensemble/028/ckpt/model_state.pt.

Epoch 115/400
Train		loss: 70676264.00	correlation: 0.3017
Validation	loss: 166645824.00	correlation: 0.2495
Elapse: 545.22s

Epoch 116/400
Train		loss: 70640760.00	correlation: 0.3023
Validation	loss: 166649888.00	correlation: 0.2495
Elapse: 545.14s

Epoch 117/400
Train		loss: 70651464.00	correlation: 0.3017
Validation	loss: 166644896.00	correlation: 0.2495
Elapse: 545.75s

Epoch 118/400
Train		loss: 70647664.00	correlation: 0.3020
Validation	loss: 166642016.00	correlation: 0.2495
Elapse: 551.10s

Epoch 119/400
Train		loss: 70647632.00	correlation: 0.3017
Validation	loss: 166642992.00	correlation: 0.2495
Elapse: 545.02s

Loaded checkpoint from epoch 114 (correlation: 0.2495).

Reduce learning rate of core to 1.0498e-06 (num. reduce: 1).
Reduce learning rate of readouts to 7.8732e-07 (num. reduce: 1).
Reduce learning rate of shifters to 7.8732e-07 (num. reduce: 1).

Epoch 120/400
Train		loss: 70673976.00	correlation: 0.3014
Validation	loss: 166641376.00	correlation: 0.2495
Elapse: 544.88s

Epoch 121/400
Train		loss: 70599064.00	correlation: 0.3023
Validation	loss: 166642464.00	correlation: 0.2495
Elapse: 544.91s

Epoch 122/400
Train		loss: 70592552.00	correlation: 0.3025
Validation	loss: 166642720.00	correlation: 0.2495
Elapse: 544.80s

Epoch 123/400
Train		loss: 70644184.00	correlation: 0.3019
Validation	loss: 166643072.00	correlation: 0.2495
Elapse: 544.57s

Epoch 124/400
Train		loss: 70629504.00	correlation: 0.3022
Validation	loss: 166642208.00	correlation: 0.2495
Elapse: 545.38s

Loaded checkpoint from epoch 114 (correlation: 0.2495).

Reduce learning rate of core to 3.1493e-07 (num. reduce: 2).
Reduce learning rate of readouts to 2.3620e-07 (num. reduce: 2).
Reduce learning rate of shifters to 2.3620e-07 (num. reduce: 2).

Epoch 125/400
Train		loss: 70617168.00	correlation: 0.3023
Validation	loss: 166641952.00	correlation: 0.2495
Elapse: 544.97s

Epoch 126/400
Train		loss: 70613328.00	correlation: 0.3026
Validation	loss: 166642704.00	correlation: 0.2495
Elapse: 544.92s

Epoch 127/400
Train		loss: 70681712.00	correlation: 0.3016
Validation	loss: 166642848.00	correlation: 0.2495
Elapse: 544.98s

Epoch 128/400
Train		loss: 70630672.00	correlation: 0.3021
Validation	loss: 166642912.00	correlation: 0.2495
Elapse: 544.96s

Epoch 129/400
Train		loss: 70600632.00	correlation: 0.3025
Validation	loss: 166642848.00	correlation: 0.2495
Elapse: 545.27s

Model has not improved after 2 LR reductions.

Loaded checkpoint from epoch 114 (correlation: 0.2495).


ValidationA: 0.2465	B: 0.2783	C: 0.2688	D: 0.2292	E: 0.2366	F: 0.2341	G: 0.2549	H: 0.2330	I: 0.2567	J: 0.2571	average: 0.2495

Results saved to /home/storage/runs/vivit_ensemble/028.