Training in progress, step 4500

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:46bfc78745340fef27b401f3b8a5de1f05a7ca7fc2b06227789acc71b8f732ce
+oid sha256:3f3b35216dc11b2d8eb1144ad6b8b6c2bbdb57c5f263916fe79dc60e3bf481c4
 size 1262817457

wandb/run-20220129_215451-1vipdbow/files/output.log

@@ -4745,3 +4745,595 @@ Configuration saved in ./checkpoint-4000/config.json
 {'eval_loss': 0.31949570775032043, 'eval_wer': 0.4180162273127179, 'eval_runtime': 296.0172, 'eval_samples_per_second': 19.566, 'eval_steps_per_second': 0.307, 'epoch': 0.58}
 Model weights saved in ./checkpoint-4000/pytorch_model.bin
 Configuration saved in ./checkpoint-4000/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+Deleting older checkpoint [checkpoint-3000] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 30%|██████████████████████████████████████████████▎                                                                                                             | 4100/13822 [4:26:11<4:30:24,  1.67s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 30%|███████████████████████████████████████████████▍                                                                                                            | 4200/13822 [4:31:23<4:27:40,  1.67s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 31%|████████████████████████████████████████████████▌                                                                                                           | 4300/13822 [4:36:35<4:20:18,  1.64s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 32%|█████████████████████████████████████████████████▋                                                                                                          | 4399/13822 [4:41:47<4:27:43,  1.70s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 33%|██████████████████████████████████████████████████▊                                                                                                         | 4499/13822 [4:46:59<4:32:21,  1.75s/it]
+ 33%|██████████████████████████████████████████████████▊                                                                                                         | 4500/13822 [4:47:01<4:21:18,  1.68s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 5792
+  Batch size = 64
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Configuration saved in ./checkpoint-4500/config.json
+{'eval_loss': 0.30754634737968445, 'eval_wer': 0.41408973899442797, 'eval_runtime': 296.2378, 'eval_samples_per_second': 19.552, 'eval_steps_per_second': 0.307, 'epoch': 0.65}
+Model weights saved in ./checkpoint-4500/pytorch_model.bin
+Configuration saved in ./checkpoint-4500/preprocessor_config.json

wandb/run-20220129_215451-1vipdbow/run-1vipdbow.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca2883907183a25ab58f3c8968be35166274236f2b82afc34594b95c94230448
-size 27912985
+oid sha256:3f46b515385f03d285516ebc3155783c7908614a53b79847fa8608c1e96b83bc
+size 31621187