Training in progress, step 8000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3a64731777bd746d359ffa586729d25bea4492bc28fa247fa4547bb22bc3101c
+oid sha256:a422bed3f403d2dc5b83d987fbc333a772df96ba06cf9954d12ef5c10f4a445f
 size 1262817457

wandb/run-20220129_215451-1vipdbow/files/output.log

@@ -8895,3 +8895,594 @@ Deleting older checkpoint [checkpoint-6000] due to args.save_total_limit
 Configuration saved in ./checkpoint-7500/config.json
 Model weights saved in ./checkpoint-7500/pytorch_model.bin
 Configuration saved in ./checkpoint-7500/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+Deleting older checkpoint [checkpoint-6500] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 55%|█████████████████████████████████████████████████████████████████████████████████████▊                                                                      | 7599/13822 [8:13:27<4:11:44,  2.43s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 56%|██████████████████████████████████████████████████████████████████████████████████████▉                                                                     | 7699/13822 [8:18:40<4:06:50,  2.42s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 56%|████████████████████████████████████████████████████████████████████████████████████████                                                                    | 7800/13822 [8:23:55<4:13:21,  2.52s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 57%|█████████████████████████████████████████████████████████████████████████████████████████▏                                                                  | 7899/13822 [8:29:05<4:01:42,  2.45s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 58%|██████████████████████████████████████████████████████████████████████████████████████████▎                                                                 | 7999/13822 [8:34:17<3:55:02,  2.42s/it]
+ 58%|██████████████████████████████████████████████████████████████████████████████████████████▎                                                                 | 8000/13822 [8:34:19<3:56:14,  2.43s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 5792
+  Batch size = 64
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Configuration saved in ./checkpoint-8000/config.json
+{'eval_loss': 0.26817116141319275, 'eval_wer': 0.3888526833718922, 'eval_runtime': 298.1504, 'eval_samples_per_second': 19.426, 'eval_steps_per_second': 0.305, 'epoch': 1.16}
+Model weights saved in ./checkpoint-8000/pytorch_model.bin
+Configuration saved in ./checkpoint-8000/preprocessor_config.json

wandb/run-20220129_215451-1vipdbow/run-1vipdbow.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2e4e819638fa4d56d583d51a37a78e1415ff0e9dd3ac011f7929b6c763317f3a
-size 53856973
+oid sha256:3920cec34508f992c87c6b894a0a1fc466949458914aec0c8f3f3e06ce7f1f8b
+size 57646698