Training in progress, step 8500

.ipynb_checkpoints/preprocessor_config-checkpoint.json

@@ -0,0 +1,9 @@
+{
+  "do_normalize": true,
+  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
+  "feature_size": 1,
+  "padding_side": "right",
+  "padding_value": 0,
+  "return_attention_mask": true,
+  "sampling_rate": 16000
+}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c7d5c5665cb2f10cbee4073c92b5d77e0f1dafd3e633c65ef200d4ca4dca469e
+oid sha256:2286263c5fbbc1e83cd9d35130517d66808e769f2dfa0b9424795299c9f1570b
 size 1262817457

wandb/run-20220129_131141-h6nhqm30/files/output.log

@@ -8470,3 +8470,534 @@ Configuration saved in ./checkpoint-8000/config.json
 {'eval_loss': 0.3026394248008728, 'eval_wer': 0.4099970068841664, 'eval_runtime': 64.4321, 'eval_samples_per_second': 28.542, 'eval_steps_per_second': 0.45, 'epoch': 1.86}
 Model weights saved in ./checkpoint-8000/pytorch_model.bin
 Configuration saved in ./checkpoint-8000/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+Deleting older checkpoint [checkpoint-6500] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 38%|██████████████████████████████████████████████████████████▋                                                                                                 | 8099/21520 [6:41:43<7:19:00,  1.96s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 38%|███████████████████████████████████████████████████████████▍                                                                                                | 8200/21520 [6:46:08<7:12:30,  1.95s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 39%|████████████████████████████████████████████████████████████▏                                                                                               | 8301/21520 [6:50:33<7:07:31,  1.94s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 39%|████████████████████████████████████████████████████████████▉                                                                                               | 8400/21520 [6:54:55<7:36:42,  2.09s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 39%|█████████████████████████████████████████████████████████████▌                                                                                              | 8499/21520 [6:59:15<7:12:41,  1.99s/it]
+ 39%|█████████████████████████████████████████████████████████████▌                                                                                              | 8500/21520 [6:59:17<7:12:02,  1.99s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 1839
+  Batch size = 64
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:56<00:00,  1.81s/it]
+
+Configuration saved in ./checkpoint-8500/config.json
+Model weights saved in ./checkpoint-8500/pytorch_model.bin
+Configuration saved in ./checkpoint-8500/preprocessor_config.json

wandb/run-20220129_131141-h6nhqm30/run-h6nhqm30.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:65aa39669da36f28c6653cdb0f6c9fe3d2f0eba29768f21178ba8144dbee3edf
-size 58269618
+oid sha256:624404f6778fa63cf42e5d65fc81506fe3d0200ae05be92d55229cbe4a878ab4
+size 62254538