## Fine-tuning run 3

Tried to improve model fine-tuned during run 1.

Checkpoint used: checkpoint-12000

* Trained for 6000 steps
* Used custom Learning Rate scheduler initialized in: `custom_trainer.Seq2SeqTrainerCustomLinearScheduler`:
  * `--learning_rate="3e-5"`
	* `--learning_rate_end="1e-5"`
* no warmup was used
* no WER improvements compared to checkpoint-12000 of run 1
* using `seed=43`
* do not upload checkpoints from that run
* uploading src, logs, tensorboard logs, trainer_state

## Advices
* I guess, we need to use warmup when resuming training and increasing LR compared to the last LR in previous run
* need to set number of steps > 6000. because model improved WER veeery slowly
* can use original Mozilla Common Voice dataset instead of a HuggingFace's one.<br>
  the reason is that original contains multiple voicings of same sentence - 
  so there is at least twice as more data.<br>
  to use this "additional" data, train, validation, test sets need to be enlarged using `validated` set - 
  the one that is absent in HuggingFace's CV11 dataset