indic-nepali-santosh-large-colab

This model is a fine-tuned version of Harveenchadha/wav2vec2-pretrained-clsril-23-10k on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.7697
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
7.8004	1.4210	400	3.7818	1.0
3.8093	2.8419	800	3.7929	1.0
3.8132	4.2629	1200	3.7744	1.0
3.8118	5.6838	1600	3.7898	1.0
3.8216	7.1048	2000	3.7773	1.0
3.8025	8.5258	2400	3.8029	1.0
3.8104	9.9467	2800	3.7783	1.0
3.8106	11.3677	3200	3.8272	1.0
3.8067	12.7886	3600	3.7776	1.0
3.8057	14.2096	4000	3.7890	1.0
3.8084	15.6306	4400	3.7765	1.0
3.8131	17.0515	4800	3.7713	1.0
3.811	18.4725	5200	3.7703	1.0
3.8088	19.8934	5600	3.8267	1.0
3.806	21.3144	6000	3.7703	1.0
3.802	22.7353	6400	3.8033	1.0
3.7906	24.1563	6800	3.7773	1.0
3.7943	25.5773	7200	3.7818	1.0
3.7808	26.9982	7600	3.7755	1.0
3.7832	28.4192	8000	3.7709	1.0
3.7741	29.8401	8400	3.7697	1.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.19.1