indic-nepali-large-large-colab

This model is a fine-tuned version of Harveenchadha/wav2vec2-pretrained-clsril-23-10k on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.7848
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 16
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
11.7001	0.9479	400	3.9168	1.0
3.8147	1.8957	800	3.8061	1.0
3.7988	2.8436	1200	3.8261	1.0
3.8006	3.7915	1600	3.7865	1.0
3.8082	4.7393	2000	3.8160	1.0
3.7948	5.6872	2400	3.7880	1.0
3.7931	6.6351	2800	3.7908	1.0
3.79	7.5829	3200	3.7990	1.0
3.7937	8.5308	3600	3.7849	1.0
3.7837	9.4787	4000	3.7994	1.0
3.7829	10.4265	4400	3.7841	1.0
3.781	11.3744	4800	3.7927	1.0
3.777	12.3223	5200	3.7849	1.0
3.7819	13.2701	5600	3.7875	1.0
3.7697	14.2180	6000	3.7855	1.0
3.7794	15.1659	6400	3.7848	1.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.19.1