metadata

license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
  - generated_from_trainer
datasets:
  - common_voice_17_0
metrics:
  - wer
  - bleu
model-index:
  - name: wav2vec2-mms-1b-malayalam-colab-CV17.0
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: common_voice_17_0
          type: common_voice_17_0
          config: ml
          split: test
          args: ml
        metrics:
          - name: Wer
            type: wer
            value: 1.0315925209542232
          - name: Bleu
            type: bleu
            value:
              bleu: 0
              precisions:
                - 0.0008639308855291577
                - 0
                - 0
                - 0
              brevity_penalty: 0.7118010694449419
              length_ratio: 0.7462927143778207
              translation_length: 2315
              reference_length: 3102

wav2vec2-mms-1b-malayalam-colab-CV17.0

This model is a fine-tuned version of facebook/mms-1b-all on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

Loss: 4.7578
Wer: 1.0316
Cer: 0.7469
Bleu: {'bleu': 0.0, 'precisions': [0.0008639308855291577, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7118010694449419, 'length_ratio': 0.7462927143778207, 'translation_length': 2315, 'reference_length': 3102}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 16
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.15
training_steps: 2000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer	Bleu
21.8941	3.1496	200	14.1370	1.0	1.0292	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.08102287291060646, 'length_ratio': 0.2846550612508059, 'translation_length': 883, 'reference_length': 3102}
8.5677	6.2992	400	6.3226	1.0071	0.8568	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.2500351312096836, 'length_ratio': 0.4190844616376531, 'translation_length': 1300, 'reference_length': 3102}
6.6037	9.4488	600	5.6520	1.0351	0.7789	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.6082895680797644, 'length_ratio': 0.6679561573178594, 'translation_length': 2072, 'reference_length': 3102}
6.0799	12.5984	800	5.2294	1.0574	0.7594	{'bleu': 0.0, 'precisions': [0.00041631973355537054, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7471989379147929, 'length_ratio': 0.7743391360412637, 'translation_length': 2402, 'reference_length': 3102}
5.7804	15.7480	1000	5.1733	1.0329	0.7629	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.6734028056038747, 'length_ratio': 0.7166344294003868, 'translation_length': 2223, 'reference_length': 3102}
5.4453	18.8976	1200	4.9821	1.0525	0.7467	{'bleu': 0.0, 'precisions': [0.0004042037186742118, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7758159728382187, 'length_ratio': 0.7975499677627337, 'translation_length': 2474, 'reference_length': 3102}
5.2983	22.0472	1400	4.8784	1.0297	0.7521	{'bleu': 0.0, 'precisions': [0.0008806693086745927, 0.0, 0.0, 0.0], 'brevity_penalty': 0.693559602972643, 'length_ratio': 0.7321083172147002, 'translation_length': 2271, 'reference_length': 3102}
5.0862	25.1969	1600	4.7948	1.0358	0.7446	{'bleu': 0.0, 'precisions': [0.0008399832003359933, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7387365301547462, 'length_ratio': 0.7675693101225016, 'translation_length': 2381, 'reference_length': 3102}
4.9884	28.3465	1800	4.7578	1.0316	0.7479	{'bleu': 0.0, 'precisions': [0.000864304235090752, 0.0, 0.0, 0.0], 'brevity_penalty': 0.711389009553914, 'length_ratio': 0.7459703417150225, 'translation_length': 2314, 'reference_length': 3102}
5.0447	31.4961	2000	4.7578	1.0316	0.7469	{'bleu': 0.0, 'precisions': [0.0008639308855291577, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7118010694449419, 'length_ratio': 0.7462927143778207, 'translation_length': 2315, 'reference_length': 3102}

Framework versions

Transformers 4.42.4
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1