--- library_name: transformers language: - bem license: cc-by-nc-4.0 base_model: facebook/mms-1b-all tags: - generated_from_trainer datasets: - BIG_C/Bemba metrics: - wer model-index: - name: facebook/mms-1b-all results: - task: name: Automatic Speech Recognition type: automatic-speech-recognition dataset: name: BIG_C type: BIG_C/Bemba metrics: - name: Wer type: wer value: 0.4170597821425508 --- # facebook/mms-1b-all This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the BIG_C dataset. It achieves the following results on the evaluation set: - Loss: 0.3475 - Model Preparation Time: 0.0112 - Wer: 0.4171 - Cer: 0.0777 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 4 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 16 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 100 - num_epochs: 100 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Model Preparation Time | Wer | Cer | |:-------------:|:-----:|:------:|:---------------:|:----------------------:|:------:|:------:| | 0.9189 | 1.0 | 3096 | 0.5510 | 0.0112 | 0.4860 | 0.1248 | | 0.5994 | 2.0 | 6192 | 0.5258 | 0.0112 | 0.4870 | 0.1264 | | 0.5721 | 3.0 | 9288 | 0.5079 | 0.0112 | 0.4638 | 0.1264 | | 0.5573 | 4.0 | 12384 | 0.4963 | 0.0112 | 0.4410 | 0.1128 | | 0.5442 | 5.0 | 15480 | 0.4938 | 0.0112 | 0.4449 | 0.1202 | | 0.5347 | 6.0 | 18576 | 0.4837 | 0.0112 | 0.4348 | 0.1165 | | 0.5261 | 7.0 | 21672 | 0.4795 | 0.0112 | 0.4205 | 0.1092 | | 0.5203 | 8.0 | 24768 | 0.4791 | 0.0112 | 0.4237 | 0.1076 | | 0.5132 | 9.0 | 27864 | 0.4745 | 0.0112 | 0.4159 | 0.1073 | | 0.5073 | 10.0 | 30960 | 0.4696 | 0.0112 | 0.4162 | 0.1072 | | 0.5037 | 11.0 | 34056 | 0.4696 | 0.0112 | 0.4227 | 0.1136 | | 0.4974 | 12.0 | 37152 | 0.4702 | 0.0112 | 0.4042 | 0.1047 | | 0.4924 | 13.0 | 40248 | 0.4761 | 0.0112 | 0.3916 | 0.1028 | | 0.4873 | 14.0 | 43344 | 0.4617 | 0.0112 | 0.3977 | 0.1046 | | 0.4846 | 15.0 | 46440 | 0.4756 | 0.0112 | 0.3926 | 0.1025 | | 0.4798 | 16.0 | 49536 | 0.4614 | 0.0112 | 0.3980 | 0.1066 | | 0.4767 | 17.0 | 52632 | 0.4606 | 0.0112 | 0.3940 | 0.1031 | | 0.473 | 18.0 | 55728 | 0.4712 | 0.0112 | 0.3919 | 0.1030 | | 0.4685 | 19.0 | 58824 | 0.4586 | 0.0112 | 0.3913 | 0.1032 | | 0.4656 | 20.0 | 61920 | 0.4713 | 0.0112 | 0.3892 | 0.1034 | | 0.4618 | 21.0 | 65016 | 0.4627 | 0.0112 | 0.3854 | 0.1020 | | 0.4577 | 22.0 | 68112 | 0.4565 | 0.0112 | 0.3880 | 0.1015 | | 0.4554 | 23.0 | 71208 | 0.4593 | 0.0112 | 0.3862 | 0.1034 | | 0.4523 | 24.0 | 74304 | 0.4624 | 0.0112 | 0.3818 | 0.1015 | | 0.4486 | 25.0 | 77400 | 0.4638 | 0.0112 | 0.3855 | 0.1024 | | 0.446 | 26.0 | 80496 | 0.4574 | 0.0112 | 0.3929 | 0.1029 | | 0.4426 | 27.0 | 83592 | 0.4662 | 0.0112 | 0.3851 | 0.1033 | | 0.4407 | 28.0 | 86688 | 0.4633 | 0.0112 | 0.3898 | 0.1032 | | 0.4378 | 29.0 | 89784 | 0.4614 | 0.0112 | 0.3800 | 0.1017 | | 0.4337 | 30.0 | 92880 | 0.4595 | 0.0112 | 0.3809 | 0.1009 | | 0.4312 | 31.0 | 95976 | 0.4681 | 0.0112 | 0.3841 | 0.1046 | | 0.4277 | 32.0 | 99072 | 0.4575 | 0.0112 | 0.3749 | 0.0999 | | 0.4257 | 33.0 | 102168 | 0.4583 | 0.0112 | 0.3760 | 0.1021 | | 0.4224 | 34.0 | 105264 | 0.4566 | 0.0112 | 0.3830 | 0.1012 | | 0.4205 | 35.0 | 108360 | 0.4718 | 0.0112 | 0.3698 | 0.1004 | | 0.4159 | 36.0 | 111456 | 0.4642 | 0.0112 | 0.3734 | 0.1008 | | 0.4141 | 37.0 | 114552 | 0.4601 | 0.0112 | 0.3802 | 0.1024 | | 0.4118 | 38.0 | 117648 | 0.4665 | 0.0112 | 0.3718 | 0.1004 | | 0.408 | 39.0 | 120744 | 0.4665 | 0.0112 | 0.3745 | 0.1041 | | 0.4062 | 40.0 | 123840 | 0.4743 | 0.0112 | 0.3708 | 0.0994 | | 0.4033 | 41.0 | 126936 | 0.4700 | 0.0112 | 0.3720 | 0.1003 | | 0.4017 | 42.0 | 130032 | 0.4755 | 0.0112 | 0.3841 | 0.1057 | | 0.3982 | 43.0 | 133128 | 0.4798 | 0.0112 | 0.3671 | 0.0984 | | 0.3966 | 44.0 | 136224 | 0.4738 | 0.0112 | 0.3752 | 0.1010 | | 0.3944 | 45.0 | 139320 | 0.4749 | 0.0112 | 0.3727 | 0.1010 | | 0.3923 | 46.0 | 142416 | 0.4702 | 0.0112 | 0.3792 | 0.1033 | | 0.389 | 47.0 | 145512 | 0.4723 | 0.0112 | 0.3714 | 0.1014 | | 0.3874 | 48.0 | 148608 | 0.4750 | 0.0112 | 0.3733 | 0.1004 | | 0.3851 | 49.0 | 151704 | 0.4708 | 0.0112 | 0.3747 | 0.1024 | | 0.3831 | 50.0 | 154800 | 0.4768 | 0.0112 | 0.3688 | 0.0996 | | 0.3803 | 51.0 | 157896 | 0.4807 | 0.0112 | 0.3673 | 0.0996 | | 0.379 | 52.0 | 160992 | 0.4823 | 0.0112 | 0.3648 | 0.0991 | | 0.3772 | 53.0 | 164088 | 0.4809 | 0.0112 | 0.3723 | 0.1002 | | 0.3741 | 54.0 | 167184 | 0.4807 | 0.0112 | 0.3705 | 0.0996 | | 0.3726 | 55.0 | 170280 | 0.4866 | 0.0112 | 0.3669 | 0.0990 | | 0.371 | 56.0 | 173376 | 0.4793 | 0.0112 | 0.3738 | 0.1027 | | 0.3695 | 57.0 | 176472 | 0.4789 | 0.0112 | 0.3768 | 0.1025 | | 0.3668 | 58.0 | 179568 | 0.4774 | 0.0112 | 0.3681 | 0.1015 | | 0.3656 | 59.0 | 182664 | 0.4783 | 0.0112 | 0.3684 | 0.1000 | | 0.3639 | 60.0 | 185760 | 0.4848 | 0.0112 | 0.3714 | 0.1002 | | 0.3615 | 61.0 | 188856 | 0.4893 | 0.0112 | 0.3721 | 0.0996 | | 0.3596 | 62.0 | 191952 | 0.4808 | 0.0112 | 0.3654 | 0.1008 | ### Framework versions - Transformers 4.47.0.dev0 - Pytorch 2.1.0+cu118 - Datasets 3.1.0 - Tokenizers 0.20.1