mosesdaudu/afrolinguahub_accented_english_ASR

Afrolinguahub Accented English ASR

This model builds upon the capabilities of Whisper small (a pre-trained model for speech recognition and translation trained on a massive 680k hour dataset). While Whisper demonstrates impressive generalization abilities, this model takes it a step further to be very specific for Nigerian accents.

Fine-tuned on the NaijaSpeech dataset, specifically designed for Nigerian accents, this model offers enhanced performance for speech recognition tasks on Nigerian accents

Transcription

In this example, the context tokens are 'unforced', meaning the model automatically predicts the output language (English) and task (transcribe).

>>> from transformers import WhisperProcessor, WhisperForConditionalGeneration
>>> from datasets import load_dataset

>>> # load model and processor
>>> processor = WhisperProcessor.from_pretrained("AWARRITech/naijaspeech-Whisper-small-en")
>>> model = WhisperForConditionalGeneration.from_pretrained("AWARRITech/naijaspeech-Whisper-small-en")
>>> model.config.forced_decoder_ids = None

>>> # load dummy dataset and read audio files
>>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
>>> sample = ds[0]["audio"]
>>> input_features = processor(sample["array"], sampling_rate=sample["sampling_rate"], return_tensors="pt").input_features 

>>> # generate token ids
>>> predicted_ids = model.generate(input_features)
>>> # decode token ids to text
>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=False)
['<|startoftranscript|><|en|><|transcribe|><|notimestamps|> Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.<|endoftext|>']

>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
[' Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.']

The context tokens can be removed from the start of the transcription by setting skip_special_tokens=True.