File size: 3,089 Bytes
648ff66 0f1237f 648ff66 4236973 991f757 4236973 648ff66 991f757 648ff66 991f757 a2c5972 03d45b4 a2c5972 169131d d0bdfb8 648ff66 c5f890a 648ff66 a2c5972 648ff66 c5f890a 4236973 648ff66 4236973 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
library_name: transformers
license: mit
base_model: openai/whisper-large-v3-turbo
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: whisper-large-v3-turbo-OpenHQ-GL
results: []
datasets:
- juanjucm/OpenHQ-SpeechT-GL-EN
language:
- gl
---
# whisper-large-v3-turbo-OpenHQ-GL
This model is a fine-tuned version of [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo) trained on [juanjucm/OpenHQ-SpeechT-GL-EN](https://huggingface.co/datasets/juanjucm/OpenHQ-SpeechT-GL-EN) for **Galician Text to Speech** task. It takes galician speech audios as input and generates the correspondant transcription.
This Automatic Speech Recognition model, was developed to be the first stage of a Speech Translation cascade system for transcribing and translating Galician audios into English texts. After this first STT step, this [Galician-to-English MT model](https://huggingface.co/juanjucm/nllb-200-distilled-600M-OpenSLR-GL-EN) can be applied over the generated Galician transcriptions to get English text translations.
The motivation behind this work is to increase the visibility of the Galician language, making it more accessible for non-Galician speakers to understand and engage with Galician audio content.
This model was developed during a 3-week Speech Translation workshop organised by [Yasmin Moslem](https://huggingface.co/ymoslem).
### Performance and training details
Baseline model achieved a WER score of **20.1** on the evaluation dataset.
After fine-tuning, it achieves the following results on the evaluation set:
- Loss: 0.1613
- **WER: 10.6845**
The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- total_eval_batch_size: 16
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
### Training results
We used [WER (Word Error Rate)](https://en.wikipedia.org/wiki/Word_error_rate) as our reference transcription metric for selecting the best checkpoint after training.
| Training Loss | Epoch | Step | Validation Loss | Wer |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| 0.2739 | 1.0 | 75 | 0.1898 | 11.4023 |
| 0.1841 | 2.0 | 150 | 0.1819 | 10.3673 |
| 0.0542 | 3.0 | 225 | 0.1919 | 10.6177 |
| 0.0399 | 4.0 | 300 | 0.1934 | 11.1352 |
| 0.0264 | 5.0 | 375 | 0.2042 | 11.2688 |
| 0.0143 | 6.0 | 450 | 0.2075 | 10.3840 |
| 0.0056 | 7.0 | 525 | 0.2198 | 10.8347 |
| 0.0063 | 8.0 | 600 | 0.2217 | 10.9683 |
| 0.0037 | 9.0 | 675 | 0.2258 | 10.5509 |
| 0.0042 | 10.0 | 750 | 0.2278 | 10.6845 |
### Framework versions
- Transformers 4.47.1
- Pytorch 2.4.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0 |