xezpeleta commited on
Commit
80c7c09
·
verified ·
1 Parent(s): 9205682

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +64 -49
README.md CHANGED
@@ -16,80 +16,95 @@ model-index:
16
  name: Automatic Speech Recognition
17
  type: automatic-speech-recognition
18
  dataset:
19
- name: asierhv/composite_corpus_eu_v2.1
20
- type: asierhv/composite_corpus_eu_v2.1
21
  metrics:
22
  - name: Wer
23
  type: wer
24
- value: 9.104533533158486
 
 
25
  ---
26
 
27
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
28
- should probably proofread and complete it, then remove this comment. -->
29
-
30
  # Whisper Medium Basque
31
 
32
- This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on the asierhv/composite_corpus_eu_v2.1 dataset.
33
- It achieves the following results on the evaluation set:
34
- - Loss: 0.2488
35
- - Wer: 9.1045
 
 
36
 
37
  ## Model description
38
 
39
- More information needed
40
 
41
  ## Intended uses & limitations
42
 
43
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
  ## Training and evaluation data
46
 
47
- More information needed
 
48
 
49
  ## Training procedure
50
 
51
  ### Training hyperparameters
52
 
53
  The following hyperparameters were used during training:
54
- - learning_rate: 6.25e-06
55
- - train_batch_size: 16
56
- - eval_batch_size: 8
57
- - seed: 42
58
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
59
- - lr_scheduler_type: linear
60
- - lr_scheduler_warmup_steps: 500
61
- - training_steps: 10000
62
- - mixed_precision_training: Native AMP
63
 
64
- ### Training results
 
 
 
 
 
 
 
 
65
 
66
- | Training Loss | Epoch | Step | Validation Loss | Wer |
67
- |:-------------:|:-----:|:-----:|:---------------:|:-------:|
68
- | 0.3412 | 0.05 | 500 | 0.5112 | 28.8685 |
69
- | 0.1464 | 0.1 | 1000 | 0.4178 | 20.5570 |
70
- | 0.2504 | 0.15 | 1500 | 0.3625 | 18.1279 |
71
- | 0.2615 | 0.2 | 2000 | 0.3236 | 15.5364 |
72
- | 0.1648 | 0.25 | 2500 | 0.3209 | 13.8129 |
73
- | 0.0933 | 0.3 | 3000 | 0.2991 | 12.8887 |
74
- | 0.1016 | 0.35 | 3500 | 0.2823 | 12.4329 |
75
- | 0.1449 | 0.4 | 4000 | 0.2741 | 11.7460 |
76
- | 0.151 | 0.45 | 4500 | 0.2791 | 11.5774 |
77
- | 0.0917 | 0.5 | 5000 | 0.2744 | 11.2402 |
78
- | 0.0913 | 0.55 | 5500 | 0.2901 | 11.1340 |
79
- | 0.1085 | 0.6 | 6000 | 0.2663 | 10.3285 |
80
- | 0.0928 | 0.65 | 6500 | 0.2705 | 10.2910 |
81
- | 0.0725 | 0.7 | 7000 | 0.2506 | 10.3035 |
82
- | 0.1216 | 0.75 | 7500 | 0.2758 | 9.7103 |
83
- | 0.131 | 0.8 | 8000 | 0.2519 | 9.4292 |
84
- | 0.0525 | 0.85 | 8500 | 0.2602 | 9.3106 |
85
- | 0.0729 | 0.9 | 9000 | 0.2549 | 9.3606 |
86
- | 0.0939 | 0.95 | 9500 | 0.2470 | 9.1920 |
87
- | 0.0639 | 1.0 | 10000 | 0.2488 | 9.1045 |
88
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
 
90
  ### Framework versions
91
 
92
- - Transformers 4.49.0.dev0
93
- - Pytorch 2.6.0+cu124
94
- - Datasets 3.3.1.dev0
95
- - Tokenizers 0.21.0
 
16
  name: Automatic Speech Recognition
17
  type: automatic-speech-recognition
18
  dataset:
19
+ name: Mozilla Common Voice 18.0
20
+ type: mozilla-foundation/common_voice_18_0
21
  metrics:
22
  - name: Wer
23
  type: wer
24
+ value: 7.14
25
+ language:
26
+ - eu
27
  ---
28
 
 
 
 
29
  # Whisper Medium Basque
30
 
31
+ This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) specifically for Basque (eu) language Automatic Speech Recognition (ASR). It was trained on the [asierhv/composite_corpus_eu_v2.1](https://huggingface.co/datasets/asierhv/composite_corpus_eu_v2.1) dataset, which is a composite corpus designed to improve Basque ASR performance.
32
+
33
+ **Key improvements and results compared to the base model:**
34
+
35
+ * **Significant WER reduction:** The fine-tuned model achieves a Word Error Rate (WER) of 9.1045 on the validation set of the `asierhv/composite_corpus_eu_v2.1` dataset, demonstrating improved accuracy compared to the base `whisper-medium` model for Basque.
36
+ * **Performance on Common Voice:** When evaluated on the Mozilla Common Voice 18.0 dataset, the model achieved a WER of 7.14. This indicates strong generalization capabilities and highlights the benefits of the medium-sized model for enhanced accuracy.
37
 
38
  ## Model description
39
 
40
+ This model utilizes the `whisper-medium` architecture, which offers a substantial increase in capacity compared to smaller variants, leading to improved accuracy. By fine-tuning this model on a dedicated Basque speech corpus, it specializes in accurately transcribing Basque speech. The `whisper-medium` model strikes a balance between high accuracy and manageable computational requirements.
41
 
42
  ## Intended uses & limitations
43
 
44
+ **Intended uses:**
45
+
46
+ * High-precision automatic transcription of Basque speech for professional and research applications.
47
+ * Development of advanced Basque speech-based applications requiring very high accuracy.
48
+ * Research in Basque speech processing where the highest possible accuracy is crucial.
49
+ * Professional transcription services and applications where accuracy is paramount.
50
+ * Use in scenarios where the computational cost is acceptable for the significant improvement in accuracy.
51
+
52
+ **Limitations:**
53
+
54
+ * Performance remains influenced by audio quality, with challenges arising from background noise and poor recording conditions.
55
+ * Accuracy may be affected by highly dialectal or informal Basque speech.
56
+ * Despite improved performance, the model may still produce errors, particularly with complex linguistic structures or rare words.
57
+ * The medium model is larger than the small, base, and tiny models, so inference will be slower and require more resources.
58
 
59
  ## Training and evaluation data
60
 
61
+ * **Training dataset:** [asierhv/composite_corpus_eu_v2.1](https://huggingface.co/datasets/asierhv/composite_corpus_eu_v2.1). This dataset is a meticulously curated collection of Basque speech data, designed to maximize the performance of Basque ASR systems.
62
+ * **Evaluation Dataset:** The `test` split of `asierhv/composite_corpus_eu_v2.1`.
63
 
64
  ## Training procedure
65
 
66
  ### Training hyperparameters
67
 
68
  The following hyperparameters were used during training:
 
 
 
 
 
 
 
 
 
69
 
70
+ * **learning_rate:** 6.25e-06
71
+ * **train_batch_size:** 16
72
+ * **eval_batch_size:** 8
73
+ * **seed:** 42
74
+ * **optimizer:** AdamW with betas=(0.9, 0.999) and epsilon=1e-08
75
+ * **lr_scheduler_type:** linear
76
+ * **lr_scheduler_warmup_steps:** 500
77
+ * **training_steps:** 10000
78
+ * **mixed_precision_training:** Native AMP
79
 
80
+ ### Training results
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
 
82
+ | Training Loss | Epoch | Step | Validation Loss | WER |
83
+ |---------------|-------|-------|-----------------|----------|
84
+ | 0.3412 | 0.05 | 500 | 0.5112 | 28.8685 |
85
+ | 0.1464 | 0.1 | 1000 | 0.4178 | 20.5570 |
86
+ | 0.2504 | 0.15 | 1500 | 0.3625 | 18.1279 |
87
+ | 0.2615 | 0.2 | 2000 | 0.3236 | 15.5364 |
88
+ | 0.1648 | 0.25 | 2500 | 0.3209 | 13.8129 |
89
+ | 0.0933 | 0.3 | 3000 | 0.2991 | 12.8887 |
90
+ | 0.1016 | 0.35 | 3500 | 0.2823 | 12.4329 |
91
+ | 0.1449 | 0.4 | 4000 | 0.2741 | 11.7460 |
92
+ | 0.151 | 0.45 | 4500 | 0.2791 | 11.5774 |
93
+ | 0.0917 | 0.5 | 5000 | 0.2744 | 11.2402 |
94
+ | 0.0913 | 0.55 | 5500 | 0.2901 | 11.1340 |
95
+ | 0.1085 | 0.6 | 6000 | 0.2663 | 10.3285 |
96
+ | 0.0928 | 0.65 | 6500 | 0.2705 | 10.2910 |
97
+ | 0.0725 | 0.7 | 7000 | 0.2506 | 10.3035 |
98
+ | 0.1216 | 0.75 | 7500 | 0.2758 | 9.7103 |
99
+ | 0.131 | 0.8 | 8000 | 0.2519 | 9.4292 |
100
+ | 0.0525 | 0.85 | 8500 | 0.2602 | 9.3106 |
101
+ | 0.0729 | 0.9 | 9000 | 0.2549 | 9.3606 |
102
+ | 0.0939 | 0.95 | 9500 | 0.2470 | 9.1920 |
103
+ | 0.0639 | 1.0 | 10000 | 0.2488 | 9.1045 |
104
 
105
  ### Framework versions
106
 
107
+ * Transformers 4.49.0.dev0
108
+ * Pytorch 2.6.0+cu124
109
+ * Datasets 3.3.1.dev0
110
+ * Tokenizers 0.21.0