add the model card
Browse files
README.md
ADDED
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
language: tr
|
4 |
+
tags:
|
5 |
+
- automatic-speech-recognition
|
6 |
+
- common_voice
|
7 |
+
- mozilla-foundation/common_voice_8_0
|
8 |
+
- tr
|
9 |
+
- robust-speech-event
|
10 |
+
datasets:
|
11 |
+
- mozilla-foundation/common_voice_8_0
|
12 |
+
model-index:
|
13 |
+
- name: mpoyraz/wav2vec2-xls-r-300m-cv8-turkish
|
14 |
+
results:
|
15 |
+
- task:
|
16 |
+
name: Automatic Speech Recognition
|
17 |
+
type: automatic-speech-recognition
|
18 |
+
dataset:
|
19 |
+
name: Common Voice 8
|
20 |
+
type: mozilla-foundation/common_voice_8_0
|
21 |
+
args: tr
|
22 |
+
metrics:
|
23 |
+
- name: Test WER
|
24 |
+
type: wer
|
25 |
+
value: 10.61
|
26 |
+
- name: Test CER
|
27 |
+
type: cer
|
28 |
+
value: 2.67
|
29 |
+
- task:
|
30 |
+
name: Automatic Speech Recognition
|
31 |
+
type: automatic-speech-recognition
|
32 |
+
dataset:
|
33 |
+
name: Robust Speech Event - Dev Data
|
34 |
+
type: speech-recognition-community-v2/dev_data
|
35 |
+
args: tr
|
36 |
+
metrics:
|
37 |
+
- name: Test WER
|
38 |
+
type: wer
|
39 |
+
value: 36.46
|
40 |
+
- name: Test CER
|
41 |
+
type: cer
|
42 |
+
value: 12.38
|
43 |
+
---
|
44 |
+
|
45 |
+
# wav2vec2-xls-r-300m-cv8-turkish
|
46 |
+
|
47 |
+
## Model description
|
48 |
+
This ASR model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on Turkish language.
|
49 |
+
|
50 |
+
## Training and evaluation data
|
51 |
+
The following datasets were used for finetuning:
|
52 |
+
- [Common Voice 8.0 TR](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0) All `validated` split except `test` split was used for training.
|
53 |
+
|
54 |
+
## Training procedure
|
55 |
+
To support the datasets above, custom pre-processing and loading steps was performed and [wav2vec2-turkish](https://github.com/mpoyraz/wav2vec2-turkish) repo was used for that purpose.
|
56 |
+
|
57 |
+
### Training hyperparameters
|
58 |
+
The following hypermaters were used for finetuning:
|
59 |
+
- learning_rate 2.5e-4
|
60 |
+
- num_train_epochs 20
|
61 |
+
- warmup_steps 500
|
62 |
+
- freeze_feature_extractor
|
63 |
+
- mask_time_prob 0.1
|
64 |
+
- mask_feature_prob 0.1
|
65 |
+
- feat_proj_dropout 0.05
|
66 |
+
- attention_dropout 0.05
|
67 |
+
- final_dropout 0.1
|
68 |
+
- activation_dropout 0.05
|
69 |
+
- per_device_train_batch_size 8
|
70 |
+
- per_device_eval_batch_size 8
|
71 |
+
- gradient_accumulation_steps 8
|
72 |
+
|
73 |
+
### Framework versions
|
74 |
+
- Transformers 4.17.0.dev0
|
75 |
+
- Pytorch 1.10.1
|
76 |
+
- Datasets 1.17.0
|
77 |
+
- Tokenizers 0.10.3
|
78 |
+
|
79 |
+
## Language Model
|
80 |
+
N-gram language model is trained on a Turkish Wikipedia articles using KenLM and [ngram-lm-wiki](https://github.com/mpoyraz/ngram-lm-wiki) repo was used to generate arpa LM and convert it into binary format.
|
81 |
+
|
82 |
+
## Evaluation Commands
|
83 |
+
Please install [unicode_tr](https://pypi.org/project/unicode_tr/) package before running evaluation. It is used for Turkish text processing.
|
84 |
+
1. To evaluate on `mozilla-foundation/common_voice_8_0` with split `test`
|
85 |
+
```bash
|
86 |
+
python eval.py --model_id mpoyraz/wav2vec2-xls-r-300m-cv8-turkish --dataset mozilla-foundation/common_voice_8_0 --config tr --split test
|
87 |
+
```
|
88 |
+
|
89 |
+
2. To evaluate on `speech-recognition-community-v2/dev_data`
|
90 |
+
|
91 |
+
```bash
|
92 |
+
python eval.py --model_id mpoyraz/wav2vec2-xls-r-300m-cv8-turkish --dataset speech-recognition-community-v2/dev_data --config tr --split validation --chunk_length_s 5.0 --stride_length_s 1.0
|
93 |
+
```
|
94 |
+
## Evaluation results:
|
95 |
+
|
96 |
+
| Dataset | WER | CER |
|
97 |
+
|---|---|---|
|
98 |
+
|Common Voice 8 TR test split| 10.61 | 2.67 |
|
99 |
+
|Speech Recognition Community dev data| 36.46 | 12.38 |
|