huseinzol05
commited on
Commit
·
57efae9
1
Parent(s):
adf6534
Update README.md
Browse files
README.md
CHANGED
@@ -11,37 +11,61 @@ probably proofread and complete it, then remove this comment. -->
|
|
11 |
|
12 |
# wav2vec2-xls-r-300m-mixed
|
13 |
|
14 |
-
|
15 |
-
It achieves the following results on the evaluation set:
|
16 |
|
|
|
17 |
|
18 |
-
|
|
|
|
|
19 |
|
20 |
-
|
21 |
|
22 |
-
##
|
23 |
|
24 |
-
|
25 |
|
26 |
-
|
|
|
|
|
|
|
27 |
|
28 |
-
|
29 |
|
30 |
-
|
31 |
|
32 |
-
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
-
|
35 |
-
- optimizer: None
|
36 |
-
- training_precision: float32
|
37 |
|
38 |
-
|
|
|
|
|
|
|
|
|
|
|
39 |
|
|
|
40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
41 |
|
42 |
-
|
43 |
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
|
|
|
|
|
|
|
|
|
11 |
|
12 |
# wav2vec2-xls-r-300m-mixed
|
13 |
|
14 |
+
Finetuned https://huggingface.co/facebook/wav2vec2-xls-r-300m on https://github.com/huseinzol05/malaya-speech/tree/master/data/mixed-stt
|
|
|
15 |
|
16 |
+
This model was finetuned on 3 languages,
|
17 |
|
18 |
+
1. Malay
|
19 |
+
2. Singlish
|
20 |
+
3. Mandarin
|
21 |
|
22 |
+
**This model trained on a single RTX 3090 Ti 24GB VRAM, provided by https://mesolitica.com/**.
|
23 |
|
24 |
+
## Evaluation set
|
25 |
|
26 |
+
Evaluation set from https://github.com/huseinzol05/malaya-speech/tree/master/pretrained-model/prepare-stt with sizes,
|
27 |
|
28 |
+
```
|
29 |
+
len(malay), len(singlish), len(mandarin)
|
30 |
+
-> (765, 3579, 614)
|
31 |
+
```
|
32 |
|
33 |
+
It achieves the following results on the evaluation set based on [evaluate-wav2vec2-xls-r-300m-mixed.ipynb](evaluate-wav2vec2-xls-r-300m-mixed.ipynb):
|
34 |
|
35 |
+
Mixed evaluation,
|
36 |
|
37 |
+
```
|
38 |
+
CER: 0.05082346216269688
|
39 |
+
WER: 0.14251665517797765
|
40 |
+
CER with LM: 0.042868860764264445
|
41 |
+
WER with LM: 0.10380217528405207
|
42 |
+
```
|
43 |
|
44 |
+
Malay evaluation,
|
|
|
|
|
45 |
|
46 |
+
```
|
47 |
+
CER: 0.05027226867066105
|
48 |
+
WER: 0.21723938552369926
|
49 |
+
CER with LM: 0.03601546154013878
|
50 |
+
WER with LM: 0.13593624603428525
|
51 |
+
```
|
52 |
|
53 |
+
Singlish evaluation,
|
54 |
|
55 |
+
```
|
56 |
+
CER: 0.05161275767676772
|
57 |
+
WER: 0.1331819722523124
|
58 |
+
CER with LM: 0.04419848182804781
|
59 |
+
WER with LM: 0.09859626021111582
|
60 |
+
```
|
61 |
|
62 |
+
Mandarin evaluation,
|
63 |
|
64 |
+
```
|
65 |
+
CER: 0.04690941391603209
|
66 |
+
WER: 0.10382926344585862
|
67 |
+
CER with LM: 0.0436573568867001
|
68 |
+
WER with LM: 0.09411065398455744
|
69 |
+
```
|
70 |
+
|
71 |
+
Language model from https://huggingface.co/huseinzol05/language-model-bahasa-manglish-combined
|