AndrewMcDowell commited on
Commit
fe7dd5e
·
1 Parent(s): 700a297

Update README.md

Browse files

Add eval results on dev data.

Files changed (1) hide show
  1. README.md +27 -2
README.md CHANGED
@@ -27,7 +27,20 @@ model-index:
27
  - name: Test CER
28
  type: cer
29
  value: 23.64
30
-
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  ---
32
 
33
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -37,7 +50,13 @@ should probably proofread and complete it, then remove this comment. -->
37
 
38
  This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - JA dataset.
39
 
40
- Kanji are converted into Hiragana using the [pykakasi](https://pykakasi.readthedocs.io/en/latest/index.html) library during training and evaluation. The model can output both Hiragana and Katakana characters.
 
 
 
 
 
 
41
 
42
  It achieves the following results on the evaluation set:
43
  - Loss: 0.5212
@@ -98,4 +117,10 @@ The following hyperparameters were used during training:
98
 
99
  ```bash
100
  python ./eval.py --model_id AndrewMcDowell/wav2vec2-xls-r-300m-japanese --dataset mozilla-foundation/common_voice_8_0 --config ja --split test --log_outputs
 
 
 
 
 
 
101
  ```
 
27
  - name: Test CER
28
  type: cer
29
  value: 23.64
30
+ - task:
31
+ name: Automatic Speech Recognition
32
+ type: automatic-speech-recognition
33
+ dataset:
34
+ name: Robust Speech Event - Dev Data
35
+ type: speech-recognition-community-v2/dev_data
36
+ args: de
37
+ metrics:
38
+ - name: Test WER
39
+ type: wer
40
+ value: 1.0
41
+ - name: Test CER
42
+ type: cer
43
+ value: 30.99
44
  ---
45
 
46
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
50
 
51
  This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - JA dataset.
52
 
53
+ Kanji are converted into Hiragana using the [pykakasi](https://pykakasi.readthedocs.io/en/latest/index.html) library during training and evaluation. The model can output both Hiragana and Katakana characters. Since there is no spacing, WER is not a suitable metric for evaluating performance and CER is more suitable.
54
+
55
+ On mozilla-foundation/common_voice_8_0 it achieved:
56
+ - cer: 23.64%
57
+
58
+ On speech-recognition-community-v2/dev_data it achieved:
59
+ - cer: 30.99%
60
 
61
  It achieves the following results on the evaluation set:
62
  - Loss: 0.5212
 
117
 
118
  ```bash
119
  python ./eval.py --model_id AndrewMcDowell/wav2vec2-xls-r-300m-japanese --dataset mozilla-foundation/common_voice_8_0 --config ja --split test --log_outputs
120
+ ```
121
+
122
+ 2. To evaluate on `mozilla-foundation/common_voice_8_0` with split `test`
123
+
124
+ ```bash
125
+ python ./eval.py --model_id AndrewMcDowell/wav2vec2-xls-r-300m-japanese --dataset speech-recognition-community-v2/dev_data --config de --split validation --chunk_length_s 5.0 --stride_length_s 1.0
126
  ```