dalmeow commited on
Commit
faba537
·
1 Parent(s): abbe4df

readme updated

Browse files
Files changed (4) hide show
  1. README.md +25 -38
  2. __pycache__/app.cpython-38.pyc +0 -0
  3. app.py +7 -1
  4. formula.png +0 -0
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
  title: WER
3
- emoji: 🤗
4
  colorFrom: blue
5
  colorTo: red
6
  sdk: gradio
@@ -9,44 +9,31 @@ app_file: app.py
9
  pinned: false
10
  tags:
11
  - evaluate
12
- - metric
13
- description: >-
14
- Word error rate (WER) is a common metric of the performance of an automatic
15
- speech recognition system.
16
-
17
- The general difficulty of measuring performance lies in the fact that the
18
- recognized word sequence can have a different length from the reference word
19
- sequence (supposedly the correct one). The WER is derived from the Levenshtein
20
- distance, working at the word level instead of the phoneme level. The WER is a
21
- valuable tool for comparing different systems as well as for evaluating
22
- improvements within one system. This kind of measurement, however, provides no
23
- details on the nature of translation errors and further work is therefore
24
- required to identify the main source(s) of error and to focus any research
25
- effort.
26
-
27
- This problem is solved by first aligning the recognized word sequence with the
28
- reference (spoken) word sequence using dynamic string alignment. Examination
29
- of this issue is seen through a theory called the power law that states the
30
- correlation between perplexity and word error rate.
31
-
32
- Word error rate can then be computed as:
33
-
34
- WER = (S + D + I) / N = (S + D + I) / (S + D + C)
35
-
36
- where
37
-
38
- S is the number of substitutions, D is the number of deletions, I is the
39
- number of insertions, C is the number of correct words, N is the number of
40
- words in the reference (N=S+D+C).
41
-
42
- This value indicates the average number of errors per reference word. The
43
- lower the value, the better the performance of the ASR system with a WER of 0
44
- being a perfect score.
45
- duplicated_from: evaluate-metric/wer
46
  ---
47
 
48
- # WER hmm
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
 
50
- ## Metric description
51
 
52
- lorem ipsum
 
1
  ---
2
  title: WER
3
+ emoji: 🖩
4
  colorFrom: blue
5
  colorTo: red
6
  sdk: gradio
 
9
  pinned: false
10
  tags:
11
  - evaluate
12
+ - wer
13
+ - neuralspace
14
+ - STT
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ---
16
 
17
+ ## About this Demo
18
+
19
+ This demo was built as a part of NeuralSpace's VoiceAI blog on [Word Error Rate 101: Your Guide to STT Vendor Evaluation](voice.neuralspace.ai).
20
+
21
+ ## What is WER?
22
+
23
+ WER or Word Error Rate is a metric used primarily in the field of speech recognition to measure the performance of an automatic speech recognition (ASR) system. WER calculates the minimum number of operations (substitutions, deletions, and insertions) required to change the system's transcription (prediction) into the reference transcription (truth), divided by the number of words in the reference.
24
+
25
+ ![Formula](formula.png)
26
+
27
+ WER can range from 0 to infinite. The closer the WER is to 0, the better. WER is often also represented as a percentage. It is usually calculated by just multiplying 100 to it. For example, a WER of 0.15 might also be represented as 15%.
28
+
29
+ WER is important because it provides:
30
+ * **Performance Measure**: It gives an objective measure of how well an ASR system is transcribing speech into text.
31
+ * **Comparison**: It allows for comparison between different ASR systems or versions of a system.
32
+
33
+
34
+ ## References
35
+
36
+ * Python package to calculate WER: [jiwer](https://jitsi.github.io/jiwer/)
37
+
38
 
 
39
 
 
__pycache__/app.cpython-38.pyc ADDED
Binary file (1.48 kB). View file
 
app.py CHANGED
@@ -28,7 +28,13 @@ def compute(input):
28
  return err
29
 
30
 
31
- description = "This is a calculator that you can use to find WER"
 
 
 
 
 
 
32
 
33
  demo = gr.Interface(
34
  fn=compute,
 
28
  return err
29
 
30
 
31
+ description = """
32
+ To calculate WER:
33
+
34
+ * Type the `prediction` and the `truth` in the respective columns in the below calculator.
35
+ * You can insert multiple predictions and truths by clicking on the `New row` button.
36
+ * To calculate the WER after inserting all the texts, click on `Submit`.
37
+ """
38
 
39
  demo = gr.Interface(
40
  fn=compute,
formula.png ADDED