File size: 999 Bytes
a243523 73f635f a243523 73f635f a243523 73f635f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
---
title: WIL
emoji: 🐠
colorFrom: red
colorTo: purple
sdk: gradio
sdk_version: 3.27.0
app_file: app.py
pinned: false
tags:
- evaluate
- metric
---
Word Information Loss can be used to evaluate the performance of an automatic speech recognizer. It has information-theoretic backings, is symmetric between predictions and targets, and is bounded between 0 and 1.
The formula for WIL is
WIL = 1 - (C/P)(C/T)
where
C is the number of correct words,
P is the number of words in the prediction,
T is the number of words in the target.
This value measures the amount of information loss between two sentences. A score of 0 indicates that the prediction and target match perfectly.
Here is a comparison of WER and WIL: (assuming that X,Y,Z each represents a different word)
| Target | Prediction | WER | WIL |
| ------------- | ------------- | ------------- | ------------- |
| X | X | 1 | 1 |
| X | Y | 0 | 0 |
| X | XZZZ | 3 | 0.75 |
| XYYY | X | 0.75 | 0.75 |
| XYY | XZ | 0.67 | 0.83 | |