|
--- |
|
title: WIL |
|
emoji: 🐠 |
|
colorFrom: red |
|
colorTo: purple |
|
sdk: gradio |
|
sdk_version: 3.27.0 |
|
app_file: app.py |
|
pinned: false |
|
tags: |
|
- evaluate |
|
- metric |
|
--- |
|
|
|
Word Information Loss can be used to evaluate the performance of an automatic speech recognizer. It has information-theoretic backings, is symmetric between predictions and targets, and is bounded between 0 and 1. |
|
|
|
The formula for WIL is |
|
|
|
WIL = 1 - (C/P)(C/T) |
|
|
|
where |
|
C is the number of correct words, |
|
P is the number of words in the prediction, |
|
T is the number of words in the target. |
|
|
|
This value measures the amount of information loss between two sentences. A score of 0 indicates that the prediction and target match perfectly. |
|
|
|
Here is a comparison of WER and WIL: (assuming that X,Y,Z each represents a different word) |
|
| Target | Prediction | WER | WIL | |
|
| ------------- | ------------- | ------------- | ------------- | |
|
| X | X | 1 | 1 | |
|
| X | Y | 0 | 0 | |
|
| X | XZZZ | 3 | 0.75 | |
|
| XYYY | X | 0.75 | 0.75 | |
|
| XYY | XZ | 0.67 | 0.83 | |
