Spaces:
Runtime error
Runtime error
File size: 2,580 Bytes
c8a3800 c56e83f 4564a8c c8a3800 c56e83f c8a3800 4564a8c c56e83f 4564a8c c56e83f 4564a8c c56e83f 4564a8c c56e83f 4564a8c c56e83f 4564a8c c56e83f 4564a8c c56e83f 4564a8c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
---
title: accuracyk
datasets:
-
tags:
- evaluate
- metric
- accuracy
description: "computes the accuracy at k for a set of predictions as labels"
sdk: gradio
sdk_version: 3.0.2
app_file: app.py
pinned: false
---
# accuracyk
## Metric Description
Computes the accuracy at k for a set of predictions. The accuracy at k is the number of instances where the real label is in the set of the k most probable
classes.
The parameter k is inferred from the shape of the array passed. If you want the accuracy at 5 the shape needs to be (N, 5) where N is the number of examples.
## How to Use
```
predictions = np.array([
[0, 7, 1, 3, 5],
[0, 2, 9, 8, 4],
[8, 4, 0, 1, 3],
])
references = np.array([
3,
5,
0
])
results = accuracyk.compute(predictions=predictions, references=references)
# 2/3 of the labels are in the corresponding rows
# the shape of the array predictions is (3, 5) so accuracy at 5 has been computed
# { accuracy: 0.6 }
```
### Inputs
- **predictions**: An array of shape (N, K) where N is the number of examples and K is the desired k (5 for accuracy at 5)
- **references**: An array of the true labels for the examples
### Output Values
The metric returns outputs between 0 and 1. With 0 being that no value is in its corresponding row and 1 being that every value occurs in its row (higher is better).
### Examples
```python
>>> accuracyk = evaluate.load("KevinSpaghetti/accuracyk")
>>> # with numpy arrays
>>> predictions = np.array([
>>> [0, 7, 1, 3, 5],
>>> [0, 2, 9, 8, 4],
>>> [8, 4, 0, 1, 3],
>>> ])
>>> references = np.array([
>>> 3,
>>> 4,
>>> 0
>>> ])
>>> results = accuracyk.compute(predictions=predictions, references=references)
{ accuracy: 1 } # every label is in its row
>>> # With lists
>>> predictions = [
>>> [0, 7, 1, 3, 5],
>>> [0, 2, 9, 8, 4],
>>> [8, 4, 0, 1, 3],
>>> ]
>>> references = [
>>> 3,
>>> 5,
>>> 0
>>> ]
>>> results = accuracyk.compute(predictions=predictions, references=references)
{ accuracy: 0.6 }
>>> # 3 is in the first row,
>>> # 5 is not in the second row,
>>> # 0 is in the third row
>>> # with numpy for a batch of examples
>>> k=5
>>> # get the 5 highest probabilities
>>> top5_probs = np.argpartition(logits, -k, axis=-1)[:, -k:]
>>> results = accuracyk.compute(references=top5_probs, predictions=labels)
>>> # computing the accuracy at 1
>>> predictions = np.array([ 3, 8, 1 ])
>>> references = np.array([ 3, 4, 0 ])
>>> results = accuracyk.compute(predictions=np.expand_dims(predictions, axis=1), references=references)
``` |