README.md

---
title: Perplexity
emoji: 🌖
colorFrom: purple
colorTo: pink
sdk: gradio
sdk_version: 4.7.1
app_file: app.py
pinned: false
license: apache-2.0
tags:
- evaluate
- metric
description: >-
  Perplexity metric implemented by d-Matrix.
  Perplexity (PPL) is one of the most common metrics for evaluating language models.
  It is defined as the exponentiated average negative log-likelihood of a sequence, calculated with exponent base `e`.
  For more information, see https://huggingface.co/docs/transformers/perplexity
---

# Metric Card for Perplexity


## Metric Description

Perplexity metric implemented by d-Matrix.
Perplexity (PPL) is one of the most common metrics for evaluating language models.
It is defined as the exponentiated average negative log-likelihood of a sequence, calculated with exponent base `e`.
For more information, see https://huggingface.co/docs/transformers/perplexity

## How to Use
At minimum, this metric requires the model and text as inputs.
```python
>>> perplexity = evaluate.load("d-matrix/perplexity", module_type="metric")
>>> input_texts = ["lorem ipsum", "Happy Birthday!", "Bienvenue"]
>>> results = perplexity.compute(model='distilgpt2',text=input_texts)
>>> print(results)
{'accuracy': 1.0}
```