metadata

title: RougeRu
emoji: 🌖
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 3.19.1
app_file: app.py
pinned: false
tags:
  - evaluate
  - metric
description: >-
  ROUGE RU is russian language variant of ROUGE with stemmer and stop words but
  without synonymas. It is case insensitive, meaning that upper case letters are
  treated the same way as lower case letters.
This metrics is a wrapper around Google Research reimplementation of ROUGE: https://github.com/google-research/google-research/tree/master/rouge

Metric Card for RougeRu

Metric Description

RougeRu is a russian language variant of ROUGE with stemmer and stop words but without synonymas. It is case insensitive, meaning that upper case letters are treated the same way as lower case letters. This metrics is a wrapper around Google Research reimplementation of ROUGE: https://github.com/google-research/google-research/tree/master/rouge

How to Use

import evaluate
import nltk
nltk.download("stopwords")
nltk.download("punkt")
nltk.download("punkt_tab")
rouge_ru = evaluate.load('Remeris/rouge_ru')
predictions = ["ну, привет!", "адмирал кеноби!"]
references = ["Ну, привет!", "Генерал Кеноби!"]
results = rouge_ru.compute(predictions=predictions, references=references)
print(results)
{'rouge1': {'recall': 0.75, 'precision': 0.75, 'fmeasure': 0.75},
 'rouge2': {'recall': 0.0, 'precision': 0.0, 'fmeasure': 0.0},
 'rougeL': {'recall': 0.75, 'precision': 0.75, 'fmeasure': 0.75},
 'rougeLsum': {'recall': 0.75, 'precision': 0.75, 'fmeasure': 0.75}}

Inputs

predictions (list): list of predictions to evaluate. Each prediction should be a string with tokens separated by spaces and punctuation.
references (list or list[list]): list of reference for each prediction. Each reference should be a string with tokens separated by spaces and punctuation.

Output Values

This metric outputs a dictionary, containing the scores.

There are precision, recall, F1 values for rouge1, rouge2 and rougeL.

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference