{{cookiecutter.repo_name}}/README.md

---
benchmark: superb
type: model-upload
submission_nme: none
---

# SUPERB submissions for {{cookiecutter.repo_name}}

## Submitting to the leaderboard

To make a submission to the [leaderboard](https://huggingface.co/spaces/superb/superb-leaderboard)), there are three main steps:

1. Generate predictions on the unlabeled test set of each task
2. Validate the predictions are compatible with the evaluation framework
3. Push the predictions to the Hub!

See the instructions below for more details.

### Rules

1. To prevent overfitting to the public leaderboard, we only evaluate **one submission per week**. You can push predictions to the Hub as many times as you wish, but we will only evaluate the most recent commit in a given week. 
2. Transfer or meta-learning using other datasets, including further pre-training on other corpora, is allowed.
3. Use of unlabeled test data is allowed, as is it always available in the applied setting. For example, further pre-training using the unlabeled data for a task would be permitted.
4. Systems may be augmented with information retrieved from the internet, e.g. via automated web searches.

### Submission file format

### Validate your submission

To ensure that your submission files are correctly formatted, run the following command from the root of the repository:

```
python cli.py validate
```

If everything is correct, you should see the following message:

```
All submission files validated! ✨ 🚀 ✨
Now you can make a submission 🤗
```

### Push your submission to the Hugging Face Hub!

The final step is to commit your files and push them to the Hub:

```
python cli.py submit
```

If there are no errors, you should see the following message:

```
Submission successful! 🎉 🥳 🎉
Your submission will be fine-tuned and evaulated on Sunday 05 September 2021 ⏳
```

where the evaluation is run every Sunday and your results will be visible on the leaderboard.