Upload from GitHub Actions: Make community links work, add CONTRIBUTING
Browse files- CONTRIBUTING.md +16 -0
- frontend/src/App.js +5 -2
CONTRIBUTING.md
ADDED
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Contributing
|
2 |
+
|
3 |
+
## Adding a new benchmark dataset
|
4 |
+
|
5 |
+
- Create a new file in `evals/datasets/` for loading the dataset(s) for your task, ideally via HuggingFace's `datasets` library.
|
6 |
+
- Define the task and its evaluation metric(s) in `evals/tasks.py`.
|
7 |
+
- Update `evals/backend.py` and `frontend/src/components/ScoreColumns.js` to include the new task and its metrics.
|
8 |
+
- Submit a pull request.
|
9 |
+
|
10 |
+
## Adding a new model
|
11 |
+
|
12 |
+
You can submit requests [here](https://forms.gle/ckvY9pS7XLcHYnaV8), and we will take care of the rest.
|
13 |
+
|
14 |
+
## Adding a new language
|
15 |
+
|
16 |
+
We believe that we already have all of the languages. If this is not the case, open an issue!
|
frontend/src/App.js
CHANGED
@@ -80,11 +80,14 @@ function App () {
|
|
80 |
<li><b>Policymakers and funders</b> can identify and prioritize neglected languages.</li>
|
81 |
<li><b>Model developers</b> can compete on our <i>AI Language Proficiency</i> metric.</li>
|
82 |
</ul>
|
83 |
-
<p>We invite the community to <a href="
|
84 |
<p>Benchmark results automatically refresh every night and include the most popular models on <a href="https://openrouter.ai" target="_blank" rel="noopener noreferrer">OpenRouter</a>, plus community-listed models.</p>
|
85 |
-
<p>For a detailed methodlogy, see <a href="#">XXX</a>.</p>
|
86 |
<p>The AI Language Proficiency Monitor is a collaboration between BMZ's <a href="https://www.bmz-digital.global/en/overview-of-initiatives/the-bmz-data-lab/" target="_blank" rel="noopener noreferrer">Data Lab</a>, GIZ's <a href="https://www.giz.de/expertise/html/61982.html" target="_blank" rel="noopener noreferrer">FairForward</a> initiative, and the <a href="https://www.dfki.de/en/web/research/research-departments/multilinguality-and-language-technology/ee-team" target="_blank" rel="noopener noreferrer">E&E group</a> of DFKI's Multilinguality and Language Technology Lab.</p>
|
87 |
</div>
|
|
|
|
|
|
|
88 |
</header>
|
89 |
<main
|
90 |
style={{
|
|
|
80 |
<li><b>Policymakers and funders</b> can identify and prioritize neglected languages.</li>
|
81 |
<li><b>Model developers</b> can compete on our <i>AI Language Proficiency</i> metric.</li>
|
82 |
</ul>
|
83 |
+
<p>We invite the community to <a href="https://forms.gle/ckvY9pS7XLcHYnaV8" target="_blank" rel="noopener noreferrer">submit</a> their custom finetuned models, and to <a href="https://github.com/datenlabor-bmz/ai-language-monitor/blob/main/CONTRIBUTING.md" target="_blank" rel="noopener noreferrer">integrate</a> benchmarks for more languages and tasks.</p>
|
84 |
<p>Benchmark results automatically refresh every night and include the most popular models on <a href="https://openrouter.ai" target="_blank" rel="noopener noreferrer">OpenRouter</a>, plus community-listed models.</p>
|
85 |
+
{/* <p>For a detailed methodlogy, see <a href="#">XXX</a>.</p> */}
|
86 |
<p>The AI Language Proficiency Monitor is a collaboration between BMZ's <a href="https://www.bmz-digital.global/en/overview-of-initiatives/the-bmz-data-lab/" target="_blank" rel="noopener noreferrer">Data Lab</a>, GIZ's <a href="https://www.giz.de/expertise/html/61982.html" target="_blank" rel="noopener noreferrer">FairForward</a> initiative, and the <a href="https://www.dfki.de/en/web/research/research-departments/multilinguality-and-language-technology/ee-team" target="_blank" rel="noopener noreferrer">E&E group</a> of DFKI's Multilinguality and Language Technology Lab.</p>
|
87 |
</div>
|
88 |
+
<a href="https://github.com/datenlabor-bmz/ai-language-monitor" target="_blank" rel="noopener noreferrer" style={{ textDecoration: 'none', color: 'inherit', marginTop: '2rem' }}>
|
89 |
+
<i className="pi pi-github" style={{ fontSize: '1.5rem' }} />
|
90 |
+
</a>
|
91 |
</header>
|
92 |
<main
|
93 |
style={{
|