|
--- |
|
library_name: transformers |
|
tags: [] |
|
--- |
|
|
|
norbert3-small trained on wikiann (fo/is), sucx3 (se), dane (da) and norne (nb/nn) |
|
|
|
added a custom clf head along with a character-level cnn for adding a tiny extra signal for the classification. |
|
|
|
results: |
|
```css |
|
Eval on wikiann - fo |
|
index 0 |
|
tokens [Byrta, -, Aftur, og, aftur] |
|
ner_tags [3, 0, 0, 0, 0] |
|
subset fo |
|
dataset wikiann |
|
Name: 0, dtype: object |
|
shape: (100, 5) |
|
100% |
|
5/5 [00:01<00:00, 3.92it/s] |
|
Loss: 0.2276667356491089 |
|
O O |
|
B-ORG B-ORG |
|
B-ORG B-ORG |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
Validation Loss: 0.26530784368515015 |
|
Validation Accuracy: 0.9228951181745751 |
|
precision recall f1-score support |
|
|
|
LOC 0.86 0.81 0.83 154 |
|
ORG 0.67 0.73 0.70 125 |
|
PER 0.87 0.91 0.89 79 |
|
|
|
micro avg 0.79 0.80 0.80 358 |
|
macro avg 0.80 0.82 0.81 358 |
|
weighted avg 0.79 0.80 0.80 358 |
|
|
|
________________________________________ |
|
Eval on wikiann - is |
|
index 100 |
|
tokens [Beltaþyrill, ''Ceryle, alcyon, '', Sjaldséð] |
|
ner_tags [5, 0, 0, 0, 0] |
|
subset is |
|
dataset wikiann |
|
Name: 0, dtype: object |
|
shape: (1000, 5) |
|
100% |
|
50/50 [00:10<00:00, 5.02it/s] |
|
Loss: 0.22668001055717468 |
|
O O |
|
B-LOC B-LOC |
|
B-LOC B-LOC |
|
B-LOC B-LOC |
|
B-LOC B-LOC |
|
B-LOC B-LOC |
|
B-LOC B-LOC |
|
O O |
|
O O |
|
O O |
|
Validation Loss: 0.2526825902983546 |
|
Validation Accuracy: 0.9360383541181041 |
|
precision recall f1-score support |
|
|
|
LOC 0.84 0.85 0.84 1983 |
|
ORG 0.81 0.80 0.80 1762 |
|
PER 0.89 0.89 0.89 1020 |
|
|
|
micro avg 0.84 0.84 0.84 4765 |
|
macro avg 0.84 0.85 0.85 4765 |
|
weighted avg 0.84 0.84 0.84 4765 |
|
|
|
________________________________________ |
|
Eval on dane - default |
|
index 1100 |
|
tokens [To, kendte, russiske, historikere, Andronik, ... |
|
ner_tags [0, 0, 7, 0, 1, 2, 0, 1, 2, 0, 0, 0, 0, 5, 0, ... |
|
subset default |
|
dataset dane |
|
Name: 0, dtype: object |
|
shape: (565, 5) |
|
100% |
|
29/29 [00:06<00:00, 4.75it/s] |
|
Loss: 0.12037135660648346 |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
B-MISC B-MISC |
|
O O |
|
O O |
|
B-PER B-PER |
|
B-PER B-PER |
|
Validation Loss: 0.11113663488228259 |
|
Validation Accuracy: 0.972018408457994 |
|
precision recall f1-score support |
|
|
|
LOC 0.78 0.86 0.82 225 |
|
MISC 0.72 0.52 0.61 333 |
|
ORG 0.72 0.69 0.71 379 |
|
PER 0.96 0.92 0.94 298 |
|
|
|
micro avg 0.80 0.73 0.76 1235 |
|
macro avg 0.80 0.75 0.77 1235 |
|
weighted avg 0.79 0.73 0.76 1235 |
|
|
|
________________________________________ |
|
Eval on norne - bokmaal-7 |
|
index 1665 |
|
tokens [Honnørordene, er, ", dristig, formspråk, ", ,... |
|
ner_tags [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] |
|
subset bokmaal-7 |
|
dataset norne |
|
Name: 0, dtype: object |
|
shape: (1939, 5) |
|
100% |
|
97/97 [00:20<00:00, 4.56it/s] |
|
Loss: 0.0011819382198154926 |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
Validation Loss: 0.04194018930858649 |
|
Validation Accuracy: 0.9876322465792248 |
|
precision recall f1-score support |
|
|
|
LOC 0.85 0.90 0.87 498 |
|
MISC 0.81 0.74 0.78 363 |
|
ORG 0.77 0.83 0.80 499 |
|
PER 0.93 0.96 0.95 845 |
|
|
|
micro avg 0.86 0.88 0.87 2205 |
|
macro avg 0.84 0.86 0.85 2205 |
|
weighted avg 0.86 0.88 0.87 2205 |
|
|
|
________________________________________ |
|
Eval on norne - nynorsk-7 |
|
index 3604 |
|
tokens [Den, er, mettande, og, smakfull, ,, og, det, ... |
|
ner_tags [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... |
|
subset nynorsk-7 |
|
dataset norne |
|
Name: 0, dtype: object |
|
shape: (1511, 5) |
|
100% |
|
76/76 [00:15<00:00, 5.82it/s] |
|
Loss: 0.0790824368596077 |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
Validation Loss: 0.05325472676725583 |
|
Validation Accuracy: 0.9867293689853402 |
|
precision recall f1-score support |
|
|
|
LOC 0.77 0.91 0.84 365 |
|
MISC 0.80 0.76 0.78 295 |
|
ORG 0.83 0.82 0.82 397 |
|
PER 0.98 0.95 0.97 664 |
|
|
|
micro avg 0.87 0.88 0.87 1721 |
|
macro avg 0.85 0.86 0.85 1721 |
|
weighted avg 0.87 0.88 0.87 1721 |
|
|
|
________________________________________ |
|
Eval on sucx3_ner - original_cased |
|
index 5115 |
|
tokens [Just, i, dag, är, Saabs, företagsledning, där... |
|
ner_tags [0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... |
|
subset original_cased |
|
dataset sucx3_ner |
|
Name: 0, dtype: object |
|
shape: (14383, 5) |
|
100% |
|
720/720 [02:36<00:00, 5.02it/s] |
|
Loss: 0.04177908971905708 |
|
Loss: 0.08230985613484489 |
|
Loss: 0.08399457804886486 |
|
Loss: 0.06163447560524267 |
|
Loss: 0.04787629511204947 |
|
Loss: 0.03949779063830233 |
|
Loss: 0.03397762095776484 |
|
Loss: 0.030040143460689266 |
|
O O |
|
O O |
|
O O |
|
O O |
|
O O |
|
B-ORG B-ORG |
|
B-ORG B-ORG |
|
O O |
|
O O |
|
O O |
|
Validation Loss: 0.02938824465528948 |
|
Validation Accuracy: 0.9919830972756728 |
|
precision recall f1-score support |
|
|
|
LOC 0.88 0.91 0.90 4202 |
|
MISC 0.65 0.59 0.62 1899 |
|
ORG 0.74 0.76 0.75 3015 |
|
PER 0.92 0.93 0.92 5778 |
|
|
|
micro avg 0.84 0.84 0.84 14894 |
|
macro avg 0.80 0.80 0.80 14894 |
|
weighted avg 0.84 0.84 0.84 14894 |
|
|
|
________________________________________ |
|
``` |
|
|
|
|
