rasyosef commited on
Commit
b955352
·
verified ·
1 Parent(s): 17f2a60

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -32
README.md CHANGED
@@ -54,42 +54,35 @@ You can use this model directly with a pipeline for masked language modeling:
54
  'sequence': 'ከሀገራቸው ከኢትዮጵያ ከወጡ ግማሽ ምዕተ ዓመታትን ተቆጥሯል ።'}]
55
  ```
56
 
57
- # Fine-tuning
58
 
59
- The following github repository contains a [notebook](https://github.com/rasyosef/amharic-news-category-classification/blob/main/%5Bbert-mini-amharic%5D%20Amharic%20News%20Category%20Classification.ipynb) that fine-tunes this model for an Amharic text classification task.
60
 
61
- https://github.com/rasyosef/amharic-news-category-classification
 
 
 
 
 
62
 
63
- #### Fine-tuned Model Performance
64
- Since this is a multi-class classification task, the reported precision, recall, and f1 metrics are macro averages.
65
 
66
- |Model|Size(# params)|Accuracy|Precision|Recall|F1|
67
- |-----|--------------|--------|---------|------|--|
68
- |bert-mini-amharic|9.67M|0.87|0.83|0.83|0.83|
69
- |bert-small-amharic|25.7M|0.89|0.86|0.87|0.86|
70
- |xlm-roberta-base|279M|0.9|0.88|0.88|0.88|
71
-
72
- ### Sentiment Classification
73
- The model was finetuned on the [amharic-sentiment](https://huggingface.co/datasets/rasyosef/amharic-sentiment)
74
- dataset to classify the given text as having `positive` or `negative` sentiment
75
 
76
- |Model|Size (# params)| Accuracy | Precision | Recall | F1 |
77
- | --- | ------------- | -------- | --------- | ------ | -- |
78
- |bert-medium-amharic|40.5M|0.83|0.83|0.82|0.83|
79
- |bert-small-amharic|27.8M|0.83|0.83|0.82|0.83|
80
- |**bert-mini-amharic**|10.7M|0.81|0.81|0.81|0.81|
81
- |bert-tiny-amharic|4.18M|0.79|0.79|0.79|0.79|
82
- |xlm-roberta-base|279M|0.83|0.83|0.83|0.83|
83
- |am-roberta|443M|0.82|0.83|0.82|0.82|
84
 
85
- ### Named Entity Recognition
86
- The model was finetuned on the [amharic-named-entity-recognition](https://huggingface.co/datasets/rasyosef/amharic-named-entity-recognition) dataset.
87
 
88
- |Model|Size (# params)| Precision | Recall | F1 |
89
- | --- | ------------- | --------- |------- | -- |
90
- |bert-medium-amharic|40.5M|0.64|0.73|0.68|
91
- |bert-small-amharic|27.8M|0.64|0.72|0.68|
92
- |**bert-mini-amharic**|10.7M|0.60|0.67|0.64|
93
- |bert-tiny-amharic|4.18M|0.50|0.59|0.54|
94
- |xlm-roberta-base|279M|0.69|0.79|0.73|
95
- |am-roberta|443M|0.67|0.72|0.69|
 
54
  'sequence': 'ከሀገራቸው ከኢትዮጵያ ከወጡ ግማሽ ምዕተ ዓመታትን ተቆጥሯል ።'}]
55
  ```
56
 
57
+ # Finetuning
58
 
59
+ This model was finetuned and evaluated on the following amharic nlp tasks
60
 
61
+ - Sentiment Classification
62
+ - Dataset: [amharic-sentiment](https://huggingface.co/datasets/rasyosef/amharic-sentiment)
63
+ - Named Entity Recognition
64
+ - Dataset: [amharic-named-entity-recognition](https://huggingface.co/datasets/rasyosef/amharic-named-entity-recognition)
65
+ - News Category Classification
66
+ - Dataset: [amharic-news-category-classification](https://github.com/rasyosef/amharic-news-category-classification)
67
 
68
+ ### Finetuned Model Performance
69
+ The reported F1 scores are macro averages.
70
 
71
+ |Model|Size (# params)| Perplexity|Sentiment (F1)| Named Entity Recognition (F1)|
72
+ |-----|---------------|-----------|--------------|------------------------------|
73
+ |bert-medium-amharic|40.5M|13.74|0.83|0.68|
74
+ |bert-small-amharic|27.8M|15.96|0.83|0.68|
75
+ |**bert-mini-amharic**|**10.7M**|**22.42**|**0.81**|**0.64**|
76
+ |bert-tiny-amharic|4.18M|71.52|0.79|0.54|
77
+ |xlm-roberta-base|279M||0.83|0.73|
78
+ |am-roberta|443M||0.82|0.69|
 
79
 
80
+ ### Amharic News Category Classification
 
 
 
 
 
 
 
81
 
82
+ The following github repository contains a [notebook](https://github.com/rasyosef/amharic-news-category-classification/blob/main/%5Bbert-mini-amharic%5D%20Amharic%20News%20Category%20Classification.ipynb) that fine-tunes this model for an Amharic text classification task using the [amharic-news-category-classification](https://github.com/rasyosef/amharic-news-category-classification) dataset.
 
83
 
84
+ |Model|Size(# params)|Accuracy|Precision|Recall|F1|
85
+ |-----|--------------|--------|---------|------|--|
86
+ |bert-small-amharic|25.7M|0.89|0.86|0.87|0.86|
87
+ |**bert-mini-amharic**|9.67M|0.87|0.83|0.83|0.83|
88
+ |xlm-roberta-base|279M|0.9|0.88|0.88|0.88|