Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,8 @@ Our User Genrated Content (UGC) is comments written on articles collected from 3
|
|
13 |
In order to valid the annotation, we search an agreement between raters to emotion in each sentence using krippendorff's alpha [(krippendorff, 1970)](https://journals.sagepub.com/doi/pdf/10.1177/001316447003000105). We left sentences that got alpha > 0.7. Note that while we found a general agreement between raters about emotion like happy, trust and disgust, there are few emotion with general disagreement about them, apparently given the complexity of finding them in the text (e.g. expectation and surprise).
|
14 |
|
15 |
### Performance
|
16 |
-
#### sentiment analysis
|
|
|
17 |
| | precision | recall | f1-score |
|
18 |
|--------------|-----------|--------|----------|
|
19 |
| natural | 0.83 | 0.56 | 0.67 |
|
@@ -23,6 +24,51 @@ In order to valid the annotation, we search an agreement between raters to emoti
|
|
23 |
| macro avg | 0.92 | 0.82 | 0.86 |
|
24 |
| weighted avg | 0.96 | 0.97 | 0.96 |
|
25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
## Stay tuned!
|
28 |
We are still working on our model and will edit this page as we progress.<br>
|
@@ -34,7 +80,7 @@ our git: https://github.com/avichaychriqui/HeBERT
|
|
34 |
Chriqui, A., & Yahav, I. (2021). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. arXiv preprint arXiv:2102.01909.
|
35 |
```
|
36 |
@article{chriqui2021hebert,
|
37 |
-
title={HeBERT
|
38 |
author={Chriqui, Avihay and Yahav, Inbal},
|
39 |
journal={arXiv preprint arXiv:2102.01909},
|
40 |
year={2021}
|
|
|
13 |
In order to valid the annotation, we search an agreement between raters to emotion in each sentence using krippendorff's alpha [(krippendorff, 1970)](https://journals.sagepub.com/doi/pdf/10.1177/001316447003000105). We left sentences that got alpha > 0.7. Note that while we found a general agreement between raters about emotion like happy, trust and disgust, there are few emotion with general disagreement about them, apparently given the complexity of finding them in the text (e.g. expectation and surprise).
|
14 |
|
15 |
### Performance
|
16 |
+
#### sentiment analysis
|
17 |
+
|
18 |
| | precision | recall | f1-score |
|
19 |
|--------------|-----------|--------|----------|
|
20 |
| natural | 0.83 | 0.56 | 0.67 |
|
|
|
24 |
| macro avg | 0.92 | 0.82 | 0.86 |
|
25 |
| weighted avg | 0.96 | 0.97 | 0.96 |
|
26 |
|
27 |
+
## How to use
|
28 |
+
### For masked-LM model (can be fine-tunned to any down-stream task)
|
29 |
+
from transformers import AutoTokenizer, AutoModel
|
30 |
+
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT")
|
31 |
+
model = AutoModel.from_pretrained("avichr/heBERT")
|
32 |
+
|
33 |
+
from transformers import pipeline
|
34 |
+
fill_mask = pipeline(
|
35 |
+
"fill-mask",
|
36 |
+
model="avichr/heBERT",
|
37 |
+
tokenizer="avichr/heBERT"
|
38 |
+
)
|
39 |
+
fill_mask("ืืงืืจืื ื ืืงืื ืืช [MASK] ืืื ื ืื ื ืฉืืจ ืืืจ.")
|
40 |
+
|
41 |
+
### For sentiment classification model (polarity ONLY):
|
42 |
+
from transformers import AutoTokenizer, AutoModel, pipeline
|
43 |
+
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT_sentiment_analysis") #same as 'avichr/heBERT' tokenizer
|
44 |
+
model = AutoModel.from_pretrained("avichr/heBERT_sentiment_analysis")
|
45 |
+
|
46 |
+
# how to use?
|
47 |
+
sentiment_analysis = pipeline(
|
48 |
+
"sentiment-analysis",
|
49 |
+
model="avichr/heBERT_sentiment_analysis",
|
50 |
+
tokenizer="avichr/heBERT_sentiment_analysis",
|
51 |
+
return_all_scores = True
|
52 |
+
)
|
53 |
+
|
54 |
+
sentiment_analysis('ืื ื ืืชืืื ืื ืืืืื ืืืจืืืช ืฆืืจืืื')
|
55 |
+
>>> [[{'label': 'natural', 'score': 0.9978172183036804},
|
56 |
+
>>> {'label': 'positive', 'score': 0.0014792329166084528},
|
57 |
+
>>> {'label': 'negative', 'score': 0.0007035882445052266}]]
|
58 |
+
|
59 |
+
sentiment_analysis('ืงืคื ืื ืืขืื')
|
60 |
+
>>> [[{'label': 'natural', 'score': 0.00047328314394690096},
|
61 |
+
>>> {'label': 'possitive', 'score': 0.9994067549705505},
|
62 |
+
>>> {'label': 'negetive', 'score': 0.00011996887042187154}]]
|
63 |
+
|
64 |
+
sentiment_analysis('ืื ื ืื ืืืื ืืช ืืขืืื')
|
65 |
+
>>> [[{'label': 'natural', 'score': 9.214012970915064e-05},
|
66 |
+
>>> {'label': 'possitive', 'score': 8.876807987689972e-05},
|
67 |
+
>>> {'label': 'negetive', 'score': 0.9998190999031067}]]
|
68 |
+
|
69 |
+
|
70 |
+
Our model is also available on AWS! for more information visit [AWS' git](https://github.com/aws-samples/aws-lambda-docker-serverless-inference/tree/main/hebert-sentiment-analysis-inference-docker-lambda)
|
71 |
+
|
72 |
|
73 |
## Stay tuned!
|
74 |
We are still working on our model and will edit this page as we progress.<br>
|
|
|
80 |
Chriqui, A., & Yahav, I. (2021). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. arXiv preprint arXiv:2102.01909.
|
81 |
```
|
82 |
@article{chriqui2021hebert,
|
83 |
+
title={HeBERT \\& HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition},
|
84 |
author={Chriqui, Avihay and Yahav, Inbal},
|
85 |
journal={arXiv preprint arXiv:2102.01909},
|
86 |
year={2021}
|