File size: 7,846 Bytes
d44dab0 7c0c353 d44dab0 5ba617d d44dab0 7c0c353 d44dab0 7c0c353 d44dab0 7c0c353 d44dab0 7c0c353 d44dab0 7c0c353 a6ef7b6 d44dab0 4d48827 ab20ce4 d44dab0 f75676a d44dab0 f75676a d44dab0 069bc85 d44dab0 5ba617d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 |
---
language:
- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- dair-ai/emotion
metrics:
- accuracy
- f1
base_model: distilbert-base-uncased
model-index:
- name: distilbert-base-uncased-finetuned-emotion
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: emotion
type: emotion
config: split
split: validation
args: split
metrics:
- type: accuracy
value: 0.9375
name: Accuracy
- type: f1
value: 0.937890467332837
name: F1
- task:
type: text-classification
name: Text Classification
dataset:
name: dair-ai/emotion
type: dair-ai/emotion
config: split
split: test
metrics:
- type: accuracy
value: 0.93
name: Accuracy
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTExYzEzMWNmYTNlYmI0NWNjYTIwMzU3MmUyYmY0ZDZjMjQwOTMzYWMwOTZiY2U4YTA2ZDE0NmM2YzNlMzNkZiIsInZlcnNpb24iOjF9.rwu31KKjXkNu7uVA-vxi4NX8Fd2cJrnAmWbIIt174dmi24nlB56g7IDBfTrGzFdnMzkCuDpLng8pnvXFoN3ZCg
- type: f1
value: 0.8869023464973423
name: F1 Macro
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzI0NjEzNmEwM2VjNTg0MjY2ZmYwZTA0ZGJkOTI2ZWFlNTUxNzA1ZGNkNzNhZGQ1NGZlZTVhZGY4ZGUwZjc5YyIsInZlcnNpb24iOjF9.PLxM2vSrYDzbdKVaK3QqI_J8ujKvTUfpSfQmC-MsHNgTw7329UaiROWhe1bhadQcgNolLgtwlLFhXyR593fGAA
- type: f1
value: 0.93
name: F1 Micro
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDFlMGY1ZDljZTQ3NzI5N2FmNTFkOWY0N2EzNTk5MGZhOWM4MDVkYjQ2NDk1NTU3MWZiMDBhNTc4YWE2MTFkOSIsInZlcnNpb24iOjF9.5wCoKKKKl0p9S0nAN2OuiPe3c9VnBmTHnJjWWdHgBmcbJ2CrVjZzejUXnfpsuaVJSxSmOZfdI6h18z_fQRgqAQ
- type: f1
value: 0.9300315549555708
name: F1 Weighted
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjcxNGMyNWE5ODg3MDczMzM5ZjE0YmU3YTRmNTM4MTQwYjhmMDcxOGU1NGU4YTBmZGI5NmM5OTRiY2VhYzQ3ZCIsInZlcnNpb24iOjF9.l1MbXmlI8txam4EttXSOWaIgfN9sKe0ZBKc_TXwWre8DNgPFVwVD4jWeQxlMRC0LtWIL5fIqEdv8qj5DqVz1BQ
- type: precision
value: 0.892405250997362
name: Precision Macro
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjg4Y2U3ODNkZjRkMWQ1YzZjMWUzYWRkOTk1YzJkNmU1NmVkZGNjZDA5NmQzNmVkNDExNzBlZDZiZDg1YWU2YSIsInZlcnNpb24iOjF9.WoctTcjRNIbgZo4pqUQVbKmOc0iudqmqO9ABr043llfHLIWe8ZjMyFek3OBUTgw7wVe35UDNGeRc308OxS7CAQ
- type: precision
value: 0.93
name: Precision Micro
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTczZGUwMTQ1MWY2NGE4YWU0NWU3YjA0MzdhMGUyYWUzYjAxOTgzYmYwNmNhYWIxZTBhODE4YjMyOTc5NDAwYSIsInZlcnNpb24iOjF9.Iy2em6yVS3K4izKHRhnap2RWHgZQ5hup8nmtNVmb7avz5x3HWUnzwAUq_EsWht_7Hf59YUPuWW_xv9EZXZYVDQ
- type: precision
value: 0.9314726605632766
name: Precision Weighted
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDljOTNmMjZjNWFkZGQzNTIxNzZhNTRjYTEwN2E0MjdmYWY2MzdjMmUyMzZlZGQzZmFlMDJiNmIxMDUxYTM1MyIsInZlcnNpb24iOjF9.S2ERnUMdG3WZyRIpYY4ZOPPMdy3VvUVcNA3sQ8uij5S-1upDs5DeSxqgXKGggxANkj1lSOda8sDSLYgswTV4DQ
- type: recall
value: 0.8858832612260938
name: Recall Macro
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjllMGU4MmRiYjljNDQ0NTEwOGIzOTMzMDI1NThjZDdhYjc0OTNmYTcwODU0ZTNmYjEwOWQ4Mjg0MDAyNjNkOSIsInZlcnNpb24iOjF9.yl3TFQCdTFGGdxbDdTFK4yAY8segyexnvqY0uViW9bEQsfXVgP-lE3veKnsakAKo4HD6nlthHUxtgJLZhYcfAg
- type: recall
value: 0.93
name: Recall Micro
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWZkNDg2YjdiZTMzMDljNWNiMzk0MzQ2MjA0NzBjMzkwMWE3OWI1YTgwNTkxODhjYzFkMjQ2Yjg4ZjkyZDE1ZCIsInZlcnNpb24iOjF9.I76FPjzyPTVxC8w5ZgDOnsGOx1weTzizDujT98WPGO3AnibtlJJJhPeQdBKxe80LGE7QpjLx0e-R9UvjpnJmAQ
- type: recall
value: 0.93
name: Recall Weighted
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOGY2ZmI1OWU3YWVjMDM3NjZjNjhlNWZiMDhlMzA3MDU0ZjExYTA5OTA1ZDBjY2YwYTA0NDE2ODY1ZGIwYmU2OCIsInZlcnNpb24iOjF9.9NACQL-gIUD57HmB62GIo5nqJJVy9k_iUo72onHCyfpCa9K0XKhi_UoKJNoV5htumHD2zKKucwUx1p8X17bUAg
- type: loss
value: 0.1600879579782486
name: loss
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTYzNzkwMWY1ZDMwMTVhN2Y4NDkwNTgyYWNjNjkwMTIwYTNjZjY0N2M0MWM5NjNlNjA3ZWJmY2VhNTQ2MGUyOCIsInZlcnNpb24iOjF9.YYmdyZk5D8B0Fb7M2ysyLBGdSVxGOSgWGfXhz7h0UZaeLijo04IiFQfNWiYNtd_UZ2QcpMeZtkuWdMK8ZHPxBQ
---
# distilbert-base-uncased-finetuned-emotion
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the emotion dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1448
- Accuracy: 0.9375
- F1: 0.9379
The notebook used to fine-tune this model may be found [HERE](https://www.kaggle.com/marcoloureno/distilbert-base-uncased-finetuned-emotion).
## Model description
DistilBERT is a transformers model, smaller and faster than BERT, which was pretrained on the same corpus in a
self-supervised fashion, using the BERT base model as a teacher. This means it was pretrained on the raw texts only,
with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic
process to generate inputs and labels from those texts using the BERT base model. More precisely, it was pretrained
with three objectives:
- Distillation loss: the model was trained to return the same probabilities as the BERT base model.
- Masked language modeling (MLM): this is part of the original training loss of the BERT base model. When taking a
sentence, the model randomly masks 15% of the words in the input then run the entire masked sentence through the
model and has to predict the masked words. This is different from traditional recurrent neural networks (RNNs) that
usually see the words one after the other, or from autoregressive models like GPT which internally mask the future
tokens. It allows the model to learn a bidirectional representation of the sentence.
- Cosine embedding loss: the model was also trained to generate hidden states as close as possible as the BERT base
model.
This way, the model learns the same inner representation of the English language than its teacher model, while being
faster for inference or downstream tasks.
## Intended uses & limitations
[Emotion](https://huggingface.co/datasets/dair-ai/emotion) is a dataset of English Twitter messages with six basic emotions: anger, fear, joy, love, sadness, and surprise. This dataset was developed for the paper entitled "CARER: Contextualized Affect Representations for Emotion Recognition" (Saravia et al.) through noisy labels, annotated via distant
supervision as in the paper"Twitter sentiment classification using distant supervision" (Go et al).
The DistilBERT model was fine-tuned to this dataset, allowing for the classification of sentences into one of the six basic emotions (anger, fear, joy, love, sadness, and surprise).
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
### Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
| 0.5337 | 1.0 | 250 | 0.1992 | 0.927 | 0.9262 |
| 0.1405 | 2.0 | 500 | 0.1448 | 0.9375 | 0.9379 |
### Framework versions
- Transformers 4.30.2
- Pytorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.13.3 |