Accuracy Drop
#1
by
mhyatt000
- opened
I tried to reproduce the results mentioned on this model card. Seems like my results do not match the claimed accuracy in the model card. I cannot figure out how to get the correct numbers, can you help me find my mistake?
- Claimed accuracy: 92.22%
- Received accuracy: 0.55%
Here are the details for my validation:
- I instantiate pre-trained model with transformers.pipeline() and use measure accuracy
- Evaluation was performed on CPU.
- Dataset was downloaded from locally from https://huggingface.co/datasets/emotion using load_dataset()