library_name: transformers
license: apache-2.0
base_model: distilbert-base-uncased
tags:
- classification
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: imdb-distilbert
results: []
imdb-distilbert
This model is a fine-tuned version of distilbert-base-uncased. It achieves the following results on the evaluation set:
- Loss: 1.4670
- Accuracy: 0.8528
Model description
This model, named imdb-distilbert, is fine-tuned from the distilbert-base-uncased checkpoint on the IMDB movie review dataset for the sentiment classification task. It's designed to predict whether a movie review is positive or negative based on the textual content of the review. This model can be used to automatically classify new movie reviews into positive or negative categories.
Intended uses & limitations
While effective within its trained domain, the model may exhibit reduced performance on text that diverges significantly from movie reviews in style or content, such as professional critiques or reviews from non-English sources translated to English. The training dataset predominantly contains informal consumer reviews, which might limit the model's effectiveness with formally written text.
Training and evaluation data
The model was trained on the IMDB dataset, which contains 50,000 movie reviews split evenly into 25,000 training and 25,000 testing datasets. Each entry is labeled as either 0 (negative) or 1 (positive). It achieves the following results on the evaluation set:
- Loss: 1.4670
- Accuracy: 0.8528
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.3777 | 1.0 | 3125 | 0.4630 | 0.8263 |
| 0.2795 | 2.0 | 6250 | 0.4771 | 0.8549 |
| 0.1698 | 3.0 | 9375 | 0.5689 | 0.8526 |
| 0.1093 | 4.0 | 12500 | 0.9568 | 0.8460 |
| 0.0664 | 5.0 | 15625 | 1.0550 | 0.8470 |
| 0.0333 | 6.0 | 18750 | 1.1734 | 0.8487 |
| 0.0238 | 7.0 | 21875 | 1.1931 | 0.8482 |
| 0.0123 | 8.0 | 25000 | 1.2663 | 0.8507 |
| 0.0056 | 9.0 | 28125 | 1.3256 | 0.8549 |
| 0.0022 | 10.0 | 31250 | 1.4670 | 0.8528 |
Framework versions
- Transformers 4.47.0
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0