---
library_name: transformers
license: apache-2.0
base_model: distilbert-base-uncased
tags:
- classification
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: imdb-distilbert
  results: []
---

# imdb-distilbert

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased).
It achieves the following results on the evaluation set:
- Loss: 1.4670
- Accuracy: 0.8528

## Model description

This model, named imdb-distilbert, is fine-tuned from the distilbert-base-uncased checkpoint on the IMDB movie review dataset for the sentiment classification task. 
It's designed to predict whether a movie review is positive or negative based on the textual content of the review.
This model can be used to automatically classify new movie reviews into positive or negative categories.


## Intended uses & limitations

While effective within its trained domain, the model may exhibit reduced performance on text that diverges significantly from movie reviews in style or content, 
such as professional critiques or reviews from non-English sources translated to English. The training dataset predominantly contains informal consumer reviews, 
which might limit the model's effectiveness with formally written text.

## Training and evaluation data

The model was trained on the IMDB dataset, which contains 50,000 movie reviews split evenly into 25,000 training  and 25,000 testing datasets. 
Each entry is labeled as either 0 (negative) or 1 (positive).
It achieves the following results on the evaluation set:
- Loss: 1.4670
- Accuracy: 0.8528

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.3777        | 1.0   | 3125  | 0.4630          | 0.8263   |
| 0.2795        | 2.0   | 6250  | 0.4771          | 0.8549   |
| 0.1698        | 3.0   | 9375  | 0.5689          | 0.8526   |
| 0.1093        | 4.0   | 12500 | 0.9568          | 0.8460   |
| 0.0664        | 5.0   | 15625 | 1.0550          | 0.8470   |
| 0.0333        | 6.0   | 18750 | 1.1734          | 0.8487   |
| 0.0238        | 7.0   | 21875 | 1.1931          | 0.8482   |
| 0.0123        | 8.0   | 25000 | 1.2663          | 0.8507   |
| 0.0056        | 9.0   | 28125 | 1.3256          | 0.8549   |
| 0.0022        | 10.0  | 31250 | 1.4670          | 0.8528   |


### Framework versions

- Transformers 4.47.0
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0