imdb-distilbert

This model is a fine-tuned version of distilbert-base-uncased. It achieves the following results on the evaluation set:

• Loss: 1.4670
• Accuracy: 0.8528

Model description

This model, named imdb-distilbert, is fine-tuned from the distilbert-base-uncased checkpoint on the IMDB movie review dataset for the sentiment classification task. It's designed to predict whether a movie review is positive or negative based on the textual content of the review. This model can be used to automatically classify new movie reviews into positive or negative categories.

Intended uses & limitations

While effective within its trained domain, the model may exhibit reduced performance on text that diverges significantly from movie reviews in style or content, such as professional critiques or reviews from non-English sources translated to English. The training dataset predominantly contains informal consumer reviews, which might limit the model's effectiveness with formally written text.

Training and evaluation data

The model was trained on the IMDB dataset, which contains 50,000 movie reviews split evenly into 25,000 training and 25,000 testing datasets. Each entry is labeled as either 0 (negative) or 1 (positive). It achieves the following results on the evaluation set:

• Loss: 1.4670
• Accuracy: 0.8528

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.3777	1.0	3125	0.4630	0.8263
0.2795	2.0	6250	0.4771	0.8549
0.1698	3.0	9375	0.5689	0.8526
0.1093	4.0	12500	0.9568	0.8460
0.0664	5.0	15625	1.0550	0.8470
0.0333	6.0	18750	1.1734	0.8487
0.0238	7.0	21875	1.1931	0.8482
0.0123	8.0	25000	1.2663	0.8507
0.0056	9.0	28125	1.3256	0.8549
0.0022	10.0	31250	1.4670	0.8528

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0