---
license: unknown
language:
- en
library_name: transformers
pipeline_tag: text-classification
---
# Distil BERT Base Uncased Text Classification Model

This repository contains a Distil BERT Base Uncased model that has been fine-tuned for a custom text classification use case. The model is designed to classify text into nine different classes based on the following categories:

1. **Neutral**: This class is for any other sort of sentences that do not fall into the specific categories below.

2. **Play**: Use this class to classify the intent of the user to listen to music or audio.

3. **Visit**: This class is intended for classifying the user's intent to visit or open a website in a web browser.

4. **ImgReco**: Use this class to make the bot process an image for image-to-text conversion or any image recognition tasks.

5. **Close**: For classifying sentences that indicate the user's intent to close a specific application or software.

6. **Web Search**: This class is designed to identify the user's intent to search the internet for information.

7. **Open**: Use this class for classifying the intent of opening a specific app or software.

8. **ImgGen**: This class is for sentences related to text-to-image processing or image generation tasks.

9. **Math**: Classify sentences that include mathematical equations or expressions using this category.

## Model Details

- **Model Architecture**: Distil BERT Base Uncased
- **Number of Classes**: 9
- **Training Data**: The model was trained on a custom dataset for text classification tasks related to the mentioned categories.

## Usage

You can use this fine-tuned Distil BERT model for text classification in your own applications or projects. Here's a simple example of how to use the model in Python:

```python
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
import torch

# Load the pre-trained model and tokenizer
tokenizer = DistilBertTokenizer.from_pretrained("Meshwa/Distill-Bert-Automation-Command-Classification")
model = DistilBertForSequenceClassification.from_pretrained("Meshwa/Distill-Bert-Automation-Command-Classification")

# Prepare the input text
text = "Your input text goes here."

# Tokenize the text and classify it
input_ids = tokenizer(text, padding=True, truncation=True, max_length=512, return_tensors="pt")
output = model(**input_ids)
logits = outputs[0].softmax(1)
probabilities = probs.argmax()

# Get the class with the highest probability
pred_label = model.config.id2label[pred_label_idx.item()]
pred_score = probs.max().item()

# The pred_label variable will contain the predicted class label.
print(f"'label': {pred_label}, 'score': {pred_score}")
```

Remember to replace `"your_model_directory_path"` with the actual path to the fine-tuned model on your system.

## Training

I used my custom labeled dataset for finetuning this model, the dataset contained 700k samples of labled sentences, containing 342k unique sentences and some little bit of synthetic data. I used [Gretel AI](https://gretel.ai/) for generating the synthetic data.

## License

Please refer to the licenses associated with the Distil BERT Base Uncased model and any other relevant libraries or datasets you used during fine-tuning.

If you have any questions or need further assistance, please feel free to reach out. [Email Me](mailto:meshupatel2847@gmail.com)

# Model Stats after Finetuning
## After Training is complete

| Epoch | Training Loss | Validation Loss | Accuracy | F1       | Precision | Recall    |
|-------|---------------|-----------------|----------|----------|-----------|-----------|
| 1     | 0.004100      | 0.002462        | 0.999403 | 0.999114 | 0.999298  | 0.998933  |
| 2     | 0.002000      | 0.002127        | 0.999372 | 0.999031 | 0.999202  | 0.998863  |
| 3     | 0.001200      | 0.001835        | 0.999478 | 0.999172 | 0.999349  | 0.998999  |


## After Evaluation

|          | eval_loss | eval_Accuracy | eval_F1  | eval_Precision | eval_Recall |
|----------|-----------|---------------|----------|----------------|------------|
| train    | 0.000917  | 0.999567      | 0.999280 | 0.999474       | 0.999089   |
| val      | 0.001835  | 0.999478      | 0.999172 | 0.999349       | 0.998999   |
| test     | 0.002225  | 0.999369      | 0.999051 | 0.999203       | 0.998901   |


# BibTeX entry and citation info of the initial Distil-Bert-Base-Uncased
```
@article{Sanh2019DistilBERTAD,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},
  journal={ArXiv},
  year={2019},
  volume={abs/1910.01108}
}
```

## Citation

If you use the fine-tuned DistilBERT model provided in this repository, please cite it as follows:

```
@article{Meshwa2023,
  title={Fine-Tuned DistilBERT for Text Classification},
  author={Meshwa},
  journal={Hugging Face Model Hub},
  year={2023},
  url={https://huggingface.co/Meshwa/Distill-Bert-Automation-Command-Classification}
```

Meshwa. (2023). Fine-Tuned DistilBERT for Text Classification. Hugging Face Model Hub. URL: [https://huggingface.co/Meshwa/Distill-Bert-Automation-Command-Classification](https://huggingface.co/Meshwa/Distill-Bert-Automation-Command-Classification)

I appreciate your acknowledgment if you find this model useful in your work.