widget:
- text: A family hiking in the mountains
example_title: Safe
- text: A child playing with a puppy
example_title: Safe
- text: A couple kissing passionately in bed
example_title: Nsfw
- text: A woman naked
example_title: Nsfw
- text: A man killing people
example_title: Nsfw
- text: A mass shooting
example_title: Nsfw
license: apache-2.0
language:
- en
metrics:
- f1
pipeline_tag: text-classification
tags:
- Transformers
- ' PyTorch'
- safety
- innapropriate
- distilbert
Model Card for Model ID
This model is designed to categorize text into two classes: "safe", or "nsfw" (not safe for work), which makes it suitable for content moderation and filtering applications.
The model was trained using a dataset containing 190,000 labeled text samples, distributed among the two classes of "safe" and "nsfw".
The model is based on the Distilbert-base model.
In terms of performance, the model has achieved a score of 0.97 for F1.
To improve the performance of the model, it is necessary to preprocess the input text. You can refer to the preprocess function in the app.py file in the following space: https://huggingface.co/spaces/eliasalbouzidi/distilbert-nsfw-text-classifier.
Model Description
The model can be used directly to classify text into one of the two classes. It takes in a string of text as input and outputs a probability distribution over the two classes. The class with the highest probability is selected as the predicted class.
- Developed by: Centrale Supélec Students
- Model type: 60M
- Language(s) (NLP): English
- License: apache-2.0
Training Procedure
The model was trained utilizing the Hugging Face Transformers library. The training approach involved fine-tuning the DistilBERT-base model. To optimize memory usage and accelerate training, mixed precision FP16 was used.
Training Data
The training data for finetuning the text classification model consists of a large corpus of text labeled with one of the two classes: "safe" and "nsfw". The dataset contains a total of 190,000 examples, which are distributed as follows:
100,000 examples labeled as "safe"
90,000 examples labeled as "nsfw"
The data was preprocessed for example to remove numbers, punctuation, urls ...
Uses
The model can be integrated into larger systems for content moderation or filtering.
Out-of-Scope Use
It should not be used for any illegal activities.
Bias, Risks, and Limitations
The model may exhibit biases based on the training data used. It may not perform well on text that is written in languages other than English. It may also struggle with sarcasm, irony, or other forms of figurative language. The model may produce false positives or false negatives, which could lead to incorrect categorization of text.
Recommendations
Users should be aware of the limitations and biases of the model and use it accordingly. They should also be prepared to handle false positives and false negatives. It is recommended to fine-tune the model for specific downstream tasks and to evaluate its performance on relevant datasets.
Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("eliasalbouzidi/distilbert-nsfw-text-classifier")
model = AutoModelForSequenceClassification.from_pretrained("eliasalbouzidi/distilbert-nsfw-text-classifier")
Use a pipeline
from transformers import pipeline
pipe = pipeline("text-classification", model="eliasalbouzidi/distilbert-nsfw-text-classifier")
Contact
Please reach out to eliasalbouzidi@gmail.com if you have any questions or feedback.