Spaces:
Runtime error
Runtime error
XquanL
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -22,16 +22,23 @@ pinned: false
|
|
22 |
[](https://github.com/YZhu0225/reddit_text_classification/actions/workflows/main.yml)
|
23 |
[](https://github.com/YZhu0225/reddit_text_classification/actions/workflows/sync_to_hugging_face_hub.yml)
|
24 |
|
|
|
|
|
25 |
In this project, we created a text classifier Hugging Face Spaces app and Gradio interface that classifies not safe for work (NSFW) content, specifically text that is considered inappropriate and unprofessional. We used a pre-trained DistilBERT transformer model for the sentiment analysis. The model was fine-tuned on Reddit posts and predicts 2 classes - which are NSFW and safe for work (SFW).
|
26 |
|
|
|
|
|
27 |
### Get Reddit data
|
28 |
* Data pulled in notebook `reddit_data/reddit_new.ipynb`
|
|
|
29 |
### Verify GPU works
|
30 |
* Run pytorch training test: `python utils/quickstart_pytorch.py`
|
31 |
* Run pytorch CUDA test: `python utils/verify_cuda_pytorch.py`
|
32 |
* Run tensorflow training test: `python utils/quickstart_tf2.py`
|
33 |
* Run nvidia monitoring test: `nvidia-smi -l 1`
|
34 |
|
|
|
|
|
35 |
### Finetune text classifier model and upload to Hugging Face
|
36 |
* In terminal, run `huggingface-cli login`
|
37 |
* Run `python fine_tune_berft.py` to finetune the model on Reddit data
|
@@ -39,3 +46,7 @@ In this project, we created a text classifier Hugging Face Spaces app and Gradio
|
|
39 |
* Check out the fine-tuned model [here](https://huggingface.co/michellejieli/inappropriate_text_classifier)
|
40 |
* Check out the spaces app [Spaces APP](https://huggingface.co/spaces/yjzhu0225/reddit_text_classification_app)
|
41 |
|
|
|
|
|
|
|
|
|
|
22 |
[](https://github.com/YZhu0225/reddit_text_classification/actions/workflows/main.yml)
|
23 |
[](https://github.com/YZhu0225/reddit_text_classification/actions/workflows/sync_to_hugging_face_hub.yml)
|
24 |
|
25 |
+
## Introduction
|
26 |
+
|
27 |
In this project, we created a text classifier Hugging Face Spaces app and Gradio interface that classifies not safe for work (NSFW) content, specifically text that is considered inappropriate and unprofessional. We used a pre-trained DistilBERT transformer model for the sentiment analysis. The model was fine-tuned on Reddit posts and predicts 2 classes - which are NSFW and safe for work (SFW).
|
28 |
|
29 |
+
## Workflow
|
30 |
+
|
31 |
### Get Reddit data
|
32 |
* Data pulled in notebook `reddit_data/reddit_new.ipynb`
|
33 |
+
|
34 |
### Verify GPU works
|
35 |
* Run pytorch training test: `python utils/quickstart_pytorch.py`
|
36 |
* Run pytorch CUDA test: `python utils/verify_cuda_pytorch.py`
|
37 |
* Run tensorflow training test: `python utils/quickstart_tf2.py`
|
38 |
* Run nvidia monitoring test: `nvidia-smi -l 1`
|
39 |
|
40 |
+
### DistilBERT transformer model
|
41 |
+
|
42 |
### Finetune text classifier model and upload to Hugging Face
|
43 |
* In terminal, run `huggingface-cli login`
|
44 |
* Run `python fine_tune_berft.py` to finetune the model on Reddit data
|
|
|
46 |
* Check out the fine-tuned model [here](https://huggingface.co/michellejieli/inappropriate_text_classifier)
|
47 |
* Check out the spaces app [Spaces APP](https://huggingface.co/spaces/yjzhu0225/reddit_text_classification_app)
|
48 |
|
49 |
+
### Gradio interface
|
50 |
+
* In terminal, run `python3 app.py`
|
51 |
+
* Put reddit URL in *input_url* and get output
|
52 |
+

|