|
--- |
|
license: mit |
|
datasets: |
|
- CodeHima/TOS_DatasetV3 |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
- precision |
|
base_model: FacebookAI/roberta-base |
|
pipeline_tag: text-classification |
|
--- |
|
# TOSRoberta-base |
|
|
|
## Model Overview |
|
|
|
**Model Name:** TOSRoberta-base |
|
**Model Type:** Sequence Classification |
|
**Base Model:** [RoBERTa-base](https://huggingface.co/roberta-base) |
|
**Language:** English |
|
**Task:** Classification of unfairness levels in Terms of Service (ToS) documents |
|
|
|
**Model Card Version:** 1.0 |
|
**Author:** CodeHima |
|
|
|
## Model Description |
|
|
|
The `TOSRoberta-base` model is a fine-tuned version of `RoBERTa-base` for classifying clauses in Terms of Service (ToS) documents into three categories: |
|
- **Clearly Fair** |
|
- **Potentially Unfair** |
|
- **Clearly Unfair** |
|
|
|
This model has been fine-tuned on a custom dataset labeled with the above categories to help identify unfair practices in ToS documents. |
|
|
|
## Intended Use |
|
|
|
### Primary Use Case |
|
The primary use case of this model is to classify text from Terms of Service documents into different levels of fairness. It can be particularly useful for legal analysts, researchers, and consumer protection agencies to quickly identify potentially unfair clauses in ToS documents. |
|
|
|
### Limitations |
|
- **Dataset Bias:** The model has been trained on a specific dataset, which may introduce biases. It may not generalize well to all types of ToS documents. |
|
- **Context Understanding:** The model may struggle with clauses that require deep contextual or legal understanding. |
|
|
|
## Performance |
|
|
|
### Training Configuration |
|
- **Batch Size:** 32 (training), 16 (evaluation) |
|
- **Learning Rate:** 1e-5 |
|
- **Epochs:** 10 |
|
- **Optimizer:** AdamW |
|
- **Scheduler:** Linear with warmup |
|
- **Training Framework:** PyTorch using Hugging Face's `transformers` library |
|
- **Mixed Precision Training:** Enabled (fp16) |
|
- **Resource:** Trained on a single NVIDIA T4 GPU (15 GB VRAM) |
|
|
|
### Training Metrics |
|
|
|
| Epoch | Training Loss | Validation Loss | Accuracy | F1 | Precision | Recall | |
|
|-------|---------------|-----------------|----------|------|-----------|--------| |
|
| 1 | 0.668100 | 0.620207 | 0.740000 | 0.727| 0.728 | 0.740 | |
|
| 2 | 0.439800 | 0.463925 | 0.824762 | 0.821| 0.826 | 0.825 | |
|
| 3 | 0.373500 | 0.432604 | 0.831429 | 0.832| 0.834 | 0.831 | |
|
| 4 | 0.342800 | 0.402661 | 0.854286 | 0.854| 0.853 | 0.854 | |
|
| 5 | 0.283800 | 0.434868 | 0.829524 | 0.832| 0.840 | 0.830 | |
|
| 6 | 0.218000 | 0.437268 | 0.859048 | 0.859| 0.859 | 0.859 | |
|
| 7 | 0.266800 | 0.508120 | 0.820952 | 0.824| 0.834 | 0.821 | |
|
| 8 | 0.139600 | 0.486364 | 0.855238 | 0.856| 0.856 | 0.855 | |
|
| 9 | 0.085000 | 0.530111 | 0.844762 | 0.846| 0.850 | 0.845 | |
|
| 10 | 0.103600 | 0.528026 | 0.842857 | 0.844| 0.847 | 0.843 | |
|
|
|
**Final Validation Accuracy:** 85.90% |
|
**Final Test Accuracy:** 85.65% |
|
|
|
### Evaluation Metrics |
|
- **Accuracy:** 85.65% |
|
- **F1 Score:** 85.60% |
|
- **Precision:** 85.61% |
|
- **Recall:** 85.65% |
|
|
|
## Dataset |
|
|
|
The model was trained on the `CodeHima/TOS_DatasetV3`, which includes labeled clauses from ToS documents. The dataset is split into training, validation, and test sets to ensure reliable performance evaluation. |
|
|
|
**Dataset Labels:** |
|
- `clearly_fair` |
|
- `potentially_unfair` |
|
- `clearly_unfair` |
|
|
|
## How to Use |
|
|
|
Here’s how you can use the model with the Hugging Face `transformers` library: |
|
|
|
```python |
|
from transformers import RobertaTokenizer, RobertaForSequenceClassification |
|
|
|
# Load the model |
|
model = RobertaForSequenceClassification.from_pretrained('CodeHima/TOSRoberta-base') |
|
tokenizer = RobertaTokenizer.from_pretrained('CodeHima/TOSRoberta-base') |
|
|
|
# Predict the unfairness level of a clause |
|
text = "Insert clause text here." |
|
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128) |
|
outputs = model(**inputs) |
|
predicted_class = outputs.logits.argmax(-1).item() |
|
|
|
# Map the predicted class to the corresponding label |
|
label_mapping = {0: 'clearly_fair', 1: 'potentially_unfair', 2: 'clearly_unfair'} |
|
predicted_label = label_mapping[predicted_class] |
|
print(f"Predicted Label: {predicted_label}") |
|
``` |
|
|
|
## Ethical Considerations |
|
|
|
- **Bias:** The model's predictions may reflect biases present in the training data. |
|
- **Fair Use:** Ensure the model is used responsibly, especially in legal contexts where human oversight is critical. |
|
|
|
## Conclusion |
|
|
|
The `TOSRoberta-base` model is a reliable tool for identifying unfair clauses in Terms of Service documents. While it performs well, it should be used in conjunction with expert analysis, particularly in legally sensitive contexts. |
|
|
|
**Model Repository:** [CodeHima/TOSRoberta-base](https://huggingface.co/CodeHima/TOSRoberta-base) |
