metadata

license: mit
language:
  - en
widget:
  - text: >-
      You have the right to use CommunityConnect for its intended purpose of
      connecting with others, sharing content responsibly, and engaging in
      constructive dialogue. You are responsible for the content you post and
      must respect the rights and privacy of others.
    example_title: Fair Clause
  - text: ' We reserve the right to suspend, terminate, or restrict your access to the platform at any time and for any reason, without prior notice or explanation. This includes but is not limited to violations of our community guidelines or terms of service, as determined solely by ConnectWorld.'
    example_title: Unfair Clause
metrics:
  - accuracy
  - precision
  - f1
  - recall
library_name: transformers
pipeline_tag: text-classification

Tos-Roberta: Terms of Service Fairness Classifier

Model Description

Tos-Roberta is a fine-tuned RoBERTa-large model designed to classify clauses in Terms of Service (ToS) documents based on their fairness level. The model categorizes clauses into three classes: clearly fair, potentially unfair, and clearly unfair.

Task

The model performs multi-class classification on individual sentences or clauses, categorizing them into three levels of unfairness:

Clearly Fair
Potentially Unfair
Clearly Unfair

Key Features

Based on the RoBERTa-large architecture
Fine-tuned on a specialized dataset of ToS clauses
Achieves high accuracy in distinguishing between fair and unfair clauses
Suitable for legal text analysis and consumer rights applications

Performance

The model demonstrates strong performance on the task of ToS clause classification:

Validation Accuracy: 89.64%
Test Accuracy: 85.84%

Detailed performance metrics per epoch:

Epoch	Training Loss	Validation Loss	Accuracy	F1 Score	Precision	Recall
1	0.443500	0.398950	0.874699	0.858838	0.862516	0.874699
2	0.416400	0.438409	0.853012	0.847317	0.849916	0.853012
3	0.227700	0.505879	0.896386	0.893325	0.891521	0.896386
4	0.052600	0.667532	0.891566	0.893167	0.895115	0.891566
5	0.124200	0.747090	0.884337	0.887412	0.891807	0.884337

Training Details

Base Model: RoBERTa-large
Dataset: CodeHima/TOS_DatasetV2
Training Time: 3310.09 seconds
Epochs: 5
Batch Size: 8
Learning Rate: Started at 2e-5 with a warmup period and decay
Optimizer: AdamW
Loss Function: Cross-Entropy
Training Strategy: Mixed precision training (FP16)

Usage

To use this model for inference:

from transformers import RobertaTokenizer, RobertaForSequenceClassification
import torch

# Load model and tokenizer
model = RobertaForSequenceClassification.from_pretrained("YourHuggingFaceUsername/Tos-Roberta")
tokenizer = RobertaTokenizer.from_pretrained("YourHuggingFaceUsername/Tos-Roberta")

# Prepare input text
text = "Your Terms of Service clause here"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)

# Make prediction
with torch.no_grad():
    outputs = model(**inputs)

probabilities = torch.softmax(outputs.logits, dim=-1)
predicted_class = torch.argmax(probabilities, dim=-1).item()

# Map prediction to label
label_map = {0: "clearly_fair", 1: "potentially_unfair", 2: "clearly_unfair"}
predicted_label = label_map[predicted_class]

print(f"Predicted class: {predicted_label}")
print(f"Probabilities: {probabilities[0].tolist()}")

Limitations and Bias

The model's performance may vary depending on the legal jurisdiction and specific domain of the ToS.
It may not capture nuanced legal interpretations that require human expertise.
The training data may contain biases present in existing ToS documents.

Ethical Considerations

While this model can assist in identifying potentially unfair clauses in ToS documents, it should not be used as a substitute for professional legal advice. The model's predictions should be reviewed by qualified legal professionals before making any decisions based on its output.

Citation

If you use this model in your research or application, please cite it as:

@misc{Tos-Roberta,
  author = {Himanshu Mohanty},
  title = {Tos-Roberta: RoBERTa-large model for Terms of Service Fairness Classification},
  year = {2024},
  publisher = {HuggingFace},
  journal = {CodeHima/Tos-Roberta},
  howpublished = {\url{https://huggingface.co/CodeHima/Tos-Roberta}}
}

Contact

For questions, feedback, or collaborations, please open an issue on the model's Hugging Face repository or contact [Your Contact Information].