File size: 3,555 Bytes
4ecc869 59932e0 4ecc869 59932e0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 |
---
license: mit
language:
- en
pipeline_tag: sentence-similarity
datasets:
- darrow-ai/LegalLensNLI
metrics:
- f1
base_model:
- ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli
library_name: transformers
---
# roberta_cnn_legal
## Overview
This repository hosts the uOttawa model developed for Subtask B (Legal Natural Language Inference) in the LegalLens-2024 shared task. The task focuses on classifying relationships between legal texts, such as determining if a premise (e.g., a summary of a legal complaint) entails, contradicts, or is neutral with respect to a hypothesis (e.g., an online review).
## Model Details
- **Model Type**: Transformer-based model combined with a Convolutional Neural Network (CNN)
- **Framework**: PyTorch, Transformers library
- **Training Data**: LegalLensNLI dataset provided by the LegalLens-2024 organizers
- **Architecture**: Integration of RoBERTa (ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli) with a custom CNN for keyword pattern detection
- **Use Case**: Classifying relationships between legal documents for applications like legal case matching and automated reasoning
## Model Architecture
The model architecture consists of:
- **RoBERTa model**: Responsible for capturing contextual information from the input text.
- **CNN model**: Used for keyword detection, including an embedding layer and three convolutional layers with filter sizes (2, 3, 4).
- **Fully connected layer**: Combines the outputs from RoBERTa and CNN for the final classification.
## Installation
To use this model, clone this repository and make sure to have the following installed:
```bash
pip install torch
pip install transformers
```
## Quick Start
Load the model and run inference using the Hugging Face Transformers library:
```code
from transformers import AutoTokenizer, AutoModelForSequenceClassification
# Load the model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("nimamegh/roberta_cnn_legal")
tokenizer = AutoTokenizer.from_pretrained("nimamegh/roberta_cnn_legal")
# Example inputs
premise = "The cat is on the mat."
hypothesis = "The animal is on the mat."
inputs = tokenizer(premise, hypothesis, return_tensors='pt')
# Get predictions
outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=-1)
# Print the prediction result
print("Predicted class:", predictions.item())
# Interpretation (optional)
label_map = {0: "Entailment", 1: "Neutral", 2: "Contradiction"}
print("Result:", label_map[predictions.item()])
```
## Training Configuration
- Learning Rate: 2e-5
- Batch Size: 4 (train and evaluation)
- Number of Epochs: 20
- Weight Decay: 0.01
- Optimizer: AdamW
- Trainer Class: Used for fine-tuning with early stopping and warmup steps
## Evaluation Metrics
The model was evaluated using an F1-score across multiple domains in the validation set:
- Average F1-score: 88.6%
## Result
- Performance on Hidden Test Set: F1-score of 0.724, achieving 5th place in the LegalLens-2024 competition.
- Comparison:
- Falcon 7B: 81.02% (average across domains)
- RoBERTa base: 71.02% (average)
- uOttawa Model: 88.6% (average on validation)
## Citation
```bibtex
@misc{meghdadi2024uottawalegallens2024transformerbasedclassification,
title={uOttawa at LegalLens-2024: Transformer-based Classification Experiments},
author={Nima Meghdadi and Diana Inkpen},
year={2024},
eprint={2410.21139},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2410.21139},
}
``` |