File size: 4,844 Bytes
01b69fc ca41227 01b69fc |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
---
language:
- it
tags:
- text2text-generation
- summarization
- legal-ai
- italian-law
license: mit
datasets:
- joelniklaus/Multi_Legal_Pile
library_name: transformers
pipeline_tag: text2text-generation
widget:
- text: "<mask> 1234: Il contratto si intende concluso quando..."
base_model:
- morenolq/bart-it
---
# 📌 Model Card: LEGIT-BART Series
## 🏛️ Model Overview
The **LEGIT-BART** models are a family of **pre-trained transformer-based models** for **Italian legal text processing**.
They build upon **BART-IT** ([`morenolq/bart-it`](https://huggingface.co/morenolq/bart-it)) and are further pre-trained on **Italian legal corpora**.
💡 Key features:
- **Extended context length** with **Local-Sparse-Global (LSG) Attention** (up to **16,384 tokens**) 📜
- **Trained on legal documents** such as **statutes, case law, and contracts** 📑
- **Not fine-tuned for specific tasks** (requires further adaptation)
## 📂 Available Models
| Model | Description | Link |
|--------|-------------|------|
| **LEGIT-BART** | Continued pre-training of `morenolq/bart-it` on **Italian legal texts** | [🔗 Link](https://huggingface.co/morenolq/LEGIT-BART) |
| **LEGIT-BART-LSG-4096** | Continued pre-training of `morenolq/bart-it`, supporting **4,096 tokens** | [🔗 Link](https://huggingface.co/morenolq/LEGIT-BART-LSG-4096) |
| **LEGIT-BART-LSG-16384** | Continued pre-training of `morenolq/bart-it`, supporting **16,384 tokens** | [🔗 Link](https://huggingface.co/morenolq/LEGIT-BART-LSG-16384) |
| **LEGIT-SCRATCH-BART** | Trained from scratch on **Italian legal texts** | [🔗 Link](https://huggingface.co/morenolq/LEGIT-SCRATCH-BART) |
| **LEGIT-SCRATCH-BART-LSG-4096** | Trained from scratch with **LSG attention**, supporting **4,096 tokens** | [🔗 Link](https://huggingface.co/morenolq/LEGIT-SCRATCH-BART-LSG-4096) |
| **LEGIT-SCRATCH-BART-LSG-16384** | Trained from scratch with **LSG attention**, supporting **16,384 tokens** | [🔗 Link](https://huggingface.co/morenolq/LEGIT-SCRATCH-BART-LSG-16384) |
| **BART-IT-LSG-4096** | `morenolq/bart-it` with **LSG attention**, supporting **4,096 tokens** (no legal adaptation) | [🔗 Link](https://huggingface.co/morenolq/BART-IT-LSG-4096)
| **BART-IT-LSG-16384** | `morenolq/bart-it` with **LSG attention**, supporting **16,384 tokens** (no legal adaptation) | [🔗 Link](https://huggingface.co/morenolq/BART-IT-LSG-16384) |
---
## 🛠️ Model Details
🔹 **Architecture**
- Base Model: [`morenolq/bart-it`](https://huggingface.co/morenolq/bart-it)
- Transformer Encoder-Decoder
- **LSG Attention** for long documents
- Specific tokenizers for models trained from scratch (underperforming continual pre-training in our experiments).
🔹 **Training Data**
- Dataset: [`joelniklaus/Multi_Legal_Pile`](https://huggingface.co/datasets/joelniklaus/Multi_Legal_Pile)
- Types of legal texts used:
- **Legislation** (laws, codes, amendments)
- **Case law** (judicial decisions)
- **Contracts** (public legal agreements)
---
## 🚀 How to Use
```python
from transformers import BartForConditionalGeneration, AutoTokenizer
# Load tokenizer and model
model_name = "morenolq/LEGIT-SCRATCH-BART-LSG-4096"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = BartForConditionalGeneration.from_pretrained(model_name)
# Example input
input_text = "<mask> 1234: Il contratto si intende concluso quando..."
inputs = tokenizer(input_text, return_tensors="pt", max_length=4096, truncation=True)
# Generate summary
summary_ids = model.generate(inputs.input_ids, max_length=150, num_beams=4, early_stopping=True)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
print("📝 Summary:", summary)
```
---
⚠️ Limitations & Ethical Considerations
- **Not fine-tuned for specific tasks**: The models are pre-trained on legal texts and may require further adaptation for specific legal NLP tasks (e.g., summarization, question-answering).
- **Bias and fairness**: Legal texts may contain biases present in the legal system. Care should be taken to ensure fairness and ethical use of the models.
- **Legal advice**: The models are not a substitute for professional legal advice. Always consult a qualified legal professional for legal matters.
---
## 📚 Reference
The paper presenting LEGIT-BART models is currently under review and will be updated here once published.
```bibtex
@article{benedetto2025legitbart,
title = {LegItBART: a summarization model for Italian legal documents},
author = {Benedetto, Irene and La Quatra, Moreno and Cagliero, Luca},
year = 2025,
journal = {Artificial Intelligence and Law},
publisher = {Springer},
pages = {1--31},
doi = {10.1007/s10506-025-09436-y},
url = {doi.org/10.1007/s10506-025-09436-y}
}
```
--- |