File size: 3,960 Bytes
34195cf b00fcda 34195cf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
# Grammar Correction with Text-to-Text Transfer Transformer
## 📌 Overview
This repository hosts the quantized version of the T5 model fine-tuned for Grammar Correction. The model has been trained on the JFLEG dataset from Hugging Face to enhance grammatical accuracy in given text inputs. The model is quantized to Float16 (FP16) to optimize inference speed and efficiency while maintaining high performance.
## 🏗 Model Details
- **Model Architecture:** t5-base
- **Task:** Grammar Correction
- **Dataset:** Hugging Face's `jfleg`
- **Quantization:** Float16 (FP16) for optimized inference
- **Fine-tuning Framework:** Hugging Face Transformers
## 🚀 Usage
### Installation
```bash
pip install transformers torch
```
### Loading the Model
```python
from transformers import T5Tokenizer, T5ForConditionalGeneration
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
model_name = "AventIQ-AI/t5-grammar-correction"
model = T5ForConditionalGeneration.from_pretrained(model_name).to(device)
tokenizer = T5Tokenizer.from_pretrained(model_name)
```
### Grammar Correction Inference
```python
def correct_grammar(text, model, tokenizer, device):
prefix = "correct grammar: "
input_text = prefix + text
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(device)
outputs = model.generate(
input_ids,
max_length=128,
num_beams=5,
early_stopping=True,
)
corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
return corrected_text
# 🔍 Test Example
test_sentences = [
"He go to the store yesterday.",
"They was running in the park.",
"She dont like pizza.",
"We has completed the project already.",
]
for sentence in test_sentences:
corrected = correct_grammar(sentence, model, tokenizer, device)
print(f"Original: {sentence}")
print(f"Corrected: {corrected}")
print("---")
```
## 📊 Evaluation Metric: BLEU Score
For grammar correction, a high BLEU score indicates that the model’s corrected sentences closely match human-annotated corrections.
## **Interpreting Our BLEU Score**
Our model achieved a **BLEU score of 0.8888**, which indicates:
✅ **Good grammar correction ability**
✅ **Moderate sentence fluency**
BLEU is computed by comparing the **1-gram, 2-gram, 3-gram, and 4-gram overlaps** between the model’s output and the reference sentence while applying a **brevity penalty** if the model generates shorter sentences.
### **BLEU Score Ranges for Grammar Correction**
| BLEU Score | Interpretation |
| --- | --- |
| **0.8 - 1.0** | Near-perfect corrections, closely matching human annotations. |
| **0.7 - 0.8** | High-quality corrections, minor variations in phrasing. |
| **0.6 - 0.7** | Good corrections, but with some grammatical errors or missing words. ✅ _(Our Model)_ |
| **0.5 - 0.6** | Decent corrections, noticeable mistakes, lacks fluency. |
| **Below 0.5** | Needs improvement, frequent incorrect corrections. |
## ⚡ Quantization Details
Post-training quantization was applied using PyTorch's built-in quantization framework. The model was quantized to Float16 (FP16) to reduce model size and improve inference efficiency while balancing accuracy.
## 📂 Repository Structure
```
.
├── model/ # Contains the quantized model files
├── tokenizer_config/ # Tokenizer configuration and vocabulary files
├── model.safetensors/ # Quantized Model
├── README.md # Model documentation
```
## ⚠️ Limitations
- The model may struggle with highly ambiguous sentences.
- Quantization may lead to slight degradation in accuracy compared to full-precision models.
- Performance may vary across different writing styles and sentence structures.
## 🤝 Contributing
Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.
|