File size: 2,984 Bytes
7da1b58 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
# T5-Small for News Headline Generation
This is a T5-Small model fine-tuned for generating concise and informative news topics from content summaries. It is useful for news agencies, content creators, and media professionals to generate headlines efficiently.
# Model Details
**Model Type:** Sequence-to-Sequence Transformer
**Base Model:** t5-small
**Maximum Sequence Length:** 128 tokens (input and output)
**Output:** News headlines based on input summaries
**Task:** Text Summarization (Headline Generation)
# Model Sources
**Documentation:** T5 Model Documentation
**Repository:** Hugging Face Model Hub
**Hugging Face Model:** Available on Hugging Face
# Full Model Architecture
```
T5ForConditionalGeneration(
(shared): Embedding(32128, 512)
(encoder): T5Stack(
(embed_tokens): Embedding(32128, 512)
(block): ModuleList(...)
(final_layer_norm): LayerNorm((512,), eps=1e-12)
(dropout): Dropout(p=0.1)
)
(decoder): T5Stack(
(embed_tokens): Embedding(32128, 512)
(block): ModuleList(...)
(final_layer_norm): LayerNorm((512,), eps=1e-12)
(dropout): Dropout(p=0.1)
)
(lm_head): Linear(in_features=512, out_features=32128, bias=False)
)
```
# Installation and Setup
```bash
pip install -U transformers torch datasets
```
# Load the Model and Run Inference
```python
from transformers import T5ForConditionalGeneration, T5Tokenizer
import torch
# Model Name
model_name = "your_fine_tuned_model_id"
tokenizer = T5Tokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)
# Move model to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
# Inference
news_summary = "Ministry of Education has announced a major reform in the national curriculum to enhance digital literacy among students."
inputs = tokenizer(news_summary, max_length=128, truncation=True, padding="max_length", return_tensors="pt").to(device)
outputs = model.generate(
input_ids=inputs["input_ids"],
attention_mask=inputs["attention_mask"],
max_length=20,
num_beams=5,
early_stopping=True
)
headline = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Generated Headline: {headline}")
```
# Training Details
### Training Dataset
**Dataset Name:** News Headlines Dataset
**Size:** 30,000 rows
**Columns:** article_summary (input), headline (output)
# Approximate Statistics
```
article_summary:
Type: string
Min length: ~20 tokens
Mean length: ~50-60 tokens (estimated)
Max length: ~128 tokens
headline:
Type: string
Min length: ~5 tokens
Mean length: ~10-15 tokens
Max length: ~20 tokens
```
# Training Hyperparameters
- **per_device_train_batch_size:** 8
- **per_device_eval_batch_size:** 8
- **gradient_accumulation_steps:** 2
- **num_train_epochs:** 4
- **learning_rate:** 5e-5
- **fp16:** True
This model is optimized for **content topic generation**, ensuring concise, accurate, and informative outputs. 🚀
|