File size: 2,915 Bytes
b7636e1 fd285c1 b7636e1 fd285c1 b7636e1 fd285c1 b7636e1 fd285c1 b7636e1 fd285c1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
library_name: transformers
license: apache-2.0
base_model: t5-small
tags:
- generated_from_trainer
metrics:
- rouge
- accuracy
model-index:
- name: T5_Fine_Tuned_on_Arxiv_Dataset
results: []
datasets:
- ccdv/arxiv-summarization
language:
- en
---
# T5_Fine_Tuned_on_Arxiv_Dataset
## Model Description
This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) designed for summarizing research papers from the Arxiv dataset. It utilizes an abstractive summarization approach to generate concise summaries that capture the main findings and contributions of the papers, facilitating easier understanding of complex academic content.
## Evaluation
It achieves the following results on the evaluation set:
- Loss: 2.7599
- Rouge1: 0.1635
- Rouge2: 0.0548
- Rougel: 0.1311
- Rougelsum: 0.1311
- Generated Length: 18.9852
## Model Overview
- **Model Name**: Arxiv Summarization Model
- **Model Type**: Summarization (Abstractive)
- **Version**: 1.0
- **Date**: [28-Sep-2024]
- **Authors**: Muhammad Ibtisam Afzal
- **Contact Information**: [email protected]
## Dataset
- **Dataset Name**: ccdv/arxiv-summarization
- **Dataset Description**: This dataset consists of articles from the Arxiv repository, paired with their respective abstracts. It is intended for training and evaluating summarization models in the academic domain.
- **Training/Validation/Test Split**: The dataset was split into training (80%), validation (10%), and test (10%) sets.
- **Data Source**: Hugging Face Datasets Hub
## Limitations
The model may struggle with highly technical content or specialized jargon that is not well-represented in the training dataset. Additionally, it may produce summaries that lack coherence or completeness for particularly long documents.
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Generated Length |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:----------------:|
| No log | 1.0 | 305 | 2.8130 | 0.1569 | 0.05 | 0.1256 | 0.1255 | 18.9852 |
| 3.0803 | 2.0 | 610 | 2.7704 | 0.1634 | 0.0546 | 0.1312 | 0.1311 | 18.9852 |
| 3.0803 | 3.0 | 915 | 2.7599 | 0.1635 | 0.0548 | 0.1311 | 0.1311 | 18.9852 |
### Framework versions
- Transformers 4.44.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.19.1
## Acknowledgments
Thanks to Hugging Face for providing the infrastructure and datasets necessary for developing and evaluating this model.
|