File size: 2,543 Bytes
fc91e6e aa63059 fc91e6e 924827e aa63059 fc91e6e f9a0822 fc91e6e f9a0822 fc91e6e 61679dc f9a0822 fc91e6e f9a0822 fc91e6e 35192d6 fc91e6e f9a0822 35192d6 fc91e6e 924827e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
language:
- en
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
pipeline_tag: text2text-generation
base_model: facebook/bart-base
model-index:
- name: bart-base-News_Summarization_CNN
results: []
---
# bart-base-News_Summarization_CNN
This model is a fine-tuned version of [facebook/bart-base](https://huggingface.co/facebook/bart-base).
It achieves the following results on the evaluation set:
- Loss: 0.1603
## Model description
For more information on how it was created, check out the following link: https://github.com/DunnBC22/NLP_Projects/blob/main/Text%20Summarization/CNN%20News%20Text%20Summarization/CNN%20News%20Text%20Summarization.ipynb
## Intended uses & limitations
I used this to improve my skillset. I thank all of authors of the different technologies and dataset(s) for their contributions that have made this possible.
Please make sure to properly cite the authors of the different technologies and dataset(s) as they absolutely deserve credit for their contributions.
## Training and evaluation data
Dataset Source: https://www.kaggle.com/datasets/hadasu92/cnn-articles-after-basic-cleaning
## Training procedure
CPU trained on all samples where the article length is less than 820 words and the summary length is no more than 52 words in length. Additionally, any sample that was missing a new article or summarization was removed. In all, 24,911 out of the possible 42,025 samples were used for training/testing/evaluation.
Here is the link to the code that was used to train this model:
https://github.com/DunnBC22/NLP_Projects/blob/main/Text%20Summarization/CNN%20News%20Text%20Summarization.ipynb
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 16
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 2
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | RougeL | RougeLsum |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:----------:|:----------:|
| 0.7491 | 1.0 | 1089 | 0.1618 | N/A | N/A | N/A | N/A |
| 0.1641 | 2.0 | 2178 | 0.1603 | 0.834343 | 0.793822 | 0.823824 | 0.823778 |
### Framework versions
- Transformers 4.21.3
- Pytorch 1.12.1
- Datasets 2.4.0
- Tokenizers 0.12.1 |