metadata

license: apache-2.0
base_model: google/mt5-small
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: mt5-small-sport
    results: []

mt5-small-sport

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.2444
Rouge1: 22.6291
Rouge2: 9.9519
Rougel: 18.0362
Rougelsum: 19.4768
Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.3085	1.0	2897	1.6680	21.9624	8.3028	16.8626	18.4892	19.0
1.9601	2.0	5794	1.5129	22.355	8.0976	17.1001	18.4518	19.0
1.8149	3.0	8691	1.4461	21.7553	8.2693	16.8813	18.1753	19.0
1.6939	4.0	11588	1.3859	22.1485	8.5444	17.1957	18.5641	19.0
1.6661	5.0	14485	1.3600	22.1464	8.5701	17.1911	18.5834	19.0
1.5776	6.0	17382	1.3366	22.188	8.6102	17.2114	18.6171	19.0
1.5635	7.0	20279	1.3036	22.2216	8.8053	17.3255	18.7127	19.0
1.5286	8.0	23176	1.2969	22.6098	9.4618	17.7417	19.2344	19.0
1.5034	9.0	26073	1.2774	22.7211	9.8323	17.9709	19.456	19.0
1.4808	10.0	28970	1.2697	22.6057	9.778	17.9176	19.3593	19.0
1.468	11.0	31867	1.2612	22.6437	9.8167	17.9253	19.3505	19.0
1.458	12.0	34764	1.2527	22.744	10.0172	18.0459	19.5324	19.0
1.4494	13.0	37661	1.2491	22.687	9.9128	17.9941	19.4752	19.0
1.4286	14.0	40558	1.2490	22.6855	9.9731	18.0414	19.5128	19.0
1.4448	15.0	43455	1.2436	22.6476	9.9704	18.0385	19.498	19.0
1.4324	16.0	46352	1.2444	22.6291	9.9519	18.0362	19.4768	19.0

Framework versions

Transformers 4.38.1
Pytorch 2.1.0+cu121
Datasets 2.18.0
Tokenizers 0.15.2