kobart_4_5.6e-5_datav2_min30_lp5.0_temperature1.0

This model is a fine-tuned version of gogamza/kobart-base-v2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9891
• Rouge1: 35.4597
• Rouge2: 12.0824
• Rougel: 23.0161
• Bleu1: 29.793
• Bleu2: 16.882
• Bleu3: 9.6468
• Bleu4: 5.3654
• Gen Len: 50.6014

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 4
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Bleu1	Bleu2	Bleu3	Bleu4	Gen Len
2.3968	0.47	5000	2.9096	32.7469	10.9679	21.4954	27.0594	15.1133	8.4503	4.564	48.5501
2.2338	0.94	10000	2.8002	33.2148	11.5121	22.7066	26.4886	15.0125	8.5792	4.8523	41.1049
1.9652	1.42	15000	2.7699	34.4269	11.8551	22.8478	28.2628	16.0909	9.0427	4.9254	46.9744
2.001	1.89	20000	2.7201	34.157	11.8683	22.6775	28.3593	16.1361	9.221	4.8616	46.979
1.6433	2.36	25000	2.7901	33.6354	11.5761	22.6878	27.6475	15.6571	8.8372	4.8672	43.9953
1.6204	2.83	30000	2.7724	34.9611	12.1606	23.0246	29.1014	16.6689	9.3661	5.1916	48.8811
1.2955	3.3	35000	2.8970	35.896	12.7037	23.3781	29.9701	17.3963	10.2978	5.9339	49.5921
1.3501	3.78	40000	2.8854	35.2981	12.1133	23.1845	29.483	16.7795	9.4124	5.2042	48.5897
1.0865	4.25	45000	2.9912	35.581	12.5145	23.2262	29.9364	17.2064	10.0427	5.62	48.31
1.052	4.72	50000	2.9891	35.4597	12.0824	23.0161	29.793	16.882	9.6468	5.3654	50.6014

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2