text_shortening_model_v36

This model is a fine-tuned version of facebook/bart-large-xsum on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9367
• Rouge1: 0.4747
• Rouge2: 0.2555
• Rougel: 0.4212
• Rougelsum: 0.4204
• Bert precision: 0.8696
• Bert recall: 0.87
• Average word count: 8.6396
• Max word count: 17
• Min word count: 4
• Average token count: 16.6216
• % shortened texts with length > 12: 6.3063

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.571	1.0	73	2.1904	0.4703	0.2572	0.4224	0.4216	0.8718	0.8704	8.2583	14	3	14.2973	1.5015
0.8809	2.0	146	1.9224	0.4588	0.2414	0.4177	0.417	0.8734	0.8673	8.2492	22	3	15.8078	3.9039
0.7135	3.0	219	2.7437	0.2535	0.082	0.2289	0.2294	0.8131	0.8149	8.4324	11	5	14.973	0.0
0.5646	4.0	292	2.0495	0.4689	0.249	0.4155	0.4156	0.8653	0.8736	9.8438	20	4	18.0961	21.3213
0.4158	5.0	365	2.0101	0.4707	0.2539	0.4241	0.4243	0.8688	0.8725	8.9009	14	4	14.7988	4.2042
0.3445	6.0	438	2.0642	0.4606	0.255	0.4133	0.4132	0.866	0.8705	9.0991	15	4	14.955	6.3063
0.2473	7.0	511	2.2675	0.4668	0.2441	0.4137	0.413	0.8683	0.8694	8.7177	19	3	16.5766	9.3093
0.2084	8.0	584	2.4474	0.4793	0.2608	0.4256	0.4257	0.8701	0.8741	9.1021	17	4	17.048	9.9099
0.1703	9.0	657	2.3961	0.4754	0.2609	0.4253	0.4253	0.8676	0.8749	9.2943	17	4	17.2402	11.4114
0.1293	10.0	730	2.4721	0.4581	0.2463	0.409	0.4082	0.8657	0.8671	8.7057	18	4	16.3514	6.9069
0.1312	11.0	803	2.4027	0.4667	0.2497	0.4117	0.4113	0.868	0.8683	8.4925	18	4	15.4294	5.1051
0.1424	12.0	876	2.5041	0.476	0.2506	0.4214	0.4214	0.8715	0.8699	8.6186	18	4	16.1862	6.006
0.0926	13.0	949	2.7011	0.4723	0.2582	0.4238	0.4227	0.8695	0.8701	8.6096	19	4	16.4505	5.7057
0.0663	14.0	1022	2.8149	0.467	0.2504	0.4157	0.4153	0.8674	0.8675	8.6336	17	4	16.4985	5.1051
0.0684	15.0	1095	2.9367	0.4747	0.2555	0.4212	0.4204	0.8696	0.87	8.6396	17	4	16.6216	6.3063

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3