layout-lm

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6748
• Answer: {'precision': 0.7245575221238938, 'recall': 0.8096415327564895, 'f1': 0.7647402218330415, 'number': 809}
• Header: {'precision': 0.3464566929133858, 'recall': 0.3697478991596639, 'f1': 0.35772357723577236, 'number': 119}
• Question: {'precision': 0.7756183745583038, 'recall': 0.8244131455399061, 'f1': 0.7992717341829768, 'number': 1065}
• Overall Precision: 0.7291
• Overall Recall: 0.7913
• Overall F1: 0.7589
• Overall Accuracy: 0.8136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7656	1.0	10	1.5482	{'precision': 0.030390738060781478, 'recall': 0.02595797280593325, 'f1': 0.028, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.31988041853512705, 'recall': 0.20093896713615023, 'f1': 0.24682814302191464, 'number': 1065}	0.1728	0.1179	0.1402	0.3707
1.4041	2.0	20	1.1664	{'precision': 0.15247252747252749, 'recall': 0.13720642768850433, 'f1': 0.14443721535458687, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.46790409899458624, 'recall': 0.568075117370892, 'f1': 0.5131467345207804, 'number': 1065}	0.3539	0.3593	0.3566	0.6164
1.0549	3.0	30	0.8895	{'precision': 0.521044992743106, 'recall': 0.4437577255871446, 'f1': 0.479305740987984, 'number': 809}	{'precision': 0.25, 'recall': 0.08403361344537816, 'f1': 0.12578616352201258, 'number': 119}	{'precision': 0.5932336742722266, 'recall': 0.707981220657277, 'f1': 0.6455479452054794, 'number': 1065}	0.5615	0.5635	0.5625	0.7226
0.8144	4.0	40	0.7445	{'precision': 0.621978021978022, 'recall': 0.6996291718170581, 'f1': 0.658522396742292, 'number': 809}	{'precision': 0.2753623188405797, 'recall': 0.15966386554621848, 'f1': 0.20212765957446807, 'number': 119}	{'precision': 0.6641477749790092, 'recall': 0.7427230046948357, 'f1': 0.7012411347517731, 'number': 1065}	0.6341	0.6904	0.6611	0.7620
0.6601	5.0	50	0.6786	{'precision': 0.6608505997818975, 'recall': 0.7490729295426453, 'f1': 0.7022016222479722, 'number': 809}	{'precision': 0.34615384615384615, 'recall': 0.226890756302521, 'f1': 0.27411167512690354, 'number': 119}	{'precision': 0.6853932584269663, 'recall': 0.8018779342723005, 'f1': 0.7390739939420164, 'number': 1065}	0.6635	0.7461	0.7024	0.7912
0.558	6.0	60	0.6751	{'precision': 0.6495375128468653, 'recall': 0.7812113720642769, 'f1': 0.7093153759820426, 'number': 809}	{'precision': 0.34615384615384615, 'recall': 0.226890756302521, 'f1': 0.27411167512690354, 'number': 119}	{'precision': 0.7348017621145374, 'recall': 0.7830985915492957, 'f1': 0.7581818181818181, 'number': 1065}	0.6830	0.7491	0.7145	0.7873
0.4876	7.0	70	0.6439	{'precision': 0.6867469879518072, 'recall': 0.7750309023485785, 'f1': 0.7282229965156795, 'number': 809}	{'precision': 0.2672413793103448, 'recall': 0.2605042016806723, 'f1': 0.26382978723404255, 'number': 119}	{'precision': 0.735144312393888, 'recall': 0.8131455399061033, 'f1': 0.7721801159161837, 'number': 1065}	0.6905	0.7647	0.7257	0.8059
0.431	8.0	80	0.6333	{'precision': 0.7019650655021834, 'recall': 0.7948084054388134, 'f1': 0.7455072463768115, 'number': 809}	{'precision': 0.3157894736842105, 'recall': 0.3025210084033613, 'f1': 0.30901287553648066, 'number': 119}	{'precision': 0.7440878378378378, 'recall': 0.8272300469483568, 'f1': 0.7834593152512227, 'number': 1065}	0.7046	0.7827	0.7416	0.8119
0.3849	9.0	90	0.6338	{'precision': 0.713495575221239, 'recall': 0.7972805933250927, 'f1': 0.7530647985989491, 'number': 809}	{'precision': 0.3465346534653465, 'recall': 0.29411764705882354, 'f1': 0.3181818181818182, 'number': 119}	{'precision': 0.7697022767075307, 'recall': 0.8253521126760563, 'f1': 0.7965564114182148, 'number': 1065}	0.7261	0.7822	0.7531	0.8189
0.3741	10.0	100	0.6533	{'precision': 0.7054429028815368, 'recall': 0.8170580964153276, 'f1': 0.7571592210767468, 'number': 809}	{'precision': 0.31092436974789917, 'recall': 0.31092436974789917, 'f1': 0.31092436974789917, 'number': 119}	{'precision': 0.7736185383244206, 'recall': 0.8150234741784037, 'f1': 0.7937814357567444, 'number': 1065}	0.7190	0.7858	0.7509	0.8133
0.3184	11.0	110	0.6556	{'precision': 0.7065803667745415, 'recall': 0.8096415327564895, 'f1': 0.7546082949308756, 'number': 809}	{'precision': 0.3203125, 'recall': 0.3445378151260504, 'f1': 0.33198380566801616, 'number': 119}	{'precision': 0.7630901287553649, 'recall': 0.8347417840375587, 'f1': 0.7973094170403587, 'number': 1065}	0.7140	0.7953	0.7524	0.8104
0.3038	12.0	120	0.6681	{'precision': 0.72271714922049, 'recall': 0.8022249690976514, 'f1': 0.7603983596953721, 'number': 809}	{'precision': 0.3305084745762712, 'recall': 0.3277310924369748, 'f1': 0.32911392405063294, 'number': 119}	{'precision': 0.7851387645478961, 'recall': 0.8234741784037559, 'f1': 0.8038496791934006, 'number': 1065}	0.7337	0.7852	0.7586	0.8155
0.2922	13.0	130	0.6667	{'precision': 0.7233809001097695, 'recall': 0.8145859085290482, 'f1': 0.7662790697674419, 'number': 809}	{'precision': 0.36036036036036034, 'recall': 0.33613445378151263, 'f1': 0.34782608695652173, 'number': 119}	{'precision': 0.7810599478714162, 'recall': 0.844131455399061, 'f1': 0.8113718411552348, 'number': 1065}	0.7354	0.8018	0.7672	0.8150
0.2685	14.0	140	0.6738	{'precision': 0.7296996662958843, 'recall': 0.8108776266996292, 'f1': 0.7681498829039812, 'number': 809}	{'precision': 0.3384615384615385, 'recall': 0.3697478991596639, 'f1': 0.35341365461847385, 'number': 119}	{'precision': 0.7788546255506608, 'recall': 0.8300469483568075, 'f1': 0.8036363636363637, 'number': 1065}	0.7320	0.7948	0.7621	0.8131
0.2668	15.0	150	0.6748	{'precision': 0.7245575221238938, 'recall': 0.8096415327564895, 'f1': 0.7647402218330415, 'number': 809}	{'precision': 0.3464566929133858, 'recall': 0.3697478991596639, 'f1': 0.35772357723577236, 'number': 119}	{'precision': 0.7756183745583038, 'recall': 0.8244131455399061, 'f1': 0.7992717341829768, 'number': 1065}	0.7291	0.7913	0.7589	0.8136

Framework versions

• Transformers 4.43.3
• Pytorch 2.4.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1