layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7211
• Answer: {'precision': 0.7268722466960352, 'recall': 0.8158220024721878, 'f1': 0.7687827606290041, 'number': 809}
• Header: {'precision': 0.31343283582089554, 'recall': 0.35294117647058826, 'f1': 0.3320158102766798, 'number': 119}
• Question: {'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065}
• Overall Precision: 0.7332
• Overall Recall: 0.8013
• Overall F1: 0.7658
• Overall Accuracy: 0.7963

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7901	1.0	10	1.5938	{'precision': 0.017361111111111112, 'recall': 0.012360939431396786, 'f1': 0.014440433212996389, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.32525252525252524, 'recall': 0.1511737089201878, 'f1': 0.20641025641025643, 'number': 1065}	0.1597	0.0858	0.1116	0.3415
1.4447	2.0	20	1.2469	{'precision': 0.22497522299306244, 'recall': 0.28059332509270707, 'f1': 0.24972497249724973, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4132890365448505, 'recall': 0.584037558685446, 'f1': 0.4840466926070039, 'number': 1065}	0.3377	0.4260	0.3767	0.5933
1.0816	3.0	30	0.9331	{'precision': 0.5004995004995005, 'recall': 0.619283065512979, 'f1': 0.5535911602209945, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5584415584415584, 'recall': 0.7267605633802817, 'f1': 0.631578947368421, 'number': 1065}	0.5260	0.6397	0.5773	0.7095
0.8262	4.0	40	0.7964	{'precision': 0.5772357723577236, 'recall': 0.7021013597033374, 'f1': 0.633575013943112, 'number': 809}	{'precision': 0.15625, 'recall': 0.08403361344537816, 'f1': 0.10928961748633881, 'number': 119}	{'precision': 0.6492659053833605, 'recall': 0.7474178403755869, 'f1': 0.6948930597992143, 'number': 1065}	0.6042	0.6894	0.6440	0.7450
0.6674	5.0	50	0.7441	{'precision': 0.6445182724252492, 'recall': 0.7194066749072929, 'f1': 0.6799065420560747, 'number': 809}	{'precision': 0.22105263157894736, 'recall': 0.17647058823529413, 'f1': 0.19626168224299065, 'number': 119}	{'precision': 0.6424870466321243, 'recall': 0.8150234741784037, 'f1': 0.7185430463576157, 'number': 1065}	0.6262	0.7381	0.6776	0.7673
0.5736	6.0	60	0.7005	{'precision': 0.6451942740286298, 'recall': 0.7799752781211372, 'f1': 0.7062115277000558, 'number': 809}	{'precision': 0.20454545454545456, 'recall': 0.15126050420168066, 'f1': 0.17391304347826086, 'number': 119}	{'precision': 0.7412891986062717, 'recall': 0.7990610328638498, 'f1': 0.7690917306823317, 'number': 1065}	0.6775	0.7526	0.7131	0.7755
0.5042	7.0	70	0.6801	{'precision': 0.6768743400211193, 'recall': 0.792336217552534, 'f1': 0.7300683371298405, 'number': 809}	{'precision': 0.22018348623853212, 'recall': 0.20168067226890757, 'f1': 0.21052631578947367, 'number': 119}	{'precision': 0.7412765957446809, 'recall': 0.8178403755868544, 'f1': 0.7776785714285714, 'number': 1065}	0.6885	0.7707	0.7273	0.7841
0.4479	8.0	80	0.6712	{'precision': 0.6687565308254964, 'recall': 0.7911001236093943, 'f1': 0.7248018120045301, 'number': 809}	{'precision': 0.20610687022900764, 'recall': 0.226890756302521, 'f1': 0.21600000000000003, 'number': 119}	{'precision': 0.7404006677796328, 'recall': 0.8328638497652582, 'f1': 0.7839151568714097, 'number': 1065}	0.6798	0.7797	0.7263	0.7900
0.3931	9.0	90	0.6806	{'precision': 0.7054263565891473, 'recall': 0.7873918417799752, 'f1': 0.7441588785046728, 'number': 809}	{'precision': 0.2809917355371901, 'recall': 0.2857142857142857, 'f1': 0.2833333333333333, 'number': 119}	{'precision': 0.7510620220900595, 'recall': 0.8300469483568075, 'f1': 0.7885816235504014, 'number': 1065}	0.7065	0.7802	0.7415	0.7955
0.3875	10.0	100	0.6819	{'precision': 0.7014767932489452, 'recall': 0.8220024721878862, 'f1': 0.7569721115537849, 'number': 809}	{'precision': 0.3, 'recall': 0.3025210084033613, 'f1': 0.301255230125523, 'number': 119}	{'precision': 0.7633851468048359, 'recall': 0.8300469483568075, 'f1': 0.7953216374269007, 'number': 1065}	0.7120	0.7953	0.7514	0.7952
0.3309	11.0	110	0.7016	{'precision': 0.7204419889502762, 'recall': 0.8059332509270705, 'f1': 0.7607934655775962, 'number': 809}	{'precision': 0.2949640287769784, 'recall': 0.3445378151260504, 'f1': 0.31782945736434104, 'number': 119}	{'precision': 0.7535864978902953, 'recall': 0.8384976525821596, 'f1': 0.7937777777777778, 'number': 1065}	0.7115	0.7958	0.7513	0.7969
0.3142	12.0	120	0.7081	{'precision': 0.7178924259055982, 'recall': 0.8084054388133498, 'f1': 0.7604651162790698, 'number': 809}	{'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119}	{'precision': 0.7768014059753954, 'recall': 0.8300469483568075, 'f1': 0.8025419881979118, 'number': 1065}	0.7245	0.7918	0.7567	0.7993
0.2992	13.0	130	0.7160	{'precision': 0.716304347826087, 'recall': 0.8145859085290482, 'f1': 0.7622903412377097, 'number': 809}	{'precision': 0.304, 'recall': 0.31932773109243695, 'f1': 0.31147540983606553, 'number': 119}	{'precision': 0.7796167247386759, 'recall': 0.8403755868544601, 'f1': 0.8088567555354722, 'number': 1065}	0.7259	0.7988	0.7606	0.7938
0.2746	14.0	140	0.7194	{'precision': 0.7238723872387238, 'recall': 0.8133498145859085, 'f1': 0.7660069848661233, 'number': 809}	{'precision': 0.32061068702290074, 'recall': 0.35294117647058826, 'f1': 0.336, 'number': 119}	{'precision': 0.7859030837004405, 'recall': 0.8375586854460094, 'f1': 0.8109090909090909, 'number': 1065}	0.7320	0.7988	0.7639	0.7957
0.2735	15.0	150	0.7211	{'precision': 0.7268722466960352, 'recall': 0.8158220024721878, 'f1': 0.7687827606290041, 'number': 809}	{'precision': 0.31343283582089554, 'recall': 0.35294117647058826, 'f1': 0.3320158102766798, 'number': 119}	{'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065}	0.7332	0.8013	0.7658	0.7963

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1