layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7099
• Answer: {'precision': 0.7126948775055679, 'recall': 0.7911001236093943, 'f1': 0.7498535442296427, 'number': 809}
• Header: {'precision': 0.3793103448275862, 'recall': 0.3697478991596639, 'f1': 0.374468085106383, 'number': 119}
• Question: {'precision': 0.7863397548161121, 'recall': 0.8431924882629108, 'f1': 0.813774354327141, 'number': 1065}
• Overall Precision: 0.7338
• Overall Recall: 0.7938
• Overall F1: 0.7626
• Overall Accuracy: 0.8008

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8413	1.0	10	1.6504	{'precision': 0.011467889908256881, 'recall': 0.006180469715698393, 'f1': 0.008032128514056226, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2831168831168831, 'recall': 0.10234741784037558, 'f1': 0.15034482758620688, 'number': 1065}	0.1389	0.0572	0.0810	0.3247
1.5029	2.0	20	1.3220	{'precision': 0.1353811149032992, 'recall': 0.14709517923362175, 'f1': 0.1409952606635071, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.40875370919881304, 'recall': 0.5173708920187794, 'f1': 0.4566929133858268, 'number': 1065}	0.3009	0.3362	0.3175	0.5584
1.1608	3.0	30	1.0033	{'precision': 0.4221267454350161, 'recall': 0.4857849196538937, 'f1': 0.45172413793103444, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5333333333333333, 'recall': 0.6910798122065728, 'f1': 0.6020449897750512, 'number': 1065}	0.4871	0.5665	0.5238	0.6833
0.9008	4.0	40	0.8415	{'precision': 0.5449101796407185, 'recall': 0.6749072929542645, 'f1': 0.6029817780231916, 'number': 809}	{'precision': 0.11904761904761904, 'recall': 0.04201680672268908, 'f1': 0.06211180124223603, 'number': 119}	{'precision': 0.6373355263157895, 'recall': 0.7276995305164319, 'f1': 0.6795265234546252, 'number': 1065}	0.5867	0.6653	0.6236	0.7396
0.7085	5.0	50	0.7531	{'precision': 0.6281628162816282, 'recall': 0.7058096415327565, 'f1': 0.6647264260768335, 'number': 809}	{'precision': 0.16, 'recall': 0.10084033613445378, 'f1': 0.12371134020618556, 'number': 119}	{'precision': 0.6728110599078341, 'recall': 0.8225352112676056, 'f1': 0.7401774397972117, 'number': 1065}	0.6382	0.7321	0.6819	0.7708
0.6016	6.0	60	0.7178	{'precision': 0.6607515657620042, 'recall': 0.7824474660074165, 'f1': 0.7164685908319186, 'number': 809}	{'precision': 0.2235294117647059, 'recall': 0.15966386554621848, 'f1': 0.18627450980392157, 'number': 119}	{'precision': 0.7422145328719724, 'recall': 0.8056338028169014, 'f1': 0.7726249437190456, 'number': 1065}	0.6867	0.7577	0.7204	0.7819
0.5255	7.0	70	0.6773	{'precision': 0.6886688668866887, 'recall': 0.7737948084054388, 'f1': 0.7287543655413271, 'number': 809}	{'precision': 0.3235294117647059, 'recall': 0.2773109243697479, 'f1': 0.2986425339366516, 'number': 119}	{'precision': 0.748932536293766, 'recall': 0.8234741784037559, 'f1': 0.7844364937388193, 'number': 1065}	0.7039	0.7707	0.7358	0.7985
0.4664	8.0	80	0.6865	{'precision': 0.6846652267818575, 'recall': 0.7836835599505563, 'f1': 0.730835734870317, 'number': 809}	{'precision': 0.24299065420560748, 'recall': 0.2184873949579832, 'f1': 0.2300884955752212, 'number': 119}	{'precision': 0.7593397046046916, 'recall': 0.8206572769953052, 'f1': 0.7888086642599278, 'number': 1065}	0.7024	0.7697	0.7345	0.7950
0.4092	9.0	90	0.6843	{'precision': 0.6929046563192904, 'recall': 0.7725587144622992, 'f1': 0.7305669199298657, 'number': 809}	{'precision': 0.3050847457627119, 'recall': 0.3025210084033613, 'f1': 0.3037974683544304, 'number': 119}	{'precision': 0.7587085811384877, 'recall': 0.8384976525821596, 'f1': 0.7966101694915255, 'number': 1065}	0.7073	0.7797	0.7418	0.8013
0.4007	10.0	100	0.6826	{'precision': 0.6887921653971708, 'recall': 0.7824474660074165, 'f1': 0.7326388888888888, 'number': 809}	{'precision': 0.3142857142857143, 'recall': 0.2773109243697479, 'f1': 0.29464285714285715, 'number': 119}	{'precision': 0.7761578044596913, 'recall': 0.8497652582159625, 'f1': 0.811295383236217, 'number': 1065}	0.7174	0.7883	0.7511	0.8001
0.3396	11.0	110	0.6904	{'precision': 0.6922246220302376, 'recall': 0.792336217552534, 'f1': 0.7389048991354467, 'number': 809}	{'precision': 0.32231404958677684, 'recall': 0.3277310924369748, 'f1': 0.32499999999999996, 'number': 119}	{'precision': 0.7778745644599303, 'recall': 0.8384976525821596, 'f1': 0.8070492544057841, 'number': 1065}	0.7166	0.7893	0.7512	0.8036
0.3223	12.0	120	0.7032	{'precision': 0.7138084632516704, 'recall': 0.792336217552534, 'f1': 0.7510251903925014, 'number': 809}	{'precision': 0.3669724770642202, 'recall': 0.33613445378151263, 'f1': 0.3508771929824562, 'number': 119}	{'precision': 0.788546255506608, 'recall': 0.8403755868544601, 'f1': 0.8136363636363636, 'number': 1065}	0.7358	0.7908	0.7623	0.8012
0.3079	13.0	130	0.7098	{'precision': 0.6950431034482759, 'recall': 0.7972805933250927, 'f1': 0.7426597582037997, 'number': 809}	{'precision': 0.3652173913043478, 'recall': 0.35294117647058826, 'f1': 0.35897435897435903, 'number': 119}	{'precision': 0.7906360424028268, 'recall': 0.8403755868544601, 'f1': 0.81474738279472, 'number': 1065}	0.7274	0.7938	0.7591	0.8027
0.2866	14.0	140	0.7096	{'precision': 0.7103218645948945, 'recall': 0.7911001236093943, 'f1': 0.7485380116959064, 'number': 809}	{'precision': 0.36065573770491804, 'recall': 0.3697478991596639, 'f1': 0.36514522821576767, 'number': 119}	{'precision': 0.787719298245614, 'recall': 0.8431924882629108, 'f1': 0.8145124716553288, 'number': 1065}	0.7314	0.7938	0.7613	0.8007
0.2847	15.0	150	0.7099	{'precision': 0.7126948775055679, 'recall': 0.7911001236093943, 'f1': 0.7498535442296427, 'number': 809}	{'precision': 0.3793103448275862, 'recall': 0.3697478991596639, 'f1': 0.374468085106383, 'number': 119}	{'precision': 0.7863397548161121, 'recall': 0.8431924882629108, 'f1': 0.813774354327141, 'number': 1065}	0.7338	0.7938	0.7626	0.8008

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1