layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7080
• Answer: {'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809}
• Header: {'precision': 0.3359375, 'recall': 0.36134453781512604, 'f1': 0.3481781376518218, 'number': 119}
• Question: {'precision': 0.7817531305903399, 'recall': 0.8206572769953052, 'f1': 0.8007329363261567, 'number': 1065}
• Overall Precision: 0.7260
• Overall Recall: 0.7842
• Overall F1: 0.7540
• Overall Accuracy: 0.8073

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.4164	1.0	10	1.1867	{'precision': 0.21566110397946084, 'recall': 0.207663782447466, 'f1': 0.21158690176322417, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.48124557678697805, 'recall': 0.6384976525821596, 'f1': 0.5488297013720743, 'number': 1065}	0.3869	0.4255	0.4053	0.6139
1.0235	2.0	20	0.8815	{'precision': 0.578494623655914, 'recall': 0.6650185414091471, 'f1': 0.6187464059804485, 'number': 809}	{'precision': 0.05555555555555555, 'recall': 0.008403361344537815, 'f1': 0.014598540145985401, 'number': 119}	{'precision': 0.6398687448728466, 'recall': 0.7323943661971831, 'f1': 0.6830122591943958, 'number': 1065}	0.6087	0.6618	0.6341	0.7403
0.7822	3.0	30	0.7564	{'precision': 0.6335403726708074, 'recall': 0.7564894932014833, 'f1': 0.6895774647887324, 'number': 809}	{'precision': 0.13559322033898305, 'recall': 0.06722689075630252, 'f1': 0.0898876404494382, 'number': 119}	{'precision': 0.6905158069883528, 'recall': 0.7793427230046949, 'f1': 0.7322452580502868, 'number': 1065}	0.6511	0.7275	0.6872	0.7697
0.6495	4.0	40	0.6955	{'precision': 0.6533333333333333, 'recall': 0.7873918417799752, 'f1': 0.7141255605381165, 'number': 809}	{'precision': 0.19480519480519481, 'recall': 0.12605042016806722, 'f1': 0.15306122448979592, 'number': 119}	{'precision': 0.7162276975361087, 'recall': 0.7915492957746478, 'f1': 0.752007136485281, 'number': 1065}	0.6707	0.7501	0.7082	0.7915
0.5641	5.0	50	0.6796	{'precision': 0.6843267108167771, 'recall': 0.7663782447466008, 'f1': 0.7230320699708457, 'number': 809}	{'precision': 0.275, 'recall': 0.18487394957983194, 'f1': 0.22110552763819097, 'number': 119}	{'precision': 0.7565217391304347, 'recall': 0.8169014084507042, 'f1': 0.7855530474040633, 'number': 1065}	0.7079	0.7587	0.7324	0.7899
0.4862	6.0	60	0.6563	{'precision': 0.6844978165938864, 'recall': 0.7750309023485785, 'f1': 0.7269565217391305, 'number': 809}	{'precision': 0.28, 'recall': 0.23529411764705882, 'f1': 0.2557077625570776, 'number': 119}	{'precision': 0.7420168067226891, 'recall': 0.8291079812206573, 'f1': 0.7831485587583149, 'number': 1065}	0.6972	0.7717	0.7326	0.8007
0.4389	7.0	70	0.6444	{'precision': 0.6868365180467091, 'recall': 0.799752781211372, 'f1': 0.7390062821245003, 'number': 809}	{'precision': 0.28703703703703703, 'recall': 0.2605042016806723, 'f1': 0.27312775330396477, 'number': 119}	{'precision': 0.7411167512690355, 'recall': 0.8225352112676056, 'f1': 0.7797062750333779, 'number': 1065}	0.6962	0.7797	0.7356	0.8040
0.3912	8.0	80	0.6505	{'precision': 0.7074527252502781, 'recall': 0.7861557478368356, 'f1': 0.7447306791569087, 'number': 809}	{'precision': 0.3392857142857143, 'recall': 0.31932773109243695, 'f1': 0.32900432900432897, 'number': 119}	{'precision': 0.7689594356261023, 'recall': 0.8187793427230047, 'f1': 0.793087767166894, 'number': 1065}	0.7207	0.7757	0.7472	0.8073
0.3511	9.0	90	0.6696	{'precision': 0.7147577092511013, 'recall': 0.8022249690976514, 'f1': 0.7559697146185206, 'number': 809}	{'precision': 0.296, 'recall': 0.31092436974789917, 'f1': 0.30327868852459017, 'number': 119}	{'precision': 0.7589833479404031, 'recall': 0.8131455399061033, 'f1': 0.7851314596554851, 'number': 1065}	0.7139	0.7787	0.7449	0.8042
0.3166	10.0	100	0.6746	{'precision': 0.7190265486725663, 'recall': 0.8034610630407911, 'f1': 0.7589025102159953, 'number': 809}	{'precision': 0.35398230088495575, 'recall': 0.33613445378151263, 'f1': 0.3448275862068966, 'number': 119}	{'precision': 0.7753108348134992, 'recall': 0.819718309859155, 'f1': 0.7968963943404839, 'number': 1065}	0.7294	0.7842	0.7558	0.8081
0.2925	11.0	110	0.6839	{'precision': 0.7160356347438753, 'recall': 0.7948084054388134, 'f1': 0.753368482718219, 'number': 809}	{'precision': 0.3208955223880597, 'recall': 0.36134453781512604, 'f1': 0.33992094861660077, 'number': 119}	{'precision': 0.7803780378037803, 'recall': 0.8140845070422535, 'f1': 0.796875, 'number': 1065}	0.7247	0.7792	0.7510	0.8087
0.2837	12.0	120	0.6853	{'precision': 0.7161862527716186, 'recall': 0.7985166872682324, 'f1': 0.7551139684395091, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.3445378151260504, 'f1': 0.33884297520661155, 'number': 119}	{'precision': 0.7751322751322751, 'recall': 0.8253521126760563, 'f1': 0.7994542974079127, 'number': 1065}	0.7253	0.7858	0.7543	0.8064
0.265	13.0	130	0.7016	{'precision': 0.7069154774972558, 'recall': 0.796044499381953, 'f1': 0.7488372093023256, 'number': 809}	{'precision': 0.31654676258992803, 'recall': 0.3697478991596639, 'f1': 0.3410852713178294, 'number': 119}	{'precision': 0.7867513611615246, 'recall': 0.8140845070422535, 'f1': 0.8001845869866173, 'number': 1065}	0.7226	0.7802	0.7503	0.8076
0.2475	14.0	140	0.7055	{'precision': 0.7084708470847084, 'recall': 0.796044499381953, 'f1': 0.749708963911525, 'number': 809}	{'precision': 0.32575757575757575, 'recall': 0.36134453781512604, 'f1': 0.3426294820717131, 'number': 119}	{'precision': 0.771806167400881, 'recall': 0.8225352112676056, 'f1': 0.7963636363636363, 'number': 1065}	0.7183	0.7842	0.7498	0.8054
0.2423	15.0	150	0.7080	{'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809}	{'precision': 0.3359375, 'recall': 0.36134453781512604, 'f1': 0.3481781376518218, 'number': 119}	{'precision': 0.7817531305903399, 'recall': 0.8206572769953052, 'f1': 0.8007329363261567, 'number': 1065}	0.7260	0.7842	0.7540	0.8073

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1