layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7271
• Answer: {'precision': 0.7216157205240175, 'recall': 0.8170580964153276, 'f1': 0.766376811594203, 'number': 809}
• Header: {'precision': 0.3046875, 'recall': 0.3277310924369748, 'f1': 0.31578947368421056, 'number': 119}
• Question: {'precision': 0.7844905320108205, 'recall': 0.8169014084507042, 'f1': 0.8003679852805888, 'number': 1065}
• Overall Precision: 0.7292
• Overall Recall: 0.7878
• Overall F1: 0.7574
• Overall Accuracy: 0.8010

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7704	1.0	10	1.5853	{'precision': 0.0226628895184136, 'recall': 0.019777503090234856, 'f1': 0.02112211221122112, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.26928471248246844, 'recall': 0.18028169014084508, 'f1': 0.2159730033745782, 'number': 1065}	0.1466	0.1044	0.1219	0.3606
1.4598	2.0	20	1.2597	{'precision': 0.14809384164222875, 'recall': 0.12484548825710753, 'f1': 0.13547954393024816, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.3946969696969697, 'recall': 0.4892018779342723, 'f1': 0.4368972746331237, 'number': 1065}	0.3107	0.3121	0.3114	0.5801
1.1343	3.0	30	0.9794	{'precision': 0.45892018779342725, 'recall': 0.48331273176761436, 'f1': 0.47080072245635163, 'number': 809}	{'precision': 0.09523809523809523, 'recall': 0.01680672268907563, 'f1': 0.02857142857142857, 'number': 119}	{'precision': 0.578684429641965, 'recall': 0.6525821596244131, 'f1': 0.6134157105030891, 'number': 1065}	0.5246	0.5459	0.5350	0.7059
0.8715	4.0	40	0.8169	{'precision': 0.5791245791245792, 'recall': 0.6378244746600742, 'f1': 0.6070588235294119, 'number': 809}	{'precision': 0.20833333333333334, 'recall': 0.08403361344537816, 'f1': 0.11976047904191618, 'number': 119}	{'precision': 0.6848816029143898, 'recall': 0.7061032863849765, 'f1': 0.6953305594082293, 'number': 1065}	0.6274	0.6412	0.6342	0.7404
0.6755	5.0	50	0.7034	{'precision': 0.6616541353383458, 'recall': 0.761433868974042, 'f1': 0.7080459770114943, 'number': 809}	{'precision': 0.2, 'recall': 0.14285714285714285, 'f1': 0.16666666666666666, 'number': 119}	{'precision': 0.6862745098039216, 'recall': 0.7887323943661971, 'f1': 0.7339449541284403, 'number': 1065}	0.6576	0.7391	0.6960	0.7888
0.563	6.0	60	0.6924	{'precision': 0.6618998978549541, 'recall': 0.8009888751545118, 'f1': 0.7248322147651008, 'number': 809}	{'precision': 0.2191780821917808, 'recall': 0.13445378151260504, 'f1': 0.16666666666666669, 'number': 119}	{'precision': 0.7177489177489178, 'recall': 0.7784037558685446, 'f1': 0.7468468468468469, 'number': 1065}	0.6765	0.7491	0.7110	0.7869
0.4764	7.0	70	0.6676	{'precision': 0.7162011173184357, 'recall': 0.792336217552534, 'f1': 0.7523474178403756, 'number': 809}	{'precision': 0.26126126126126126, 'recall': 0.24369747899159663, 'f1': 0.25217391304347825, 'number': 119}	{'precision': 0.7538726333907056, 'recall': 0.8225352112676056, 'f1': 0.7867085765603951, 'number': 1065}	0.7131	0.7757	0.7431	0.8032
0.4205	8.0	80	0.6759	{'precision': 0.7108953613807982, 'recall': 0.8145859085290482, 'f1': 0.7592165898617511, 'number': 809}	{'precision': 0.2564102564102564, 'recall': 0.25210084033613445, 'f1': 0.2542372881355932, 'number': 119}	{'precision': 0.7594501718213058, 'recall': 0.8300469483568075, 'f1': 0.7931807985643785, 'number': 1065}	0.7124	0.7893	0.7489	0.8005
0.3675	9.0	90	0.6917	{'precision': 0.7132034632034632, 'recall': 0.8145859085290482, 'f1': 0.7605308713214081, 'number': 809}	{'precision': 0.25984251968503935, 'recall': 0.2773109243697479, 'f1': 0.2682926829268293, 'number': 119}	{'precision': 0.7740213523131673, 'recall': 0.8169014084507042, 'f1': 0.7948835084513477, 'number': 1065}	0.7182	0.7837	0.7495	0.7982
0.3596	10.0	100	0.6906	{'precision': 0.7193932827735645, 'recall': 0.8207663782447466, 'f1': 0.766743648960739, 'number': 809}	{'precision': 0.3, 'recall': 0.2773109243697479, 'f1': 0.28820960698689957, 'number': 119}	{'precision': 0.7866786678667866, 'recall': 0.8206572769953052, 'f1': 0.8033088235294117, 'number': 1065}	0.7327	0.7883	0.7595	0.8061
0.3121	11.0	110	0.6999	{'precision': 0.7300884955752213, 'recall': 0.8158220024721878, 'f1': 0.7705779334500875, 'number': 809}	{'precision': 0.3082706766917293, 'recall': 0.3445378151260504, 'f1': 0.3253968253968254, 'number': 119}	{'precision': 0.7694974003466204, 'recall': 0.8338028169014085, 'f1': 0.8003605227579991, 'number': 1065}	0.7252	0.7973	0.7596	0.8035
0.2902	12.0	120	0.7153	{'precision': 0.7124183006535948, 'recall': 0.8084054388133498, 'f1': 0.7573827446438912, 'number': 809}	{'precision': 0.3185840707964602, 'recall': 0.3025210084033613, 'f1': 0.3103448275862069, 'number': 119}	{'precision': 0.7887067395264117, 'recall': 0.8131455399061033, 'f1': 0.8007397133610726, 'number': 1065}	0.7309	0.7807	0.7550	0.8029
0.2776	13.0	130	0.7184	{'precision': 0.728587319243604, 'recall': 0.8096415327564895, 'f1': 0.7669789227166277, 'number': 809}	{'precision': 0.3, 'recall': 0.3277310924369748, 'f1': 0.3132530120481928, 'number': 119}	{'precision': 0.7759226713532513, 'recall': 0.8291079812206573, 'f1': 0.8016341352700863, 'number': 1065}	0.7277	0.7913	0.7582	0.8015
0.2604	14.0	140	0.7218	{'precision': 0.7272727272727273, 'recall': 0.8108776266996292, 'f1': 0.766803039158387, 'number': 809}	{'precision': 0.31746031746031744, 'recall': 0.33613445378151263, 'f1': 0.32653061224489793, 'number': 119}	{'precision': 0.7809439002671416, 'recall': 0.8234741784037559, 'f1': 0.8016453382084096, 'number': 1065}	0.7313	0.7893	0.7592	0.8021
0.2644	15.0	150	0.7271	{'precision': 0.7216157205240175, 'recall': 0.8170580964153276, 'f1': 0.766376811594203, 'number': 809}	{'precision': 0.3046875, 'recall': 0.3277310924369748, 'f1': 0.31578947368421056, 'number': 119}	{'precision': 0.7844905320108205, 'recall': 0.8169014084507042, 'f1': 0.8003679852805888, 'number': 1065}	0.7292	0.7878	0.7574	0.8010

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1