lmv2-g-rai-auth-02-14

This model is a fine-tuned version of microsoft/layoutlmv2-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0368
• Dob Key Precision: 0.5057
• Dob Key Recall: 0.5205
• Dob Key F1: 0.5130
• Dob Key Number: 171
• Dob Value Precision: 0.8071
• Dob Value Recall: 0.9191
• Dob Value F1: 0.8595
• Dob Value Number: 173
• Patient Name Key Precision: 0.6923
• Patient Name Key Recall: 0.7219
• Patient Name Key F1: 0.7068
• Patient Name Key Number: 187
• Patient Name Value Precision: 0.9235
• Patient Name Value Recall: 0.9628
• Patient Name Value F1: 0.9427
• Patient Name Value Number: 188
• Provider Name Key Precision: 0.6930
• Provider Name Key Recall: 0.7065
• Provider Name Key F1: 0.6997
• Provider Name Key Number: 460
• Provider Name Value Precision: 0.9353
• Provider Name Value Recall: 0.9476
• Provider Name Value F1: 0.9414
• Provider Name Value Number: 458
• Overall Precision: 0.7796
• Overall Recall: 0.8082
• Overall F1: 0.7936
• Overall Accuracy: 0.9944

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Dob Key Precision	Dob Key Recall	Dob Key F1	Dob Key Number	Dob Value Precision	Dob Value Recall	Dob Value F1	Dob Value Number	Patient Name Key Precision	Patient Name Key Recall	Patient Name Key F1	Patient Name Key Number	Patient Name Value Precision	Patient Name Value Recall	Patient Name Value F1	Patient Name Value Number	Provider Name Key Precision	Provider Name Key Recall	Provider Name Key F1	Provider Name Key Number	Provider Name Value Precision	Provider Name Value Recall	Provider Name Value F1	Provider Name Value Number	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.1221	1.0	241	0.4373	0.0	0.0	0.0	171	0.0	0.0	0.0	173	0.0	0.0	0.0	187	0.0	0.0	0.0	188	0.0	0.0	0.0	460	0.0	0.0	0.0	458	0.0	0.0	0.0	0.9696
0.258	2.0	482	0.1408	0.0385	0.0351	0.0367	171	0.9778	0.2543	0.4037	173	0.0385	0.0053	0.0094	187	0.1739	0.0426	0.0684	188	0.0286	0.0043	0.0075	460	0.6628	0.7424	0.7003	458	0.4685	0.2450	0.3217	0.9782
0.1066	3.0	723	0.0774	0.4011	0.4386	0.4190	171	0.8404	0.9133	0.8753	173	0.5097	0.5615	0.5344	187	0.4804	0.7181	0.5757	188	0.5108	0.5674	0.5376	460	0.8841	0.9323	0.9075	458	0.6255	0.7092	0.6648	0.9920
0.0685	4.0	964	0.0585	0.4229	0.4327	0.4277	171	0.8495	0.9133	0.8802	173	0.5479	0.5508	0.5493	187	0.9005	0.9628	0.9306	188	0.6362	0.6957	0.6646	460	0.9315	0.9498	0.9405	458	0.7390	0.7764	0.7572	0.9938
0.0532	5.0	1205	0.0486	0.4432	0.4561	0.4496	171	0.8634	0.9133	0.8876	173	0.6862	0.6898	0.688	187	0.905	0.9628	0.9330	188	0.7106	0.7152	0.7129	460	0.9375	0.9498	0.9436	458	0.7826	0.8002	0.7913	0.9943
0.0453	6.0	1446	0.0429	0.4277	0.4327	0.4302	171	0.8971	0.9075	0.9023	173	0.6806	0.6952	0.6878	187	0.8835	0.9681	0.9239	188	0.7181	0.7087	0.7133	460	0.9332	0.9454	0.9393	458	0.7829	0.7954	0.7891	0.9943
0.0392	7.0	1687	0.0392	0.4432	0.4561	0.4496	171	0.8177	0.9075	0.8603	173	0.6875	0.7059	0.6966	187	0.9333	0.9681	0.9504	188	0.7045	0.7152	0.7098	460	0.9353	0.9476	0.9414	458	0.7782	0.8015	0.7896	0.9944
0.0351	8.0	1928	0.0368	0.5057	0.5205	0.5130	171	0.8071	0.9191	0.8595	173	0.6923	0.7219	0.7068	187	0.9235	0.9628	0.9427	188	0.6930	0.7065	0.6997	460	0.9353	0.9476	0.9414	458	0.7796	0.8082	0.7936	0.9944
0.0326	9.0	2169	0.0354	0.4375	0.4503	0.4438	171	0.8438	0.9364	0.8877	173	0.6943	0.7166	0.7053	187	0.9235	0.9628	0.9427	188	0.7063	0.7109	0.7086	460	0.9353	0.9476	0.9414	458	0.7809	0.8033	0.7919	0.9944
0.0313	10.0	2410	0.0350	0.4886	0.5029	0.4957	171	0.8777	0.9538	0.9141	173	0.6959	0.7219	0.7087	187	0.9188	0.9628	0.9403	188	0.6674	0.7022	0.6843	460	0.9333	0.9476	0.9404	458	0.7770	0.8088	0.7926	0.9944

Framework versions

• Transformers 4.27.0.dev0
• Pytorch 1.13.1+cu116
• Datasets 2.2.2
• Tokenizers 0.13.2