GerMedBert_NORMTOP50_V02_BRONCO

This model is a fine-tuned version of GerMedBERT/medbert-512 on the BRONCO150 dataset. The task is to normalize entities into the top50 codes from the ICD10GM, OPS and ATC catalogue. It achieves the following results on the evaluation set:

• Loss: 0.0166
• F1 Score: 0.8624
• Precision: 0.8939
• Recall: 0.8329
• Accuracy: 0.8594
• Num Input Tokens Seen: 15165836

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Input Tokens Seen
No log	0.25	81	0.1649	0.0	1.0	0.0	0.5920	190512
0.2195	0.5	162	0.0988	0.0	1.0	0.0	0.5920	380384
0.2195	0.75	243	0.0825	0.0	1.0	0.0	0.5920	570256
0.0855	1.0	324	0.0768	0.0	1.0	0.0	0.5920	759687
0.0855	1.25	405	0.0744	0.0	1.0	0.0	0.5920	948599
0.0737	1.5	486	0.0688	0.0	1.0	0.0	0.5920	1138791
0.0737	1.75	567	0.0652	0.1979	0.94	0.1106	0.6059	1327703
0.0666	2.0	648	0.0600	0.0982	0.9565	0.0518	0.5972	1516814
0.0666	2.25	729	0.0553	0.1231	0.9333	0.0659	0.5972	1706686
0.0572	2.5	810	0.0521	0.4093	0.8394	0.2706	0.6458	1896878
0.0572	2.75	891	0.0490	0.4326	0.8777	0.2871	0.6510	2086110
0.0499	3.0	972	0.0456	0.2897	1.0	0.1694	0.6476	2275221
0.0499	3.25	1053	0.0423	0.5060	0.925	0.3482	0.6667	2465413
0.0441	3.5	1134	0.0406	0.5386	0.8913	0.3859	0.6736	2654965
0.0441	3.75	1215	0.0396	0.5979	0.8419	0.4635	0.6858	2845157
0.0393	4.0	1296	0.0377	0.6517	0.9004	0.5106	0.7083	3034556
0.0393	4.25	1377	0.0357	0.6319	0.9075	0.4847	0.6997	3224428
0.0361	4.5	1458	0.0346	0.6154	0.9245	0.4612	0.7066	3414620
0.0361	4.75	1539	0.0334	0.6258	0.8987	0.48	0.7101	3604172
0.032	5.0	1620	0.0321	0.6775	0.9124	0.5388	0.7292	3793603
0.032	5.25	1701	0.0306	0.7081	0.9176	0.5765	0.7378	3983155
0.0293	5.5	1782	0.0302	0.6928	0.9019	0.5624	0.7361	4172387
0.0293	5.75	1863	0.0292	0.6657	0.9247	0.52	0.7240	4362579
0.0273	6.0	1944	0.0287	0.7365	0.9253	0.6118	0.7691	4552330
0.0273	6.25	2025	0.0275	0.7215	0.9328	0.5882	0.7552	4741882
0.0258	6.5	2106	0.0272	0.7275	0.9024	0.6094	0.7517	4930794
0.0258	6.75	2187	0.0260	0.7451	0.9204	0.6259	0.7726	5120026
0.0228	7.0	2268	0.0260	0.7247	0.9203	0.5976	0.7656	5309137
0.0228	7.25	2349	0.0249	0.7867	0.9077	0.6941	0.7969	5499649
0.0218	7.5	2430	0.0246	0.7572	0.9079	0.6494	0.7778	5688881
0.0218	7.75	2511	0.0239	0.7779	0.9088	0.68	0.7882	5878113
0.02	8.0	2592	0.0239	0.7835	0.8994	0.6941	0.7899	6067224
0.02	8.25	2673	0.0229	0.7711	0.9159	0.6659	0.7917	6256456
0.0184	8.5	2754	0.0227	0.7705	0.8969	0.6753	0.7917	6446328
0.0184	8.75	2835	0.0226	0.7782	0.8671	0.7059	0.7899	6636520
0.0182	9.0	2916	0.0224	0.7937	0.8988	0.7106	0.8003	6825951
0.0182	9.25	2997	0.0217	0.7815	0.8939	0.6941	0.7951	7015183
0.0172	9.5	3078	0.0213	0.8156	0.9101	0.7388	0.8212	7205375
0.0172	9.75	3159	0.0211	0.8063	0.9086	0.7247	0.8142	7394927
0.0154	10.0	3240	0.0216	0.8246	0.8820	0.7741	0.8212	7583366
0.0154	10.25	3321	0.0204	0.7831	0.8943	0.6965	0.8021	7772598
0.0145	10.5	3402	0.0201	0.8185	0.9034	0.7482	0.8229	7962470
0.0145	10.75	3483	0.0200	0.8261	0.9048	0.76	0.8264	8152662
0.0143	11.0	3564	0.0198	0.8238	0.8929	0.7647	0.8281	8341773
0.0143	11.25	3645	0.0196	0.8229	0.8972	0.76	0.8264	8531645
0.0131	11.5	3726	0.0193	0.8231	0.8817	0.7718	0.8212	8720877
0.0131	11.75	3807	0.0195	0.8152	0.8822	0.7576	0.8177	8910109
0.0129	12.0	3888	0.0192	0.8263	0.9119	0.7553	0.8299	9099860
0.0129	12.25	3969	0.0188	0.8229	0.8972	0.76	0.8212	9289412
0.0116	12.5	4050	0.0191	0.8123	0.8883	0.7482	0.8247	9479284
0.0116	12.75	4131	0.0181	0.8417	0.9030	0.7882	0.8472	9669156
0.0115	13.0	4212	0.0180	0.8398	0.8895	0.7953	0.8420	9857947
0.0115	13.25	4293	0.0177	0.8445	0.9126	0.7859	0.8455	10045899
0.0108	13.5	4374	0.0179	0.8426	0.8901	0.8	0.8438	10236091
0.0108	13.75	4455	0.0179	0.8519	0.8961	0.8118	0.8524	10426283
0.0103	14.0	4536	0.0177	0.8392	0.9003	0.7859	0.8420	10615394
0.0103	14.25	4617	0.0176	0.8603	0.9062	0.8188	0.8594	10805906
0.0097	14.5	4698	0.0173	0.8475	0.904	0.7976	0.8507	10995458
0.0097	14.75	4779	0.0175	0.8511	0.9003	0.8071	0.8524	11185650
0.0095	15.0	4860	0.0173	0.8501	0.8979	0.8071	0.8490	11375081
0.0095	15.25	4941	0.0175	0.8451	0.8927	0.8024	0.8472	11564633
0.009	15.5	5022	0.0174	0.8483	0.8912	0.8094	0.8490	11754185
0.009	15.75	5103	0.0172	0.8490	0.8956	0.8071	0.8438	11943417
0.009	16.0	5184	0.0174	0.8547	0.8883	0.8235	0.8490	12132528
0.009	16.25	5265	0.0171	0.8519	0.8961	0.8118	0.8490	12322400
0.0085	16.5	5346	0.0169	0.8537	0.8861	0.8235	0.8524	12511632
0.0085	16.75	5427	0.0169	0.8498	0.8915	0.8118	0.8524	12701184
0.0084	17.0	5508	0.0169	0.8487	0.8892	0.8118	0.8507	12889655
0.0084	17.25	5589	0.0167	0.8634	0.8962	0.8329	0.8524	13079527
0.0079	17.5	5670	0.0168	0.8504	0.8958	0.8094	0.8472	13269399
0.0079	17.75	5751	0.0168	0.8606	0.8957	0.8282	0.8594	13458631
0.0081	18.0	5832	0.0167	0.8564	0.8949	0.8212	0.8542	13648382
0.0081	18.25	5913	0.0166	0.8620	0.8959	0.8306	0.8576	13838254
0.0078	18.5	5994	0.0166	0.8533	0.8964	0.8141	0.8542	14028126
0.0078	18.75	6075	0.0167	0.8610	0.8937	0.8306	0.8611	14217678
0.0078	19.0	6156	0.0166	0.8610	0.8937	0.8306	0.8594	14407109
0.0078	19.25	6237	0.0166	0.8637	0.8942	0.8353	0.8594	14596341
0.0072	19.5	6318	0.0166	0.8620	0.8959	0.8306	0.8594	14786213
0.0072	19.75	6399	0.0166	0.8624	0.8939	0.8329	0.8594	14976085
0.008	20.0	6480	0.0166	0.8624	0.8939	0.8329	0.8594	15165836

Framework versions

• Transformers 4.40.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1