my_awesome_ner-token_classification_v1.0.7-5

This model is a fine-tuned version of lilyyellow/my_awesome_ner-token_classification_v1.0.7-5 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6412
• Age: {'precision': 0.8231292517006803, 'recall': 0.9166666666666666, 'f1': 0.8673835125448027, 'number': 132}
• Datetime: {'precision': 0.7318548387096774, 'recall': 0.7378048780487805, 'f1': 0.7348178137651821, 'number': 984}
• Disease: {'precision': 0.677536231884058, 'recall': 0.6607773851590106, 'f1': 0.669051878354204, 'number': 283}
• Event: {'precision': 0.30604982206405695, 'recall': 0.32575757575757575, 'f1': 0.3155963302752293, 'number': 264}
• Gender: {'precision': 0.7419354838709677, 'recall': 0.8070175438596491, 'f1': 0.7731092436974789, 'number': 114}
• Law: {'precision': 0.5209003215434084, 'recall': 0.6403162055335968, 'f1': 0.5744680851063829, 'number': 253}
• Location: {'precision': 0.7192796610169492, 'recall': 0.7424822307271733, 'f1': 0.7306967984934087, 'number': 1829}
• Organization: {'precision': 0.656473649967469, 'recall': 0.7176386913229018, 'f1': 0.6856948691811077, 'number': 1406}
• Person: {'precision': 0.7022955523672884, 'recall': 0.7333333333333333, 'f1': 0.717478930010993, 'number': 1335}
• Phone: {'precision': 0.8837209302325582, 'recall': 0.9743589743589743, 'f1': 0.9268292682926831, 'number': 78}
• Product: {'precision': 0.4470046082949309, 'recall': 0.37890625, 'f1': 0.4101479915433404, 'number': 256}
• Quantity: {'precision': 0.5621890547263682, 'recall': 0.6231617647058824, 'f1': 0.5911072362685265, 'number': 544}
• Role: {'precision': 0.47593582887700536, 'recall': 0.5144508670520231, 'f1': 0.49444444444444446, 'number': 519}
• Transportation: {'precision': 0.5028571428571429, 'recall': 0.6376811594202898, 'f1': 0.5623003194888179, 'number': 138}
• Overall Precision: 0.6503
• Overall Recall: 0.6868
• Overall F1: 0.6680
• Overall Accuracy: 0.8884

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Age	Datetime	Disease	Event	Gender	Law	Location	Organization	Person	Phone	Product	Quantity	Role	Transportation	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.132	1.9991	2313	0.4438	{'precision': 0.8482758620689655, 'recall': 0.9318181818181818, 'f1': 0.8880866425992779, 'number': 132}	{'precision': 0.7001897533206831, 'recall': 0.75, 'f1': 0.7242394504416094, 'number': 984}	{'precision': 0.7104247104247104, 'recall': 0.6501766784452296, 'f1': 0.6789667896678967, 'number': 283}	{'precision': 0.30029154518950435, 'recall': 0.39015151515151514, 'f1': 0.3393739703459638, 'number': 264}	{'precision': 0.7647058823529411, 'recall': 0.7982456140350878, 'f1': 0.7811158798283262, 'number': 114}	{'precision': 0.5693430656934306, 'recall': 0.616600790513834, 'f1': 0.5920303605313093, 'number': 253}	{'precision': 0.7090248962655602, 'recall': 0.7474029524330235, 'f1': 0.7277082778812883, 'number': 1829}	{'precision': 0.6387607119314437, 'recall': 0.6891891891891891, 'f1': 0.6630174478275744, 'number': 1406}	{'precision': 0.6537414965986394, 'recall': 0.7198501872659177, 'f1': 0.685204991087344, 'number': 1335}	{'precision': 0.7888888888888889, 'recall': 0.9102564102564102, 'f1': 0.8452380952380951, 'number': 78}	{'precision': 0.37606837606837606, 'recall': 0.34375, 'f1': 0.35918367346938773, 'number': 256}	{'precision': 0.6036217303822937, 'recall': 0.5514705882352942, 'f1': 0.5763688760806917, 'number': 544}	{'precision': 0.4448462929475588, 'recall': 0.47398843930635837, 'f1': 0.458955223880597, 'number': 519}	{'precision': 0.4489795918367347, 'recall': 0.6376811594202898, 'f1': 0.5269461077844311, 'number': 138}	0.6320	0.6742	0.6524	0.8866
0.1236	3.9983	4626	0.4916	{'precision': 0.8299319727891157, 'recall': 0.9242424242424242, 'f1': 0.8745519713261649, 'number': 132}	{'precision': 0.6889952153110048, 'recall': 0.7317073170731707, 'f1': 0.7097092163627403, 'number': 984}	{'precision': 0.6178343949044586, 'recall': 0.6855123674911661, 'f1': 0.6499162479061976, 'number': 283}	{'precision': 0.26216216216216215, 'recall': 0.36742424242424243, 'f1': 0.305993690851735, 'number': 264}	{'precision': 0.7711864406779662, 'recall': 0.7982456140350878, 'f1': 0.7844827586206897, 'number': 114}	{'precision': 0.5325077399380805, 'recall': 0.6798418972332015, 'f1': 0.5972222222222223, 'number': 253}	{'precision': 0.6995329527763363, 'recall': 0.7370147621651175, 'f1': 0.7177848775292864, 'number': 1829}	{'precision': 0.6458598726114649, 'recall': 0.7211948790896159, 'f1': 0.6814516129032258, 'number': 1406}	{'precision': 0.64526588845655, 'recall': 0.7453183520599251, 'f1': 0.691692735488356, 'number': 1335}	{'precision': 0.9156626506024096, 'recall': 0.9743589743589743, 'f1': 0.9440993788819876, 'number': 78}	{'precision': 0.3524904214559387, 'recall': 0.359375, 'f1': 0.3558994197292069, 'number': 256}	{'precision': 0.5358851674641149, 'recall': 0.6176470588235294, 'f1': 0.5738684884713919, 'number': 544}	{'precision': 0.4106060606060606, 'recall': 0.5221579961464354, 'f1': 0.45971162001696353, 'number': 519}	{'precision': 0.5416666666666666, 'recall': 0.6594202898550725, 'f1': 0.5947712418300652, 'number': 138}	0.6138	0.6907	0.6500	0.8800
0.0909	5.9974	6939	0.5451	{'precision': 0.8413793103448276, 'recall': 0.9242424242424242, 'f1': 0.8808664259927798, 'number': 132}	{'precision': 0.7414684591520165, 'recall': 0.7286585365853658, 'f1': 0.735007688364941, 'number': 984}	{'precision': 0.7054263565891473, 'recall': 0.6431095406360424, 'f1': 0.6728280961182995, 'number': 283}	{'precision': 0.33613445378151263, 'recall': 0.30303030303030304, 'f1': 0.3187250996015936, 'number': 264}	{'precision': 0.7627118644067796, 'recall': 0.7894736842105263, 'f1': 0.7758620689655172, 'number': 114}	{'precision': 0.5496688741721855, 'recall': 0.6561264822134387, 'f1': 0.5981981981981982, 'number': 253}	{'precision': 0.7087024491922876, 'recall': 0.7435757244395844, 'f1': 0.7257203842049093, 'number': 1829}	{'precision': 0.6441326530612245, 'recall': 0.7183499288762447, 'f1': 0.6792199058507061, 'number': 1406}	{'precision': 0.6782246879334258, 'recall': 0.7325842696629213, 'f1': 0.7043572200216061, 'number': 1335}	{'precision': 0.8941176470588236, 'recall': 0.9743589743589743, 'f1': 0.9325153374233129, 'number': 78}	{'precision': 0.43564356435643564, 'recall': 0.34375, 'f1': 0.38427947598253276, 'number': 256}	{'precision': 0.5513866231647635, 'recall': 0.6213235294117647, 'f1': 0.5842696629213483, 'number': 544}	{'precision': 0.4785046728971963, 'recall': 0.4932562620423892, 'f1': 0.4857685009487666, 'number': 519}	{'precision': 0.50920245398773, 'recall': 0.6014492753623188, 'f1': 0.5514950166112956, 'number': 138}	0.6483	0.6817	0.6646	0.8882
0.0531	7.9965	9252	0.6110	{'precision': 0.8356164383561644, 'recall': 0.9242424242424242, 'f1': 0.8776978417266188, 'number': 132}	{'precision': 0.7186274509803922, 'recall': 0.7449186991869918, 'f1': 0.7315369261477046, 'number': 984}	{'precision': 0.6541095890410958, 'recall': 0.6749116607773852, 'f1': 0.6643478260869565, 'number': 283}	{'precision': 0.30662020905923343, 'recall': 0.3333333333333333, 'f1': 0.3194192377495463, 'number': 264}	{'precision': 0.71875, 'recall': 0.8070175438596491, 'f1': 0.7603305785123967, 'number': 114}	{'precision': 0.5838926174496645, 'recall': 0.6877470355731226, 'f1': 0.6315789473684211, 'number': 253}	{'precision': 0.7138348237769595, 'recall': 0.7419354838709677, 'f1': 0.7276139410187666, 'number': 1829}	{'precision': 0.6483375959079284, 'recall': 0.7211948790896159, 'f1': 0.6828282828282827, 'number': 1406}	{'precision': 0.6911250873515025, 'recall': 0.7408239700374532, 'f1': 0.7151120751988431, 'number': 1335}	{'precision': 0.8837209302325582, 'recall': 0.9743589743589743, 'f1': 0.9268292682926831, 'number': 78}	{'precision': 0.45045045045045046, 'recall': 0.390625, 'f1': 0.4184100418410041, 'number': 256}	{'precision': 0.5617792421746294, 'recall': 0.6268382352941176, 'f1': 0.5925282363162467, 'number': 544}	{'precision': 0.4652777777777778, 'recall': 0.5163776493256262, 'f1': 0.48949771689497723, 'number': 519}	{'precision': 0.49444444444444446, 'recall': 0.644927536231884, 'f1': 0.559748427672956, 'number': 138}	0.6448	0.6926	0.6678	0.8877
0.0441	9.9957	11565	0.6412	{'precision': 0.8231292517006803, 'recall': 0.9166666666666666, 'f1': 0.8673835125448027, 'number': 132}	{'precision': 0.7318548387096774, 'recall': 0.7378048780487805, 'f1': 0.7348178137651821, 'number': 984}	{'precision': 0.677536231884058, 'recall': 0.6607773851590106, 'f1': 0.669051878354204, 'number': 283}	{'precision': 0.30604982206405695, 'recall': 0.32575757575757575, 'f1': 0.3155963302752293, 'number': 264}	{'precision': 0.7419354838709677, 'recall': 0.8070175438596491, 'f1': 0.7731092436974789, 'number': 114}	{'precision': 0.5209003215434084, 'recall': 0.6403162055335968, 'f1': 0.5744680851063829, 'number': 253}	{'precision': 0.7192796610169492, 'recall': 0.7424822307271733, 'f1': 0.7306967984934087, 'number': 1829}	{'precision': 0.656473649967469, 'recall': 0.7176386913229018, 'f1': 0.6856948691811077, 'number': 1406}	{'precision': 0.7022955523672884, 'recall': 0.7333333333333333, 'f1': 0.717478930010993, 'number': 1335}	{'precision': 0.8837209302325582, 'recall': 0.9743589743589743, 'f1': 0.9268292682926831, 'number': 78}	{'precision': 0.4470046082949309, 'recall': 0.37890625, 'f1': 0.4101479915433404, 'number': 256}	{'precision': 0.5621890547263682, 'recall': 0.6231617647058824, 'f1': 0.5911072362685265, 'number': 544}	{'precision': 0.47593582887700536, 'recall': 0.5144508670520231, 'f1': 0.49444444444444446, 'number': 519}	{'precision': 0.5028571428571429, 'recall': 0.6376811594202898, 'f1': 0.5623003194888179, 'number': 138}	0.6503	0.6868	0.6680	0.8884

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1