metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 1.0307
- Answer: {'precision': 0.3855302279484638, 'recall': 0.48084054388133496, 'f1': 0.4279427942794279, 'number': 809}
- Header: {'precision': 0.34782608695652173, 'recall': 0.2689075630252101, 'f1': 0.3033175355450237, 'number': 119}
- Question: {'precision': 0.48268238761974946, 'recall': 0.6150234741784038, 'f1': 0.5408753096614369, 'number': 1065}
- Overall Precision: 0.4378
- Overall Recall: 0.5399
- Overall F1: 0.4835
- Overall Accuracy: 0.6393
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.7508 | 1.0 | 10 | 1.5163 | {'precision': 0.07105263157894737, 'recall': 0.10012360939431397, 'f1': 0.08311954848640328, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2490566037735849, 'recall': 0.18591549295774648, 'f1': 0.2129032258064516, 'number': 1065} | 0.1442 | 0.1400 | 0.1421 | 0.3638 |
| 1.4483 | 2.0 | 20 | 1.3842 | {'precision': 0.19585898153329603, 'recall': 0.4326328800988875, 'f1': 0.2696456086286595, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.27010309278350514, 'recall': 0.36901408450704226, 'f1': 0.3119047619047619, 'number': 1065} | 0.2286 | 0.3728 | 0.2834 | 0.4135 |
| 1.3068 | 3.0 | 30 | 1.2439 | {'precision': 0.2390092879256966, 'recall': 0.47713226205191595, 'f1': 0.3184818481848185, 'number': 809} | {'precision': 0.03125, 'recall': 0.01680672268907563, 'f1': 0.02185792349726776, 'number': 119} | {'precision': 0.32887189292543023, 'recall': 0.48450704225352115, 'f1': 0.39179954441913445, 'number': 1065} | 0.2783 | 0.4536 | 0.3450 | 0.4631 |
| 1.1868 | 4.0 | 40 | 1.1443 | {'precision': 0.25613802256138024, 'recall': 0.47713226205191595, 'f1': 0.33333333333333337, 'number': 809} | {'precision': 0.1797752808988764, 'recall': 0.13445378151260504, 'f1': 0.15384615384615385, 'number': 119} | {'precision': 0.3619233268356075, 'recall': 0.5230046948356808, 'f1': 0.42780337941628266, 'number': 1065} | 0.3059 | 0.4812 | 0.3740 | 0.5267 |
| 1.0837 | 5.0 | 50 | 1.1479 | {'precision': 0.27571728481455565, 'recall': 0.48702101359703337, 'f1': 0.3521000893655049, 'number': 809} | {'precision': 0.2696629213483146, 'recall': 0.20168067226890757, 'f1': 0.23076923076923078, 'number': 119} | {'precision': 0.3705616526791478, 'recall': 0.5389671361502347, 'f1': 0.4391736801836266, 'number': 1065} | 0.3234 | 0.4977 | 0.3921 | 0.5252 |
| 1.0102 | 6.0 | 60 | 1.1154 | {'precision': 0.29912810194500333, 'recall': 0.5512978986402967, 'f1': 0.3878260869565217, 'number': 809} | {'precision': 0.2604166666666667, 'recall': 0.21008403361344538, 'f1': 0.23255813953488375, 'number': 119} | {'precision': 0.44872918492550395, 'recall': 0.4807511737089202, 'f1': 0.4641885766092475, 'number': 1065} | 0.3603 | 0.4932 | 0.4164 | 0.5831 |
| 0.9349 | 7.0 | 70 | 1.0180 | {'precision': 0.3333333333333333, 'recall': 0.4289245982694685, 'f1': 0.37513513513513513, 'number': 809} | {'precision': 0.32558139534883723, 'recall': 0.23529411764705882, 'f1': 0.2731707317073171, 'number': 119} | {'precision': 0.42487046632124353, 'recall': 0.615962441314554, 'f1': 0.5028746646224608, 'number': 1065} | 0.3860 | 0.5173 | 0.4421 | 0.6121 |
| 0.8786 | 8.0 | 80 | 1.0198 | {'precision': 0.3177723177723178, 'recall': 0.4796044499381953, 'f1': 0.3822660098522168, 'number': 809} | {'precision': 0.2815533980582524, 'recall': 0.24369747899159663, 'f1': 0.26126126126126126, 'number': 119} | {'precision': 0.4321808510638298, 'recall': 0.6103286384976526, 'f1': 0.5060334760607241, 'number': 1065} | 0.3773 | 0.5354 | 0.4426 | 0.6088 |
| 0.8204 | 9.0 | 90 | 1.0123 | {'precision': 0.3665987780040733, 'recall': 0.44499381953028433, 'f1': 0.40201005025125625, 'number': 809} | {'precision': 0.2903225806451613, 'recall': 0.226890756302521, 'f1': 0.25471698113207547, 'number': 119} | {'precision': 0.45675482487491065, 'recall': 0.6, 'f1': 0.5186688311688312, 'number': 1065} | 0.4147 | 0.5148 | 0.4594 | 0.6320 |
| 0.8126 | 10.0 | 100 | 1.0461 | {'precision': 0.37877312560856863, 'recall': 0.48084054388133496, 'f1': 0.42374727668845313, 'number': 809} | {'precision': 0.3, 'recall': 0.226890756302521, 'f1': 0.25837320574162675, 'number': 119} | {'precision': 0.4764521193092622, 'recall': 0.5699530516431925, 'f1': 0.5190252244548953, 'number': 1065} | 0.4279 | 0.5133 | 0.4667 | 0.6288 |
| 0.7357 | 11.0 | 110 | 1.0160 | {'precision': 0.3771839671120247, 'recall': 0.453646477132262, 'f1': 0.4118967452300786, 'number': 809} | {'precision': 0.29357798165137616, 'recall': 0.2689075630252101, 'f1': 0.28070175438596495, 'number': 119} | {'precision': 0.4672639558924879, 'recall': 0.6366197183098592, 'f1': 0.5389507154213037, 'number': 1065} | 0.4252 | 0.5404 | 0.4759 | 0.6369 |
| 0.7249 | 12.0 | 120 | 1.0246 | {'precision': 0.38046795523906407, 'recall': 0.4622991347342398, 'f1': 0.4174107142857143, 'number': 809} | {'precision': 0.29411764705882354, 'recall': 0.25210084033613445, 'f1': 0.27149321266968324, 'number': 119} | {'precision': 0.4727403156384505, 'recall': 0.6187793427230047, 'f1': 0.5359902399349329, 'number': 1065} | 0.4288 | 0.5334 | 0.4754 | 0.6387 |
| 0.7015 | 13.0 | 130 | 1.0335 | {'precision': 0.36654135338345867, 'recall': 0.4820766378244747, 'f1': 0.416444207154298, 'number': 809} | {'precision': 0.31521739130434784, 'recall': 0.24369747899159663, 'f1': 0.27488151658767773, 'number': 119} | {'precision': 0.4788104089219331, 'recall': 0.6046948356807512, 'f1': 0.5344398340248964, 'number': 1065} | 0.4250 | 0.5334 | 0.4731 | 0.6326 |
| 0.6696 | 14.0 | 140 | 1.0364 | {'precision': 0.3841121495327103, 'recall': 0.5080346106304079, 'f1': 0.43746673762639704, 'number': 809} | {'precision': 0.32941176470588235, 'recall': 0.23529411764705882, 'f1': 0.2745098039215686, 'number': 119} | {'precision': 0.48804934464148036, 'recall': 0.5943661971830986, 'f1': 0.5359864521591872, 'number': 1065} | 0.4372 | 0.5379 | 0.4823 | 0.6394 |
| 0.6661 | 15.0 | 150 | 1.0307 | {'precision': 0.3855302279484638, 'recall': 0.48084054388133496, 'f1': 0.4279427942794279, 'number': 809} | {'precision': 0.34782608695652173, 'recall': 0.2689075630252101, 'f1': 0.3033175355450237, 'number': 119} | {'precision': 0.48268238761974946, 'recall': 0.6150234741784038, 'f1': 0.5408753096614369, 'number': 1065} | 0.4378 | 0.5399 | 0.4835 | 0.6393 |
Framework versions
- Transformers 4.38.2
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2