File size: 9,424 Bytes
9dbfb4d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
library_name: transformers
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7276
- Answer: {'precision': 0.7205240174672489, 'recall': 0.8158220024721878, 'f1': 0.7652173913043478, 'number': 809}
- Header: {'precision': 0.2903225806451613, 'recall': 0.3025210084033613, 'f1': 0.2962962962962963, 'number': 119}
- Question: {'precision': 0.7903508771929825, 'recall': 0.8460093896713615, 'f1': 0.8172335600907029, 'number': 1065}
- Overall Precision: 0.7326
- Overall Recall: 0.8013
- Overall F1: 0.7654
- Overall Accuracy: 0.8111
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.735 | 1.0 | 10 | 1.5211 | {'precision': 0.04572098475967175, 'recall': 0.048207663782447466, 'f1': 0.04693140794223826, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2578616352201258, 'recall': 0.26948356807511736, 'f1': 0.26354453627180896, 'number': 1065} | 0.1658 | 0.1636 | 0.1647 | 0.4315 |
| 1.353 | 2.0 | 20 | 1.1828 | {'precision': 0.18625954198473282, 'recall': 0.1508034610630408, 'f1': 0.16666666666666669, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.48657445077298617, 'recall': 0.5615023474178403, 'f1': 0.5213600697471664, 'number': 1065} | 0.3808 | 0.3613 | 0.3708 | 0.5930 |
| 1.027 | 3.0 | 30 | 0.9100 | {'precision': 0.53, 'recall': 0.5241038318912238, 'f1': 0.5270354257302672, 'number': 809} | {'precision': 0.16981132075471697, 'recall': 0.07563025210084033, 'f1': 0.10465116279069768, 'number': 119} | {'precision': 0.6330434782608696, 'recall': 0.6835680751173709, 'f1': 0.6573363431151242, 'number': 1065} | 0.5796 | 0.5825 | 0.5811 | 0.7286 |
| 0.7694 | 4.0 | 40 | 0.7622 | {'precision': 0.6295546558704453, 'recall': 0.7688504326328801, 'f1': 0.6922648859209795, 'number': 809} | {'precision': 0.22727272727272727, 'recall': 0.12605042016806722, 'f1': 0.16216216216216214, 'number': 119} | {'precision': 0.704, 'recall': 0.7436619718309859, 'f1': 0.7232876712328766, 'number': 1065} | 0.6558 | 0.7170 | 0.6850 | 0.7708 |
| 0.6233 | 5.0 | 50 | 0.7113 | {'precision': 0.6527196652719666, 'recall': 0.7713226205191595, 'f1': 0.7070821529745043, 'number': 809} | {'precision': 0.26136363636363635, 'recall': 0.19327731092436976, 'f1': 0.22222222222222224, 'number': 119} | {'precision': 0.6977309562398704, 'recall': 0.8084507042253521, 'f1': 0.7490213136146151, 'number': 1065} | 0.6620 | 0.7566 | 0.7062 | 0.7895 |
| 0.531 | 6.0 | 60 | 0.6976 | {'precision': 0.6386138613861386, 'recall': 0.7972805933250927, 'f1': 0.709180868609126, 'number': 809} | {'precision': 0.22972972972972974, 'recall': 0.14285714285714285, 'f1': 0.17616580310880825, 'number': 119} | {'precision': 0.7175572519083969, 'recall': 0.7943661971830986, 'f1': 0.7540106951871658, 'number': 1065} | 0.6664 | 0.7566 | 0.7086 | 0.7866 |
| 0.4577 | 7.0 | 70 | 0.6823 | {'precision': 0.675531914893617, 'recall': 0.7849196538936959, 'f1': 0.7261292166952545, 'number': 809} | {'precision': 0.21951219512195122, 'recall': 0.226890756302521, 'f1': 0.2231404958677686, 'number': 119} | {'precision': 0.7434819175777965, 'recall': 0.8300469483568075, 'f1': 0.7843833185448092, 'number': 1065} | 0.6865 | 0.7757 | 0.7284 | 0.8001 |
| 0.3982 | 8.0 | 80 | 0.6871 | {'precision': 0.6847710330138446, 'recall': 0.7948084054388134, 'f1': 0.7356979405034326, 'number': 809} | {'precision': 0.2621359223300971, 'recall': 0.226890756302521, 'f1': 0.24324324324324326, 'number': 119} | {'precision': 0.7569386038687973, 'recall': 0.8450704225352113, 'f1': 0.7985803016858917, 'number': 1065} | 0.7037 | 0.7878 | 0.7434 | 0.8091 |
| 0.3614 | 9.0 | 90 | 0.6850 | {'precision': 0.7039045553145337, 'recall': 0.8022249690976514, 'f1': 0.7498555748122473, 'number': 809} | {'precision': 0.2692307692307692, 'recall': 0.23529411764705882, 'f1': 0.25112107623318386, 'number': 119} | {'precision': 0.7635593220338983, 'recall': 0.8460093896713615, 'f1': 0.8026726057906458, 'number': 1065} | 0.7153 | 0.7918 | 0.7516 | 0.8101 |
| 0.354 | 10.0 | 100 | 0.6937 | {'precision': 0.7171270718232045, 'recall': 0.8022249690976514, 'f1': 0.7572928821470245, 'number': 809} | {'precision': 0.30275229357798167, 'recall': 0.2773109243697479, 'f1': 0.28947368421052627, 'number': 119} | {'precision': 0.7840616966580977, 'recall': 0.8591549295774648, 'f1': 0.8198924731182796, 'number': 1065} | 0.7322 | 0.8013 | 0.7652 | 0.8140 |
| 0.2994 | 11.0 | 110 | 0.7161 | {'precision': 0.7063236870310825, 'recall': 0.8145859085290482, 'f1': 0.7566016073478761, 'number': 809} | {'precision': 0.2631578947368421, 'recall': 0.29411764705882354, 'f1': 0.27777777777777773, 'number': 119} | {'precision': 0.7885816235504014, 'recall': 0.8300469483568075, 'f1': 0.808783165599268, 'number': 1065} | 0.7215 | 0.7918 | 0.7550 | 0.8067 |
| 0.2908 | 12.0 | 120 | 0.7068 | {'precision': 0.7208287895310797, 'recall': 0.8170580964153276, 'f1': 0.7659327925840093, 'number': 809} | {'precision': 0.3, 'recall': 0.2773109243697479, 'f1': 0.28820960698689957, 'number': 119} | {'precision': 0.7865266841644795, 'recall': 0.844131455399061, 'f1': 0.8143115942028986, 'number': 1065} | 0.7341 | 0.7993 | 0.7653 | 0.8134 |
| 0.2689 | 13.0 | 130 | 0.7206 | {'precision': 0.7186477644492911, 'recall': 0.8145859085290482, 'f1': 0.7636152954808806, 'number': 809} | {'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119} | {'precision': 0.7954345917471466, 'recall': 0.8507042253521127, 'f1': 0.822141560798548, 'number': 1065} | 0.7331 | 0.8023 | 0.7662 | 0.8120 |
| 0.2527 | 14.0 | 140 | 0.7260 | {'precision': 0.724972497249725, 'recall': 0.8145859085290482, 'f1': 0.7671711292200234, 'number': 809} | {'precision': 0.2892561983471074, 'recall': 0.29411764705882354, 'f1': 0.2916666666666667, 'number': 119} | {'precision': 0.7900696864111498, 'recall': 0.8516431924882629, 'f1': 0.8197017623136014, 'number': 1065} | 0.7351 | 0.8033 | 0.7677 | 0.8104 |
| 0.2511 | 15.0 | 150 | 0.7276 | {'precision': 0.7205240174672489, 'recall': 0.8158220024721878, 'f1': 0.7652173913043478, 'number': 809} | {'precision': 0.2903225806451613, 'recall': 0.3025210084033613, 'f1': 0.2962962962962963, 'number': 119} | {'precision': 0.7903508771929825, 'recall': 0.8460093896713615, 'f1': 0.8172335600907029, 'number': 1065} | 0.7326 | 0.8013 | 0.7654 | 0.8111 |
### Framework versions
- Transformers 4.46.3
- Pytorch 2.5.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3
|