File size: 9,370 Bytes
6e01f7a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7211
- Answer: {'precision': 0.7268722466960352, 'recall': 0.8158220024721878, 'f1': 0.7687827606290041, 'number': 809}
- Header: {'precision': 0.31343283582089554, 'recall': 0.35294117647058826, 'f1': 0.3320158102766798, 'number': 119}
- Question: {'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065}
- Overall Precision: 0.7332
- Overall Recall: 0.8013
- Overall F1: 0.7658
- Overall Accuracy: 0.7963
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7901 | 1.0 | 10 | 1.5938 | {'precision': 0.017361111111111112, 'recall': 0.012360939431396786, 'f1': 0.014440433212996389, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.32525252525252524, 'recall': 0.1511737089201878, 'f1': 0.20641025641025643, 'number': 1065} | 0.1597 | 0.0858 | 0.1116 | 0.3415 |
| 1.4447 | 2.0 | 20 | 1.2469 | {'precision': 0.22497522299306244, 'recall': 0.28059332509270707, 'f1': 0.24972497249724973, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4132890365448505, 'recall': 0.584037558685446, 'f1': 0.4840466926070039, 'number': 1065} | 0.3377 | 0.4260 | 0.3767 | 0.5933 |
| 1.0816 | 3.0 | 30 | 0.9331 | {'precision': 0.5004995004995005, 'recall': 0.619283065512979, 'f1': 0.5535911602209945, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5584415584415584, 'recall': 0.7267605633802817, 'f1': 0.631578947368421, 'number': 1065} | 0.5260 | 0.6397 | 0.5773 | 0.7095 |
| 0.8262 | 4.0 | 40 | 0.7964 | {'precision': 0.5772357723577236, 'recall': 0.7021013597033374, 'f1': 0.633575013943112, 'number': 809} | {'precision': 0.15625, 'recall': 0.08403361344537816, 'f1': 0.10928961748633881, 'number': 119} | {'precision': 0.6492659053833605, 'recall': 0.7474178403755869, 'f1': 0.6948930597992143, 'number': 1065} | 0.6042 | 0.6894 | 0.6440 | 0.7450 |
| 0.6674 | 5.0 | 50 | 0.7441 | {'precision': 0.6445182724252492, 'recall': 0.7194066749072929, 'f1': 0.6799065420560747, 'number': 809} | {'precision': 0.22105263157894736, 'recall': 0.17647058823529413, 'f1': 0.19626168224299065, 'number': 119} | {'precision': 0.6424870466321243, 'recall': 0.8150234741784037, 'f1': 0.7185430463576157, 'number': 1065} | 0.6262 | 0.7381 | 0.6776 | 0.7673 |
| 0.5736 | 6.0 | 60 | 0.7005 | {'precision': 0.6451942740286298, 'recall': 0.7799752781211372, 'f1': 0.7062115277000558, 'number': 809} | {'precision': 0.20454545454545456, 'recall': 0.15126050420168066, 'f1': 0.17391304347826086, 'number': 119} | {'precision': 0.7412891986062717, 'recall': 0.7990610328638498, 'f1': 0.7690917306823317, 'number': 1065} | 0.6775 | 0.7526 | 0.7131 | 0.7755 |
| 0.5042 | 7.0 | 70 | 0.6801 | {'precision': 0.6768743400211193, 'recall': 0.792336217552534, 'f1': 0.7300683371298405, 'number': 809} | {'precision': 0.22018348623853212, 'recall': 0.20168067226890757, 'f1': 0.21052631578947367, 'number': 119} | {'precision': 0.7412765957446809, 'recall': 0.8178403755868544, 'f1': 0.7776785714285714, 'number': 1065} | 0.6885 | 0.7707 | 0.7273 | 0.7841 |
| 0.4479 | 8.0 | 80 | 0.6712 | {'precision': 0.6687565308254964, 'recall': 0.7911001236093943, 'f1': 0.7248018120045301, 'number': 809} | {'precision': 0.20610687022900764, 'recall': 0.226890756302521, 'f1': 0.21600000000000003, 'number': 119} | {'precision': 0.7404006677796328, 'recall': 0.8328638497652582, 'f1': 0.7839151568714097, 'number': 1065} | 0.6798 | 0.7797 | 0.7263 | 0.7900 |
| 0.3931 | 9.0 | 90 | 0.6806 | {'precision': 0.7054263565891473, 'recall': 0.7873918417799752, 'f1': 0.7441588785046728, 'number': 809} | {'precision': 0.2809917355371901, 'recall': 0.2857142857142857, 'f1': 0.2833333333333333, 'number': 119} | {'precision': 0.7510620220900595, 'recall': 0.8300469483568075, 'f1': 0.7885816235504014, 'number': 1065} | 0.7065 | 0.7802 | 0.7415 | 0.7955 |
| 0.3875 | 10.0 | 100 | 0.6819 | {'precision': 0.7014767932489452, 'recall': 0.8220024721878862, 'f1': 0.7569721115537849, 'number': 809} | {'precision': 0.3, 'recall': 0.3025210084033613, 'f1': 0.301255230125523, 'number': 119} | {'precision': 0.7633851468048359, 'recall': 0.8300469483568075, 'f1': 0.7953216374269007, 'number': 1065} | 0.7120 | 0.7953 | 0.7514 | 0.7952 |
| 0.3309 | 11.0 | 110 | 0.7016 | {'precision': 0.7204419889502762, 'recall': 0.8059332509270705, 'f1': 0.7607934655775962, 'number': 809} | {'precision': 0.2949640287769784, 'recall': 0.3445378151260504, 'f1': 0.31782945736434104, 'number': 119} | {'precision': 0.7535864978902953, 'recall': 0.8384976525821596, 'f1': 0.7937777777777778, 'number': 1065} | 0.7115 | 0.7958 | 0.7513 | 0.7969 |
| 0.3142 | 12.0 | 120 | 0.7081 | {'precision': 0.7178924259055982, 'recall': 0.8084054388133498, 'f1': 0.7604651162790698, 'number': 809} | {'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119} | {'precision': 0.7768014059753954, 'recall': 0.8300469483568075, 'f1': 0.8025419881979118, 'number': 1065} | 0.7245 | 0.7918 | 0.7567 | 0.7993 |
| 0.2992 | 13.0 | 130 | 0.7160 | {'precision': 0.716304347826087, 'recall': 0.8145859085290482, 'f1': 0.7622903412377097, 'number': 809} | {'precision': 0.304, 'recall': 0.31932773109243695, 'f1': 0.31147540983606553, 'number': 119} | {'precision': 0.7796167247386759, 'recall': 0.8403755868544601, 'f1': 0.8088567555354722, 'number': 1065} | 0.7259 | 0.7988 | 0.7606 | 0.7938 |
| 0.2746 | 14.0 | 140 | 0.7194 | {'precision': 0.7238723872387238, 'recall': 0.8133498145859085, 'f1': 0.7660069848661233, 'number': 809} | {'precision': 0.32061068702290074, 'recall': 0.35294117647058826, 'f1': 0.336, 'number': 119} | {'precision': 0.7859030837004405, 'recall': 0.8375586854460094, 'f1': 0.8109090909090909, 'number': 1065} | 0.7320 | 0.7988 | 0.7639 | 0.7957 |
| 0.2735 | 15.0 | 150 | 0.7211 | {'precision': 0.7268722466960352, 'recall': 0.8158220024721878, 'f1': 0.7687827606290041, 'number': 809} | {'precision': 0.31343283582089554, 'recall': 0.35294117647058826, 'f1': 0.3320158102766798, 'number': 119} | {'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065} | 0.7332 | 0.8013 | 0.7658 | 0.7963 |
### Framework versions
- Transformers 4.41.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
|