File size: 9,424 Bytes
edfb1c7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd-c
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd-c
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7152
- Answer: {'precision': 0.7134894091415831, 'recall': 0.7911001236093943, 'f1': 0.7502930832356389, 'number': 809}
- Header: {'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119}
- Question: {'precision': 0.7805309734513274, 'recall': 0.828169014084507, 'f1': 0.8036446469248291, 'number': 1065}
- Overall Precision: 0.7245
- Overall Recall: 0.7837
- Overall F1: 0.7530
- Overall Accuracy: 0.8069
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7835 | 1.0 | 10 | 1.5696 | {'precision': 0.02753303964757709, 'recall': 0.030902348578491966, 'f1': 0.029120559114735003, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.23644444444444446, 'recall': 0.24976525821596243, 'f1': 0.24292237442922376, 'number': 1065} | 0.1431 | 0.1460 | 0.1446 | 0.4162 |
| 1.4134 | 2.0 | 20 | 1.2167 | {'precision': 0.15942028985507245, 'recall': 0.13597033374536466, 'f1': 0.1467645096731154, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.42325227963525835, 'recall': 0.5230046948356808, 'f1': 0.4678706425871483, 'number': 1065} | 0.3322 | 0.3347 | 0.3334 | 0.5768 |
| 1.0829 | 3.0 | 30 | 0.9351 | {'precision': 0.4783599088838269, 'recall': 0.519159456118665, 'f1': 0.4979253112033195, 'number': 809} | {'precision': 0.034482758620689655, 'recall': 0.008403361344537815, 'f1': 0.013513513513513513, 'number': 119} | {'precision': 0.6103896103896104, 'recall': 0.6619718309859155, 'f1': 0.6351351351351351, 'number': 1065} | 0.5461 | 0.5650 | 0.5554 | 0.7105 |
| 0.8077 | 4.0 | 40 | 0.7702 | {'precision': 0.6122233930453108, 'recall': 0.7181705809641533, 'f1': 0.6609783845278726, 'number': 809} | {'precision': 0.2033898305084746, 'recall': 0.10084033613445378, 'f1': 0.13483146067415733, 'number': 119} | {'precision': 0.6381631037212985, 'recall': 0.7568075117370892, 'f1': 0.6924398625429553, 'number': 1065} | 0.6160 | 0.7020 | 0.6562 | 0.7659 |
| 0.6407 | 5.0 | 50 | 0.7146 | {'precision': 0.6491978609625668, 'recall': 0.7503090234857849, 'f1': 0.6961009174311926, 'number': 809} | {'precision': 0.2948717948717949, 'recall': 0.19327731092436976, 'f1': 0.233502538071066, 'number': 119} | {'precision': 0.6921221864951769, 'recall': 0.8084507042253521, 'f1': 0.7457773928107406, 'number': 1065} | 0.6606 | 0.7481 | 0.7016 | 0.7869 |
| 0.5585 | 6.0 | 60 | 0.6995 | {'precision': 0.673866090712743, 'recall': 0.7713226205191595, 'f1': 0.7193083573487031, 'number': 809} | {'precision': 0.3372093023255814, 'recall': 0.24369747899159663, 'f1': 0.2829268292682927, 'number': 119} | {'precision': 0.7374784110535406, 'recall': 0.8018779342723005, 'f1': 0.768331084120558, 'number': 1065} | 0.6945 | 0.7561 | 0.7240 | 0.7948 |
| 0.4934 | 7.0 | 70 | 0.6852 | {'precision': 0.6681222707423581, 'recall': 0.7564894932014833, 'f1': 0.7095652173913044, 'number': 809} | {'precision': 0.37777777777777777, 'recall': 0.2857142857142857, 'f1': 0.3253588516746411, 'number': 119} | {'precision': 0.7634408602150538, 'recall': 0.8, 'f1': 0.7812929848693261, 'number': 1065} | 0.7059 | 0.7516 | 0.7281 | 0.7979 |
| 0.4384 | 8.0 | 80 | 0.6731 | {'precision': 0.6920492721164614, 'recall': 0.7639060568603214, 'f1': 0.7262044653349001, 'number': 809} | {'precision': 0.3008130081300813, 'recall': 0.31092436974789917, 'f1': 0.3057851239669422, 'number': 119} | {'precision': 0.7508503401360545, 'recall': 0.8291079812206573, 'f1': 0.788041053101294, 'number': 1065} | 0.7016 | 0.7717 | 0.7350 | 0.8021 |
| 0.3737 | 9.0 | 90 | 0.6766 | {'precision': 0.6993392070484582, 'recall': 0.7849196538936959, 'f1': 0.7396622015142692, 'number': 809} | {'precision': 0.2992125984251969, 'recall': 0.31932773109243695, 'f1': 0.30894308943089427, 'number': 119} | {'precision': 0.7890974084003575, 'recall': 0.8291079812206573, 'f1': 0.8086080586080587, 'number': 1065} | 0.7224 | 0.7807 | 0.7504 | 0.8046 |
| 0.341 | 10.0 | 100 | 0.6950 | {'precision': 0.6888888888888889, 'recall': 0.7663782447466008, 'f1': 0.7255705090696314, 'number': 809} | {'precision': 0.3619047619047619, 'recall': 0.31932773109243695, 'f1': 0.33928571428571425, 'number': 119} | {'precision': 0.7859030837004405, 'recall': 0.8375586854460094, 'f1': 0.8109090909090909, 'number': 1065} | 0.7243 | 0.7777 | 0.7501 | 0.8088 |
| 0.3178 | 11.0 | 110 | 0.6979 | {'precision': 0.7157534246575342, 'recall': 0.7750309023485785, 'f1': 0.7442136498516321, 'number': 809} | {'precision': 0.375, 'recall': 0.35294117647058826, 'f1': 0.3636363636363636, 'number': 119} | {'precision': 0.7805092186128183, 'recall': 0.8347417840375587, 'f1': 0.8067150635208712, 'number': 1065} | 0.7325 | 0.7817 | 0.7563 | 0.8059 |
| 0.2998 | 12.0 | 120 | 0.7019 | {'precision': 0.7027624309392265, 'recall': 0.7861557478368356, 'f1': 0.7421236872812136, 'number': 809} | {'precision': 0.32061068702290074, 'recall': 0.35294117647058826, 'f1': 0.336, 'number': 119} | {'precision': 0.7885816235504014, 'recall': 0.8300469483568075, 'f1': 0.808783165599268, 'number': 1065} | 0.7242 | 0.7837 | 0.7528 | 0.8069 |
| 0.2809 | 13.0 | 130 | 0.7056 | {'precision': 0.7177777777777777, 'recall': 0.7985166872682324, 'f1': 0.7559976594499708, 'number': 809} | {'precision': 0.3565217391304348, 'recall': 0.3445378151260504, 'f1': 0.3504273504273504, 'number': 119} | {'precision': 0.7911504424778761, 'recall': 0.8394366197183099, 'f1': 0.8145785876993167, 'number': 1065} | 0.7371 | 0.7933 | 0.7641 | 0.8097 |
| 0.2656 | 14.0 | 140 | 0.7117 | {'precision': 0.718609865470852, 'recall': 0.792336217552534, 'f1': 0.7536743092298648, 'number': 809} | {'precision': 0.33884297520661155, 'recall': 0.3445378151260504, 'f1': 0.3416666666666667, 'number': 119} | {'precision': 0.7888198757763976, 'recall': 0.8347417840375587, 'f1': 0.8111313868613138, 'number': 1065} | 0.7341 | 0.7883 | 0.7602 | 0.8098 |
| 0.2669 | 15.0 | 150 | 0.7152 | {'precision': 0.7134894091415831, 'recall': 0.7911001236093943, 'f1': 0.7502930832356389, 'number': 809} | {'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119} | {'precision': 0.7805309734513274, 'recall': 0.828169014084507, 'f1': 0.8036446469248291, 'number': 1065} | 0.7245 | 0.7837 | 0.7530 | 0.8069 |
### Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0
|