metadata

language:
  - multilingual
  - en
  - de
  - fr
  - ja
license: mit
tags:
  - object-detection
  - vision
  - generated_from_trainer
  - DocLayNet
  - COCO
  - PDF
  - IBM
  - Financial-Reports
  - Finance
  - Manuals
  - Scientific-Articles
  - Science
  - Laws
  - Law
  - Regulations
  - Patents
  - Government-Tenders
  - object-detection
  - image-segmentation
  - token-classification
inference: false
datasets:
  - pierreguillou/DocLayNet-base
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: >-
      lilt-xlm-roberta-base-finetuned-with-DocLayNet-base-at-paragraphlevel-ml512
    results:
      - task:
          name: Token Classification
          type: token-classification
        metrics:
          - name: f1
            type: f1
            value: 0.8634

Document Understanding model (at paragraph level)

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base with the DocLayNet base dataset. It achieves the following results on the evaluation set:

Loss: 0.4104
Precision: 0.8634
Recall: 0.8634
F1: 0.8634
Accuracy: 0.8634

References

Notebooks (line level)

APP

You can test this model with this APP in Hugging Face Spaces: Inference APP for Document Understanding at paragraph level (v1).

You can run as well the corresponding notebook: Document AI | Inference APP at paragraph level with a Document Understanding model (LiLT fine-tuned on DocLayNet dataset)

DocLayNet dataset

DocLayNet dataset (IBM) provides page-by-page layout segmentation ground-truth using bounding-boxes for 11 distinct class labels on 80863 unique pages from 6 document categories.

Until today, the dataset can be downloaded through direct links or as a dataset from Hugging Face datasets:

direct links: doclaynet_core.zip (28 GiB), doclaynet_extra.zip (7.5 GiB)
Hugging Face dataset library: dataset DocLayNet

Paper: DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis (06/02/2022)

Model description

The model was finetuned at paragraph level on chunk of 512 tokens with overlap of 128 tokens. Thus, the model was trained with all layout and text data of all pages of the dataset.

At inference time, a calculation of best probabilities give the label to each paragraph bounding boxes.

Inference

See notebook: Document AI | Inference at paragraph level with a Document Understanding model (LiLT fine-tuned on DocLayNet dataset)

Training and evaluation data

See notebook: Document AI | Fine-tune LiLT on DocLayNet base in any language at paragraph level (chunk of 512 tokens with overlap)

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 1
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	0.05	100	0.9875	0.6585	0.6585	0.6585	0.6585
No log	0.11	200	0.7886	0.7551	0.7551	0.7551	0.7551
No log	0.16	300	0.5894	0.8248	0.8248	0.8248	0.8248
No log	0.21	400	0.4794	0.8396	0.8396	0.8396	0.8396
0.7446	0.27	500	0.3993	0.8703	0.8703	0.8703	0.8703
0.7446	0.32	600	0.3631	0.8857	0.8857	0.8857	0.8857
0.7446	0.37	700	0.4096	0.8630	0.8630	0.8630	0.8630
0.7446	0.43	800	0.4492	0.8528	0.8528	0.8528	0.8528
0.7446	0.48	900	0.3839	0.8834	0.8834	0.8834	0.8834
0.4464	0.53	1000	0.4365	0.8498	0.8498	0.8498	0.8498
0.4464	0.59	1100	0.3616	0.8812	0.8812	0.8812	0.8812
0.4464	0.64	1200	0.3949	0.8796	0.8796	0.8796	0.8796
0.4464	0.69	1300	0.4184	0.8613	0.8613	0.8613	0.8613
0.4464	0.75	1400	0.4130	0.8743	0.8743	0.8743	0.8743
0.3672	0.8	1500	0.4535	0.8289	0.8289	0.8289	0.8289
0.3672	0.85	1600	0.3681	0.8713	0.8713	0.8713	0.8713
0.3672	0.91	1700	0.3446	0.8857	0.8857	0.8857	0.8857
0.3672	0.96	1800	0.4104	0.8634	0.8634	0.8634	0.8634

Framework versions

Transformers 4.26.1
Pytorch 1.13.1+cu116
Datasets 2.9.0
Tokenizers 0.13.2