---
language: nl
license: mit
pipeline_tag: text-classification
inference: false
---

# A-PROOF Binary Sentence Classification 

## Description

A fine-tuned binary text classification model that determines whether a sentence is relevant for WHO-ICF category classification.

Since 95% of the sentences in clinical notes is not relevant for ICF classification, it makes sense to filter the relevant sentences before applying other classification processes. Using the binary classification, the processing of large volumes of data can be optimised as only 5% of the sentences needs to be classified for the level of functioning.
For further classification of relevant sentences, you can use the multilabel classifier: https://huggingface.co/CLTL/icf-domains and the any of the relevant regression classifiers for obtaining a level score.

Relevant sentences are likely to be express patient's functioning for the following 9 ICF categories:

ICF code | Domain | name in repo
---|---|---
b440 | Respiration functions | ADM
b140 | Attention functions | ATT
d840-d859 | Work and employment | BER
b1300 | Energy level | ENR
d550 | Eating | ETN
d450 | Walking | FAC
b455 | Exercise tolerance functions | INS
b530 | Weight maintenance functions | MBW
b152 | Emotional functions | STM


## Intended use and limitations
- The model was fine-tuned (trained, validated and tested) on medical records from the Amsterdam UMC (the two academic medical centers of Amsterdam). It might perform differently on text from a different hospital or text from non-hospital sources (e.g. GP records).
- The model only distinguishes sentences on the basis of the 9 ICF categories.

## How to use
To generate predictions with the model, use the [Transformers](https://huggingface.co/docs/transformers) library:

```
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline('text-classification', model='CLTL/binary_icf_classifier')
result = pipe('De patient is erg moe')
print(result)
[{'label': 'pos', 'score': 0.9977788329124451}]
```

```
# load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer. from_pretrained ('CLTL/binary_icf_classifier')
model = AutoModelForSequenceClassification.from_pretrained('CLTL/binary_icf_classifier')
```

## Training data
- The training data consists of clinical notes from medical records (in Dutch) of the Amsterdam UMC. Due to privacy constraints, the data cannot be released.
- The annotation guidelines used for the project can be found [here](https://github.com/cltl/a-proof-zonmw/tree/main/resources/annotation_guidelines).

## Evaluation results
The evaluation is done on a sentence-level (the classification unit): .97 precision, .96 recall, .97 f1.

## Contact
Piek Vossen, piek.vossen@vu.nl

## References
https://github.com/cltl-students/Cecilia_Kuan_data_augmentation
Cecilia Kuan, 2023, Generative Approach of Data Augmentation for Pre-Trained Clinical NLP System, MA Thesis, Vrije Universiteit Amsterdam