---
license: mit
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: im-bin-tf-abstr
  results: []
---

# im-bin-tf-abstr

This model is a fine-tuned version of [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext)
on a dataset of medical articles (titles only). The dataset is balanced between articles from Internal Medicine (IM) and non-IM specialties (e.g. surgery, etc.).
Class labels (non-IM = 0 vs IM = 1) are derived from the respective journal the article appeared in, i.e. if the source journal topically belonged to any subspecialty of
Internal Medicine, the class label was 1. The task was classification of titles as either 0 (non-IM) or 1 (IM).

The model achieves the following results on the evaluation set:

- Loss: 0.2750
- Accuracy: 0.9261
- F1: 0.9259
- Precision: 0.9311
- Recall: 0.9207

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Accuracy | F1     | Precision | Recall |
|:-------------:|:-----:|:------:|:---------------:|:--------:|:------:|:---------:|:------:|
| 0.2484        | 1.0   | 30000  | 0.2765          | 0.9192   | 0.9209 | 0.9039    | 0.9386 |
| 0.2141        | 2.0   | 60000  | 0.2750          | 0.9261   | 0.9259 | 0.9311    | 0.9207 |
| 0.1991        | 3.0   | 90000  | 0.2952          | 0.9271   | 0.9275 | 0.9248    | 0.9303 |
| 0.1661        | 4.0   | 120000 | 0.3409          | 0.9274   | 0.9275 | 0.9284    | 0.9266 |


### Framework versions

- Transformers 4.31.0.dev0
- Pytorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.13.3