distilbert-finetuned-ner-for-articles

This model is a fine-tuned version of distilbert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4393
• Precision: 0.6848
• Recall: 0.6391
• F1: 0.6611
• Accuracy: 0.8736

Model description

Distilbert finetuned for detecting crime, accidents, and natural disaster occurrences.

Tags (IOBES/BIOES tagging format):

• O: not an entity
• S-CRIME
• S-CRIMINAL
• S-VICTIM
• S-SUSPECT
• S-TIMEDATE: date with month, day, year, either one, two, or all of them together
• S-TIMEWORD: words signifying time (last, weekend, earlier, week, today, etc.)
• S-TIMEDAY: days of the week
• S-TIMEDAYPART: morning, afternoon, evening, night
• S-TIMENUM: 4:31, 6:30, etc.
• S-TIMEMISC: New Year, Christmas, etc.
• S-LOC: location word (mentioned alone)
• B-LOC: beginning (part of a series of location names mentioned)
• I-LOC: inside
• E-LOC: end (the last location word specified)
• S-LOCWORD: junction, island, street, etc.
• S-LOCDIR: north, south, etc.
• S-ACCIDENT
• S-NATDISAS: type of natural disaster
• S-OTHEROCC: other occurrences (not really labeled much in the dataset)

Dataset used is of size 502, manually annotated the dataset from the paper "MN-DS: A Multilabeled News Dataset for News Articles Hierarchical Classification" using Doccano (a free NER annotation tool).

Intended uses & limitations

• Needs a bigger dataset.
• More training is highly recommended.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.4324	1.0	51	0.4654	0.6744	0.5577	0.6105	0.8598
0.3106	2.0	102	0.4438	0.7041	0.6026	0.6494	0.8674
0.2886	3.0	153	0.4378	0.6744	0.5987	0.6343	0.8678
0.2724	4.0	204	0.4443	0.6788	0.6449	0.6614	0.8736
0.2504	5.0	255	0.4393	0.6848	0.6391	0.6611	0.8736

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1