---
library_name: peft
license: apache-2.0
base_model: distilbert-base-uncased
tags:
- generated_from_trainer
datasets:
- conll2002
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: distilbert-ner-qlorafinetune-runs
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-ner-qlorafinetune-runs

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the conll2002 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2164
- Precision: 0.6299
- Recall: 0.6227
- F1: 0.6263
- Accuracy: 0.9372

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0004
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- training_steps: 640
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 1.0711        | 0.0766 | 20   | 0.7111          | 0.0       | 0.0    | 0.0    | 0.8570   |
| 0.5291        | 0.1533 | 40   | 0.5467          | 0.0       | 0.0    | 0.0    | 0.8570   |
| 0.4545        | 0.2299 | 60   | 0.4850          | 0.2172    | 0.1687 | 0.1899 | 0.8769   |
| 0.4113        | 0.3065 | 80   | 0.4470          | 0.3227    | 0.1765 | 0.2282 | 0.8816   |
| 0.3837        | 0.3831 | 100  | 0.4049          | 0.4187    | 0.3840 | 0.4006 | 0.8896   |
| 0.334         | 0.4598 | 120  | 0.3639          | 0.4695    | 0.4276 | 0.4476 | 0.8981   |
| 0.3342        | 0.5364 | 140  | 0.3499          | 0.5104    | 0.4520 | 0.4794 | 0.8997   |
| 0.322         | 0.6130 | 160  | 0.3281          | 0.4939    | 0.4920 | 0.4929 | 0.9091   |
| 0.2868        | 0.6897 | 180  | 0.3021          | 0.5207    | 0.4646 | 0.4911 | 0.9145   |
| 0.2788        | 0.7663 | 200  | 0.2878          | 0.5361    | 0.5064 | 0.5209 | 0.9185   |
| 0.2748        | 0.8429 | 220  | 0.2864          | 0.5419    | 0.5232 | 0.5324 | 0.9197   |
| 0.2435        | 0.9195 | 240  | 0.2750          | 0.5306    | 0.5294 | 0.5300 | 0.9205   |
| 0.238         | 0.9962 | 260  | 0.2636          | 0.5525    | 0.5623 | 0.5573 | 0.9239   |
| 0.2465        | 1.0728 | 280  | 0.2616          | 0.5574    | 0.5602 | 0.5588 | 0.9255   |
| 0.2296        | 1.1494 | 300  | 0.2607          | 0.5859    | 0.5409 | 0.5625 | 0.9252   |
| 0.2141        | 1.2261 | 320  | 0.2491          | 0.5728    | 0.5841 | 0.5784 | 0.9279   |
| 0.2229        | 1.3027 | 340  | 0.2483          | 0.5849    | 0.5767 | 0.5808 | 0.9289   |
| 0.2234        | 1.3793 | 360  | 0.2413          | 0.5906    | 0.5712 | 0.5808 | 0.9310   |
| 0.2217        | 1.4559 | 380  | 0.2416          | 0.5890    | 0.5944 | 0.5917 | 0.9321   |
| 0.208         | 1.5326 | 400  | 0.2337          | 0.6117    | 0.5889 | 0.6001 | 0.9326   |
| 0.1961        | 1.6092 | 420  | 0.2387          | 0.5950    | 0.6018 | 0.5984 | 0.9321   |
| 0.2237        | 1.6858 | 440  | 0.2263          | 0.6230    | 0.6094 | 0.6161 | 0.9353   |
| 0.2029        | 1.7625 | 460  | 0.2262          | 0.6377    | 0.6045 | 0.6207 | 0.9353   |
| 0.203         | 1.8391 | 480  | 0.2229          | 0.6246    | 0.6167 | 0.6206 | 0.9358   |
| 0.2098        | 1.9157 | 500  | 0.2221          | 0.6277    | 0.6264 | 0.6270 | 0.9363   |
| 0.1907        | 1.9923 | 520  | 0.2237          | 0.6197    | 0.6186 | 0.6191 | 0.9355   |
| 0.1774        | 2.0690 | 540  | 0.2214          | 0.6284    | 0.6170 | 0.6226 | 0.9365   |
| 0.1822        | 2.1456 | 560  | 0.2213          | 0.6267    | 0.6211 | 0.6239 | 0.9368   |
| 0.1783        | 2.2222 | 580  | 0.2180          | 0.6308    | 0.6266 | 0.6287 | 0.9371   |
| 0.1856        | 2.2989 | 600  | 0.2174          | 0.6289    | 0.6206 | 0.6247 | 0.9369   |
| 0.1773        | 2.3755 | 620  | 0.2172          | 0.6192    | 0.6278 | 0.6235 | 0.9362   |
| 0.1647        | 2.4521 | 640  | 0.2164          | 0.6299    | 0.6227 | 0.6263 | 0.9372   |


### Framework versions

- PEFT 0.13.2
- Transformers 4.46.3
- Pytorch 2.5.1
- Datasets 3.1.0
- Tokenizers 0.20.3