distilbert-base-cased-distilled-squad-finetuned-squad

This model is a fine-tuned version of distilbert-base-cased-distilled-squad on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.3012

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	66	2.4569
No log	2.0	132	2.3201
No log	3.0	198	2.3281
No log	4.0	264	2.3895
No log	5.0	330	2.7032
No log	6.0	396	2.9461
No log	7.0	462	3.2856
1.4273	8.0	528	3.4643
1.4273	9.0	594	3.7349
1.4273	10.0	660	4.1377
1.4273	11.0	726	4.2097
1.4273	12.0	792	4.3009
1.4273	13.0	858	4.4249
1.4273	14.0	924	4.5736
1.4273	15.0	990	4.7452
0.222	16.0	1056	4.8124
0.222	17.0	1122	4.8169
0.222	18.0	1188	5.1680
0.222	19.0	1254	5.0830
0.222	20.0	1320	5.1588
0.222	21.0	1386	5.1933
0.222	22.0	1452	5.2828
0.0343	23.0	1518	5.2054
0.0343	24.0	1584	5.2622
0.0343	25.0	1650	5.2089
0.0343	26.0	1716	5.2026
0.0343	27.0	1782	5.3382
0.0343	28.0	1848	5.2926
0.0343	29.0	1914	5.3003
0.0343	30.0	1980	5.3012

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3