gpt2_finetuned_wolfram

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.2595

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	113	6.3789
No log	2.0	226	6.0746
No log	3.0	339	5.7649
No log	4.0	452	5.4453
5.9875	5.0	565	5.2142
5.9875	6.0	678	5.0967
5.9875	7.0	791	5.0143
5.9875	8.0	904	4.9429
4.5754	9.0	1017	4.8936
4.5754	10.0	1130	4.8722
4.5754	11.0	1243	4.8700
4.5754	12.0	1356	4.8362
4.5754	13.0	1469	4.8246
4.0366	14.0	1582	4.8242
4.0366	15.0	1695	4.8149
4.0366	16.0	1808	4.8062
4.0366	17.0	1921	4.8065
3.8118	18.0	2034	4.8288
3.8118	19.0	2147	4.8035
3.8118	20.0	2260	4.8009
3.8118	21.0	2373	4.7835
3.8118	22.0	2486	4.7865
3.6394	23.0	2599	4.7833
3.6394	24.0	2712	4.7776
3.6394	25.0	2825	4.8030
3.6394	26.0	2938	4.7684
3.5105	27.0	3051	4.7724
3.5105	28.0	3164	4.7803
3.5105	29.0	3277	4.7792
3.5105	30.0	3390	4.8027
3.38	31.0	3503	4.8000
3.38	32.0	3616	4.8046
3.38	33.0	3729	4.7751
3.38	34.0	3842	4.7774
3.38	35.0	3955	4.7733
3.2382	36.0	4068	4.7886
3.2382	37.0	4181	4.7892
3.2382	38.0	4294	4.7876
3.2382	39.0	4407	4.7965
3.1022	40.0	4520	4.7879
3.1022	41.0	4633	4.7829
3.1022	42.0	4746	4.7884
3.1022	43.0	4859	4.7845
3.1022	44.0	4972	4.8193
2.9571	45.0	5085	4.7947
2.9571	46.0	5198	4.7968
2.9571	47.0	5311	4.7894
2.9571	48.0	5424	4.7892
2.7555	49.0	5537	4.7914
2.7555	50.0	5650	4.8099
2.7555	51.0	5763	4.8029
2.7555	52.0	5876	4.8000
2.7555	53.0	5989	4.8092
2.5656	54.0	6102	4.8111
2.5656	55.0	6215	4.8257
2.5656	56.0	6328	4.8109
2.5656	57.0	6441	4.8457
2.3501	58.0	6554	4.8428
2.3501	59.0	6667	4.8519
2.3501	60.0	6780	4.8652
2.3501	61.0	6893	4.8788
2.141	62.0	7006	4.8910
2.141	63.0	7119	4.8928
2.141	64.0	7232	4.9112
2.141	65.0	7345	4.9219
2.141	66.0	7458	4.9403
1.9122	67.0	7571	4.9585
1.9122	68.0	7684	4.9726
1.9122	69.0	7797	4.9904
1.9122	70.0	7910	5.0118
1.7176	71.0	8023	5.0129
1.7176	72.0	8136	5.0303
1.7176	73.0	8249	5.0529
1.7176	74.0	8362	5.0610
1.7176	75.0	8475	5.0821
1.5292	76.0	8588	5.0931
1.5292	77.0	8701	5.1154
1.5292	78.0	8814	5.1319
1.5292	79.0	8927	5.1394
1.3843	80.0	9040	5.1529
1.3843	81.0	9153	5.1711
1.3843	82.0	9266	5.1802
1.3843	83.0	9379	5.1952
1.3843	84.0	9492	5.2088
1.2643	85.0	9605	5.2170
1.2643	86.0	9718	5.2160
1.2643	87.0	9831	5.2267
1.2643	88.0	9944	5.2346
1.1928	89.0	10057	5.2418
1.1928	90.0	10170	5.2463
1.1928	91.0	10283	5.2505
1.1928	92.0	10396	5.2522
1.1556	93.0	10509	5.2538
1.1556	94.0	10622	5.2557
1.1556	95.0	10735	5.2566
1.1556	96.0	10848	5.2585
1.1556	97.0	10961	5.2594
1.1268	98.0	11074	5.2596
1.1268	99.0	11187	5.2595
1.1268	100.0	11300	5.2595

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.10.0
• Tokenizers 0.13.2