random_sentence_conv

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4956

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: reduce_lr_on_plateau
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
4.5924	0.0254	1000	4.3868
4.2112	0.0508	2000	4.0928
4.0634	0.0762	3000	3.9684
3.9751	0.1016	4000	3.8877
3.91	0.1270	5000	3.8313
3.8653	0.1524	6000	3.7860
3.8183	0.1778	7000	3.7501
3.7884	0.2032	8000	3.7205
3.7561	0.2286	9000	3.6964
3.7302	0.2540	10000	3.6737
3.713	0.2794	11000	3.6561
3.6922	0.3048	12000	3.6403
3.6732	0.3302	13000	3.6259
3.6638	0.3556	14000	3.6114
3.6422	0.3810	15000	3.6047
3.6275	0.4064	16000	3.5950
3.6258	0.4318	17000	3.5839
3.6078	0.4572	18000	3.5757
3.596	0.4826	19000	3.5690
3.5875	0.5080	20000	3.5619
3.5735	0.5334	21000	3.5565
3.5707	0.5588	22000	3.5468
3.5587	0.5842	23000	3.5455
3.5521	0.6096	24000	3.5381
3.5464	0.6350	25000	3.5353
3.5389	0.6604	26000	3.5322
3.5345	0.6858	27000	3.5250
3.5285	0.7112	28000	3.5243
3.5167	0.7366	29000	3.5192
3.5175	0.7620	30000	3.5147
3.5108	0.7874	31000	3.5148
3.5022	0.8128	32000	3.5104
3.5043	0.8382	33000	3.5070
3.4927	0.8636	34000	3.5048
3.4914	0.8890	35000	3.4997
3.4909	0.9144	36000	3.4984
3.4809	0.9398	37000	3.4973
3.4807	0.9652	38000	3.4950
3.4797	0.9906	39000	3.4956

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.1