lemexp-processed-task1_min_symbols_template_small-deepseek-coder-1.3b-base-test

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1481

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.3008	0.2000	5030	0.2892
0.2627	0.4001	10060	0.2511
0.2327	0.6001	15090	0.2334
0.2229	0.8001	20120	0.2116
0.2074	1.0002	25150	0.2014
0.1832	1.2002	30180	0.1921
0.1709	1.4002	35210	0.1860
0.1692	1.6003	40240	0.1758
0.1559	1.8003	45270	0.1679
0.1624	2.0003	50300	0.1645
0.1233	2.2003	55330	0.1600
0.1206	2.4004	60360	0.1539
0.1296	2.6004	65390	0.1516
0.1277	2.8004	70420	0.1481

Framework versions

• PEFT 0.14.0
• Transformers 4.47.0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0