SolCoderFuncs

This model is a fine-tuned version of Salesforce/codet5p-220m on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5574

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 37
  • eval_batch_size: 37
  • seed: 100
  • distributed_type: multi-GPU
  • num_devices: 4
  • total_train_batch_size: 148
  • total_eval_batch_size: 148
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 40

Training results

Training Loss Epoch Step Validation Loss
0.8793 1.0 3600 0.7881
0.7622 2.0 7200 0.7190
0.7077 3.0 10800 0.6769
0.659 4.0 14400 0.6518
0.6212 5.0 18000 0.6300
0.589 6.0 21600 0.6119
0.562 7.0 25200 0.6014
0.5361 8.0 28800 0.5905
0.5171 9.0 32400 0.5799
0.4973 10.0 36000 0.5747
0.4772 11.0 39600 0.5666
0.4619 12.0 43200 0.5610
0.4443 13.0 46800 0.5588
0.4335 14.0 50400 0.5571
0.4192 15.0 54000 0.5534
0.4062 16.0 57600 0.5512
0.3977 17.0 61200 0.5513
0.3864 18.0 64800 0.5515
0.3791 19.0 68400 0.5507
0.3718 20.0 72000 0.5510
0.4132 21.0 75600 0.5551
0.4079 22.0 79200 0.5499
0.3957 23.0 82800 0.5522
0.3895 24.0 86400 0.5482
0.3797 25.0 90000 0.5477
0.3686 26.0 93600 0.5486
0.3628 27.0 97200 0.5491
0.3518 28.0 100800 0.5502
0.3452 29.0 104400 0.5494
0.3379 30.0 108000 0.5546
0.3292 31.0 111600 0.5486
0.3232 32.0 115200 0.5522
0.3146 33.0 118800 0.5524
0.31 34.0 122400 0.5505
0.3057 35.0 126000 0.5538
0.301 36.0 129600 0.5549
0.2955 37.0 133200 0.5557
0.2901 38.0 136800 0.5554
0.2872 39.0 140400 0.5564
0.2844 40.0 144000 0.5574

Framework versions

  • Transformers 4.33.0
  • Pytorch 2.1.0+cu121
  • Datasets 2.11.0
  • Tokenizers 0.13.3
Downloads last month
20
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Pipper/SolCoderFuncs

Finetuned
(28)
this model