metadata

base_model: bigcode/starencoder
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - accuracy
model-index:
  - name: stack-edu-classifier-ruby
    results: []

stack-edu-classifier-ruby

This model is a fine-tuned version of bigcode/starencoder on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.3282
Precision: 0.4623
Recall: 0.3260
F1 Macro: 0.3536
Accuracy: 0.6657
F1 Binary Minimum3: 0.6101

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 64
eval_batch_size: 256
seed: 0
distributed_type: multi-GPU
num_devices: 2
total_train_batch_size: 128
total_eval_batch_size: 512
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 200
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy	F1 Binary Minimum3
No log	0	0	4.7647	0.0010	0.1667	0.0020	0.0062	0
0.358	1.4368	1000	0.3549	0.4093	0.2953	0.3072	0.6513	0.5882
0.3559	2.8736	2000	0.3425	0.4649	0.3087	0.3294	0.6571	0.6143
0.3534	4.3103	3000	0.3391	0.4318	0.3144	0.3349	0.6586	0.6149
0.3539	5.7471	4000	0.3394	0.4219	0.3244	0.3446	0.6579	0.6298
0.3585	7.1839	5000	0.3359	0.4756	0.3106	0.3350	0.6622	0.6069
0.3476	8.6207	6000	0.3339	0.4551	0.3178	0.3415	0.6638	0.6082
0.3496	10.0575	7000	0.3307	0.4512	0.3263	0.3505	0.6656	0.6204
0.3362	11.4943	8000	0.3307	0.4657	0.3228	0.3485	0.6640	0.6178
0.3442	12.9310	9000	0.3307	0.4771	0.3248	0.3517	0.6677	0.6095
0.344	14.3678	10000	0.3287	0.4774	0.3222	0.3496	0.6660	0.6147
0.3332	15.8046	11000	0.3281	0.4678	0.3240	0.3504	0.6658	0.6168
0.3359	17.2414	12000	0.3300	0.4658	0.3203	0.3471	0.6643	0.6100
0.3306	18.6782	13000	0.3282	0.4623	0.3260	0.3536	0.6657	0.6101

Framework versions

Transformers 4.43.4
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1