--- library_name: transformers license: apache-2.0 base_model: HuggingFaceTB/SmolLM2-135M tags: - generated_from_trainer metrics: - accuracy model-index: - name: smol-135-tq-closure-synthetic results: [] --- # smol-135-tq-closure-synthetic This model is a fine-tuned version of [HuggingFaceTB/SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.2292 - < Precision: 0.8939 - < Recall: 0.8974 - < F1-score: 0.8956 - < Support: 4036.0 - > Precision: 0.9419 - > Recall: 0.9383 - > F1-score: 0.9401 - > Support: 3681.0 - = Precision: 0.7952 - = Recall: 0.8187 - = F1-score: 0.8068 - = Support: 1622.0 - - Precision: 0.7872 - - Recall: 0.7277 - - F1-score: 0.7563 - - Support: 661.0 - Accuracy: 0.8885 - Macro Avg Precision: 0.8546 - Macro Avg Recall: 0.8455 - Macro Avg F1-score: 0.8497 - Macro Avg Support: 10000.0 - Weighted Avg Precision: 0.8885 - Weighted Avg Recall: 0.8885 - Weighted Avg F1-score: 0.8884 - Weighted Avg Support: 10000.0 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.001 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - gradient_accumulation_steps: 2 - total_train_batch_size: 512 - total_eval_batch_size: 256 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: reduce_lr_on_plateau - num_epochs: 30 ### Training results | Training Loss | Epoch | Step | Validation Loss | < Precision | < Recall | < F1-score | < Support | > Precision | > Recall | > F1-score | > Support | = Precision | = Recall | = F1-score | = Support | - Precision | - Recall | - F1-score | - Support | Accuracy | Macro Avg Precision | Macro Avg Recall | Macro Avg F1-score | Macro Avg Support | Weighted Avg Precision | Weighted Avg Recall | Weighted Avg F1-score | Weighted Avg Support | |:-------------:|:-----:|:-----:|:---------------:|:-----------:|:--------:|:----------:|:---------:|:-----------:|:--------:|:----------:|:---------:|:-----------:|:--------:|:----------:|:---------:|:-----------:|:--------:|:----------:|:---------:|:--------:|:-------------------:|:----------------:|:------------------:|:-----------------:|:----------------------:|:-------------------:|:---------------------:|:--------------------:| | 0.4785 | 1.0 | 1354 | 0.2004 | 0.8789 | 0.8665 | 0.8726 | 4036.0 | 0.8989 | 0.9226 | 0.9106 | 3681.0 | 0.7986 | 0.7700 | 0.7841 | 1622.0 | 0.7467 | 0.7670 | 0.7567 | 661.0 | 0.8649 | 0.8308 | 0.8315 | 0.8310 | 10000.0 | 0.8645 | 0.8649 | 0.8646 | 10000.0 | | 0.3428 | 2.0 | 2708 | 0.1891 | 0.8766 | 0.8959 | 0.8862 | 4036.0 | 0.9215 | 0.9313 | 0.9264 | 3681.0 | 0.8175 | 0.7762 | 0.7963 | 1622.0 | 0.7967 | 0.7413 | 0.7680 | 661.0 | 0.8793 | 0.8531 | 0.8362 | 0.8442 | 10000.0 | 0.8783 | 0.8793 | 0.8786 | 10000.0 | | 0.3423 | 3.0 | 4062 | 0.1858 | 0.9031 | 0.8749 | 0.8887 | 4036.0 | 0.9234 | 0.9337 | 0.9285 | 3681.0 | 0.7783 | 0.8224 | 0.7998 | 1622.0 | 0.7645 | 0.7564 | 0.7605 | 661.0 | 0.8802 | 0.8423 | 0.8469 | 0.8444 | 10000.0 | 0.8812 | 0.8802 | 0.8805 | 10000.0 | | 0.3044 | 4.0 | 5416 | 0.1870 | 0.8875 | 0.8853 | 0.8864 | 4036.0 | 0.9168 | 0.9397 | 0.9281 | 3681.0 | 0.8172 | 0.7965 | 0.8067 | 1622.0 | 0.7710 | 0.7231 | 0.7463 | 661.0 | 0.8802 | 0.8481 | 0.8362 | 0.8419 | 10000.0 | 0.8792 | 0.8802 | 0.8796 | 10000.0 | | 0.2814 | 5.0 | 6770 | 0.1857 | 0.8880 | 0.8902 | 0.8891 | 4036.0 | 0.9335 | 0.9348 | 0.9342 | 3681.0 | 0.8006 | 0.8095 | 0.8050 | 1622.0 | 0.7818 | 0.7428 | 0.7618 | 661.0 | 0.8838 | 0.8510 | 0.8443 | 0.8475 | 10000.0 | 0.8836 | 0.8838 | 0.8837 | 10000.0 | | 0.2861 | 6.0 | 8124 | 0.1893 | 0.8928 | 0.8977 | 0.8952 | 4036.0 | 0.9361 | 0.9315 | 0.9338 | 3681.0 | 0.7930 | 0.8150 | 0.8039 | 1622.0 | 0.7810 | 0.7231 | 0.7510 | 661.0 | 0.8852 | 0.8508 | 0.8419 | 0.8460 | 10000.0 | 0.8852 | 0.8852 | 0.8851 | 10000.0 | | 0.3019 | 7.0 | 9478 | 0.1982 | 0.8693 | 0.9093 | 0.8888 | 4036.0 | 0.9314 | 0.9326 | 0.9320 | 3681.0 | 0.8340 | 0.7620 | 0.7964 | 1622.0 | 0.7787 | 0.7186 | 0.7474 | 661.0 | 0.8814 | 0.8533 | 0.8306 | 0.8412 | 10000.0 | 0.8804 | 0.8814 | 0.8804 | 10000.0 | | 0.2531 | 8.0 | 10832 | 0.2028 | 0.8955 | 0.8858 | 0.8906 | 4036.0 | 0.9245 | 0.9416 | 0.9330 | 3681.0 | 0.8103 | 0.7873 | 0.7986 | 1622.0 | 0.7438 | 0.7685 | 0.7560 | 661.0 | 0.8826 | 0.8435 | 0.8458 | 0.8445 | 10000.0 | 0.8823 | 0.8826 | 0.8824 | 10000.0 | | 0.1957 | 9.0 | 12186 | 0.2206 | 0.8882 | 0.9036 | 0.8958 | 4036.0 | 0.9264 | 0.9478 | 0.9370 | 3681.0 | 0.8272 | 0.7824 | 0.8042 | 1622.0 | 0.7896 | 0.7095 | 0.7474 | 661.0 | 0.8874 | 0.8579 | 0.8358 | 0.8461 | 10000.0 | 0.8859 | 0.8874 | 0.8863 | 10000.0 | | 0.1648 | 10.0 | 13540 | 0.2256 | 0.8905 | 0.8902 | 0.8903 | 4036.0 | 0.9290 | 0.9421 | 0.9355 | 3681.0 | 0.8080 | 0.7885 | 0.7981 | 1622.0 | 0.7596 | 0.7458 | 0.7527 | 661.0 | 0.8833 | 0.8468 | 0.8417 | 0.8442 | 10000.0 | 0.8826 | 0.8833 | 0.8829 | 10000.0 | | 0.1691 | 11.0 | 14894 | 0.2292 | 0.8939 | 0.8974 | 0.8956 | 4036.0 | 0.9419 | 0.9383 | 0.9401 | 3681.0 | 0.7952 | 0.8187 | 0.8068 | 1622.0 | 0.7872 | 0.7277 | 0.7563 | 661.0 | 0.8885 | 0.8546 | 0.8455 | 0.8497 | 10000.0 | 0.8885 | 0.8885 | 0.8884 | 10000.0 | | 0.1592 | 12.0 | 16248 | 0.2357 | 0.8904 | 0.8895 | 0.8899 | 4036.0 | 0.9351 | 0.9348 | 0.9349 | 3681.0 | 0.7989 | 0.8033 | 0.8011 | 1622.0 | 0.7382 | 0.7337 | 0.7360 | 661.0 | 0.8819 | 0.8406 | 0.8403 | 0.8405 | 10000.0 | 0.8819 | 0.8819 | 0.8819 | 10000.0 | | 0.1783 | 13.0 | 17602 | 0.2415 | 0.8879 | 0.8972 | 0.8925 | 4036.0 | 0.9355 | 0.9378 | 0.9366 | 3681.0 | 0.8083 | 0.7928 | 0.8005 | 1622.0 | 0.7598 | 0.7368 | 0.7481 | 661.0 | 0.8846 | 0.8479 | 0.8411 | 0.8444 | 10000.0 | 0.8841 | 0.8846 | 0.8843 | 10000.0 | | 0.1395 | 14.0 | 18956 | 0.2466 | 0.8951 | 0.8905 | 0.8928 | 4036.0 | 0.9340 | 0.9343 | 0.9341 | 3681.0 | 0.8029 | 0.8064 | 0.8047 | 1622.0 | 0.7374 | 0.7519 | 0.7446 | 661.0 | 0.8838 | 0.8424 | 0.8458 | 0.8440 | 10000.0 | 0.8841 | 0.8838 | 0.8839 | 10000.0 | ### Framework versions - Transformers 4.47.1 - Pytorch 2.5.1+cu124 - Datasets 3.0.1 - Tokenizers 0.21.0