--- tags: - generated_from_trainer model-index: - name: DNADebertaK6f results: [] --- # DNADebertaK6f This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset. It achieves the following results on the evaluation set: - Loss: 1.3707 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 150 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:--------:|:---------------:| | 2.1545 | 1.24 | 100000 | 1.5895 | | 1.5727 | 2.49 | 200000 | 1.5383 | | 1.5368 | 3.73 | 300000 | 1.5117 | | 1.5143 | 4.97 | 400000 | 1.4926 | | 1.4969 | 6.22 | 500000 | 1.4789 | | 1.4841 | 7.46 | 600000 | 1.4677 | | 1.475 | 8.7 | 700000 | 1.4599 | | 1.4677 | 9.95 | 800000 | 1.4533 | | 1.4623 | 11.19 | 900000 | 1.4491 | | 1.4576 | 12.43 | 1000000 | 1.4461 | | 1.4544 | 13.68 | 1100000 | 1.4420 | | 1.451 | 14.92 | 1200000 | 1.4381 | | 1.4482 | 16.16 | 1300000 | 1.4359 | | 1.446 | 17.4 | 1400000 | 1.4345 | | 1.443 | 18.65 | 1500000 | 1.4320 | | 1.4412 | 19.89 | 1600000 | 1.4295 | | 1.4385 | 21.13 | 1700000 | 1.4278 | | 1.4368 | 22.38 | 1800000 | 1.4249 | | 1.4346 | 23.62 | 1900000 | 1.4237 | | 1.433 | 24.86 | 2000000 | 1.4219 | | 1.4315 | 26.11 | 2100000 | 1.4201 | | 1.4297 | 27.35 | 2200000 | 1.4198 | | 1.4282 | 28.59 | 2300000 | 1.4180 | | 1.4266 | 29.84 | 2400000 | 1.4142 | | 1.4253 | 31.08 | 2500000 | 1.4146 | | 1.4238 | 32.32 | 2600000 | 1.4130 | | 1.4228 | 33.57 | 2700000 | 1.4113 | | 1.4221 | 34.81 | 2800000 | 1.4100 | | 1.42 | 36.05 | 2900000 | 1.4097 | | 1.4188 | 37.3 | 3000000 | 1.4085 | | 1.4174 | 38.54 | 3100000 | 1.4067 | | 1.4161 | 39.78 | 3200000 | 1.4064 | | 1.4149 | 41.03 | 3300000 | 1.4058 | | 1.4139 | 42.27 | 3400000 | 1.4024 | | 1.4134 | 43.51 | 3500000 | 1.4022 | | 1.4126 | 44.76 | 3600000 | 1.4025 | | 1.4117 | 46.0 | 3700000 | 1.4015 | | 1.411 | 47.24 | 3800000 | 1.4001 | | 1.4098 | 48.49 | 3900000 | 1.3968 | | 1.4096 | 49.73 | 4000000 | 1.3997 | | 1.4089 | 50.97 | 4100000 | 1.3974 | | 1.4084 | 52.21 | 4200000 | 1.3972 | | 1.4072 | 53.46 | 4300000 | 1.3965 | | 1.4066 | 54.7 | 4400000 | 1.3974 | | 1.4062 | 55.94 | 4500000 | 1.3960 | | 1.4058 | 57.19 | 4600000 | 1.3958 | | 1.4053 | 58.43 | 4700000 | 1.3950 | | 1.4041 | 59.67 | 4800000 | 1.3936 | | 1.4041 | 60.92 | 4900000 | 1.3963 | | 1.4031 | 62.16 | 5000000 | 1.3915 | | 1.4023 | 63.4 | 5100000 | 1.3917 | | 1.4022 | 64.65 | 5200000 | 1.3930 | | 1.4017 | 65.89 | 5300000 | 1.3904 | | 1.4009 | 67.13 | 5400000 | 1.3899 | | 1.4007 | 68.38 | 5500000 | 1.3892 | | 1.3997 | 69.62 | 5600000 | 1.3910 | | 1.3996 | 70.86 | 5700000 | 1.3892 | | 1.3991 | 72.11 | 5800000 | 1.3890 | | 1.3983 | 73.35 | 5900000 | 1.3870 | | 1.3985 | 74.59 | 6000000 | 1.3889 | | 1.3975 | 75.84 | 6100000 | 1.3865 | | 1.3973 | 77.08 | 6200000 | 1.3852 | | 1.3969 | 78.32 | 6300000 | 1.3869 | | 1.3964 | 79.57 | 6400000 | 1.3843 | | 1.396 | 80.81 | 6500000 | 1.3853 | | 1.3955 | 82.05 | 6600000 | 1.3844 | | 1.3952 | 83.3 | 6700000 | 1.3863 | | 1.395 | 84.54 | 6800000 | 1.3835 | | 1.3948 | 85.78 | 6900000 | 1.3841 | | 1.394 | 87.02 | 7000000 | 1.3850 | | 1.3934 | 88.27 | 7100000 | 1.3827 | | 1.3932 | 89.51 | 7200000 | 1.3830 | | 1.3929 | 90.75 | 7300000 | 1.3821 | | 1.392 | 92.0 | 7400000 | 1.3820 | | 1.392 | 93.24 | 7500000 | 1.3837 | | 1.3913 | 94.48 | 7600000 | 1.3817 | | 1.3909 | 95.73 | 7700000 | 1.3836 | | 1.3906 | 96.97 | 7800000 | 1.3811 | | 1.3903 | 98.21 | 7900000 | 1.3806 | | 1.3902 | 99.46 | 8000000 | 1.3807 | | 1.3896 | 100.7 | 8100000 | 1.3804 | | 1.3895 | 101.94 | 8200000 | 1.3805 | | 1.3891 | 103.19 | 8300000 | 1.3821 | | 1.3889 | 104.43 | 8400000 | 1.3833 | | 1.3881 | 105.67 | 8500000 | 1.3788 | | 1.388 | 106.92 | 8600000 | 1.3818 | | 1.3876 | 108.16 | 8700000 | 1.3806 | | 1.387 | 109.4 | 8800000 | 1.3766 | | 1.387 | 110.65 | 8900000 | 1.3765 | | 1.3865 | 111.89 | 9000000 | 1.3800 | | 1.3864 | 113.13 | 9100000 | 1.3830 | | 1.386 | 114.38 | 9200000 | 1.3770 | | 1.3853 | 115.62 | 9300000 | 1.3771 | | 1.3852 | 116.86 | 9400000 | 1.3772 | | 1.3848 | 118.1 | 9500000 | 1.3771 | | 1.384 | 119.35 | 9600000 | 1.3749 | | 1.3843 | 120.59 | 9700000 | 1.3764 | | 1.3836 | 121.83 | 9800000 | 1.3802 | | 1.3833 | 123.08 | 9900000 | 1.3756 | | 1.3831 | 124.32 | 10000000 | 1.3748 | | 1.3821 | 125.56 | 10100000 | 1.3755 | | 1.3817 | 126.81 | 10200000 | 1.3744 | | 1.3819 | 128.05 | 10300000 | 1.3763 | | 1.381 | 129.29 | 10400000 | 1.3743 | | 1.3805 | 130.54 | 10500000 | 1.3762 | | 1.3804 | 131.78 | 10600000 | 1.3725 | | 1.3795 | 133.02 | 10700000 | 1.3753 | | 1.3791 | 134.27 | 10800000 | 1.3780 | | 1.3785 | 135.51 | 10900000 | 1.3749 | | 1.3781 | 136.75 | 11000000 | 1.3749 | | 1.3779 | 138.0 | 11100000 | 1.3737 | | 1.3772 | 139.24 | 11200000 | 1.3715 | | 1.3763 | 140.48 | 11300000 | 1.3759 | | 1.3761 | 141.73 | 11400000 | 1.3745 | | 1.3752 | 142.97 | 11500000 | 1.3708 | | 1.3744 | 144.21 | 11600000 | 1.3735 | | 1.3736 | 145.46 | 11700000 | 1.3720 | | 1.3728 | 146.7 | 11800000 | 1.3702 | | 1.3714 | 147.94 | 11900000 | 1.3696 | | 1.3706 | 149.19 | 12000000 | 1.3707 | ### Framework versions - Transformers 4.27.3 - Pytorch 2.0.0 - Datasets 2.10.1 - Tokenizers 0.13.2