--- license: other tags: - generated_from_trainer model-index: - name: EnabledChat-v2-lora results: [] --- <!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. --> # EnabledChat-v2-lora This model is a fine-tuned version of [0x70DA/EnabledChat](https://huggingface.co/0x70DA/EnabledChat) on the None dataset. It achieves the following results on the evaluation set: - Loss: 1.2828 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0002 - train_batch_size: 16 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 100 - num_epochs: 2 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | 1.3247 | 0.07 | 200 | 1.3711 | | 1.4053 | 0.14 | 400 | 1.3470 | | 1.3431 | 0.21 | 600 | 1.3355 | | 1.4012 | 0.27 | 800 | 1.3278 | | 1.3189 | 0.34 | 1000 | 1.3235 | | 1.3581 | 0.41 | 1200 | 1.3181 | | 1.3417 | 0.48 | 1400 | 1.3138 | | 1.3111 | 0.55 | 1600 | 1.3110 | | 1.3547 | 0.62 | 1800 | 1.3068 | | 1.3725 | 0.68 | 2000 | 1.3058 | | 1.3574 | 0.75 | 2200 | 1.3028 | | 1.3773 | 0.82 | 2400 | 1.2999 | | 1.2944 | 0.89 | 2600 | 1.2979 | | 1.2433 | 0.96 | 2800 | 1.2962 | | 1.2349 | 1.03 | 3000 | 1.2952 | | 1.3126 | 1.1 | 3200 | 1.2934 | | 1.284 | 1.16 | 3400 | 1.2925 | | 1.2194 | 1.23 | 3600 | 1.2912 | | 1.2885 | 1.3 | 3800 | 1.2902 | | 1.2159 | 1.37 | 4000 | 1.2888 | | 1.2832 | 1.44 | 4200 | 1.2881 | | 1.2233 | 1.51 | 4400 | 1.2868 | | 1.2565 | 1.57 | 4600 | 1.2857 | | 1.2618 | 1.64 | 4800 | 1.2855 | | 1.3954 | 1.71 | 5000 | 1.2846 | | 1.3082 | 1.78 | 5200 | 1.2840 | | 1.3149 | 1.85 | 5400 | 1.2834 | | 1.2558 | 1.92 | 5600 | 1.2830 | | 1.2299 | 1.98 | 5800 | 1.2828 | ### Framework versions - Transformers 4.29.2 - Pytorch 2.0.0 - Datasets 2.11.0 - Tokenizers 0.13.3