Model save

README.md

@@ -1,8 +1,13 @@
 ---
-base_model: TinyPixel/small-llama2
 library_name: peft
+base_model: TinyPixel/small-llama2
 tags:
 - generated_from_trainer
+metrics:
+- accuracy
+- precision
+- recall
+- f1
 model-index:
 - name: debug_test
   results: []
@@ -14,6 +19,12 @@ should probably proofread and complete it, then remove this comment. -->
 # debug_test
 
 This model is a fine-tuned version of [TinyPixel/small-llama2](https://huggingface.co/TinyPixel/small-llama2) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.7894
+- Accuracy: 0.4982
+- Precision: 0.3939
+- Recall: 0.7114
+- F1: 0.5071
 
 ## Model description
 
@@ -33,27 +44,30 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2e-05
-- train_batch_size: 5
-- eval_batch_size: 5
+- train_batch_size: 8
+- eval_batch_size: 8
 - seed: 42
 - distributed_type: multi-GPU
 - num_devices: 4
-- gradient_accumulation_steps: 5
-- total_train_batch_size: 100
-- total_eval_batch_size: 20
-- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 64
+- total_eval_batch_size: 32
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.1
 - num_epochs: 1
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|
+| 0.8214        | 1.0   | 5    | 0.7894          | 0.4982   | 0.3939    | 0.7114 | 0.5071 |
 
 
 ### Framework versions
 
-- PEFT 0.12.0
+- PEFT 0.13.2
 - Transformers 4.46.0
-- Pytorch 2.4.0+cu118
-- Datasets 3.0.0
-- Tokenizers 0.20.1
+- Pytorch 2.5.1+cu124
+- Datasets 3.1.0
+- Tokenizers 0.20.3

adapter_config.json

@@ -14,10 +14,7 @@
   "lora_dropout": 0.05,
   "megatron_config": null,
   "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "classifier",
-    "score"
-  ],
+  "modules_to_save": null,
   "peft_type": "LORA",
   "r": 16,
   "rank_pattern": {},
@@ -26,7 +23,7 @@
     "v_proj",
     "q_proj"
   ],
-  "task_type": "TOKEN_CLS",
+  "task_type": "CAUSAL_LM",
   "use_dora": false,
   "use_rslora": false
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:295fbb6912088e76227fdd91a6b85ca49225304bb0ef961fa0ec23e73b1aef67
-size 3160464
+oid sha256:de830bbf06fd72c2f571570e38cad20ccd7f037c099ab1023751fe4f4abc47e2
+size 3152080

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cbfa9dbe3b714e5e01f73c5e480df616663aca49b5819d36459e038d0a510ef7
+oid sha256:7104657a3a775262b9f821d30163b8cb0a53129465c1e5c5d23a3645ea1cf4fa
 size 5240