End of training

README.md

@@ -12,12 +12,10 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/noc-lab/PMC_LLAMA_7B_trainer_Wiki_lora/runs/fm9skl5h)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/noc-lab/PMC_LLAMA_7B_trainer_Wiki_lora/runs/dtbm1bv9)
 # PMC_LLAMA_7B_trainer_Wiki_lora
 
 This model is a fine-tuned version of [chaoyi-wu/PMC_LLAMA_7B](https://huggingface.co/chaoyi-wu/PMC_LLAMA_7B) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 1.8246
 
 ## Model description
 
@@ -49,9 +47,6 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.0899        | 0.9985 | 569  | 1.8246          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,8 +20,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj"
+    "v_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6ebe2593d39d58f23a485a5ecd8f98e8bb84266885cecf02ca59b21d6da1bc65
+oid sha256:010bf851e8d728cef779c2aa76071dc6d5dfba86fae2562e857b090b21a0401d
 size 16794200

trainer_peft.log

@@ -279,3 +279,35 @@
 2024-05-30 02:08 - Setup PEFT
 2024-05-30 02:08 - Setup optimizer
 2024-05-30 02:08 - Start training
+2024-05-30 17:35 - Training complete!!!
+2024-05-30 19:45 - Cuda check
+2024-05-30 19:45 - True
+2024-05-30 19:45 - 1
+2024-05-30 19:45 - Configue Model and tokenizer
+2024-05-30 19:45 - Memory usage in   25.17 GB
+2024-05-30 19:45 - Dataset loaded successfully:
+ train-Jingmei/Pandemic_ECDC
+ test -Jingmei/Pandemic_WHO
+2024-05-30 19:46 - Tokenize data: DatasetDict({
+    train: Dataset({
+        features: ['input_ids', 'attention_mask'],
+        num_rows: 7008
+    })
+    test: Dataset({
+        features: ['input_ids', 'attention_mask'],
+        num_rows: 8264
+    })
+})
+2024-05-30 19:49 - Split data into chunks:DatasetDict({
+    train: Dataset({
+        features: ['input_ids', 'attention_mask'],
+        num_rows: 103936
+    })
+    test: Dataset({
+        features: ['input_ids', 'attention_mask'],
+        num_rows: 198964
+    })
+})
+2024-05-30 19:49 - Setup PEFT
+2024-05-30 19:49 - Setup optimizer
+2024-05-30 19:49 - Start training

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e87c7f15854d6ac2a1b9f59debb7097d4a1af8482eadaafa4a458af315f2cd5a
+oid sha256:8fb92caab4943784c256b92dc8713b57b617869aef8b999c4479dbbfa339daa4
 size 5176