llamantino7b_question_answering_finetuining

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [swap-uniba/LLaMAntino-2-7b-hf-ITA](https://huggingface.co/swap-uniba/LLaMAntino-2-7b-hf-ITA) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.2277
+- Loss: 1.4340
 
 ## Model description
 
@@ -39,27 +39,19 @@ More information needed
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
 - train_batch_size: 4
-- eval_batch_size: 4
+- eval_batch_size: 1
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.03
-- num_epochs: 10
+- num_epochs: 2
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.6498        | 1.0   | 90   | 0.0351          |
-| 1.1895        | 2.0   | 180  | 0.0237          |
-| 0.8519        | 3.0   | 270  | 0.0219          |
-| 0.5762        | 4.0   | 360  | 0.0155          |
-| 0.3852        | 5.0   | 450  | 0.0987          |
-| 0.2445        | 6.0   | 540  | 0.1373          |
-| 0.1554        | 7.0   | 630  | 0.2655          |
-| 0.0989        | 8.0   | 720  | 0.0678          |
-| 0.0607        | 9.0   | 810  | 0.1935          |
-| 0.0397        | 10.0  | 900  | 0.2277          |
+| 1.4152        | 1.0   | 3    | 1.4624          |
+| 1.3209        | 2.0   | 6    | 1.4340          |
 
 
 ### Framework versions

adapter_config.json

@@ -19,8 +19,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj"
+    "v_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_rslora": false

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5cca94709411e6f43a71c53735a3796e0415d3643f0c62de07cddeba3c06cbea
+oid sha256:eb8e6c929e9d22e8c7f5910f58212d2876b080e3e4b8ae99d32e569eaba24e59
 size 134235048

tokenizer.json

@@ -1,6 +1,11 @@
 {
   "version": "1.0",
-  "truncation": null,
+  "truncation": {
+    "direction": "Right",
+    "max_length": 1024,
+    "strategy": "LongestFirst",
+    "stride": 0
+  },
   "padding": null,
   "added_tokens": [
     {

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:de3f6ccc8a4fe4d9d7af9f98b6ced1b4b4aa7db3a8e15dde4f62a932939aaecf
+oid sha256:88218a9037911cdc6f26e5076c993ebc2d7b2abe368fd7f6168160042fa72cb1
 size 4411