Model save

README.md

@@ -1,36 +1,68 @@
 ---
-{}
+tags:
+- generated_from_trainer
+datasets:
+- squad_v2
+model-index:
+- name: distilbert-finetuned-uncased-squad_v2
+  results: []
 ---
 
-    ---
-    language: en
-    license: mit
-    ---
-
-    # distilbert-finetuned-uncased Model
-
-    This model is fine-tuned on SQuAD v2 for question answering tasks.
-
-    ## Training Procedure
-    - Number of Epochs: 4
-    - Learning Rate: 2e-05
-    - Batch Size: 128 (per device)
-    - Evaluation Strategy: Every 100 steps
-    - Save Strategy: Every 100 steps
-    - FP16 Training: Yes
-
-    ## Evaluation Results
-    - Exact Match: 23.347090036216628
-    - F1 Score: 26.869992349988973
-    - Total: 11873
-    - Has Answer Exact: 38.630229419703106
-    - Has Answer F1: 45.686136837283904
-    - Has Answer Total: 5928
-    - No Answer Exact: 8.107653490328007
-    - No Answer F1: 8.107653490328007
-    - No Answer Total: 5945
-    - Best Exact: 50.11370336056599
-    - Best Exact Threshold: 0.0
-    - Best F1: 50.11370336056599
-    - Best F1 Threshold: 0.0
-    
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# distilbert-finetuned-uncased-squad_v2
+
+This model was trained from scratch on the squad_v2 dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.3930
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 128
+- eval_batch_size: 128
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 512
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 3.6437        | 0.39  | 100  | 2.1780          |
+| 2.1596        | 0.78  | 200  | 1.6557          |
+| 1.8138        | 1.18  | 300  | 1.5683          |
+| 1.6987        | 1.57  | 400  | 1.5076          |
+| 1.6586        | 1.96  | 500  | 1.5350          |
+| 1.5957        | 1.18  | 600  | 1.4431          |
+| 1.5825        | 1.37  | 700  | 1.4955          |
+| 1.5523        | 1.57  | 800  | 1.4444          |
+| 1.5346        | 1.76  | 900  | 1.3930          |
+| 1.5098        | 1.96  | 1000 | 1.4285          |
+
+
+### Framework versions
+
+- Transformers 4.34.1
+- Pytorch 2.1.0+cu118
+- Datasets 2.14.5
+- Tokenizers 0.14.1

repo_card.md

@@ -6,10 +6,10 @@
     language: en
     license: mit
     ---
-    
+
     # distilbert-finetuned-uncased Model
 
-    This model is fine-tuned on the SQuAD v2 dataset for the task of question answering.
+    This model is fine-tuned on SQuAD v2 for question answering tasks.
 
     ## Training Procedure
     - Number of Epochs: 4

tokenizer.json

@@ -3,13 +3,11 @@
   "truncation": {
     "direction": "Right",
     "max_length": 512,
-    "strategy": "LongestFirst",
-    "stride": 0
+    "strategy": "OnlySecond",
+    "stride": 128
   },
   "padding": {
-    "strategy": {
-      "Fixed": 512
-    },
+    "strategy": "BatchLongest",
     "direction": "Right",
     "pad_to_multiple_of": null,
     "pad_id": 0,

trainer_state.json

@@ -153,15 +153,15 @@
       "step": 1020,
       "total_flos": 5.148633647651021e+16,
       "train_loss": 0.028946983113008386,
-      "train_runtime": 27.4849,
-      "train_samples_per_second": 18992.679,
-      "train_steps_per_second": 37.111
+      "train_runtime": 26.7453,
+      "train_samples_per_second": 19517.888,
+      "train_steps_per_second": 38.138
     },
     {
       "epoch": 4.0,
       "eval_loss": 1.3930128812789917,
-      "eval_runtime": 8.2728,
-      "eval_samples_per_second": 1446.798,
+      "eval_runtime": 8.2725,
+      "eval_samples_per_second": 1446.843,
       "eval_steps_per_second": 11.363,
       "step": 1020
     }