End of training

README.md

@@ -1,5 +1,6 @@
 ---
 license: apache-2.0
+base_model: allenai/longformer-base-4096
 tags:
 - generated_from_trainer
 metrics:
@@ -19,11 +20,11 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.1860
-- Accuracy: 0.8377
-- F1: 0.9093
-- Precision: 0.8493
-- Recall: 0.9785
+- Loss: 0.4961
+- Accuracy: 0.9068
+- F1: 0.9452
+- Precision: 0.9265
+- Recall: 0.9646
 
 ## Model description
 
@@ -42,27 +43,33 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
+- learning_rate: 2e-05
 - train_batch_size: 1
-- eval_batch_size: 2
+- eval_batch_size: 1
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3
-- mixed_precision_training: Native AMP
+- num_epochs: 10
 
 ### Training results
 
 | Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     | Precision | Recall |
 |:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:---------:|:------:|
-| 0.0053        | 1.0   | 5152  | 1.4254          | 0.8315   | 0.9080 | 0.8315    | 1.0    |
-| 0.0107        | 2.0   | 10304 | 1.2232          | 0.8315   | 0.9080 | 0.8315    | 1.0    |
-| 1.2863        | 3.0   | 15456 | 1.1860          | 0.8377   | 0.9093 | 0.8493    | 0.9785 |
+| 0.8485        | 1.0   | 5152  | 0.8574          | 0.8323   | 0.9085 | 0.8323    | 1.0    |
+| 0.7968        | 2.0   | 10304 | 0.8441          | 0.8323   | 0.9085 | 0.8323    | 1.0    |
+| 0.847         | 3.0   | 15456 | 0.8049          | 0.8323   | 0.9085 | 0.8323    | 1.0    |
+| 0.8677        | 4.0   | 20608 | 0.7919          | 0.8323   | 0.9085 | 0.8323    | 1.0    |
+| 0.8778        | 5.0   | 25760 | 0.8980          | 0.8323   | 0.9085 | 0.8323    | 1.0    |
+| 0.7563        | 6.0   | 30912 | 0.8299          | 0.8323   | 0.9085 | 0.8323    | 1.0    |
+| 0.661         | 7.0   | 36064 | 0.6065          | 0.8882   | 0.9357 | 0.8973    | 0.9776 |
+| 0.8207        | 8.0   | 41216 | 0.5387          | 0.8975   | 0.9410 | 0.9038    | 0.9813 |
+| 0.6872        | 9.0   | 46368 | 0.5960          | 0.8602   | 0.9212 | 0.8680    | 0.9813 |
+| 0.4596        | 10.0  | 51520 | 0.4961          | 0.9068   | 0.9452 | 0.9265    | 0.9646 |
 
 
 ### Framework versions
 
-- Transformers 4.27.4
-- Pytorch 2.2.0+cu121
-- Datasets 2.18.0
-- Tokenizers 0.13.2
+- Transformers 4.32.0.dev0
+- Pytorch 2.0.1
+- Datasets 2.14.4
+- Tokenizers 0.13.3

config.json

@@ -43,11 +43,10 @@
   "num_hidden_layers": 12,
   "onnx_export": false,
   "pad_token_id": 1,
-  "position_embedding_type": "absolute",
   "problem_type": "single_label_classification",
   "sep_token_id": 2,
   "torch_dtype": "float32",
-  "transformers_version": "4.27.4",
+  "transformers_version": "4.32.0.dev0",
   "type_vocab_size": 1,
   "vocab_size": 50265
 }

merges.txt

@@ -1,4 +1,4 @@
-#version: 0.2 - Trained by `huggingface/tokenizers`
+#version: 0.2
 Ġ t
 Ġ a
 h e

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:178624f5e416d2324eb092aca88314c7e41c27ae3cd6683746e1fa059dbcf8d8
-size 594739726
+oid sha256:f743a83cf538002482f27db4cbfc382cb110c6dcd8ecc9a5125adb8a051f37e8
+size 594739281

tokenizer.json

@@ -82,6 +82,7 @@
     "continuing_subword_prefix": "",
     "end_of_word_suffix": "",
     "fuse_unk": false,
+    "byte_fallback": false,
     "vocab": {
       "<s>": 0,
       "<pad>": 1,

tokenizer_config.json

@@ -1,6 +1,7 @@
 {
   "add_prefix_space": false,
   "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
   "cls_token": "<s>",
   "eos_token": "</s>",
   "errors": "replace",
@@ -9,7 +10,6 @@
   "model_max_length": 4096,
   "pad_token": "<pad>",
   "sep_token": "</s>",
-  "special_tokens_map_file": null,
   "tokenizer_class": "LongformerTokenizer",
   "trim_offsets": true,
   "unk_token": "<unk>"

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:01c47c7ef56337c9b0d198aefe3555b2889a6b6917880f0aec799c150894cb3d
-size 3960
+oid sha256:4788a8133052e4ee70ab0ba139a04d730dc16b230624e8e6d542478eabdc68b6
+size 4091