End of training

README.md

@@ -0,0 +1,162 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model: Qwen/Qwen2-0.5B
+tags:
+- trl
+- reward-trainer
+- generated_from_trainer
+metrics:
+- accuracy
+model-index:
+- name: reward-model
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# reward-model
+
+This model is a fine-tuned version of [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5182
+- Accuracy: 0.73
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 8
+- total_train_batch_size: 64
+- total_eval_batch_size: 64
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 1.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:--------:|
+| 0.7794        | 0.0103 | 10   | 0.6609          | 0.603    |
+| 0.6484        | 0.0206 | 20   | 0.6517          | 0.639    |
+| 0.6565        | 0.0310 | 30   | 0.6356          | 0.626    |
+| 0.6522        | 0.0413 | 40   | 0.6274          | 0.649    |
+| 0.6354        | 0.0516 | 50   | 0.6114          | 0.665    |
+| 0.6337        | 0.0619 | 60   | 0.6120          | 0.674    |
+| 0.6133        | 0.0722 | 70   | 0.5871          | 0.677    |
+| 0.6075        | 0.0826 | 80   | 0.5823          | 0.685    |
+| 0.5795        | 0.0929 | 90   | 0.5812          | 0.686    |
+| 0.585         | 0.1032 | 100  | 0.5759          | 0.691    |
+| 0.5664        | 0.1135 | 110  | 0.5713          | 0.687    |
+| 0.5772        | 0.1238 | 120  | 0.5619          | 0.692    |
+| 0.5762        | 0.1342 | 130  | 0.5701          | 0.701    |
+| 0.5726        | 0.1445 | 140  | 0.5550          | 0.707    |
+| 0.5889        | 0.1548 | 150  | 0.5864          | 0.678    |
+| 0.5758        | 0.1651 | 160  | 0.5623          | 0.702    |
+| 0.5721        | 0.1754 | 170  | 0.5510          | 0.708    |
+| 0.5595        | 0.1858 | 180  | 0.5458          | 0.707    |
+| 0.5601        | 0.1961 | 190  | 0.5531          | 0.709    |
+| 0.5646        | 0.2064 | 200  | 0.5473          | 0.702    |
+| 0.5467        | 0.2167 | 210  | 0.5586          | 0.704    |
+| 0.5558        | 0.2270 | 220  | 0.5539          | 0.7      |
+| 0.5457        | 0.2374 | 230  | 0.5744          | 0.686    |
+| 0.5848        | 0.2477 | 240  | 0.5571          | 0.712    |
+| 0.5478        | 0.2580 | 250  | 0.5432          | 0.701    |
+| 0.5461        | 0.2683 | 260  | 0.5447          | 0.708    |
+| 0.5607        | 0.2786 | 270  | 0.5433          | 0.711    |
+| 0.5886        | 0.2890 | 280  | 0.5378          | 0.716    |
+| 0.5616        | 0.2993 | 290  | 0.5354          | 0.724    |
+| 0.5426        | 0.3096 | 300  | 0.5327          | 0.722    |
+| 0.5553        | 0.3199 | 310  | 0.5358          | 0.728    |
+| 0.5335        | 0.3302 | 320  | 0.5259          | 0.732    |
+| 0.5354        | 0.3406 | 330  | 0.5277          | 0.725    |
+| 0.5437        | 0.3509 | 340  | 0.5344          | 0.727    |
+| 0.5317        | 0.3612 | 350  | 0.5372          | 0.718    |
+| 0.5609        | 0.3715 | 360  | 0.5422          | 0.715    |
+| 0.5522        | 0.3818 | 370  | 0.5349          | 0.721    |
+| 0.5714        | 0.3922 | 380  | 0.5329          | 0.728    |
+| 0.5099        | 0.4025 | 390  | 0.5296          | 0.719    |
+| 0.5148        | 0.4128 | 400  | 0.5337          | 0.721    |
+| 0.5366        | 0.4231 | 410  | 0.5308          | 0.733    |
+| 0.5432        | 0.4334 | 420  | 0.5270          | 0.733    |
+| 0.5431        | 0.4438 | 430  | 0.5318          | 0.725    |
+| 0.5531        | 0.4541 | 440  | 0.5364          | 0.728    |
+| 0.5585        | 0.4644 | 450  | 0.5436          | 0.731    |
+| 0.5275        | 0.4747 | 460  | 0.5348          | 0.721    |
+| 0.553         | 0.4850 | 470  | 0.5315          | 0.728    |
+| 0.5152        | 0.4954 | 480  | 0.5309          | 0.721    |
+| 0.5405        | 0.5057 | 490  | 0.5332          | 0.721    |
+| 0.5089        | 0.5160 | 500  | 0.5339          | 0.727    |
+| 0.5334        | 0.5263 | 510  | 0.5321          | 0.725    |
+| 0.535         | 0.5366 | 520  | 0.5342          | 0.719    |
+| 0.5228        | 0.5470 | 530  | 0.5327          | 0.725    |
+| 0.5384        | 0.5573 | 540  | 0.5316          | 0.713    |
+| 0.5306        | 0.5676 | 550  | 0.5274          | 0.728    |
+| 0.5183        | 0.5779 | 560  | 0.5230          | 0.724    |
+| 0.5205        | 0.5882 | 570  | 0.5259          | 0.723    |
+| 0.5152        | 0.5986 | 580  | 0.5282          | 0.718    |
+| 0.5266        | 0.6089 | 590  | 0.5236          | 0.716    |
+| 0.5141        | 0.6192 | 600  | 0.5223          | 0.721    |
+| 0.5527        | 0.6295 | 610  | 0.5169          | 0.727    |
+| 0.5037        | 0.6398 | 620  | 0.5172          | 0.733    |
+| 0.5305        | 0.6502 | 630  | 0.5174          | 0.733    |
+| 0.5277        | 0.6605 | 640  | 0.5217          | 0.724    |
+| 0.5157        | 0.6708 | 650  | 0.5191          | 0.725    |
+| 0.4997        | 0.6811 | 660  | 0.5216          | 0.723    |
+| 0.5298        | 0.6914 | 670  | 0.5262          | 0.716    |
+| 0.509         | 0.7018 | 680  | 0.5245          | 0.723    |
+| 0.5216        | 0.7121 | 690  | 0.5199          | 0.727    |
+| 0.5012        | 0.7224 | 700  | 0.5175          | 0.724    |
+| 0.544         | 0.7327 | 710  | 0.5200          | 0.722    |
+| 0.513         | 0.7430 | 720  | 0.5202          | 0.723    |
+| 0.5153        | 0.7534 | 730  | 0.5220          | 0.721    |
+| 0.5065        | 0.7637 | 740  | 0.5232          | 0.721    |
+| 0.5291        | 0.7740 | 750  | 0.5230          | 0.721    |
+| 0.5506        | 0.7843 | 760  | 0.5203          | 0.717    |
+| 0.517         | 0.7946 | 770  | 0.5211          | 0.719    |
+| 0.5102        | 0.8050 | 780  | 0.5212          | 0.718    |
+| 0.4861        | 0.8153 | 790  | 0.5225          | 0.715    |
+| 0.5407        | 0.8256 | 800  | 0.5236          | 0.718    |
+| 0.5067        | 0.8359 | 810  | 0.5254          | 0.718    |
+| 0.5207        | 0.8462 | 820  | 0.5269          | 0.718    |
+| 0.5134        | 0.8566 | 830  | 0.5256          | 0.723    |
+| 0.5116        | 0.8669 | 840  | 0.5241          | 0.724    |
+| 0.5256        | 0.8772 | 850  | 0.5235          | 0.723    |
+| 0.5233        | 0.8875 | 860  | 0.5222          | 0.727    |
+| 0.4895        | 0.8978 | 870  | 0.5194          | 0.727    |
+| 0.4877        | 0.9082 | 880  | 0.5185          | 0.725    |
+| 0.5299        | 0.9185 | 890  | 0.5194          | 0.725    |
+| 0.4815        | 0.9288 | 900  | 0.5196          | 0.725    |
+| 0.477         | 0.9391 | 910  | 0.5204          | 0.728    |
+| 0.5368        | 0.9494 | 920  | 0.5199          | 0.73     |
+| 0.5133        | 0.9598 | 930  | 0.5184          | 0.733    |
+| 0.4974        | 0.9701 | 940  | 0.5183          | 0.731    |
+| 0.4948        | 0.9804 | 950  | 0.5183          | 0.728    |
+| 0.5217        | 0.9907 | 960  | 0.5182          | 0.73     |
+
+
+### Framework versions
+
+- Transformers 4.44.2
+- Pytorch 2.4.0+cu121
+- Datasets 2.21.0
+- Tokenizers 0.19.1

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

config.json

@@ -0,0 +1,35 @@
+{
+  "_name_or_path": "Qwen/Qwen2-0.5B",
+  "architectures": [
+    "Qwen2ForSequenceClassification"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "hidden_act": "silu",
+  "hidden_size": 896,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4864,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "max_position_embeddings": 131072,
+  "max_window_layers": 24,
+  "model_type": "qwen2",
+  "num_attention_heads": 14,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d59b1043a328078221fdec071b7c48249b6d2577f39e3277398f29d4893d3ece
+size 1976167144

special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6666915e34b0d49c1e937d28940b9d8ebbbf3ffaf7ad86d949fca19c463a46cc
+size 5240