Hugging Face's logo

gokulsrinivasagan
/
gpt_train_12_256

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
gpt_train_12_256 / README.md
gokulsrinivasagan's picture
gokulsrinivasagan
End of training
1907629 verified
preview code
|
raw
history blame contribute delete
19 kB
metadata
license: mit
base_model: openai-community/gpt2
tags:
  - generated_from_trainer
datasets:
  - gokuls/wiki_book_corpus_raw_dataset_tiny
metrics:
  - accuracy
model-index:
  - name: gpt_train_12_256
    results:
      - task:
          name: Causal Language Modeling
          type: text-generation
        dataset:
          name: gokuls/wiki_book_corpus_raw_dataset_tiny
          type: gokuls/wiki_book_corpus_raw_dataset_tiny
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.08778952977191952

gpt_train_12_256

This model is a fine-tuned version of openai-community/gpt2 on the gokuls/wiki_book_corpus_raw_dataset_tiny dataset. It achieves the following results on the evaluation set:

  • Loss: 9.6016
  • Accuracy: 0.0878

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 36
  • eval_batch_size: 36
  • seed: 10
  • distributed_type: multi-GPU
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
10.875 0.0001 1 10.875 0.0031
10.875 0.0001 2 10.875 0.0031
10.875 0.0002 3 10.875 0.0031
10.875 0.0002 4 10.875 0.0031
10.8672 0.0003 5 10.875 0.0031
10.875 0.0003 6 10.875 0.0031
10.8672 0.0004 7 10.875 0.0031
10.875 0.0004 8 10.875 0.0031
10.875 0.0005 9 10.875 0.0031
10.875 0.0005 10 10.875 0.0031
10.875 0.0006 11 10.875 0.0031
10.875 0.0007 12 10.875 0.0031
10.875 0.0007 13 10.875 0.0031
10.875 0.0008 14 10.875 0.0031
10.875 0.0008 15 10.875 0.0031
10.875 0.0009 16 10.875 0.0031
10.8672 0.0009 17 10.875 0.0031
10.875 0.0010 18 10.8047 0.0103
10.8125 0.0010 19 10.75 0.0119
10.7578 0.0011 20 10.6953 0.0180
10.7188 0.0011 21 10.6562 0.0319
10.6719 0.0012 22 10.625 0.0470
10.6328 0.0013 23 10.5938 0.0530
10.6172 0.0013 24 10.5703 0.0542
10.5859 0.0014 25 10.5469 0.0543
10.5547 0.0014 26 10.5312 0.0540
10.5391 0.0015 27 10.5156 0.0534
10.5547 0.0015 28 10.5 0.0531
10.5156 0.0016 29 10.4844 0.0535
10.4844 0.0016 30 10.4766 0.0542
10.4844 0.0017 31 10.4609 0.0548
10.4766 0.0017 32 10.4531 0.0551
10.4766 0.0018 33 10.4453 0.0557
10.4531 0.0019 34 10.4375 0.0565
10.4453 0.0019 35 10.4297 0.0570
10.4375 0.0020 36 10.4219 0.0575
10.4375 0.0020 37 10.4141 0.0581
10.4453 0.0021 38 10.4141 0.0583
10.3984 0.0021 39 10.4062 0.0585
10.4141 0.0022 40 10.3984 0.0586
10.4062 0.0022 41 10.3906 0.0587
10.3984 0.0023 42 10.3906 0.0587
10.3906 0.0023 43 10.3828 0.0588
10.4062 0.0024 44 10.375 0.0591
10.375 0.0025 45 10.375 0.0592
10.3984 0.0025 46 10.3672 0.0592
10.3828 0.0026 47 10.3594 0.0593
10.375 0.0026 48 10.3516 0.0597
10.3594 0.0027 49 10.3516 0.0599
10.3516 0.0027 50 10.3438 0.0602
10.3438 0.0028 51 10.3359 0.0604
10.3516 0.0028 52 10.3281 0.0606
10.3594 0.0029 53 10.3281 0.0607
10.3438 0.0029 54 10.3203 0.0608
10.3281 0.0030 55 10.3125 0.0608
10.3281 0.0031 56 10.3125 0.0607
10.3281 0.0031 57 10.3047 0.0607
10.3438 0.0032 58 10.3047 0.0607
10.3125 0.0032 59 10.2969 0.0609
10.3203 0.0033 60 10.2969 0.0612
10.3125 0.0033 61 10.2891 0.0615
10.2969 0.0034 62 10.2812 0.0618
10.2891 0.0034 63 10.2812 0.0620
10.2969 0.0035 64 10.2734 0.0622
10.2891 0.0035 65 10.2734 0.0622
10.2734 0.0036 66 10.2656 0.0623
10.2656 0.0037 67 10.2656 0.0623
10.2656 0.0037 68 10.2578 0.0623
10.2578 0.0038 69 10.25 0.0622
10.25 0.0038 70 10.25 0.0622
10.2656 0.0039 71 10.2422 0.0623
10.2344 0.0039 72 10.2422 0.0626
10.2578 0.0040 73 10.2344 0.0629
10.2266 0.0040 74 10.2344 0.0632
10.2422 0.0041 75 10.2266 0.0633
10.2656 0.0041 76 10.2266 0.0633
10.2266 0.0042 77 10.2188 0.0632
10.2422 0.0043 78 10.2188 0.0631
10.2031 0.0043 79 10.2109 0.0630
10.2031 0.0044 80 10.2109 0.0631
10.2188 0.0044 81 10.2031 0.0633
10.2188 0.0045 82 10.2031 0.0637
10.2344 0.0045 83 10.1953 0.0641
10.2188 0.0046 84 10.1953 0.0647
10.2031 0.0046 85 10.1875 0.0653
10.2266 0.0047 86 10.1875 0.0657
10.2109 0.0047 87 10.1797 0.0660
10.1641 0.0048 88 10.1797 0.0660
10.1953 0.0048 89 10.1719 0.0660
10.1875 0.0049 90 10.1719 0.0658
10.2031 0.0050 91 10.1641 0.0658
10.1719 0.0050 92 10.1641 0.0658
10.1953 0.0051 93 10.1562 0.0660
10.1641 0.0051 94 10.1562 0.0665
10.1797 0.0052 95 10.1484 0.0673
10.1797 0.0052 96 10.1484 0.0682
10.1406 0.0053 97 10.1406 0.0690
10.1562 0.0053 98 10.1406 0.0696
10.1406 0.0054 99 10.1328 0.0699
10.1641 0.0054 100 10.1328 0.0700
10.1797 0.0055 101 10.125 0.0699
10.1484 0.0056 102 10.125 0.0699
10.1406 0.0056 103 10.1172 0.0701
10.1328 0.0057 104 10.1172 0.0706
10.0938 0.0057 105 10.1094 0.0712
10.1016 0.0058 106 10.1094 0.0719
10.1016 0.0058 107 10.1016 0.0725
10.1094 0.0059 108 10.1016 0.0728
10.1016 0.0059 109 10.1016 0.0729
10.1016 0.0060 110 10.0938 0.0729
10.0781 0.0060 111 10.0938 0.0728
10.0938 0.0061 112 10.0859 0.0727
10.1172 0.0062 113 10.0859 0.0725
10.1016 0.0062 114 10.0781 0.0725
10.0938 0.0063 115 10.0781 0.0726
10.1016 0.0063 116 10.0703 0.0730
10.0703 0.0064 117 10.0703 0.0733
10.0938 0.0064 118 10.0625 0.0738
10.0859 0.0065 119 10.0625 0.0742
10.0781 0.0065 120 10.0625 0.0744
10.0625 0.0066 121 10.0547 0.0745
10.0547 0.0066 122 10.0547 0.0746
10.0781 0.0067 123 10.0469 0.0746
10.0625 0.0068 124 10.0469 0.0745
10.0781 0.0068 125 10.0391 0.0745
10.0781 0.0069 126 10.0391 0.0747
10.0703 0.0069 127 10.0391 0.0752
10.0547 0.0070 128 10.0312 0.0758
10.0469 0.0070 129 10.0312 0.0762
10.0391 0.0071 130 10.0234 0.0765
10.0391 0.0071 131 10.0234 0.0765
10.0469 0.0072 132 10.0156 0.0764
10.0469 0.0072 133 10.0156 0.0761
10.0234 0.0073 134 10.0156 0.0759
10.0312 0.0074 135 10.0078 0.0757
10.0312 0.0074 136 10.0078 0.0757
10.0078 0.0075 137 10.0 0.0759
10.0 0.0075 138 10.0 0.0763
10.0078 0.0076 139 10.0 0.0768
10.0234 0.0076 140 9.9922 0.0774
9.9922 0.0077 141 9.9922 0.0779
10.0234 0.0077 142 9.9844 0.0782
9.9766 0.0078 143 9.9844 0.0783
10.0156 0.0078 144 9.9844 0.0782
9.9844 0.0079 145 9.9766 0.0780
9.9922 0.0080 146 9.9766 0.0778
9.9844 0.0080 147 9.9688 0.0776
10.0 0.0081 148 9.9688 0.0775
9.9766 0.0081 149 9.9688 0.0776
9.9688 0.0082 150 9.9609 0.0778
9.9844 0.0082 151 9.9609 0.0782
9.9766 0.0083 152 9.9531 0.0785
9.9766 0.0083 153 9.9531 0.0787
9.9922 0.0084 154 9.9453 0.0787
9.9688 0.0084 155 9.9453 0.0787
9.9141 0.0085 156 9.9453 0.0785
9.9453 0.0086 157 9.9375 0.0783
9.9375 0.0086 158 9.9375 0.0782
9.9453 0.0087 159 9.9375 0.0782
9.9531 0.0087 160 9.9297 0.0784
9.9297 0.0088 161 9.9297 0.0788
9.9375 0.0088 162 9.9219 0.0793
9.9219 0.0089 163 9.9219 0.0797
9.9297 0.0089 164 9.9219 0.0799
9.9219 0.0090 165 9.9141 0.0802
9.9141 0.0090 166 9.9141 0.0801
9.9141 0.0091 167 9.9062 0.0799
9.9219 0.0092 168 9.9062 0.0797
9.9062 0.0092 169 9.9062 0.0795
9.9062 0.0093 170 9.8984 0.0795
9.9062 0.0093 171 9.8984 0.0797
9.9297 0.0094 172 9.8906 0.0800
9.8984 0.0094 173 9.8906 0.0804
9.875 0.0095 174 9.8906 0.0808
9.8984 0.0095 175 9.8828 0.0810
9.8828 0.0096 176 9.8828 0.0811
9.8828 0.0096 177 9.8828 0.0811
9.875 0.0097 178 9.875 0.0808
9.8828 0.0098 179 9.875 0.0805
9.8906 0.0098 180 9.8672 0.0803
9.8594 0.0099 181 9.8672 0.0803
9.8828 0.0099 182 9.8672 0.0804
9.8906 0.0100 183 9.8594 0.0807
9.8438 0.0100 184 9.8594 0.0809
9.8672 0.0101 185 9.8516 0.0810
9.8828 0.0101 186 9.8516 0.0811
9.8828 0.0102 187 9.8516 0.0811
9.8594 0.0102 188 9.8438 0.0811
9.8672 0.0103 189 9.8438 0.0811
9.8516 0.0104 190 9.8438 0.0812
9.8281 0.0104 191 9.8359 0.0813
9.8359 0.0105 192 9.8359 0.0816
9.8359 0.0105 193 9.8281 0.0818
9.8516 0.0106 194 9.8281 0.0819
9.8125 0.0106 195 9.8281 0.0817
9.8047 0.0107 196 9.8203 0.0815
9.8203 0.0107 197 9.8203 0.0814
9.8438 0.0108 198 9.8203 0.0814
9.8281 0.0108 199 9.8125 0.0815
9.8516 0.0109 200 9.8125 0.0819
9.8125 0.0110 201 9.8047 0.0823
9.7969 0.0110 202 9.8047 0.0826
9.8359 0.0111 203 9.8047 0.0827
9.8359 0.0111 204 9.7969 0.0828
9.8281 0.0112 205 9.7969 0.0826
9.8359 0.0112 206 9.7969 0.0824
9.8125 0.0113 207 9.7891 0.0823
9.8281 0.0113 208 9.7891 0.0824
9.8203 0.0114 209 9.7812 0.0826
9.7891 0.0114 210 9.7812 0.0826
9.7734 0.0115 211 9.7812 0.0826
9.7734 0.0116 212 9.7734 0.0830
9.7969 0.0116 213 9.7734 0.0835
9.7969 0.0117 214 9.7656 0.0840
9.7656 0.0117 215 9.7656 0.0844
9.7891 0.0118 216 9.7656 0.0844
9.7812 0.0118 217 9.7578 0.0845
9.7812 0.0119 218 9.7578 0.0844
9.7891 0.0119 219 9.7578 0.0844
9.7734 0.0120 220 9.75 0.0844
9.75 0.0120 221 9.75 0.0844
9.7578 0.0121 222 9.7422 0.0843
9.7422 0.0122 223 9.7422 0.0842
9.7578 0.0122 224 9.7422 0.0843
9.7344 0.0123 225 9.7344 0.0845
9.7578 0.0123 226 9.7344 0.0848
9.7734 0.0124 227 9.7344 0.0851
9.7266 0.0124 228 9.7266 0.0851
9.7344 0.0125 229 9.7266 0.0849
9.7344 0.0125 230 9.7266 0.0849
9.6875 0.0126 231 9.7188 0.0850
9.75 0.0126 232 9.7188 0.0854
9.7188 0.0127 233 9.7109 0.0857
9.7109 0.0128 234 9.7109 0.0860
9.7031 0.0128 235 9.7109 0.0861
9.7422 0.0129 236 9.7031 0.0861
9.7266 0.0129 237 9.7031 0.0861
9.7109 0.0130 238 9.7031 0.0858
9.7422 0.0130 239 9.6953 0.0856
9.6875 0.0131 240 9.6953 0.0854
9.7109 0.0131 241 9.6953 0.0853
9.6953 0.0132 242 9.6875 0.0853
9.7109 0.0132 243 9.6875 0.0856
9.6719 0.0133 244 9.6797 0.0859
9.7109 0.0134 245 9.6797 0.0863
9.6719 0.0134 246 9.6797 0.0866
9.7109 0.0135 247 9.6719 0.0867
9.7031 0.0135 248 9.6719 0.0866
9.6641 0.0136 249 9.6719 0.0866
9.6953 0.0136 250 9.6641 0.0866
9.6641 0.0137 251 9.6641 0.0866
9.6719 0.0137 252 9.6641 0.0868
9.6719 0.0138 253 9.6562 0.0869
9.6797 0.0138 254 9.6562 0.0870
9.6797 0.0139 255 9.6484 0.0870
9.6641 0.0139 256 9.6484 0.0870
9.6562 0.0140 257 9.6484 0.0869
9.6562 0.0141 258 9.6406 0.0867
9.6562 0.0141 259 9.6406 0.0865
9.6641 0.0142 260 9.6406 0.0866
9.6406 0.0142 261 9.6328 0.0868
9.6484 0.0143 262 9.6328 0.0871
9.6484 0.0143 263 9.6328 0.0873
9.6328 0.0144 264 9.625 0.0874
9.625 0.0144 265 9.625 0.0875
9.6328 0.0145 266 9.6172 0.0877
9.6641 0.0145 267 9.6172 0.0877
9.6484 0.0146 268 9.6172 0.0877
9.6328 0.0147 269 9.6094 0.0877
9.625 0.0147 270 9.6094 0.0875
9.625 0.0148 271 9.6094 0.0875
9.6094 0.0148 272 9.6016 0.0875
9.6172 0.0149 273 9.6016 0.0877
9.625 0.0149 274 9.6016 0.0878

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.1.0a0+32f93b1
  • Datasets 2.20.0
  • Tokenizers 0.19.1