Hugging Face's logo

gokulsrinivasagan
/
gpt_train_6_256

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
gpt_train_6_256 / README.md
gokulsrinivasagan's picture
gokulsrinivasagan
End of training
f22b430 verified
preview code
|
raw
history blame contribute delete
22.9 kB
metadata
license: mit
base_model: openai-community/gpt2
tags:
  - generated_from_trainer
datasets:
  - gokuls/wiki_book_corpus_raw_dataset_tiny
metrics:
  - accuracy
model-index:
  - name: gpt_train_6_256
    results:
      - task:
          name: Causal Language Modeling
          type: text-generation
        dataset:
          name: gokuls/wiki_book_corpus_raw_dataset_tiny
          type: gokuls/wiki_book_corpus_raw_dataset_tiny
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.08509852797509099

gpt_train_6_256

This model is a fine-tuned version of openai-community/gpt2 on the gokuls/wiki_book_corpus_raw_dataset_tiny dataset. It achieves the following results on the evaluation set:

  • Loss: 9.4766
  • Accuracy: 0.0851

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 36
  • eval_batch_size: 36
  • seed: 10
  • distributed_type: multi-GPU
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
10.8672 0.0001 1 10.8672 0.0045
10.8672 0.0001 2 10.8672 0.0045
10.8672 0.0002 3 10.8672 0.0045
10.8672 0.0002 4 10.8672 0.0045
10.8594 0.0003 5 10.8672 0.0045
10.8672 0.0003 6 10.8672 0.0045
10.8672 0.0004 7 10.8672 0.0045
10.8672 0.0004 8 10.8672 0.0045
10.8672 0.0005 9 10.8672 0.0045
10.8594 0.0005 10 10.8672 0.0045
10.8672 0.0006 11 10.8672 0.0045
10.8672 0.0007 12 10.8672 0.0045
10.8594 0.0007 13 10.8672 0.0045
10.8672 0.0008 14 10.8672 0.0045
10.8594 0.0008 15 10.8672 0.0045
10.8594 0.0009 16 10.8672 0.0045
10.8672 0.0009 17 10.8672 0.0045
10.8672 0.0010 18 10.8359 0.0086
10.8359 0.0010 19 10.8047 0.0108
10.8047 0.0011 20 10.7734 0.0113
10.7891 0.0011 21 10.75 0.0115
10.7578 0.0012 22 10.7266 0.0119
10.7188 0.0013 23 10.7031 0.0129
10.7188 0.0013 24 10.6797 0.0147
10.6953 0.0014 25 10.6641 0.0179
10.6719 0.0014 26 10.6406 0.0231
10.6562 0.0015 27 10.625 0.0286
10.6641 0.0015 28 10.6094 0.0347
10.6328 0.0016 29 10.5938 0.0399
10.6016 0.0016 30 10.5781 0.0436
10.6016 0.0017 31 10.5703 0.0463
10.5938 0.0017 32 10.5547 0.0479
10.5781 0.0018 33 10.5469 0.0484
10.5547 0.0019 34 10.5312 0.0484
10.5469 0.0019 35 10.5234 0.0484
10.5391 0.0020 36 10.5156 0.0482
10.5312 0.0020 37 10.5078 0.0475
10.5312 0.0021 38 10.4922 0.0475
10.4922 0.0021 39 10.4844 0.0476
10.5078 0.0022 40 10.4844 0.0477
10.4922 0.0022 41 10.4766 0.0481
10.4844 0.0023 42 10.4688 0.0486
10.4766 0.0023 43 10.4609 0.0493
10.4844 0.0024 44 10.4531 0.0495
10.4688 0.0025 45 10.4453 0.0503
10.4844 0.0025 46 10.4453 0.0513
10.4609 0.0026 47 10.4375 0.0522
10.4453 0.0026 48 10.4297 0.0526
10.4453 0.0027 49 10.4297 0.0532
10.4297 0.0027 50 10.4219 0.0537
10.4219 0.0028 51 10.4141 0.0544
10.4297 0.0028 52 10.4141 0.0548
10.4375 0.0029 53 10.4062 0.0554
10.4219 0.0029 54 10.4062 0.0558
10.4141 0.0030 55 10.3984 0.0565
10.4141 0.0031 56 10.3906 0.0574
10.4219 0.0031 57 10.3906 0.0583
10.4219 0.0032 58 10.3828 0.0591
10.3984 0.0032 59 10.3828 0.0598
10.3984 0.0033 60 10.375 0.0603
10.3984 0.0033 61 10.375 0.0607
10.3906 0.0034 62 10.3672 0.0611
10.3672 0.0034 63 10.3672 0.0615
10.3906 0.0035 64 10.3594 0.0616
10.3828 0.0035 65 10.3594 0.0615
10.3594 0.0036 66 10.3516 0.0614
10.3516 0.0037 67 10.3438 0.0610
10.3516 0.0037 68 10.3438 0.0609
10.3438 0.0038 69 10.3359 0.0611
10.3594 0.0038 70 10.3359 0.0610
10.3594 0.0039 71 10.3281 0.0610
10.3203 0.0039 72 10.3281 0.0610
10.3516 0.0040 73 10.3203 0.0610
10.3203 0.0040 74 10.3125 0.0611
10.3281 0.0041 75 10.3125 0.0612
10.3438 0.0041 76 10.3047 0.0614
10.2969 0.0042 77 10.3047 0.0618
10.3281 0.0043 78 10.2969 0.0622
10.2891 0.0043 79 10.2969 0.0628
10.3047 0.0044 80 10.2891 0.0632
10.2969 0.0044 81 10.2812 0.0637
10.2891 0.0045 82 10.2812 0.0643
10.3125 0.0045 83 10.2734 0.0649
10.2891 0.0046 84 10.2734 0.0654
10.2812 0.0046 85 10.2656 0.0657
10.3047 0.0047 86 10.2656 0.0659
10.2969 0.0047 87 10.2578 0.0660
10.2578 0.0048 88 10.25 0.0661
10.2812 0.0048 89 10.25 0.0662
10.2734 0.0049 90 10.2422 0.0663
10.2891 0.0050 91 10.2422 0.0664
10.2578 0.0050 92 10.2344 0.0666
10.2734 0.0051 93 10.2344 0.0668
10.2266 0.0051 94 10.2266 0.0671
10.2578 0.0052 95 10.2266 0.0674
10.25 0.0052 96 10.2188 0.0676
10.2266 0.0053 97 10.2188 0.0678
10.2266 0.0053 98 10.2109 0.0679
10.2344 0.0054 99 10.2109 0.0681
10.2422 0.0054 100 10.2031 0.0682
10.2422 0.0055 101 10.2031 0.0683
10.2266 0.0056 102 10.1953 0.0685
10.2188 0.0056 103 10.1953 0.0686
10.2109 0.0057 104 10.1875 0.0687
10.1797 0.0057 105 10.1875 0.0689
10.1797 0.0058 106 10.1797 0.0691
10.1719 0.0058 107 10.1797 0.0693
10.1875 0.0059 108 10.1719 0.0696
10.1797 0.0059 109 10.1719 0.0698
10.1797 0.0060 110 10.1641 0.0700
10.1406 0.0060 111 10.1641 0.0702
10.1719 0.0061 112 10.1641 0.0704
10.1953 0.0062 113 10.1562 0.0706
10.1719 0.0062 114 10.1562 0.0708
10.1641 0.0063 115 10.1484 0.0710
10.1719 0.0063 116 10.1484 0.0712
10.1484 0.0064 117 10.1406 0.0713
10.1562 0.0064 118 10.1406 0.0715
10.1562 0.0065 119 10.1328 0.0716
10.1484 0.0065 120 10.1328 0.0718
10.1406 0.0066 121 10.125 0.0719
10.1328 0.0066 122 10.125 0.0721
10.1641 0.0067 123 10.1172 0.0722
10.1328 0.0068 124 10.1172 0.0723
10.1484 0.0068 125 10.1094 0.0725
10.1406 0.0069 126 10.1094 0.0726
10.1406 0.0069 127 10.1016 0.0728
10.125 0.0070 128 10.1016 0.0729
10.1172 0.0070 129 10.0938 0.0731
10.1016 0.0071 130 10.0938 0.0732
10.1172 0.0071 131 10.0859 0.0733
10.1172 0.0072 132 10.0859 0.0734
10.1172 0.0072 133 10.0859 0.0736
10.0938 0.0073 134 10.0781 0.0737
10.1094 0.0074 135 10.0781 0.0738
10.1094 0.0074 136 10.0703 0.0740
10.0703 0.0075 137 10.0703 0.0742
10.0781 0.0075 138 10.0625 0.0743
10.0781 0.0076 139 10.0625 0.0745
10.0781 0.0076 140 10.0547 0.0746
10.0625 0.0077 141 10.0547 0.0747
10.0781 0.0077 142 10.0469 0.0749
10.0391 0.0078 143 10.0469 0.0750
10.0703 0.0078 144 10.0469 0.0751
10.0391 0.0079 145 10.0391 0.0753
10.0469 0.0080 146 10.0391 0.0754
10.0547 0.0080 147 10.0312 0.0755
10.0703 0.0081 148 10.0312 0.0756
10.0469 0.0081 149 10.0234 0.0757
10.0391 0.0082 150 10.0234 0.0759
10.0391 0.0082 151 10.0156 0.0760
10.0391 0.0083 152 10.0156 0.0761
10.0391 0.0083 153 10.0156 0.0762
10.0469 0.0084 154 10.0078 0.0763
10.0312 0.0084 155 10.0078 0.0765
9.9844 0.0085 156 10.0 0.0766
10.0 0.0086 157 10.0 0.0767
10.0078 0.0086 158 9.9922 0.0768
10.0078 0.0087 159 9.9922 0.0769
10.0234 0.0087 160 9.9922 0.0770
9.9922 0.0088 161 9.9844 0.0771
9.9922 0.0088 162 9.9844 0.0772
9.9766 0.0089 163 9.9766 0.0773
9.9922 0.0089 164 9.9766 0.0773
9.9766 0.0090 165 9.9688 0.0774
9.9844 0.0090 166 9.9688 0.0775
9.9766 0.0091 167 9.9688 0.0776
9.9844 0.0092 168 9.9609 0.0777
9.9609 0.0092 169 9.9609 0.0778
9.9766 0.0093 170 9.9531 0.0778
9.9531 0.0093 171 9.9531 0.0779
9.9922 0.0094 172 9.9531 0.0780
9.9531 0.0094 173 9.9453 0.0781
9.9375 0.0095 174 9.9453 0.0781
9.9688 0.0095 175 9.9375 0.0782
9.9453 0.0096 176 9.9375 0.0783
9.9375 0.0096 177 9.9375 0.0783
9.9375 0.0097 178 9.9297 0.0784
9.9453 0.0098 179 9.9297 0.0785
9.9453 0.0098 180 9.9219 0.0786
9.9297 0.0099 181 9.9219 0.0787
9.9375 0.0099 182 9.9141 0.0787
9.9375 0.0100 183 9.9141 0.0788
9.8984 0.0100 184 9.9141 0.0789
9.9375 0.0101 185 9.9062 0.0790
9.9297 0.0101 186 9.9062 0.0791
9.9297 0.0102 187 9.8984 0.0791
9.9141 0.0102 188 9.8984 0.0792
9.9219 0.0103 189 9.8984 0.0793
9.8984 0.0104 190 9.8906 0.0793
9.8828 0.0104 191 9.8906 0.0794
9.8984 0.0105 192 9.8828 0.0795
9.8906 0.0105 193 9.8828 0.0796
9.9062 0.0106 194 9.8828 0.0797
9.875 0.0106 195 9.875 0.0798
9.8594 0.0107 196 9.875 0.0798
9.8828 0.0107 197 9.875 0.0799
9.8984 0.0108 198 9.8672 0.0800
9.8906 0.0108 199 9.8672 0.0801
9.9062 0.0109 200 9.8594 0.0801
9.8672 0.0110 201 9.8594 0.0802
9.8672 0.0110 202 9.8594 0.0803
9.8906 0.0111 203 9.8516 0.0804
9.8828 0.0111 204 9.8516 0.0804
9.8906 0.0112 205 9.8438 0.0805
9.8828 0.0112 206 9.8438 0.0805
9.8594 0.0113 207 9.8438 0.0806
9.875 0.0113 208 9.8359 0.0806
9.8594 0.0114 209 9.8359 0.0807
9.8516 0.0114 210 9.8281 0.0808
9.8359 0.0115 211 9.8281 0.0809
9.8281 0.0116 212 9.8281 0.0810
9.8516 0.0116 213 9.8203 0.0810
9.8516 0.0117 214 9.8203 0.0811
9.8281 0.0117 215 9.8203 0.0811
9.8438 0.0118 216 9.8125 0.0812
9.8359 0.0118 217 9.8125 0.0813
9.8281 0.0119 218 9.8047 0.0814
9.8281 0.0119 219 9.8047 0.0815
9.8281 0.0120 220 9.8047 0.0815
9.7969 0.0120 221 9.7969 0.0816
9.8281 0.0121 222 9.7969 0.0816
9.8047 0.0122 223 9.7891 0.0817
9.8047 0.0122 224 9.7891 0.0818
9.8047 0.0123 225 9.7891 0.0818
9.8047 0.0123 226 9.7812 0.0819
9.8281 0.0124 227 9.7812 0.0819
9.7812 0.0124 228 9.7812 0.0819
9.7891 0.0125 229 9.7734 0.0820
9.7969 0.0125 230 9.7734 0.0821
9.7578 0.0126 231 9.7656 0.0821
9.8125 0.0126 232 9.7656 0.0822
9.7734 0.0127 233 9.7656 0.0823
9.7656 0.0128 234 9.7578 0.0823
9.7578 0.0128 235 9.7578 0.0824
9.7891 0.0129 236 9.7578 0.0824
9.7812 0.0129 237 9.75 0.0824
9.7656 0.0130 238 9.75 0.0825
9.7969 0.0130 239 9.75 0.0825
9.75 0.0131 240 9.7422 0.0825
9.7734 0.0131 241 9.7422 0.0825
9.7578 0.0132 242 9.7344 0.0825
9.7656 0.0132 243 9.7344 0.0825
9.7266 0.0133 244 9.7344 0.0826
9.75 0.0134 245 9.7266 0.0826
9.7422 0.0134 246 9.7266 0.0827
9.75 0.0135 247 9.7266 0.0827
9.7656 0.0135 248 9.7188 0.0828
9.7266 0.0136 249 9.7188 0.0828
9.75 0.0136 250 9.7109 0.0828
9.7266 0.0137 251 9.7109 0.0829
9.7266 0.0137 252 9.7109 0.0829
9.7266 0.0138 253 9.7031 0.0829
9.7266 0.0138 254 9.7031 0.0829
9.7344 0.0139 255 9.7031 0.0829
9.7109 0.0139 256 9.6953 0.0829
9.7109 0.0140 257 9.6953 0.0829
9.7109 0.0141 258 9.6953 0.0830
9.7031 0.0141 259 9.6875 0.0830
9.7109 0.0142 260 9.6875 0.0831
9.6953 0.0142 261 9.6797 0.0832
9.7031 0.0143 262 9.6797 0.0832
9.6953 0.0143 263 9.6797 0.0832
9.6875 0.0144 264 9.6719 0.0833
9.6719 0.0144 265 9.6719 0.0833
9.6797 0.0145 266 9.6719 0.0832
9.7188 0.0145 267 9.6641 0.0833
9.6953 0.0146 268 9.6641 0.0833
9.6797 0.0147 269 9.6641 0.0833
9.6719 0.0147 270 9.6562 0.0834
9.6875 0.0148 271 9.6562 0.0834
9.6641 0.0148 272 9.6484 0.0835
9.6719 0.0149 273 9.6484 0.0836
9.6719 0.0149 274 9.6484 0.0836
9.6406 0.0150 275 9.6406 0.0837
9.6641 0.0150 276 9.6406 0.0837
9.6328 0.0151 277 9.6406 0.0838
9.6328 0.0151 278 9.6328 0.0838
9.6484 0.0152 279 9.6328 0.0838
9.6484 0.0153 280 9.6328 0.0838
9.6875 0.0153 281 9.625 0.0838
9.6328 0.0154 282 9.625 0.0838
9.6562 0.0154 283 9.6172 0.0838
9.6719 0.0155 284 9.6172 0.0838
9.6641 0.0155 285 9.6172 0.0838
9.6328 0.0156 286 9.6094 0.0838
9.6328 0.0156 287 9.6094 0.0839
9.625 0.0157 288 9.6094 0.0839
9.6328 0.0157 289 9.6016 0.0840
9.6172 0.0158 290 9.6016 0.0840
9.6172 0.0159 291 9.6016 0.0841
9.6094 0.0159 292 9.5938 0.0841
9.6172 0.0160 293 9.5938 0.0842
9.6094 0.0160 294 9.5938 0.0842
9.6328 0.0161 295 9.5859 0.0842
9.5938 0.0161 296 9.5859 0.0842
9.5938 0.0162 297 9.5781 0.0842
9.6016 0.0162 298 9.5781 0.0842
9.5781 0.0163 299 9.5781 0.0842
9.5938 0.0163 300 9.5703 0.0843
9.5938 0.0164 301 9.5703 0.0843
9.6016 0.0165 302 9.5703 0.0844
9.5781 0.0165 303 9.5625 0.0845
9.6016 0.0166 304 9.5625 0.0845
9.5703 0.0166 305 9.5625 0.0845
9.5781 0.0167 306 9.5547 0.0845
9.5938 0.0167 307 9.5547 0.0846
9.5391 0.0168 308 9.5547 0.0846
9.5625 0.0168 309 9.5469 0.0846
9.5547 0.0169 310 9.5469 0.0846
9.5703 0.0169 311 9.5469 0.0846
9.5625 0.0170 312 9.5391 0.0846
9.5469 0.0171 313 9.5391 0.0846
9.5469 0.0171 314 9.5391 0.0846
9.5391 0.0172 315 9.5312 0.0847
9.5781 0.0172 316 9.5312 0.0847
9.5469 0.0173 317 9.5312 0.0847
9.5312 0.0173 318 9.5234 0.0848
9.5703 0.0174 319 9.5234 0.0848
9.5312 0.0174 320 9.5234 0.0848
9.5703 0.0175 321 9.5156 0.0848
9.5312 0.0175 322 9.5156 0.0849
9.5391 0.0176 323 9.5078 0.0849
9.5156 0.0177 324 9.5078 0.0849
9.5234 0.0177 325 9.5078 0.0849
9.5391 0.0178 326 9.5 0.0849
9.5078 0.0178 327 9.5 0.0849
9.5312 0.0179 328 9.5 0.0848
9.5078 0.0179 329 9.4922 0.0848
9.5234 0.0180 330 9.4922 0.0847
9.5078 0.0180 331 9.4922 0.0848
9.4922 0.0181 332 9.4844 0.0848
9.5 0.0181 333 9.4844 0.0849
9.5078 0.0182 334 9.4844 0.0850
9.4766 0.0183 335 9.4766 0.0851
9.5 0.0183 336 9.4766 0.0851

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.1.0a0+32f93b1
  • Datasets 2.20.0
  • Tokenizers 0.19.1