Hugging Face's logo

gokulsrinivasagan
/
gpt_train_12_384

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
gpt_train_12_384 / README.md
gokulsrinivasagan's picture
gokulsrinivasagan
End of training
674480c verified
preview code
|
raw
history blame contribute delete
23.2 kB
metadata
license: mit
base_model: openai-community/gpt2
tags:
  - generated_from_trainer
datasets:
  - gokuls/wiki_book_corpus_raw_dataset_tiny
metrics:
  - accuracy
model-index:
  - name: gpt_train_12_384
    results:
      - task:
          name: Causal Language Modeling
          type: text-generation
        dataset:
          name: gokuls/wiki_book_corpus_raw_dataset_tiny
          type: gokuls/wiki_book_corpus_raw_dataset_tiny
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.10244681503029747

gpt_train_12_384

This model is a fine-tuned version of openai-community/gpt2 on the gokuls/wiki_book_corpus_raw_dataset_tiny dataset. It achieves the following results on the evaluation set:

  • Loss: 8.8125
  • Accuracy: 0.1024

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 10
  • distributed_type: multi-GPU
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
10.8984 0.0000 1 10.9062 0.0001
10.8984 0.0001 2 10.9062 0.0001
10.8984 0.0001 3 10.9062 0.0001
10.8984 0.0002 4 10.9062 0.0001
10.9062 0.0002 5 10.9062 0.0001
10.8984 0.0003 6 10.9062 0.0001
10.9062 0.0003 7 10.9062 0.0001
10.9062 0.0004 8 10.9062 0.0001
10.9062 0.0004 9 10.9062 0.0001
10.8984 0.0005 10 10.9062 0.0001
10.8984 0.0005 11 10.9062 0.0001
10.8984 0.0006 12 10.9062 0.0001
10.8984 0.0006 13 10.9062 0.0001
10.9062 0.0007 14 10.9062 0.0001
10.8984 0.0007 15 10.9062 0.0001
10.8984 0.0008 16 10.9062 0.0001
10.9062 0.0008 17 10.9062 0.0001
10.9062 0.0009 18 10.7578 0.0110
10.7734 0.0009 19 10.6562 0.0285
10.6797 0.0010 20 10.5781 0.0469
10.6016 0.0010 21 10.5234 0.0485
10.5234 0.0011 22 10.4766 0.0478
10.5 0.0011 23 10.4375 0.0483
10.4531 0.0012 24 10.4062 0.0507
10.4141 0.0012 25 10.3828 0.0531
10.3672 0.0013 26 10.3594 0.0556
10.3828 0.0013 27 10.3359 0.0562
10.3594 0.0014 28 10.3203 0.0562
10.3281 0.0014 29 10.3047 0.0559
10.3203 0.0015 30 10.2969 0.0563
10.3281 0.0015 31 10.2812 0.0566
10.3359 0.0015 32 10.2734 0.0566
10.2656 0.0016 33 10.2656 0.0570
10.2656 0.0016 34 10.2578 0.0561
10.2656 0.0017 35 10.2422 0.0562
10.2656 0.0017 36 10.2344 0.0575
10.2656 0.0018 37 10.2266 0.0586
10.2109 0.0018 38 10.2188 0.0593
10.2656 0.0019 39 10.2109 0.0596
10.2266 0.0019 40 10.2031 0.0599
10.2109 0.0020 41 10.1953 0.0601
10.2109 0.0020 42 10.1797 0.0604
10.2109 0.0021 43 10.1719 0.0608
10.1484 0.0021 44 10.1641 0.0610
10.1875 0.0022 45 10.1484 0.0611
10.1719 0.0022 46 10.1406 0.0612
10.1484 0.0023 47 10.1328 0.0615
10.1172 0.0023 48 10.1172 0.0622
10.1797 0.0024 49 10.1094 0.0632
10.1016 0.0024 50 10.1016 0.0642
10.1406 0.0025 51 10.0938 0.0651
10.1406 0.0025 52 10.0859 0.0658
10.1094 0.0026 53 10.0781 0.0663
10.1016 0.0026 54 10.0703 0.0669
10.0781 0.0027 55 10.0625 0.0672
10.0703 0.0027 56 10.0547 0.0678
10.0703 0.0028 57 10.0469 0.0681
10.0469 0.0028 58 10.0391 0.0686
10.1016 0.0029 59 10.0312 0.0689
10.0547 0.0029 60 10.0312 0.0694
10.0391 0.0030 61 10.0234 0.0695
10.0547 0.0030 62 10.0156 0.0692
10.0312 0.0031 63 10.0078 0.0688
10.0547 0.0031 64 10.0 0.0687
10.0547 0.0031 65 9.9922 0.0693
9.9922 0.0032 66 9.9844 0.0697
10.0234 0.0032 67 9.9766 0.0705
10.0 0.0033 68 9.9688 0.0711
10.0 0.0033 69 9.9609 0.0715
9.9688 0.0034 70 9.9609 0.0716
9.9922 0.0034 71 9.9531 0.0717
9.9844 0.0035 72 9.9453 0.0716
9.9688 0.0035 73 9.9375 0.0718
9.9453 0.0036 74 9.9297 0.0726
9.9375 0.0036 75 9.9219 0.0734
9.9141 0.0037 76 9.9141 0.0744
9.9062 0.0037 77 9.9062 0.0751
9.9219 0.0038 78 9.9062 0.0755
9.9219 0.0038 79 9.8984 0.0756
9.9219 0.0039 80 9.8906 0.0757
9.875 0.0039 81 9.8828 0.0759
9.9219 0.0040 82 9.875 0.0760
9.875 0.0040 83 9.875 0.0763
9.8672 0.0041 84 9.8672 0.0765
9.9062 0.0041 85 9.8594 0.0769
9.8828 0.0042 86 9.8516 0.0773
9.8594 0.0042 87 9.8516 0.0775
9.8906 0.0043 88 9.8438 0.0777
9.8047 0.0043 89 9.8359 0.0777
9.8203 0.0044 90 9.8359 0.0778
9.8594 0.0044 91 9.8281 0.0781
9.8438 0.0045 92 9.8203 0.0786
9.8438 0.0045 93 9.8203 0.0790
9.8438 0.0046 94 9.8125 0.0793
9.8359 0.0046 95 9.8047 0.0794
9.8281 0.0046 96 9.8047 0.0795
9.8516 0.0047 97 9.7969 0.0796
9.8281 0.0047 98 9.7891 0.0797
9.7734 0.0048 99 9.7891 0.0798
9.8125 0.0048 100 9.7812 0.0802
9.8203 0.0049 101 9.7734 0.0806
9.8281 0.0049 102 9.7734 0.0809
9.7734 0.0050 103 9.7656 0.0811
9.7891 0.0050 104 9.7578 0.0813
9.8047 0.0051 105 9.7578 0.0814
9.7578 0.0051 106 9.75 0.0815
9.7734 0.0052 107 9.75 0.0816
9.7891 0.0052 108 9.7422 0.0818
9.75 0.0053 109 9.7344 0.0819
9.75 0.0053 110 9.7344 0.0821
9.7266 0.0054 111 9.7266 0.0823
9.7656 0.0054 112 9.7188 0.0824
9.7812 0.0055 113 9.7188 0.0824
9.7734 0.0055 114 9.7109 0.0824
9.7266 0.0056 115 9.7109 0.0824
9.7266 0.0056 116 9.7031 0.0826
9.7109 0.0057 117 9.6953 0.0828
9.6719 0.0057 118 9.6953 0.0829
9.6953 0.0058 119 9.6875 0.0830
9.6719 0.0058 120 9.6875 0.0831
9.6953 0.0059 121 9.6797 0.0831
9.6875 0.0059 122 9.6797 0.0831
9.6719 0.0060 123 9.6719 0.0832
9.6719 0.0060 124 9.6641 0.0833
9.625 0.0061 125 9.6641 0.0833
9.6719 0.0061 126 9.6562 0.0834
9.6953 0.0062 127 9.6562 0.0836
9.6719 0.0062 128 9.6484 0.0837
9.6797 0.0062 129 9.6406 0.0838
9.6484 0.0063 130 9.6406 0.0839
9.6719 0.0063 131 9.6328 0.0839
9.6328 0.0064 132 9.6328 0.0839
9.6719 0.0064 133 9.625 0.0839
9.6484 0.0065 134 9.6172 0.0840
9.6406 0.0065 135 9.6172 0.0841
9.6094 0.0066 136 9.6094 0.0843
9.625 0.0066 137 9.6094 0.0845
9.6562 0.0067 138 9.6016 0.0846
9.6172 0.0067 139 9.6016 0.0847
9.6094 0.0068 140 9.5938 0.0847
9.6562 0.0068 141 9.5859 0.0847
9.6562 0.0069 142 9.5859 0.0847
9.6562 0.0069 143 9.5781 0.0848
9.6016 0.0070 144 9.5781 0.0849
9.6094 0.0070 145 9.5703 0.0850
9.5938 0.0071 146 9.5703 0.0851
9.5703 0.0071 147 9.5625 0.0851
9.5859 0.0072 148 9.5625 0.0851
9.625 0.0072 149 9.5547 0.0852
9.5859 0.0073 150 9.5469 0.0854
9.5625 0.0073 151 9.5469 0.0855
9.5547 0.0074 152 9.5391 0.0856
9.5703 0.0074 153 9.5391 0.0858
9.5391 0.0075 154 9.5312 0.0858
9.5391 0.0075 155 9.5312 0.0859
9.5 0.0076 156 9.5234 0.0861
9.5547 0.0076 157 9.5156 0.0863
9.5391 0.0077 158 9.5156 0.0863
9.5312 0.0077 159 9.5156 0.0864
9.5391 0.0077 160 9.5078 0.0864
9.4688 0.0078 161 9.5 0.0866
9.5547 0.0078 162 9.5 0.0867
9.5078 0.0079 163 9.4922 0.0869
9.5078 0.0079 164 9.4922 0.0870
9.5 0.0080 165 9.4844 0.0872
9.5312 0.0080 166 9.4844 0.0875
9.5156 0.0081 167 9.4766 0.0877
9.4844 0.0081 168 9.4766 0.0878
9.4688 0.0082 169 9.4688 0.0878
9.5156 0.0082 170 9.4609 0.0879
9.4922 0.0083 171 9.4609 0.0879
9.4844 0.0083 172 9.4531 0.0878
9.5234 0.0084 173 9.4531 0.0879
9.4844 0.0084 174 9.4453 0.0879
9.4219 0.0085 175 9.4453 0.0880
9.4062 0.0085 176 9.4375 0.0881
9.4375 0.0086 177 9.4375 0.0883
9.4375 0.0086 178 9.4297 0.0885
9.4688 0.0087 179 9.4297 0.0887
9.4453 0.0087 180 9.4219 0.0888
9.4219 0.0088 181 9.4219 0.0890
9.4141 0.0088 182 9.4141 0.0890
9.4375 0.0089 183 9.4062 0.0890
9.3984 0.0089 184 9.4062 0.0890
9.4297 0.0090 185 9.3984 0.0891
9.3984 0.0090 186 9.3984 0.0891
9.3906 0.0091 187 9.3906 0.0892
9.4219 0.0091 188 9.3906 0.0893
9.4062 0.0092 189 9.3828 0.0895
9.375 0.0092 190 9.3828 0.0897
9.3828 0.0093 191 9.375 0.0898
9.3906 0.0093 192 9.375 0.0898
9.3906 0.0093 193 9.3672 0.0899
9.4141 0.0094 194 9.3672 0.0898
9.3203 0.0094 195 9.3594 0.0898
9.3906 0.0095 196 9.3594 0.0898
9.3594 0.0095 197 9.3516 0.0900
9.3516 0.0096 198 9.3516 0.0901
9.3438 0.0096 199 9.3438 0.0902
9.3516 0.0097 200 9.3438 0.0904
9.3125 0.0097 201 9.3359 0.0906
9.3516 0.0098 202 9.3359 0.0907
9.3359 0.0098 203 9.3281 0.0908
9.3516 0.0099 204 9.3281 0.0907
9.3281 0.0099 205 9.3203 0.0906
9.375 0.0100 206 9.3125 0.0905
9.2812 0.0100 207 9.3125 0.0904
9.3281 0.0101 208 9.3047 0.0906
9.3281 0.0101 209 9.3047 0.0908
9.3594 0.0102 210 9.2969 0.0912
9.3438 0.0102 211 9.2969 0.0915
9.2891 0.0103 212 9.2891 0.0916
9.3438 0.0103 213 9.2891 0.0916
9.3047 0.0104 214 9.2812 0.0915
9.2656 0.0104 215 9.2812 0.0914
9.2734 0.0105 216 9.2734 0.0913
9.2891 0.0105 217 9.2734 0.0913
9.2969 0.0106 218 9.2656 0.0913
9.25 0.0106 219 9.2656 0.0914
9.2578 0.0107 220 9.2578 0.0915
9.25 0.0107 221 9.2578 0.0916
9.2656 0.0108 222 9.25 0.0920
9.2578 0.0108 223 9.25 0.0923
9.2734 0.0108 224 9.2422 0.0926
9.2891 0.0109 225 9.2422 0.0929
9.25 0.0109 226 9.2344 0.0928
9.2344 0.0110 227 9.2344 0.0928
9.2656 0.0110 228 9.2266 0.0927
9.2656 0.0111 229 9.2266 0.0928
9.2656 0.0111 230 9.2188 0.0930
9.25 0.0112 231 9.2188 0.0933
9.2891 0.0112 232 9.2109 0.0937
9.2188 0.0113 233 9.2031 0.0938
9.2578 0.0113 234 9.2031 0.0939
9.2422 0.0114 235 9.1953 0.0938
9.2109 0.0114 236 9.1953 0.0935
9.1797 0.0115 237 9.1953 0.0935
9.1953 0.0115 238 9.1875 0.0938
9.1797 0.0116 239 9.1875 0.0943
9.2266 0.0116 240 9.1797 0.0948
9.2109 0.0117 241 9.1719 0.0951
9.1719 0.0117 242 9.1719 0.0954
9.2031 0.0118 243 9.1719 0.0955
9.1953 0.0118 244 9.1641 0.0954
9.1875 0.0119 245 9.1641 0.0950
9.2031 0.0119 246 9.1562 0.0949
9.1797 0.0120 247 9.1484 0.0950
9.1484 0.0120 248 9.1484 0.0952
9.1406 0.0121 249 9.1484 0.0954
9.1641 0.0121 250 9.1406 0.0956
9.1406 0.0122 251 9.1406 0.0956
9.1719 0.0122 252 9.1328 0.0954
9.125 0.0123 253 9.1328 0.0953
9.1719 0.0123 254 9.125 0.0950
9.1797 0.0124 255 9.125 0.0950
9.0859 0.0124 256 9.1172 0.0951
9.1875 0.0124 257 9.1172 0.0957
9.1094 0.0125 258 9.1094 0.0963
9.0938 0.0125 259 9.1094 0.0968
9.1016 0.0126 260 9.1016 0.0969
9.1406 0.0126 261 9.1016 0.0969
9.0781 0.0127 262 9.0938 0.0966
9.1094 0.0127 263 9.0938 0.0963
9.1172 0.0128 264 9.0859 0.0959
9.1172 0.0128 265 9.0859 0.0956
9.125 0.0129 266 9.0859 0.0955
9.1094 0.0129 267 9.0781 0.0957
9.0781 0.0130 268 9.0781 0.0964
9.125 0.0130 269 9.0703 0.0973
9.0547 0.0131 270 9.0703 0.0980
9.0781 0.0131 271 9.0625 0.0983
9.1016 0.0132 272 9.0625 0.0981
9.0703 0.0132 273 9.0547 0.0975
9.0547 0.0133 274 9.0547 0.0969
9.0312 0.0133 275 9.0469 0.0964
9.0938 0.0134 276 9.0469 0.0964
9.0156 0.0134 277 9.0391 0.0967
9.1094 0.0135 278 9.0391 0.0973
9.0859 0.0135 279 9.0312 0.0980
9.0234 0.0136 280 9.0312 0.0984
9.0781 0.0136 281 9.0234 0.0984
9.0547 0.0137 282 9.0234 0.0983
9.0234 0.0137 283 9.0156 0.0979
9.0312 0.0138 284 9.0156 0.0978
9.0391 0.0138 285 9.0078 0.0978
9.0312 0.0139 286 9.0078 0.0980
9.0625 0.0139 287 9.0078 0.0982
9.0234 0.0139 288 9.0 0.0986
9.0078 0.0140 289 9.0 0.0990
9.0 0.0140 290 8.9922 0.0996
9.0078 0.0141 291 8.9922 0.0997
9.0 0.0141 292 8.9844 0.0999
9.0078 0.0142 293 8.9844 0.0999
8.9922 0.0142 294 8.9766 0.0995
9.0078 0.0143 295 8.9766 0.0990
8.9844 0.0143 296 8.9688 0.0985
8.9766 0.0144 297 8.9688 0.0983
8.9531 0.0144 298 8.9609 0.0985
8.9688 0.0145 299 8.9609 0.0988
9.0312 0.0145 300 8.9531 0.0994
9.0156 0.0146 301 8.9531 0.0998
8.9688 0.0146 302 8.9453 0.0999
9.0 0.0147 303 8.9453 0.0997
8.9375 0.0147 304 8.9375 0.0996
8.9766 0.0148 305 8.9375 0.0994
8.9375 0.0148 306 8.9375 0.0994
8.9688 0.0149 307 8.9297 0.0997
8.9531 0.0149 308 8.9297 0.0999
8.9531 0.0150 309 8.9219 0.1002
8.9062 0.0150 310 8.9219 0.1003
8.9375 0.0151 311 8.9141 0.1004
8.8828 0.0151 312 8.9141 0.1003
8.9219 0.0152 313 8.9062 0.1003
8.9219 0.0152 314 8.9062 0.1004
8.9297 0.0153 315 8.9062 0.1009
8.9922 0.0153 316 8.8984 0.1011
8.9062 0.0154 317 8.8984 0.1011
8.9297 0.0154 318 8.8906 0.1011
8.9531 0.0155 319 8.8906 0.1008
8.9531 0.0155 320 8.8828 0.1006
8.9375 0.0155 321 8.8828 0.1004
8.9219 0.0156 322 8.875 0.1002
8.9062 0.0156 323 8.875 0.1004
8.8906 0.0157 324 8.875 0.1006
8.8906 0.0157 325 8.8672 0.1011
8.8672 0.0158 326 8.8672 0.1016
8.875 0.0158 327 8.8594 0.1019
8.8516 0.0159 328 8.8594 0.1022
8.8672 0.0159 329 8.8516 0.1020
8.8984 0.0160 330 8.8516 0.1018
8.875 0.0160 331 8.8438 0.1016
8.8828 0.0161 332 8.8438 0.1014
8.8438 0.0161 333 8.8359 0.1014
8.7969 0.0162 334 8.8359 0.1017
8.8828 0.0162 335 8.8281 0.1020
8.8281 0.0163 336 8.8281 0.1025
8.8203 0.0163 337 8.8281 0.1027
8.8594 0.0164 338 8.8203 0.1028
8.8594 0.0164 339 8.8203 0.1027
8.8203 0.0165 340 8.8125 0.1025
8.8359 0.0165 341 8.8125 0.1024

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.1.0a0+32f93b1
  • Datasets 2.20.0
  • Tokenizers 0.19.1