Create With one layer, n_layer 1, n_embd 4 is failure. but n_embd 6 is marginal success.

With one layer, n_layer 1, n_embd 4 is failure. but n_embd 6 is marginal success.

@@ -0,0 +1,58 @@
+At n_embd': 4, there was no coherence obtained.
+
+At n_embd': 6, 'n_layer': 1, 'n_head': 1, 'n_inner': 64, the Toy Gettysburg GPT-2 model got a good start with "four score and seven years ago our fathers brought forth on this continent , a new nation , conceived in" before some mistakes. But resumed another whole part of the Gettysburg speech: "that all men are created equal . now we are engaged in a great civil war , testing whether that nation , or any nation so conceived and so dedicated , can long endure . we are met on a great battle - field of that war . we have come to dedicate a portion of that field , as a final resting place for those who here gave their lives that that nation might endure "
+
+Adding a second layer ('n_layer': 2,) did not solve the problem: 
+Epoch 22361/100000, Loss: 0.0054
+           LOSS IS BELOW 0.01
+Epoch 22362/100000, Loss: 0.0033
+           LOSS IS BELOW 0.01
+Epoch 22363/100000, Loss: 0.0044
+           LOSS IS BELOW 0.01
+Epoch 22364/100000, Loss: 0.0032
+
+
+Epoch 26651/100000, Loss: 0.0024
+           LOSS IS BELOW 0.01
+Epoch 26652/100000, Loss: 0.0039
+           LOSS IS BELOW 0.01
+Epoch 26653/100000, Loss: 0.0024
+           LOSS IS BELOW 0.01
+Epoch 26654/100000, Loss: 0.0034
+           LOSS IS BELOW 0.01
+
+           
+
+#################################### n_embd': 6, 'n_layer': 1, 'n_head': 1, 'n_inner': 64,  ###############################################
+Epoch 99983/100000, Loss: 0.0474
+Epoch 99984/100000, Loss: 0.1334
+Epoch 99985/100000, Loss: 0.0775
+Epoch 99986/100000, Loss: 0.0629
+Epoch 99987/100000, Loss: 0.1047
+Epoch 99988/100000, Loss: 0.0988
+Epoch 99989/100000, Loss: 0.0666
+Epoch 99990/100000, Loss: 0.0633
+Epoch 99991/100000, Loss: 0.1468
+Epoch 99992/100000, Loss: 0.0667
+Epoch 99993/100000, Loss: 0.1081
+Epoch 99994/100000, Loss: 0.0680
+Epoch 99995/100000, Loss: 0.0754
+Epoch 99996/100000, Loss: 0.0507
+Epoch 99997/100000, Loss: 0.1052
+Epoch 99998/100000, Loss: 0.0613
+Epoch 99999/100000, Loss: 0.2482
+Epoch 100000/100000, Loss: 0.0892
+
+# --- Inference Examples --- at script line 431
+# Example 1: Recite the Gettysburg Address at script line 435
+Prompt: four score
+
+Response:
+ four score and seven years ago our fathers brought forth on this continent , a new nation , conceived in nation - to , it gave by that all men are created equal . now we are engaged in a great civil war , testing whether that nation , or any nation so conceived and so dedicated , can long endure . we are met on a great battle - field of that war . we have come to dedicate a portion of that field , as a final resting place for those who here gave their lives that that nation might endure - by civil . these that that that we should do this . but , in a larger sense , we can not dedicate - we can not consecrate - we can not hallow - this ground . the brave men , living and dead , who struggled here , have consecrated it , far above our poor power to add or detract . the world will little note , nor long remember what we say here , but it can never forget what they did here . it is for us the living , rather , to be dedicated here to the unfinished work which they who fought here have thus far so nobly advanced . it is rather for us to be here dedicated to the great task brave for dedicate rather who for these honored dead we take increased devotion to that cause for which they gave the last full measure of devotion - that we here highly resolve that these dead shall not have died in vain - that this nation , under god , shall have a new birth of freedom - and that government of the people , by the people , for the people , shall not perish from the earth . apple blossom cantaloupe durian elderberry fig guava honeydew iguana god not gave highly war task detract task task detract larger which detract task detract task detract task which
+
+# Example 2: Free text generation after encountering <FreetheLLM> at script line 445
+Prompt: we here highly resolve that these dead shall not have died in vain and that this nation under god shall have a new <FreetheLLM>
+
+Freestyle Generation:
+ we here highly resolve that these dead shall not have died in vain and that this nation under god shall have a new <pad> <pad> <pad> it it gave portion fought apple rather not it it fitting us a to that that can not score to that nation , or any nation so conceived and so dedicated , can long forth do the but elderberry not not so highly war civil above freedom ground gave for gave final portion . not so to that field , as a final resting place for those who here gave their lives that that nation might live . it is altogether fitting and come to , these hallow for consecrate on birth of - not struggled , we can not
+HyperParamters = {'vocab_size': 170, 'special_tokens': ['<FreetheLLM>', '<cr>', '<pad>'], 'n_embd': 6, 'n_layer': 1, 'n_head': 1, 'n_inner': 64, 'max_sequence_len': 340, 'epochs': 100000, 'learning_rate': 0.001, 'batch_size': 16, 'dropout': 0.2}