Update README.md

README.md

@@ -5,31 +5,34 @@ tags:
 model-index:
 - name: retnet-mini-shakespeare
   results: []
+pipeline_tag: text-generation
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 # retnet-mini-shakespeare
 
-This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 2.7718
+This model was trained from scratch on "tinyshakespeare" text file.
 
 ## Model description
 
-More information needed
+A tiny model similar to jploski/falcon-mini-shakespeare, to demonstrate training and recurrent inference using a retention network (https://arxiv.org/pdf/2307.08621.pdf).
+The code utilizes Sehyun Choi's implementation of retention network (https://github.com/syncdoth/RetNet) with configuration parameters changed to make it a very tiny model.
+
+- **License:** Apache 2.0.
 
 ## Intended uses & limitations
 
-More information needed
+Intended to demonstrate training and (recurrent O(1)) inference using a retention network
 
 ## Training and evaluation data
 
-More information needed
+https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
 
 ## Training procedure
 
+Just used the single tinyshakespeare text file as both the training and validation set (split up into paragraphs). See:
+
+https://colab.research.google.com/drive/1wZnM7FCe4TsQpoamJ7NDAuQfA3DYiwHi?usp=sharing
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
@@ -59,4 +62,4 @@ The following hyperparameters were used during training:
 - Transformers 4.31.0
 - Pytorch 2.0.1+cu118
 - Datasets 2.14.3
-- Tokenizers 0.13.3
+- Tokenizers 0.13.3