Update README.md

README.md

@@ -11,22 +11,47 @@ should probably proofread and complete it, then remove this comment. -->
 
 # gpt2-arxiv
 
-This model was trained from scratch on an unknown dataset.
+A [gpt2](https://huggingface.co/gpt2) powered predictive keyboard trained on ~1.6M manuscript abstracts from the ArXiv. 
+This model uses https://www.kaggle.com/datasets/Cornell-University/arxiv
+
+```python
+from transformers import pipeline
+from transformers import GPT2TokenizerFast
+
+tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")
+llm = pipeline('text-generation',model='pearsonkyle/gpt2-arxiv', tokenizer=tokenizer)
+
+texts = llm("Directly imaged exoplanets probe", 
+             max_length=50, do_sample=True, num_return_sequences=5, 
+             penalty_alpha=0.65, top_k=4, repetition_penalty=1.2)
+
+for i in range(5):
+    print(texts[i]['generated_text']+'\n')
+```
+
+- *The reflectance of Earth's vegetation suggests* `that large, deciduous forest fires are composed of mostly dry, unprocessed material that is distributed in a nearly patchy fashion. The distributions of these fires are correlated with temperature, and also with vegetation...`
+- *Directly imaged exoplanets probe* the atmospheres of giant planets. The detection of such planets requires high-quality imaging with high contrast and angular resolution, as well as
+- *We can remotely sense an atmosphere by observing its reflected, transmitted, or emitted light in varying geometries. This light will contain information on* the planetary conditions including atmospheric temperature and cloud properties, which is essential for understanding how the planet interacts with the atmosphere and how it affects the climate. The primary science objective of this paper is to develop a methodology that can be applied to any kind of observation and measurement data, and to provide a framework that enables the detection and characterisation of the atmospheres of ex
+
 
 ## Model description
 
-More information needed
+[GPT-2](https://huggingface.co/transformers/v2.2.0/pretrained_models.html): 12-layer, 768-hidden, 12-heads, 117M parameters
 
 ## Intended uses & limitations
 
-More information needed
+Coming soon...
+- Predictive Keyboard using text generation
+- Realtime reference recommendations using nearest neighbors of embeddings
 
-## Training and evaluation data
+Be careful when generating a lot of text...The things it says are not that truthful, maybe more training is needed
 
-More information needed
+- The surface of Mars is composed of a thin layer of water ice, that was discovered by the Cassini spacecraft after its impact on the Earth's surface.
 
 ## Training procedure
 
+~49 hours on a 3090 training for 1.25M iterations
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
@@ -36,7 +61,7 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 50
+- num_epochs: 10
 
 ### Framework versions