Update README.md

README.md

@@ -8,9 +8,57 @@ pipeline_tag: text-generation
 tags:
 - Sports
 ---
+# OnlySportsLM
 
-OnlySportsLM is A 196 million parameter RWKV-v6 based sports language model trained on half of the OnlySports Dataset. 
-In our OnlySports Benchmark, OnlySportsLM outperforms the preceding SOTA general purpose 135M/360M language model by 37.62%/34.08%.
+## Model Overview
 
-Please download the model file and use RWKV-v6-demo.py for inference.
+OnlySportsLM is a 196 million parameter language model specifically designed and trained for sports-related natural language processing tasks. It is part of the larger OnlySports collection, which aims to advance domain-specific language modeling in the sports domain.
 
+## Model Architecture
+
+- Base architecture: RWKV-v6
+- Parameters: 196 million
+- Structure: 20 layers, 640 dimensions
+
+## Training
+
+- Dataset: OnlySports Dataset (subset of 315B tokens out of 600B total)
+- Training setup: 8 H100 GPUs
+- Optimizer: AdamW with weight decay of 0.1
+- Learning rate: Initially 6e-4, adjusted to 1e-4 due to observed loss spikes
+- Context length: 1024 tokens
+
+## Performance
+
+OnlySportsLM shows impressive performance on sports-related tasks:
+
+- Outperforms previous SOTA 135M/360M models by 37.62%/34.08% on the OnlySports Benchmark
+- Competitive with larger models like SomlLM 1.7B and Qwen 1.5B in the sports domain
+- Demonstrates improved performance on general zero-shot tasks throughout training
+
+For detailed performance metrics, please refer to our [technical report](https://github.com/chrischenhub/OnlySportsLM).
+
+## Usage
+
+You can use this model for various sports-related content generation.
+
+Download all files in this repo. Open RWKV_v6_demo.py for inference.
+
+## Limitations
+
+- The model is specifically trained on sports-related content and may not perform as well on general topics
+- Training was stopped at 315B tokens due to resource constraints, potentially limiting its full capabilities
+
+## Related Resources
+
+- [OnlySports Dataset](https://huggingface.co/collections/Chrisneverdie/onlysports-66b3e5cf595eb81220cc27a6)
+- [Sports Text Classifier](https://huggingface.co/Chrisneverdie/OnlySports_Classifier)
+- [GitHub Repository](https://github.com/chrischenhub/OnlySportsLM)
+
+## Citation
+
+If you use OnlySportsLM in your research, please cite our paper (citation details to be added upon publication).
+
+## Contact
+
+For more information or inquiries about OnlySportsLM, please visit our [GitHub repository](https://github.com/chrischenhub/OnlySportsLM) or email [email protected].