Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ tags:
|
|
12 |
|
13 |
## Model Overview
|
14 |
|
15 |
-
OnlySportsLM is a
|
16 |
|
17 |
## Model Architecture
|
18 |
|
@@ -22,9 +22,9 @@ OnlySportsLM is a 196 million parameter language model specifically designed and
|
|
22 |
|
23 |
## Training
|
24 |
|
25 |
-
- Dataset: OnlySports Dataset (subset of 315B tokens out of 600B total)
|
26 |
- Training setup: 8 H100 GPUs
|
27 |
-
- Optimizer: AdamW
|
28 |
- Learning rate: Initially 6e-4, adjusted to 1e-4 due to observed loss spikes
|
29 |
- Context length: 1024 tokens
|
30 |
|
|
|
12 |
|
13 |
## Model Overview
|
14 |
|
15 |
+
OnlySportsLM is a 196M language model specifically designed and trained for sports-related natural language processing tasks. It is part of the larger OnlySports collection, which aims to advance domain-specific language modeling in sports.
|
16 |
|
17 |
## Model Architecture
|
18 |
|
|
|
22 |
|
23 |
## Training
|
24 |
|
25 |
+
- Dataset: [OnlySports Dataset](https://huggingface.co/datasets/Chrisneverdie/OnlySports_Dataset) (subset of 315B tokens out of 600B total)
|
26 |
- Training setup: 8 H100 GPUs
|
27 |
+
- Optimizer: AdamW
|
28 |
- Learning rate: Initially 6e-4, adjusted to 1e-4 due to observed loss spikes
|
29 |
- Context length: 1024 tokens
|
30 |
|