Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
Paper
•
2407.13623
•
Published
•
54
📈 Scaling Laws with Vocabulary
Increase your vocabulary size when you scale up your language model
Increase your vocabulary size when you scale up your language model
Predict optimal vocabulary size based on model parameters
