Scaling laws for language models
Loss scales as a power law with model size, dataset size and compute used for training.
Official source
Scaling laws for neural language modelsopenai.com · published
Loss scales as a power law with model size, dataset size and compute used for training.
Scaling laws for neural language modelsopenai.com · published