GPT-2
A 1.5B-parameter language model trained to predict the next word; only a much smaller model was released.
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.
Official source
Better language models and their implicationsopenai.com · published