DialoGPT

DialoGPT, released by Microsoft Research in November 2019 (arXiv:1911.00536, ACL 2020 demo), was one of the first serious attempts to scale pretraining specifically for dialogue. Starting from GPT-2, the team trained on 147 million conversation-like exchanges extracted from Reddit comment chains spanning 2005–2017, releasing small (117M), medium (345M), and large (762M) checkpoints.

Human evaluations found its single-turn responses comparable to human quality under a Turing-test-style setup, and it set the template for the conversational pretraining that later chat models industrialized. As a historical artifact it carries honest caveats: Reddit-trained models inherit Reddit's biases and toxicity, and the work predates RLHF-era safety practice. Code and weights were released openly, and the model remains a standard reference in dialogue research.