Nougat

Nougat (Neural Optical Understanding for Academic Documents) is a visual transformer from Meta AI that performs end-to-end OCR on scientific documents. Instead of relying on embedded PDF text, it reads the rasterized page image and generates a lightweight markup language, which lets it recover semantic structure that conventional extraction loses, especially mathematical expressions and tables.

The model pairs a Swin Transformer vision encoder with an mBART text decoder and was trained on millions of academic papers from arXiv and PubMed via an automated PDF-to-source matching pipeline. It outperformed GROBID and raw PDF text extraction in the paper's evaluations and can handle scanned documents. Meta released the models and code (arXiv 2308.13418, August 2023); check the Hugging Face model card for current license terms before commercial use.