LayoutLMv3

LayoutLMv3 is Microsoft's multimodal pre-trained Transformer for Document AI, released in April 2022. It unifies text and image masking objectives and adds a word-patch alignment objective that teaches cross-modal alignment between words and image patches.

The simple unified architecture made it a general-purpose backbone for both text-centric tasks (form understanding, receipt understanding, document VQA) and image-centric ones (document classification, layout analysis), reaching state of the art on both at release. It is MIT licensed via the unilm repository.