DiT
Microsoft's self-supervised Document Image Transformer — a BEiT-style encoder pre-trained on 42 million document images.
About
DiT (Document Image Transformer) was Microsoft's answer to a simple observation: there are no human-labeled document image datasets at ImageNet scale, so document vision needed self-supervision. It takes the BEiT recipe (masked patch prediction against a discrete VAE tokenizer) and pre-trains on IIT-CDIP, a collection of 42 million document images, learning a general visual representation of documents.
The pre-trained encoder is a backbone, not a finished product: fine-tune it for document image classification, layout analysis, table detection, or text detection for OCR. The paper (arXiv 2203.02378, 2022) reports new state-of-the-art on all four of those downstream tasks, and checkpoints from base to large live on Hugging Face under the microsoft org.
It is showing its age next to modern VLMs, but as a pure-vision document backbone it is still one of the most reused pre-training starting points in Document AI research.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Author-reported (arXiv 2203.02378, 2022): document image classification 92.69 (from 91.11), layout analysis 94.9 (from 91.0), ICDAR 2019 table detection 96.55, text detection for OCR 94.29 — new SOTA on all four at publication
- Pre-trained on IIT-CDIP (42M document images), self-supervised
- No independent replication of the headline numbers; widely used as a backbone since
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (TIER verified, metrics-v1). Evidence: Microsoft Research paper (arXiv 2203.02378, Mar 2022) with released code and checkpoints on Hugging Face (microsoft/dit-base and larger), heavily reused as a Document AI backbone. Strengths: fully open, foundational pre-training approach, strong adoption as a backbone. Docked for: (1) all headline numbers are author-reported; (2) 2022-era — superseded by modern VLMs for end-to-end tasks, maintenance is archival. License: Microsoft research release; no explicit license field on the HF card — verify before commercial use.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · Microsoft Research paper (arXiv 2203.02378, Mar 2022): BEiT-style masked-patch pre-training on IIT-CDIP (42M document images).
- · Released checkpoints (microsoft/dit-base/large and fine-tunes) on Hugging Face; heavily reused as a Document AI backbone.
- · Author-reported SOTA on 4 downstream tasks at publication (RVL-CDIP 92.69, PubLayNet 94.9, table detection 96.55, text detection 94.29); 2022-era, superseded by modern VLMs for end-to-end tasks.