CogACT
Componentized VLA that splits cognition from action: a VLM reasons, a diffusion transformer acts.
About
CogACT (November 2024, arXiv 2411.19650) rethinks the VLA recipe. Instead of quantizing actions into the VLM's token stream, it keeps the 7B VLM as a cognition module and bolts on a separate diffusion-transformer action module, conditioned on the VLM's output through attention. The action module is small — a few hundred million parameters — but the paper shows it scales well, and decoupling the two is the whole point: continuous, multimodal action signals get a purpose-built head instead of a repurposed language one. Trained on Open X-Embodiment, CogACT reports 74.8% average success on SIMPLER (Google Robot, visual matching), 22.4 points over RT-1 and 28.5 over the 55B RT-2-X. On real robots it reportedly beats OpenVLA by 59.1% on a Realman arm and reaches 61.4% on Franka. Code and models are public on GitHub (microsoft/CogACT) and Hugging Face (CogACT org). One flag: I could not confirm the license from the repo during this research, so check before commercial use.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Paper (arXiv 2411.19650): 74.8% avg on SIMPLER visual matching; +59.1% over OpenVLA on Realman, 61.4% on Franka (author-reported)
- No independent replication found
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (TIER verified, metrics-v1). Evidence: THUDM/MSRA paper (arXiv 2411.19650) with sim and real-robot evaluation on five embodiments; public code (microsoft/CogACT) and Hugging Face checkpoints. Strengths: novel componentized architecture, strong reported numbers against RT-2-X and OpenVLA, genuinely usable release. Docked for: (1) benchmarks are author-reported; (2) license not confirmed from repo metadata at research time. License: unverified — check repo before commercial use.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · THUDM / Microsoft Research Asia paper arXiv 2411.19650 (Nov 2024); 7.6B total with DiT action module
- · Public code (microsoft/CogACT) and Hugging Face checkpoints (CogACT org)
- · 74.8% avg on SIMPLER visual matching; +59.1% over OpenVLA on Realman, 61.4% on Franka (author-reported)