HiRobot
Hierarchical VLA: a high-level VLM turns open-ended instructions into commands, a low-level policy executes them.
About
HiRobot (February 2025, arXiv 2502.19417) tackles a problem flat VLAs fumble: open-ended instructions like 'clean up the table but leave my mug alone.' It splits the job in two. A high-level VLM policy watches the scene and the user's request and emits short language commands; a low-level VLA executes them. Because the high level 'speaks' in language, a human evaluator can judge whether it understood the intent — DeepMind's key metric, Instruction Accuracy, alongside task-progress scoring on long-horizon tabletop tasks. The system consistently beat GPT-4o-as-policy and flat VLA baselines on following nuanced instructions. As with most DeepMind robot work, no code or weights were released. It stands as a peer-visible design with 20-trials-per-task human evaluation, not a product.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Paper (arXiv 2502.19417): beats GPT-4o policy and flat VLA baselines on Instruction Accuracy and Task Progress across long-horizon tasks (author-reported, human-evaluated, 20 trials per task)
- No independent replication found
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (TIER metrics, metrics-v1). Evidence: Google DeepMind paper (arXiv 2502.19417) with human-evaluated open-ended instruction following against GPT-4o and flat VLA baselines. Strengths: interpretable hierarchy, careful human evaluation protocol. Docked for: (1) no public code or weights; (2) all results are author-reported. License: none — no public release.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 4.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Google DeepMind paper arXiv 2502.19417 (Feb 2025); hierarchical VLA with language sub-command interface
- · Instruction Accuracy + Task Progress human evaluation, 20 trials per task
- · Beats GPT-4o-as-policy and flat VLA baselines on nuanced instructions (author-reported)