WiLoR
End-to-end in-the-wild 3D hand localization and reconstruction — detection plus mesh recovery in one pipeline.
About
Most 3D hand papers assume someone already cropped the hand for you. WiLoR does the whole job: a real-time fully-convolutional detector finds hands in full images, then a transformer-based reconstruction model recovers the 3D hand mesh for each detection — end to end, including smooth multi-hand tracking from monocular video without any temporal modeling tricks. To make the detector robust, the authors built WHIM, a 2M+ image in-the-wild hand dataset covering diverse lighting, occlusion and viewpoints.
The pipeline is open on GitHub with a Gradio demo, and per the authors it beats prior methods on both 2D and 3D benchmarks. The license is the thing to read carefully: the models are CC-BY-NC-ND, so research and demos are fine but commercial use and redistribution of derivatives are off the table. It also inherits the MANO license for the hand model, which needs a separate registration.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Paper (arXiv 2409.12259): outperforms prior methods on 2D and 3D hand benchmarks (author-reported)
- Trained with the 2M+ image WHIM in-the-wild hand dataset
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 6/10
Scored 6/10 (TIER metrics, metrics-v1). Evidence: arXiv paper (2409.12259) with benchmark evals; official repo (rolpotamias/WiLoR) with models on Hugging Face; WHIM 2M-image dataset release; project page live. Strengths: end-to-end pipeline fills a real gap (detection + reconstruction); dataset scale is a plus. Docked for: (1) models under CC-BY-NC-ND — no commercial use, no derivatives; (2) smaller community and newer than HaMeR; (3) headline numbers are author-reported. License: CC-BY-NC-ND.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 5.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 6.0/10
- Manual review · sanity read + red-flag check
- 6.0/10
- · arXiv paper 2409.12259; authors at Imperial College London (Potamias, Zhang, Deng, Zafeiriou)
- · Official repo rolpotamias/WiLoR; models on Hugging Face; WHIM 2M-image in-the-wild hand dataset released
- · Models under CC-BY-NC-ND — no commercial use, no derivatives