HaMeR

HaMeR reconstructs a full 3D hand mesh from one RGB image using a fully transformer-based architecture: a large Vision Transformer backbone extracts features, and a transformer decoder regresses MANO hand parameters. The key insight was scale — the authors trained on a much larger mix of 2D- and 3D-annotated hand datasets than prior work, which is why it holds up on in-the-wild inputs that break older methods: occluded fingers, hands holding objects, gloves, egocentric viewpoints, even painted or mechanical hands.

It was a CVPR 2024 paper and took 2nd place in the Ego-Pose Hands task of the 2024 Ego-Exo4D Challenge. The code and weights are MIT-licensed, with a Hugging Face demo, and it's become a go-to component in robotics and AR hand-tracking pipelines. You'll need to register on the MANO website to download the hand model itself, but that's a one-time formality.

• Paper (arXiv 2312.05251): consistent improvements over prior baselines on FreiHAND and HO-3D
• Introduced the HInt in-the-wild hand annotation set
• 2nd place, Ego-Pose Hands task, Ego-Exo4D Challenge (2024)