Listening to Hands: Robust Acoustic Multi-Hand Pose Reconstruction with RAM-Hand

Hands are one of our most natural tools for interacting with the world around us. We use them to point, grasp, hold, move, and manipulate objects, and we also use them to communicate with other people. To make future interactions, from virtual environments to smart devices feel this natural, a system needs more than the ability to recognize a few predefined hand gestures. It needs to recover the 3D hand pose, capturing where the hand is, how the fingers bend, and how one or more hands move over time.

This capability could make human-computer interaction more natural and intuitive. In augmented and virtual reality, users could manipulate virtual objects with bare hands instead of relying on controllers. In remote collaboration, hand poses could help people point to, organize, and move shared content more easily. Beyond virtual environments, the same idea could support touch-free smart-home control or allow a user's hand motion to guide a robotic arm in manipulating physical objects.