Publication

Online Optical Marker-based Hand Tracking with Deep Labels

Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH)


Abstract

Optical marker-based motion capture is the dominant way for obtaining high-fidelity human body animation for special effects, movies, and video games. However, motion capture has seen limited application to the human hand due to the difficulty of automatically identifying (or labeling) identical markers on self-similar fingers. We propose a technique that frames the labeling problem as a keypoint regression problem conducive to a solution using convolutional neural networks. We demonstrate robustness of our labeling solution to occlusion, ghost markers, hand shape, and even motions involving two hands or handheld objects. Our technique is equally applicable to sparse or dense marker sets and can run in real-time to support interaction prototyping with high-fidelity hand tracking and hand presence in virtual reality.

Related Publications

All Publications

Open Source Evolutionary Structured Optimization

Jeremy Rapin, Pauline Bennet, Emmanuel Centeno, Daniel Haziza, Antoine Moreau, Olivier Teytaud

Evolutionary Computation Software Systems Workshop at ​GECCO - July 9, 2020

Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning

Tianyu Li, Nathan Lambert, Roberto Calandra, Franziska Meier, Akshara Rai

ICRA - June 1, 2020

Large Scale Audiovisual Learning of Sounds with Weakly Labeled Data

Haytham M. Fayek, Anurag Kumar

IJCAI - July 11, 2020

A Synthesis-Aided Compiler for DSP Architectures (WiP Paper)

Alexa VanHattum, Rachit Nigam, Vincent T. Lee, James Bornholt, Adrian Sampson

LCTES - June 15, 2020

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy