Publication

Gripmarks: Using Hand Grips to Transform In-Hand Objects into Mixed Reality Input

Conference on Human Factors in Computing Systems (CHI)


Abstract

We introduce Gripmarks, a system that enables users to opportunistically use objects they are already holding as input surfaces for mixed reality head-mounted displays (HMD). Leveraging handheld objects reduces the need for users to free up their hands or acquire a controller to interact with their HMD. Gripmarks associate a particular hand grip with the shape primitive of the physical object without the need of object recognition or instrumenting the object. From the grip pose and shape primitive we can infer the surface of the object. With an activation gesture, we can enable the object for use as input to the HMD. With five gripmarks we demonstrate a recognition rate of 94.2%; we show that our grip detection benefits from the physical constraints of holding an object. We explore two categories of input objects 1) tangible surfaces and 2) tangible tools and present two representative applications. We discuss the design and technical challenges for expanding the concept.

Related Publications

All Publications

ARCH: Animatable Reconstruction of Clothed Humans

Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, Tony Tung

CVPR - June 15, 2020

Head-Coupled Kinematic Template Matching: A Prediction Model for Ray Pointing in VR

Rorik Henrikson, Tovi Grossman, Sean Trowbridge, Daniel Wigdor, Hrvoje Benko

CHI - April 27, 2020

Crossing the Chasm: Linking with the Virtual World through a Compact Haptic Actuator

Pornthep Preechayasomboon, Ali Israr

ACM CHI - April 25, 2020

Explorations of Wrist Haptic Feedback for AR/VR Interactions with Tasbi

Evan Pezent, Marcia K. O’Malley, Ali Israr, Majed Samad, Shea Robinson, Priyanshu Agarwal, Hrvoje Benko, Nicholas Colonnese

CHI - April 25, 2020

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy