Publications

January 1, 2020

Ethics in Design and Communication

Virtual Reality (VR) designers accept the ethical responsibilities of removing a user’s entire world and superseding it with a fabricated reality. These unique immersive design challenges are intensified when virtual experiences become public and socially-driven. As female VR designers in 2018, we see an opportunity to fold the language of consent into the design practice of virtual reality—as a means to design safe, accessible, virtual spaces.

Publication will be made available in 2020.

Areas: AR/VR Human Computer Interaction & UX

July 31, 2019

SIGGRAPH

To overcome memory limitations of voxel-based representations, we learn a dynamic irregular grid structure implemented with a warp field during ray-marching. This structure greatly improves the apparent resolution and reduces grid-like artifacts and jagged motion. Finally, we demonstrate how to incorporate surface-based representations into our volumetric-learning framework for applications where the highest resolution is required, using facial performance capture as a case in point.

Areas: AR/VR Computer Vision Machine Learning

July 12, 2019

SIGGRAPH

In this work, we present a bidirectional system that can animate avatar heads of both users’ full likeness using consumer-friendly headset mounted cameras (HMC). There are two main challenges in doing this: unaccommodating camera views and the image-to-avatar domain gap. We address both challenges by leveraging constraints imposed by multiview geometry to establish precise image-to-avatar correspondence, which are then used to learn an end-to-end model for real-time tracking.

Areas: AR/VR Computer Vision Facebook AI Research Machine Learning

July 7, 2019

IEEE World Haptics Conference

Inter-personal touch is a powerful aspect of social interaction that we expect to be particularly important for emotional communication. We studied the capacity of closely acquainted humans to signal the meaning of several word cues (e.g. gratitude, sadness) using touch sensation alone.

Areas: AR/VR

July 7, 2019

IEEE World Haptics Conference

Couples often communicate their emotions, e.g., love or sadness, through physical expressions of touch. Prior efforts have used visual observation to distinguish emotional touch communications by certain gestures tied to one’s hand contact, velocity and position. The work herein describes an automated approach to eliciting the essential features of these gestures.

Areas: AR/VR

July 7, 2019

IEEE World Haptics Conference

Human-to-human touch conveys rich, meaningful social and emotional sentiment. At present, however, we understand neither the physical attributes that underlie such touch, nor how the attributes evoke responses in unique types of peripheral afferents. Indeed, nearly all electrophysiological studies use well-controlled but non-ecological stimuli. Here, we develop motion tracking and algorithms to quantify physical attributes – indentation depth, shear velocity, contact area, and distance to the cutaneous sensory space (receptive field) of the afferent – underlying human-to-human touch.

Areas: AR/VR

July 7, 2019

IEEE World Haptics Conference

We present a skin-shear actuator based on the lead screw mechanism. The lead screw mechanism is simple, reliable, offers fewer components, and accommodates into compact form-factors. We show mechanical design of a single assembly unit and implement multiple units in a single handheld device. We evaluate the actuator in one instrumentation-based test and one preliminary user study.

Areas: AR/VR

June 14, 2019

Conference Computer Vision and Pattern Recognition (CVPR)

Binaural audio provides a listener with 3D sound sensation, allowing a rich perceptual experience of the scene. However, binaural recordings are scarcely available and require nontrivial expertise and equipment to obtain. We propose to convert common monaural audio into binaural audio by leveraging video.

Areas: AR/VR Computer Vision Facebook AI Research

May 31, 2019

IEEE International Conference on Acoustics, Speech and Signal Processing

This paper addresses the problem of soundfield reconstruction over a large area using a distributed array of higher-order microphones. Given an area enclosed by the array, one can distinguish between two components of the soundfield: the interior soundfield generated by sources outside of the enclosed area and the exterior soundfield generated by sources inside the enclosed area.

Areas: AR/VR

May 17, 2019

SID Display Week

Little is known about user sensitivity to flicker and eye movement-induced ghosting at refresh rates and persistence levels relevant for head-mounted displays (HMDs). In this report, we describe a pair of psychophysical experiments that we performed to comprehensively quantify these artifacts and make general recommendations for HMD design.

Areas: AR/VR Human Computer Interaction & UX