Research Area
Year Published

105 Results

August 10, 2018

Effects of virtual acoustics on target-word identification performance in multi-talker environments

ACM Symposium on Applied Perception (SAP)

Many virtual reality applications let multiple users communicate in a multi-talker environment, recreating the classic cocktail-party effect. While there is a vast body of research focusing on the perception and intelligibility of human speech in real-world scenarios with cocktail party effects, there is little work in accurately modeling and evaluating the effect in virtual environments.

By: Atul Rungta, Nicholas Rewkowski, Carl Schissler, Philip Robinson, Ravish Mehra, Dinesh Manocha
Areas: AR/VR

August 10, 2018

Learning to Feel Words: A Comparison of Learning Approaches to Acquire Haptic Words

ACM Symposium on Applied Perception (SAP)

Recent studies have shown that decomposing spoken or written language into phonemes and transcribing each phoneme into a unique vibrotactile pattern enables people to receive lexical messages on the arm. A potential barrier to adopting this new communication system is the time and effort required to learn the association between phonemes and vibrotactile patterns. Therefore, in this study, we compared the learnability and generalizability of different learning approaches, including guided learning, self-directed learning, and a mnemonic device.

By: Jennifer Chen, Robert Turcott, Pablo Castillo, Wahyudinata Setiawan, Frances Lau, Ali Israr
Areas: AR/VR

July 30, 2018

Instant 3D Photography


We present an algorithm for constructing 3D panoramas from a sequence of aligned color-and-depth image pairs. Such sequences can be conveniently captured using dual lens cell phone cameras that reconstruct depth maps from synchronized stereo image capture.

By: Peter Hedman, Johannes Kopf

July 29, 2018

Online Optical Marker-based Hand Tracking with Deep Labels

Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH)

We propose a technique that frames the labeling problem as a keypoint regression problem conducive to a solution using convolutional neural networks.

By: Shangchen Han, Beibei Liu, Robert Wang, Yuting Ye, Christopher D. Twigg, Kenrick Kin

July 7, 2018

Reconstructing Scenes with Mirror and Glass Surfaces


We introduce a fully automatic pipeline that allows us to reconstruct the geometry and extent of planar glass and mirror surfaces while being able to distinguish between the two.

By: Thomas Whelan, Michael Goesele, Steven J. Lovegrove, Julian Straub, Simon Green, Richard Szeliski, Steven Butterfield, Shobhit Verma, Richard Newcombe

July 6, 2018

DeepFocus: Learned Image Synthesis for Computational Display

ACM SIGGRAPH (Talks Program)

In this paper, we introduce Deep-Focus, a generic, end-to-end trainable convolutional neural network designed to efficiently solve the full range of computational tasks for accommodation-supporting HMDs.

By: Lei Xiao, Anton Kaplanyan, Alexander Fix, Matt Chapman, Douglas Lanman

June 18, 2018

Modeling Facial Geometry using Compositional VAEs

Computer Vision and Pattern Recognition (CVPR)

We propose a method for learning non-linear face geometry representations using deep generative models. Our model is a variational autoencoder with multiple levels of hidden variables where lower layers capture global geometry and higher ones encode more local deformations.

By: Timur Bagautdinov, Chenglei Wu, Jason Saragih, Pascal Fua, Yaser Sheikh

June 18, 2018

Eye In-Painting with Exemplar Generative Adversarial Networks

Computer Vision and Pattern Recognition (CVPR)

This paper introduces a novel approach to in-painting where the identity of the object to remove or change is preserved and accounted for at inference time: Exemplar GANs (ExGANs). ExGANs are a type of conditional GAN that utilize exemplar information to produce high-quality, personalized in-painting results.

By: Brian Dolhansky, Cristian Canton Ferrer

June 18, 2018

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

Computer Vision and Pattern Recognition (CVPR)

We present a unified deformation model for the markerless capture of human movement at multiple scales, including facial expressions, body motion, and hand gestures.

By: Hanbyul Joo, Tomas Simon, Yaser Sheikh

June 18, 2018

Audio to Body Dynamics

Computer Vision and Pattern Recognition (CVPR)

We present a method that gets as input an audio of violin or piano playing, and outputs a video of skeleton predictions which are further used to animate an avatar. The key idea is to create an animation of an avatar that moves their hands similarly to how a pianist or violinist would do, just from audio.

By: Eli Shlizerman, Lucio Dery, Hayden Schoen, Ira Kemelmacher Shlizerman