Explore the latest research from Facebook

All Publications

October 11, 2021 Hao Jiang, Vamsi Krishna Ithapu
Paper

Egocentric Pose Estimation from Human Vision Span

In this paper, we tackle the egopose estimation from a more natural human vision span, where camera wearer can be seen in the peripheral view and depending on the head pose the wearer may become invisible or has a limited partial view.
Paper
October 1, 2021 Hung Le, Chinnadhurai Sankar, Seungwhan Moon, Ahmad Beirami, Alborz Geramifard, Satwik Kottur
Paper

DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue

The dataset is designed to contain minimal biases and has detailed annotations for the different types of reasoning over the spatio-temporal space of video. Dialogues are synthesized over multiple question turns, each of which is injected with a set of cross-turn semantic relationships. We use DVD to analyze existing approaches, providing interesting insights into their abilities and limitations.
Paper
September 16, 2021 Alex Adkins, Lorraine Lin, Aline Normoyle, Ryan Canales, Yuting Ye, Sophie Jörg
Paper

Evaluating Grasping Visualizations and Control Modes in a VR Game

In this paper, we investigate if results of such studies translate into a real application and game-like experience. We designed a virtual escape room in which participants interact with various objects to gather clues and complete puzzles.
Paper
August 23, 2021 Jacob Donley, Vladimir Tourbabin, Boaz Rafaely, Ravish Mehra
Paper

Adaptive Multi-Channel Signal Enhancement Based on Multi-Source Contribution Estimation

This paper outlines a new method to adapt to desired and undesired signals using their spatial statistics, independent of their temporal characteristics. The method uses a linearly constrained minimum variance (LCMV) beamformer to estimate the relative source contribution of each source in a mixture, which is then used to weight statistical estimates of the spatial characteristics of each source used for final separation.
Paper
August 9, 2021 Jungdam Won, Deepak Gopinath, Jessica Hodgins
Paper

Control Strategies for Physically Simulated Characters Performing Two-player Competitive Sports

In this paper, we develop a learning framework that generates control policies for physically simulated athletes who have many degrees-of-freedom. Our framework uses a two step-approach, learning basic skills and learning boutlevel strategies, with deep reinforcement learning, which is inspired by the way that people how to learn competitive sports.
Paper
August 9, 2021 Cheng Zhang, Zhao Dong, Michael Doggett, Shuang Zhao
Paper

Antithetic Sampling for Monte Carlo Differentiable Rendering

In this paper, we demonstrate that path sampling techniques developed for forward rendering can become inefficient for differentiable rendering of glossy materials—especially when estimating derivatives with respect to global scene geometries. To address this problem, we introduce antithetic sampling of BSDFs and light-transport paths, allowing significantly faster convergence and can be easily integrated into existing differentiable rendering pipelines.
Paper
August 9, 2021 He Zhang, Yuting Ye, Takaaki Shiratori, Taku Komura
Paper

ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation

In this paper, we propose a hand-object spatial representation that can achieve generalization from limited data. Our representation combines the global object shape as voxel occupancies with local geometric details as samples of closest distances. This representation is used by a neural network to regress finger motions from input trajectories of wrists and objects.
Paper
August 9, 2021 Sai Bi, Stephen Lombardi, Shunsuke Saito, Tomas Simon, Shih-En Wei, Kevyn Mcphail, Ravi Ramamoorthi, Yaser Sheikh, Jason Saragih
Paper

Deep Relightable Appearance Models for Animatable Faces

We present a method for building high-fidelity animatable 3D face models that can be posed and rendered with novel lighting environments in real-time.
Paper
August 9, 2021 Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, Jason Saragih
Paper

Mixture of Volumetric Primitives for Efficient Neural Rendering

We present Mixture of Volumetric Primitives (MVP), a representation for rendering dynamic 3D content that combines the completeness of volumetric representations with the efficiency of primitive-based rendering, e.g., point-based or mesh-based methods. Our approach achieves this by leveraging spatially shared computation with a convolutional architecture and by minimizing computation in empty regions of space with volumetric primitives that can move to cover only occupied regions.
Paper
August 9, 2021 Timur Bagautdinov, Chenglei Wu, Tomas Simon, Fabián Prada, Takaaki Shiratori, Shih-En Wei, Weipeng Xu, Yaser Sheikh, Jason Saragih
Paper

Driving-Signal Aware Full-Body Avatars

The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation.
Paper