Explore the latest research from Facebook

All Publications

August 9, 2021 Sai Bi, Stephen Lombardi, Shunsuke Saito, Tomas Simon, Shih-En Wei, Kevyn Mcphail, Ravi Ramamoorthi, Yaser Sheikh, Jason Saragih
Paper

Deep Relightable Appearance Models for Animatable Faces

We present a method for building high-fidelity animatable 3D face models that can be posed and rendered with novel lighting environments in real-time.
Paper
August 9, 2021 Jungdam Won, Deepak Gopinath, Jessica Hodgins
Paper

Control Strategies for Physically Simulated Characters Performing Two-player Competitive Sports

In this paper, we develop a learning framework that generates control policies for physically simulated athletes who have many degrees-of-freedom. Our framework uses a two step-approach, learning basic skills and learning boutlevel strategies, with deep reinforcement learning, which is inspired by the way that people how to learn competitive sports.
Paper
August 9, 2021 Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, Jason Saragih
Paper

Mixture of Volumetric Primitives for Efficient Neural Rendering

We present Mixture of Volumetric Primitives (MVP), a representation for rendering dynamic 3D content that combines the completeness of volumetric representations with the efficiency of primitive-based rendering, e.g., point-based or mesh-based methods. Our approach achieves this by leveraging spatially shared computation with a convolutional architecture and by minimizing computation in empty regions of space with volumetric primitives that can move to cover only occupied regions.
Paper
August 9, 2021 He Zhang, Yuting Ye, Takaaki Shiratori, Taku Komura
Paper

ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation

In this paper, we propose a hand-object spatial representation that can achieve generalization from limited data. Our representation combines the global object shape as voxel occupancies with local geometric details as samples of closest distances. This representation is used by a neural network to regress finger motions from input trajectories of wrists and objects.
Paper
June 19, 2021 Shugao Ma, Tomas Simon, Jason Saragih, Dawei Wang, Yuecheng Li, Fernando De la Torre, Yaser Sheikh
Paper

Pixel Codec Avatars

In this work, we present the Pixel Codec Avatars (PiCA): a deep generative model of 3D human faces that achieves state of the art reconstruction performance while being computationally efficient and adaptive to the rendering conditions during execution.
Paper
June 19, 2021 Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih
Paper

SimPoE: Simulated Character Control for 3D Human Pose Estimation

Accurate estimation of 3D human motion from monocular video requires modeling both kinematics (body motion without physical forces) and dynamics (motion with physical forces).
Paper
June 7, 2021 Lior Arbel, Zamir Ben-Hur, David Lou Alon, Boaz Rafaely
Paper

Applied Methods for Sparse Sampling of Head-related Transfer Functions

This paper describes the application of two methods for ear-aligned HRTF interpolation by sparse sampling: Orthogonal Matching Pursuit and Principal Component Analysis.
Paper
June 6, 2021 Tom Shlomo, Boaz Rafaely
Paper

Blind Amplitude Estimation of Early Room Reflections Using Alternating Least Squares

This work presents a preliminary attempt to blindly estimate reflection amplitudes. An iterative estimator is suggested, based on maximum likelihood and alternating least squares.
Paper
June 6, 2021 Panagiotis Tzirakis, Anurag Kumar, Jacob Donley
Paper

Multi-Channel Speech Enhancement Using Graph Neural Networks

In this paper, we introduce a different research direction by viewing each audio channel as a node lying in a non-Euclidean space and, specifically, a graph.
Paper
June 6, 2021 Yaxuan Zhou, Hao Jiang, Vamsi Krishna Ithapu
Paper

On the Predictability of HRTFs from Ear Shapes Using Deep Networks

Using 3D ear shapes as inputs, we explore the bounds of HRTF predictability using deep neural networks. To that end, we propose and evaluate two models, and identify the lowest achievable spectral distance error when predicting the true HRTF magnitude spectra.
Paper