Explore the latest research from Facebook

All Publications

January 1, 2022 Mike Salvato, Sophia R. Williams, Cara M. Nunez, Xin Zhu, Ali Israr, Frances Lau, Keith Klumb, Freddy Abnousi, Allison M. Okamura, Heather Culbertson
Paper

Data-driven sparse skin stimulation can convey social touch information to humans

Here we explore whether a sparse representation of human touch is sufficient to convey social touch signals. To test this we collected a dataset of social touch interactions using a soft wearable pressure sensor array, developed an algorithm to map recorded data to an array of actuators, then applied our algorithm to create signals that drive an array of normal indentation actuators placed on the arm.
Paper
December 14, 2021 Donglai Xiang, Fabián Prada, Timur Bagautdinov, Weipeng Xu, Yuan Dong, He Wen, Jessica Hodgins, Chenglei Wu
Paper

Modeling Clothing as a Separate Layer for an Animatable Human Avatar

We then train a new two-layer codec avatar with separate modeling of the upper clothing and the inner body layer. To learn the interaction between the body dynamics and clothing states, we use a temporal convolution network to predict the clothing latent code based on a sequence of input skeletal poses. We show photorealistic animation output for three different actors, and demonstrate the advantage of our clothed-body avatars over the single-layer avatars used in previous work.
Paper
December 5, 2021 Xiaofan Zhang, Dawei Wang, Pierce Chuang, Shugao Ma, Deming Chen, Yuecheng Li
Paper

F-CAD: A Framework to Explore Hardware Accelerators for Codec Avatar Decoding

To the best of our knowledge, F-CAD is the first automation tool that supports the whole design flow of hardware acceleration of codec avatar decoders, allowing joint optimization on decoder designs in popular machine learning frameworks and corresponding customized accelerator design with cycle-accurate evaluation.
Paper
November 18, 2021 Bjoern Haefner, Simon Green, Alan Oursland, Daniel Andersen, Michael Goesele, Daniel Cremers, Richard Newcombe, Thomas Whelan
Paper

Recovering Real-World Reflectance Properties and Shading From HDR Imagery

We propose a method to estimate the bidirectional reflectance distribution function (BRDF) and shading of complete scenes under static illumination given the 3D scene geometry and a corresponding high dynamic range (HDR) video.
Paper
November 5, 2021 Lior Yariv, Jiatao Gu, Yoni Kasten, Yaron Lipman
Paper

Volume Rendering of Neural Implicit Surfaces

The goal of this paper is to improve geometry representation and reconstruction in neural volume rendering. We achieve that by modeling the volume density as a function of the geometry.
Paper
November 2, 2021 Cole Hawkins, Haichuan Yang, Meng Li, Liangzhen Lai, Vikas Chandra
Paper

Low-Rank+Sparse Tensor Compression For Neural Networks

We propose to combine low-rank tensor decomposition with sparse pruning in order to take advantage of both coarse and fine structure for compression. We compress weights in SOTA architectures (MobileNetv3, EfficientNet, Vision Transformer) and compare this approach to sparse pruning and tensor decomposition alone.
Paper
October 28, 2021 Sunny Ye, Ni Huo, Wyatt Tenhaeff
Paper

Halogenated high index polymer coating

In this work, a series of halogenated polymer thin films with high refractive indexes (n =1.7 - 2.0) and outstanding optical transparency in the visible and infrared range, were prepared via a unique vapor phase process – initiated chemical vapor deposition (iCVD).
Paper
October 26, 2021 Edward J. Smith, David Meger, Luis Pineda, Roberto Calandra, Jitendra Malik, Adriana Romero-Soriano, Michal Drozdzal
Paper

Active 3D Shape Reconstruction from Vision and Touch

In this paper, we focus on this problem and introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile signals; and 3) a set of data-driven solutions with either tactile or visuotactile priors to guide the shape exploration.
Paper
October 25, 2021 James Hegarty, Omar Eldash, Amr Suleiman, Armin Alaghi
Paper

HWTool: Fully Automatic Mapping of an Extensible C++ Image Processing Language to Hardware

In this paper, we present HWTool, a novel system for automatically mapping image processing and computer vision algorithms to hardware. Our system maps between two domains: HWImg, an extensible C++ image processing library containing common image processing and parallel computing operators, and Rigel2, a library of optimized hardware implementations of HWImg’s operators and backend Verilog compiler.
Paper
October 23, 2021 Ahsan Abdullah, Jan Kolkmeier, Vivian Lo, Michael Neff
Paper

Videoconference and Embodied VR: Communication Patterns Across Task and Medium

This paper describes a 210 participant study focused on behavioral measures that compares multiparty interaction in videoconference and embodied VR across a range of task types: a factual intellective task, a subjective judgment task and two negotiation tasks, one with visual grounding.
Paper