7 Results
June 18, 2018
Audio to Body Dynamics
Computer Vision and Pattern Recognition (CVPR)
We present a method that gets as input an audio of violin or piano playing, and outputs a video of skeleton predictions which are further used to animate an avatar. The key idea is to create an animation of an avatar that moves their hands similarly to how a pianist or violinist would do, just from audio.
By: Eli Shlizerman, Lucio Dery, Hayden Schoen, Ira Kemelmacher Shlizerman
AR/VR 
Computational Photography & Intelligent Cameras 
Computer Vision 
Facebook AI Research November 27, 2017
Casual 3D Photography
SIGGRAPH ASIA
We present an algorithm that enables casual 3D photography. Given a set of input photos captured with a hand-held cell phone or DSLR camera, our algorithm reconstructs a 3D photo, a central panoramic, textured, normal mapped, multi-layered geometric mesh representation. Our geometric representation also allows interacting with the scene using 3D geometry-aware effects, such as adding new objects to the scene and artistic lighting effects.
By: Peter Hedman, Suhib Alsisan, Richard Szeliski, Johannes KopfNovember 27, 2017
Bringing Portraits to Life
SIGGRAPH ASIA
We present a technique to automatically animate a still portrait, making it possible for the subject in the photo to come to life and express various emotions. We use a driving video (of a different subject) and develop means to transfer the expressiveness of the subject in the driving video to the target portrait. In contrast to previous work that requires an input video of the target face to reenact a facial performance, our technique uses only a single target image.
By: Hadar Averbuch-Elor, Daniel Cohen-Or, Johannes Kopf, Michael CohenOctober 22, 2017
Mask R-CNN
International Conference on Computer Vision (ICCV)
We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition.
By: Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross GirshickJuly 30, 2017
Low-Cost 360 Stereo Photography and Video Capture
SIGGRAPH 2017
In this work, we describe a method that takes images from two 360◦ spherical cameras and synthesizes an omni-directional stereo panorama with stereo in all directions. Our proposed method has a lower equipment cost than camera-ring alternatives, can be assembled with currently available off-the-shelf equipment, and is relatively small and light-weight compared to the alternatives.
By: Kevin Matzen, Michael Cohen, Bryce Evans, Johannes Kopf, Richard SzeliskiDecember 5, 2016
Temporally Coherent Completion of Dynamic Video
SIGGRAPH ASIA
We present an automatic video completion algorithm that synthesizes missing regions in videos in a temporally coherent fashion.
By: Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, Johannes KopfDecember 5, 2016
360◦ Video Stabilization
SIGGRAPH ASIA
We present a hybrid 3D-2D algorithm for stabilizing 360◦ video using a deformable rotation motion model.
By: Johannes Kopf