June 18, 2018
Audio to Body Dynamics
Computer Vision and Pattern Recognition (CVPR)
We present a method that gets as input an audio of violin or piano playing, and outputs a video of skeleton predictions which are further used to animate an avatar. The key idea is to create an animation of an avatar that moves their hands similarly to how a pianist or violinist would do, just from audio.
By: Eli Shlizerman, Lucio Dery, Hayden Schoen, Ira Kemelmacher Shlizerman
Computational Photography & Intelligent Cameras
Facebook AI Research