Semi-supervised Synthesis of High-Resolution Editable Textures for 3D Humans

Conference on Computer Vision and Pattern Recognition (CVPR)


We introduce a novel approach to generate diverse high fidelity texture maps for 3D human meshes in a semi-supervised setup. Given a segmentation mask defining the layout of the semantic regions in the texture map, our network generates high-resolution textures with a variety of styles, that are then used for rendering purposes. To accomplish this task, we propose a Region-adaptive Adversarial Variational AutoEncoder (ReAVAE) that learns the probability distribution of the style of each region individually so that the style of the generated texture can be controlled by sampling from the region-specific distributions. In addition, we introduce a data generation technique to augment our training set with data lifted from single-view RGB in- puts. Our training strategy allows the mixing of reference image styles with arbitrary styles for different regions, a property which can be valuable for virtual try-on AR/VR applications. Experimental results show that our method synthesizes better texture maps compared to prior work while enabling independent layout and style controllability.



Related Publications

All Publications

Presentation on Late-Breaking Work at CHI - May 9, 2021

A Multichannel Pneumatic Analog Control System for Haptic Displays

Benjamin Stephens-Fripp, Ali Israr, Carine Rognon

ACM CHI Virtual Conference on Human Factors in Computing Systems (CHI) - May 8, 2021

Armstrong: An Empirical Examination of Pointing at Non-Dominant Arm-Anchored UIs in Virtual Reality

Zhen Li, Joannes Chan, Joshua Walton, Hrvoje Benko, Daniel Wigdor, Michael Glueck

CHI - May 8, 2021

Understanding, Detecting and Mitigating the Effects of Coactivations in Ten-Finger Mid-Air Typing in Virtual Reality

Conor R. Foy, John J. Dudley, Aakar Gupta, Hrvoje Benko, Per Ola Kristensson

CHI - May 8, 2021

StickyPie: A Gaze-Based, Scale-Invariant Marking Menu Optimized for AR/VR

Sunggeun Ahn, Stephanie Santosa, Mark Parent, Daniel Wigdor, Tovi Grossman, Marcello Giordano

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy