Hasti: Haptic and Audio Synthesis for Texture Interactions

The IEEE World Haptics Conference (IEEE WHC)


Multi-sensory stimuli can greatly enhance immersion in interactive virtual environments. Advances in graphics algorithms and technologies like VR displays have pushed the appearance of interactive virtual worlds to unprecedented fidelity, but rendering sound and, especially, touch feedback of comparable quality remains a challenge. We describe a method for real-time synthesis of vibrotactile haptic and audio stimuli for interactions with textured surfaces in 3-D virtual environments. Standard descriptions of object geometry and material properties, including displacement and roughness texture maps typically used for physically-based visual rendering, are employed to generate realistic sound and touch feedback consistent with appearance. Our method reconstructs meso-and microscopic surface features on the fly along a contact trajectory, and runs a micro-contact dynamics simulation whose outputs drive vibrotactile haptic actuators and modal sound synthesis. An exploratory, absolute identification user study was conducted as an initial evaluation of our synthesis methods.

Related Publications

All Publications

The Journal of the Acoustical Society of America - February 4, 2021

Perceptual implications of different Ambisonics-based methods for binaural reverberation

Isaac Engel, Craig Henry, Sebastià V. Amengual Garí, Philip W. Robinson, Lorenzo Picinali

ICASSP - May 13, 2021

Room Impulse Response Interpolation From A Sparse Set Of Measurements Using A Modal Architecture

Orchisama Das, Paul Calamia, Sebastià V. Amengual Garí

IEEE Transactions on Image Processing - August 6, 2021

Subjective and Objective Quality Assessment of 2D and 3D Foveated Video Compression in Virtual Reality

Yize Jin, Meixu Chen, Todd Goodall, Anjul Patney, Alan C. Bovik

Nature Protocols Journal (Nat. Protoc) - February 24, 2021

Making bioinspired 3D-printed autonomic perspiring hydrogel actuators

Anand Kumar Mishra, Wenyang Pan, Emmanuel P. Giannelis, Robert F. Shepherd, Thomas J. Wallin

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy