Spatial Covariance Matrix Estimation for Reverberant Speech with Application to Speech Enhancement



A wide range of applications in speech and audio signal processing incorporate a model of room reverberation based on the spatial covariance matrix (SCM). Typically, a diffuse sound field model is used, but although the diffuse model simplifies formulations, it may lead to limited accuracy in realistic sound fields, resulting in potential degradation in performance. While some extensions to the diffuse field SCM recently have been presented, accurate modeling for real sound fields remains an open problem. In this paper, a method for estimating the SCM of reverberant speech is proposed, based on the selection of time-frequency bins dominated by reverberation. The method is data-based and estimates the SCM for a specific acoustic scene. It is therefore applicable to realistic reverberant fields. An application of the proposed method to optimal beamforming for speech enhancement is presented, using the plane wave density function in the spherical harmonics (SH) domain. It is shown that the use of the proposed SCM outperforms the commonly used diffuse field SCM, suggesting the method is more successful in capturing the statistics of the late part of the reverberation.

Related Publications

All Publications

ICASSP - June 6, 2021

Multi-Channel Speech Enhancement Using Graph Neural Networks

Panagiotis Tzirakis, Anurag Kumar, Jacob Donley

SIGGRAPH - August 17, 2020

Consistent Video Depth Estimation

Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, Johannes Kopf

ICSE - November 23, 2020

Predictive Test Selection

Mateusz Machalica, Alex Samylkin, Meredith Porth, Satish Chandra

3DV - November 25, 2020

MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video

Donglai Xiang, Fabian Prada, Chenglei Wu, Jessica Hodgins

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy