DDTSE: Discriminative Diffusion Model
for Target Speech Extraction


Abstract

Diffusion models have gained attention in speech enhancement tasks, providing an alternative to conventional discriminative methods. However, research on target speech extraction under multi-speaker noisy conditions remains relatively unexplored. Moreover, the superior quality of diffusion methods typically comes at the cost of slower inference speed. In this paper, we introduce the Discriminative Diffusion model for Target Speech Extraction (DDTSE). We apply the same forward process as diffusion models and utilize the reconstruction loss similar to discriminative methods. Furthermore, we devise a two-stage training strategy to emulate the inference process during model training. DDTSE not only works as a standalone system, but also can further improve the performance of discriminative models without additional retraining. Experimental results demonstrate that DDTSE not only achieves higher perceptual quality but also accelerates the inference process by 3 times compared to the conventional diffusion model.

🔥 Codes are available at https://github.com/vivian556123/slt2024-ddtse/tree/code
📝 Paper is available at https://arxiv.org/abs/2309.13874

Multi-speaker noisy scenario

Mixture: The input multi-speaker mixture with background noise.
Ground Truth: The clean speech of the target speaker.
DDTSE-only: Extraction results of our proposed DDTSE-only method.
DPCCN: Extraction results of discriminative DPCCN method.
DPCCN+DDTSE: Extraction results of DPCCN+DDTSE method.
We also provide the corresponding mel spectrogram of each audio file for better visualization.
WAV_ID Mixture Ground Truth DDTSE-only DPCCN DPCCN+DDTSE

Multi-speaker clean scenario

Mixture: The input multi-speaker mixture without background noise.
Ground Truth: The clean speech of the target speaker.
DDTSE-only: Extraction results of our proposed DDTSE-only method.
DPCCN: Extraction results of discriminative DPCCN method.
DPCCN+DDTSE: Extraction results of diffusion-based DPCCN+DDTSE method.
We also provide the corresponding mel spectrogram of each audio file for better visualization.
WAV_ID Mixture Ground Truth DDTSE-only DPCCN DPCCN+DDTSE

Single-speaker noisy scenario

Mixture: The input noisy single-speaker speech.
Ground Truth: The clean speech of the target speaker.
DDTSE-only: Enhancement results of our proposed DDTSE-only method.
DCCRN: Enhancement results of discriminative DCCRN method.
SGMSE+: Enhancement results of diffusion based SGMSE+ method.
We also provide the corresponding mel spectrogram of each audio file for better visualization.
WAV_ID Mixture Ground Truth DDTSE-only DCCRN SGMSE+