DualSpecSE

A Dual-Path Speech Enhancement Network Integrating Mel and Complex Spectrograms

Anonymous Author(s)
Under Review

Abstract

In this paper, we propose DualSpecSE, a speech enhancement framework that jointly models Mel-spectrogram and complex spectrogram in a dual-path architecture for improved ASR performance and higher-quality speech reconstruction. The Mel branch learns coarse-grained acoustic representations and produces enhanced Mel-spectrograms for direct ASR usage, while the complex branch refines fine-grained spectral details for high-fidelity waveform reconstruction. Built upon the cross-band and narrow-band blocks from CleanMel, DualSpecSE introduces an interaction module and a fusion module to enable effective information exchange between the two branches. The model simultaneously outputs enhanced Mel and complex spectrogram without requiring a pretrained vocoder. Experimental results demonstrate consistent improvements in speech fidelity, perceptual quality, and ASR performance.

Note: Our code will be open-sourced upon paper acceptance.

Audio Demonstrations

We present audio samples comparing DualSpecSE with baseline methods (SpatialNet and PGUSE) on three DNS Challenge test sets.

DNS Challenge: No-Reverb Test Set

Sample 1

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

sample 2

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

sample 3

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

DNS Challenge: With-Reverb Test Set

Sample 1

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

Sample 2

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

Sample 3

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

DNS Challenge: Real-Recording Test Set

Sample 1

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

Sample 2

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram

Sample 3

Noisy Speech SpatialNet PGUSE DualSpecSE (Ours)
Noisy Spectrogram
SpatialNet Spectrogram
PGUSE Spectrogram
DualSpecSE Spectrogram