In this paper, we propose DualSpecSE, a speech enhancement framework that jointly models Mel-spectrogram and complex spectrogram in a dual-path architecture for improved ASR performance and higher-quality speech reconstruction. The Mel branch learns coarse-grained acoustic representations and produces enhanced Mel-spectrograms for direct ASR usage, while the complex branch refines fine-grained spectral details for high-fidelity waveform reconstruction. Built upon the cross-band and narrow-band blocks from CleanMel, DualSpecSE introduces an interaction module and a fusion module to enable effective information exchange between the two branches. The model simultaneously outputs enhanced Mel and complex spectrogram without requiring a pretrained vocoder. Experimental results demonstrate consistent improvements in speech fidelity, perceptual quality, and ASR performance.
Note: Our code will be open-sourced upon paper acceptance.
We present audio samples comparing DualSpecSE with baseline methods (SpatialNet and PGUSE) on three DNS Challenge test sets.
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|
| Noisy Speech | SpatialNet | PGUSE | DualSpecSE (Ours) |
|---|---|---|---|
|
|
|
|