mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR
Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN),...
Gardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Artigo |
| Publicado: |
2026
|
| Materias: | |
| Acceso en liña: | https://arxiv.org/abs/2602.22431 http://arxiv.org/abs/2602.22431 |
| Etiquetas: |
Sen Etiquetas, Sexa o primeiro en etiquetar este rexistro!
|
| _version_ | 1879179914643505152 |
|---|---|
| author | Karani, Jash Chittem, Adithya Roy, Deepan Joshi, Sandeep |
| author_browse | Chittem, Adithya Joshi, Sandeep Karani, Jash Roy, Deepan |
| author_facet | Karani, Jash Chittem, Adithya Roy, Deepan Joshi, Sandeep |
| author_sort | Karani, Jash |
| building | arXiv |
| collection | Acceso abierto |
| description | Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/. |
| first_indexed | 2026-10-16T01:40:11Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-2602.22431</identifier><datestamp>2026-08-18</datestamp>
<dc:title>mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR</dc:title>
<dc:creator>Karani, Jash</dc:creator>
<dc:creator>Chittem, Adithya</dc:creator>
<dc:creator>Roy, Deepan</dc:creator>
<dc:creator>Joshi, Sandeep</dc:creator>
<dc:subject>Sound</dc:subject>
<dc:subject>Machine Learning</dc:subject>
<dc:description>Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/. </dc:description>
<dc:description>Accepted at Interspeech 2026</dc:description>
<dc:date>2026-02-25</dc:date>
<dc:date>2026-08-15</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/2602.22431</dc:identifier>
<dc:identifier>http://arxiv.org/abs/2602.22431</dc:identifier>
</oai_dc:dc> |
| id | arxiv-2602.22431 |
| institution | arXiv |
| last_indexed | 2026-10-16T01:40:11Z |
| publishDate | 2026 |
| publishDateRange | 2026 |
| publishDateSort | 2026 |
| record_format | oai_dc |
| spelling | arxiv-2602.224312026-08-18 mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR Karani, Jash Chittem, Adithya Roy, Deepan Joshi, Sandeep Sound Machine Learning Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/. Accepted at Interspeech 2026 2026-02-25 2026-08-15 text https://arxiv.org/abs/2602.22431 http://arxiv.org/abs/2602.22431 |
| spellingShingle | Sound Machine Learning Karani, Jash Chittem, Adithya Roy, Deepan Joshi, Sandeep mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR |
| title | mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR |
| title_full | mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR |
| title_fullStr | mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR |
| title_full_unstemmed | mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR |
| title_short | mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR |
| title_sort | mmwave radar aware dual conditioned gan for speech reconstruction of signals with low snr |
| topic | Sound Machine Learning |
| url | https://arxiv.org/abs/2602.22431 http://arxiv.org/abs/2602.22431 |
| work_keys_str_mv | AT karanijash mmwaveradarawaredualconditionedganforspeechreconstructionofsignalswithlowsnr AT chittemadithya mmwaveradarawaredualconditionedganforspeechreconstructionofsignalswithlowsnr AT roydeepan mmwaveradarawaredualconditionedganforspeechreconstructionofsignalswithlowsnr AT joshisandeep mmwaveradarawaredualconditionedganforspeechreconstructionofsignalswithlowsnr |