Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR
<p>Virtual reality offers powerful opportunities to explore complex scientific datasets through immersive interaction, yet most VR visualization systems rely exclusively on visual channels, leaving human auditory perception largely untapped. This chapter presents a framework for multimodal scientifi...
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Capítulo de libro |
| Lenguaje: | inglés |
| Publicado: |
IntechOpen
2026
|
| Materias: | |
| Acceso en línea: | https://www.intechopen.com/chapters/1246021 |
| Etiquetas: |
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| Sumario: | <p>Virtual reality offers powerful opportunities to explore complex scientific datasets through immersive interaction, yet most VR visualization systems rely exclusively on visual channels, leaving human auditory perception largely untapped. This chapter presents a framework for multimodal scientific visualization that combines high-fidelity 3D rendering with spatial audio sonification. We establish psychoacoustic foundations grounded in Weber–Fechner logarithmic scaling, Stevens’ power-law loudness compression, and Bregman’s auditory stream segregation, then introduce a structured parameter mapping design framework with four formal criteria: polarity consistency, perceptual orthogonality, dynamic range utilization, and listener comfort. We detail HRTF-based binaural rendering using the MIT KEMAR database with barycentric interpolation, and demonstrate the approach through a computational fluid dynamics (CFD) case study employing psychoacoustically-motivated mappings: velocity magnitude to pitch, vorticity to spectral roughness, pressure deviation to sub-bass content, and turbulent kinetic energy to amplitude modulation. A within-subjects evaluation with 12 participants using a counterbalanced design across visual-only and multimodal conditions yielded a 31% improvement in feature identification tasks (paired t-tests, p < 0.002, Cohen’s d > 1.2), with median spatial localization accuracy of 8.4° azimuth and 11.3° elevation using audio cues alone. Pilot studies with 10 participants informed iterative mapping design, and a 12-student educational deployment in a graduate CFD course confirmed practical utility. Post-session Likert surveys reported high subjective scores for spatial awareness (4.5/5) and feature identification (4.3/5). The framework generalizes to medical imaging, atmospheric science, molecular dynamics, and geophysics, providing conceptual foundations and practical implementation guidance for multimodal VR visualization systems.</p> |
|---|---|
| ISBN: | 978-1-80631-325-9 |
| DOI: | 10.5772/intechopen.1015241 |