Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR
<p>Virtual reality offers powerful opportunities to explore complex scientific datasets through immersive interaction, yet most VR visualization systems rely exclusively on visual channels, leaving human auditory perception largely untapped. This chapter presents a framework for multimodal scientifi...
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Capítulo de libro |
| Lenguaje: | inglés |
| Publicado: |
IntechOpen
2026
|
| Materias: | |
| Acceso en línea: | https://www.intechopen.com/chapters/1246021 |
| Etiquetas: |
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1879164894746509312 |
|---|---|
| author | Boulanger, Pierre |
| author_browse | Boulanger, Pierre |
| author_facet | Boulanger, Pierre |
| author_sort | Boulanger, Pierre |
| building | IntechOpen |
| collection | Acceso abierto |
| description | <p>Virtual reality offers powerful opportunities to explore complex scientific datasets through immersive interaction, yet most VR visualization systems rely exclusively on visual channels, leaving human auditory perception largely untapped. This chapter presents a framework for multimodal scientific visualization that combines high-fidelity 3D rendering with spatial audio sonification. We establish psychoacoustic foundations grounded in Weber–Fechner logarithmic scaling, Stevens’ power-law loudness compression, and Bregman’s auditory stream segregation, then introduce a structured parameter mapping design framework with four formal criteria: polarity consistency, perceptual orthogonality, dynamic range utilization, and listener comfort. We detail HRTF-based binaural rendering using the MIT KEMAR database with barycentric interpolation, and demonstrate the approach through a computational fluid dynamics (CFD) case study employing psychoacoustically-motivated mappings: velocity magnitude to pitch, vorticity to spectral roughness, pressure deviation to sub-bass content, and turbulent kinetic energy to amplitude modulation. A within-subjects evaluation with 12 participants using a counterbalanced design across visual-only and multimodal conditions yielded a 31% improvement in feature identification tasks (paired t-tests, p < 0.002, Cohen’s d > 1.2), with median spatial localization accuracy of 8.4° azimuth and 11.3° elevation using audio cues alone. Pilot studies with 10 participants informed iterative mapping design, and a 12-student educational deployment in a graduate CFD course confirmed practical utility. Post-session Likert surveys reported high subjective scores for spatial awareness (4.5/5) and feature identification (4.3/5). The framework generalizes to medical imaging, atmospheric science, molecular dynamics, and geophysics, providing conceptual foundations and practical implementation guidance for multimodal VR visualization systems.</p> |
| doi_str_mv | 10.5772/intechopen.1015241 |
| first_indexed | 2026-10-15T21:41:35Z |
| format | Book Chapter |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>intech-1246021</identifier><datestamp>2026-08-19</datestamp>
<dc:title>Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR</dc:title>
<dc:creator>Pierre Boulanger</dc:creator>
<dc:subject>Physical Sciences, Engineering and Technology</dc:subject>
<dc:description><p>Virtual reality offers powerful opportunities to explore complex scientific datasets through immersive interaction, yet most VR visualization systems rely exclusively on visual channels, leaving human auditory perception largely untapped. This chapter presents a framework for multimodal scientific visualization that combines high-fidelity 3D rendering with spatial audio sonification. We establish psychoacoustic foundations grounded in Weber&ndash;Fechner logarithmic scaling, Stevens&rsquo; power-law loudness compression, and Bregman&rsquo;s auditory stream segregation, then introduce a structured parameter mapping design framework with four formal criteria: polarity consistency, perceptual orthogonality, dynamic range utilization, and listener comfort. We detail HRTF-based binaural rendering using the MIT KEMAR database with barycentric interpolation, and demonstrate the approach through a computational fluid dynamics (CFD) case study employing psychoacoustically-motivated mappings: velocity magnitude to pitch, vorticity to spectral roughness, pressure deviation to sub-bass content, and turbulent kinetic energy to amplitude modulation. A within-subjects evaluation with 12 participants using a counterbalanced design across visual-only and multimodal conditions yielded a 31% improvement in feature identification tasks (paired t-tests, p &lt; 0.002, Cohen&rsquo;s d &gt; 1.2), with median spatial localization accuracy of 8.4&deg; azimuth and 11.3&deg; elevation using audio cues alone. Pilot studies with 10 participants informed iterative mapping design, and a 12-student educational deployment in a graduate CFD course confirmed practical utility. Post-session Likert surveys reported high subjective scores for spatial awareness (4.5/5) and feature identification (4.3/5). The framework generalizes to medical imaging, atmospheric science, molecular dynamics, and geophysics, providing conceptual foundations and practical implementation guidance for multimodal VR visualization systems.</p></dc:description>
<dc:publisher>IntechOpen</dc:publisher>
<dc:date>2026-08-19</dc:date>
<dc:type>Chapter, Part Of Book</dc:type>
<dc:identifier>https://www.intechopen.com/chapters/1246021</dc:identifier>
<dc:identifier>doi:10.5772/intechopen.1015241</dc:identifier>
<dc:language>en</dc:language>
<dc:relation>ISBN:978-1-80631-325-9</dc:relation>
<dc:rights>https://creativecommons.org/licenses/by/3.0/</dc:rights>
<dc:source>https://www.intechopen.com/books/1005054 ; Virtual Reality - Theoretical Foundations to Practical Applications</dc:source>
</oai_dc:dc> |
| id | intech-1246021 |
| institution | IntechOpen |
| isbn | 978-1-80631-325-9 |
| language | English |
| last_indexed | 2026-10-15T21:41:35Z |
| publishDate | 2026 |
| publishDateRange | 2026 |
| publishDateSort | 2026 |
| publisher | IntechOpen |
| publisherStr | IntechOpen |
| record_format | oai_dc |
| spelling | intech-12460212026-08-19 Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR Pierre Boulanger Physical Sciences, Engineering and Technology <p>Virtual reality offers powerful opportunities to explore complex scientific datasets through immersive interaction, yet most VR visualization systems rely exclusively on visual channels, leaving human auditory perception largely untapped. This chapter presents a framework for multimodal scientific visualization that combines high-fidelity 3D rendering with spatial audio sonification. We establish psychoacoustic foundations grounded in Weber–Fechner logarithmic scaling, Stevens’ power-law loudness compression, and Bregman’s auditory stream segregation, then introduce a structured parameter mapping design framework with four formal criteria: polarity consistency, perceptual orthogonality, dynamic range utilization, and listener comfort. We detail HRTF-based binaural rendering using the MIT KEMAR database with barycentric interpolation, and demonstrate the approach through a computational fluid dynamics (CFD) case study employing psychoacoustically-motivated mappings: velocity magnitude to pitch, vorticity to spectral roughness, pressure deviation to sub-bass content, and turbulent kinetic energy to amplitude modulation. A within-subjects evaluation with 12 participants using a counterbalanced design across visual-only and multimodal conditions yielded a 31% improvement in feature identification tasks (paired t-tests, p < 0.002, Cohen’s d > 1.2), with median spatial localization accuracy of 8.4° azimuth and 11.3° elevation using audio cues alone. Pilot studies with 10 participants informed iterative mapping design, and a 12-student educational deployment in a graduate CFD course confirmed practical utility. Post-session Likert surveys reported high subjective scores for spatial awareness (4.5/5) and feature identification (4.3/5). The framework generalizes to medical imaging, atmospheric science, molecular dynamics, and geophysics, providing conceptual foundations and practical implementation guidance for multimodal VR visualization systems.</p> IntechOpen 2026-08-19 Chapter, Part Of Book https://www.intechopen.com/chapters/1246021 doi:10.5772/intechopen.1015241 en ISBN:978-1-80631-325-9 https://creativecommons.org/licenses/by/3.0/ https://www.intechopen.com/books/1005054 ; Virtual Reality - Theoretical Foundations to Practical Applications |
| spellingShingle | Physical Sciences, Engineering and Technology Boulanger, Pierre Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR |
| title | Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR |
| title_full | Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR |
| title_fullStr | Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR |
| title_full_unstemmed | Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR |
| title_short | Hearing the Invisible: Spatial Sonification for Scientific Discovery in VR |
| title_sort | hearing the invisible spatial sonification for scientific discovery in vr |
| topic | Physical Sciences, Engineering and Technology |
| url | https://www.intechopen.com/chapters/1246021 |
| work_keys_str_mv | AT boulangerpierre hearingtheinvisiblespatialsonificationforscientificdiscoveryinvr |