From Lab to Code #2: Exploring Spatial Audio with Python and SciPy

Most of us can recognize where a sound is coming from without consciously thinking about it.

If someone speaks to our left, we naturally turn in that direction. If a car passes behind us, we can often estimate its movement without seeing it.

But how does our auditory system determine where a sound is located?

Interestingly, much of this ability depends on small differences between the signals arriving at our left and right ears.

A sound may reach one ear slightly earlier, or with a different intensity, than the other.

These differences can be analyzed using signal processing techniques such as cross-correlation, convolution, and impulse response analysis.

For the second project in my From Lab to Code series, I wanted to explore these concepts through an interactive Python application.

The result is Spatial Audio Playground: a small experimental application that lets users position virtual sound sources, compare mono and binaural audio, and visualize some of the signals responsible for directional hearing.

GitHub repository: https://github.com/Ertugrulmutlu/from-lab-to-code

Project 02 — Spatial Audio Playground: https://github.com/Ertugrulmutlu/from-lab-to-code/tree/main/02-spatial-audio

What Are We Building?

The application explores four main ideas:

1. Cross-correlation to measure the relative delay between signals.
2. Interaural Time Difference (ITD) and Interaural Level Difference (ILD) to understand directional hearing.
3. Binaural synthesis to generate separate audio signals for the left and right ears.
4. Interactive spatial audio using Python, SciPy, Streamlit, and a small JavaScript-based soundstage.

Users can move two virtual sound sources around a listener, adjust their positions and gains, and compare the resulting audio through headphones.

The project uses procedurally generated audio and educational impulse-response models by default.

No pretrained models, external audio datasets, or API calls are required.

---

1. Cross-Correlation: Detecting Delays Between Signals

Before exploring spatial audio, we need a way to measure the relative timing of two signals.

Consider a simple example.

We have two discrete signals containing impulses:

• The first signal has an impulse at sample index 5.
• The second signal has impulses at indices 9 and 14, with amplitudes of 1.0 and 0.5.

The corresponding differences in sample positions are +4 and +9.

How can we detect these shifts automatically?

Understanding Cross-Correlation

Cross-correlation measures similarity between two signals as one is shifted relative to the other.

For discrete signals, one convention is:

R_yx[k] = Σ y[n] · conjugate(x[n-k])

The resulting function tells us how strongly the signals align at different sample lags.

For simple impulse signals, alignment produces distinct correlation peaks.

Implementing Cross-Correlation with SciPy

First, generate the signals:

import numpy as np
from scipy import signal

num_samples = 20

reference = signal.unit_impulse(
    num_samples, idx=5
)

shifted = (
    signal.unit_impulse(num_samples, idx=9)
    + 0.5 * signal.unit_impulse(
        num_samples, idx=14
    )
)

Next, calculate the correlation and its corresponding lags:

correlation = signal.correlate(
    shifted,
    reference,
    mode="full"
)

lags = signal.correlation_lags(
    len(shifted),
    len(reference),
    mode="full"
)

The correlation contains two nonzero peaks:

Lag	Correlation amplitude
+4	1.0
+9	0.5

The first peak is larger because the corresponding impulse has a higher amplitude.

An important detail is the order of the arguments.

In this example, correlate(shifted, reference) produces positive lags for impulses occurring later in the shifted signal.

Reversing the arguments changes the lag sign convention.

From Sample Lag to Time Delay

A sample shift can also be expressed as a time difference.

If the sampling frequency is fs, then:

delay = lag / fs

For example, at a sampling frequency of 44,100 Hz, a delay of four samples corresponds to approximately 0.091 milliseconds.

This becomes important when comparing the arrival times of audio signals at our two ears.

Key takeaway: Cross-correlation gives us a practical way to estimate relative timing differences, provided the signals contain sufficiently identifiable patterns.

---

2. How Do We Hear Direction? ITD and ILD

Our ears receive two slightly different versions of the same sound.

The differences between them provide important information about the location of a sound source.

Two particularly useful cues are:

Interaural Time Difference (ITD)

Imagine a sound source located to your left.

The sound generally reaches your left ear before your right ear because the path to the right ear is longer.

This arrival-time difference is called the Interaural Time Difference.

In the application, I define:

ITD = t_right - t_left

Therefore:

• Positive ITD: the sound arrives at the left ear first.
• Negative ITD: the sound arrives at the right ear first.
• ITD near zero: both ears have approximately equal arrival times.

To estimate ITD, the project applies a low-pass filter to the left and right impulse responses before calculating their cross-correlation.

sos = signal.butter(
    4,
    1500,
    btype="lowpass",
    fs=44100,
    output="sos"
)

low_left = signal.sosfilt(
    sos, hrir[0]
)

low_right = signal.sosfilt(
    sos, hrir[1]
)

lags = signal.correlation_lags(
    len(low_right),
    len(low_left),
    mode="full"
)

correlation = signal.correlate(
    low_right,
    low_left,
    mode="full"
)

The implementation restricts the lag search to a physically relevant window of ±1 millisecond.

The lag corresponding to the highest correlation value within this window provides an estimate of the timing difference.

Interaural Level Difference (ILD)

Time is not the only difference between our ears.

The head also affects how much sound energy reaches each ear.

For some directions, the head partially blocks the sound travelling toward the opposite ear.

This effect is especially important at higher frequencies.

The resulting difference in sound level is called the Interaural Level Difference.

In the project, a broadband ILD estimate is calculated using the energy of the two impulse responses:

ILD = 10 · log10(E_right / E_left)

where:

E = Σ h[n]²

In Python:

energy_left = np.sum(hrir[0] ** 2)
energy_right = np.sum(hrir[1] ** 2)

ild_db = 10 * np.log10(
    energy_right / energy_left
)

A negative ILD means the right channel contains less impulse-response energy than the left channel.

This is a broadband estimate. In real acoustic systems, ILD depends strongly on frequency, so frequency-band analysis can provide more detailed information.

Why Both Cues Matter

ITD and ILD complement each other.

ITD is particularly useful for understanding low-frequency directional cues, while ILD becomes especially relevant at higher frequencies.

However, these two values alone cannot completely explain spatial hearing.

They do not reliably resolve all front/back or elevation ambiguities.

For that, we need to consider how the head, torso, and outer ears modify the sound spectrum.

Key takeaway: Spatial hearing depends on both when sound reaches each ear and how its level and spectrum change along the way.

---

3. Head-Related Impulse Responses: Modeling the Path to Our Ears

To simulate the sound arriving at each ear, we need to model how the human body modifies incoming audio.

This is where Head-Related Transfer Functions (HRTFs) and Head-Related Impulse Responses (HRIRs) become useful.

An HRTF describes the directional filtering of sound in the frequency domain.

An HRIR represents the corresponding impulse response in the time domain.

For each sound source direction, we need two impulse responses:

• h_left[n] for the left ear.
• h_right[n] for the right ear.

These responses describe how an incoming signal is transformed on its way to the two ears.

Generating Educational HRIRs

For this project, I implemented a small synthetic HRIR generator.

Instead of relying on a measured dataset, the default model approximates three effects:

1. Arrival-time differences between the ears.
2. Head-shadow attenuation.
3. Mild spectral coloration related to elevation.

The lateral direction is calculated using the source angles:

lateral = (
    np.sin(np.deg2rad(azimuth))
    * np.cos(np.deg2rad(elevation))
)

The application uses the following coordinate convention:

• 0°: front
• +90°: left
• -90°: right
• ±180°: behind

The synthetic model then converts this lateral position into an approximate sample delay.

It also reduces the far-ear amplitude to illustrate head-shadow effects.

This provides a controllable environment for investigating spatial cues without requiring external recordings.

An Important Limitation

The synthetic HRIR model is intentionally simplified.

Real HRIRs contain much more detailed information about reflections, diffraction, and resonances caused by an individual's anatomy.

Therefore, the built-in model is most useful for understanding basic left/right directional cues.

It should not be treated as a physically accurate or personalized 3D audio renderer.

For more realistic experiments, the application can optionally load compatible measured HRIR data stored in the SOFA (Spatially Oriented Format for Acoustics) format.

The current implementation uses the nearest available measurement direction rather than interpolating between measurements.

---

4. Binaural Synthesis: Turning Mono into Directional Stereo

Now we have the essential components:

• A mono audio signal.
• A left-ear impulse response.
• A right-ear impulse response.

The next step is to combine them.

This is done using convolution.

For a mono input signal x[n], we calculate:

y_left[n] = x[n] * h_left[n]

y_right[n] = x[n] * h_right[n]

Here, * represents the convolution operation.

Each output contains a filtered version of the original sound, corresponding to a different ear.

Implementing Convolution with SciPy

The actual project uses scipy.signal.oaconvolve, which implements overlap-add convolution.

from scipy import signal
import numpy as np

def render_one(mono, hrir, gain_db=0.0):
    channels = [
        signal.oaconvolve(
            mono,
            hrir[ear],
            mode="full"
        )
        for ear in range(2)
    ]

    stereo = np.column_stack(channels)

    return stereo * (
        10.0 ** (gain_db / 20.0)
    )

The result is a stereo array containing separate left and right waveforms.

When these signals are played through headphones, the listener receives different directional information at each ear.

Combining Multiple Sound Sources

The application supports up to two independent sound sources.

Each source can have its own:

• Audio recording.
• Azimuth and elevation.
• Gain.
• HRIR pair.

Every mono source is processed separately.

The resulting stereo signals are then summed to create a binaural mix.

To make the mono/binaural comparison more consistent, the application applies one shared clipping-prevention gain to both outputs when necessary.

This avoids independently peak-normalizing the two versions, which could introduce an unnecessary gain difference.

It is important to note that a common gain does not guarantee equal perceived loudness.

Key takeaway: Convolution lets us transform ordinary mono audio into two direction-dependent ear signals using impulse responses.

---

5. Building the Interactive Spatial Audio Playground

Understanding the equations is useful, but I also wanted a way to experiment with them directly.

That became the main motivation behind the application's interface.

The interface is built using Streamlit, with an additional HTML, CSS, SVG, and JavaScript component for the interactive soundstage.

Moving Virtual Sound Sources

Instead of manually entering every angle, users can drag two sound-source markers around a listener.

Each marker represents an independent virtual audio source.

When a source is moved, its new azimuth is sent to the Python application.

The application selects or generates the corresponding impulse responses and renders new audio clips.

The soundstage also supports keyboard interaction and directional presets.

Importantly, this is offline rendering, not continuous real-time spatial audio.

Moving the markers changes the next generated clip rather than streaming a moving sound source.

Comparing Mono and Binaural Playback

One of the central features is a direct listening comparison.

The application produces two outputs from the same input sources:

Centered mono: A reference mix without separate left/right spatial filtering.

Binaural stereo: A direction-dependent mix created using individual impulse responses for each ear.

Users can listen to both versions and download the binaural WAV output.

Headphones are recommended because they deliver the two channels separately to the listener's ears.

Inspecting the Signals

The interface also displays:

• Estimated ITD in milliseconds.
• Estimated ILD in decibels.
• Left and right HRIR waveforms.
• The beginning of the rendered stereo waveform.

This allows users to connect the perceptual experience with the underlying signal-processing operations.

The application also includes a separate cross-correlation experiment and a short theory section.

---

Running the Project Locally

The complete implementation is available on GitHub:

https://github.com/Ertugrulmutlu/from-lab-to-code/tree/main/02-spatial-audio

You will need Python 3.11–3.13.

Clone the repository:

git clone https://github.com/Ertugrulmutlu/from-lab-to-code.git
cd from-lab-to-code/02-spatial-audio

Create a virtual environment:

python -m venv .venv

Install the dependencies:

python -m pip install -r requirements.txt

Launch the application:

python -m streamlit run app.py

The application opens in your browser, typically at http://localhost:8501.

You can immediately experiment with the built-in audio examples.

The project also supports uploading your own short WAV recordings and compatible SOFA datasets.

To execute the automated tests:

python -m unittest discover -s tests -v

The project includes 18 automated unit and integration tests covering signal processing, audio rendering, and related functionality.

The main implementation is organized into separate files:

• correlation.py — cross-correlation and interaural cue estimation.
• binaural.py — convolution, HRIR loading, and WAV handling.
• audio_pipeline.py — rendering and mixing multiple sound sources.
• synthetic.py — original demonstration sounds and educational HRIRs.
• app.py — the interactive Streamlit application.
• frontend/index.html — the draggable soundstage component.

This separation keeps the audio algorithms independent of the user interface.

---

What I Learned

Working on this project helped me connect several ideas that initially seemed independent.

First, small timing differences can carry meaningful spatial information.

Cross-correlation is not only a mathematical technique for comparing signals. It also provides a practical way to investigate the delays that contribute to directional hearing.

Second, impulse responses provide an elegant connection between physical systems and software.

Once we represent the acoustic path to each ear as an impulse response, generating the corresponding output becomes a convolution problem.

Third, a signal can be mathematically correct without producing a perfectly realistic perceptual result.

Our synthetic HRIR model demonstrates timing and level differences, but human spatial hearing also depends on detailed anatomy-dependent spectral cues.

Finally, I learned that making an engineering concept interactive can change how it is understood.

Adjusting an azimuth angle, listening to the result, and examining the corresponding waveforms makes the relationship between theory and perception much more tangible.

Continuing From Lab to Code

In the first project of this series, I explored classical computer vision using filtering, Fourier analysis, thresholding, and clustering.

This second project moves from image processing to audio signal processing.

Both follow the same philosophy:

Take an engineering concept, build an independent implementation, make the results observable, and explain what is happening underneath.

Not every engineering experiment needs to become a large software product.

Sometimes, a small interactive application is enough to reveal how an interesting concept works.

The goal of From Lab to Code is to keep exploring these ideas through practical implementations.

All bundled demonstration sounds in this project are generated programmatically, and no university course materials or proprietary recordings are redistributed.

If you're interested in signal processing, spatial audio, or simply curious about how computers can simulate directional hearing, I hope this project gives you something interesting to experiment with.

Explore the full series: https://github.com/Ertugrulmutlu/from-lab-to-code

Explore Spatial Audio Playground: https://github.com/Ertugrulmutlu/from-lab-to-code/tree/main/02-spatial-audio

Thanks for reading!