Echo Cancellation with Web Audio Api and Chromium

Background

Recently, I ran into a doozy of a bug in Chromium. My pair and I were working on an application that was taking advantage of audio processing using the WebAudio Api. Part of the processing in question was the ability to spatially interact with noise based on the user's position in the application. We were building this application on top of a conferencing api stream that already had it's own processing built in. This added more complexity to the problem, as we had to yank the media stream coming through that api to apply the spatial processing, and then feed it back to the user all tidied up and ready to blow their minds.

The Problem

However, then came Chromium… The spatial awesomeness was working, and pretty neat, but the minute you took your headphones off- BAM, a sound only fit to be heard at experimental electronic basement shows came wallowing through our speakers. An extensive google search revealed an open bug lurking in the issues log.

It seems Chromium does not apply echo cancellation to any remote streams- including WebAudio Api streams. To save others the trouble in working with the WebAudio Api in Chromium, we wanted to collect and share our findings from the community, and ultimately what worked for us.

Step 1:

Create your audio processing and apply it to your stream

const audioContext = new AudioContext()
const destinationNode = audioContext.createMediaStreamDestination();
audioContext.createMediaStreamSource(new MediaStream([audioStream.getAudioTracks()[0]]))
        .connect(createDistortion(audioContext))
        .connect(destinationNode);

Note: audioStream is our current media stream that we want to apply audio processing to.

Be careful, as there are two destination nodes as part of Web Audio API. One is the audioContext.destination, and the other is audioContext.createMediaStreamDestination(); ---> use this one.

Step 2:

Create your audio loopback stream

const loopbackStream = await createLoopbackConnection(destinationNode);

The function createLoopbackConnection is setting up a local webRTC loopback connection. Chromium then picks this up and applies echo cancellation. Voila! Here's a helpful gist in the Chromium issue log we found to see how to do this.

Note: One important hiccup that tripped us up in the creation of a local loopback connection, was trying to use addStream which is deprecated- use addTrack to circumvent this.

Step 3:

Feed the loopback stream into your audio element.

In our case we had an existing audio element that was backed by the conferencing api we were using. We did not have to create one, but rather feed the processed stream into the existing audio element.

const audioElement = document.getElementById(`audio-element`);
audioElement.srcObject = loopbackStream;

Step 4:

Rejoice in the fact that your ears no longer hear feedback!
You no longer need to use headphones after applying audio processing in Chromium!

Lessons learned:

• Debugging audio feedback with a pair is really hard.
• Debugging audio feedback without a pair is even harder.
• Always check issues logs.
• Take care of your ears!

Comments

[James McMahon]

I was able to pair with Katy a bit on this problem and let me tell you it was a dozy. Great write up!

This is us pairing through the Chrome Echo loop...

[Oliver Switzer]

Hi,

Thanks for writing this helpful article up. Gain nodes seemed to work using this technique, but we couldn't seem to get our stereo audio panner node to affect the stream after running it through the loopback function. You mentioned that you were spatializing your audio, so I was wondering if you could share the audio pipeline code that does this?

We've tried putting our stereo panner node both before and after the loopback stream with no luck. Gain nodes totally work though. Very strange.

Thanks again,
Oliver

[djokage]

Thank you for this great workaround, I didnt yet test it but I wonder how I can make it to work with a local audio file like an MP3 that I want to route through this loop to get an echo cancellation, can you help with that please

[Tasha]

We made a demo and post talking about working with the web audio API to connect nodes into a node chain for processing. You can check it out here: daily.co/blog/managing-users-media...

[Ryan Taylor]

This is a simple workaround to a known issue that could grind development speed to a halt if you're not aware of the Chromium bug report. Thanks for sharing!

[ageofcoding]

Tried the code, what I got is:
"createLoopbackConnection is not defined"
Where the clue of createLoopConnection can be found?