In an attempt to learn THREE.js — the 3D rendering WebGL framework and WebAudio API, I made something that visualises the music in a very simple way. This article documents the whole process.
Final thing first:
(Just use a .mp3 / .mp4 / .wav file to see it work. If you are out, you can use this)
A Primer on WebAudio API
The HTML5’s tag when combined with the WebAudio API, becomes quite powerful. It’s a dynamic tool that lets you process and adds audio effects dynamically to any kind of audio.
The Web Audio API involves handling audio operations inside an audio context and has been designed to allow modular routing. Basic audio operations are performed with audio nodes, which are linked together to form an audio routing graph. Several sources — with different types of channel layouts — are supported even within a single context. This modular design provides the flexibility to create complex audio functions with dynamic effects.
The audio pipeline starts by creating an audio context. It should have at least a single audio source — which can be thought of as an entry point for external files, mic input, oscillators, etc. Once we have a source in place, the signal is processed and moved ahead in the pipeline using audio nodes. After processing, the signal(s) are routed to the audio destination, which can only be a single one in the whole context.
Modular Routing
In an attempt to learn THREE.js — the 3D rendering WebGL framework and WebAudio API, I made something that visualises the music in a very simple way. This article documents the whole process.
Final thing first:
(Just use a .mp3 / .mp4 / .wav file to see it work. If you are out, you can use this)
A Primer on WebAudio API
The HTML5’s tag when combined with the WebAudio API, becomes quite powerful. It’s a dynamic tool that lets you process and adds audio effects dynamically to any kind of audio.
The Web Audio API involves handling audio operations inside an audio context and has been designed to allow modular routing. Basic audio operations are performed with audio nodes, which are linked together to form an audio routing graph. Several sources — with different types of channel layouts — are supported even within a single context. This modular design provides the flexibility to create complex audio functions with dynamic effects.
The audio pipeline starts by creating an audio context. It should have at least a single audio source — which can be thought of as an entry point for external files, mic input, oscillators, etc. Once we have a source in place, the signal is processed and moved ahead in the pipeline using audio nodes. After processing, the signal(s) are routed to the audio destination, which can only be a single one in the whole context.
Modular Routing
The simplest illustration has a single source and a destination, without any effects or processing, inside the context. Why would anyone use this? Maybe they just want to play the sound without any changes.
On the left is an example of a much more complex setup, which can also be made using this API.
Let’s look at the relevant code from the visualiser:
With respect to the WebAudio API, our only purpose of using it in this project is to extract the attributes of the audio signal and use those to update the visuals. If you look at the code above, the analyser node helps us to analyse these musical attributes in real time. Since it does not interferes with the signal and does not changes it in any way, it’s the perfect interface for our use.
This interface (Analyser Node Interface) represents a node which is able to provide real-time frequency and time-domain analysis information. The audio stream will be passed un-processed from input to output.
The core idea of the visualiser was about modulating a sphere’s size based on the beat signature, and deform its surface based on the vocal. To make it interesting I used some procedural noise that adds some physical texture to the ball, using audio data as an input.
The default value of FFT Size is 2048, but we chose a lower resolution of 512 as it’s far easier to compute — considering the very primitive beat detection method that I utilised do not need very high resolution and in addition there will be additional computations for the real time 3D updates, so we can safely go ahead with this value for starters.
The simplest illustration has a single source and a destination, without any effects or processing, inside the context. Why would anyone use this? Maybe they just want to play the sound without any changes.
Refer the complete article here:- https://www.epicprogrammer.com/2021/11/music-visualiser-with-threejs-web-audio.html
Content Inspired by Epic Programmer
Top comments (5)
Is there a specific site from where you are learning three.js or just developing by refering the documentation?
Yep, I'm following epic programmer those guy makes good projects...
Hey I like this. Would love to try this weekend.
Hi, is possible to add a obj or a 3d that I builded in 3d I will like to use a head or a face, thank you
Very cool project! I’d love to try it soon