When working on the latest feature I had to build, I was tasked with creating and saving audio in the browser through exploring the Web Audio API. As a result I was able to execute audio nodes in the MVP as well as manipulate the frequency to be saved to the database for my dev team's purposes. Ordinarily, I'd say I was satisfied, but just having audio with nothing to visualize it gave me this feeling like I could do something more with my MVP. As luck would have it, one of my teammates was already working on implementing wavesurfer.js which got me interested because I like to try and understand the technologies used by cohorts as much as I can. So I picked wavesurfer.js and decided to obtain a deeper understanding of it. So what is wavesurfer.js?
What is wavesurfer.js?
At its core, wavesurfer.js is an open-source audio visualization library that can be implemented to create interactive and customizable wave forms from decoded audio files. As noted in its name, wavesurfer is a a JavaScript library that is built on top of the Web Audio API and is mostly used in acoustic phonetics research. For reference, the Web Audio API allows developers to interact with web audio in the browser as it is built into the browser. In addition to web audio's support, HTML5 Audio provides additional assistance further expanding wavesurfer's flexibility.
The flexibility of wavesurfer.js allows it to run on most platforms such as Microsoft Windows, Mac OS X, Linux, and HP-UX due to being written Tcl/Tk. Basically Tk exists as a cross-platform widget toolkit that provides a library that consists of GUI (Graphic User Interface) widgets, which are simple elements used to build a GUI. For wavesurfer.js, Tcl/Tk uses the Snack Sound Toolkit which is another cross-platform library with bindings for the scripting languages, Tcl, Python, and Ruby. One of the main reasons for this architectural decision lies in SST's functionality that handles audio I/O, audio analysis, and processing. This allows wavesurfer.js the capacity for spectral analysis, pitch tracking, filtering, and displaying sound pressure waveforms.
These displays can render in various forms with customizable values that help developers build dynamic/interactive UIs for their applications or for researching audio and its visualization. But, what does this all look like and how can a developer retain access to the functions in this library and implement them?
Building a Waveform
Now that we've gone through a brief overview of wavesurfer.js and its architecture, let's explore the library and how to use it. Simply begin by importing the library into your file.
import WaveSurfer from 'wavesurfer.js';
Now that we have WaveSurfer imported in our file, we can create our first wavesurfer visualization.
const visualWave = WaveSurfer.create({
container: document.body,
waveColor: 'rgb(204, 0, 255)',
progressColor: 'rgb(0, 51, 0)',
url: '/examples/audio/audio.wav',
barGap: 3,
})
visualWave.on('click', () => {
wavesurfer.play()
})
In the snippet above, I initialize a new instance of a waveform by invoking the .create() method and assigned some required properties to render the desired visualization. The container property establishes the HTML element that will contain the visualized waveform. The waveColor property is the initial color upon rendering and the progressColor property tracks the progress of the audio file's play through and shows you where you are in the recording. The url property contains the location of the audio file and the barGap property sets the space between each bar in pixels.
With these tools provided I was able to build my own waveform as seen below. The result of the code shows what I implemented and this can be done in the wavesurfer.js documentation. If you're curious then give it shot, the documentation will guide you to your desired result.
As you can see, this is just a basic implementation. However, the library contains more methods and functionality that can be utilized for more advanced uses. That being said, there are other features known as plugins that further enhance the flexibility of what developers can achieve. There are currently seven plugins that can be imported into your file which are: Regions, Timeline, Minimap, Envelope, Record, Spectrogram, and Hover. Listed with them are ways of importing them into you file. While they are part of the wavesurfer library, they are still "plugins" and need to be imported separately.
Regions
import Regions from 'wavesurfer.js/dist/plugins/regions.js';
Provides a visual overlay and markers for regions of the audio. By being able to drag the overlay across the wave, you can isolate smaller chunks of the audio to play back.
Timeline
import Timeline from 'wavesurfer.js/dist/plugins/timeline.js';
Displays "notches" and time tables below the waveform allowing you to keep track of the specific time frequencies occur.
Minimap
import Minimap from 'wavesurfer.js/dist/plugins/minimap.js';
A small waveform that serves as a scrollbar for the main waveform which is useful when the waveform is long and requires more navigation. Located beneath the main waveform
Envelope
import Envelope from 'wavesurfer.js/dist/plugins/envelope.js';
Provides a graphical interface that controls volume as well as fading the audio in and out.
Record
import Record from 'wavesurfer.js/dist/plugins/Record.js';
Records audio from the microphone and renders that audio as a waveform. This plugin is more in depth because of the added tasks of starting, stopping, and recording the audio in order to decode the audio's URL.
Spectrogram
import Spectrogram from 'wavesurfer.js/dist/plugins/spectrogram.js';
A visualization of an audio frequency spectrum. This renders a more graphical interface that relates the visualization of the spectrum of frequencies over time.
Hover
import Hover from 'wavesurfer.js/dist/plugins/hover.js';
Shows a vertical line and timestamp on waveform hover.
Pros/Cons
As with most, if not all, technologies, there can't be pros without the cons and vice versa. While wavesurfer.js has its strengths, there are some limitations and it's up to us developers to decide for or against importing into our projects.
Pros
For starters, the features and plugins nested in wavesurfer.js contribute a great deal to the library's flexibility. The waveforms generated can be given tools and properties for customization that provides more to the "wow" factor to the UI. As for UX, the methods and plugins contribute the necessary tools to developers for more dynamic and interactive audio based web applications. These experiences that can be provided to end users with audio interactions can fit comfortably in a social communication channel that prioritized audio over text. Lastly, which is perhaps the most important upside to wavesurfer.js is the efficient processes of decoding, processing, and analyzing audio files.
Cons
Speaking on audio files, there is a catch to this decoding process in that the audio files in question may fail if the audio file is large. This increases the risk of failing to decode due to memory constraints. Streaming audio isn't supported either because wavesurfer needs to download the entire audio file for decoding and rendering. The fact that wavesurfer decodes audio entirely in the browser with Web Audio, it's best practice to maintain interactions with smaller audio files. Lastly, even though the documentation is thorough and interactive, there is not much documentation that relates making research potentially more difficult if a developer is stuck.
Conclusion
Even though the documentation is lacking, the learning curve for wavesurfer.js isn't that steep and can be considered beginner to intermediate friendly. It seems the pros and cons evenly weigh each other in a fine balancing act depending on what you see as holding more weight on the scale. While wavesurfer.js doesn't have much recreational entertainment use cases currently as it does in acoustic phonetics research, it doesn't mean there can't be. This library will continue to expand and update as it is still relatively young. So who's to say someone like you or me won't come along and enhance this library?
Top comments (0)