Live Audio Storage using Firebase and Blobs

I've been coming back to one of the apps I'm most proud of on my portfolio, Virtual Canvas, and attempting to get to a feature I'd imagined in my original vision of the project. The original app lets users create and share canvases to collaboratively build an audio visualizer together.

Now I'd gotten to implementing animations in p5 and using web sockets in action cable to have a functioning project. Users can input mp3s to see their visualizer in action, moving and gyrating to the different audio frequencies. But I'd originally imagined using firebase for its file storage capabilities to let users store and share mp3s they input with others over the canvas as well. I'd seen this medium posting detailing the way they used both firebase storage and firebase's real time database to implement shared audio.

However, the main difference for my case was that the author used the download URL firebase gives you to embed the audio in an html tag. That unfortunately didn't work for me because p5 sound, which syncs the music to the animations, requires a local file rather than an external URL. This means I had to deal with CORS issues to actually download the file from firebase. So in this post I'll try to fill in the major distinctions between my implementation and Luis Miguel Rincon's medium post.

Storing to Firebase

To get a database and cloud storage bucket up and running is pretty well documented on the Firebase docs so I'd refer you over there (database and storage). Plenty of good resources but essentially you'll know you're good to go when when you have the firebase config object to put in your app. Pretty much if you follow along to the end of this video. If you're using React then you'll wanna download the appropriate npm package to deal with firebase.

Once you have a bucket for storage and a database, and you have a file ready to upload to storage then you'll wanna run something like this:

 const musicRef = firebase.storage().ref(`/music/canvas-${this.props.canvas.id}/${file.file.name}`)
const musicRef = firebase.storage().ref(`/music/canvas-${this.props.canvas.id}/${file.file.name}`)

        musicRef.put(file.file).then(() => {
            const storageRef = firebase.storage().ref(`/music/canvas-${this.props.canvas.id}`)
            storageRef.child(file.file.name).getDownloadURL()
                .then((url) => {
                    const databaseRef = firebase.database().ref(`canvas-${this.props.canvas.id}`)
                    databaseRef.push({
                        songName: file.name,
                        url: url
                        })
                })
            })

In my case I created a bucket and file structure that went like "music/[canvasId]/[songName].mp3". And when putting storage, you have to create a reference in firebase storage as seen by this line "firebase.storage().ref(...String Reference). This uses the firebase object imported with

import firebase from 'firebase'

Once you've created a reference, you can then ".put" or place a file into that reference. For me I was storing an inputed file triggered from an onChange input event. The event gave me a file object where the mp3 to store was located at "file.file" to which I have a ".then" attached to then grab the firebase download URL and using the storage reference. I simply followed the firebase convention found here. Once I have the url, I then created a database reference to store the url in. You could definitely store this in possibly your own backend database. But, firebase has some capabilities to enable realtime database events, like file creation. So one user inputing a file can trigger an event so that another user's machine will be able to change state accordingly.

This isn't really anything outside of the scope of Luis's article (albeit some syntax differences), but I figured I'd cover it just to be fully clear.

Retrieving the Data

Now this is where the major departure comes in. When accessing the database where the URLs are stored you'll do the same reference creating as before but use firebase's ".on" method. Mine looked like this

 const database = firebase.database().ref(`canvas-${this.props.canvas.id}`)
        database.on('value', loadData, errData)

loadData = (data) => {
        console.log(data.val())
    }

    errData = (err) => {
        console.log(err)
    }

Before actually accessing the url, you'll have to configure cors requests by following the "CORS Configuration Steps" by firebase found here. Once that is ok you should be able to download the files from the appropriate url.

The reference is highly dependent on your configuration but "database.on('value', this.loadData, this.errData)" is the important part. This "database.on('value',...callbacks)" method will fire anytime a creation in the database occurs and the "loadData" method will be run. This should have the url desired in it and once there you'll want to have something like this using the url:

let objURL;
let xhr = new XMLHttpRequest();
                xhr.responseType = 'blob';
                xhr.onload = function(event) {
                    let blob = xhr.response;
                    objURL = URL.createObjectURL(blob)
                    // ...code that uses objURL can be put here
                };
                xhr.open('GET', url);
                xhr.send();

This uses XML requests which I'm not the most comfortable in, having been taught to use the fetch api. However, this is pretty much lifted off the same page where CORS configuration happens. The onload function is just run once the XML request is completed. All that is added from firebase's code is using the URL objects "createObjectURL" function that takes the xhr blob and puts stores it in its own local url path.

And that's about it. The steps we took were:

1. Setting up a database and file storage in firebase
2. Putting a file in storage using the appropriate reference
3. Then using that storage url to hold in the database so that we'll get nice state change handlers nicely given to us by firebase
4. Using the database reference and the ".on" method to access the file urls
5. Configuring CORS
6. Finally using the download url we want to make an XML request, store that response in a blob and create an object URL to store this blob

The objURL variable will store a string for the url path for us to access. This means we have the downloaded mp3 or img or what have you at a local url for us to do whatever we wish with.

Thanks for following along. I kinda jumped around a bit because the necessary information was naturally in a ton of different places and putting them together was my main hurdle. Feel free to comment if you have any q's.