loading...
Play Button Pause Button

Seeing Sound with Amit Nambiar

amitlzkpa profile image Amit Nambiar ・1 min read

Hey, I'm Amit! I'm an architect turned computational designer who now helps architects and engineers make buildings faster and better.

In this talk I'm going to talk about a passion project I've been working on where I'm trying to visualise audio in 3D. Sound is an excitation of a medium like air which moves like a wave from the source. While this is well known when it comes to understanding the physics of it we are often introduced to squiggly lines on a paper followed by abstract interpretation of it in math. In this talk I'll be talking about my research project where I parse audio using the WebAudioAPI and visualise it in its complete volumetric glory in the browser. I'll talk about my assumptions, limitations and challenges encountered in the process and attempt to breakdown the physics of some by showing its wavelike behaviour and how I use this information to create a 3D visualization.

Check it out at:
https://lotusaudio.herokuapp.com/view/5ee4fe4610a0ec114483fd4e

Slides:

Here is a download link to the talk slides (PDF)


This talk will be presented as part of CodeLand:Distributed on July 23. After the talk is streamed as part of the conference, it will be added to this post as a recorded video.

Posted on by:

amitlzkpa profile

Amit Nambiar

@amitlzkpa

Love to design with code.

Discussion

markdown guide
 

This is fascinating. (I'm a fan of making academic definitions real and concrete, with physical experiments)

 
 

I'm really looking forward to your talk, Amit. I studied physics and have been thinking about I can add that knowledge to increase my value in the marketplace. All I've really thought about is making animation that obeys physics laws.

 

oops. This was for the next talk.

 

This is incredible!!!! I was just telling my musician friend that coding is in nearly everything! Going to show his students soon!

 

That is absolutely beautiful to hear and see. What frequencies have you found to be the most fascinating to work with and have you subjected these visuals to binaural beats at all?

 

I'm still figuring the range that I like to work with most. So far my understanding is that the actually frequency range shouldn't matter too much for what I create. Sort of like which colors are used isn't the only thing that makes a painting good/bad. That said I know people (me included) do have a preference for colour choices. So my hunch is that it'll be an open question forever.

 

Also I'm remapping the range to make it perceptible for us humans in 3d space. The frequency spectrum sits well on a logarithmic scale which becomes too difficult to visualise in a 3D space designed for people.
So there's always that bit of distortion in effect.

 

Great talk! Did you end up doing the FFT on a server side or on the JS client?

 

FFT happens on the client side.
The WebAudioAPI actually does it and is pretty straightforward to work with.

 
 

@dev it seems many parts of this video has been replaced by Vadehi's talk on Cost of Data!

 

pinging @andy because I know he was helping out with tech issues for the conference today 🙏

 

I'm really looking forward to your talk, Amit. I studied physics and have been thinking about I can add that knowledge to increase my value in the marketplace. All I've really thought about is making animation that obeys physics laws.

 

That's awesome!
Please do share if you have any thoughts!

 

I just bought a book called The Jazz of Physics. Seems to have a similar theme.

 

Just checking that out. Seem neat!
Would like to see how different genres "look" like one day.

 

It might just be me but can I just say how beautiful I thought the forms were at each of the frequencies in the example provided during the talk.

 
 

What an exciting way to "see" sound.

 

If you could suspend particles in the air could you apply this technology to create a visual representation of prayer chants that a person could sit inside of?

 

Yeah. That's sort of what I was going for.
The real world unfortunately has forces of gravity which makes it difficult to recreate :(

 
 
 
 

What an amazing talk! Thanks so much for sharing. I'm really interested in Web Audio and MIR in general. What's the best way to get started with the Web Audio API? It's so vast!

 

Can't quite profess to be an expert on this.
But visualizing the spectrogram might the best way to get started and get a sense of how it works.
After that you could in whichever direction interests you.
Besides visualizing, you could also procedurally create sound/music. So it's pretty broad in what you can do with it.

 

Thanks for the tip! I'll look deeper into that and see where the tools take me!

 

these visualizations makes me think of the show devs! beautiful.

 

I love how it starts out looking like a galaxy of stars, then the interference makes it change into patterns. The temple audio was beautiful.

 

Amazing talk, the visualizations are beautiful thank you for sharing this Amit!

 

Amazing presentation Amit! Thanks for helping us "see" sound!

 

I'm in total awe!!! This is so cool!

 

Literally mind blown!! This is beautiful!!!