Seeing Sound with Amit Nambiar

Amit Nambiar — Thu, 23 Jul 2020 11:52:46 +0000

Hey, I'm Amit! I'm an architect turned computational designer who now helps architects and engineers make buildings faster and better.

In this talk I'm going to talk about a passion project I've been working on where I'm trying to visualise audio in 3D. Sound is an excitation of a medium like air which moves like a wave from the source. While this is well known when it comes to understanding the physics of it we are often introduced to squiggly lines on a paper followed by abstract interpretation of it in math. In this talk I'll be talking about my research project where I parse audio using the WebAudioAPI and visualise it in its complete volumetric glory in the browser. I'll talk about my assumptions, limitations and challenges encountered in the process and attempt to breakdown the physics of some by showing its wavelike behaviour and how I use this information to create a 3D visualization.

Check it out at:
https://lotusaudio.herokuapp.com/view/5ee4fe4610a0ec114483fd4e

Slides:

Here is a download link to the talk slides (PDF)

This talk will be presented as part of CodeLand:Distributed on July 23. After the talk is streamed as part of the conference, it will be added to this post as a recorded video.

Lotus Audio: Create and share virtual environments from the browser

Amit Nambiar — Sun, 14 Jun 2020 11:06:55 +0000

What I built

Lotus Audio is a web app to create and share virtual music environments from the browser. The environments can respond to music so it behaves like a 3D music visualizer that you can get immersed in with virtual reality(VR) devices.

Artists can create scripts which act as plugins into the LotusAudio engine.
The engine processes and passes byte information about any accompanying audio file to the scripts and designers can use this to update their environments in whatever way they choose.
The results of the scripts are rendered as virtual environments compatible with VR devices.

Artists can choose to monetize their creations with Web Monetization. If they have it enabled users would be able to access their virtual worlds of art only with payment enabled. It is a research and explorations into this medium for creative expressions. I work with a community of livecoders and digital performers and excited to show this to them.

Submission Category:

Creative Catalyst

Demo

Link to Code

Github

How I built it

The 3D environments are created using threejs and uses the WebAudio API to parse audio. The environments are WebXR compatible which makes sure they can be accessed through VR devices. The platform is maintained as a web-app running on MEVN stack. In the future I would like to make the full app infrastructure completely decentralized.

Additional Resources/Info

I am pretty excited about this project. I would like extend it by adding support for multi-user experiences using WebRTC. That way several people can get together for virtual algoraves where we party within our creations.

DEV Community: Amit Nambiar