DEV Community

julianrubisch
julianrubisch

Posted on

Audioproxy: Simple audio variants for your app

#audio #opensource #webdev #architecture

Let's say you're running a podcast hosting platform, or a social network. A user uploads a 40-minute interview in uncompressed format (WAV).

Before anyone can hear it, your app needs four things that did not exist a second ago:

  1. a 30-second preview for the feed,
  2. waveform data so the player can draw something,
  3. transcoded variants in Opus, MP3, etc.,

and the whole thing loudness-normalised, because half of what people upload is recorded too quiet (or too loud).

None of that is hard. ffmpeg does each one in a line. The hard part is everything you build around those lines.

The stack

It usually goes like this:

  • You add a background job, because transcoding in a request would take far too long.
  • You add a database table to remember which variants exist for which upload, because you need to know whether to render or to serve.
  • You add a state column, because a job can fail.
  • You add a backfill task, because the first time you change the preview from 30 seconds to 45, every existing row is wrong.
  • Then someone asks for waveforms and you bolt on a second pipeline that runs audiowaveform, stores the output somewhere, and needs its own backfill.

The result is a media pipeline, which is not what you set out to build, and which now has to be maintained by whoever inherits the codebase.

The part that stings is the ratio. Most of those pre-rendered variants are never requested. You are paying storage and complexity to have answers ready for questions nobody asked.

What if a variant were a URL, not a row?

There is another framing, and it is the one imgproxy made normal for images: stop treating a variant as a thing you make and store, and start treating it as a thing you describe. The description goes in the URL. Nothing exists until someone requests it.

That is what audioproxy does for audio. A URL like this:

GET /{signature}/f:opus/br:96/t:0:30/fade:1:1/plain/s3://masters/interview.wav
Enter fullscreen mode Exit fullscreen mode

is a 30-second Opus preview at 96 kbps with one-second fades, taken from a master stored in your own bucket. Reading it left to right: f: picks the format, br: the bitrate in kbps, t: the trim as start and duration in seconds, fade: the fade in and out. The leading segment is a signature over the rest of the path, so nobody can point your proxy at arbitrary work by editing the URL.

Three things follow from that shape:

The variant is fully cacheable. The options fully describe the output, so the same request always yields the same output. Responses can be Cache-Control: immutable, because a variant that is fully described like this cannot become stale.

The first request streams while it still renders. It is answered as 200 chunked, bytes leaving as the encoder produces them, and can be written back into a variant store at the same time. Nobody waits for the whole encode before hearing anything.

Concurrent requests for the same variant coalesce. Ten people hitting one uncached preview produce one render process, not ten.

Changing your mind about the preview length is now a different URL. There is no migration and no backfill, because there was never a database entry.

What it looks like to run

One single Docker container - no database, no queue. Try it out like this:

docker run --rm -p 4000:4000 \
  -e AP_ALLOW_INSECURE=true \
  -e AP_LOCAL_ROOT=/audio \
  -v /path/to/your/audio:/audio:ro \
  ghcr.io/audioproxy/audioproxy:0.7.2
Enter fullscreen mode Exit fullscreen mode

That mounts a directory read only and runs unsigned, which is meant only for development purposes. Then:

BASE=localhost:4000
SRC='plain/local://interview.wav'

# 30 second Opus preview, faded
curl -o preview.opus "$BASE/insecure/f:opus/br:96/t:0:30/fade:1:1/$SRC"

# Normalised to EBU R128, as an MP3
curl -o normalised.mp3 "$BASE/insecure/f:mp3/br:128/norm:ebu/$SRC"

# Duration, sample rate, channels, tags
curl "$BASE/insecure/info/$SRC"
Enter fullscreen mode Exit fullscreen mode

Sources can be a mounted directory like the one above, or any S3-compatible bucket. Rendered variants go back to a directory or a bucket you own.

Waveforms are free

This is the part I most wanted when I was building it all by hand. Waveform data for graphic display purposes is not a separate service, it is a format:

curl "$BASE/insecure/f:peaks/pts:800/$SRC"
Enter fullscreen mode Exit fullscreen mode

You can retrieve it in audiowaveform's own JSON or its compact binary, which is what peaks.js and friends already read. It is signed, cached, and invalidated exactly like audio, because as far as the proxy is concerned, it's just another format. The contract described above still applies.

From your application

Signing a URL is an HMAC over the path, so it is a few lines in any language. If you are in Rails, it is a gem:

# Gemfile
gem "audioproxy-rails"
Enter fullscreen mode Exit fullscreen mode 
<%= audioproxy_audio_tag @recording.audio,
      format: "opus", bitrate: 96,
      html: { controls: true } %>
Enter fullscreen mode Exit fullscreen mode

That takes an ActiveStorage attachment, works out the source string from whichever service the blob lives on, signs the URL, and hands it to Rails' own audio_tag.

The next post in this series goes through the Rails side properly.

Try it

There is a demo on the project's home page that builds URLs against a real deployment, with no signup: change an option and the waveform is redrawn from a fresh render. On that box a cold 30-second Opus preview lands in roughly 600 ms and about 400 KB; the same URL afterwards comes back from the store in tens of milliseconds.

Next in this series: the same job from Rails, without the job queue.

Top comments (0)