Building an API for Audio Mix Analysis — Tech Stack and Lessons

I've been building mixdiagnose, an API that analyzes audio mixes and gives you actionable feedback on loudness, dynamics, frequency balance, and more. This post covers the tech stack, the API surface, how the Mix Score works, the frequency banding model, and the lessons I picked up along the way.

Tech Stack

Layer	Tool	Why
API framework	FastAPI	Async, automatic OpenAPI docs, multipart uploads, great DX
Audio analysis	librosa	Spectral features, STFT, frequency extraction
Loudness	pyloudnorm	ITU-R BS.1770 / EBU R128 LUFS measurement
Math / DSP	numpy	Vectorized array ops, FFT, statistics
Storage	SQLite	Zero-ops, file-based, more than enough for this use case
CDN / TLS	Cloudflare	Edge caching, DDoS protection, free TLS

FastAPI handles multipart audio uploads and JSON responses. librosa and numpy do the heavy DSP lifting. pyloudnorm gives me broadcast-standard LUFS. SQLite stores shared analysis results. Cloudflare sits in front for caching and TLS.

The API

Three endpoints make up the core:

POST /api/analyze

Multipart upload — you send an audio file (wav, mp3, flac, etc.), and the server returns a full JSON analysis report. This is the main workhorse:

curl -F "file=@my-mix.wav" https://mixdiagnose.com/api/analyze

The response includes loudness (LUFS, dB peak, dB true-peak), dynamics (crest factor, RMS), frequency band energies, stereo width, and an overall Mix Score.

GET /api/shared/{short_id}

Retrieves a previously shared analysis by its short ID. When you share a mix analysis, you get a short URL like /api/shared/ab12cd. Anyone with that link can view the full report without uploading the file again.

POST /api/share

Saves an analysis result and returns a short_id you can distribute. This is what powers the shareable links — you analyze once, share the ID, and others can view the results.

Mix Score Calculation

The Mix Score is a single number from 0–100 that summarizes overall mix quality. Here's how it works:

1. 12 metrics are evaluated. Each metric is scored as good, warn, or bad.
2. Each metric maps to a numeric value: good = 100, warn = 50, bad = 0.
3. The raw average is computed across all 12 metrics.
4. For every metric flagged as Critical (i.e., a hard failure that seriously impacts playability), 8 points are subtracted from the average.
5. The final score is clamped to 0–100.

def compute_mix_score(metrics: list[dict]) -> float:
    """
    Each metric dict has: name, status (good/warn/bad), critical (bool)
    """
    score_map = {"good": 100, "warn": 50, "bad": 0}
    raw = sum(score_map[m["status"]] for m in metrics) / len(metrics)
    critical_count = sum(1 for m in metrics if m["critical"])
    final = raw - (8 * critical_count)
    return max(0.0, min(100.0, final))

The idea: a mix with one or two warnings can still score well, but critical issues (like clipping, extremely low LUFS, or severe phase problems) drag the score down fast. The -8 per Critical penalty ensures that you can't game the score by being mediocre-but-not-terrible across the board.

The 12 Metrics

The 12 metrics include things like:

• Integrated LUFS — is it in a reasonable range (targeting ~-14 for streaming, ~-9 for loud masters)?
• True peak — is it below -1 dBTP to avoid inter-sample clipping?
• Crest factor — dynamics check (too low = over-compressed)
• RMS level — overall energy
• Low-frequency energy — is the sub-bass region balanced?
• High-frequency energy — is the air band present or harsh?
• Frequency balance — are any bands dramatically over/under-represented?
• Stereo width — mono compatibility and width consistency
• DC offset — is there a DC bias?
• Spectral centroid — overall brightness indicator
• Dynamic range — difference between loudest and quietest sections
• Loudness range — variation in loudness over time (EBU R128)

Each one is evaluated against thresholds tuned from real-world reference mixes.

The 6 Frequency Bands

Audio is split into 6 frequency bands for analysis. This is more granular than the typical 3-band (low/mid/high) split and maps well to how engineers actually think about the spectrum:

Band	Range	What it covers
Sub-bass	20–60 Hz	Kick drum fundamental, sub-bass synths, rumble
Low-mid	60–250 Hz	Bass guitar body, low-end warmth
Midrange	250 Hz–2 kHz	Vocals, snare, guitar presence — the core of the mix
High-mid	2–6 kHz	Attack, presence, intelligibility
High	6–12 kHz	Cymbals, air on vocals, brightness
Air	12–20 kHz	Sparkle, openness, "hi-fi" sheen

FREQUENCY_BANDS = [
    ("sub_bass", 20, 60),
    ("low_mid", 60, 250),
    ("midrange", 250, 2000),
    ("high_mid", 2000, 6000),
    ("high", 6000, 12000),
    ("air", 12000, 20000),
]

def band_energies(y: np.ndarray, sr: int) -> dict[str, float]:
    """Compute per-band energy from audio signal."""
    fft = np.fft.rfft(y)
    mag = np.abs(fft)
    freqs = np.fft.rfftfreq(len(y), 1 / sr)
    result = {}
    for name, lo, hi in FREQUENCY_BANDS:
        mask = (freqs >= lo) & (freqs < hi)
        result[name] = float(np.sum(mag[mask] ** 2) / max(np.sum(mask), 1))
    return result

Energy distribution across these bands reveals common mix problems: too much sub-bass energy (muddy), too much high-mid (harsh), missing air band (dull). The API returns per-band energy and flags outliers.

Lessons Learned

pyloudnorm for LUFS — the right tool for the job

LUFS (Loudness Units Full Scale) is the modern standard for loudness measurement — it's what Spotify, Apple Music, and YouTube use for normalization. Don't try to hand-roll it. pyloudnorm implements ITU-R BS.1770-4 and EBU R128, which is exactly what you need:

import pyloudnorm as pyln
import librosa

y, sr = librosa.load("my-mix.wav", sr=None)
meter = pyln.Meter(sr)  # creates a BS.1770 meter
loudness = meter.integrated_loudness(y)
print(f"Integrated LUFS: {loudness:.1f}")

One gotcha: pyloudnorm expects float audio in the range [-1, 1]. librosa returns this by default, so they work together cleanly. Just make sure you don't apply any additional normalization before measuring.

Crest factor as a dynamics metric

Crest factor = peak amplitude / RMS. It's a simple, effective proxy for dynamic range:

crest_factor = np.max(np.abs(y)) / np.sqrt(np.mean(y ** 2))

• High crest factor (6+): lots of dynamics, good headroom
• Low crest factor (<3): over-compressed, squashed — the mix will sound lifeless

I tried dynamic range meters and loudness range (LRA) from EBU R128, but crest factor ended up being the most intuitive single number for users. It's not perfect, but it's a great quick-check.

SQLite over Postgres — and I don't regret it

For this project, SQLite was the right call:

• Single-server deployment. The analysis is CPU-bound (librosa/numpy), not DB-bound. The database just stores shared analysis JSON — read-heavy, write-light.
• Zero ops. No database server to manage, no connection pool, no migrations beyond schema init.
• Portability. The entire database is one file. Back up by copying it.
• Performance is fine. SQLite handles hundreds of concurrent reads without breaking a sweat. Writes are serialized, but share-creation is rare.

If I ever need horizontal scaling or multi-writer scenarios, Postgres is the obvious migration path. But for a single-instance API doing DSP work, SQLite + WAL mode is more than sufficient:

import sqlite3
conn = sqlite3.connect("mixdiagnose.db", check_same_thread=False)
conn.execute("PRAGMA journal_mode=WAL")

CLI Tool

There's also a CLI so you can analyze mixes without hitting the API:

pip install mixdiagnose
mixdiagnose analyze my-mix.wav

This is great for quick checks during mixing. It runs the same analysis locally and prints the report to your terminal. No upload, no network, no rate limits.

Rate Limiting

The API uses slowapi for rate limiting. Free tier: 3 analyses per IP address. This keeps abuse in check without requiring auth:

from slowapi import Limiter
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)

@app.post("/api/analyze")
@limiter.limit("3 per day")
async def analyze(request: Request, file: UploadFile):
    # ... analysis logic ...
    pass

The limit is per-IP, per-day. If you need more, there's a paid tier with higher limits and API key auth.

Links

• API docs: https://mixdiagnose.com/api-docs
• Famous mixes analysis: https://mixdiagnose.com/famous-mixes

The famous mixes page runs real hit songs through the analyzer so you can see what good looks like — super useful for calibrating your expectations.

---

That's the whole stack. FastAPI + librosa + pyloudnorm + numpy + SQLite + Cloudflare. It's simple, cheap to run, and the analysis is genuinely useful. If you're building something similar, I hope this saves you some time. Questions? Hit me up in the comments.