Auto-Tag Your Music Library — With Confidence Scores

#music #api #machinelearning #python

"Tag my whole catalogue by mood, energy and genre" is one of those jobs that sounds like a weekend script and turns into a procurement cycle. The enterprise tagging tools — Cyanite is the one people name — do the job well, but the shape of the deal is the friction: you upload your audio, you sign up for a seat, and what comes back is a confident flat label (energetic, happy, techno) with no indication of how it was decided or how much to trust any single tag. For a side project, an internal catalogue tool, or a feature you're still prototyping, that's a lot of ceremony for some adjectives.

This post is a practical walkthrough of the self-serve alternative: tag tracks by name (no audio upload) in about 30 lines of Python against the FreqBlog Music API's GET /tag — and, crucially, get a confidence and a provenance on every tag, so you know which ones are a measurement and which ones are a hint.

What /tag actually is. It's a tag-shaped projection of the same open-data analysis /lookup already returns — no audio upload, no new compute on your request. If you want the full numeric feature set (raw bpm, key, the [0,1] floats), use /lookup; if you want the nearest-sounding tracks to a seed, use /similar. /tag is the "just give me the labels, honestly" surface.

Why honesty is the feature

Audio tagging is not one problem — it's a stack of problems with very different reliability. A track's energy or danceability is a measurement off the waveform: reproducible, defensible, and the same every time. A one-word mood is a derivation from those measurements via a rule. A fine-grained mood like "aggressive: 0.71" is a model estimate from a research-grade classifier that was honest enough about its own limits that its authors deprecated it. Genre is a broad catalogue tag, not a taxonomy you should bet a recommender on.

Most taggers flatten all four into the same confident-looking string. FreqBlog's /tag refuses to: every tag in the response carries a confidence and a provenance, so a measured high-energy and a model-estimated aggressive never look equally authoritative. That's the differentiator — not "more tags," but tags you can reason about.

The call — one track, in

GET /tag resolves a track by exactly one of: track (plus optional artist), isrc, mbid, spotify_id, or track_id (a catalog itunes_track_id) — the same identifiers /lookup accepts. No Spotify account, no audio file.

import requests

API = "https://api.freqblog.com"
HEADERS = {"X-Api-Key": "sk_live_your_key_here"}

def tag(track, artist=None):
    params = {"track": track}
    if artist:
        params["artist"] = artist
    r = requests.get(f"{API}/tag", params=params, headers=HEADERS, timeout=30)
    r.raise_for_status()
    return r.json()

data = tag("Blinding Lights", "The Weeknd")
for t in data["tags"]:
    val = "" if t["value"] is None else f"  ({t['value']})"
    print(f"  {t['category']:16} {t['tag']:18} [{t['confidence']}]{val}")

For Blinding Lights that prints a handful of tags across the categories — a measured high-energy (1.0), very-danceable and acoustic; a derived mood of tense; and a catalog genre of synthwave. The response envelope is { track, count, tags:[…], disclaimer }: track is the resolved stub (name, artist, ids), count is the number of tags, and disclaimer is a plain-English note about provenance you can surface in a tooltip.

The four confidence levels — what each one means

Every tag's confidence + provenance tells you which bucket it came from. There are exactly four, and they're ordered most-to-least authoritative:

confidence	provenance	what it is	value
`measured`	`essentia`	Our own Essentia analysis of the recording — energy, danceability, valence, acousticness, instrumentalness — bucketed to a human label.	the [0,1] score
`derived`	`valence+energy`	A single MIREX-style mood category computed from the measured valence + energy (e.g. `tense`, `calm`).	`null` (label only)
`model-estimated`	`acousticbrainz`	AcousticBrainz mood SVM probabilities (happy / sad / aggressive / relaxed / party). Research-grade — treat as a hint.	the raw probability
`catalog-genre`	`catalog`	The broad catalogue genre tag (iTunes / Last.fm chain). Coarse, not a fine taxonomy.	`null` (label only)

So the rule for a consumer is simple and the API does the heavy lifting of being honest: trust the measured tags as fact, treat derived as a reasonable summary, and gate any product decision on a model-estimated mood by its value — it's only emitted when the model's probability clears 0.5, and the raw number is right there for you to threshold higher.

Two things we deliberately do NOT serve — and why it makes the output better. (1) AcousticBrainz also ships a genre column, but it's roughly 80% mislabelled "electronic" — so we serve the broad catalogue genre instead and never expose the AB genre column. (2) AB ships a single argmax mood label ("this track is happy"); we don't serve that verdict either — we expose the per-axis SVM probabilities with the raw numbers, so you can see it's happy: 0.58 / relaxed: 0.55 rather than a confident coin-flip dressed as a fact. Leaving bad data out is part of the honesty.

Tagging a whole library

The endpoint is one track per call, so "tag my library" is a loop — resolve each (title, artist) once, keep the tags you care about, and you're done. A small, readable version that pulls just the high-confidence labels for a playlist:

library = [
    ("Blinding Lights", "The Weeknd"),
    ("Bohemian Rhapsody", "Queen"),
    ("One More Time", "Daft Punk"),
]

def trusted_tags(track, artist):
    """Measured + derived tags only — the ones safe to file on."""
    data = tag(track, artist)
    out = {}
    for t in data["tags"]:
        if t["confidence"] in ("measured", "derived", "catalog-genre"):
            out.setdefault(t["category"], t["tag"])
    return out

for title, artist in library:
    tags = trusted_tags(title, artist)
    print(f"{title} — {artist}")
    print(f"   {tags.get('energy')}, {tags.get('danceability')}, "
          f"mood={tags.get('mood')}, genre={tags.get('genre')}")

That's the whole pattern: a function per track, a filter on confidence, and a dict you write to your own store. If you want the model-estimated mood axes too, drop the filter and read value — the data's there, you just choose how much to trust it. Each successful call is one quota unit (it's a /lookup projection — same data, same cost), and you're only charged on a served 200; a miss (404) or a request with no/invalid identifier (422) costs nothing. The free tier's 1,000 requests/month is enough to tag a real playlist before you ever need a paid plan.

What you can and can't tag — the honest coverage

This is the part most "tag any track" pitches get wrong, so here it is plainly. There are two coverage stories and they're different:

The measured tags — the broad, reliable coverage. Energy, danceability, valence, acousticness, instrumentalness and the derived mood come from our own Essentia analysis over the pre-analysed catalogue (hundreds of thousands of tracks), and any track not yet in it gets pulled in on demand the first time you look it up. You can tag these by name. Lead with this — it's where /tag is dependable.
The model-estimated mood axes — reachable when you supply the identifier. The AcousticBrainz SVM probabilities sit over 7.5M+ recordings, but they're keyed by MusicBrainz ID. So you can reach that slice by passing an mbid or an isrc — when you supply the identifier — not by track name alone. Only a small fraction of AB rows are resolvable by name today.

The one-line version: tag any track by name (the full analysed catalogue + on-demand backfill) for the measured features, or by MBID / ISRC against 7.5M+ AcousticBrainz recordings when you supply the identifier for the model-estimated mood. It is not "7.5M tracks by name" — and we'd rather tell you that up front than have you discover it in production.

How it compares to enterprise auto-tagging — honestly

No upload, self-serve. You tag by name / ISRC / MBID against open-data analysis — there's no audio to upload, no seat to provision, and you can be tagging a playlist 60 seconds after you get a free key.
Confidence + provenance on every tag. This is the headline difference. A closed tagger gives you a label; /tag gives you a label plus where it came from and how much to trust it — so you can file the measured tags and gate the model-estimated ones.
Open data, not a black box. The measured tags are our Essentia pipeline; the mood axes are AcousticBrainz; the genre is the broad catalogue tag. Nothing here is a proprietary score you can't reason about — and we leave the known-bad fields (the ~80%-"electronic" AB genre column, the argmax mood verdict) out on purpose.
Coverage, not magic. The measured tags need the track in the catalogue (or pulled in on demand); the model-estimated mood needs an MBID/ISRC. A track we can't resolve simply comes back without those tags rather than with a confident guess — better an honest gap than a wrong label.

Try FreqBlog — free tier, no card: https://freqblog.com/