How I fixed the slug collision that silenced aiappdex.com for 36 hours

I noticed aiappdex.com was showing blank model listings on July 21. The nightly HuggingFace ETL had run successfully on July 20, then — nothing. No GitHub Actions failure notification. No red workflow. Just stale data accumulating while the refresh silently stopped working.

Two hours of log reading later, I found two bugs that had compounded against each other.

What the outage looked like from the outside

The ETL workflow runs on a nightly cron, fetches the top 100 HuggingFace models by downloads, and upserts them into a Turso libSQL database. Astro builds the directory pages statically at deploy time, so a failed upsert doesn't immediately produce 500 errors — it produces pages that never update. The site keeps serving whatever was last built. Models launched after the last successful refresh simply don't appear.

The workflow itself finished with exit code 0. That's the detail that made it hard to spot quickly. The crash happened inside a nested try/catch that was supposed to handle UNIQUE violations gracefully but wasn't recognizing them — so the exception re-threw, propagated up, terminated the for-loop mid-batch, and the outer job swallowed it with a misleading success status.

I should have been watching for unexpected console.warn output in structured alerts. I wasn't. The boring part of every postmortem is the monitoring gap.

Bug one: slugify() collapses distinct HuggingFace model IDs

The slugify function I wrote reduces model IDs to URL-safe strings:

export function slugify(id: string): string {
  return id
    .toLowerCase()
    .replace(/[^a-z0-9/-]/g, "-")
    .replace(/\//g, "-")
    .replace(/-+/g, "-")
    .replace(/^-|-$/g, "")
    .slice(0, 100);
}

Punctuation collapse means org/model-v1.0 and org/model_v1_0 both become org-model-v1-0. These are two different HuggingFace model IDs with different id primary keys in the models table. But after slugification they resolve to the same string, and slug has a separate UNIQUE constraint.

The INSERT ... ON CONFLICT(id) DO UPDATE clause handles the common rerun case cleanly: the same model ID appears in the top-100 list again, and the row gets refreshed in place. I'd written earlier about where ON CONFLICT DO NOTHING beats DO UPDATE in ETL pipelines — but neither pattern covers the cross-ID slug collision case, because the collision is on slug, not id. The ON CONFLICT(id) clause isn't watching the slug column at all.

What changed around July 21: a popular HuggingFace organization released two models with IDs that slugified identically. Both appeared in the top-100 by downloads for the first time on the same nightly run. The first model upserted correctly. The second threw a UNIQUE violation. My handler tried to catch it and failed.

Bug two: @libsql/client changed its error shape between 0.14 and 0.17

This is what made the first bug fatal rather than annoying.

My original UNIQUE violation check:

function isUniqueViolation(err: unknown): boolean {
  if (typeof err !== "object" || err === null) return false;
  const e = err as { code?: string };
  return e.code === "SQLITE_CONSTRAINT_UNIQUE";
}

In @libsql/client 0.14, this worked. The code field on a constraint error was "SQLITE_CONSTRAINT_UNIQUE". I'd tested it, it held up, I moved on.

Between 0.14 and 0.17, the shape changed. The code field became "SQLITE_CONSTRAINT" — the broader category. The specific constraint type moved to a new field: extendedCode: "SQLITE_CONSTRAINT_UNIQUE". My check was reading e.code === "SQLITE_CONSTRAINT_UNIQUE", which now always returned false for any constraint violation.

So when the slug collision hit, isUniqueViolation() said "not a UNIQUE violation" and re-threw. The for-loop crashed. Models after the collision in the sorted list were never stored. The directory showed stale data for 36 hours.

I'd been running routine pnpm update without specifically tracking whether @libsql/client changed its error interface. The change wasn't flagged as breaking in the release notes — it showed up in a GitHub issue, not the CHANGELOG. The credential-isolation setup I'd done earlier gives me confidence that auth isn't the issue when the ETL fails, but it doesn't protect against API surface changes in the client library itself.

The fix: stable rawCode detection and hash-suffix fallback

For detection, I added rawCode: 2067 as a cross-version anchor:

function isUniqueViolation(err: unknown): boolean {
  if (typeof err !== "object" || err === null) return false;
  const e = err as { code?: string; extendedCode?: string; rawCode?: number };
  return (
    e.code === "SQLITE_CONSTRAINT_UNIQUE" ||
    e.extendedCode === "SQLITE_CONSTRAINT_UNIQUE" ||
    e.rawCode === 2067
  );
}

SQLite's extended result code for UNIQUE constraint violations is 2067 — that's SQLITE_CONSTRAINT_UNIQUE in the SQLite extended result codes spec. It's part of the SQLite specification, not a libsql convention, and it hasn't changed since SQLite 3.7.16. The string checks stay in the function for readability and work in any version where the strings are correct. The rawCode === 2067 check is the thing that actually holds across client library updates.

For the slug collision itself, I added a deterministic hash-suffix fallback:

export function collisionSlug(id: string): string {
  const hash = createHash("sha1").update(id).digest("hex").slice(0, 6);
  return `${slugify(id).slice(0, 93)}-${hash}`;
}

And the upsert now retries once before giving up:

try {
  await upsert(slugify(id));
} catch (err) {
  if (!isUniqueViolation(err)) throw err;
  // Slug is taken by a DIFFERENT model. Retry with hash suffix.
  try {
    await upsert(collisionSlug(id));
    console.warn(`[etl] slug collision for ${id} — stored as ${collisionSlug(id)}`);
  } catch (err2) {
    if (!isUniqueViolation(err2)) throw err2;
    console.warn(`[etl] skipping ${id}: slug collision even after hash suffix`);
    continue;
  }
}

First attempt uses the plain slug. If that hits a UNIQUE violation on slug — meaning a different model ID already claims that slug — we retry with the 6-character hex suffix appended. If even that collides (6 hex chars = ~16 million values; against a directory with a few hundred slugs this is near-impossible), we log and skip rather than crashing the entire run.

SHA-1 is fine here. I'm not using it for security — I'm using it for determinism and brevity. The same model ID produces the same 6-character suffix across every nightly run, so the URL doesn't change between refreshes. The cryptographic weakness of SHA-1 against adversarial preimage attacks is irrelevant when the input space is HuggingFace model ID strings.

Why ON CONFLICT(id) DO UPDATE wasn't enough

When I designed the upsert, I assumed ON CONFLICT(id) covered the conflict surface. It does cover the most common case: the same model appears in the top-100 list on consecutive nights, and the row gets refreshed. This works correctly and I've had no issues with it.

What it doesn't cover is two different IDs competing for the same slug value. They have different primary keys, so ON CONFLICT(id) never fires. The slug TEXT NOT NULL UNIQUE constraint is a separate guard, and there's no ON CONFLICT(slug) clause — I hadn't thought to write one.

I briefly considered adding ON CONFLICT(slug) DO NOTHING to the insert. I decided against it: DO NOTHING silently drops the second model with no record of what happened. The exception-first retry approach at least produces a warning log and stores the model under a differentiated slug. Looking at my ETL health queries confirmed that a few collision-slug entries would be visible and auditable, rather than invisible in a silent skip.

The trade-off: the hash-suffix slug for the colliding model differs from what slugify(id) would produce. If that model later becomes more popular and someone has linked to it, the URL is stable as long as the hash is deterministic — which it is. If the first claimant of the slug is later removed from the top-100 and its row is cleaned up, the collision model retains its hash-suffixed URL permanently. That's an edge case I'm comfortable living with.

What I'd do differently

Three things, in priority order:

Route console.warn to a notification sink. Slug collisions now log to stdout, which GitHub Actions captures. But I only see it when I go looking. The pipeline health watchdog I built intercepts certain ETL failure signals and opens a GitHub Issue — I should extend it to treat [etl] slug collision warnings as notification-worthy. A collision isn't a crisis with the fix in place, but I want to know within hours, not when the directory goes dark.

Write a version-pinned test for isUniqueViolation(). The test doesn't need a live database. It needs a mock error object shaped like what @libsql/client@0.14 throws and another shaped like what 0.17 throws, and it should assert that isUniqueViolation() returns true for both. That test would have caught the regression when I ran pnpm update. I'd also benefit from tighter version pinning in CI: pnpm update pulling a minor bump that changes error shape is not behavior I want to merge without a test run that exercises the error path.

Consider a pre-flight slug lookup before insert. Before inserting a model, I could query SELECT id FROM models WHERE slug = ? and check whether that slug is taken by a different ID. If it is, I skip straight to collisionSlug() without going through the exception path. The downside is two DB round-trips per model instead of the common case of one. Against Turso's edge-deployed replicas, the per-query latency is low enough that 200 extra round-trips in a 100-model batch probably wouldn't dominate the refresh time. But the exception-first pattern is still correct for the no-collision case, which is 99% of runs — pre-flight optimizes for the rare case at the cost of extra queries in every run. I'll revisit if the collision rate climbs.

The compounding factor

Neither bug alone would have caused the 36-hour outage.

If isUniqueViolation() had been working correctly on 0.17, the first slug collision would have been caught, the hash-suffix retry would have run (an earlier version of this existed), and the model would have been stored under a differentiated slug. The run would have completed. Nobody would have known there was a collision.

If slugify() hadn't produced a collision in the first place, isUniqueViolation() never would have been called for a UNIQUE violation, so its broken behavior would have been invisible.

It took both bugs together to produce the outage: the collision triggering a UNIQUE exception, and the exception handler failing to recognize it as UNIQUE, causing the exception to propagate up and crash the for-loop mid-batch.

This is the compounding pattern I tend to miss in code review — each piece looks reasonable in isolation, and the combined failure mode requires reading two different parts of the codebase at the same time. The four GitHub Actions ETL patterns I use include workflow-level retry on non-zero exit codes, but that only helps when the job actually fails. A job that exits 0 while doing the wrong thing is the hardest failure mode to catch automatically. Exit code 0 is the tell — not "succeeded" but "didn't throw at the top level."

---

Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.