DEV Community

Daniel Pertu
Daniel Pertu

Posted on

22 lines of ULID, and the three different jobs we were asking one id to do

Munchable has no ulid dependency. It has this, in apps/web/lib/ulid.ts:

// Dependency-free ULID generator (lean mandate: no `ulid` package for ~20
// lines of code). 48-bit millisecond timestamp + 80 bits of crypto randomness,
// Crockford base32, lexically sortable by creation time. Used as the app-side
// primary key for catalog.product_revisions (the schema deliberately has no DB
// default for it).

const ENC = '0123456789ABCDEFGHJKMNPQRSTVWXYZ';

export function ulid(now: number = Date.now()): string {
  let ts = '';
  let t = now;
  for (let i = 0; i < 10; i++) {
    ts = ENC[t % 32] + ts;
    t = Math.floor(t / 32);
  }
  const bytes = crypto.getRandomValues(new Uint8Array(16));
  let rand = '';
  for (let i = 0; i < 16; i++) rand += ENC[bytes[i] % 32];
  return ts + rand;
}
Enter fullscreen mode Exit fullscreen mode

Five call sites use it: a contribution revision, a settled reward row, a support ticket, a support message, and an entry in the curated ingredient overlay. What I want to write about is not the twenty lines. It is the three separate things we were originally asking a single identifier to be, and the bug that lives in the gap between them.

Why the generator is app side and the column has no default

The schema comment is blunt about it:

// ULID, generated app side (lexically sortable by time). No DB default.
id: text('id').primaryKey(),
Enter fullscreen mode Exit fullscreen mode

A gen_random_uuid() default is the obvious alternative and it was the wrong shape for us, because the insert order is load bearing. A capture writes an append-only revision row first, and then upserts the product row with current_revision_id pointing at it. If the database invents the id, the application does not know it until the insert comes back, and the write becomes two round trips with a window in between. Generating it first makes both writes part of one statement sequence the caller fully describes before it starts.

Leaving the column without a default is the deliberate half of that. An insert that forgets the id fails loudly at the boundary instead of silently minting a value whose format nothing else in the codebase expects.

The two characters nobody counts

Ten base32 characters hold 50 bits. A millisecond timestamp needs 48. That slack is why the loop can be a plain t % 32 with no masking: the value cannot overflow ten characters until long after the heat death of this product.

It also means every id sorts correctly as text, which is the actual reason for the format. Crockford base32 drops I, L, O and U to avoid transcription errors, and the remaining alphabet is still in ascending ASCII order:

0123456789ABCDEFGHJKMNPQRSTVWXYZ
Enter fullscreen mode Exit fullscreen mode

So a lexicographic comparison of two ids is a numeric comparison of their timestamps. Postgres sorts them with a plain B-tree on text, no expression index, no custom collation, and a keyset-paginated admin queue gets a tiebreak that happens to be creation order.

One honest limitation: there is no monotonic counter inside a millisecond. Two ids minted in the same millisecond are ordered by their random tails, which is to say arbitrarily. Everything we use the ordering for needs a stable total order rather than true creation order at sub-millisecond resolution, so this is fine, but it is the kind of thing that is worth knowing before you build a feature on top of it.

128 bits drawn, 80 bits kept, zero bias

The random half looks wasteful, and it is:

const bytes = crypto.getRandomValues(new Uint8Array(16));
for (let i = 0; i < 16; i++) rand += ENC[bytes[i] % 32];
Enter fullscreen mode Exit fullscreen mode

Sixteen bytes is 128 bits of randomness, and the result keeps 80 of them: five bits per character, sixteen characters. The other 48 bits are thrown away. Drawing one byte per output character and discarding three of its bits is a line of code anybody can read, and the entropy saved by bit-packing has no value to spend.

What matters is that % 32 is exactly the case where modulo is not biased. 256 divides by 32 with no remainder, so each of the 32 characters has precisely eight of the 256 byte values mapping to it. Uniform in, uniform out.

Change the alphabet length and that evaporates. Base62 is the usual example:

256 = 4 × 62 + 8
Enter fullscreen mode Exit fullscreen mode

Residues 0 through 7 get five byte values each, residues 8 through 61 get four. The first eight characters of your alphabet come out roughly 25 per cent more often than the rest. That is the bug in every hand-rolled short-id helper that reaches for bytes[i] % alphabet.length, and it is invisible in testing because the ids still look random. It is only safe here because of a property of the number 32, not because modulo is fine.

The three jobs, separated

A support ticket is where the pressure on a single identifier showed up, because the id was being asked to be three incompatible things at once.

One: the primary key. The ULID. Sortable, opaque, appears in URLs the signed-in owner can see.

Two: something a human can read back to you over email. A 26-character string is not that, so there is a reference derived from it:

export function ticketRef(id: string): string {
  return `MU-${id.slice(-6).toUpperCase()}`;
}
Enter fullscreen mode Exit fullscreen mode

Six characters off the random tail, not the timestamp head. Taking it off the front would give near-identical refs to two tickets opened in the same second, which is precisely when a support queue most needs to tell two things apart. The ref is 32 to the power of 6, or 1,073,741,824 possibilities, and by the birthday bound two tickets would share one at around 38,000 tickets. That is acceptable only because the ref is a label, never a lookup key: it is printed in the acknowledgement email and in the admin list, and no code path resolves a ticket from it. If it had become a lookup key, that number would be a liability.

Three: the secret that authorises an inbound email onto the thread. Not the id. The id loses that competition the moment it is allowed into a URL. So it is a separate value from a separate generator:

const id = ulid();
const threadToken = crypto.randomUUID().replace(/-/g, '');
Enter fullscreen mode Exit fullscreen mode

Support replies come back by email, and the thread token lives in the Reply-To address. That address is the only thing proving an inbound message belongs on a particular conversation, which makes it a bearer credential. It never appears in a DTO, never reaches a client, and goes to the route only so the route can build the Reply-To header. If the sortable id in the URL had doubled as that token, anyone holding a ticket link could append to someone else's thread.

Sortable, memorable, unguessable. Pick any one per value.

See it working

  • munchable.app/support is the form. Submit anything and the confirmation hands you back an MU- reference, which is the six characters above. You do not need an account to use it, which is the point: the person who cannot sign in is the one who most needs to reach us.
  • munchable.app/licenses explains what an accepted capture becomes, which is the append-only revision row whose primary key this generator exists for.
  • munchable.app/answers is the published side of the curated ingredient data, where the overlay entries that use the fifth call site end up, as pages like is sorbitol low FODMAP.

If you have a hand-rolled id helper in your codebase, the one thing worth checking today is the alphabet length. Thirty-two and sixty-four are fine. Anything that does not divide 256 is quietly skewing your ids, and nothing will ever tell you.

Top comments (0)