Anton Brilliantov

Posted on Aug 17

Snowflake IDs - Part 1: Why int8 + Snowflake, and Never UUID

#database #architecture #go #php

An identifier is load-bearing infrastructure, not a boring column. Part 1 of a short series on how I pick identifiers and why: a 64-bit Snowflake in a BIGINT, generated at the edge, time-ordered, and small - versus a random UUID that shreds your indexes or a BIGSERIAL that prints your growth rate on every URL. No custom generator - the good implementations already exist; this is about the reasoning, not the wheel.

👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go. Most of what I write is about the slow, careful business of breaking a large PHP monolith into Go services while it keeps serving real customers, and identifiers are the one thing that has to stay coherent while everything else moves. This little two-part series pulls the ID decision out on its own. Part 1 (this one): why int8 + Snowflake is the right primary key. Part 2: how you move that id across a service mesh, a public URL, and a browser without leaking or corrupting it. Running notes live on my GitHub: github.com/brilliant-almazov. No hype, just the real work.

An ID is the most-touched value in the whole system. It's the primary key, the foreign key, the URL segment, the cache key, the gRPC field, the log line you grep at 3am. Pick it badly and the cost doesn't show up in a demo - it shows up months later as index bloat, a competitor reading your order count, or a support ticket that says "it edited the wrong record and nobody knows why." So I treat the ID as a design decision, not a default, and I make it deliberately.

An ID has to be two things at once: small and unguessable-ish

Small, because it's in every index and every join. Unguessable-ish, because it ends up in places I don't fully control. Those two pulls usually fight - the value that's cheapest to store (a sequential counter) is the one that leaks the most, and the value that leaks the least (a random 128-bit UUID) is the one that wrecks your indexes. A Snowflake in a BIGINT is the point where they stop fighting.

64-bit BIGINT: eight bytes, and the B-tree loves it

A BIGINT is 8 bytes. A UUID is 16. That 2× isn't just disk - it's every index page, every foreign key, every tuple in the cache. Double the key width and you halve how many entries fit in a B-tree page, which means more pages, deeper trees, more cache misses on every lookup.

But the width is the smaller half of the story. The bigger half is ordering. A Snowflake's high bits are a timestamp, so IDs come out time-ordered - roughly sorted by creation. New rows land at the right edge of the index, next to the last one, so inserts append to warm pages instead of scattering across cold ones. The index stays tight.

64-bit Snowflake ID   (fits in a signed BIGINT / int64)

  1 bit    41 bits                 10 bits      12 bits
 ┌──────┬─────────────────────┬───────────┬────────────┐
 │ sign │  timestamp (ms)     │ worker id │  sequence  │
 │  0   │  ms since a custom  │  0..1023  │  0..4095   │
 │      │  epoch              │  which    │  per-ms    │
 │      │                     │  node     │  counter   │
 └──────┴─────────────────────┴───────────┴────────────┘
  unused  high bits = time ->    where it   uniqueness
          IDs sort by creation   was minted within a ms

  time in the HIGH bits  =>  k-sorted IDs  =>  append-y, tight B-tree

Why not UUID

UUID is the reflexive "distributed id" answer, and it's the wrong shape for a primary key.

UUIDv4 is 128 bits of randomness. Every insert lands in a random spot in the index, forcing page splits, fragmenting the tree, and turning what should be an append into a random-write storm. On a hot table you feel it: write throughput drops and the index bloats far past the data it indexes. You pay that on every table, forever, for a property (global uniqueness) a 64-bit Snowflake already gives you at half the width.

UUIDv7 fixes the ordering problem - it puts a timestamp up front, so it's k-sorted like a Snowflake. It's a genuine improvement and a fine choice if you're wedded to the UUID type. But it's still 128 bits - twice the storage of what I need, in every index and every FK - and the timestamp-in-the-front makes it more enumerable, not less. A time-ordered 64-bit id gives me the same locality win at half the width, in a native BIGINT the database and every driver already understand, with a worker-id field that makes multi-node generation collision-proof by construction.

So: UUIDv4 loses on locality, UUIDv7 loses on width. Snowflake int8 wins both.

Already on UUIDs? They're just integers too

Here's the thing that makes this whole approach not a religious int-vs-UUID war: a UUID is only 128 bits, and 128 bits is just numbers. Any UUID can be represented as integers - two int64 (a hi/lo pair), a single 128-bit big integer, or, depending on the version, just the bytes that carry meaning (a v7's 48-bit timestamp prefix, a v4's random bits). Nothing about "it's a UUID" stops you from treating it as an int.

Which means: even if your primary keys are already UUIDs, you don't have to migrate them to get most of this. You keep the UUID column and, at the boundary, represent it as int(s) and do exactly the same thing - the same hashids/Sqids treatment (those libraries encode a list of numbers, so a 128-bit UUID is simply two numbers), the same per-(tenant × type) salt, the same rotation-with-audit, the same int64-as-string for the browser (Part 2). The principle is identical; the only thing that changes between UUID versions is which bytes you read.

Be honest about what that does and doesn't buy you, though. Representing a UUID as ints gives you the whole boundary story for free - clean URLs, hashids, salt rotation, no naked ids leaking. It does not fix the storage story: a UUID is still 128 bits in every index, so the locality and width costs from above stay until you actually move the primary key to a Snowflake int8. Treat-as-int is the zero-migration on-ramp; Snowflake int8 is the destination.

UUID versions: which bytes matter, and when a UUID actually earns it

"Represent it as ints" means read the bytes that matter for that version - and the versions are very different animals:

v1 (time + node) - a 60-bit timestamp split awkwardly (low bits first) plus a clock sequence and the machine's MAC. Time is in there but leaks the node; rarely what you want.
v4 (random) - 122 bits of pure randomness, no structure to read. Carry it as the full 128-bit (hi/lo) pair; there's nothing to sort by. This is the one that shreds index locality.
v5 (name-based, SHA-1) - deterministic: uuidv5(namespace, name) hashes a namespace + a name and always returns the same UUID for the same input. The namespace behaves like a salt. No randomness, no coordination - the id is a pure function of stable inputs.
v7 (unix-ms + random) - a 48-bit millisecond timestamp in the high bytes, then random. Read the top 48 bits and you've got a time-ordered key, k-sorted like a Snowflake - which is exactly why it's the good modern default if you're staying on UUIDs.

Why I'd actually reach for a UUID - v5 in particular. Snowflake int8 is my default for a minted primary key. But v5's determinism is a genuinely great property when you need to control the id instead of generating a fresh one: derive a stable id straight from a business key so the same input always maps to the same row (idempotency and dedup for free), content-address something, or reproduce an id across systems with no shared sequence and no lookup table. "The id comes from the data" is a real superpower for those cases - and it's precisely what a random Snowflake can't do.

Honest pros and cons, so it's a choice and not a reflex:

UUID good: a 128-bit space you'll never exhaust; fully decentralized; v5 deterministic/derivable (control the id); v7 time-ordered; a universally understood type with native columns everywhere.
UUID bad: 16 bytes in every index and FK (2× a BIGINT); v4 destroys locality; heavier logs, URLs, cache keys; and it's still not a secret, so it needs the exact same boundary treatment (hashids, per-tenant salt) as an int.

So the rule I actually use: default to Snowflake int8 for minted keys; reach for UUID v5 when I specifically need a derived, reproducible id; and whichever I hold, carry it as int(s) at the boundary and apply Part 2 unchanged. I use it this way because the id strategy should follow what the id is for - minted-and-fast versus derived-and-controllable - not which type is fashionable.

Snowflake = generate the ID at the edge, before the INSERT

The layout is the whole trick: timestamp | worker id | per-ms sequence. The timestamp gives ordering. The worker id says which node minted it, so two machines never collide. The sequence counts multiple IDs inside the same millisecond on one node. Put together, any node can mint a globally-unique, time-ordered ID locally, with no round-trip to a central sequence and no coordination.

That last part matters more than it sounds. It means the service about to create a row already knows the row's ID before it talks to the database. I can build the whole object graph - parent, children, the foreign keys between them - in memory, then insert it in one shot. No "insert, read back the generated id, insert the children" dance.

The generation is a solved problem - don't write your own. The layout is a well-known bit formula and there are mature, battle-tested libraries in every language. Pick one and move on:

Go: bwmarrin/snowflake, sony/sonyflake (a slightly different bit split), godruoyi/go-snowflake.
PHP: godruoyi/php-snowflake.
The original is Twitter's Snowflake; Instagram, Discord and Sony all shipped documented variants. Any of them is fine - what matters is the shape, not the repo.

The whole algorithm, as pseudocode, is just a couple of shifts - so you can see there's nothing to hand-roll:

# what every Snowflake library does, once you strip the mutex and clock handling
id = (ms_since_epoch << 22)   # 41 bits of time, in the high bits
   | (worker_id      << 12)   # 10 bits: which node minted it
   |  sequence                # 12 bits: per-ms counter, wraps at 4096

# time in the high bits => ids sort by creation
# worker_id => no cross-node collision
# sequence  => up to 4096 ids per node per millisecond

You configure the worker id per node and the custom epoch once, then call Next() from the library. That's it.

Where Snowflake comes from - the original, and the descendants

Snowflake isn't a pattern someone blogged into existence - it's a specific system Twitter built and open-sourced in 2010, and the choices above are load-bearing decisions from a real migration, not arbitrary bit-counting.

Twitter announced it in Announcing Snowflake (June 2010). The trigger was concrete: they were moving off MySQL (whose auto-increment handed them ids) onto Cassandra, which has no built-in id generation. They needed ids that were (1) roughly time-ordered so they'd sort, and (2) generated with no coordination - no central sequence, no lock, no single point of failure - at tens of thousands per second. That brief is the two properties this article keeps circling back to.

The original is a Scala network service (Thrift-based, predating Finagle), still readable in the archived repo twitter-archive/snowflake at the snowflake-2010 tag. Its layout is the one everyone copied: 1 sign bit · 41 bits ms timestamp (custom epoch, ~69 years of range) · 10 bits machine id (Twitter split it 5 datacenter + 5 worker = 1024 nodes) · 12 bits sequence (4096 ids per node per millisecond). It's now enough of a standard to have its own Snowflake ID writeup.

Those 10 machine bits are the whole multi-DC / multi-node story - why you can run this across datacenters with zero coordination:

  10 machine bits  =  5 datacenter  +  5 worker   =>  1024 independent minters

  DC A                      DC B                      DC C
  ┌────────────────┐        ┌────────────────┐        ┌────────────────┐
  │ worker 0 → id  │        │ worker 0 → id  │        │ worker 0 → id  │
  │ worker 1 → id  │        │ worker 1 → id  │        │ worker 1 → id  │
  │ worker 2 → id  │        │ worker 2 → id  │        │ worker 2 → id  │
  └────────────────┘        └────────────────┘        └────────────────┘
   no lock · no              no lock · no              no lock · no
   central sequence          central sequence          central sequence

  every node mints LOCALLY:  id = (time | dc | worker | sequence)
  the (dc,worker) field makes each node's id-stream DISJOINT  → collisions impossible
  no cross-node / cross-DC talk on the write path             → no bottleneck, no SPOF
  scale out = hand the new node a (dc,worker) id              → it just starts minting

The idea then spread by being re-balanced for different scale points:

Instagram - Sharding & IDs at Instagram brought the same idea inside Postgres with PL/pgSQL, minting time-ordered ids per shard (41 bits time, 13 bits shard, 10 bits per-shard sequence).
Sony - Sonyflake re-cut the bits for more machines: 39 bits of 10-ms time, 8 sequence, 16 machine bits (65,536 nodes) - a different point on the same trade-off curve.
Discord adopted the format wholesale and, tellingly, returns every id as a string in JSON to dodge the exact 2⁵³ browser bug Part 2 is about.

And you never hand-roll the bit-twiddling - mature libraries carry it: Go bwmarrin/snowflake and sony/sonyflake; every language has an equivalent. Pick an epoch and a worker id, call Next().

vs BIGSERIAL / SERIAL: three problems Snowflake doesn't have

BIGSERIAL is the reflexive default in Postgres, and it's fine right up until it isn't. It backs the column with a central sequence, and that sequence is three problems wearing a trench coat:

It's a coordination point. Every insert asks the sequence for the next value. On one box that's cheap; across shards or services it's a round-trip and a bottleneck you can't shard away.
You can't know the ID before the INSERT. The value is assigned by the database, on write. So you insert, read back the generated key, then insert anything that references it - extra round-trips, and awkward the moment you want to assemble a graph before persisting it.
Cross-shard collisions. Two shards each running their own sequence both hand out 42. Merge or federate them and the keys clash. Snowflake's per-node worker id makes that structurally impossible.

Snowflake fixes all three: no central sequence, ID known at the edge, no collision across nodes.

Property	UUIDv4	UUIDv7	BIGSERIAL	Snowflake int8
Width	128-bit / 16 B	128-bit / 16 B	64-bit / 8 B	64-bit / 8 B
Ordering	random	time-ordered	sequential	time-ordered (k-sorted)
Index locality	poor (page splits)	good	excellent	excellent
Where it's generated	client / anywhere	client / anywhere	central sequence	at the edge, per node
Know the ID before INSERT	yes	yes	no	yes
Cross-shard collision	~never	~never	possible	~never
Casual enumeration leak	none	some (time prefix)	trivial	hard

Generation performance, and the control-vs-throughput trade-off

The reason Snowflake feels free is that it is almost free. Minting an id is a couple of bit-shifts under a mutex - no I/O, no network. The mature libraries do millions of ids per second per process, and the only hard ceiling is the sequence field: 4096 ids per node per millisecond (~4M/s). Hit that ceiling inside one millisecond and the library simply waits for the next one - bwmarrin/snowflake blocks to the next ms, sony/sonyflake trades time resolution (10-ms units) for far more machine bits. Crucially it scales linearly: two nodes mint twice as fast because they share nothing. There's no contention to fight.

Now put BIGSERIAL next to that. Every id is a round-trip to a shared sequence - network latency plus lock contention on one hot object - and your id throughput is bounded by the database, not the app, and can't scale past that one authority. That's the SERIAL story in one line: the sequence is a single, coordinated, and therefore rate-limited source of truth.

Which surfaces the real axis you're choosing on - control vs generation:

Central sequence (SERIAL) - maximum control: dense, gapless, strictly monotonic, one authority that can hand you "invoice #1001 with no gaps." The price is coordination: a round-trip, a bottleneck, a single point, and no id before insert.
Snowflake int8 - maximum generation: local, uncoordinated, linear-scaling, id known before insert. The price is you give up density and a single authority - ids have gaps, ordering is only k-sorted, and it leans on the node clock.
UUID v4 - maximum independence, zero order or control (and the widest storage).
UUID v5 - control of a different kind: not dense/monotonic, but deterministic - the id is derived from the data (see the UUID section).

Balance it by the job, and don't be afraid to mix: Snowflake int8 on the hot write paths where throughput and edge-generation win, and a real sequence exactly where a business rule needs gapless, auditable numbering (invoice or receipt numbers - which are a domain concern, not a primary key). Pick control where the requirement is control; pick generation where the requirement is throughput.

The security angle - a SECOND control, never the gate

Here's where a sequential BIGSERIAL quietly hurts you, and it has nothing to do with the database.

Sequential IDs leak. Two ways:

Enumeration / IDOR. If order 41 exists, so do 40 and 42. An attacker who sees one URL can walk the whole table by adding and subtracting one. If your authorization is weak anywhere, sequential IDs hand out a map of everything to try.
The German-tank problem. In WWII the Allies estimated German tank production from the serial numbers on captured tanks - sequential numbers leak totals. Same math, your business: a competitor signs up, notes their user_id is 50100, waits a week, signs up again as 51200, and now knows you added ~1,100 users that week. Your growth rate, order volume, invoice count - all readable from the increment. That's business intelligence you're printing on every URL.

Snowflake helps because its IDs are not neatly sequential - the low bits move, the timestamp is in milliseconds off a custom epoch, so you can't add one to get the next valid id and you can't subtract two to get a count. Casual enumeration and count-leaking get much, much harder.

But - and I want to be blunt about this - an ID is NEVER a secret, and it is NEVER an authorization check. "Hard to guess" is not "safe." The real control is an ownership/authz check on every single request: does this authenticated caller have the right to touch this object? - enforced server-side, every time, no exceptions. The unguessable-ish id is defense in depth layered on top of that check, not a replacement for it. Anyone who treats a hard-to-guess id as the gate has built an IDOR with extra steps. The Snowflake shape buys you a second wall; it does not excuse a missing first one. (There's a lot more to say on the security side - a later piece can pivot fully to it.)

Next: getting the id out of the building

That's the primary key settled: 64-bit Snowflake, native BIGINT, generated at the edge by a library you didn't write. Inside the trust boundary, that raw int64 is exactly what every service should speak.

The moment it has to leave - into a public URL, a third-party API, a browser - the rules change completely. Raw ints don't go out the door, browsers can't even count that high without corrupting them, and a leaked link needs to be revocable. That's Part 2: hashids at the edge, composite tokens, self-serve salt rotation with an audit tripwire, and why every int64 becomes a string before it touches a browser.

If you build serious backends - Symfony, Go, or the messy space between a monolith and the services growing out of it - follow along on github.com/brilliant-almazov. And if your primary keys are UUIDs right now: what's the index bloat costing you that you haven't measured yet? I'd genuinely like to compare notes.

DEV Community