If you run a distributed system in production, you are likely aware of the primary issue of ID generation: guaranteeing uniqueness across a distributed system, especially when low-latency is required.
Universally Unique Identifier (UUID)
UUID is a standard for ID generation. A UUIDv4 consists of 128 random bits, with a few altered according to the specification. This 'ensures' uniqueness by simply making it incredibly improbable to generate the same UUID twice (0.0000000000000004 to generate one clash per 2^36 ids). Google provides a go UUID generator here.
Caveats
- If your random number generator does not produce random enough numbers, it may have a negative effect on the uniqueness of UUIDS
- 128 bits is large and unwieldy
ID generation service
It is, of course, possible to create a single service responsible for ID allocation - one which guarantees uniqueness, by incrementing an integer.
Caveats
- If you are running a highly distributed system with servers which are physically distant, there will be an impact on latency.
- Single point of failure
Snowflakes
Snowflakes were initially introduced by Twitter - they are 64-bit integers comprised of the time (in ms), a node id and a counter. That means, that each node/machine is allowed to generate its own IDs, which will still be unique. I recommend this library because it allows the customization of how many bits are allocated to the timestamp, node id, and counter and supports a custom epoch so it can be customized to your exact use case (e.g. if you need to generate more IDs per second per node, you can increase the size of the counter). It achieves a performance of < 35 ns/id - allowing the generation of aprox. 28 thousand Snowflakes per millisecond (28 million per second) per node.
Caveats
- Some languages (such as JavaScript) do not support 64 bit integers, so must interpret ids as strings
Pros
- Suitable for highly distributed systems
- Only 8 bytes!
- Sortable by creation time 🕒
- Flexible bit allocations (depending on library)
full disclosure: I am, at the time of writing, the primary contributor to github.com/ocuppi/snowflake
Top comments (0)