DEV Community

Cover image for Twitter Added id_str in 2010 Because JavaScript Can't Hold a Java long
Gabriel Anhaia
Gabriel Anhaia

Posted on AI-assisted

Twitter Added id_str in 2010 Because JavaScript Can't Hold a Java long

#javascript #typescript #kotlin #java

On October 18, 2010, Twitter told every developer on its API that tweet IDs were about to change shape. A new generator called Snowflake would produce 64-bit IDs, and TechCrunch's write-up of the notice put the date plainly: "on November 26, these IDs will go longer than 53 bits." The fix Twitter shipped was blunt. Starting October 22, every ID in a JSON response would come back twice, once as a number and once as a string. In the words of developer advocate Matt Harris, quoted in the same article: "To allow javascript and JSON parsers to read the IDs we need to include a string version of any ID."

That string field is id_str, and it is still in the API. The current X developer docs tell v1.1 clients to "always use id_str", and v2 skipped the number entirely: IDs come back as strings by default.

Sixteen years later, the same bug still ships. A Spring Boot or Ktor service returns a Long ID, a TypeScript client calls JSON.parse, and the last few digits change. No exception. No warning in the console. The client then sends the wrong ID back to the server, and you get a 404 on a record that exists, or a hit on a different record whose ID happens to be the rounded value.

Why Twitter needed 64 bits in the first place

Snowflake was built for Twitter's move from MySQL to Cassandra. The original README is direct about the motive: "there is no sequential id generation facility in Cassandra, nor should there be." The requirements were uncoordinated generation across machines, rough time ordering, and IDs that stay under 64 bits.

The layout it settled on:

  • 41 bits of millisecond timestamp against a custom epoch ("gives us 69 years")
  • 10 bits of machine ID, for up to 1024 machines
  • 12 bits of sequence number, rolling over every 4096 IDs per machine

That is 63 bits of payload, which fits a signed 64-bit integer: a Java long, a Kotlin Long, a Scala Long. The Java Language Specification gives long the range -9223372036854775808 to 9223372036854775807. Every JVM backend can hold a Snowflake ID without thinking about it. The trouble starts when the ID leaves the JVM.

A 64-bit integer bar being squeezed into a shorter 53-slot container, with the extra bits spilling off the end
A JVM long has 64 bits to spend. A JavaScript number has 53 for the integer part.

53 bits of mantissa versus 64 bits of long

A JavaScript number is an IEEE 754 double. MDN's page on Number.MAX_SAFE_INTEGER spells out the consequence: a double "only has 52 bits to represent the mantissa," plus one implicit leading bit, so it can only safely represent integers between -(2^53 - 1) and 2^53 - 1. That ceiling is 9007199254740991.

Past it, a double still exists for every power of two, but not for every integer. Just above 2^53 the gap between neighbouring doubles is 2. Just above 2^60 it is 2^(60-52), which is 256. Modern Snowflake IDs live up there, so a parsed ID snaps to the nearest representable double, and two real IDs generated a moment apart can land on the same one.

You can see it in four lines:

const body = '{"id": 9007199254740993}';
const order = JSON.parse(body) as { id: number };

console.log(order.id);                       // 9007199254740992
console.log(Number.isSafeInteger(order.id)); // false
Enter fullscreen mode Exit fullscreen mode

MDN shows the same collapse from the arithmetic side: Number.MAX_SAFE_INTEGER + 1 === Number.MAX_SAFE_INTEGER + 2 evaluates to true. The X docs use 10765432100123456789, which is past even a Java long's ceiling, and show it printing back as 10765432100123458000.

None of this breaks the JSON spec. RFC 8259, section 6 "allows implementations to set limits on the range and precision of numbers accepted," and names only the integers in [-(2^53)+1, (2^53)-1] as the range where implementations "will agree exactly on their numeric values." Jackson writing 9007199254740993 is valid JSON. JSON.parse rounding it is also valid. The contract between them is the bug.

TypeScript does not help here. JSON.parse returns any, your as { id: number } is a promise you made to the compiler, and the value already satisfies number. The type system checks the shape, and the shape is fine.

Why BigInt doesn't fix JSON.parse

The natural reaction from a JVM developer is: JavaScript has arbitrary-precision integers now, so parse into BigInt. Two things get in the way.

First, JSON.parse never produces a BigInt. Numbers become doubles during parsing, before your code sees anything. The MDN JSON.parse page is explicit that the reviver "is run after the value is parsed," so numbers "will have already been converted to JavaScript numbers, and may lose precision in the process." Writing BigInt(value) inside a classic two-argument reviver gives you an exact BigInt of the wrong number.

Second, the trip back is closed by default. Per MDN's BigInt page, calling JSON.stringify() on any value containing a BigInt throws a TypeError unless you supply a toJSON or a replacer.

So BigInt is a fine type to hold an ID once you have the exact digits. It cannot recover digits the parser already threw away. The digits have to survive the parse, which means either they arrive as a string, or the parser hands you the source text.

Two different ID cards on a conveyor belt merging into a single card as they pass through a funnel
Above 2^53, distinct IDs can parse to the same JavaScript number.

Fix it on the JVM side: make the ID a string in the contract

The fix Twitter chose, and the one Discord documents for its own Snowflake IDs ("they are always returned as strings in the HTTP API"), is to put the ID on the wire as a JSON string. Keep Long in the database and in your domain model. Change only the serialized form.

With Jackson, a per-field annotation does it:

import com.fasterxml.jackson.annotation.JsonFormat;

public record OrderDto(
    @JsonFormat(shape = JsonFormat.Shape.STRING)
    long id,
    long totalCents
) {}
Enter fullscreen mode Exit fullscreen mode

In Jackson 2, the number serializers return ToStringSerializer.instance when the shape is STRING, so id goes out as "9007199254740993" with the quotes. The annotation lives in jackson-annotations, which the Spring team notes kept its package unchanged when Spring Boot 4 moved to Jackson 3 under tools.jackson, so the same line works across that upgrade.

If every Long in your API is an identifier, you can register the serializer by type instead. On Spring Boot 3 with Jackson 2:

@Bean
Jackson2ObjectMapperBuilderCustomizer longsAsStrings() {
    return b -> b
        .serializerByType(Long.class, ToStringSerializer.instance)
        .serializerByType(Long.TYPE, ToStringSerializer.instance);
}
Enter fullscreen mode Exit fullscreen mode

Be careful with that one. It also quotes counts, cents and timestamps that never needed it. Jackson's blunter switch, JsonWriteFeature.WRITE_NUMBERS_AS_STRINGS (off by default), "forces all regular number values to be written as JSON Strings," doubles included. Per field is usually the honest scope.

On Ktor with kotlinx.serialization, the library ships a serializer for exactly this case. The LongAsStringSerializer docs describe it as "intended to be used for interoperability with external clients (mainly JavaScript ones)":

import kotlinx.serialization.Serializable
import kotlinx.serialization.builtins.LongAsStringSerializer

@Serializable
data class OrderDto(
    @Serializable(with = LongAsStringSerializer::class)
    val id: Long,
    val totalCents: Long,
)
Enter fullscreen mode Exit fullscreen mode

Then write it down. If you publish an OpenAPI spec, the ID is type: string, not type: integer, format: int64, so generated TypeScript clients get string too.

Fix it in the TypeScript client

On the client, type the ID as what it is on the wire: a string. A branded type keeps it from mixing with other strings:

type OrderId = string & { readonly __brand: "OrderId" };

interface Order {
  id: OrderId;
  totalCents: number;
}
Enter fullscreen mode Exit fullscreen mode

You compare it, put it in URLs and send it back. You almost never do arithmetic on an ID, so a string loses nothing. If you need ordering or math, convert with BigInt(order.id) at that moment, from exact digits.

Sometimes you don't own the server and it still sends bare numbers. For that case, TC39 added source text access to JSON.parse; the proposal is at stage 4. For primitive values, the reviver gets a third context argument whose source property is "the original JSON string representing this value" (MDN). MDN's compatibility data lists it in Chrome 114, Firefox 135, Safari 18.4, Node.js 21, Deno 1.33 and Bun 1.1.43.

type Ctx = { source?: string };

function idsAsStrings(key: string, value: unknown, ctx?: Ctx) {
  if (key !== "id" || typeof value !== "number") return value;
  if (ctx?.source === undefined) {
    throw new Error("JSON.parse source access not supported");
  }
  return ctx.source; // the exact digits, before rounding
}

const order = JSON.parse(text, idsAsStrings) as Order;
Enter fullscreen mode Exit fullscreen mode

The reviver throws when the runtime doesn't pass context, because falling back to String(value) would hand you the rounded digits as a string, which is the original bug wearing a disguise. And you need the raw text to call it: Response.json() takes no reviver, so read the body with await res.text() and parse it yourself.

The same proposal added JSON.rawJSON for the outbound direction. A replacer like typeof v === "bigint" ? JSON.rawJSON(v.toString()) : v lets JSON.stringify emit a BigInt as a bare JSON number without precision loss, when a server insists on numeric IDs in request bodies.

If you own the server, the one-line @JsonFormat change is the whole fix, and it costs less to ship before your ID generator crosses 2^53 than after the first 404 on a record that exists.

If this was useful

This bug lives exactly on the seam between the JVM and TypeScript: a Long that is perfectly safe on one side and quietly lossy on the other. Kotlin and Java to TypeScript is the book I wrote for that seam, mapping what you already know from Java and Kotlin (variance, null safety, sealed hierarchies, coroutines) onto how TypeScript does the same jobs.

It's part of The TypeScript Library, five books with a clear path through them. Books 1 and 2 are the core path. If you come from the JVM, book 3 stands in for book 1. If you ship TypeScript at work, book 5 is the production layer:

All five books ship in ebook, paperback, and hardcover.

The TypeScript Library — the 5-book collection

Top comments (1)

Collapse
 
sgaggjhkjh profile image
Arjun Sharma •

The JSON.parse detail is the part that keeps biting people: precision is already lost before the reviver runs, so even a BigInt-aware reviver only recovers an exact copy of the wrong number. Serializing IDs as strings at the boundary remains the only fix that actually works.