DEV Community

Taylor Wang
Taylor Wang

Posted on

The Webhook Updated the Wrong Row. The Logs Showed the Perfect ID.

I ran a small payment-sync service on a free server, and for two weeks it quietly marked the wrong orders as paid before I noticed. The symptom was infuriating because every log line looked flawless: the right event type, the right timestamp, and an order ID that matched the provider dashboard exactly. Yet the database row that changed belonged to a different order, and only a handful of events were affected. Why would the same code path work for most payloads and silently misfire on a few?

This is the story of that bug, the one-digit difference behind it, and the debugging habits that finally exposed it. If your service ingests JSON from any external system, the lesson applies to you too.

The symptom

The service receives payment events from an external provider and marks the matching order as paid. The flow looks innocent enough: parse the JSON body, look up the order by external_id, update its status, and log the result.

const event = JSON.parse(rawBody);
const order = await db.orders.findByExternalId(event.order_id);
await db.orders.update(order.id, { status: "paid" });
Enter fullscreen mode Exit fullscreen mode

The first clue came from support, not from my dashboards. A customer reported that their order was marked paid, but a different order number appeared on the invoice. I pulled the event from the provider dashboard, found the matching log line, and confirmed the ID looked right. The code path was trivial, the input appeared correct, and the output was still wrong. How could that even happen?

Hypothesis one: a race condition

My first instinct was concurrency. Two webhooks for the same merchant arriving in parallel could both read the same fallback row and update it twice. I added a unique constraint, replayed the exact payloads, and watched the bug refuse to reproduce with my local fixtures. That refusal was the real clue, and I almost missed it.

The clue: the size of the ID

Every affected event had an order_id larger than nine quadrillion, and that number should have triggered something immediately. JavaScript numbers are only safe up to Number.MAX_SAFE_INTEGER, which is 9007199254740991 — about nine quadrillion. Any integer above that loses precision the moment JSON.parse converts it to a Number.

Here is the minimal reproduction that finally made it obvious:

const raw = '{"order_id": 9223372036854775807}';
const parsed = JSON.parse(raw).order_id;
console.log(parsed);                    // 9223372036854775808
console.log(Number.isSafeInteger(parsed)); // false
Enter fullscreen mode Exit fullscreen mode

The last digit silently changed from 7 to 8. My log line printed the rounded value, so it looked correct, and the database lookup used the rounded value too. The lookup found a different order whose external ID was exactly 9223372036854775808 — a row created earlier by a test import — and the update landed there. The logs were perfect because the logs were lying in a very subtle way.

You can reproduce the same surprise in one line without any framework: node -e 'console.log(JSON.parse("9223372036854775807"))' prints 9223372036854775808. Once I saw that, the whole incident rearranged itself.

Why the free server exposed it

Locally, my fixtures used small IDs, so every test passed and every replay succeeded. The free server was the first place the service saw real traffic, and real payment providers hand out 64-bit integer IDs that routinely exceed the safe range. The shared database on that server also contained adjacent IDs from previous imports, which is exactly the condition that turns a rounding error into a wrong-row update instead of a clean "not found."

That combination — real payloads plus a database full of lookalike IDs — is why the bug only appeared in production. If the lookup had returned nothing, the failure would have been loud and easy to trace. Instead, it returned the wrong row, and the logs had no error to report.

The debugging workflow I should have used from day one

Looking back, three habits would have found this in minutes instead of days.

  1. Log the raw payload alongside the parsed object. Comparing rawBody with the parsed fields immediately shows whether parsing changed anything. I had only logged the parsed object, so the rounding was invisible.

  2. Check Number.isSafeInteger on every external ID. A one-line validation guard turns a silent corruption into a loud, actionable error.

  3. Reproduce with the exact production payload, not a fixture. The moment I pasted the real event body into a standalone script, the bug appeared. My test data was as much the problem as the code.

The fix

The durable fix is to stop parsing external identifiers as JavaScript numbers entirely. Since I could not change the provider's format, I switched to a lossless JSON parser that keeps large integers as strings.

const JSONBigInt = require("json-bigint");

const event = JSONBigInt({ storeAsString: true }).parse(rawBody);
// event.order_id === "9223372036854775807" — no rounding
const order = await db.orders.findByExternalId(event.order_id);
Enter fullscreen mode Exit fullscreen mode

I also added a validation step that rejects any payload containing an unsafe integer, because silently rounding data you did not create is never acceptable.

function assertSafeIds(event) {
  for (const [key, value] of Object.entries(event)) {
    if (key.endsWith("_id") && typeof value === "number" && !Number.isSafeInteger(value)) {
      throw new Error(`Unsafe integer in ${key}: ${value}`);
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

After deploying, I replayed the affected events from the provider's dashboard, and every single one updated the correct row. The invoice bug disappeared, and the validation guard has been silent ever since.

How a free model helped me see it

I stared at those logs for a full day before I asked a free model to look. I pasted the raw payload and the parsed output into MonkeyCode's free model access, and it pointed out that the difference between the two values was exactly one unit in the last place — the signature of IEEE-754 rounding, not a race condition. It also generated the two-line reproduction above, which turned my vague suspicion into a test I could run.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free model did not find the bug for me; it gave me the right vocabulary to search for it. That distinction matters, because the real skill is knowing that a 64-bit integer cannot survive a JavaScript Number, and no model can replace the discipline of logging raw input.

Limitations and who should not use this approach

This fix assumes you control the consumer and can change how IDs are parsed. If you are stuck with a legacy schema that stores external IDs as integers, the string conversion will not help — you need to fix the schema or migrate the column first.

The lossless parser also has a cost: every large integer becomes a string, so any code that expects a number for arithmetic or comparisons needs a deliberate conversion. And the validation guard will reject legitimate payloads if the provider ever sends a genuinely unsafe integer, which is correct behavior but requires an alerting path so the rejection is never silent.

If your service only handles small IDs, this entire article does not apply to you. The moment an external system can grow past nine quadrillion, though, you have a time bomb, and the logs will not tell you when it goes off.

The takeaway

The most dangerous bugs are the ones where every log line looks correct, because the corruption happens before your code ever runs. When a webhook updates the wrong row and the logs show the right ID, stop staring at the business logic and look at the boundary where the data enters your system. Parse it, print it, and compare it with the raw bytes. The difference will be one digit, and that digit is the whole story.

Top comments (0)