The lost-update bug that quietly corrupts homegrown wallet balances — and the five disciplines we used across PayWithToken to make money movement correct under concurrency.
There is a bug that lives in a large share of the world's homegrown wallet systems. It doesn't throw an error. It doesn't show up in tests. It surfaces months later as a balance that is quietly, inexplicably wrong — and in a payments system, a wrong balance is either a customer who has lost money or a company that has given it away. This is the story of that bug, why the "obvious" wallet code causes it, and the handful of disciplines we used across PayWithToken to make money movement correct under concurrency.
The bug: lost updates
Here is wallet code almost everyone writes first. Credit a user's balance:
// DON'T do this
$row = $db->query("SELECT balance FROM users WHERE id = $id")->fetch();
$new = $row['balance'] + $amount;
$db->exec("UPDATE users SET balance = $new WHERE id = $id");
Read the balance, add to it in PHP, write it back. It works perfectly — until two things happen at the same time.
Picture a wallet at ₦1,000. Two credits of ₦500 arrive simultaneously — say a bank webhook and the user tapping "confirm" on their phone:
Request A reads balance = 1000.
Request B reads balance = 1000 (A hasn't written yet).
A computes 1500, writes 1500.
B computes 1500, writes 1500.
Two credits landed; the balance rose by ₦500. ₦500 vanished. This is a lost update, and it is a race condition, which means it is invisible until you have real concurrent traffic — exactly when you can least afford it. The debit version of the same bug lets a balance go negative or double-spends a token.
Fix #1: let the database do the arithmetic
The read-modify-write happened in PHP, across three round trips, with a gap where another request could interleave. The fix is to make the update a single atomic statement and let the database's row lock serialise it:
// DO this — one atomic statement
$db->prepare("UPDATE users SET balance = balance + :amt WHERE id = :id")
->execute([':amt' => $amount, ':id' => $id]);
Now the addition happens inside the database, under the row lock it already takes for an UPDATE. Concurrent credits queue behind each other and both land. There is no window to interleave because there is no gap between read and write — they are the same operation. In PayWithToken we converted every wallet credit and debit site — 21 of them — to this pattern. Consistency matters more than cleverness: one read-modify-write left in the codebase is enough to corrupt a balance.
Fix #2: store money as DECIMAL, never FLOAT
The second silent corruption is the column type. Floating-point numbers cannot represent most decimal fractions exactly:
0.1 + 0.2 → 0.30000000000000004
Store balances as FLOAT/DOUBLE and those tiny errors accumulate across millions of operations until statements no longer reconcile to the penny. Money must be exact, so money columns are DECIMAL:
ALTER TABLE users MODIFY balance DECIMAL(18,2) NOT NULL DEFAULT 0.00;
DECIMAL(18,2) stores an exact base-10 number with two fractional digits and headroom for very large totals. Combined with atomic updates, arithmetic is now both race-free and exact.
Fix #3: guard the debit
A credit can always succeed; a debit must not overdraw. The guard belongs in the same atomic statement, not in a separate SELECT:
$stmt = $db->prepare(
"UPDATE users SET balance = balance - :amt
WHERE id = :id AND balance >= :amt");
$stmt->execute([':amt' => $amount, ':id' => $id]);
if ($stmt->rowCount() === 0) {
// No row updated → insufficient funds (or wrong id). The debit did NOT happen.
throw new InsufficientFunds();
}
The WHERE balance >= :amt makes the check and the deduction one indivisible act. You then trust rowCount(): if zero rows changed, the money did not move. A separate "check balance, then debit" would reopen exactly the race we just closed.
Fix #4: credit at confirm time, keyed to a specific payment
Atomicity stops corruption. It does not stop duplication. In a real payments system the same credit can be triggered from several places — a bank webhook, a scheduled reconciliation job, a user pressing "confirm." If each one credits the wallet, the user is paid three times for one deposit.
The naive guard — "have we credited this user today?" — is wrong, because a user can legitimately deposit twice in a day. The credit must be tied to the specific payment artifact: a bank reference, a transaction id. Record that a given reference has been settled, and make claiming it a one-time, atomic act:
// Idempotent: only the FIRST caller for this bank reference wins
$claimed = $db->prepare(
"INSERT IGNORE INTO settled_refs (bank_ref) VALUES (:ref)");
$claimed->execute([':ref' => $ref]);
if ($claimed->rowCount() === 1) {
// We are the first — safe to credit exactly once
creditWallet($userId, $amount);
}
// rowCount() === 0 → already settled → do nothing
A UNIQUE constraint on bank_ref plus INSERT IGNORE turns "have we already handled this payment?" into a single race-free question. Webhook, cron and manual reconciliation can now all run — even simultaneously — and the wallet is credited exactly once. We applied the same "execute-once claim" to dispute resolutions, so a refund can never be paid twice.
Fix #5: make the whole movement transactional
When money moves between two places — debit one wallet, credit another, write a ledger row — those writes must all commit or all roll back. A crash between the debit and the credit must not leave money destroyed:
$db->beginTransaction();
try {
debit($sender, $amount); // atomic, guarded
credit($receiver, $amount); // atomic
writeLedger($sender, $receiver, $amount);
$db->commit();
} catch (Throwable $e) {
$db->rollBack(); // nothing moved
throw $e;
}
The disciplines, distilled
None of this is exotic. It is five disciplines applied without exception:
Never read-modify-write a balance in application code. Let the database add and subtract in one statement.
Store money as DECIMAL, never floating point.
Guard debits in the WHERE clause and trust the affected-row count.
Make credits idempotent by keying them to a specific payment reference, claimed exactly once.
Wrap multi-step movements in a transaction so they are all-or-nothing.
The reason to be absolute about it is that concurrency bugs don't fail loudly. They fail as a slow leak of trust — a balance that's a little off, a reconciliation that won't tie out, a customer who swears they were charged twice. In payments, "usually correct" is another way of saying "wrong." Money-grade software earns its name by being correct every time two things happen at once — and that is a property you design in deliberately, one atomic statement at a time.
Top comments (0)