Why Building a Ledger Is Harder Than It Looks

Tomas Almeida — Thu, 11 Jun 2026 16:54:47 +0000

On August 1, 2012, Knight Capital pushed a faulty deployment to production. Within 45 minutes, its router fired more than 4 million unintended orders into the market, and the firm lost over $460 million (U.S. SEC, 2013). One build, one morning, one company nearly gone.

Most money systems don’t fail like that. They fail quietly: a balance drifting by a cent every few thousand operations, a webhook replay that doubles a payout, a reconciliation job that “almost matches” forever.

We’ve spent years building a double-entry ledger for a crypto payments system. What follows isn’t theory. It’s the set of constraints we ended up needing once the system started seeing real money, retries, and adversarial failure modes.

Why Floating Point Breaks Finance Systems

Floating point isn’t “slightly imprecise.” It’s structurally incompatible with exact accounting.

$1.23 USD    ->  123
1.23 USDC     ->  1230000
0.5 ETH       ->  500000000000000000

The rule we landed on is simple: money is always an integer in the smallest unit of the asset.

Not decimals. Not floats. Not “precision-aware” types that still round under the hood.

We use numeric(38, 0) so fractional units cannot exist at rest. If a half-unit appears in a calculation, it fails before it becomes debt.

The real complexity shows up elsewhere: fees.

A 1% fee on 99,999,999 units produces a remainder that must be deterministically assigned. If you don’t decide where that remainder goes, your ledger will drift. Slowly. Invisibly. Permanently.

Enforcing Double-Entry at the Database Layer

Application-level invariants don’t survive production reality.

Backfills, scripts, incident patches, and “temporary” admin queries bypass them instantly.

So we push the invariant into Postgres.

The rule is strict:

Within a transaction, debits must equal credits per currency.

And it is enforced at commit time, not row time.

CREATE CONSTRAINT TRIGGER postings_must_balance
AFTER INSERT ON postings
DEFERRABLE INITIALLY DEFERRED
FOR EACH ROW
EXECUTE FUNCTION enforce_balanced_postings();

The important part is not the trigger itself. It’s the deferred validation.

Without deferral, you can’t validate a group of rows that doesn’t exist yet.

With it, Postgres becomes a transactional accounting engine.

flowchart TD
  A[Insert postings rows] --> B[Transaction open]
  B --> C[Deferred trigger at commit]
  C --> D{Debits == Credits per currency?}
  D -->|Yes| E[Commit]
  D -->|No| F[Reject transaction]

This is one of those rare cases where “database constraint” is not defensive programming. It is the system boundary.

Idempotency Is Not a Feature, It’s the System Model

Every payment system is at-least-once.

Not “sometimes.” Always.

Stripe explicitly retries webhooks for days and may deliver duplicates by design.

That means your system is not correct if retries are handled at the edge. It must be correct under duplication everywhere.

We treat idempotency as a hard uniqueness rule:

One intent → one transaction, forever.

flowchart TD
  A[Incoming request] --> B{Idempotency key seen?}

  B -->|Yes| C[Return existing transaction]
  B -->|No| D[Insert transaction]

  D --> E{Unique constraint hit?}
  E -->|Yes| C
  E -->|No| F[Create postings]

The real guarantee is the database index:

(ledger_id, idempotency_key) is the source of truth
cache is only a performance optimization
concurrency collapses naturally into a single winner

If two requests race, one wins the insert, the other resolves to it. No locks. No coordination layer. Just relational constraints doing what they were designed for.

Append-Only Is an Audit Model, Not a Storage Choice

The hardest habit to break is mutability.

CRUD thinking says: fix incorrect rows.

Ledger thinking says: never change history, only extend it.

flowchart LR
  A[Original entry] --> B[Posted]
  B --> C{Error discovered}
  C --> D[Reversal entry]
  D --> E[Corrective entry]

This is not aesthetic purity. It’s operational safety.

If you allow updates:

you destroy audit trails
you break reproducibility
you make reconciliation non-deterministic

If you forbid updates:

every correction becomes explicit
every mistake is traceable
every balance is recomputable from first principles

We enforce this at two levels:

application layer rejects updates/deletes
database triggers enforce it as a final gate

The only mutable fields are metadata (tags, references). Money fields are immutable by design.

Exchange Rates Must Be Frozen, Not Recomputed

Any call to “current price” inside a ledger pipeline is a correctness bug.

Not a race condition. A correctness bug.

Rates are captured once:

exchange_rate   decimal(38, 18)
base_currency   text
quote_currency  text

And stored on every affected entry.

flowchart TD
  A[Payment confirmed] --> B[Fetch FX rate]
  B --> C[Write rate into ledger entries]
  C --> D[Downstream systems use snapshot only]

This ensures that:

retries don’t change accounting
reports are deterministic across time
historical balances remain reproducible

A refund six months later uses the original rate, not today’s market rate. That’s not “approximate fairness.” It’s consistency.

Reconciliation Is Where Systems Stop Agreeing With Each Other

Everything before this point ensures internal consistency.

Reconciliation is where reality enters.

And reality is messy.

External systems differ in:

timing
batching
rounding rules
partial settlements
eventual consistency windows

So reconciliation cannot be a simple equality check.

It becomes a matching engine with strategies.

flowchart TD
  A[External records] --> B[Reconciliation engine]

  B --> C[Exact match]
  B --> D[Windowed sum match]
  B --> E[Tolerance-based match]

  C --> F[Mark reconciled]
  D --> F
  E --> F

  B --> G[Unmatched queue for review]

Key design choice: reconciliation writes back into the ledger.

A posting is either:

reconciled
tolerably auto-matched
or explicitly flagged for review

No spreadsheets. No external truth tables.

The ledger is the system of record, not the reconciliation output.

The Minimal Model Behind a Payment Gateway

At runtime, the entire system collapses into four accounts:

graph TD
  A[Crypto Received] --> B[Merchant Payable]
  A --> C[Fee Revenue]
  B --> D[Payout Disbursed]

That simplicity is misleading.

The correctness doesn’t come from the graph.

It comes from everything around it:

integer-only math
deferred constraints
idempotency at the database level
append-only history
frozen exchange rates
reconciliation as a write-back process

Without those, the diagram is just wishful thinking.

With them, it becomes a financial system.

Closing Thought

None of this is exotic. That’s the uncomfortable part.

Each rule is simple in isolation. Every failure mode only appears under production pressure: retries, concurrency, partial failure, and time.

The systems that survive are the ones that assume:

the network will duplicate requests
the database is the only trustworthy boundary
time will invalidate assumptions
humans will eventually need to audit everything

A ledger isn’t a data model.

It’s a collection of enforced invariants that happen to store money.

If you don’t want to build all of this from scratch, there’s also SouthPay Ledger: SouthPay Ledger

DEV Community: Tomas Almeida