Design a Payment System: the system design interview walkthrough

Design a Payment System: the system design interview walkthrough

The request to design a payment system flips the usual interview instinct on its head. Most "design X" rounds are about scale — more requests, more data, more machines. A payment backend is about correctness: it may only process a few hundred payments a second, but every single one is money, and a bug does not drop a tweet — it double-charges a customer or loses track of a dollar. That is why the system design interview loves this question: it tests whether you reach for consistency, idempotency, and auditability instead of caches and shards.

The round turns on two decisions. First, how you guarantee that one logical payment causes exactly one charge even when the client retries — the answer is an idempotency key. Second, how you keep the money provably accounted for — the answer is a double-entry ledger whose debits and credits must always balance. This walkthrough follows the path a strong candidate draws on the whiteboard — requirements, estimate, API and data model, high-level design, then the deep dives — and at each step it names the follow-up the interviewer uses to tell a memorised answer from an engineer who understands the tradeoff.

How the payment system round is actually run

A payment system round runs about 60 minutes on a shared canvas — you draw the boxes and arrows while you talk, and the interviewer interrupts. It is a favourite because the naive design (take card, charge it, mark the order paid) hides every hard question: what if the charge succeeds but your database write fails, what if the customer clicks twice, how do you ever prove the books are right. Get those clean and you have shown the whole loop in one problem.

The interviewer rarely lets the happy path stand. "The customer taps Pay, the response times out, and they tap again. Walk me through both requests — how many times is the card charged?" That question — exactly-once under retries — is the one a copied answer can never rehearse, and the one this round is really testing.

The rounds here are modelled on how the loop runs, not scraped from any candidate's recall. What matters for prep is the shape: a blank canvas, a clock, and a follow-up after every decision you make.

The five-step framework for any HLD round

Every high-level design answer fits the same spine. Say it out loud at the start so the interviewer knows you have a plan, then walk it:

1. Clarify requirements — functional and non-functional, and what you are explicitly not building.
2. Estimate — volume and ledger growth, to justify every later choice.
3. High-level design — the boxes and arrows: client, payment service, processor, ledger.
4. Deep dive — the two or three decisions that actually decide this system.
5. Follow-ups — failure modes, consistency, and how you prove correctness.

What the debrief measures

The score is not "right or wrong design." A good HLD debrief reads on four dimensions, and every move below maps to one: problem solving (did you see this as a correctness problem and design for exactly-once), communication (did you state the idempotency and double-entry rules in a sentence each), technical execution (atomic payment-plus-idempotency write, balanced ledger entries, idempotent webhook handling), and time management (did you spend your 60 minutes on money-correctness instead of the checkout form). Reading a walkthrough teaches the design; it does not teach you to deliver it on the clock while someone pushes back.

Step 1 — Requirements: what a payment system must do

Spend the first few minutes here. Jumping to boxes before you have scoped the problem is the fastest way to design the wrong system.

Type	Requirement
Functional	Accept a payment (charge a card via a processor); record every money movement; support refunds; reconcile internal records against the processor.
Functional (optional)	Payouts to merchants, multi-currency, holds/captures, fraud checks — call these out, then scope most of them for later.
Non-functional	Correctness above all — exactly one charge per logical payment; the books are always balanced and auditable; strong consistency over raw throughput; and, for compliance, we never store raw card numbers — a third-party processor does.

The sentence that earns points: "This is a correctness and consistency problem, not a scale problem — so I'll delegate the actual card movement to a payment processor like Stripe, keep an immutable internal ledger as the source of truth, and make every operation idempotent so retries can't corrupt the money." Say that and you have framed the entire design — and named all three deep dives — before drawing a box.

Step 2 — Capacity estimation (back of the envelope)

Estimation here proves the opposite of most rounds: the numbers are small, so correctness, not capacity, is the whole game.

Quantity	Assumption	Result
Payments	~10M/day	~115 payments/sec — modest
Ledger entries	2+ per payment (double-entry)	~20M+ entries/day, append-only
Peak	10× during sales events	~1,150/sec — still small
Retention	years, for audit & disputes	storage grows but is not the bottleneck

Two conclusions fall out, and you should say both out loud. First, the throughput is low by system-design standards — a single well-designed service and database can handle it — so you should not waste the round on sharding theatre. Second, because every record is money and must survive forever for audit, the design is dominated by durability, consistency, and an append-only ledger. You have now pointed the interview at correctness — where it belongs — before anyone asked.

Step 3 — API and data model

The API is small; what matters is that the charge endpoint takes an idempotency key so retries are safe.

Endpoint	Purpose
POST /payments + Idempotency-Key header	Create a payment; safe to retry — same key returns the same result.
GET /payments/{id}	Fetch a payment and its status.
POST /payments/{id}/refund	Refund a captured payment (itself idempotent).

The data model has three load-bearing pieces: the payment, the idempotency record that de-dupes retries, and the immutable ledger entries that account for the money:

Entity	Notes
payment	payment_id, amount, currency, status (pending → captured / failed / refunded), provider_charge_id.
idempotency_key	(caller_id, key) unique; stores request_hash, response, status.
ledger_entry	Append-only; account, debit/credit, amount, payment_id, created_at — never updated in place.

Step 4 — High-level design: payment service, processor, and ledger

Now the boxes. A client calls the payment service, which orchestrates three things and owns the money-correctness guarantees:

• Idempotency store: before doing anything, the service checks the (caller_id, key) record — a repeat of an in-flight or completed request returns the stored response instead of charging again.
• Payment processor (third-party, e.g. Stripe): the service sends a tokenized card (never a raw card number) to the processor, which moves the money and returns a provider_charge_id. This keeps the system out of PCI scope for card data.
• Double-entry ledger: every state change writes balanced debit and credit entries to an append-only journal — the internal source of truth for what happened.

Two more pieces make it robust: the processor confirms asynchronously via a webhook (consumed through a queue, idempotently), and a periodic reconciliation job compares the ledger against the processor's settlement file. Say explicitly that the ledger — not the processor — is your source of truth, and that you reconcile the two rather than trusting either blindly.

Deep dive: idempotency — never charge twice

This is the first decision that actually decides the round. Networks time out, users double-click, and clients retry — and a payment must cause exactly one charge regardless.

The mechanism is an idempotency key: the client generates a unique key per logical payment and sends it with the request. The payment service stores it as (caller_id, idempotency_key) -> request_hash, response, status and uses it as a gate:

• New key: process the payment, call the processor, and store the key with the result — all in one atomic transaction with the payment record, so the key and the charge commit together or not at all.
• Same key, same payload (a retry): return the stored response without calling the processor again. The second tap gets the same payment_id, not a second charge.
• Same key, different payload: reject with 409 Conflict — the key is being reused for a different request, which is a client bug you must not paper over.

def create_payment(caller_id, key, request):
    existing = store.get(caller_id, key)
    if existing:
        if existing.request_hash != hash(request):
            raise Conflict(409)          # same key, different payload
        return existing.response         # safe retry -> original result
    # atomic: write idempotency record + payment in one transaction
    resp = charge_via_processor(request) # tokenized card -> provider_charge_id
    store.put(caller_id, key, hash(request), resp, status="captured")
    return resp

"They tapped Pay twice — how many charges?" One. The first request stores the idempotency key with its result in the same transaction as the charge; the second request finds that key and returns the stored response without touching the processor. Naming the atomic write of key-plus-payment is the senior move.

System Design
HLD · Payment System · 60 min
Design a payment system out loud, with follow-ups

Practise on Zynter →

Deep dive: the double-entry ledger

This is where you prove the money is right. The naive approach — a mutable balance column you add to and subtract from — cannot answer "why is this number what it is?" and quietly corrupts under concurrency. The interview answer is a double-entry ledger.

Every money movement is recorded as balanced entries: at least one debit and one credit of equal total, so that for every transaction ∑ debits = ∑ credits. A captured payment might debit customer_pending and credit merchant_receivable; settlement later moves merchant_receivable to merchant_settled. Three properties make it interview-grade:

• Immutable / append-only. You never edit an entry. To correct a mistake you append a reversal (an equal, opposite entry) — so the full history, including the error, stays auditable months later during a dispute.
• Balances are materialized views. The real source of truth is the stream of entries; an account's balance is derived by summing them (and can be cached/snapshotted). Never let a mutable balance be the primary record.
• The invariant is testable. Because ∑ debits = ∑ credits must always hold, you can continuously assert it — if it ever breaks, you have a bug alarm before a customer does.

"How do you know your numbers are right?" The ledger is double-entry and append-only, so every movement is two balanced halves and the invariant ∑debits = ∑credits holds by construction; balances are derived from entries, and corrections are reversals, never edits. That is the answer that separates a payments engineer from someone who reached for a balance column.

System Design
HLD · Ledger & correctness · 60 min
Defend a balanced ledger out loud

Practise on Zynter →

Deep dive: reconciliation and duplicate webhooks

Your ledger is your truth, but the processor has its own — and the two drift. Two mechanisms keep them honest.

Idempotent webhook consumers. The processor confirms the real outcome of a charge asynchronously, via a webhook, and that webhook can arrive more than once (at-least-once delivery) or out of order. So the consumer must be idempotent: dedupe on the provider_charge_id (or the event id) with a unique constraint, and use an outbox pattern so your own events are published exactly once alongside the ledger write. Processing the same webhook twice must change nothing the second time.

Reconciliation against the settlement file. On a schedule, the system pulls the processor's settlement file and matches each line against the ledger on stable keys — provider_charge_id, amount, currency, merchant, settlement date. Mismatches are expected and must be handled, not ignored: processor fees, partial captures, chargebacks, currency rounding, and timing gaps where a payment straddles the cutoff. Anything that still does not match is flagged for investigation. This is how you catch the case the interviewer loves: your system thinks a payment captured, but the processor shows it failed.

"Your system says captured, the processor says failed — how do you ever catch that?" Reconciliation: a scheduled job matches the ledger against the processor's settlement file on provider_charge_id, amount and date, and flags the discrepancy for correction via a reversal entry — you never just trust the internal record. Saying that unprompted is the answer they want.

The follow-ups that decide the payment system round

Scan the probes from each deep dive and they rhyme. Master these five and you are ready for a version of the question that is not drawn exactly like this one.

• "They tapped Pay twice — how many charges?" → One; the idempotency key is stored atomically with the charge and a retry returns the stored response. (technical execution)
• "Same key, different amount — now what?" → 409 Conflict; a reused key with a changed payload is a client bug, not a second payment. (problem solving)
• "How do you know your numbers are right?" → A double-entry, append-only ledger where ∑debits = ∑credits; balances are derived, corrections are reversals. (communication)
• "The processor webhook arrives twice — what happens?" → Nothing the second time; consumers dedupe on provider_charge_id with a unique constraint and an outbox. (technical execution)
• "Internal says captured, processor says failed — how do you catch it?" → Reconciliation against the settlement file, matched on stable keys, with mismatches flagged. (problem solving)

Each maps to a debrief dimension. Reading this list teaches you the answers; it does not teach you to deliver them while someone interrupts — that only comes from saying them out loud.

How to prepare: a one-week plan

You do not need fifty systems. You need this one cold and the framework automatic, so the next "design X" is just new nouns in the same five steps.

• Days 1–2: Learn the five-step framework until it is reflex. Do the payment system end to end on paper — requirements, estimate, API, data model, the service-processor-ledger boxes.
• Days 3–4: Drill idempotency: the (caller_id, key) record, same-key-same-payload vs 409, the atomic write. Write the create-payment function from memory.
• Days 5–6: Own the ledger and reconciliation — balanced entries, the ∑debits=∑credits invariant, reversals, idempotent webhooks, settlement-file matching. Answer each of the five follow-ups out loud.
• Day 7: A full timed 60-minute round, narrating the whole way, with someone (or something) interrupting. This is the rep that counts.

This post is the deep walkthrough behind one row of the broader Top 25 system design interview questions — once the payment system is automatic, the same spine carries the rest of the list.

Frequently asked questions

How do I design a payment system step by step?

Follow the five-step HLD framework. Clarify requirements (charge, refund, record every movement, reconcile, never store raw cards, correctness over throughput); estimate to show the volume is modest so consistency — not scale — is the point; define POST /payments with an idempotency key plus a thin model of payment, idempotency record, and append-only ledger entries; draw the high-level design (client → payment service → idempotency store + tokenized processor + double-entry ledger, with async webhooks and a reconciliation job); then deep-dive the decisions that matter — idempotency, the ledger, and reconciliation. Finish on the follow-ups about retries and mismatches.

How do you prevent double charging in a payment system?

With an idempotency key. The client sends a unique key per logical payment; the service stores it as (caller_id, key) -> request_hash, response, status and writes that record in the same transaction as the charge. A retry with the same key and payload returns the stored response instead of charging again, so a double-click or timeout retry produces exactly one charge. If the same key arrives with a different payload, the service rejects it with 409 Conflict because that indicates a client bug rather than a legitimate second payment.

What is a double-entry ledger and why use one?

A double-entry ledger records every money movement as balanced entries — at least one debit and one credit of equal total — so the invariant ∑ debits = ∑ credits always holds. It is append-only: you never edit an entry, and you correct mistakes by appending a reversal, which keeps the full history auditable for disputes months later. Account balances are materialized views derived by summing entries, not a mutable column. This design makes the books provably correct and is the expected answer in a payment system interview.

How does reconciliation work in a payment system?

Reconciliation compares your internal ledger against the payment processor's settlement file on a schedule. Each settlement line is matched to a ledger record on stable keys — provider_charge_id, amount, currency, merchant, and settlement date. Differences from fees, partial captures, chargebacks, currency rounding, and timing gaps are expected and handled explicitly; anything left unmatched is flagged for investigation and corrected with a reversal entry. Reconciliation is how you catch cases where the internal state and the processor disagree, instead of blindly trusting either side.

How do you handle duplicate payment webhooks?

Make the webhook consumer idempotent. Processor callbacks are delivered at least once, so the same event can arrive multiple times or out of order. Dedupe on a stable identifier such as the provider_charge_id or event id using a unique constraint, so processing the same webhook twice changes nothing the second time. Pair this with an outbox pattern so your own downstream events publish exactly once alongside the ledger write, and make every handler replay-safe.

How do I practise a system design interview out loud?

Reading a walkthrough is not the same as defending a design while someone interrupts. On Zynter, the HLD track runs live voice rounds modelled on how real system design loops run — you talk through a shared design on the canvas, and a voice interviewer pushes with the follow-ups above one at a time, then gives you a debrief on problem solving, communication, technical execution, and time management. The score is a practice signal, not a hiring decision.

Practise it on Zynter

You can read every rule. The round is won on the retry and the mismatch.

Pick the Payment System round, draw the service-processor-ledger design on the canvas, and defend it out loud to a voice interviewer that keeps asking "they tapped Pay twice — how many charges?" and "how do you know your numbers are right?" — then read the debrief. Zynter.ai is on-demand mock interviews modelled on real company loops, with 150+ rounds across 20 companies including the HLD system design track (Sep 2026).

Practise the Payment System round →
Browse all interview tracks