Back to Code | Ep 06: State Hell and Idempotency

#programming #ai #distributedsystems #typescript

The 15-week technical battle of LogiFlow — a company waking up from the illusion created by artificial intelligence and returning to real engineering.

The Story

A network fluctuation hit the payment gateway. The system received a "Timeout" and retried the operation. The customer's card was charged twice. AI, when writing HTTP requests, had no concept of "Idempotency."

The customer support team noticed it first. Then the finance team. Then legal. A single network hiccup had turned into a compliance incident, and the root cause was embarrassingly simple: the retry logic had no deduplication mechanism.

Technical Autopsy: AI Only Sees the Happy Path

// Dangerous: No idempotency
async function chargeCustomer(
  orderId: string, amount: number
) {
  await stripe.charges.create({
    amount, currency: 'usd'
  });
}

In AI's world, networks are reliable. Requests always succeed on the first try. Timeouts don't exist. And if they do, you just retry — what could go wrong?

Everything. Everything could go wrong.

When the payment gateway timed out, the charge had actually succeeded on Stripe's side. The timeout was on the response, not the request. The retry created a second, duplicate charge.

The Human Fix: Managing Distributed Chaos

// Safe: With idempotency key
async function chargeCustomer(
  orderId: string,
  amount: number,
  idempotencyKey: string
) {
  // Retries with the same key are rejected
  await stripe.charges.create(
    { amount, currency: 'usd' },
    { idempotencyKey }
  );
}

Fallacies of Distributed Computing: The network is unreliable. It delays, disconnects, sends duplicate packets. Async operations written without idempotency keys are financial disasters.

The Eight Fallacies

Peter Deutsch's eight fallacies of distributed computing, written in 1994, remain as true today as ever:

The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.
Topology doesn't change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.

AI assumes all eight. Humans know better.

Lessons from Episode 6

1. Idempotency: Every payment, every order confirmation, every inventory deduction must be idempotent.

2. Retry Strategies: Without exponential backoff + jitter + dead-letter queue, async systems collapse.

3. Event Sourcing: Storing events instead of state preserves the source of truth in distributed systems.

This is Episode 6 of the "Back to Code" series. Next up: Episode 7 — The Lies of Mocks: Reality with Testcontainers.

Series: back.to.code · 2026

DEV Community