DEV Community: Muhammad Abdullah

A future revision should not change yesterday's data selection

Muhammad Abdullah — Sat, 24 Oct 2026 21:47:28 +0000

An as-of join can run successfully and still answer the wrong historical question. The important timestamp is when a particular version became available, not just the period the value describes.

Suppose a synthetic September fact is published on October 3 with value 100, then revised on October 7 to 80. For an October 4 decision, the eligible value is 100. A pipeline that joins today's latest revision to every earlier date silently returns 80 instead.

There is a useful regression property here: adding a future revision must not change a past selection.

from datetime import datetime

def available(value):
    result = datetime.fromisoformat(value)
    if result.tzinfo is None:
        raise ValueError("Explicit timezone required")
    return result

def select(rows, entity, period, decision):
    cutoff = available(decision)
    eligible = [r for r in rows
                if r["entity"] == entity and r["period"] == period
                and available(r["available_at"]) <= cutoff]
    if not eligible:
        return None
    stamps = [available(r["available_at"]) for r in eligible]
    if len(stamps) != len(set(stamps)):
        raise ValueError("Ambiguous revision order")
    return max(eligible, key=lambda r: available(r["available_at"]))["value"]

original = [{"entity": "DEMO", "period": "2026-09-30",
             "available_at": "2026-10-03T09:00:00+00:00", "value": 100}]
revised = original + [{"entity": "DEMO", "period": "2026-09-30",
                      "available_at": "2026-10-07T09:00:00+00:00", "value": 80}]
decision = "2026-10-04T12:00:00+00:00"

assert select(original, "DEMO", "2026-09-30", decision) == 100
assert select(revised, "DEMO", "2026-09-30", decision) == 100
assert select(revised, "DEMO", "2026-09-30", "2026-10-08T12:00:00+00:00") == 80
assert select(revised, "DEMO", "2026-09-30", "2026-10-01T12:00:00+00:00") is None
print("All four synthetic selection checks passed")

This is a deliberately small selection example. It isn't a trading strategy, a performance result or a complete financial data model. Equal-time revisions are rejected because file order isn't an evidence-based tie-breaker.

The availability contract still needs review. Replaying what a live application knew requires actual receipt history as well as publication time. A hypothetical research archive can use declared delivery-lag assumptions, but that does not reconstruct your own system's history.

In a larger pipeline, also check the entity and fact keys, timezone precision, missing observations and revision identifiers. A backward join does not repair unreliable timestamps, retrospective universe membership, leaked feature transforms or a model trained after the historical cutoff.

The useful engineering question is whether you can preserve and test these boundaries, rather than whether a chart looks convincing.

I’m affiliated with StackEndBox. This adaptation uses synthetic examples from our tested point-in-time data guide. It contains no client data or claimed customer outcome. Prepared with AI assistance, with the displayed code executed and checked.

Before retrying a failed integration, ask whether the write already happened

Muhammad Abdullah — Sat, 24 Oct 2026 20:45:29 +0000

A timeout tells your application that it didn't receive a result in time. It doesn't establish that the receiving service rejected the operation. That distinction matters when an integration creates a contact, opens a support request or performs another consequential write.

This is an illustrative design discussion, not a report about a customer incident or a tested implementation for a specific vendor.

Start with a business operation reference

Give the intended action a stable reference before attempting delivery. Keep that reference when the browser refreshes or the worker restarts. A newly generated reference on every retry makes it harder to recognize that two requests represent the same intended action.

Separate what you know

The operation can be prepared but not attempted, conclusively rejected, confirmed by the receiving service, or uncertain after an attempted write. An HTTP timeout belongs in that last category until further evidence resolves it. A generic failed label loses the distinction that tells an operator whether another write is safe.

Use the provider's actual recovery capability

Some providers offer idempotency keys. Others expose a reference lookup or an operation-status endpoint. Their scope, retention and concurrency behavior differ. Verify those details for the endpoint you use. A disabled submit button prevents some repeated clicks, but it doesn't establish exactly-once processing across retries, workers and provider failures.

If supported read-only checks find the result, save that evidence against the original operation. If the provider can't establish what happened, stop for review. Don't convert uncertainty into an automatic retry simply because retry code is easy to write.

Make the operator path part of acceptance

For a labelled fixture, simulate a lost response after a successful write. Refresh the interface, open the original operation and check the result without resubmitting it. Inspect the destination record and the stored operation state. Then test a genuine rejection separately. Those are different recovery cases.

Useful operational evidence includes timestamps, operation references, provider references and redacted error categories. Agree access and retention. Private enquiry text, credentials and unnecessary customer details don't belong in ordinary logs.

Define the boundary honestly

This pattern improves traceability and helps control duplicate actions. It doesn't provide a universal exactly-once guarantee. The strongest claim you can make depends on the provider's documented behavior and the concurrency and lost-response tests you've actually run.

Disclosure and optional reference

The author works with StackEndBox. Our public integration project checklist expands the buyer-facing questions around field ownership, sync direction and operator recovery: https://www.stackendbox.com/blog/api-integration-project-checklist

This contribution was prepared with AI assistance. Its examples are illustrative, with no customer data or claimed production benchmark.

Reliable Webhooks in TypeScript: Idempotency, Deduplication and Failure Recovery

Muhammad Abdullah — Thu, 01 Oct 2026 10:34:34 +0000

Webhook providers deliver at least once, not exactly once. That single fact explains most webhook bugs: duplicate emails, double-credited balances, orders marked paid twice. Here is a pattern for handling deliveries safely with a Postgres table and a small amount of TypeScript, plus the failure cases it is designed around.

What the provider actually promises

Read the delivery semantics of the provider you integrate with. Stripe's documentation says endpoints may receive the same event more than once and recommends logging processed event IDs to skip repeats (Stripe webhooks docs). GitHub allows redelivery of a delivery, and each delivery carries a unique ID in a header (GitHub: handling webhook deliveries). The details differ, but the shape is the same: retries happen, ordering is not guaranteed, and the receiver has to cope.

So the receiver needs three properties:

Authenticity. Only accept payloads you can verify.
Idempotency. Processing the same event twice has the same effect as processing it once.
Recoverability. If processing fails halfway, you can retry without corrupting state.

Step 1: Verify the signature against the raw body

Signature verification must use the exact bytes the provider signed. The classic mistake is parsing JSON first and re-serializing it, which changes whitespace and key order.

import crypto from "node:crypto";

export function verifySignature(rawBody: Buffer, header: string, secret: string) {
  const expected = crypto.createHmac("sha256", secret).update(rawBody).digest("hex");
  const a = Buffer.from(expected);
  const b = Buffer.from(header);
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

Header formats vary by provider (some include a timestamp, some a prefix such as sha256=), so adapt this to the provider's spec. Use a constant-time comparison, and reject stale timestamps if the provider supplies one.

Step 2: Record the event before you act on it

The core of deduplication is a table with a unique constraint on the provider's event ID.

CREATE TABLE webhook_events (
  provider      text        NOT NULL,
  event_id      text        NOT NULL,
  event_type    text        NOT NULL,
  payload       jsonb       NOT NULL,
  status        text        NOT NULL DEFAULT 'received',  -- received | processing | processed | failed
  attempts      int         NOT NULL DEFAULT 0,
  last_error    text,
  received_at   timestamptz NOT NULL DEFAULT now(),
  processed_at  timestamptz,
  PRIMARY KEY (provider, event_id)
);

Insert with ON CONFLICT DO NOTHING (PostgreSQL INSERT docs). The insert itself is the deduplication check, and it stays correct under concurrent deliveries because the database enforces uniqueness, not your application code.

const inserted = await db.query(
  `INSERT INTO webhook_events (provider, event_id, event_type, payload)
   VALUES ($1, $2, $3, $4)
   ON CONFLICT (provider, event_id) DO NOTHING
   RETURNING event_id`,
  [provider, event.id, event.type, event]
);

if (inserted.rowCount === 0) {
  // Seen before. Acknowledge so the provider stops retrying.
  return res.status(200).end();
}

Notice what this does not do: it does not check "have I processed this?" before inserting. A read-then-write check has a race window. Two simultaneous deliveries can both read "not seen" and both proceed.

Step 3: Separate acknowledging from processing

Providers typically time out slow endpoints and retry. If your handler does heavy work inline, a slow downstream call can trigger a retry while the first attempt is still running.

A safer shape is: verify, store, return 2xx quickly, process asynchronously. A queue is the usual tool, but a simple worker that polls webhook_events for status = 'received' is enough for an early-stage product and has fewer moving parts.

Claim work with a row lock so two workers do not take the same event:

UPDATE webhook_events
SET status = 'processing', attempts = attempts + 1
WHERE (provider, event_id) = (
  SELECT provider, event_id FROM webhook_events
  WHERE status = 'received'
  ORDER BY received_at
  FOR UPDATE SKIP LOCKED
  LIMIT 1
)
RETURNING *;

Step 4: Make the business effect idempotent too

Deduplicating the event is not enough, because the same real-world action can arrive under different event IDs (a provider retry that creates a new event, or a manual replay). Where a handler changes money, entitlements, or state, anchor the effect to a natural key.

For example, record a payment against the provider's payment ID with a unique constraint, rather than incrementing a balance:

await db.query(
  `INSERT INTO payments (provider_payment_id, user_id, amount_cents)
   VALUES ($1, $2, $3)
   ON CONFLICT (provider_payment_id) DO NOTHING`,
  [payment.id, userId, payment.amount]
);

Compute balances from rows, or update them in the same transaction as the insert. The rule of thumb: an operation that is naturally "set to X" is safer than one that is "add 1."

Do the state change and the status = 'processed' update in a single transaction. If the process dies between them, you will either redo the whole thing (safe, because of the natural key) or none of it.

Step 5: Plan for out-of-order delivery

Do not assume invoice.paid arrives after invoice.created. Two defenses work well:

Fetch current state when it matters. Treat the webhook as a notification and read the object from the provider's API before acting, so a late, old event cannot overwrite newer state.
Use version or timestamp guards. Only apply an update if the incoming event is newer than what you have stored.

Also decide what happens when an event references something you have not seen yet. Parking it as failed with a retry is better than dropping it.

Step 6: Give failures an explicit state

An event that fails permanently should not disappear into logs. With the table above, a retry policy is a few lines:

async function handle(row: WebhookEvent) {
  try {
    await process(row);
    await markProcessed(row);
  } catch (err) {
    const exhausted = row.attempts >= MAX_ATTEMPTS;
    await markFailed(row, String(err), exhausted ? "failed" : "received");
  }
}

Retries with backoff for transient errors, a terminal failed state after the limit, and an alert on that state. A failed row with the original payload and last error is something a person can inspect and replay after fixing the bug. That recovery path is the reason to store the payload at all.

Failure scenarios worth testing

Write tests for these specifically. They are the ones that break in production:

The same event delivered twice, sequentially and concurrently.
A handler that throws after a partial side effect.
A process crash between processing and marking processed.
An older event arriving after a newer one.
A valid payload with an invalid signature, and a valid signature on a modified body.

Limits of this approach

A Postgres table is a reasonable default for a young product, but it is not a universal answer. High event volume may justify a proper queue, and retention needs a plan so the events table does not grow without bound. If handlers call external APIs that are not idempotent, you also need idempotency keys on those outbound calls. The IETF has an in-progress draft for an Idempotency-Key header that describes the idea (draft-ietf-httpapi-idempotency-key-header).

Summary

Verify the raw body, record the event ID under a unique constraint, acknowledge fast, process separately, anchor effects to natural keys, and give failures a visible state. None of it is clever, and that is the point: webhook code should be boring enough that retries stop being scary.

I'm Muhammad Abdullah, a full-stack engineer working as AbdullahBuilt. I build SaaS products, integrations and AI-assisted workflows.

Disclosure: this post was drafted with AI assistance and edited and reviewed by me.