A future revision should not change yesterday's data selection

Muhammad Abdullah — Sat, 24 Oct 2026 21:47:28 +0000

An as-of join can run successfully and still answer the wrong historical question. The important timestamp is when a particular version became available, not just the period the value describes.

Suppose a synthetic September fact is published on October 3 with value 100, then revised on October 7 to 80. For an October 4 decision, the eligible value is 100. A pipeline that joins today's latest revision to every earlier date silently returns 80 instead.

There is a useful regression property here: adding a future revision must not change a past selection.

from datetime import datetime

def available(value):
    result = datetime.fromisoformat(value)
    if result.tzinfo is None:
        raise ValueError("Explicit timezone required")
    return result

def select(rows, entity, period, decision):
    cutoff = available(decision)
    eligible = [r for r in rows
                if r["entity"] == entity and r["period"] == period
                and available(r["available_at"]) <= cutoff]
    if not eligible:
        return None
    stamps = [available(r["available_at"]) for r in eligible]
    if len(stamps) != len(set(stamps)):
        raise ValueError("Ambiguous revision order")
    return max(eligible, key=lambda r: available(r["available_at"]))["value"]

original = [{"entity": "DEMO", "period": "2026-09-30",
             "available_at": "2026-10-03T09:00:00+00:00", "value": 100}]
revised = original + [{"entity": "DEMO", "period": "2026-09-30",
                      "available_at": "2026-10-07T09:00:00+00:00", "value": 80}]
decision = "2026-10-04T12:00:00+00:00"

assert select(original, "DEMO", "2026-09-30", decision) == 100
assert select(revised, "DEMO", "2026-09-30", decision) == 100
assert select(revised, "DEMO", "2026-09-30", "2026-10-08T12:00:00+00:00") == 80
assert select(revised, "DEMO", "2026-09-30", "2026-10-01T12:00:00+00:00") is None
print("All four synthetic selection checks passed")

This is a deliberately small selection example. It isn't a trading strategy, a performance result or a complete financial data model. Equal-time revisions are rejected because file order isn't an evidence-based tie-breaker.

The availability contract still needs review. Replaying what a live application knew requires actual receipt history as well as publication time. A hypothetical research archive can use declared delivery-lag assumptions, but that does not reconstruct your own system's history.

In a larger pipeline, also check the entity and fact keys, timezone precision, missing observations and revision identifiers. A backward join does not repair unreliable timestamps, retrospective universe membership, leaked feature transforms or a model trained after the historical cutoff.

The useful engineering question is whether you can preserve and test these boundaries, rather than whether a chart looks convincing.

I’m affiliated with StackEndBox. This adaptation uses synthetic examples from our tested point-in-time data guide. It contains no client data or claimed customer outcome. Prepared with AI assistance, with the displayed code executed and checked.

Before retrying a failed integration, ask whether the write already happened

Muhammad Abdullah — Sat, 24 Oct 2026 20:45:29 +0000

A timeout tells your application that it didn't receive a result in time. It doesn't establish that the receiving service rejected the operation. That distinction matters when an integration creates a contact, opens a support request or performs another consequential write.

This is an illustrative design discussion, not a report about a customer incident or a tested implementation for a specific vendor.

Start with a business operation reference

Give the intended action a stable reference before attempting delivery. Keep that reference when the browser refreshes or the worker restarts. A newly generated reference on every retry makes it harder to recognize that two requests represent the same intended action.

Separate what you know

The operation can be prepared but not attempted, conclusively rejected, confirmed by the receiving service, or uncertain after an attempted write. An HTTP timeout belongs in that last category until further evidence resolves it. A generic failed label loses the distinction that tells an operator whether another write is safe.

Use the provider's actual recovery capability

Some providers offer idempotency keys. Others expose a reference lookup or an operation-status endpoint. Their scope, retention and concurrency behavior differ. Verify those details for the endpoint you use. A disabled submit button prevents some repeated clicks, but it doesn't establish exactly-once processing across retries, workers and provider failures.

If supported read-only checks find the result, save that evidence against the original operation. If the provider can't establish what happened, stop for review. Don't convert uncertainty into an automatic retry simply because retry code is easy to write.

Make the operator path part of acceptance

For a labelled fixture, simulate a lost response after a successful write. Refresh the interface, open the original operation and check the result without resubmitting it. Inspect the destination record and the stored operation state. Then test a genuine rejection separately. Those are different recovery cases.

Useful operational evidence includes timestamps, operation references, provider references and redacted error categories. Agree access and retention. Private enquiry text, credentials and unnecessary customer details don't belong in ordinary logs.

Define the boundary honestly

This pattern improves traceability and helps control duplicate actions. It doesn't provide a universal exactly-once guarantee. The strongest claim you can make depends on the provider's documented behavior and the concurrency and lost-response tests you've actually run.

Disclosure and optional reference

The author works with StackEndBox. Our public integration project checklist expands the buyer-facing questions around field ownership, sync direction and operator recovery: https://www.stackendbox.com/blog/api-integration-project-checklist

This contribution was prepared with AI assistance. Its examples are illustrative, with no customer data or claimed production benchmark.

DEV Community: StackEndBox