The Lookup That Lies: Why Your Agent’s Retry Logic Has a Hidden Race Condition

Your agent dispatches an action. You run a provider lookup immediately after to confirm it happened. The lookup returns “not found.” You mark it failed. You retry.

Except the action succeeded. Your lookup just ran inside the provider’s replication window.

You just caused a duplicate.

This isn’t a bug in your retry logic. It’s a bug in your trust model for the lookup itself.

The Problem: Write Success ≠ Read Availability

Some providers are eventually consistent. The write succeeds — HTTP 200, no error. But the artifact isn’t immediately queryable on the read replica. Depending on the provider and load, the replication lag might be 200ms, 2 seconds, or 30 seconds.

A lookup that runs immediately after dispatch is reading from a clock that hasn’t caught up.

Two failure modes that look identical:

Lookup result: "not found"
Case A: Action genuinely failed — nothing was written
Case B: Action succeeded — read replica hasn't caught up yet

Both return the same thing. Only Case A should trigger a retry.

The Naive State Machine Gets This Wrong

`suspend fun confirmDispatch(record: ActionRecord, provider: ActionProvider): ActionRecord {
    val artifact = provider.lookup(record.dispatchId)

    return if (artifact != null) {
        record.copy(delivery = DeliveryState.CONFIRMED)
    } else {
        // BUG: This assumes "not found" means "failed"
        // It might just mean "not propagated yet"
        record.copy(delivery = DeliveryState.FAILED)
    }
}`

If this runs immediately after dispatch, a “not found” response is ambiguous. You cannot distinguish a genuine failure from a replication lag.

The Fix: A Minimum Age Gate

Don’t allow a PENDING record to enter reconciliation until the consistency window has closed:

`data class ActionRecord(
    val actionId: String,
    val dispatchId: String,
    val providerObjectId: String?,
    val delivery: DeliveryState,
    val acceptance: AcceptanceState,
    val dispatchedAt: Instant,
    val confirmedAt: Instant?,
    val eligibleForReconciliationAt: Instant  // dispatched_at + consistency_window
)

fun ActionRecord.isEligibleForReconciliation(): Boolean {
    return Instant.now().isAfter(eligibleForReconciliationAt)
}`

The reconciliation loop checks eligibility before running a lookup:

`suspend fun reconcileIfEligible(
    record: ActionRecord,
    provider: ActionProvider
): ActionRecord {
    if (!record.isEligibleForReconciliation()) {
        // Too early — lookup result cannot be trusted
        return record
    }

    val artifact = provider.lookup(record.dispatchId)

    return if (artifact != null) {
        record.copy(
            providerObjectId = artifact.id,
            delivery = DeliveryState.CONFIRMED,
            confirmedAt = Instant.now()
        )
    } else {
        // Now "not found" actually means failed
        // The consistency window has closed
        record.copy(delivery = DeliveryState.FAILED)
    }
}`

Where N Comes From

The consistency window duration isn’t arbitrary:

`object ConsistencyWindows {
    // From provider documentation
    val STRIPE_EVENTS = Duration.ofSeconds(5)

    // From empirical observation (no SLA published)
    val GOOGLE_CALENDAR = Duration.ofSeconds(10)

    // Conservative default when nothing is documented
    val DEFAULT = Duration.ofSeconds(30)
}

fun ActionRecord.withConsistencyWindow(provider: ProviderType): ActionRecord {
    val window = when (provider) {
        ProviderType.STRIPE -> ConsistencyWindows.STRIPE_EVENTS
        ProviderType.GOOGLE_CALENDAR -> ConsistencyWindows.GOOGLE_CALENDAR
        else -> ConsistencyWindows.DEFAULT
    }
    return copy(
        eligibleForReconciliationAt = dispatchedAt.plus(window)
    )
}`

If the provider doesn’t publish their replication lag, measure it. Dispatch a test write, poll the read endpoint, record when it becomes visible. Do this under load, not just in ideal conditions.

The Late Confirmation Race

There’s a related problem: what happens when a confirmation arrives after you’ve already marked a record FAILED and queued a replacement?

`suspend fun handleLateConfirmation(
    record: ActionRecord,
    artifact: ProviderArtifact
): ActionRecord {
    return when (record.delivery) {
        DeliveryState.FAILED -> {
            // Late confirmation — action succeeded despite our earlier verdict
            // Cancel any queued replacement before updating state
            cancelQueuedReplacement(record.actionId)

            record.copy(
                providerObjectId = artifact.id,
                delivery = DeliveryState.CONFIRMED,
                confirmedAt = Instant.now()
            )
        }
        DeliveryState.PENDING -> {
            // Normal confirmation path
            record.copy(
                providerObjectId = artifact.id,
                delivery = DeliveryState.CONFIRMED,
                confirmedAt = Instant.now()
            )
        }
        DeliveryState.CONFIRMED -> {
            // Already confirmed — idempotent, no change
            record
        }
    }
}`

The replacement dispatch must read delivery state immediately before firing — not at the time it was scheduled. Scheduling a replacement and dispatching one are two separate moments. The state check belongs at dispatch.

What This Looks Like in the Full State Machine

`enum class DeliveryState {
    PENDING,      // Dispatched, awaiting confirmation
    CONFIRMED,    // Provider lookup returned the artifact
    FAILED        // Lookup returned nothing AFTER consistency window closed
}

// DENIED_UNRESOLVED requires a lookup outside the consistency window
// before it can transition to FAILED
fun ActionRecord.resolvedStatus(): ActionStatus = when {
    delivery == DeliveryState.PENDING && 
        !isEligibleForReconciliation() -> ActionStatus.DENIED_UNRESOLVED  // Window still open
    delivery == DeliveryState.PENDING && 
        isEligibleForReconciliation() -> ActionStatus.DENIED_UNRESOLVED   // Needs lookup
    delivery == DeliveryState.FAILED -> ActionStatus.FAILED
    delivery == DeliveryState.CONFIRMED && 
        acceptance == AcceptanceState.REJECTED -> ActionStatus.FAILED
    delivery == DeliveryState.CONFIRMED && 
        acceptance == AcceptanceState.ACCEPTED -> ActionStatus.CONFIRMED
    else -> ActionStatus.DENIED_UNRESOLVED
}
`

DENIED_UNRESOLVED doesn’t decay into FAILED on a single early lookup. It requires a lookup that runs outside the consistency window. A lookup inside the window is evidence of nothing.

The Practical Implication

The consistency window is part of your verification model. If it isn’t documented somewhere in your code — as a named constant, not a magic number — it’s an assumption. And assumptions are where duplicate dispatches live.

For StareBrain’s on-device actions, this matters particularly for anything that touches an external provider after the accessibility layer fires. The accessibility service can confirm the UI action happened. Whether the downstream provider has recorded it is a different clock entirely.

Two clocks. One verification model. The gap between them is where your duplicates come from.