Designing a Stop-on-Reply State Machine for Queued Follow-Ups

Lead-follow-up systems usually fail at the boundary between two valid events: a message is queued, then the customer replies before the worker sends it. A settings flag called "stop on reply" is not enough. The stop has to survive retries, duplicate webhooks, delayed jobs, and two workers racing each other.

This article shows a small TypeScript state machine and the tests I would require before allowing it near a production conversation.

Disclosure: I work with the Auto-Respond Team. This is an engineering pattern from our own testing, not an independent product review.

Start with an explicit conversation state

Do not infer the state from whichever table a worker happens to read. Give the conversation one authoritative state and restrict transitions.

type ConversationState =
  | "active"
  | "replied"
  | "booked"
  | "opted_out"
  | "paused"
  | "closed";

type StopReason = Exclude<ConversationState, "active">;

interface Conversation {
  id: string;
  state: ConversationState;
  version: number;
  stoppedAt?: string;
  stopReason?: StopReason;
}

The version matters. It gives the database a compare-and-set boundary, so a worker cannot save an old active snapshot over a newer replied state.

Treat inbound events as commands, not observations

A webhook saying that a customer replied is not merely an analytics event. It is a command to transition the conversation and invalidate future work.

Give every inbound event a stable provider ID. If the provider does not supply one, derive an idempotency key from the conversation ID, event type, and provider timestamp, then keep the raw payload hash for investigation.

interface StopEvent {
  eventId: string;
  conversationId: string;
  reason: StopReason;
  occurredAt: string;
}

async function applyStop(event: StopEvent): Promise<void> {
  await db.transaction(async (tx) => {
    const firstSeen = await tx.eventInbox.insertIfAbsent({
      id: event.eventId,
      receivedAt: new Date().toISOString(),
    });

    if (!firstSeen) return;

    const conversation = await tx.conversations.getForUpdate(
      event.conversationId,
    );

    if (!conversation || conversation.state !== "active") return;

    await tx.conversations.updateWhereVersion(
      conversation.id,
      conversation.version,
      {
        state: event.reason,
        version: conversation.version + 1,
        stoppedAt: event.occurredAt,
        stopReason: event.reason,
      },
    );

    await tx.messageOutbox.cancelFutureForConversation(
      conversation.id,
      event.occurredAt,
    );
  });
}

The inbox insert, state transition, and outbox cancellation belong in one transaction. Splitting them creates a failure window where the conversation says replied but a future message is still sendable.

Make the send worker prove permission twice

Cancelling future rows is necessary, but it is not sufficient. A worker may already have claimed a row before the reply arrived.

Use a two-check send path:

1. Claim one queued message with a short lease.
2. Read the authoritative conversation state immediately before delivery.
3. Atomically mark the row sending only if the conversation is still active.
4. Call the provider.
5. Record the provider result with the same attempt ID.

async function deliver(row: OutboxMessage): Promise<void> {
  const attemptId = crypto.randomUUID();

  const permitted = await db.transaction(async (tx) => {
    const conversation = await tx.conversations.getForUpdate(
      row.conversationId,
    );

    if (!conversation || conversation.state !== "active") {
      await tx.messageOutbox.cancel(row.id, "conversation_stopped");
      return false;
    }

    return tx.messageOutbox.markSendingIfQueued(row.id, attemptId);
  });

  if (!permitted) return;

  const result = await provider.send({
    to: row.destination,
    body: row.body,
    idempotencyKey: row.id,
  });

  await db.messageOutbox.recordProviderResult(row.id, attemptId, result);
}

The second check closes the most common race. It does not make a remote provider call transactional, so the UI and audit log must be honest about the final boundary.

Define the handoff boundary

Once a provider has accepted a message, cancellation may no longer be possible. Model that fact instead of presenting a perfect-looking toggle.

Useful outbox states are queued, leased, sending, accepted, delivered, cancelled, and failed. A stop event should cancel queued and expired leased rows. It should mark an in-flight sending row as stop_requested and capture whether the provider accepted it before or after the stop timestamp.

That gives support staff a concrete answer when someone asks why a message still arrived. "Automation was off" is not an audit trail.

Test the races deliberately

A happy-path unit test is almost worthless here. Add barriers so the tests can pause a worker at each boundary.

At minimum, cover these cases:

• reply before the worker claims the row;
• reply after claim but before the permission check;
• reply after permission check but before the provider accepts;
• duplicate reply webhook;
• delivery worker retry after a timeout;
• booking and reply arriving in either order;
• opt-out arriving while two follow-ups are queued;
• stale worker attempting to overwrite a newer conversation version.

A useful integration test freezes the worker after claim, applies the reply, releases the worker, then waits beyond the original scheduled time. The pass condition is not just "row cancelled." It is no provider acceptance and no retry capable of recreating the row.

it("does not send when a reply wins the claim race", async () => {
  const gate = createBarrier();
  const queued = await seedQueuedFollowUp();

  worker.beforePermissionCheck = () => gate.wait();
  const delivery = worker.runOnce();

  await gate.reached;
  await applyStop(replyEventFor(queued.conversationId));
  gate.release();

  await delivery;
  expect(provider.accepted).toHaveLength(0);
  expect(await outbox.state(queued.id)).toBe("cancelled");
});

Keep evidence that survives the dashboard

For each stop test, store the configuration version, conversation ID, provider event ID, queued message IDs, state transitions, timestamps, worker attempt IDs, and provider responses. Do not put real customer data in the test packet.

A dashboard can then show four separate facts:

• when the inbound stop event occurred;
• when the conversation state changed;
• which queued rows were cancelled;
• whether any provider accepted a message after that point.

This is the difference between a feature claim and a reproducible result.

For context, our current vendor-supplied Yelp workflow facts and stop conditions are published in the Yelp auto responder guide. Apply the same race tests to that workflow or any competing implementation before trusting the setting.

The durable design rule is simple: stopping is a state transition, not a filter in the UI. Put it in the transaction, recheck it at delivery, and test the races on purpose.