Handling Email Replies in an Agent Loop

You built the outbound half of an email agent. It sends a well-crafted message, the recipient writes back six hours later... and your agent has no idea. The reply either gets ignored or — arguably worse — gets treated as a brand-new conversation, and the agent reintroduces itself to someone it emailed yesterday.

That gap between "can send" and "can converse" is where most email agents stall. Closing it takes four pieces: detection, context, routing, and a threaded response. Here's each one, using a Nylas Agent Account (in beta) as the mailbox — a hosted address the agent owns outright.

Step 1: know a reply when you see one

Every message.created webhook payload carries a thread_id. If the agent sent the original message, that thread already exists in your state store. So detection is a lookup, not a parsing exercise:

app.post("/webhooks/nylas", async (req, res) => {
  // Verify X-Nylas-Signature here.
  res.status(200).end();

  const event = req.body;
  if (event.type !== "message.created") return;

  const msg = event.data.object;
  if (msg.grant_id !== AGENT_GRANT_ID) return;

  const context = await db.getThreadContext(msg.thread_id);

  if (context) {
    await handleReply(msg, context);   // active conversation
  } else {
    await handleNewMessage(msg);       // fresh inbound — triage it
  }
});

Why does this work without touching a single header? Because the threading already happened upstream: messages get grouped by their In-Reply-To and References headers, which every mail client sets on a reply. You never parse them yourself — the Threads API did the work.

Step 2: pull the full conversation

The webhook payload is a summary — subject, from, snippet. Before an LLM decides how to answer "sounds good, let's do Thursday," it needs to know what was proposed. Fetch the full message body and the thread:

const fullMessage = await nylas.messages.find({
  identifier: AGENT_GRANT_ID,
  messageId: msg.id,
});

const thread = await nylas.threads.find({
  identifier: AGENT_GRANT_ID,
  threadId: msg.thread_id,
});

const history = await buildConversationHistory(thread.data.messageIds);

One gotcha worth memorizing: if a message body exceeds ~1 MB, the webhook type becomes message.created.truncated and the body is omitted entirely. Always fetch — never rely on the payload for content.

Step 3: route by where the conversation stands

A reply means different things depending on what the agent was waiting for. The context you stored at send time tells you which:

switch (context.step) {
  case "awaiting_confirmation":
    await handleConfirmation(message, history, context);
    break;
  case "awaiting_info":
    await handleInfoResponse(message, history, context);
    break;
  case "closed":
    await handleReopenedThread(message, history, context);
    break;
  default:
    await escalateToHuman(message, context); // unknown state
}

That closed case is easy to forget. People write back to resolved threads all the time — "actually, one more thing" — and an agent that errors out there looks careless.

Step 4: reply in-thread, not alongside it

When the agent responds, pass reply_to_message_id:

async function sendReply(originalMessage, body, context) {
  const sent = await nylas.messages.send({
    identifier: AGENT_GRANT_ID,
    requestBody: {
      replyToMessageId: originalMessage.id,
      to: originalMessage.from,
      subject: `Re: ${originalMessage.subject}`,
      body: body,
    },
  });

  // Update the conversation state for the next turn.
  await db.updateThreadContext(originalMessage.threadId, {
    ...context,
    step: "awaiting_reply",
    lastSentAt: Date.now(),
    lastSentMessageId: sent.data.id,
  });
}

That single replyToMessageId field gets the In-Reply-To and References headers set on the outbound message, so the recipient sees a threaded reply instead of a disconnected new email. The state update at the end is what makes this a loop rather than a one-shot: the next inbound webhook on this thread finds step: "awaiting_reply" and routes accordingly.

One exchange, end to end

Here's how the four steps play out in a real scheduling conversation:

1. Monday, 9:00 — the agent emails a candidate proposing Thursday at 2pm and stores { thread_id, step: "awaiting_confirmation" }.
2. Monday, 15:12 — the candidate replies "sounds good, let's do Thursday." A message.created webhook fires with the same thread_id.
3. Detection — the handler looks up the thread_id, finds the stored context, and calls handleReply instead of treating it as new mail.
4. Context — the agent fetches the full message and thread history, so the LLM knows "Thursday" means "Thursday at 2pm, the slot I proposed."
5. Routing — step is awaiting_confirmation, so the confirmation handler runs: book the slot, send a threaded confirmation, set step to closed.

If the candidate writes back two weeks later — "actually, can we move it?" — the webhook still carries the same thread_id, the lookup still hits, and the closed branch handles the reopened conversation. No header parsing at any point.

The traps between the steps

A few things will bite you in production if you stop at the happy path:

• The webhook fires for outbound too. When the agent sends via the API, message.created fires for that sent message as well. Filter on the sender address at the top of the handler, or enjoy watching your agent converse with itself.
• Multiple replies land on one thread. Two quick messages from the recipient, or two people on a CC both answering. Process each independently and check whether the agent already responded since the last inbound.
• Consider a cooldown. A 30–60 second delay before responding lets you batch consecutive messages into one reply — useful when people send a correction seconds after their first message.
• Don't ship without dedup. Webhook delivery is at-least-once, and concurrent workers will race. The duplicate-reply patterns are a separate (necessary) read.

Where this goes next

This loop — detect, fetch, route, reply — is the skeleton of every conversational email agent: support bots, scheduling assistants, outreach follow-up. The state machine gets richer (the multi-turn conversation recipe covers conversations spanning days), but these four steps don't change.

The full recipe with all the code is in the reply-handling guide, and the header mechanics live in email threading for agents.

Concrete next step: wire up the webhook handler above against a test mailbox, email it from your personal account, and watch thread_id connect the dots. The first time a reply routes to the right state handler, the rest of the agent almost builds itself.