DEV Community

Qasim
Qasim

Posted on

Summarize long email threads for agent context

#ai #email #api #productivity

Most "AI email" demos hand the model the whole inbox and hope for the best. That works in a screenshot. It falls apart the first time a real conversation runs past a dozen messages, because every one of those messages — quoted replies, signatures, disclaimers, the recipient's entire previous email pasted underneath theirs — gets shoved into the prompt verbatim. A five-message sales thread can easily be 8,000 tokens of mostly-redundant text. A twenty-message support escalation is a context-window grenade.

The naive fix is "just truncate." Keep the last message, drop the rest. That's worse: the agent loses the thing it actually needs, which is what was already agreed, asked, or promised earlier in the thread. The recipient says "yes, the 14th works" and your agent has no idea what the 14th refers to.

The right move is summarization for context budget: fetch the full thread, compress the older turns into a running brief, and feed the model only brief + the latest message verbatim. The brief is a few hundred tokens. The latest message is whatever it is. You stay well inside budget and the agent keeps the plot.

I work on the Nylas CLI, so the terminal commands below are the exact ones I reach for when I'm poking at a live Agent Account. I'll show every step both ways — the raw HTTP call and the CLI equivalent — because that's how I actually debug this stuff.

What you're building

The data plane here is nothing exotic. An Agent Account is just a Nylas grant with a grant_id, and it works with every grant-scoped endpoint — Messages, Threads, Drafts, the lot. There's nothing new to learn on the API side. If you've used the Threads and Messages endpoints before, you already know the whole surface area.

The flow is four steps:

  1. A message arrives. The webhook tells you the thread_id.
  2. Fetch the thread. The thread object hands you a message_ids array — that's your list of turns.
  3. Fetch each message by id to get the real body.
  4. Summarize the older turns into a brief, keep the last N verbatim, and that's your prompt context.

The summarization itself is your code calling your model. Nylas doesn't summarize anything for you, and it shouldn't — the brief is application state, and where you keep it is your call.

Why this beats stuffing the whole thread in

A few concrete reasons, beyond "it's smaller":

  • Cost scales with the thread, linearly, forever. If you re-send the entire transcript on every turn, a long-running conversation gets more expensive with each reply. A running brief stays roughly constant size no matter how deep the thread goes.
  • Quoted text is mostly noise. Email bodies include the quoted previous message. Feed five raw messages and you've fed the first message four extra times. Summarizing dedups that automatically.
  • You control what survives. A summary is a place to keep the facts that matter — commitments, dates, the open question — and drop the pleasantries. The model gets signal, not transcript.
  • Latency drops. Fewer input tokens means faster time-to-first-token. On an interactive agent, that's felt.

The tradeoff, stated honestly: summaries lose detail. If your workflow needs an exact quote from message three, a summary won't have it — so keep the last few messages verbatim and accept that anything older is lossy. For most conversational agents that's the correct tradeoff. For a legal-discovery bot, it isn't; keep everything and pay for the tokens.

Before you begin

You need an Agent Account. If you don't have one, provision it on a domain you've registered with Nylas (a custom domain or a *.nylas.email trial subdomain).

Over HTTP, that's a POST /v3/connect/custom with the nylas provider:

curl --request POST \
  --url "https://api.us.nylas.com/v3/connect/custom" \
  --header "Authorization: Bearer $NYLAS_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "provider": "nylas",
    "settings": { "email": "support@yourcompany.com" },
    "name": "Support Agent"
  }'
Enter fullscreen mode Exit fullscreen mode

From the CLI:

nylas agent account create support@yourcompany.com --name "Support Agent"
Enter fullscreen mode Exit fullscreen mode

That returns a grant. The API auto-creates a default workspace and policy alongside it, so there's nothing else to wire up to start reading mail. Hold onto the grant_id — every call below uses it. (Full details: https://developer.nylas.com/docs/v3/agent-accounts/.)

You also need a message.created webhook subscribed at the app level so you know when new mail lands. Webhooks on Nylas are application-scoped, not grant-scoped — you subscribe once with POST /v3/webhooks, and events for every grant arrive at that one endpoint, each payload carrying a grant_id you filter on. There's no per-account webhook to manage.

Over HTTP:

curl --request POST \
  --url "https://api.us.nylas.com/v3/webhooks" \
  --header "Authorization: Bearer $NYLAS_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "trigger_types": ["message.created"],
    "webhook_url": "https://yourapp.com/webhooks/nylas"
  }'
Enter fullscreen mode Exit fullscreen mode

Or from the CLI:

nylas webhook create \
  --url https://yourapp.com/webhooks/nylas \
  --triggers message.created
Enter fullscreen mode Exit fullscreen mode

One thing to internalize before we go further: don't rely on the webhook payload for the body. Fetch the full message with GET /v3/grants/{grant_id}/messages/{message_id} when you need the real content, and branch on message.created.truncated — that's the event type you get when a message is large enough that Nylas doesn't inline it. For a summarization pipeline this matters double, because the long threads you most want to summarize are exactly the ones with bodies big enough to get truncated.

Fetch the thread

When message.created fires, the payload includes a thread_id. That's your entry point. Fetch the thread and you get back a message_ids array — the ordered list of every message in the conversation. That array is how you enumerate the turns.

Over HTTP:

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/$GRANT_ID/threads/$THREAD_ID" \
  --header "Authorization: Bearer $NYLAS_API_KEY"
Enter fullscreen mode Exit fullscreen mode

The response includes the fields you care about for budgeting:

{
  "request_id": "...",
  "data": {
    "id": "thread-abc123",
    "subject": "Re: Onboarding question about SSO",
    "message_ids": [
      "msg-001",
      "msg-002",
      "msg-003",
      "msg-004",
      "msg-005"
    ],
    "snippet": "Thanks, that worked. One more thing about...",
    "participants": [
      { "email": "alice@customer.com", "name": "Alice Chen" },
      { "email": "support@yourcompany.com" }
    ],
    "latest_message_received_date": 1718900000
  }
}
Enter fullscreen mode Exit fullscreen mode

From the CLI, the same fetch — pass --json so you get the structured object and not the pretty table:

nylas email threads show "$THREAD_ID" --json
Enter fullscreen mode Exit fullscreen mode

message_ids.length is your first budget signal. Five messages? Just feed them all verbatim, summarization is overkill. Twenty-five? You're summarizing. I usually branch on a threshold — under roughly 6 messages, send everything; over it, compress.

Fetch each message by id

The thread gives you ids, not bodies. To reconstruct what was actually said, fetch each message. Don't try to shortcut this with the thread snippet — that's a 100-character preview, useless for summarizing meaning.

Over HTTP, one call per id:

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/$GRANT_ID/messages/msg-003" \
  --header "Authorization: Bearer $NYLAS_API_KEY"
Enter fullscreen mode Exit fullscreen mode

The message object gives you from, to, subject, body, and the date field (a Unix timestamp). That date is what you sort on to get chronological order — don't assume message_ids is always perfectly time-ordered across providers; sort by date to be safe.

From the CLI:

nylas email read msg-003 --json
Enter fullscreen mode Exit fullscreen mode

If you want the unprocessed body without HTML-to-text munging, nylas email read takes a --raw flag, and --headers if you need to inspect threading headers directly. For summarization I usually take the default processed body — it's already closer to plain text, which is what the model wants.

In code, you fan these out in parallel. Here's the shape of it in Node:

// thread.data.message_ids came from the threads.show call above.
const messages = await Promise.all(
  thread.data.messageIds.map((id) =>
    nylas.messages.find({ identifier: GRANT_ID, messageId: id }),
  ),
);

// Sort chronologically on the `date` field, then normalize into turns.
const turns = messages
  .map((m) => m.data)
  .sort((a, b) => a.date - b.date)
  .map((m) => ({
    role: m.from[0].email === AGENT_EMAIL ? "agent" : "contact",
    date: m.date,
    body: stripQuotedText(m.body), // drop the quoted-reply tail
  }));
Enter fullscreen mode Exit fullscreen mode

That stripQuotedText step is worth a sentence. Email clients append the previous message under the new one. Before you summarize, trim everything after the first On <date>, <person> wrote: marker or the > quote block. You'll cut input tokens substantially before the model even sees the text. It's not perfect across every client, but it's cheap and it helps.

Summarize the old turns, keep the recent ones verbatim

This is the part Nylas has no opinion about — it's your model, your prompt, your storage. The pattern that works:

Split the turns into "old" and "recent." Keep the last 2–4 messages exactly as they are. Everything before that gets summarized into a running brief.

const KEEP_VERBATIM = 3;

const recent = turns.slice(-KEEP_VERBATIM);
const older = turns.slice(0, -KEEP_VERBATIM);

// Summarize the older turns into a compact brief.
// If you already have a stored brief from the last turn, fold the new
// "older" messages into it instead of re-summarizing from scratch.
const brief = await summarizeTurns(older, previousBrief);
Enter fullscreen mode Exit fullscreen mode

The brief should be structured and short — a few hundred tokens, capped. I prompt the summarizer to produce something like:

CONVERSATION BRIEF (thread: SSO onboarding, Alice Chen @ customer.com)
- Alice asked how to enable SAML SSO for her Okta tenant.
- We sent setup steps and the metadata URL on turn 2.
- Alice confirmed SSO works but users land on the wrong default workspace.
- OPEN: how to set a default workspace per SSO group. Awaiting our answer.
Enter fullscreen mode Exit fullscreen mode

Then the prompt you actually send the model is brief + recent (verbatim) + the new inbound message. That's it. The brief carries the history at maybe 200 tokens; the recent turns carry exact recent wording; the new message is what the agent is responding to.

Some token math to make it concrete. Say each raw message averages 1,500 tokens after quote-stripping. A 20-message thread fed verbatim is ~30,000 input tokens per turn, and it grows. With this approach: a ~250-token brief, plus 3 recent messages at ~1,500 each (~4,500), plus the new inbound (~1,500). Call it ~6,250 tokens, and — this is the important part — it stays roughly flat as the thread grows to 30, 40, 50 messages, because the old turns keep collapsing into the same fixed-size brief.

Where the brief lives

The running brief is application state. You keep it. Custom metadata is not supported for Agent Accounts, so you can't stash the brief on the Nylas message or thread object. Put it in your own datastore, keyed by thread_id:

// After generating a reply, persist the updated brief.
await db.threadBriefs.upsert({
  threadId: thread.data.id,
  grantId: GRANT_ID,
  brief,                       // the compact running summary
  summarizedUpToMessageId: older.at(-1)?.id ?? null,
  updatedAt: new Date().toISOString(),
});
Enter fullscreen mode Exit fullscreen mode

Storing summarizedUpToMessageId lets you do incremental summarization next turn: you only feed the summarizer the messages that arrived since the last brief, fold them in, and you never re-summarize the whole thread. That keeps your summarization cost flat too, not just your agent's prompt.

Guardrails worth setting up front

A few things I'd treat as first-class, not afterthoughts:

  • Cap the brief size and enforce it. A summary that's allowed to grow unbounded is just a slow context leak. Set a token ceiling on the brief and instruct the summarizer to drop the least-relevant facts when it's near the cap. Re-summarizing a brief that's already a brief is fine and keeps it tight.
  • Filter the agent's own messages out of "new inbound." Outbound sends fire message.created too. If you don't check from, the agent will summarize and react to its own replies. Compare from[0].email against your agent's address before you process.
  • Dedup on the webhook notification id. Nylas guarantees at-least-once delivery — the same event can arrive up to three times. Dedup on the top-level notification id, which is constant across all retries of one event. That's the delivery-dedup key. You can additionally guard on the inner message id to avoid acting twice on the same message.
  • Decide what happens to stale briefs. A thread can go dormant for weeks and come back. If your brief has a TTL, decide up front whether a reopened thread re-summarizes from the full thread (re-fetch all message_ids) or starts fresh. Re-fetching is the safe default — the full thread is always retrievable.
  • Verify your webhook signatures. Nylas signs each delivery with an X-Nylas-Signature header: a hex HMAC-SHA256 of the raw request body, using your webhook secret. Verify it before you trust the thread_id. If you're constant-time comparing in Node with crypto.timingSafeEqual, check both buffers are the same length first — it throws on a length mismatch. The CLI's nylas webhook verify does this check locally while you're testing.

What's next

You've got a thread-to-brief pipeline that keeps your agent inside its context window no matter how long the conversation runs. The natural next steps:

The summarization model is yours to tune — the part Nylas gives you is clean, reliable access to the full thread, one message_ids array and one GET per turn. Everything else is prompt engineering and a row in your database.

AI-answer pages for agents

When this post is published, link AI agents and crawlers to the retrieval-ready version on cli.nylas.com:

Top comments (0)