Daniel Kim

Posted on Aug 13

Trigger.dev's Chat Agent Lets an AI Conversation Sit Idle for Days. You Only Pay for the Seconds It's Actually Talking

#ai #webdev #opensource #architecture

Close a ChatGPT tab mid-response and the stream is gone. Reopen it and you get the finished message because someone's server kept talking to the model after you left — but wire that up yourself for your own product and you're suddenly building a job queue, a pub/sub layer, a reconnect protocol, and a place to put "assistant said X but the process died before writing it down."

Trigger.dev, the open-source background jobs platform, shipped a feature this month that turns that whole stack into one function. It's called chat.agent, it launched as Chat Agent on Product Hunt on August 13, 2026 with the tagline "AI chat that keeps running after you close the tab," and the mechanic worth understanding isn't the tab-survival — that's table stakes for anyone using Redis and Postgres. The mechanic worth understanding is what it costs you while nobody's looking at it: an idle AI conversation on Trigger.dev can sit for days without a single cent of compute billed, then wake up mid-thought the instant a message arrives.

That's a specific, checkable claim, and it's also the thing most "durable AI agent" launches don't actually deliver, because most of them are wrappers around a job queue that still bills you for holding a worker open. This piece is about what Trigger.dev built, how the underlying machine actually works, what it costs, and where the launch page goes quiet.

What happened

chat.agent isn't a brand-new company or a stealth launch — it's a new primitive bolted onto a platform that's been running background jobs in production since 2023 and has raised funding from backers including Sequoia. According to Trigger.dev's own changelog, chat.agent has been generally available since July 2, 2026, and running in production since June, serving "millions of sessions" and "more than 84 years of compute" before its public Product Hunt debut. Arena, a coding-agent product, built its "Agent Mode" on top of it; its engineer Graham Tremper is quoted in Trigger.dev's release notes saying "every conversation gets a real machine, which made our durable agents much more straightforward to build."

So this is less "brand new API, please stress-test it" and more "here's a stress-tested internal feature we're now selling directly." That matters for how much to trust the durability claims, and it doesn't fully excuse what's still rough (more on that below).

What it actually does

The pitch is a chat backend with zero API routes. You write a Trigger.dev task that takes messages and returns a stream, wire the Vercel AI SDK's useChat hook to a custom transport, and that's the whole backend:

import { chat } from "@trigger.dev/sdk/ai";
import { streamText, stepCountIs } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

export const myChat = chat.agent({
  id: "my-chat",
  run: async ({ messages, signal }) =>
    streamText({
      model: anthropic("claude-sonnet-4-5"),
      messages,
      abortSignal: signal,
      stopWhen: stepCountIs(15),
    }),
});

No database schema for messages, no Redis for durable streams, no separate worker process to coordinate. The properties Trigger.dev advertises:

Survives refreshes, redeploys, and crashes. A conversation in progress when you ship a deploy keeps streaming on the version it started on, and you move it to new code explicitly via a version-upgrade flow rather than getting yanked mid-turn.
Multi-turn history for free. Each turn is a step inside the same durable task, so the server accumulates conversation history and the client only ever ships the new message.
Fast first response despite the durability. An opt-in "Head Start" path runs the first streamText call in your own warm server process while the durable agent boots in parallel, so "durable" doesn't mean "slow to start."
Tool calls, human-in-the-loop, sub-agents. Tools can be marked needsApproval: true, which pauses the run — for days if needed — until a human approves, with zero compute billed while it waits.
Built-in tracing. Every turn is a span in the Trigger.dev dashboard, with a dedicated AI metrics view for cost, tokens, and latency.

How it actually works

This is where the launch page undersells itself, because the mechanism is more interesting than "we made chat durable." Per Trigger.dev's own architecture docs, a chat session is three things bound together:

An inbox channel (.in) that every user message lands in.
An outbox channel (.out) that every assistant chunk leaves through.
A long-lived agent task that reads .in and writes .out.

Both channels are backed by S2, a durable append-only log service — think a pair of per-session Kafka topics. Records get monotonically increasing sequence numbers, and readers resume from a cursor. That's the actual trick behind "refresh the page and the stream picks back up": your browser reconnects to .out with a Last-Event-ID, and the server replays only the chunks it missed, without re-running the LLM call. Nobody pays for the model twice because a laptop went to sleep mid-response.

The task itself moves through a small state machine:

Streaming — actively running streamText(), piping chunks to .out.
Idle — the turn finished, the task is alive and parked on a wait for the next message, but doing no work.
Suspended — after a configurable idle timeout (30 seconds by default), the engine checkpoints the entire process state — variables, in-memory caches, any sub-agent spawned mid-conversation — and frees the compute. The session row is still live and the .out stream is still readable; there's just no machine assigned to it.
Resuming — the next message restores the run from its checkpoint exactly where it left off, no cold-boot work, no re-initialization.

If the run has fully exited instead of merely idling — it hit a turn cap, the code called endRun(), it crashed, or it was cancelled — there's nothing left to resume, so the platform starts a brand-new run, restores conversation state from an S3 snapshot, and replays anything that arrived after that snapshot was taken. If the previous run died mid-stream with a half-written response sitting in .out, the framework splices that partial answer plus the message that triggered it into the new run's context, so a follow-up still has the full picture.

That checkpoint-resume system isn't chat-specific — it's the same mechanism Trigger.dev uses for every long-running task on the platform, which is presumably why a brand-new feature can already claim 84 years of aggregate compute: it's new orchestration logic sitting on old, exercised infrastructure.

There's a second layer worth noting for anyone building a real product on top of this: multi-tab handling. Open the same chat in two browser tabs and the transport uses BroadcastChannel to have the sending tab "claim" the chat ID; other tabs flip into a real-time, read-only state until the turn completes, with a 10-second heartbeat to release the claim if a tab crashes. That's the kind of bug class (duplicate sends from two open tabs, stale UI in a background tab) that's tedious to get right by hand and easy to skip until a support ticket forces the issue — here it's a config flag.

The performance number worth citing: Trigger.dev's own benchmark for the "Head Start" fast-path, measured on claude-sonnet-4-6 with the same model on both sides of the comparison, shows time-to-first-token going from 2801ms to 1218ms (a 57% cut) and total turn time from 4180ms to 2345ms (44% faster). The mechanism is straightforward — the first LLM call runs in your already-warm Next.js/Hono/SvelteKit server while the durable agent boots in parallel, and ownership of the stream only hands over to the durable side once the model wants to call a tool.

What changed versus rolling it yourself

Before something like this, "durable AI chat" meant assembling it from parts: Postgres for message history, Redis (or a database LISTEN/NOTIFY setup) for the durable stream so a browser refresh doesn't drop in-flight tokens, a background worker to keep the LLM call alive past your API route's timeout, and hand-rolled idempotency so a duplicate webhook or a flaky reconnect doesn't double-append a message. None of that is exotic engineering, but all of it is boilerplate every team building a serious AI chat product ends up writing roughly the same way, and getting subtly wrong roughly the same ways (dropped chunks on reconnect, double-charged LLM calls on retry, message ordering bugs under concurrent tabs).

Trigger.dev's answer is to fold that into infrastructure and expose two moves — declare the task, point useChat at it — while also handling a case DIY stacks usually never handle at all: needsApproval tools that can pause a run for days waiting on a human, at zero ongoing compute cost, then resume with full in-memory state intact. That's a meaningfully higher bar than "the chat history is in a database somewhere."

Why developers should actually care

Cost model. This is the least-marketed and most consequential detail. Because idle and suspended time is checkpointed and freed rather than held open, you are not paying server-uptime economics for a conversation that's just sitting there between messages. Compute is billed per second by machine size, from $0.0000169/sec for a 0.25 vCPU "Micro" machine up to $0.00068/sec for an 8 vCPU "Large 2x," plus a flat $0.000025 per run invocation. A support chat that a customer abandons for six hours and then resumes costs you six hours of storage for a session row, not six hours of compute. If you've ever run a WebSocket server that holds a connection (and a worker) open for every idle chat tab, this is a genuinely different economic shape.

Latency. The Head Start numbers above are real and specific, not marketing rounding — a 44% total-turn-time cut on identical models is worth having if you're optimizing perceived responsiveness, which for a chat product is most of the product.

DX. No API route to write, retry, or rate-limit by hand; the AI SDK's useChat just works against a different transport.

Lock-in. This is where you should slow down. chat.agent is not "durable execution with a chat flavor" — it's a specific session model built on Trigger.dev's own primitives (S2-backed streams, S3 snapshots, their checkpoint engine). There's no adapter that lets you point useChat at Inngest or Temporal instead; adopting this couples your chat backend's persistence and resume semantics to Trigger.dev's infrastructure. Trigger.dev is Apache-2.0 licensed and self-hostable, which is a real mitigation — but self-hosting an S2-backed durable stream is a materially heavier operational commitment than self-hosting a stateless job runner, and the public docs don't spell out what a self-hosted S2 story looks like versus using their managed cloud.

Security posture for tool calls. The needsApproval gate on tools — pause the run, hold state, wait for a human — is the right shape for anything that touches money or destructive actions (their own example is a refundOrder tool), and it's cheap to leave in place because a paused run costs nothing while it waits. That's a better default than teams typically build for themselves under deadline pressure.

Practical use cases

Customer support agents with real refund/cancel authority, gated by the needsApproval HITL flow so a human signs off before money moves, without the agent losing its accumulated conversation context while it waits on that approval.
Long-horizon coding or research agents where a "turn" might legitimately take minutes, and the user is expected to close the laptop and come back — the exact case serverless timeouts (Vercel's 800 seconds, Lambda's 900) are hostile to.
Multi-agent products where a parent chat spawns sub-agents — Trigger.dev's AgentChat pattern lets a sub-agent run as its own durable session whose output streams back through the parent's tool card, useful for anything shaped like "the assistant delegates to a specialist and shows its work."
Any AI product currently duct-taping Redis pub/sub to Postgres to get resumable streaming — this is a direct, drop-in replacement for that specific stack if you're willing to take the coupling.

What the launch page leaves out

A few things worth knowing before you commit a production chat feature to this:

It's genuinely new as a public product, even if the code isn't. GA since July 2, 2026 is barely over a month of public availability at the time of its Product Hunt debut. The "84 years of compute" figure is real signal that the underlying engine is exercised, but the chat-specific API surface — hooks, transport, multi-tab coordination — has had a much shorter public runway to find edge cases than the phrase "battle-tested" implies.
There are documented rough edges today. Trigger.dev's own docs flag a known bug where React Strict Mode double-fires the resume effect in useChat, throwing a caught-but-visible TypeError in dev consoles (it's an upstream AI SDK issue, with a fix PR filed but not yet merged as of this writing). Their own docs also note that useChat's built-in stop() currently doesn't work correctly after a stream has been resumed — you have to call a separate stopGeneration method to reliably kill generation. Small, documented, workaroundable — but not the frictionless story the marketing implies.
Concurrency limits gate how many conversations can be "live" at once, not just how much you spend. The free tier caps at 20 concurrent runs, Hobby at 50, Pro starts at 200-plus with metered overage. An active (Streaming or Idle) chat occupies a concurrent-run slot; Suspended chats don't. If your product can spike to hundreds of simultaneously active conversations, plan around that ceiling explicitly rather than discovering it in production.
The self-hosting story for this specific feature is under-specified. Trigger.dev is open source and self-hostable in general, but chat.agent depends on S2 for its durable streams — the public-facing self-hosting guide doesn't detail what running that dependency yourself entails versus relying on Trigger.dev's managed S2 usage.
The tooling around this is still being hardened in the open, which is a good sign to check yourself rather than take on faith. The reference implementation shipped alongside chat.agent includes MCP tooling that lets IDE agents (Claude Code, Cursor) drive a deployed chat task directly from the editor. In the pull request that introduced it, a reviewer flagged that the in-memory map tracking active chat sessions had no eviction policy and would grow unbounded, and recommended adding an idle sweeper and an LRU cap before merge. That's a normal, healthy thing to see caught in code review — but it's also a reminder that a feature announced as running on "84 years of compute" still has actively-reviewed rough edges in its newest surfaces, and it's worth reading the changelog rather than assuming everything shipped is equally seasoned.

How it compares

The nearest neighbors aren't chat products, they're durable-execution platforms, because that's the underlying category chat.agent sits inside: Inngest, Temporal, and Restate.

Inngest is the closest philosophically — TypeScript/Python-first, event-driven, step.run() wraps each side-effecting block so a crash resumes from the last completed step. It's a strong fit if you're already event-driven, but its billing counts the function run and every durable step as separate executions, so a chatty 20-turn agent with a model call and a tool call per turn can rack up roughly 40+ billed step executions for one conversation — a materially different cost shape than Trigger.dev's per-second machine billing.
Temporal is the heavyweight: workflows-as-code with deterministic replay, used for complex mission-critical orchestration across many languages. It's more powerful and more operationally involved — you're running a Temporal cluster or paying for Temporal Cloud and adopting its programming model, which is a much bigger lift than either of the TypeScript-native options for a team that just wants a chat backend.
Restate is the newer, lighter-footprint entrant — a single binary, durable functions and virtual objects, less operational overhead than Temporal. It's a durable-execution primitive, not a chat product; you'd still be building the chat-session and frontend-transport layer yourself on top of it.

None of the three ship a ready frontend ChatTransport for the AI SDK's useChat out of the box the way Trigger.dev now does. That's the actual differentiation — not "durable execution" (all four platforms do that), but "durable execution plus an opinionated, pre-wired chat protocol," which collapses a specific and common integration most teams currently build by hand regardless of which durable-execution backend they picked.

Independent read

The engineering here is legitimately well thought through — the split between engine checkpoints (survive an idle gap, same run), chat snapshots (survive a full run exit, new continuation run), and a client-side lastEventId cursor (survive a tab refresh, no server involvement needed) maps cleanly onto three genuinely different failure modes that most homegrown implementations conflate into one "just retry" pile. The Head Start latency numbers are specific and plausible rather than rounded-marketing plausible. And "waiting costs nothing" is the detail that actually makes human-in-the-loop approval workflows viable in a way that's usually just aspirational in agent demos.

The trade you're making is real, though: you're adopting a session and streaming model that is Trigger.dev's own invention, running on Trigger.dev's own infrastructure (S2, S3 snapshots, their checkpoint engine), with a chat-specific API surface that's a little over a month old in public. If Trigger.dev the company has a bad year, or S2 has an outage, or the API shape changes under you before it settles, you feel it directly in your product's chat feature, not in a swappable dependency. That's the same bet you make adopting any managed durable-execution platform — it's just worth naming plainly rather than discovering later.

Who should try it, wait, or skip it

Try it now if you're building an AI chat product from scratch, especially one with tool calls that touch real actions (refunds, bookings, code changes) where the HITL pause-for-days behavior is a feature, not a workaround. The integration cost is genuinely low — a task definition and a transport hook — so the cost of trying it against a side feature or a new product is small.
Wait a release or two if you're migrating an existing production chat feature off a working Redis+Postgres stack. The core mechanism is sound, but the two known rough edges (Strict Mode resume bug, stop() after resume) are exactly the kind of thing you want someone else to hit first, and a few more months of public GA will tell you whether the concurrency limits and S2 dependency become real friction at scale.
Skip it if avoiding vendor lock-in on your core chat infrastructure is a hard requirement, or if you need multi-language support beyond TypeScript — this is a TypeScript-native SDK on a TypeScript-native platform, and there's no path to a Python or Go backend here the way there is with Temporal.

If you're currently maintaining a hand-rolled version of this — Redis for durable streams, Postgres for history, a worker process fighting your serverless platform's timeout — the honest question worth sitting with is how much of that stack you built because you needed the control, versus how much you built because nothing shipped this as a primitive yet. Which part of your setup would you actually miss if it disappeared tomorrow?

Sources:

DEV Community