KristinZ

Posted on Nov 1 Edited on Nov 1

Your AI Agent Doesn't Belong Inside an HTTP Request

#typescript #ai #webdev #agents

A user sends a message to your AI application.

Your server receives:

POST /api/chat

So you do what seems natural:

app.post('/api/chat', async (c) => {
  const result = await runAgent(await c.req.json());

  return c.json(result);
});

It works.

Then your Agent gets smarter. Instead of one LLM call, it starts doing this:

User
  │
  ▼
Agent
  │
  ├── LLM call
  ├── search
  ├── database query
  ├── tool call
  ├── another LLM call
  ├── file generation
  └── final LLM call

The request that used to take 5 seconds now takes 30 seconds, then 2 minutes, then 10.

The HTTP request is still there, but the HTTP connection is a poor place to represent the lifetime of an Agent.

The Problem With Connection-Bound Agents

Consider this architecture:

Browser
   │
   │ POST /api/agent
   ▼
Node.js
   │
   │ await
   ▼
Agent
   │
   ├── LLM
   ├── Tool
   ├── LLM
   ├── Tool
   └── LLM
   │
   ▼
HTTP response

The entire Agent run is coupled to one HTTP request. That raises an uncomfortable question:

What happens when the browser disconnects?

A server can detect a client disconnect through the request's AbortSignal and combine it with a server timeout:

const signal = AbortSignal.any([
  timeoutSignal,
  c.req.raw.signal,
]);

Then it passes the signal into the upstream LLM request:

await fetch(LLM_URL, { signal });

For a normal streaming chat response, this is what you want. If the user closes the tab halfway through, continuing to generate is wasteful.

An Agent is different.

A Chat Response Is Not an Agent Run

Compare two operations.

Operation A: Generate an answer

User
  │
  ▼
POST /chat
  │
  ▼
LLM
  │
  ▼
stream response

The operation exists to serve the current connection. If the user leaves, cancelling the generation makes sense.

Operation B: Run an Agent

User
  │
  ▼
POST /agents
  │
  ▼
Agent
  │
  ├── search web
  ├── inspect documents
  ├── call tools
  ├── write files
  ├── run code
  └── generate final report

This is no longer a response. It is a task, and tasks have their own lifecycle.

The Browser Is Not the Owner of the Task

Tying an Agent to the connection says:

"This Agent exists as long as this browser connection exists."

What the user actually asked is:

"Perform this task for me."

Those are different lifetimes. The browser connection might last 10 seconds while the task takes 10 minutes:

Browser connection
       │
       │ short-lived
       ▼
HTTP request

                    Agent task
                       │
                       │ long-lived
                       ▼
                  Task lifecycle

This is the same boundary the previous article ran into with cancellation. A client disconnect does not prove the server cancelled the work, and a server cancellation does not prove the provider stopped inference. Each of those is a separate fact.

Now take it one step further. Suppose the Agent has already searched, queried the database, and generated a file, and then the user closes the browser. If the request was:

"Research these 20 documents and generate a report."

you probably don't want:

Browser closed
     ↓
Agent killed
     ↓
report lost

The user didn't cancel the task. They closed the UI.

The Task Gets Its Own Identity

Instead of running the Agent inside the request, the request creates a task and returns:

POST /api/agents

{
  "taskId": "task_01J..."
}

The HTTP request is finished. The Agent continues on its own:

HTTP request
     │
     ▼
create task ──────────► task_123
     │                      │
     ▼                      ▼
HTTP response             Agent
                            │
                            ├── LLM
                            ├── Tool
                            └── LLM

The task ID is an identity for the work itself:

Before:  HTTP request = work
After:   task ID      = work

That lets you store the task and make the database, not the connection, the source of truth:

type AgentTask = {
  id: string;
  status: 'queued' | 'running' | 'completed' | 'failed' | 'cancelled';
  createdAt: Date;
  startedAt?: Date;
  completedAt?: Date;
  error?: string;
};

The browser stops owning the Agent. It owns a subscription to the Agent's progress:

POST /api/agents                  → task_123
GET  /api/agents/task_123         → current status
GET  /api/agents/task_123/events  → live progress

A client can connect, disconnect, and reconnect. The task doesn't care.

Reconnection becomes ordinary. The user starts an Agent at 12:00 and the browser crashes at 12:01. At 12:05 the user reopens the app, and the frontend asks:

GET /api/agents/task_123

{
  "id": "task_123",
  "status": "completed"
}

Nothing required the original TCP connection to survive five minutes.

The code for this article is a runnable demo: aiengineering-book/agent-task-demo. It runs the same simulated Agent bound to a request and as a task, drops the connection partway through, cancels at different points, evicts a subscriber that stops reading, and checks what survives each time.

Streaming Becomes a Subscription

This doesn't mean you should stop streaming. Streaming is still the right way to deliver progress. What changes is what the stream represents:

Streaming becomes a view of a task, not the task itself.

The task emits events:

task.started
tool.started
tool.completed
llm.started
llm.delta
llm.completed
task.completed

and any number of browsers can render them as they arrive:

                  Agent task
                      │
                      │ events
                      ▼
              ┌───────────────┐
              │ Event stream  │
              └───────┬───────┘
                      │
              ┌───────┴────────┐
              ▼                ▼
          Browser A        Browser B

An SSE response is a subscription to task_123. It is not where the Agent lives:

Task lifecycle
───────────────────────────────────────►

        Agent running

Browser A
───────────────X

                         Browser B
                         ────────────────►

The subscription can disappear without destroying the work. The LLM stream itself moves one level down: it feeds the task's event stream instead of feeding the HTTP response directly.

Once the task is independent of HTTP, the server no longer has to execute it either. The API accepts the request, validates input, creates the task, and hands it to a worker. The worker owns execution.

Disconnect and Cancel Are Different Operations

With a task, "the browser went away" and "the user wants this stopped" stop being the same event.

Disconnect

Browser
   X
   │
   ▼
subscription ended
Agent continues

Cancel

POST /api/agents/task_123/cancel

Now the user has said explicitly: stop this task. That is a different semantic, and it lets you keep several policies side by side:

Event	What happens
User disconnects	The subscription ends. The Agent continues.
User presses Stop	The task is cancelled.
Task runs longer than its limit (for example 30 minutes)	The task is cancelled.
Server shuts down (SIGTERM)	Task state is persisted. The task resumes later.

These are four different events. Collapsing them into one AbortSignal hides the difference between them.

A cancel request also needs a precise answer. "Cancelled" can mean different things depending on how far the work had got, and a status that hides the difference is misleading:

Outcome	Meaning
Client detached	The subscription ended. Nothing was cancelled.
Queued work cancelled	The task had not started. Nothing ran.
Tool call cancelled	A running step stopped before it finished.
Side effect already committed	A step finished before the cancel arrived and cannot be undone.

Return which of these happened. In the demo, the cancel endpoint answers step_cancelled or side_effect_committed. Give the task an ID that appears in every log line across the browser, the API, the worker, and the provider call, so the sequence can be reconstructed afterwards.

What You Take On

Decoupling the task from the connection doesn't make the hard parts disappear. It moves them somewhere you have to design for.

Dead subscribers. A subscription that goes quiet without a clean close can leave a stale connection holding resources, and proxies and load balancers close idle connections. Send periodic heartbeats on the event stream so a healthy connection stays open. Heartbeats do not help you spot a dead one. On a one-way stream the server never hears back, and its own writes keep the socket looking active, so an idle timer cannot tell a quiet client from a gone one. What gives a stuck reader away is backpressure: its socket stops draining, writes back up, and the server can drop that subscription after a short drain timeout. The demo does exactly that, and the task keeps running.
Task state. A task moves through queued, running, and one of completed, failed, cancelled. The task decides these states, not the HTTP connection.
Worker failure. If a worker crashes mid-task, the task is still running in the database. Something has to notice that the worker disappeared, and then retry, resume, or mark the task failed.
Durable steps. An Agent that searches, writes a file, calls an API, updates a database, and sends an email can crash after step four. Restarting blindly repeats work, so the task has to record which steps are done.
Idempotency. If the Agent sends an email and crashes before recording that it did, a retry sends it twice. Steps with side effects need idempotency keys or a record of completion.
Tools that outlive the abort. If the model has emitted a tool call, cancelling the LLM stream does nothing for the work that tool already started. Tools need to check the task's cancellation signal themselves, and be safe to run twice.

None of these are specific to AI. They are the standard problems of background work, and Agents run into them early because a single Agent run is already many asynchronous steps. Each deserves more space than this article can give it.

When to Bind Work to HTTP

Not every operation needs a task. For plain chat:

request
  ↓
LLM
  ↓
stream
  ↓
response

binding the work to the connection is reasonable, and cancelling when the user leaves is often exactly right.

The previous article measured one provider on one day: DeepSeek, streaming chat requests, aborted at two points. Billing followed the abort point and stayed far below the cost of running each request to its token limit. For plain chat in that test, cancel-on-disconnect was cheap and effective. That is one provider, not a general statement, but it points to where the real reason to decouple Agents lies. It isn't cost. It is that an Agent run has side effects and results the user still wants after the tab is gone.

Ask one question:

If the user closes the browser, should the work still exist?

If the answer is no, use connection cancellation. If the answer is yes, create a task and return its ID.

You can sort most AI operations into three patterns:

Pattern	Shape	Examples	Disconnect
Request-bound	HTTP → LLM → HTTP response	chat completion, short transformations, simple streaming	Can cancel the work
Persisted	HTTP → create work → storage → background processing	document indexing, file processing, generation written to history	Should not cancel the work
Resumable Agent task	HTTP → task ID → worker (LLM, tools, checkpoints, state)	research agents, coding agents, multi-step workflows	The UI is just a client of the task

The rule isn't "never run an Agent inside an HTTP request". It is:

Don't bind the lifetime of durable work to the lifetime of an ephemeral connection.

Final Takeaway

The first generation of AI applications looked like this:

HTTP request → LLM → HTTP response

That is fine for chat. Agents introduce something different:

HTTP request → create task → Agent → many asynchronous operations → durable result

Once work can outlive the browser connection, the connection should no longer own it:

┌──────────────────┐
│   Connection     │
│                  │
│ short-lived      │
│ disposable       │
│ reconnectable    │
└────────┬─────────┘
         │
         │ subscribes to
         ▼
┌──────────────────┐
│      Task        │
│                  │
│ durable          │
│ cancellable      │
│ resumable        │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Agent Runtime   │
│                  │
│ LLMs             │
│ Tools            │
│ State            │
│ Checkpoints      │
└──────────────────┘

The browser connection is ephemeral. The Agent task is the thing that matters.

The rule

If the work should survive the connection, don't make the connection own the work.

DEV Community