Stop Keeping Agent Tool Calls Open While the Work Finishes

Some AI tool calls finish in milliseconds.

Others do not.

A database export may take 20 seconds.
A CRM sync may take longer.
A report might need several background steps.
A refund may have to move through another payment system before it reaches a final state.

If the agent keeps the original interaction open while all of that happens, the architecture starts paying for it.

Long-lived connections accumulate. Timeouts get awkward. Retries become harder to reason about. The user is left waiting for work that could have continued independently.

One part of the newer MCP direction that deserves more attention is its treatment of long-running work as explicit tasks.

That is a useful production pattern even outside MCP.

A tool call and a completed job are not always the same event

It is tempting to design an agent tool like this:

User asks for action
        ↓
Agent calls tool
        ↓
Tool starts work
        ↓
Connection stays open
        ↓
Work finishes
        ↓
Final result returned

That is perfectly reasonable for a quick database lookup or lightweight API call.

It becomes less attractive when the operation takes 10, 30, or 60 seconds.

Now the request lifecycle and the work lifecycle are forced to stay together.

A temporary network problem can interrupt the response.

A client timeout can make successful work look like failure.

A retry can accidentally start the same expensive action twice.

And the user has to wait even when there is nothing useful happening in the foreground.

Return task state instead

A cleaner pattern separates starting the work from finishing the work.

User asks for action
        ↓
Agent calls tool
        ↓
Server creates task
        ↓
Server returns immediately
        ↓
Work continues in background
        ↓
Client checks task state
        ↓
Result becomes available

Now the interaction can continue while the job runs.

The server does not need to pretend that a long-running operation is an ordinary request-response cycle.

The task becomes something the application can inspect.

That difference is small on paper and extremely useful in production.

Give every long-running action an identity

Once work becomes asynchronous, it needs an explicit identifier.

A simplified task record might look like this:

type AgentTask = {
  id: string;
  type: string;
  status: "queued" | "working" | "completed" | "failed";
  createdAt: string;
  updatedAt: string;
  result?: unknown;
  error?: string;
};

When a tool starts expensive work, it can create the task first:

async function startReportGeneration(caseId: string) {
  const task = await taskStore.create({
    type: "generate_report",
    status: "queued",
    input: { caseId }
  });

  queue.publish("generate_report", {
    taskId: task.id,
    caseId
  });

  return {
    taskId: task.id,
    status: "queued",
    message: "Report generation has started."
  };
}

The agent gets a useful response immediately.

The actual work moves somewhere designed to handle long-running execution.

The task store becomes the source of truth

This is where the architecture gets much easier to operate.

Instead of relying on one connection staying alive, the system can ask:

What is task 7f21 doing right now?

and receive something like:

{
  "taskId": "7f21",
  "status": "working",
  "progress": "extracting_sources"
}

Later:

{
  "taskId": "7f21",
  "status": "completed",
  "resultId": "report_842"
}

That state can survive:

• client reconnects
• server restarts
• load-balancer rerouting
• background-worker changes
• longer execution times

The user-facing conversation no longer has to own the lifecycle of the work.

Retries also become safer

Long-running agent actions often have side effects.

That changes how retries should behave.

Imagine a refund tool.

If the user connection times out after the payment provider accepted the refund, blindly retrying the original tool call could submit the refund twice.

A task-oriented system gives you somewhere to attach idempotency and execution state.

For example:

async function processRefundTask(taskId: string, orderId: string) {
  const task = await taskStore.get(taskId);

  if (task.status === "completed") {
    return task.result;
  }

  await taskStore.update(taskId, {
    status: "working"
  });

  try {
    const result = await payments.refund({
      orderId,
      idempotencyKey: taskId
    });

    await taskStore.update(taskId, {
      status: "completed",
      result
    });
  } catch (error) {
    await taskStore.update(taskId, {
      status: "failed",
      error: String(error)
    });

    throw error;
  }
}

Now the system has a durable answer to:

Did this action already happen?

That question matters much more than simply asking whether an HTTP request succeeded.

The user experience gets better too

Async architecture is not only an infrastructure concern.

It changes what the product can tell the user.

Instead of:

Please wait...

for 40 seconds, the application can say:

Your report is being generated. You can continue working while it finishes.

The conversation can continue.

The user can leave and come back.

The interface can show progress.

A failed task can expose a retry path without pretending the entire conversation failed.

That is a much better fit for agent products where tools increasingly perform meaningful business actions instead of quick information retrieval.

Observable states make operations easier

I would avoid a task model that only has:

pending
done

Long-running workflows become much easier to debug when the important states reflect what the system is actually doing.

For example:

queued
↓
validating_input
↓
fetching_sources
↓
generating_output
↓
saving_result
↓
completed

A failure can now tell you where it happened.

status: failed
stage: fetching_sources

That is much more useful than:

tool failed

It also gives you better operational metrics:

• queue wait time
• execution time
• failure rate by stage
• retry count
• abandoned tasks
• completion time by tool

Those signals become important once agent actions are part of a production workflow.

Not every tool needs this

There is no reason to turn a 150 ms lookup into a background task.

Synchronous tool calls are still perfectly reasonable when:

• execution is predictably fast
• retries are simple
• there are no expensive side effects
• the result is needed before anything else can continue

Async tasks become more useful when:

• execution can take several seconds or longer
• work depends on external systems
• the action has side effects
• users do not need the final result immediately
• the job should survive reconnects or infrastructure changes
• progress or retry state matters

The boundary should follow the behaviour of the work, not a blanket architecture rule.

MCP is moving in this direction too

Google's recent write-up on the 2026-07-28 MCP specification discusses a Tasks extension for long-running tool execution.

Instead of keeping the client blocked while the operation finishes, a tool can return a task identifier while execution continues separately. The client can then inspect or receive updates about that task as it progresses.

The same update also describes multi-round-trip interactions where required user input can be represented explicitly and the operation resumed later.

Those changes point toward a useful broader pattern:

agent interactions and business operations do not need to share the same lifetime.

A conversation may last minutes.

A request may last milliseconds.

A background job may last longer than either.

Treating those as separate lifecycles makes the system much easier to reason about.

A practical agent execution model

For longer-running tools, I like this separation:

Conversation layer
Handles:
- user interaction
- tool selection
- acknowledgement
- final explanation

        ↓

Task layer
Handles:
- task identity
- status
- retries
- progress
- result references

        ↓

Worker layer
Handles:
- expensive processing
- external APIs
- side effects
- long-running operations

Each layer has a clear job.

The agent does not need to sit on an open connection while the worker finishes.

The worker does not need to understand the whole conversation.

And the product has an explicit record of what is happening in between.

Final thought

As AI agents start doing more than retrieving information, more tool calls will behave like jobs rather than ordinary API requests.

Treat them that way.

Start the work.

Give it an identity.

Track its state.

Let the conversation continue.

Then bring the result back when it is ready.

That is much easier to operate than asking one request to stay alive for the entire lifetime of the action.

Source

Google Developers Blog
Scaling AI Agent Infrastructure with the MCP Stateless updates

https://developers.googleblog.com/scaling-ai-agent-infrastructure-with-the-mcp-stateless-updates/