AI Agents and Long-Running API Calls: Polling vs Webhooks

Designing Async APIs Agents Can Actually Follow

The agent calls your video transcode endpoint. The endpoint returns 202 Accepted and a job ID. The agent, which has no idea what 202 means in your system, reports that the transcode is complete and moves on to the next step—which reads a file that does not exist yet.

Try Apidog today

Long-running operations break agents in predictable ways. A synchronous call has a clear contract: send, wait, receive. An asynchronous call splits that contract into “start” and “finish,” creating a gap where agents may:

• Declare success too early
• Poll in a tight loop
• Hold a conversation turn open for several minutes

This guide shows how to design async contracts agents can follow, when to poll versus hand off, how to write reliable tools, and how to test slow and failed jobs. For general API failures, see agent error recovery.

Why agents mishandle async operations

Three habits cause most failures.

1. Models treat any 2xx response as completion

202 Accepted means that processing has started or been accepted; it does not mean the work is finished. The HTTP semantics specification is explicit about this.

Models trained on ordinary request-response traffic often interpret any 2xx as success unless the response body clearly says otherwise.

2. Polling inside the reasoning loop is expensive

If an agent polls every two seconds for a four-minute job, it may create 120 model turns. Each turn consumes tokens and context. See how tool responses fill the context window.

3. Job IDs can disappear from context

A start tool creates state that the agent must carry forward. If the job ID is buried in a long conversation, compaction can remove it and the agent may forget that work is still in flight.

Design responses the model cannot misread

The most effective fix is explicit wording in the response body:

{
  "status": "processing",
  "job_id": "job_7f21c",
  "message": "The transcode has STARTED and is NOT complete. Do not report success. Check status with getJobStatus(job_id) after at least 30 seconds.",
  "poll_after_seconds": 30,
  "estimated_duration_seconds": 240,
  "status_url": "/v1/jobs/job_7f21c"
}

This may feel heavy-handed for human API consumers, but models follow direct instructions more reliably than they infer meaning from a status code.

Three details matter:

• Say “not complete”
• Name the next tool
• Provide a minimum wait time

Google’s AIP-151 describes a consistent resource shape for long-running operations: one operation object with done, error, and response fields. Using the same shape across slow endpoints lets an agent reuse one polling pattern.

Make status responses equally explicit:

{
  "job_id": "job_7f21c",
  "status": "processing",
  "done": false,
  "progress_percent": 45,
  "elapsed_seconds": 108,
  "poll_after_seconds": 45,
  "message": "Still processing. Do not proceed to the next step."
}

When the result is small, return it inline on completion:

{
  "job_id": "job_7f21c",
  "status": "succeeded",
  "done": true,
  "result": {
    "output_url": "https://cdn.example.com/out/7f21c.mp4",
    "duration_seconds": 372
  }
}

Poll outside the model

The most important implementation choice is to put waiting in the tool wrapper—not in the agent’s reasoning loop.

import time

def start_and_await_transcode(client, source_url, max_wait=600):
    job = client.post("/v1/transcode", json={"source_url": source_url}).json()
    job_id = job["job_id"]
    delay = job.get("poll_after_seconds", 5)
    waited = 0

    while waited < max_wait:
        time.sleep(delay)
        waited += delay
        status = client.get(f"/v1/jobs/{job_id}").json()

        if status.get("done"):
            if status["status"] == "succeeded":
                return {"status": "succeeded", "result": status["result"]}
            return {"status": "failed", "error": status.get("error")}

        delay = min(int(delay * 1.5), 60)

    return {
        "status": "timed_out",
        "job_id": job_id,
        "message": f"Still running after {max_wait}s. Job {job_id} continues in the background.",
    }

From the model’s perspective, this is one tool call that eventually returns a final result. There is no polling loop in context, no forgotten job ID, and no unnecessary model turns.

The backoff keeps requests manageable, while the timeout prevents a stuck job from blocking the run forever. Before tuning these values, review AWS’s guidance on timeouts, retries, and backoff with jitter.

Follow two rules:

1. Always cap the wait.
2. Always return the job ID on timeout.

Never return an ambiguous result. Expose distinct states:

• succeeded
• failed
• timed_out

For jobs that run for hours, use two tools instead:

1. A tool to start the job
2. A tool to check its status

Store the job ID, associated task, and start time outside the conversation. The agent can read the in-flight job list at the beginning of each run.

When webhooks are better

Polling is simple and works almost everywhere. Webhooks are more efficient but require more infrastructure. See this webhooks versus polling comparison.

Use polling when:

• The job takes seconds to minutes
• The agent must wait before continuing
• You cannot host a public endpoint

Use webhooks when:

• Jobs take hours
• The agent should start work and move on
• Many concurrent jobs make polling wasteful

Webhooks require a public receiver, signature verification, retry handling, and a way to wake the agent when the callback arrives. These guides cover the foundation:

• Designing reliable webhooks
• Webhook signature verification

A middle ground is server-sent events (SSE), which stream progress over an open connection without requiring a public callback endpoint. SSE works well for interactive agents where a human is watching. See streaming API responses with SSE.

Whatever mechanism you choose, make completion handling idempotent. Webhooks retry, polls can race, and receiving succeeded twice must not start the downstream step twice. See idempotency keys for AI agents.

Test the slow path

Async bugs stay hidden when test environments are fast. A production job that takes four minutes may finish in 200 milliseconds against a local stub, so the agent never experiences the real state transitions.

Build these four scenarios deliberately:

A genuinely slow job

Return processing for several status calls, then return succeeded. Verify that the wrapper polls, backs off, and eventually returns the result.

In Apidog, use a mock that varies by request count or a control parameter so the scenario is deterministic.

A job that fails late

Return processing three times, then return failed with an error body. The agent must report failure—not confuse “the job finished” with “the job succeeded.”

A timeout

Keep returning processing beyond the wrapper’s ceiling. Assert that the tool returns timed_out with the job ID intact.

A duplicate completion

Deliver the same success twice, either through a webhook retry or a racing poll. Assert that the downstream step runs only once.

Save all four scenarios and run them in CI. The API contract testing guide covers the broader approach.

Three jobs that expose the problem

Report generation

A finance agent requests a quarterly export that takes 90 seconds. With a naive tool, the agent receives a job ID, announces that the report is ready, and sends the user a broken download link.

With a blocking wrapper, it waits 90 seconds and returns the real URL. The API is unchanged; only the location of the waiting differs.

Bulk imports

An operations agent uploads 20,000 records. The import runs for eight minutes and partially fails on row 14,000.

A naive success check sees that the job is done and assumes success. Instead, return partial outcomes explicitly:

• Processed count
• Succeeded count
• Failed count
• A URL for detailed errors

Make the agent inspect those fields before proceeding.

Model and build pipelines

An agent starts a training run or CI build that takes 40 minutes. In-wrapper polling is the wrong shape because it holds a conversation turn open too long.

Start the job, record its ID durably, end the turn, and use a scheduled check or callback to trigger the follow-up. See multi-agent handoff and context passing.

Give partial results an explicit shape

Long jobs often finish between success and failure. A two-state model forces you to misrepresent the outcome.

Use a third state:

{
  "job_id": "job_a11f",
  "status": "completed_with_errors",
  "done": true,
  "summary": {
    "processed": 20000,
    "succeeded": 19860,
    "failed": 140
  },
  "errors_url": "/v1/jobs/job_a11f/errors?limit=50",
  "message": "Import finished. 140 rows failed and were not written. Review errors before reporting success."
}

The counts are inline, so the agent can decide without another request. The detailed rows remain behind a limited URL, preventing hundreds of error objects from entering the context unexpectedly.

Make stalled jobs visible

A timeout should return a job ID and state that the work is still running. That result is useful only if it reaches someone responsible for checking it.

If the agent is your own service, route the result to the queue your team already monitors. If it runs through a coding platform, use the platform’s task and execution state.

For example, in Sharkly, a blocked run remains attached to its task with execution state and result, while the Inbox separates items requiring human reply or review from ordinary updates.

The specific platform is less important than the principle: “still running, check later” needs an owner.

Checklist

• Every slow endpoint returns a job ID, status URL, and plain-language message saying the work is unfinished.
• Status responses include a boolean done field.
• Polling runs in the tool wrapper with exponential backoff and a hard ceiling.
• Timeouts return the job ID so work can resume.
• Success, failure, and timeout are distinct return values.
• Jobs longer than a few minutes are recorded outside the conversation.
• Completion handling is idempotent for both polls and callbacks.
• Slow, late-failing, timed-out, and duplicated completions have saved tests.

With explicit response wording and a wrapper that owns the waiting, long-running operations stop being a special case. The agent calls one tool, waits, and receives an unambiguous result—the contract it handles best.

Download Apidog to build slow-job mocks alongside your tests.

Frequently asked questions

Should an async start endpoint return 202 or 200?

Return 202 Accepted. It honestly signals that processing is not finished. Do not rely on the status code alone; include explicit instructions in the response body as well.

How long should the tool wrapper wait?

Set the ceiling slightly above the endpoint’s realistic worst case, commonly two to ten minutes. Beyond that, a check-later tool is usually a better shape than blocking the conversation.

What polling interval should I use?

Start with the server’s poll_after_seconds hint. Then back off by roughly 1.5×, with a cap around 60 seconds. Fixed one-second polling wastes requests and can trigger rate limits; see the rate limit exceeded guide.

Can the agent do useful work while it waits?

Only if your orchestrator supports concurrent tool calls. If it does, start the job, perform independent work, and then check the status. Otherwise, a blocking wrapper is simpler and less error-prone than a hand-built scheduler.

How do I stop the agent from claiming success early?

Say it explicitly in the response body, expose a boolean done, and make the completion tool the only place a result appears. If the start response contains no result, the agent has nothing to report as an outcome.

Do webhooks work for agents running on a laptop?

Not directly, because a laptop usually has no public endpoint. Use a tunnel during development, as shown in this guide to testing localhost APIs with webhook services, or use polling until the agent runs somewhere addressable.