DEV Community: xn

Google ADK state_delta Not Applied on Resume: Runner.run_async Reproduction and Workaround

xn — Sun, 09 Aug 2026 13:26:30 +0000

A production-focused update based on a real project: Google ADK 2.6.2 repro: Runner.run_async ignores state_delta when resuming by invocation_id without new_message. Includes four…

If you use Google ADK resumability, there is a failure mode that is harder to notice than an exception: Runner.run_async() resumes successfully, the invocation continues, but the state_delta passed with that resume never reaches session.state.

I reproduced the behavior locally on google-adk==2.6.2 without an API key or an external model call. The boundary was consistent across both dispatch paths I tested: the Node path using LlmAgent and the legacy path using a plain BaseAgent. When the resume has no new_message, the delta is not applied. With a new_message, the same delta is persisted.

That distinction matters for approval callbacks, background-job resumes, and long-running workflows where an external system may need to update state without inventing another user message.

Short answer

My local environment:

macOS 26.5.2 arm64
Python 3.10.2
google-adk==2.6.2
ResumabilityConfig(is_resumable=True)
InMemorySessionService
an offline Echo model stub

The four-case matrix was:

Runner path	`new_message` on resume	Was `state_delta` applied?	Result
Node / `LlmAgent`	No	No	Failure reproduced
Node / `LlmAgent`	Yes	Yes	Control passes
legacy / `BaseAgent`	No	No	Failure reproduced
legacy / `BaseAgent`	Yes	Yes	Control passes

Figure 2. The same delta is lost on both resume paths without new_message, and applied on both control paths when a user message exists.

Actual output from the local reproduction:

node-no-message        new_message=False applied=False state={}
node-with-message      new_message=True  applied=True  state={'resumed_key': 'resumed_value'}
legacy-no-message      new_message=False applied=False state={}
legacy-with-message    new_message=True  applied=True  state={'resumed_key': 'resumed_value'}

The problematic call is structurally simple:

async for _ in runner.run_async(
    user_id=user_id,
    session_id=session_id,
    invocation_id=invocation_id,
    state_delta={"approved": True},
):
    pass

On ADK 2.6.2 in this test, the call can continue without an exception while the delta is not persisted. The important wording is the delta is ignored. Existing session state is not proven to be erased or reset by this test.

Why this is easy to misdiagnose

At first, this looks like a SessionService problem. You might suspect that InMemorySessionService did not persist state, that the wrong invocation_id was resumed, or that a callback later overwrote the state.

The controls narrow that down. The SessionService, agent, delta, and resumability configuration are the same; the only meaningful difference is whether a new_message exists on resume. With no message the final state is {}. With a message the final state contains:

{"resumed_key": "resumed_value"}

That makes this different from “ADK state does not work on resume.” State updates do work on the control path. The failure is tied to how state_delta reaches the event-persistence path.

What the ADK 2.6.2 source path shows

I inspected the locally installed 2.6.2 Runner implementation. On the Node execution path, appending the user event is gated by the presence of new_message:

if new_message:
    user_event = await self._append_user_event(
        ic, new_message, state_delta=state_delta
    )

_append_user_event() is where the delta is attached to EventActions:

Event(
    invocation_id=ic.invocation_id,
    author="user",
    actions=EventActions(state_delta=state_delta),
    content=content,
)

The event is then persisted through:

self.session_service.append_event(...)

That explains the control result: when a user message exists, the delta rides on the user event and reaches the SessionService. When the invocation is resumed without a new_message, that event path is skipped and there is no independent persistence path in this version to carry the supplied delta.

Figure 3. Left: the failing resume path. Right: the temporary event-level workaround tested locally. The workaround is not an upstream fix.

This matches Google ADK issue #6644. The issue states that Runner.run_async accepts state_delta as an optional state change, but the delta is silently discarded when resuming by invocation_id without new_message. The report covers both the Node and legacy dispatch paths.

As of August 9, 2026, #6644 is still open and the GitHub issue page shows no linked branch or pull request. Treat any workaround below as temporary, not as a released Google fix.

Do not use a fake `new_message` as the default fix

The controls make one tempting workaround obvious: if a message makes the state update work, why not send an empty or synthetic user message every time you resume?

I would not make that the production default.

new_message is not a state-update flag. It becomes part of the session event history and can affect callbacks, model context, auditing, and later workflow behavior. In a human-in-the-loop flow, inventing a user message only to trigger state persistence can make the event history semantically false.

A safer temporary approach is to separate two operations that your application actually means: persist the state change, then resume the invocation.

Temporary workaround I verified locally

ADK session state is updated through events carrying EventActions(state_delta=...). I therefore tested an explicit event-level path: append a content-less event containing the delta through the configured SessionService, then resume the invocation without passing a fake user message.

Core example:

from google.adk.events.event import Event
from google.adk.events.event_actions import EventActions

session = await session_service.get_session(
    app_name=app_name,
    user_id=user_id,
    session_id=session_id,
)

await session_service.append_event(
    session=session,
    event=Event(
        invocation_id=invocation_id,
        author="user",
        actions=EventActions(
            state_delta={"resumed_key": "resumed_value"}
        ),
    ),
)

async for _ in runner.run_async(
    user_id=user_id,
    session_id=session_id,
    invocation_id=invocation_id,
):
    pass

I ran that workaround on both local paths:

node       applied=True state={'resumed_key': 'resumed_value'}
legacy     applied=True state={'resumed_key': 'resumed_value'}

So in this 2.6.2 test, explicitly persisting a content-less event with EventActions(state_delta=...) avoided the loss seen when relying on Runner.run_async(state_delta=...) during a message-less resume.

This is still a workaround. I only verified it with InMemorySessionService. If you use a database-backed or managed SessionService, re-test event persistence, idempotency, concurrency, ordering, and audit semantics before using the same pattern in production.

Where this bug matters most

A normal chat application may never notice the issue because each turn naturally contains a new user message. The higher-risk cases are workflows where resume input comes from somewhere other than natural-language chat:

a human approves an action in an external UI;
a background job finishes and resumes an invocation;
a webhook changes workflow state;
an operator updates approval metadata without adding a user message;
a long-running process uses invocation_id as its resume handle.

For example:

state_delta={
    "approval_status": "approved",
    "reviewer": "human-42",
}

If that resume has no new_message, application code may see run_async() complete and assume the approval state was saved. A later node can then read the old value or no value at all.

For state that gates payments, deletion, sending, deployment, or other high-impact tools, add a write-after-read assertion instead of treating a successful Runner call as proof of persistence:

session = await session_service.get_session(...)
assert session.state.get("approval_status") == "approved"

Is every Google ADK version affected?

This article does not claim that.

The XBSTACK reproduction is scoped to google-adk==2.6.2. The upstream report also demonstrates the issue against 2.6.2-era code. A later release may change the behavior.

If you are reading this on a newer version, check two things before copying any workaround:

Check whether issue #6644 is closed and whether a fix is included in your installed release.
Run the four-case matrix locally. It requires no external model API, so it is cheap to keep as a regression test.

If the no-message cases begin returning the expected state on a newer version, remove the workaround rather than preserving obsolete event logic indefinitely.

Minimal troubleshooting checklist

When state_delta appears not to work during a Google ADK resume:

Record the exact google-adk version.
Confirm the call is resuming with invocation_id.
Confirm resumability is enabled for the app.
Check whether new_message is absent.
Re-read session.state after the run; do not rely only on the lack of an exception.
Run the same delta once with and once without new_message.
Check the current status of #6644 and your release notes.
Prefer an upstream fixed release when available; otherwise evaluate an explicit state-event workaround in your own SessionService.

FAQ

Why does `run_async()` succeed while the state remains unchanged?

Because the failure is in the state persistence path, not in invocation validation. ADK can find and resume the invocation while the 2.6.2 message-less path fails to attach the supplied delta to a persisted user event.

Does sending a `new_message` fix it?

In my four-case test, the same delta is applied when a new_message exists. That is useful as a control, but manufacturing a fake message just to trigger persistence changes your event history and is not the workaround I recommend by default.

Is `SessionService.append_event()` the official fix?

No. It is a temporary event-level workaround I verified on 2.6.2 with InMemorySessionService. The upstream issue remains open as of August 9, 2026.

Is the bug model-specific?

The reproduction does not require Gemini, OpenAI, LiteLLM, or any external model API. The Node-path test uses an offline Echo stub, and the legacy-path test uses a plain BaseAgent, so the tested failure boundary is independent of a provider call.

Final recommendation

The most important lesson here is not simply “ADK has a state bug.” The narrower production lesson is that resumability and state persistence need separate regression tests.

A run can resume without throwing while a state mutation is not persisted. For approval systems, long-running workflows, and external callbacks, verify the state after resume and keep a small no-network regression test around this boundary.

On google-adk==2.6.2, I reproduced the failure on both dispatch paths, confirmed the new_message control on both paths, and verified an explicit event-level workaround. I would still prefer an upstream fixed release as soon as one is available, then remove the temporary workaround after the same matrix passes.

References

Google ADK issue #6644: https://github.com/google/adk-python/issues/6644
Google ADK repository: https://github.com/google/adk-python
XBSTACK Production AI Agent Systems: https://www.xbstack.com/en/ai/agent/?utm_source=devto&utm_medium=referral&utm_campaign=google_adk_state_delta_resume&utm_content=related_agent_hub
XBSTACK AI Engineering Resource Hub: https://www.xbstack.com/en/ai/?utm_source=devto&utm_medium=referral&utm_campaign=google_adk_state_delta_resume&utm_content=related_ai_hub
XBSTACK AI Agent Framework Guide 2026: https://www.xbstack.com/en/ai/ai-agent-framework-comparison/?utm_source=devto&utm_medium=referral&utm_campaign=google_adk_state_delta_resume&utm_content=related_framework_guide

Reproduction assets

repro.py: four failure/control cases
workaround.py: explicit content-less state-event workaround
requirements.txt: pins google-adk==2.6.2
README.md: environment, results, and evidence boundary

Canonical article on XBSTACK：https://www.xbstack.com/en/ai/google-adk-state-delta-resume/?utm_source=devto&utm_medium=referral&utm_campaign=article_distribution&utm_content=google-adk-state-delta-resume&ref=devto

标签：#AI #SoftwareEngineering #DeveloperTools #Google ADK #Runner.run_async

OpenAI Responses API Stream Abort: Fix No tool call found

xn — Wed, 05 Aug 2026 05:29:55 +0000

A production-focused update based on a real project: A function_call can be visible before it is durable in Conversation state. This guide explains the 400 No tool call found erro…

OpenAI Responses API: Why Stream Abort Causes No tool call found for function call output

Here is the production conclusion first: seeing a streamed function_call is not the same as having a durable call that the Conversation can reference later. If a client closes the Responses API stream before the response commits, but the application already executes the tool, the next turn may reject the matching function_call_output with 400 No tool call found for function call output. The error is bad; the orphaned side effect is worse. A payment, deployment, email, or database write may already exist even though the model conversation has no call record for it.

The fastest recovery is not to keep retrying the stale output. Read the Conversation Items first. If the matching call_id exists, reuse or produce the tool result idempotently and submit it. If the call is absent, treat the streamed call as uncommitted, discard that call_id, and generate a new turn. Separately, every external write needs an application-owned idempotency key, because durable model state does not provide business exactly-once execution.

The server-side behavior described here comes from OpenAI Python SDK Issue #3561. I did not use an unapproved production API key and do not present the local lab as an independent server reproduction. The lab makes the reported persistence boundary explicit and tests which recovery policies create or prevent orphaned side effects.

1. What the 400 Error Actually Means

The representative error is:

400 No tool call found for function call output with call_id call_xxx.

This is not a missing Python function and not a JSON Schema validation failure. It is an association failure. A function_call_output says, in effect, “this value belongs to the tool call identified by call_id.” The server must be able to find that call in the current state before it can continue the model turn. If no matching call exists, accepting the output would attach data to an event that the conversation does not know happened.

Start by separating three failure classes. The first is wiring: the output was sent to the wrong Conversation, a stale ID was reused, or concurrent tabs crossed user state. The second is state-mechanism mismatch: one turn used a Conversation while another turn was built against a different context path. The third is the abort window covered here: the client observed a call in the stream, but the response never reached the state where that call was durably available in Conversation Items.

Issue #3561 documents that exact path. The reporter creates a Conversation, starts a streamed Responses request, receives a function_call in response.output_item.added, closes the stream, and then lists Conversation Items. The list is empty. The local tool is nevertheless executed. Submitting its output to the same Conversation on the next turn fails with the 400 error. The report uses Windows 10, Python 3.11.5, and openai 2.45.0.

2. Official Status as of August 5, 2026

Version numbers can be misleading here. The latest OpenAI Python SDK release is v2.53.0, published on August 3, 2026. Issue #3561 remains open, carries the bug label, and was updated on August 4. It has no linked Python fix pull request, milestone, or release note identifying a first fixed version.

The accurate statement is therefore limited: there is a runnable report in the official repository, and the report has not been closed with a Python release conclusion. You can test 2.53.0 in your own environment, but you should not remove the reconciliation gate merely because the package number is newer than 2.45.0.

Do not automatically downgrade either. The behavior crosses the client stream, the Responses service commit boundary, and Conversation persistence. A downgrade may not change server behavior and can reintroduce unrelated type, authentication, retry, or transport problems. A safer upgrade policy is to freeze the current production version, add a deterministic abort matrix, run the same matrix against a candidate release in an isolated OpenAI Project, and promote only when official status, live integration results, and side-effect counts agree.

The issue links to a related fix in the JavaScript Agents SDK, PR #1241, which reconciles streamed function calls when server-managed runs abort. That relationship makes the failure mode credible across adjacent stacks, but it is not proof that the Python Responses path is fixed. SDK runner behavior, API service behavior, and conversation storage must be verified independently.

3. Stream Visibility Is Not a Commit Token

Streaming exists to expose useful information before a full response completes. response.output_item.added tells the client that an output item has appeared in the current stream. response.function_call_arguments.done tells it that the function arguments are complete. Those events are useful for UI, validation, logging, and scheduling low-risk work. They are not, by themselves, a durable authorization token for an external side effect.

There are at least three distinct guarantees:

Boundary	What you know	What you still do not know
Client observed an event	A call and `call_id` appeared in the stream	The Conversation durably contains the call
Response completed	The model turn reached a normal completion boundary	The tool will be delivered only once
Conversation reconciliation	The server lists the matching `function_call`	The business action is exactly once

The unsafe implementation jumps from the first row directly to execution. That may be tolerable for a cacheable weather read. It is not acceptable for money movement, order creation, deployment, outbound email, access changes, or database mutation. If the stream aborts before commitment, the application has created an external fact without a model-state fact that can receive the result.

call_id is also not a sufficient business idempotency key. It is primarily the correlation ID between a model call and a tool output. A regenerated model turn can produce a new call_id for the same order or deployment. A stable key should come from the business intent, such as tenant + operation + resource + desired_version, with call_id stored as supporting evidence rather than the only uniqueness boundary.

tenant-a:deploy:release-2026-08-05:call_xxx

4. The Local Lab and Its Evidence Boundary

The public lab is located at:

experiments/openai-responses-stream-abort-tool-call-loss/

It uses only the Python standard library. provisional_items represents calls already exposed through stream events but not committed. committed_items represents calls that can be referenced by later outputs. Normal completion moves provisional calls into committed state. Abort discards them in the modeled failure path. Submitting an output for a missing call raises the same semantic error as the official report.

This is not a reverse-engineered copy of OpenAI internals. It does not prove how every production request is stored. Its purpose is narrower and honest: given the officially reported condition that the client can observe a call that the Conversation later lacks, which application policies are safe?

The lab fixes four scenarios:

complete the stream, confirm the call, then execute;
execute immediately after output_item.added, then abort;
abort, reconcile, and execute only if the call exists;
commit the call, deliver the tool job twice, and deduplicate with a business ledger.

It does not measure live API latency, token cost, browser behavior, HTTP/2 teardown, proxy buffering, model tool-selection quality, or the previous_response_id path. OpenAI behavior claims are sourced from official documentation and Issue #3561. Local results support the application control model only.

5. Four Scenarios, One Critical Invariant

The experiment writes the verified result to:

experiments/openai-responses-stream-abort-tool-call-loss/results/verification.json

Four standard-library unit tests pass. The result matrix is:

Scenario	Conversation items	Deliveries	External effects	Next turn
Complete, then execute	call + output	1	1	accepted
Execute on observed call, then abort	none	1	1	rejected with 400
Abort, reconcile, then decide	none	0	0	stale call discarded
Committed call delivered twice with idempotency	call + output	2	1	accepted

The second row is the failure that matters. The tool side effect count is already 1, while the Conversation contains no items. The output cannot be attached, so the next turn receives:

No tool call found for function call output with call_id call_aborted.

For a read-only tool, generating another turn may be enough. For a charge or deployment, regeneration may produce a second valid call and a second side effect. Refusing every retry avoids duplication but can leave the user with a failed task after the action actually succeeded. That conflict cannot be solved with prompting; it requires an application truth store and idempotent execution.

The third scenario proves the conservative gate: after abort, reconcile first. If the call is missing, do not execute. The fourth scenario proves that reconciliation is not an exactly-once mechanism. A valid committed call can still be delivered repeatedly by a queue, manual retry, or worker recovery.

6. Fast Triage for Production Incidents

Do not begin by rewriting the prompt or randomly changing SDK versions. Follow the association chain.

Step 1: Confirm the Conversation and call_id belong to the same turn

Capture complete correlation fields:

request_id
conversation_id
response_id
call_id
user_id
tenant_id
stream_status
tool_name
business_idempotency_key

Verify that the output is being sent to the same Conversation that produced the call. Check concurrent tabs, retries, and tenant boundaries. Partial IDs are often insufficient when two live requests overlap.

Step 2: Read Conversation Items

List Conversation Items and look for the exact call_id. If it is absent, stop submitting the output. Mark the application intent as uncommitted or orphaned-intent and route it to regeneration or manual reconciliation.

Step 3: Identify how the stream ended

Distinguish normal completion, user stop, component unmount, AbortController, reverse-proxy timeout, mobile network loss, worker restart, and server crash. These paths can look identical to the UI while producing different backend states.

Step 4: Check whether the tool already produced a side effect

Verify the external side effect independently. An empty Conversation does not mean the tool never ran. Check payment records, deployment IDs, message IDs, database constraints, or the application ledger.

Step 5: Choose the recovery path

Use the reconciliation result and the external side-effect state together:

Call in Conversation	Side effect	Recovery
yes	no	reserve idempotency key, execute, submit output
yes	yes	reuse stored result, submit output
no	no	discard stale call_id and generate a new turn
no	yes	do not execute again; compensate or reconcile manually, then rebuild model context

The last row should not be hidden behind a generic retry loop. The business system has a fact that the model system lacks. Whether you send that fact back as a normal message, create an auditable recovery event, or require human review depends on the risk and regulatory context.

7. A Production Tool-Call State Machine

A reliable orchestrator needs more than pending/success/failed:

stream_observed
→ call_committed
→ execution_reserved
→ execution_succeeded
→ output_submitted
→ model_acknowledged

Failure branches need names as well:

stream_aborted_before_commit
commit_unknown
execution_unknown
output_rejected
manual_reconciliation_required

stream_observed is safe for UI. It is not safe for a high-risk executor. call_committed means the Conversation reconciliation succeeded. execution_reserved means a unique business key has locked the action. execution_succeeded stores the immutable external result. output_submitted records delivery to the model, and model_acknowledged closes the complete workflow.

What the state record should store

Persist this state in a transactional store, not only in traces. A minimal record should include the business key, tenant and user, Conversation and Response IDs, call_id, tool name, argument digest, every transition timestamp, abort reason, external result ID, result reference, retry count, and last error.

intent_id
business_idempotency_key
user_id
tenant_id
conversation_id
response_id
call_id
tool_name
arguments_hash
stream_observed_at
call_committed_at
execution_reserved_at
execution_succeeded_at
output_submitted_at
model_acknowledged_at
status
abort_reason
external_result_id
result_payload_ref
retry_count
last_error_code
created_at
updated_at

Use a unique constraint on the business idempotency key and another on tenant + conversation_id + call_id.

The argument digest matters. Reusing one business key with different tool arguments should fail closed rather than silently return an old result. The external result ID should be a payment transaction, deployment ID, email Message ID, or other verifiable fact that a recovery worker can query after a crash.

Useful metrics include stream_observed_without_commit_total, tool_execution_before_commit_total, function_call_output_not_found_total, reconciliation_latency_ms, duplicate_delivery_total, idempotency_reuse_total, and manual_reconciliation_total. For high-risk tools, tool_execution_before_commit_total should remain zero. A falling 400 rate is not enough if errors are merely swallowed while side effects diverge.

8. A Safer Python Control Skeleton

The following is a control-flow example, not a complete OpenAI wrapper. The important separation is observed call, committed call, idempotency reservation, tool execution, and output submission:

from dataclasses import dataclass

@dataclass
class ToolIntent:
    conversation_id: str
    response_id: str
    call_id: str
    tool_name: str
    arguments: dict
    business_key: str


def handle_streamed_call(client, intent: ToolIntent, ledger, executor):
    items = client.conversations.items.list(
        conversation_id=intent.conversation_id,
    )

    committed = any(
        item.type == "function_call" and item.call_id == intent.call_id
        for item in items.data
    )
    if not committed:
        return {
            "status": "discarded_uncommitted_call",
            "call_id": intent.call_id,
        }

    existing = ledger.get(intent.business_key)
    if existing:
        tool_result = existing.result
    else:
        reservation = ledger.reserve(intent.business_key, intent.call_id)
        if not reservation.acquired:
            return {"status": "execution_in_progress"}

        tool_result = executor.execute(
            intent.tool_name,
            intent.arguments,
        )
        ledger.complete(intent.business_key, tool_result)

    return client.responses.create(
        conversation=intent.conversation_id,
        input=[{
            "type": "function_call_output",
            "call_id": intent.call_id,
            "output": tool_result,
        }],
    )

Two crash windows remain. The process may fail after checking Conversation Items but before output submission. It may also fail after the external action succeeds but before the ledger is completed. The ledger therefore needs durable reservation, unique constraints, external result lookup, and result reuse. A recovery worker must query the ledger and external system before deciding to execute again.

High-risk tools should add authorization and human approval before execution_reserved. See AI Agent tool authorization for a per-call Policy Gate. If your application uses OpenAI Agents SDK approval interruptions, the separate RunState resume and v0.19.3 persistence guide covers that layer.

9. Recovering Browser Disconnects and Proxy Timeouts

Production disconnects are rarely a clean stream.close(). A browser can background the tab, mobile connectivity can switch networks, a reverse proxy can close the downstream while the upstream is still working, and a server process can die after an event was received but before the log was flushed. The correct state is often “unknown,” not “failed.”

Persist one application record per streamed response:

response_id
conversation_id
last_sequence_number
last_event_type
observed_call_ids
completed_at
abort_reason
reconciliation_status

After an ambiguous disconnect, a background reconciler should list Conversation Items and classify each observed call as committed or missing. Committed calls can enter the idempotent execution queue. Missing calls must be discarded and must not receive an output. If a frontend or another service already executed the tool, the business ledger must route the operation into compensation or manual reconciliation.

Do not give the browser direct ownership of high-risk execution. The browser should receive the stream and present approval. A backend orchestrator should persist the intent, reconcile Conversation state, execute the tool, and submit the output. Closing a page then becomes a presentation failure, not a payment or deployment consistency failure.

10. Regression Tests That Are Worth Keeping

A manual Ctrl-C test is not sufficient. The regression suite should actively abort at several boundaries and verify Conversation state, ledger state, and external effects together.

Run commit-boundary tests after response.output_item.added, after response.function_call_arguments.done, after the final argument delta, immediately before completion, and after response.completed. Store the Response ID, call ID, final sequence number, and the Conversation Items result. Do not encode assumptions about which event commits; let the current API behavior populate the matrix.

Run side-effect-gate tests with a transactional fake payment, deployment table, or outbox. Before reconciliation, the count must be zero. After a committed call and first business-key reservation, it must be one. Deliver the same work two, five, or twenty times and assert that the external fact still appears once.

Inject process crashes in at least five windows:

before ledger reservation
after reservation, before tool execution
after tool success, before ledger completion
after ledger completion, before function_call_output
after output submission, before final model response

Each window has a different recovery rule. Retry is safe before reservation. A stale reservation needs ownership and expiry logic. Success before ledger completion requires external lookup. A completed ledger result should be reused. A lost final model response should not execute the tool again.

Keep a version matrix in CI:

SDK	Abort point	Call persisted	Side effects	Old output accepted	Decision
production	after item added	measured	0	measured	safe/unsafe
production	after arguments done	measured	0	measured	safe/unsafe
production	after response completed	measured	1	measured	baseline
candidate	same three points	measured	invariant	measured	promote/block

Unit tests should validate the application state machine on every commit. A live integration test should run when the SDK, streaming wrapper, proxy configuration, or cancellation logic changes. Use an isolated OpenAI Project, a low-cost model, a fake tool, and a strict spend limit. Save sanitized Conversation Items, package version, HTTP status, and timestamps as build artifacts.

The public XBSTACK lab deliberately stops before a live API call. A project adopting this pattern should add an environment-gated live_integration_test.py only after explicit credential authorization. That preserves the evidence boundary while making future official fixes comparable under the same test.

11. Common Mistakes and Final Decision

Mistake: treating function_call_arguments.done as a commit. It means argument streaming is complete, not that the Conversation can reference the call.

Mistake: using call_id as the only exactly-once key. A regenerated turn can produce a new call ID for the same order or deployment.

Mistake: retrying the same output after every 400. If the call is absent, retries do not create it.

Mistake: assuming an empty Conversation means the tool did not execute. External systems and the application ledger are the source of truth for side effects.

Mistake: deleting safeguards because the current SDK is newer than the reported version. As of August 5, 2026, the issue remains open and no Python fixed release is named.

The decision is simple:

A streamed tool call is observable data, not an execution credential. Confirm the server committed it, reserve a stable business idempotency key, execute through a backend orchestrator, and reconcile every ambiguous disconnect. If the call ID is missing, discard it.

Read-only tools may accept a looser policy if duplicate cost is low. Payments, email, deployments, writes, and permission changes should not. They need the state machine, ledger, constraints, authorization, and recovery path described above. For broader timeout and retry design, see AI Agent error recovery. For the full tool-call production boundary, continue with AI Agent Tool Use.

Frequently Asked Questions

Why can the client have a call_id while Conversation Items are empty?

The stream event and durable state are different guarantees. The client can observe a provisional function call before the response reaches its persistence boundary. If the response is aborted first, the reported failure path leaves no Conversation item to reference later.

Is waiting for response.completed enough?

It is safer than executing on the added event. For a high-risk tool, still reconcile Conversation Items before execution. Response completion addresses the model-turn boundary; an idempotency ledger addresses duplicate tool delivery.

What if the tool executed but the call_id is missing?

Do not invent a tool call or continue submitting output to the missing ID. Record the real external result, prevent duplicate execution, and start a new auditable model turn or require human reconciliation. The appropriate recovery depends on whether the business allows the model to continue automatically.

Is this the same as the OpenAI Agents SDK RunState persistence issue?

No. This page covers a Responses API Conversation that did not commit a streamed function_call. The RunState article covers approval resume and Session persistence in the Agents SDK. Both can create a mismatch between model state and external action, but the APIs and recovery boundaries differ.

Does the local lab prove the OpenAI server implementation?

No. It tests application policies under the condition documented by the official issue. OpenAI server behavior, affected versions, and the first fixed release must come from official Issues, pull requests, and release notes. This page will update its version conclusion only when new official status and comparable verification are available.

Experiment: OpenAI Responses API Stream Abort Tool Call Loss Lab

Official sources: OpenAI Python SDK Issue #3561 · Function Calling Guide · Conversation State Guide

Canonical article on XBSTACK：https://www.xbstack.com/en/ai/openai-responses-api-stream-abort-tool-call-lost/?utm_source=devto&utm_medium=referral&utm_campaign=openai-responses-stream-abort&utm_content=en%2Fopenai-responses-api-stream-abort-tool-call-lost&ref=devto

标签：#AI #SoftwareEngineering #DeveloperTools #OpenAI Responses API #Function Calling

OpenAI Agents SDK Duplicate Tool Names: Why the Later Tool Wins

xn — Mon, 03 Aug 2026 06:11:41 +0000

A production-focused update based on a real project: An offline reproduction on openai-agents 0.19.2 shows that duplicate FunctionTool names pass SDK validation, remain in Agent.g…

OpenAI Agents SDK Duplicate Tool Names: Why the Later Tool Wins

Here is the direct result: with openai-agents==0.19.2, two plain FunctionTool objects can expose the same public name without the SDK rejecting the Agent configuration. In the offline fixture, two different Python functions both use name_override="lookup". The SDK validation function returns None, Agent.get_all_tools() still returns two lookup tools, and the internal dispatch lookup map keeps only the later one. With OpenAI APIs, the request can fail with a duplicate-function-name 400. With a compatible provider that accepts duplicate names, the run may continue while the wrong local implementation executes.

The issue was filed in the official OpenAI Agents SDK repository on August 2, 2026 as issue #4116. As of August 3, it remained open with no linked pull request.

This page answers one narrow search problem:

Why can duplicate FunctionTool names enter an OpenAI Agents SDK Agent, why does local dispatch become last-wins, and how can a production service fail before sending a model request?

It is separate from the existing RunState approval resume guide. That article covers interruptions, approval, serialization, and cross-process recovery. This one covers tool-registry identity before the model call starts.

The smallest collision

The Python function names are different, but both public tool names are lookup:

from agents import function_tool


@function_tool(name_override="lookup")
def lookup_customers(query: str) -> str:
    """Look up customers."""
    return f"customer:{query}"


@function_tool(name_override="lookup")
def lookup_orders(query: str) -> str:
    """Look up orders."""
    return f"order:{query}"

The official tools documentation says @function_tool normally uses the Python function name and allows an explicit name_override. The identity that must be unique is the resulting FunctionTool.name, not the Python identifier.

This collision can emerge when:

CRM and order modules both export a generic lookup tool;
two plugins expose search;
an Agent.clone() flow appends the original tools again;
tenant or feature-flag logic adds tools dynamically;
sub-agents are converted to tools with repeated tool_name values;
multiple teams choose execute, query, or fetch as overrides;
an old and new implementation are registered during a migration.

Each module may be valid in isolation. The conflict appears only after the final registry is assembled.

Offline test environment

The fixture avoids model behavior, API keys, network calls, and provider differences:

Component	Value
Python	`3.10.2`
OpenAI Agents SDK	`0.19.2`
API key	Not required
Model call	None
Verified paths	Tool validation, Agent tool list, dispatch lookup map

Files:

experiments/openai-agents-duplicate-tool-names-repro/
├── repro.py
├── requirements.txt
├── results/verification.json
├── RESEARCH.md
└── README.md

Run it with:

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
.venv/bin/python repro.py

Layer one: the SDK validator does not reject the conflict

The SDK contains a function with an explicit purpose:

validate_function_tool_lookup_configuration(tools)

Passing the duplicate tools returns normally:

result = validate_function_tool_lookup_configuration([
    lookup_customers,
    lookup_orders,
])

assert result is None

There is no UserError, warning, or deduplication.

Issue #4116 identifies the relevant branch: the validator detects an existing owner for the same qualified name, but when neither plain tool has an explicit namespace, it executes continue. The common collision is recognized and then ignored.

The issue also notes inconsistent handling across tool categories: duplicate names from MCP servers and Codex tools already have explicit checks, while plain FunctionTools do not receive the equivalent gate.

Layer two: both duplicate tools remain visible to the model

After constructing the Agent:

from agents import Agent, RunContextWrapper

agent = Agent(
    name="Support",
    tools=[lookup_customers, lookup_orders],
)

tools = await agent.get_all_tools(
    RunContextWrapper(context=None),
)

print([tool.name for tool in tools])

The fixture prints:

['lookup', 'lookup']

The SDK does not rename, select, or remove one of them at the Agent layer. The provider request can therefore contain two function definitions with the same public name.

The upstream issue reports that OpenAI Responses and Chat Completions APIs reject this payload, producing a provider-side 400 even though the actual configuration error is local.

Layer three: local dispatch becomes last-wins

If a compatible provider accepts duplicate names and returns a tool call such as:

{
  "name": "lookup",
  "arguments": {"query": "A-100"}
}

then the SDK still needs to select a Python implementation.

The fixture builds the internal lookup map:

from agents._tool_identity import build_function_tool_lookup_map

lookup_map = build_function_tool_lookup_map([
    lookup_customers,
    lookup_orders,
])

Only one key remains:

('bare', 'lookup')

It points to the later tool:

selected = lookup_map[("bare", "lookup")]

assert selected is lookup_orders
assert selected is not lookup_customers

Verified output:

{
  "sdk_validator_returned_none": true,
  "advertised_tool_names": ["lookup", "lookup"],
  "dispatch_lookup_keys": [["bare", "lookup"]],
  "dispatch_selected_python_function": "lookup_orders",
  "first_tool_reachable_by_bare_name": false,
  "second_tool_reachable_by_bare_name": true
}

The later dictionary assignment replaces the earlier one.

Why silent last-wins is more dangerous than a 400

A provider 400 stops the request. It harms availability, but it does not execute the wrong operation. A tolerant provider can be worse: the model returns lookup, and the local SDK cannot infer whether it meant customer lookup or order lookup. It executes whichever implementation survived the map construction.

Imagine that the two underlying implementations are:

lookup_customer_account
lookup_refund_order

If both are exposed as lookup, one tool may read customer data while the other initiates a refund, delete, message send, or database write. The identity collision becomes a side-effect risk.

The failure surface includes:

two indistinguishable schemas sent to the model;
provider request rejection;
incorrect local implementation selection;
audit logs that show only the ambiguous public name;
retries that repeat the deterministic configuration problem;
different failure behavior after switching providers.

Temporary fix: validate uniqueness before model dispatch

Until the SDK rejects duplicate plain FunctionTools, applications can fail after assembling the final registry and before constructing the production request:

from collections import Counter
from collections.abc import Iterable

from agents import FunctionTool
from agents.exceptions import UserError


def find_duplicate_function_tool_names(
    tools: Iterable[object],
) -> list[str]:
    names = [
        tool.name
        for tool in tools
        if isinstance(tool, FunctionTool)
    ]

    return sorted(
        name
        for name, count in Counter(names).items()
        if count > 1
    )


def require_unique_function_tool_names(
    tools: Iterable[object],
) -> None:
    duplicates = find_duplicate_function_tool_names(tools)
    if duplicates:
        quoted = ", ".join(repr(name) for name in duplicates)
        raise UserError(
            "Duplicate FunctionTool names are not allowed: "
            f"{quoted}. Use a unique Python function name, "
            "name_override=, or a tool namespace."
        )

Apply it to the final set:

tools = load_static_tools()
tools += load_plugin_tools()
tools += await load_tenant_tools(tenant_id)

require_unique_function_tool_names(tools)

agent = Agent(
    name="Support",
    tools=tools,
)

The duplicate fixture now fails locally with an actionable message instead of waiting for a provider response:

Duplicate FunctionTool names are not allowed: 'lookup'.
Use a unique Python function name, name_override=, or a tool namespace.

Fix the public names

Give the second tool a distinct identity:

@function_tool(name_override="lookup_orders")
def lookup_orders(query: str) -> str:
    """Look up orders."""
    return f"order:{query}"

The resulting list is:

['lookup', 'lookup_orders']

Prefer business-specific names over numeric suffixes:

customer_lookup
order_lookup
invoice_lookup
knowledge_search
shipment_track

A good name helps the model distinguish capabilities as well as satisfying the uniqueness constraint.

Use namespaces for larger registries

The official tools documentation recommends namespaces where possible, especially when many related tools exist. A registry can expose clearer identities such as:

crm.lookup_customer
orders.lookup_order
billing.lookup_invoice

Namespaces reduce collisions on generic verbs such as lookup, search, and create, and give the model a better high-level surface.

They do not replace testing. CI should validate the actual assembled callable identities rather than only checking source-level function names.

Where the gate should run

Agent factory unit tests

def test_support_agent_tool_names_are_unique():
    tools = build_support_tools()
    require_unique_function_tool_names(tools)

Plugin registration

Validate after all plugins load. Per-plugin uniqueness cannot detect collisions between plugins.

Multi-tenant configuration

Different tenants can enable different combinations. Precompute valid combinations or validate and cache each final set at worker startup or request entry.

Clone and dynamic append paths

Cloning, list concatenation, feature flags, and A/B tests are common duplication sources. Validate the final list rather than the initial constant.

Release gates

CI can enumerate production factories:

for agent_name, tools in all_production_toolsets():
    try:
        require_unique_function_tool_names(tools)
    except UserError as exc:
        raise AssertionError(f"{agent_name}: {exc}") from exc

Name uniqueness is only the first registry check

A production tool registry should also validate:

Check	Failure risk
Unique `FunctionTool.name`	Provider 400 or wrong dispatch
Stable tool schema	Cache invalidation and argument drift
Distinct descriptions	Unstable model selection
Approval or guardrails for high-risk tools	Unauthorized side effects
Stable tool IDs in audit logs	Inability to identify implementation
Reproducible dynamic enablement	Different workers expose different registries

For authorization and policy enforcement, continue with the AI Agent Tool Authorization Policy Gate. This page remains limited to naming identity.

Three approaches to avoid

Waiting for the provider 400

A deterministic local error is delayed until after network work, increasing latency, retry noise, and debugging cost. A provider change can convert the visible 400 into silent wrong dispatch.

Treating list order as configuration

Import order, plugin discovery, and configuration merging can change ordering. Last-wins is not an explicit or auditable routing policy.

Checking only `name`

Different Python functions can share the same public name through name_override. Validate FunctionTool.name.

Regression test after an upstream fix

As of August 3, 2026, issue #4116 had no linked pull request. A reasonable upstream fix should reject the configuration during tool resolution with an error that names the conflict and suggests a unique override or namespace, for example:

Ambiguous function tool configuration:
the tool name `lookup` is used by multiple tools.
Pass a unique name_override= or namespace.

After upgrading, test the SDK behavior directly:

import pytest
from agents.exceptions import UserError


def test_sdk_rejects_duplicate_bare_function_tools():
    with pytest.raises(UserError):
        validate_function_tool_lookup_configuration([
            lookup_customers,
            lookup_orders,
        ])

Also verify that:

Agent.get_all_tools() cannot expose duplicate sendable names;
dynamic and static tool collisions are both detected;
namespace rules match the documented behavior;
the error identifies the conflicting name and remediation;
pre-request behavior is consistent across providers.

Relationship to the RunState article

The OpenAI Agents SDK RunState guide covers:

tool-approval interruptions;
RunState serialization;
cross-process resume;
redelivery and business idempotency;
context filtering and version governance.

This problem happens earlier. The tool registry is already ambiguous before the Agent sends a model request. A durable approval pipeline cannot make an ambiguous tool identity safe.

Conclusion

Duplicate FunctionTool names in OpenAI Agents SDK 0.19.2 have two failure modes:

strict providers reject the duplicate function definitions;
tolerant providers allow the request, while local dispatch keeps the later implementation.

The offline XBSTACK fixture verifies:

SDK validator       -> no error
Agent tools          -> ['lookup', 'lookup']
Local dispatch map   -> later lookup_orders only
First tool reachable -> False

Until an upstream release rejects the conflict, production systems should:

validate unique FunctionTool.name values after final registry assembly;
use explicit business-specific names or name_override;
organize larger tool sets with namespaces;
run the check in factory tests, plugin registration, and release gates;
keep a regression test during SDK upgrades.

This configuration should fail at local startup, not through a provider 400 or an incorrect external side effect.

References

Canonical article on XBSTACK：https://www.xbstack.com/en/ai/openai-agents-sdk-duplicate-tool-names/?utm_source=devto&utm_medium=community&utm_campaign=article_distribution&utm_content=openai-agents-sdk-duplicate-tool-names&ref=devto

标签：#AI #SoftwareEngineering #DeveloperTools #OpenAI Agents SDK #FunctionTool

Why LangGraph State Disappears After Run Cancellation: Streaming, Checkpoints and Recovery

xn — Thu, 30 Jul 2026 09:33:15 +0000

A production-focused update based on a real project: Why does streamed LangGraph output disappear after Stop? A LangGraph 1.2.9 and SQLite experiment compares 16 streaming cases, …

Why LangGraph State Disappears After Run Cancellation: Streaming, Checkpoints and Recovery Consistency

A user sees half an answer, presses Stop, and then watches that answer disappear on the next refresh or message. It is tempting to blame the frontend. It is equally tempting to switch LangGraph from the default durability mode to sync and assume the problem is solved.

The experiment in this article points to a more precise boundary:

A stream event that has reached the UI is not necessarily part of the authoritative LangGraph checkpoint. The decisive question is not only whether durability is sync, async, or exit; it is whether the visible progress crossed a completed graph-step boundary and became a state update.

I ran a deterministic matrix on Python 3.11.15, LangGraph 1.2.9, and AsyncSqliteSaver. The fixture uses no model and no external API. It compares two graph shapes, four durability inputs, and two termination paths, producing sixteen streaming cases plus one interrupt()/resume case.

On normal completion, every combination produced six visible chunks and six checkpointed chunks. When the consumer closed the stream after the third visible chunk, all four durability modes on a single long-running node produced the same mismatch: UI 3, checkpoint 0. When the same progress was modeled as completed graph steps, all four cases restored 3 of 3 chunks. The interrupt experiment added a second result: prior completed state was preserved, but code before interrupt() ran twice because the interrupted node restarted from its beginning on resume.

This is not a model-quality article. It is a state-semantics article. The practical question is:

When a user cancels a streaming LangGraph run, how do we keep the last visible UI state, the recoverable graph state, and any external side effects consistent?

Test design, environment, and explicit boundaries

Using a real LLM would introduce provider buffering, token batch size, network variance, and model latency. The fixture instead streams six deterministic strings:

LangGraph | streams | visible | progress | before | checkpoint.

There is a 60 ms delay between chunks. In the cancellation path, the consumer closes the async stream after the third visible update. That is a deliberately narrow model of a browser disconnect, a client that stops consuming, or an application that abandons a local run.

The recorded environment is:

Item	Recorded value
Python	3.11.15
LangGraph	1.2.9
langgraph-checkpoint-sqlite	3.1.0
Checkpointer	`AsyncSqliteSaver`
External model calls	0
External API calls	0
Stream format	`version="v2"`
Stream modes	`custom` and `updates`
Cancellation point	after 3 visible chunks
Durability inputs	default, sync, async, exit

The default case omits the durability argument. The current LangGraph reference documents the default as async: completed changes are persisted while the next step executes. sync persists completed changes before starting the next step. exit persists when the graph exits.

That wording matters. Durability controls when an existing graph-state change is written. It does not promise to convert arbitrary local variables or custom stream events inside an unfinished node into checkpointed state.

The three tasks were:

Monolithic stream node — one node emits six custom events, then returns its state only after the loop finishes.
Stepwise progress graph — each chunk is appended by a completed node execution, followed by a conditional edge back to the same node.
Interrupt and resume — a prepare node returns an approval payload; an approval node records a prefix event, calls interrupt(), and resumes with the same thread ID and Command(resume=True).

The fixture does not test LangSmith Agent Server cancellation endpoints, disconnect_mode, a remote Postgres checkpointer, reverse-proxy disconnect behavior, or real provider token streams. Closing a local async iterator is not identical to every managed runtime. The experiment establishes a local runtime/checkpointer boundary that a production deployment must verify again in its own stack.

The official persistence model: checkpoints are super-step state

LangGraph’s persistence documentation describes checkpoints as thread state snapshots created across graph execution. Full StateSnapshot checkpoints are associated with super-step boundaries. The runtime can also persist per-task writes so that, when one task in a super-step succeeds and another fails, the successful task does not have to be recomputed. Those pending writes are useful for fault tolerance, but they are not the same as an arbitrary snapshot of every in-memory value at every line of a node.

Streaming serves a different purpose.

custom can expose progress emitted from inside a node through get_stream_writer().
messages can expose model tokens.
updates exposes state updates after graph work produces them.
checkpoints exposes checkpoint creation events.

A UI may receive a custom event before the node returns. At that moment, three facts can all be true:

the node is still running;
the browser has rendered several chunks;
the graph state has not received those chunks as a node return value.

The checkpoint is the authoritative recovery source. Reusing a thread_id asks the checkpointer to load committed graph state. It does not ask the browser for the last text it happened to render.

Official issue #5672 reports the resulting product failure in LangGraph Platform/API: users see streamed content, cancel before the next checkpoint, and later resynchronize to an older backend state. The issue was originally filed against LangGraph 0.3.31. This article does not claim to reproduce the exact Agent Server implementation bug on 1.2.9. It reproduces the underlying local semantic boundary: a custom event emitted by an unfinished node is not automatically a state update merely because a consumer observed it.

Experiment 1: the UI saw three chunks and the checkpoint stored none

The monolithic node is intentionally simple:

async def long_streaming_node(state: StreamState) -> StreamState:
    writer = get_stream_writer()
    emitted = []

    for token in TOKENS:
        emitted.append(token)
        writer({
            "kind": "visible_chunk",
            "token": token,
            "visible_text": "".join(emitted),
        })
        await asyncio.sleep(0.06)

    return {
        "status": "completed",
        "emitted_chunks": emitted,
        "final_text": "".join(emitted),
        "step_index": len(emitted),
    }

When the node completes normally, all four durability inputs converge on the same business result:

six chunks reached the consumer;
six chunks were available from get_state();
the final text matched;
no runtime error occurred.

The SQLite write pattern differed. Default, sync, and async each recorded three checkpoints and ten write rows. exit recorded one final checkpoint and no intermediate writes. These counts are diagnostic evidence for this fixture, not a throughput benchmark.

When the stream closed after the third custom event, every durability mode produced the same state mismatch:

Durability	UI chunks	Missing	Authoritative match
default	3	3	no
sync	3	3	no
async	3	3	no
exit	3	3	no

This does not mean SQLite wrote nothing. The default, sync, and async cases each contained two checkpoints and seven write rows. exit contained one checkpoint. What they did not contain was the node-local emitted list, because the node never reached its return statement.

That distinction invalidates a common fix:

“Use durability="sync" so partial tokens are never lost.”

sync can narrow the persistence window for a completed step. It cannot persist a state update that the unfinished node never returned. There is no state update for the checkpointer to save.

This is the first production lesson: before changing durability, identify whether the disappearing object is a stream event, a returned graph-state update, a pending write, or an external business record. They are not interchangeable.

Experiment 2: completed graph steps restored all three visible chunks

The second graph changes the state boundary rather than the storage parameter. Each node execution appends one chunk and returns:

async def durable_step(state: StreamState) -> StreamState:
    index = int(state.get("step_index", 0))
    chunks = list(state.get("emitted_chunks", []))
    chunks.append(TOKENS[index])

    return {
        "status": "running" if index + 1 < len(TOKENS) else "completed",
        "emitted_chunks": chunks,
        "final_text": "".join(chunks),
        "step_index": index + 1,
    }

A conditional edge returns to durable_step until all six chunks exist. The consumer follows updates, so the visible representation now comes from a completed graph-state update rather than an arbitrary event emitted midway through a node.

Normal completion still produced six visible chunks and six checkpointed chunks for all durability inputs. The write cost increased:

default, sync, and async: eight checkpoints and thirty-five writes;
exit: one final checkpoint.

This is expected. A finer recovery boundary creates more persistent work. A production system should not create one graph node per token; the tiny steps exist only to make the semantic boundary measurable.

When the consumer closed after the third completed update, every case restored all three chunks:

Durability	UI chunks	Checkpoint chunks	Authoritative match
default	3	3	yes
sync	3	3	yes
async	3	3	yes
exit	3	3	yes

Default, sync, and async recorded four checkpoints and twenty-one writes. exit recorded one checkpoint and five writes.

The exit result needs careful wording. Closing the stream caused this graph execution to exit, and the runtime had an opportunity to persist the current state on that exit path. It does not prove that exit had stored every intermediate step while the graph was still running. It also does not prove what would happen if the process were killed before cleanup completed.

The stronger conclusion is independent of that nuance:

Durable partial progress requires an authoritative boundary. That boundary can be a completed graph step, an application-owned append-only event log, or a task record in a separate store. It cannot be merely “the browser received an event.”

What `sync`, `async`, and `exit` actually change

The three modes describe write timing for completed changes.

sync

A completed step is persisted before the next step starts. This is the strongest choice at a critical workflow boundary: approval decisions, durable task handoff, or a transition immediately before an external write. The trade-off is direct storage latency on the execution path.

async

Completed changes are persisted while the next step executes. This is the documented default and often provides a useful balance between recovery and throughput. There is still a cleanup and flush window. Discussion on issue #5672 in 2026 points to cancellation-path cleanup and pending async persistence as an implementation concern. The fixture did not patch the runtime or kill the process during cleanup, so that community analysis remains a source clue rather than a locally proven result.

exit

State is persisted when the graph exits. It can be appropriate for short, deterministic work that can be restarted as a whole. It is a poor fit when each step is an audit fact that must survive a worker crash.

A practical selection table is more useful than declaring one mode “best”:

Workload	Recommended design	Reason
State immediately before high-risk approval	`sync` and a separate node	persistence confirmed before advancing
Ordinary multi-step agent	default/`async` and idempotent nodes	balanced persistence and throughput
Short deterministic calculation	`exit`	fewer intermediate writes
Token-level UI rendering	stream plus application event log	tokens are display events, not graph checkpoints
Long external tool progress	durable task table and sequence	survives page, worker, and runtime changes
Irreversible side effect	idempotency key and domain database	checkpointing is not a business transaction

When debugging a cancellation rollback, ask these questions in order:

Was the disappearing object a stream event or graph state?
Had the node returned before cancellation?
Which source is used after refresh: browser memory, application event storage, or the checkpointer?
Did runtime cleanup complete?
Did an external side effect happen even though graph state did not advance?

Changing durability before answering those questions treats write timing as if it were state modeling.

`interrupt()` is a recoverable pause, not ordinary cancellation

Cancellation usually means “stop the current run.” interrupt() means “pause at a recoverable graph position and wait for external input.” Official documentation requires a checkpointer and a stable thread ID. Resuming with the same thread and Command(resume=...) provides the return value of the interrupt() call.

The fixture first completed a preparation node:

async def prepare(state):
    return {
        "prepared_payload": "draft-action-v1",
        "status": "prepared",
    }

The approval node then recorded observable prefix and suffix events:

def approval(state):
    side_effect_events.append("before_interrupt")

    approved = interrupt({
        "question": "Approve draft-action-v1?",
        "request_id": state["request_id"],
        "prepared_payload": state["prepared_payload"],
    })

    side_effect_events.append("after_interrupt")
    return {
        "approved": bool(approved),
        "status": "approved" if approved else "rejected",
    }

Before resume, the checkpoint contained prepared_payload: draft-action-v1 and the graph was positioned at the approval work. After Command(resume=True), the final state was approved.

The event counts expose the important behavior:

Event	Count
`before_interrupt`	2
`after_interrupt`	1

LangGraph did not restore a frozen Python stack and continue at the next line. It restarted the interrupted node from its beginning, matched the supplied resume value to interrupt(), and then continued. The result matches the official interrupt rules.

That behavior turns prefix side effects into a production risk. A call that creates an order, sends an email, appends an audit record, or decrements inventory before interrupt() may run again on resume.

Safe patterns are:

move irreversible work after interrupt();
put it in a separate node with its own checkpoint boundary;
use a stable operation ID and an idempotent upsert;
check the authoritative external result before retrying.

Official issues #6792 and #7361 report additional resume/replay edge cases involving subgraphs or a specific checkpoint ID. They are not evidence from this local fixture, but they justify a release gate: nested subgraphs, multiple interrupts, and checkpoint-specific resume paths must be tested against the exact version and deployment used in production.

Production repair: separate visible progress, recoverable state, and business truth

A cancellation model with only running and completed is too coarse. A user can request cancellation while a model is generating, a tool is executing, an external API has completed but the local result has not been committed, or async checkpointer cleanup is still running.

A more explicit state machine is:

queued
  -> running
  -> cancel_requested
      -> cancelled_confirmed
      -> completed_before_cancel
      -> externally_completed_local_pending
      -> cancel_failed
  -> failed
  -> completed

cancel_requested represents user intent. It is not proof that execution stopped. externally_completed_local_pending is essential for payment, email, ticket, upload, or job systems: the external world changed, but local graph state may still be behind. Blindly replaying from the last checkpoint can duplicate the operation.

I use three persistence layers for this class of system.

1. UI stream

This layer optimizes latency. Every event should carry at least:

run_id
sequence
event_type
timestamp
safe payload or payload reference

The UI can render it optimistically. It should not silently promote the event to durable conversation history.

2. Application event ledger

This layer stores progress the product promises to retain. A minimal append-only shape is:

run_events(
  run_id,
  sequence,
  event_type,
  payload_hash,
  safe_payload,
  created_at,
  UNIQUE(run_id, sequence)
)

It does not have to write every token. A system can batch every 200–500 ms, store sentence boundaries, record tool phases, or persist a partial message object. Its job is to reconstruct what the user was shown after page reload or device change.

3. LangGraph checkpoint

This layer stores recoverable graph state, next nodes, task writes, and thread history. It is a workflow recovery mechanism. It should not be forced to contain every animation or token, and it should not replace an order, approval, or task database.

A production cancellation flow can then be:

The client sends a cancel request containing run_id; it does not only close the SSE connection.
The backend atomically records cancel_requested, who requested it, and when.
The runtime stops cancellable work. Non-cancellable external work enters reconciliation.
The application event ledger commits its last confirmed sequence.
Completed graph steps persist according to durability.
The backend sets a confirmed final status.
Reload merges authoritative graph state with the event ledger by run and sequence.

Every layer needs idempotency. run_id + sequence deduplicates visible events. operation_id deduplicates external effects. Recovery checks the domain database before retrying a tool.

Do not turn every token into a graph node

The 3/3 stepwise result is not a recommendation to checkpoint every token.

In the recorded fixture, default, sync, and async normal completion produced:

monolithic graph: three checkpoints and ten writes;
stepwise graph: eight checkpoints and thirty-five writes.

Those counts only describe this deterministic local fixture, not production latency or cost. They still reveal the direction of the trade-off: a finer recovery boundary usually creates more serialization and storage work.

Good graph boundaries carry business meaning:

a plan is accepted;
a tool call finishes;
a batch of document pages is parsed;
an approval package is ready;
an external object ID is obtained;
a retryable shard completes.

Poor graph boundaries are display details:

every token;
animation percentage;
arbitrary character chunks;
transient UI state that can be derived later.

For long text generation, tokens can stay on the stream. An application event ledger can persist paragraphs or a partial message every bounded interval. The graph state receives the complete message when the node finishes. On cancellation, the partial object can be labeled aborted_partial rather than disappearing or pretending to be a final answer.

For a long tool task, let the tool own a durable task_id and stage status. The graph stores the task ID, last confirmed stage, and recovery policy. The page reads progress from the task API. A graph worker restart no longer erases progress that existed only in a node’s local variables.

Regression tests that prove the repair

A repair is incomplete if it only passes the normal completion path. At minimum, automate four scenarios and assert both the last visible sequence and the last authoritative backend sequence.

Natural completion

Six progress events are emitted. The UI shows six. The event ledger stores six. After the node finishes, graph state contains the complete result. This protects the ordinary path from cancellation-specific changes.

Cancellation at a fixed sequence

After the third event, send a real cancel request, wait for a confirmed terminal state, create a fresh client, and reload the same thread. The result must match product policy: either preserve three events and label them partial, or deliberately discard them and remove them before presenting a stable state. “Show three now, silently lose them after refresh” is not acceptable.

Cancellation racing an external side effect

Complete the external operation immediately before and immediately after the cancel request. Assert that the same operation_id produces one order, one ticket, one email, or one file object. The important assertion is in the external system, not only the graph output.

Worker/process failure

After one meaningful business step, terminate the worker process and restart it. Reuse the same thread and compare sync, default async, and exit. Record the last recoverable step rather than inferring it from documentation.

Logs should connect five identifiers:

thread_id — long-lived workflow;
run_id — one execution attempt;
sequence — visible event ordering;
checkpoint_id — recoverable graph snapshot;
operation_id — external side effect.

Useful production metrics include:

cancel request to confirmed stop latency;
UI sequence minus checkpoint/event-ledger sequence;
partial-message recovery success;
duplicate side-effect prevention count;
nodes re-executed after resume.

A low cancellation count does not prove consistency. A stable zero or explainable value for these divergence metrics is much stronger evidence.

Failures, limitations, and claims this test does not support

The experiment intentionally retains failure evidence.

First, all four monolithic cancellation cases failed the consistency test: UI 3, checkpoint 0. This is the central negative result.

Second, the first regression-test loader dynamically imported the experiment without registering the module in sys.modules. Python 3.11 dataclass type resolution failed. Registering the module fixed the harness, and the semantic test then passed. This was a test-harness defect, not a LangGraph defect.

Third, several orphaned Astro development processes had exhausted system file descriptors, causing the first isolated installation attempt to fail with Too many open files in system. Stopping those project processes allowed the clean Python 3.11 environment to install. That environmental incident is excluded from the LangGraph conclusions.

The fixture also does not test:

Agent Server cancellation endpoints;
disconnect_mode="cancel" versus "continue";
Postgres checkpointer behavior under process death;
parallel pending writes within one super-step;
every nested-subgraph interrupt path;
provider-specific token streaming and abort signals;
a reverse proxy disconnect while backend execution continues;
durability performance, throughput, or cost.

It would therefore be inaccurate to say “LangGraph 1.2.9 still has exactly issue #5672” or “stepwise nodes solve every cancellation problem.” The supported claim is narrower:

On the local LangGraph 1.2.9 runtime, the experiment reproduces the underlying state boundary behind the same user-visible rollback: unfinished in-node stream output is not checkpointed state, while completed graph-step updates are recoverable.

Final decision: never let a stream impersonate a database

For a personal demo where users can regenerate an answer, discarding an unfinished response may be acceptable. The UI should make that policy explicit and remove partial content consistently.

For customer support, research, finance, approvals, long-running tools, or any auditable workflow, the UI and backend cannot disagree about what happened. Such systems need:

an explicit distinction between stream events, graph state, and business facts;
state updates at meaningful business boundaries;
an application event ledger for visible progress the product promises to retain;
stable thread, run, sequence, checkpoint, and operation identifiers;
a cancellation confirmation state machine;
idempotent side effects around interrupts and retries;
real disconnect, worker-restart, and same-thread regression tests.

The experiment’s most useful result is not a winning durability parameter. It is a clean separation of responsibilities:

Streaming helps users see progress quickly. Checkpoints let the graph recover. Domain storage proves what changed in the outside world. The three layers may reference one another, but none can safely impersonate the others.

Before implementing “Stop generation,” decide what must survive: a visible partial message, recoverable workflow state, or an external business fact. Consistency becomes tractable only after those three objects have explicit owners.

FAQ

Does `durability="sync"` prevent state loss on cancellation?

It protects completed graph-step changes by persisting them before the next step. It cannot save node-local progress that has been streamed but not returned as a state update. In all four monolithic cancellation cases, the UI showed three chunks and the checkpoint held zero.

Does `interrupt()` resume at the exact Python line?

No. The interrupted node restarts from its beginning. Code before interrupt() runs again, so side effects must be idempotent, moved after the interrupt, or isolated in a separate node.

Should every token be checkpointed?

Usually not. Stream tokens for latency, persist paragraphs or partial messages in an application event ledger, and write the complete message to graph state when the node finishes.

Why did `exit` preserve three chunks in the stepwise cancellation test?

Closing the iterator caused the graph execution to exit, giving the runtime an opportunity to persist the current state. It does not mean every step had already been persisted during execution, and it does not establish behavior under hard process termination.

Is local browser storage enough for partial responses?

It can mask a rollback in one browser, but it does not survive every reload, device change, or multi-instance synchronization path. If the product promises that visible partial content will remain, persist it in a backend event ledger.

Official sources and related XBSTACK guides

Canonical article on XBSTACK：https://www.xbstack.com/en/ai/langgraph-cancel-run-streaming-checkpoint-state-loss/?utm_source=devto&utm_medium=community&utm_campaign=langgraph_cancel_checkpoint_consistency&utm_content=langgraph-cancel-run-streaming-checkpoint-state-loss&ref=devto

标签：#AI #SoftwareEngineering #DeveloperTools #LangGraph #Streaming

MCP initialize Removed: Reproducing Unknown MCP Session Across Replicas

xn — Tue, 28 Jul 2026 02:49:24 +0000

The first MCP request returns 200. The next tools/call returns 400 with Unknown MCP session. The endpoint did not change, the tool schema is valid, and the network is healthy. What changed was the replica.

In a legacy Streamable HTTP flow, initialize can create protocol state in Replica A and return Mcp-Session-Id. A round-robin proxy may send the following tool call to Replica B. If the session lives only in A's memory, B cannot resolve it.

POST /mcp initialize
  -> Replica A
  <- 200
  <- Mcp-Session-Id: session_a_123

POST /mcp tools/call
Mcp-Session-Id: session_a_123
  -> Replica B
  <- 400 Unknown MCP session

I built a small fixture with two legacy replicas, two draft-shaped stateless replicas, a real localhost round-robin proxy, and an external SQLite store. The legacy path produced a deterministic 200/400 result. The self-contained path alternated between draft-a and draft-b successfully. The wire lab passed 6/6 cases.

The important boundary is not 'state versus no state.' It is protocol state versus application state.

Protocol facts such as version, client information, capabilities, and routing metadata should be available on the current request. Durable product state still exists, but it should use explicit domain handles such as basket_id, approval_id, browser_id, or task_id, backed by storage every replica can reach.

That makes the application contract inspectable:

the handle has an owner and tenant;
the record has an explicit lifecycle;
expired or completed objects are rejected intentionally;
concurrent writes can use versions or ETags;
side-effecting tools can use idempotency keys.

Sticky sessions can reduce the immediate failure rate during migration, but they do not preserve process memory after restarts, rolling deployments, or connection movement. A shared session store can support the old protocol temporarily, but it should not become a permanent opaque blob containing client capabilities, identity, business objects, and recovery state.

The locked MCP 2026-07-28 revision is still labeled RC / pre-release as of publication. This is therefore an RC experiment and migration plan, not a final-spec conformance claim. Package version, SDK API shape, and the wire protocol also need to be verified separately; installing an SDK v2 prerelease does not prove that production traffic is speaking the new protocol.

The complete article includes the request examples, replica routing evidence, explicit-handle lifecycle, SDK v2 smoke test, old/new client-server compatibility matrix, rollout gates, and rollback criteria.

Read the full experiment and migration checklist: https://www.xbstack.com/en/ai/mcp-2026-07-28-stateless-migration/?utm_source=devto&utm_medium=referral&utm_campaign=mcp_2026_07_28_stateless_migration&utm_content=article_body

AI Agent Tool Authorization: Build a Policy Gate Before Tool Execution

xn — Mon, 27 Jul 2026 04:21:31 +0000

A tool being visible to an agent does not mean every generated call is authorized. The enforcement boundary belongs between the proposed tool call and the real executor.

An Agent that can see send_email, query_customer, and issue_refund has been given capabilities. That does not mean every model-generated call is authorized. One of the most dangerous architecture mistakes is treating “the tool is registered,” “the arguments match JSON Schema,” “the guardrail passed,” and “the user may execute this action” as the same fact.

Consider a call that is syntactically valid: query a customer record, then send the result to an external email address. The input can look normal, the arguments can match the schema, and both tools can belong to the Agent. The combination can still be data exfiltration. What is missing is a deterministic pre-execution authorization layer that reads trusted identity, tenant, scope, resource ownership, destination, and exact arguments before a real side effect occurs.

The load-bearing design rule is: put a Policy Gate between the model output and the Tool Executor, not only in the prompt, UI confirmation, or tool description. At minimum, it should return ALLOW, REQUIRE_CONFIRM, STEP_UP, or BLOCK, and generate a structured, redacted, traceable audit record for every decision.

Guardrails, HITL, and authorization solve different problems

The OpenAI Agents SDK documents Tool Guardrails and Human-in-the-loop. Tool Guardrails can inspect function-tool input and output, block or replace results, and trigger a tripwire. HITL can turn a sensitive call into an interruption that waits for approval or rejection. Guardrails Human-in-the-loop

Authorization remains distinct. Issue #2868 requests per-tool authorization middleware that evaluates identity, role, scope, rate limits, and session context before execution, supports non-binary decisions, and emits structured audit records.

Layer	Question	Typical inputs	Typical result
Authentication	Who is calling?	Token, certificate, session	Verified principal
Capability exposure	Which tools can the Agent see?	Registry, filters	Candidate tool set
Guardrails	Is the content or argument shape safe and valid?	Prompt, arguments, result	Pass, block, replace, tripwire
Authorization	May this principal execute this exact call now?	Identity, tenant, scope, resource, parameters, policy	ALLOW, BLOCK, STEP_UP, DEFER
HITL	Does a person need to decide this call?	Pending call and context	Approve, reject, edit, expire
Idempotency	Will the same side effect land only once?	Stable key and ledger	Execute once or reuse result

HITL can handle REQUIRE_CONFIRM, but it cannot replace authorization. Cross-tenant access, missing scope, and unknown tools should normally be blocked immediately rather than consuming human review capacity.

Where the Policy Gate belongs

A production topology should look like this:

User Request
  -> Agent / Model
  -> Proposed Tool Call + Arguments
  -> Policy Enforcement Point
       -> Identity verification
       -> Tenant and ownership check
       -> Scope / role check
       -> Argument and destination policy
       -> Risk and assurance evaluation
       -> ALLOW / REQUIRE_CONFIRM / STEP_UP / BLOCK
  -> Approval Service when required
  -> Idempotency Ledger
  -> Tool Executor
  -> Audit / Trace

The Policy Enforcement Point intercepts the call; a Policy Decision Point evaluates the rules. A small application can keep both in one process. A distributed platform can use a gateway or external policy service. Two conditions are non-negotiable:

the Tool Executor cannot be reached around the gate;
identity and tenant context cannot come from model-generated arguments.

The model may propose target_tenant_id=tenant-b, but the trusted tenant must come from a verified session, service token, or gateway context. Treating model-generated tenant data as an authorization fact is equivalent to letting the request sign its own access pass.

The local lab: seven per-call decisions

The experiment lives in:

experiments/agent-tool-authorization-policy-gate/

It evaluates a trusted principal and a proposed call:

Principal(
    user_id="user-17",
    tenant_id="tenant-a",
    scopes={"customer:read", "email:send", "refund:write"},
    assurance_level=1,
)

ToolCall(
    name="issue_refund",
    arguments={"amount": 8000, "currency": "CNY"},
    target_tenant_id="tenant-a",
    risk_level="write",
)

All seven deterministic cases pass:

Case	Expected decision	Result
Same-tenant customer read	ALLOW	Allowed
Cross-tenant customer read	BLOCK	Denied immediately
Customer data sent to an external recipient	REQUIRE_CONFIRM	Human confirmation
CNY 8,000 refund with low assurance	STEP_UP	Stronger authentication
Record deletion	REQUIRE_CONFIRM	Human confirmation
Refund without the required scope	BLOCK	Denied immediately
Unregistered arbitrary shell tool	BLOCK	Default deny

The audit record redacts sensitive fields:

{
  "decision": "REQUIRE_CONFIRM",
  "reason": "customer_data_to_external_recipient",
  "policy_id": "email-dlp-v1",
  "audit_id": "c3d46b41f63c84c50927",
  "redacted_arguments": {
    "recipient": "partner@external.test",
    "body": "[REDACTED]",
    "contains_customer_data": true
  }
}

The lab includes five unit tests and a generated verification.json covering all seven cases. It does not integrate an actual Agent SDK, so the evidence verifies policy semantics rather than end-to-end SDK behavior.

Why binary allow/deny is insufficient

ALLOW

Use for low-risk, same-tenant, correctly scoped operations with clear ownership, such as reading the current user's order status.

REQUIRE_CONFIRM

Use for side effects that a human can meaningfully review, such as sending customer data externally, deleting a record, or issuing a moderate refund. The Policy Gate first proves that the request is not obviously unauthorized, then routes it to HITL.

STEP_UP

Use when the principal may have permission but the current authentication assurance is too weak. A high-value refund can require password re-entry, MFA, a hardware key, or an administrator signature. This differs from ordinary approval: the goal is to strengthen the caller's identity evidence.

BLOCK

Use for cross-tenant access, missing scope, ownership mismatch, unknown tools, expired policy, or disabled targets. Fail closed when the policy service is unavailable; a timeout must not become implicit permission.

Production systems may add MODIFY or DEFER, for example capping an amount or scheduling a call inside a business window. Any policy-side mutation should preserve an original argument digest and a reason so the audit trail remains reconstructable.

Combining RBAC, ABAC, and argument-level policy

RBAC answers coarse questions such as whether a support role may access a refund tool. ABAC answers contextual questions such as whether this support user can refund this order for this tenant at this amount.

A real decision can inspect:

principal: user ID, service ID, role;
tenant and organization boundary;
scope: customer:read, refund:write;
resource: owner, region, classification;
arguments: amount, recipient, environment;
session: assurance level, device, risk score;
time: policy version, expiry, maintenance window;
rate: calls or cumulative amount per window.

For example:

IF tool = issue_refund
AND principal.scope includes refund:write
AND order.tenant_id = principal.tenant_id
AND amount <= 500
THEN ALLOW

IF amount > 500 AND amount <= 5000
THEN REQUIRE_CONFIRM

IF amount > 5000 AND assurance_level < 2
THEN STEP_UP

The model does not evaluate this policy. It can explain why a refund was requested, but it cannot decide that its caller satisfies the scope or amount threshold.

Integration with OpenAI Agents SDK, LangGraph, and MCP

OpenAI Agents SDK

needs_approval and Tool Guardrails are integration points. Evaluate the Policy Gate first: continue on ALLOW, return a denial on BLOCK, surface an interruption on REQUIRE_CONFIRM, and route STEP_UP to stronger authentication. Preserve the policy ID, decision reason, and audit ID rather than reducing the result to a boolean.

LangGraph

Add a dedicated Policy Node before the Tool Node. It reads trusted context and the proposed Tool Call, then writes the decision into graph state. Route REQUIRE_CONFIRM to interrupt(), BLOCK to a safe terminal state, and only ALLOW to the executor. The existing LangGraph Human-in-the-loop guide covers pause and resume; the Policy Gate determines whether the call is eligible to enter approval at all.

MCP

Tool filtering can reduce which MCP capabilities a client sees, but the remote server should still authorize every tools/call using trusted principal, tenant, resource, and arguments. Client-side approval cannot replace server-side authorization because a faulty or malicious client may bypass the UI and call the server directly. Continue with MCP Server production governance and MCP security best practices.

Minimum audit fields

Store at least:

audit_id
request_id / trace_id
principal_id
trusted_tenant_id
tool_name
tool_version
argument_digest
redacted_argument_summary
policy_id
policy_version
decision
reason
approval_id when present
assurance_level
created_at
executor_result_id after execution

Do not place complete customer records, tokens, or email bodies into the audit log. Store a redacted summary and irreversible digest, then use controlled access to retrieve authoritative data during an investigation.

Record the policy version as well. If an approval waits for two days, permission rules may change. Resume should re-evaluate the current policy rather than blindly trusting an old approved flag.

Why authorized calls still need idempotency

The Policy Gate answers whether a call may execute, not whether it already executed. Queue redelivery, network timeout, worker restart, or a duplicate click can send the same authorized call to the executor again.

A side-effecting tool needs a stable key:

idempotency_key = hash(
  tenant_id,
  business_action,
  business_object_id,
  approved_call_id
)

The executor checks a ledger first: return the stored result if complete, reject concurrent work if processing, and retry only under explicit failure policy. Authorization, approval, idempotency, and audit are separate layers.

Minimum production checklist

Can the Tool Executor be reached only through the Policy Gate?
Do identity and tenant come from trusted context?
Are unregistered tools default-denied?
Is cross-tenant access blocked immediately?
Do high-risk actions use REQUIRE_CONFIRM or STEP_UP?
Does a policy-service failure fail closed?
Are audit arguments redacted?
Does resume re-evaluate the current policy version?
Do writes use idempotency keys?
Has the team tested bypassing the frontend and calling the Tool API directly?
Are revocation, expiry, and emergency disable controls available?

Final decision

Agent tool safety cannot rely on a compliant model. The Tool Registry determines visible capabilities, schemas determine argument shape, guardrails inspect content and format, HITL collects human decisions, and authorization proves whether the current principal may execute this exact call.

A reliable system lets the model propose an action, lets the Policy Gate decide whether it can proceed, and lets the Tool Executor accept only authorized requests protected by idempotency. Prompt injection, model misunderstanding, or unauthorized arguments then meet a deterministic execution boundary before they can affect the real system.

Continue with AI Agent Tool Use: registry, validation, and audit, AI Agent Security, and OpenAI Agents SDK RunState approval resume.

Full experiment, code, and production checklist: https://www.xbstack.com/en/ai/ai-agent-tool-authorization-policy-gate/?utm_source=devto&utm_medium=community&utm_campaign=ai_agent_tool_authorization_20260727&utm_content=article_body&ref=devto

n8n AI Agent Not Calling Tools: The Request Contract Behind tool_choice

xn — Sun, 26 Jul 2026 15:17:04 +0000

An n8n AI Agent can have tools connected, receive a system instruction that says “you must call the tool,” and still return prose without executing anything.

The first diagnostic is not the final answer. It is the execution trace:

if the Tool node never starts, inspect the model request and provider behavior;
if the Tool node starts and fails schema validation, inspect arguments and JSON Schema;
if the Tool succeeds but the agent reports failure, inspect the returned Tool message;
if single-turn execution works but later turns do not, inspect memory persistence.

n8n issue #31135 reports a specific first-iteration boundary: tools are present, but tool_choice: required is not explicitly sent. A compatible provider that interprets the missing field as auto may answer directly.

I built a provider-free deterministic contract lab. It is not a model benchmark:

Scenario	Runs	Tool calls	Text replies
Missing `tool_choice`	20	0	20
First-iteration proxy injects `required`	20	20	0

The proxy also verifies three safeguards: do nothing when no tools exist, preserve explicit choices such as none, and stop forcing after an assistant tool call appears in the message history.

The broader debugging path still includes provider compatibility, ambiguous tool descriptions, invalid schemas, structured Tool Results, and memory backends that omit tool-call evidence.

Canonical analysis and complete code:

https://www.xbstack.com/en/ai/n8n-ai-agent-not-calling-tools/?utm_source=devto&utm_medium=referral&utm_campaign=n8n_agent_tool_call&utm_content=article_canonical

OpenAI Agents SDK RunState: Resume Tool Approvals Without Duplicate Side Effects

xn — Fri, 24 Jul 2026 06:23:10 +0000

OpenAI Agents SDK RunState can persist a run that is waiting for human approval, let the original process exit, and resume the approved or rejected run in another worker.

That solves durability. It does not solve exactly-once execution.

In my reproducible openai-agents==0.18.3 lab:

side effects before approval: 0
side effects after cross-process approval and resume: 1
side effects after rejection: 0
side effects after replaying the same approved state in two workers: 2
side effects after adding an idempotency ledger: 1

Reproduction setup

Python 3.10.2
openai-agents 0.18.3
RunState schema 1.12
SQLite
deterministic custom Model
no external model provider API

The deterministic model always emits one deploy_release tool call, then returns a final message after receiving the tool result. This isolates SDK interruption, serialization, and resume behavior from model variance and network behavior.

Pause before the side effect

from agents import function_tool

@function_tool(needs_approval=True)
def deploy_release(release: str, idempotency_key: str) -> str:
    return f"deployed:{release}"

result = await Runner.run(
    agent,
    "Deploy release 2026.07.24",
    context=app_context,
)

assert len(result.interruptions) == 1
assert effect_count() == 0
state = result.to_state()

needs_approval=True is enforced by the runner at tool execution time. The model has already proposed a call and produced arguments, but the Python function has not run.

Serialize only durable context

RunContextWrapper.context participates in RunState serialization. A plain mapping containing a token can therefore write that token into the state blob.

Use a strict serializer that keeps identifiers and reconstructs runtime dependencies later:

def context_serializer(context: AppContext):
    return {"tenant_id": context.tenant_id}

payload = state.to_json(
    context_serializer=context_serializer,
    strict_context=True,
)

On restore, obtain credentials from a secret manager instead of the blob.

The application still needs an approval record

RunState is an SDK execution snapshot. It should not be the only record for:

who requested the action
who is authorized to approve it
tenant and resource boundaries
expiry and revocation
the immutable tool argument digest
queue delivery and worker leases
whether the external side effect already happened

A production design separates:

run_state_blob       SDK execution snapshot
approval_request     authorization and lifecycle
idempotency_ledger   external-effect ownership and result reuse

The approval UI should display an immutable review snapshot. Canonicalize the tool arguments and store a SHA-256 digest. Before execution, the worker extracts the arguments from RunState and recomputes the digest. A mismatch invalidates the approval.

Resume in another process

state = await RunState.from_json(
    initial_agent,
    stored_payload,
    context_deserializer=context_deserializer,
    strict_context=True,
)

interruption = state.get_interruptions()[0]
state.approve(interruption)
await persist(state)

The worker then rebuilds a compatible agent and resumes:

agent = AGENT_FACTORIES[record.app_state_version](runtime_dependencies)
state = await load_state(agent)
result = await Runner.run(agent, state)

from_json() still requires an initial Agent because JSON cannot recreate Python functions, tool implementations, clients, database connections, or a complete executable graph.

The duplicate-delivery failure

Worker A consumed the approved state and executed the tool. I then replayed the original approved state to Worker B instead of using Worker A's new result.

{
  "worker_a": "deployed:2026.07.24",
  "worker_b": "deployed:2026.07.24",
  "effect_count": 2
}

The SDK cannot know whether this is a queue redelivery, a lost acknowledgement, an operator retry, or a second legitimate resume. The snapshot says the call is approved and unfinished, so executing it again is consistent.

Claim execution with an idempotency ledger

connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
    "SELECT result FROM idempotency_ledger WHERE idempotency_key = ?",
    (idempotency_key,),
).fetchone()

if existing:
    return f"reused:{existing[0]}"

result = perform_external_effect()
connection.execute(
    "INSERT INTO idempotency_ledger(idempotency_key, result) VALUES (?, ?)",
    (idempotency_key, result),
)
return result

The operation key should be generated by the application, for example:

tenant_id + approval_id + logical_operation + target_resource

If the downstream API supports an idempotency key, forward the same value. A local ledger cannot close the crash window where the remote operation succeeds but the local result is not recorded. In that case, query the downstream system or mark the operation as uncertain and require reconciliation.

Production checklist

verify zero side effects before approval
keep approval authorization separate from RunState
persist a tool argument digest
encrypt state blobs and apply tenant ACLs
route old states through versioned Agent factories
design queues for at-least-once delivery
make side-effecting tools idempotent
test rejection, expiry, revocation, redelivery, lost ACKs, and corrupted blobs

The complete article includes the experiment files, state shapes, SQL schema, failure logs, and architecture diagrams.

Read the canonical article on XBSTACK

AI SDK 7 Migration: The Production Breakages a Codemod Cannot Fix

xn — Fri, 24 Jul 2026 03:41:13 +0000

AI SDK 7 Migration: The Production Breakages a Codemod Cannot Fix

Most AI SDK 7 migrations look small in a pull request.

A few option names change. TypeScript points at a callback. The app builds again, the chat page streams text, and the migration is declared complete.

That is enough for a prototype. It is not enough for a production system that calls tools, stores conversation state, retries failures, resumes interrupted runs, or performs external side effects.

I built a provider-free migration lab with two isolated fixtures:

Node.js 22.18.0
ai@6.0.230
ai@7.0.31
strict TypeScript
deterministic mock language models from ai/test
no API key, paid request, provider latency, or model-quality variance

The goal was not to benchmark speed. It was to compare control flow and result semantics under identical tool-calling, persistence, failure, cancellation, retry, and timeout scenarios.

Start with the runtime boundary

Before changing any API names, check two requirements that a codemod cannot solve:

AI SDK 7 requires Node.js 22 or newer.
AI SDK 7 is ESM-only and no longer supports CommonJS require().

A minimal package boundary looks like this:

{
  "type": "module",
  "engines": {
    "node": ">=22"
  }
}

Your local environment, CI worker, serverless runtime, and production image must all satisfy the same boundary. Otherwise, the migration can pass locally and fail after deployment.

The visible renames are the easy part

The obvious API changes are straightforward:

// AI SDK 6
streamText({
  model,
  system: 'You are an order assistant.',
  onFinish({ text }) {
    console.log(text);
  },
});

// AI SDK 7
streamText({
  model,
  instructions: 'You are an order assistant.',
  onEnd({ text }) {
    console.log(text);
  },
});

The event stream also moves from fullStream to stream.

A codemod can help with syntax. It cannot decide what your application meant when it read top-level tool fields, persisted raw SDK messages, retried a side-effecting tool, or handled cancellation after a tool had already completed.

Breakage 1: top-level tool calls no longer mean what some v6 code assumes

Both fixtures ran the same two-step workflow:

user asks for order A-100
        ↓
model emits lookupOrder tool call
        ↓
tool returns ready_for_pickup
        ↓
model returns final answer

Both versions produced the same final text:

Order A-100 is ready for pickup.

But their result shapes differed:

Field	AI SDK 6 fixture	AI SDK 7 fixture
Top-level `toolCalls`	0	1
Raw `response.messages`	3	1
Final-step tool calls	read from step data	`finalStep.toolCalls = 0`

In AI SDK 7, top-level tool fields represent the full run. Code that previously treated them as “the final step” can silently change behavior.

The migration rule is explicit:

const allRunToolCalls = result.toolCalls;
const finalStepToolCalls = result.finalStep.toolCalls;

Audit every read of toolCalls and toolResults. Decide whether each call site needs full-run history or final-step-only semantics.

Breakage 2: `response.messages` is not a stable database contract

The same deterministic workflow produced:

three raw response messages in the v6 fixture;
one raw response message in the v7 fixture.

That does not make either version wrong. It means the SDK response shape is not the right long-term storage model for your product.

A safer persistence boundary is application-owned:

conversation
  └── message
      └── run
          └── step
              └── logical tool_call
                  ├── tool_attempt 1
                  ├── tool_attempt 2
                  └── tool_result

Persist normalized business records first. Keep the raw SDK payload only as optional audit evidence.

This protects your database from changes in message count, nesting, aggregation, or provider-specific metadata.

Breakage 3: abort is not rollback

I tested three interruption reasons:

page_closed
network_disconnected
manual_cancel

Each fixture completed a tool call, committed the result, and then aborted before final text generation.

The invariant was:

the user message survives;
the completed tool result survives;
the final assistant response is absent;
the run is marked aborted;
the tool executes exactly once.

Once an external side effect has succeeded, closing the page does not undo it. Recovery must resume from persisted state, not replay the tool blindly.

For every side-effecting tool, use an idempotency key such as:

tenant_id + run_id + logical_tool_call_id

Breakage 4: provider retries and tool retries are different policies

A provider retry repeats model I/O. A tool retry may repeat a payment, email, database write, deployment, or ticket creation.

The fixture models one logical tool call with two attempts:

attempt 1 → transient failure
attempt 2 → success
one successful result returned to the model

Track attempts separately from the logical call:

type ToolAttempt = {
  logicalCallId: string;
  attempt: number;
  status: 'running' | 'failed' | 'succeeded';
  error?: string;
};

Do not let an SDK step double as your retry ledger.

Breakage 5: timeout needs more than one budget

The v6 fixture uses an application-owned timer and AbortController.

The v7 fixture directly verifies a first-class total timeout:

const result = await generateText({
  model,
  prompt,
  timeout: {
    totalMs: 20,
  },
});

AI SDK 7 also exposes broader timeout controls in its public API. In production, separate budgets for the whole run, individual steps, idle chunks, and tools. Verify the exact type definitions for the version in your lockfile, and add a deterministic fixture before relying on a new timeout field.

A timeout is not complete until it is represented in persisted state and propagated through providers and tools.

Real HTTP/SSE disconnects change the recovery problem

The deterministic fixtures prove state transitions, but I also added a real localhost HTTP/SSE test using Node's native server and fetch. The client disconnected immediately after receiving a completed tool result.

In request-bound mode, the observed state was:

Field	Result
`run.status`	`aborted`
Tool executions	`1`
Tool result saved	`true`
Final response saved	`false`
Disconnect observed	`true`

Recovery reused the same run_id and idempotency key. The application checked the tool ledger before execution and reused the stored result:

tool executions before resume = 1
tool executions after resume  = 1
duplicate executions          = 0
final response saved          = true

That ordering matters. The user message, planned tool call, executing attempt, and completed tool result should be committed independently of the final assistant response. Otherwise, a broken stream can erase the evidence needed to avoid replaying a side effect.

consumeStream() is useful for continuing server-side stream consumption and completion callbacks after a client disconnect. It does not know whether an order, email, deployment, or CRM update has already happened. It cannot replace an application-owned tool ledger and idempotency check.

I also tested a detached mode in which the run was not cancelled with the HTTP connection. The client disconnected, while the run completed with one tool execution and a saved final response. This proves that request and execution lifecycles can be separated. It does not turn an in-process promise into a durable job: process restarts, deployments, instance eviction, and delayed approvals still require a queue, worker, or durable workflow owner.

`chunkMs` and Cloudflare 524 are different timeout layers

A local reverse proxy used a deliberately tiny 70 ms read-silence budget:

slow first byte      -> simulated 524
heartbeat every 35ms -> 200

This was not a real Cloudflare edge test: there was no public domain, Ray ID, or production edge node. It only verifies the boundary between application state and proxy read behavior.

The controls are separate:

Layer	What it controls	What it cannot replace
Browser/request	Page close or network disconnect	Knowledge of whether a tool completed
AI SDK `chunkMs`	Silence between stream chunks	A reverse proxy's read budget
AI SDK `toolMs`	One tool's execution budget	Side-effect reconciliation
Cloudflare 524	Proxy waiting for the origin response	Persisted run and step state

A heartbeat can keep a proxy connection active, but it does not prove business progress. A useful heartbeat should include a run_id, step_id, status, and update time, and it still needs a total SLA. Tasks that may outlive the HTTP budget should return a job_id, persist each step and tool result, and continue under a queue or durable workflow.

A safer rollout order

The migration sequence I would use is:

Upgrade local development and CI to Node.js 22 and ESM.
Run the official migration tooling for mechanical renames.
Audit all reads of top-level tool fields.
Move final-step-only logic to finalStep.
Introduce version-neutral persistence.
Propagate AbortSignal through providers and tools.
Add idempotency keys and attempt-level retry records.
Add deterministic cancellation and timeout fixtures.
Migrate one internal workflow behind a feature flag.
Compare normalized events before moving production traffic.

Reproducible evidence

The complete dual-version fixture, raw JSON results, migration diff, and persistence architecture are public:

https://github.com/xbstack/xbstack-ai-sdk-7-migration-demo

The full production migration guide, including the complete schema and rollback checklist, is available here:

https://www.xbstack.com/en/ai/vercel-ai-sdk-7-migration-production/?utm_source=devto&utm_medium=community&utm_campaign=ai_sdk7_migration&utm_content=article_body

The main conclusion is simple:

AI SDK 7 is worth upgrading to, but a safe migration begins by separating application state from SDK state.

A build that passes after a codemod proves syntax compatibility. It does not prove that your tools, persistence, cancellation, retries, or recovery semantics are still correct.

I Tested Kimi K3 on a Real Astro Codebase: Strong Cross-File Analysis, Unsafe First Fix

xn — Thu, 23 Jul 2026 03:40:54 +0000

I Tested Kimi K3 on a Real Astro Codebase: Strong Cross-File Analysis, Unsafe First Fix

Kimi K3 arrived with a 2.8-trillion-parameter architecture, up to a 1M-token context window, and a strong focus on long-horizon coding and agent tasks. Those specifications are interesting, but they do not answer the question I care about as an independent developer:

Can Kimi K3 inspect a real, existing codebase, understand relationships across files, and produce a remediation plan that is safe enough to use?

I tested K3 Max in Kimi web Chat against a sanitized evidence pack from XBSTACK, a production Astro website with bilingual content, legacy routes, Canonical rules, Content Collections, and build-time validation.

The result was mixed in a useful way. K3 understood the cross-file architecture better than a shallow code assistant. It identified the correct Collection owner, connected route generation with Canonical behavior, and noticed that an old URL could not be treated as an isolated file problem.

But its first final recommendation would have hidden a valid warning instead of fixing the underlying state.

That difference matters more than any benchmark score.

What I gave K3

I did not upload the entire private repository. I prepared a sanitized pack containing only the files required to investigate one real problem:

the Astro content configuration;
the affected Markdown frontmatter;
the route implementation;
the bilingual content mapping;
the validation output that triggered the investigation.

I used Kimi web Chat with K3 and Max reasoning. This means the model could inspect the uploaded files and reason about them, but it could not edit the local repository, run npm, execute the Astro build, or inspect files I did not provide.

That boundary is important. This was a code-review and remediation-planning test, not an autonomous coding-agent test.

What K3 got right

The strongest part of the response was architectural reconstruction.

K3 correctly inferred that the problem crossed several layers:

content frontmatter
→ Astro Content Collection ownership
→ generated route
→ Canonical URL
→ bilingual mapping
→ legacy URL behavior

It did not reduce the warning to a typo in one Markdown file. It also recognized that changing the public route could affect existing links, search indexing, and the English counterpart.

That is where a large-context model can be genuinely useful. The value is not simply reading more files. The value is preserving enough relationships to avoid treating each file as an independent snippet.

Where the first recommendation failed

The first plan contained a dangerous shortcut: suppress or hide the warning so the validation output would become clean.

The warning, however, represented a real mismatch. Removing the signal would not repair the route, Canonical, redirect, or translation relationship. It would only make the system quieter.

I added more repository evidence and asked K3 to reassess the plan. In the second round, it withdrew the suppression approach and moved toward a safer remediation:

preserve the valid public URL;
repair the content or mapping source that produced the mismatch;
verify the Chinese and English route pair;
run the local build and release checks;
inspect the final diff before publishing.

The self-correction was useful, but the failure still establishes the operating boundary:

K3 is strong as an investigator and second opinion. Its first final plan is not a production approval.

Why a 1M context window does not remove the need for scope control

Kimi Code documentation now makes the membership boundary explicit:

K3 is available to Moderato plans and above;
Moderato supports up to 256K context;
Allegretto and higher unlock up to 1M context.

A larger context window can reduce manual chunking, but it can also encourage a bad workflow: loading the entire repository without defining the question, evidence set, or success criteria.

For this test, a selected evidence pack was better than a repository dump because it made the reasoning auditable. I could see which relationships K3 derived from the provided files and which conclusions still depended on missing context.

July 21 update: subscription pause and Kimi Code 0.28

The operational picture changed after the original test.

Kimi said it temporarily paused new consumer subscriptions after K3 demand exceeded forecasts and pushed current compute clusters close to capacity. Existing paid users were prioritized while additional capacity was prepared. Reuters independently reported the pause on July 20.

Kimi Code 0.26 expanded the coder sub-agent with background tasks, todo lists, plan mode, skills, and nested agents. Version 0.27 added /copy, automatic model-list refresh for API-key users, better network error reporting, and a security fix preventing the built-in URL fetcher from reaching loopback or internal network targets through crafted domains and redirects. Version 0.28 standardized the foreground web mode as kimi web, deprecated kimi server, fixed Conservative reasoning-effort persistence for K2.5/K3 sessions and the status line, and corrected inconsistent YOLO/Auto mode descriptions.

Another practical detail can look like a sudden quota or pricing problem: switching models or reasoning effort invalidates the existing prompt cache. In a long session, the old context may need to be prefilled again. Starting with /new before selecting K3 is the safer default when previous context is unnecessary.

The workflow I would use in production

For a real repository, I would separate investigation from execution:

1. Define one concrete engineering question.
2. Provide the smallest complete evidence set.
3. Ask K3 to map files, routes, state, and risks.
4. Require it to list assumptions and missing evidence.
5. Reject any plan that merely suppresses a warning.
6. Apply changes locally through a controlled coding agent or by hand.
7. Run typecheck, tests, build, route checks, and release gates.
8. Review the final diff before commit or deployment.

K3 can shorten steps 2 through 4. It does not replace steps 6 through 8.

Final decision

Kimi K3 is worth testing for:

cross-file investigation;
architecture review;
long code-review sessions;
identifying dependencies before a change;
generating a second remediation plan after new evidence arrives.

I would not treat it as autonomous proof that a repository is fixed. The model did not run the build in this test, and its first recommendation optimized the warning rather than the system.

The useful conclusion is not that K3 is universally better. It is that K3 can hold a larger engineering picture in one investigation, while production safety still depends on evidence selection, local verification, idempotent execution, and human approval.

Read the complete evidence-backed review, including the two-round correction and Astro-specific details:

https://www.xbstack.com/en/ai/tools-lab/kimi-k3-real-astro-project-test/?utm_source=devto&utm_medium=community&utm_campaign=kimi_k3_july20_update&utm_content=article_body

Source and disclosure

This article is adapted from my original XBSTACK test.
The test used Kimi web Chat, not a locally authorized Kimi Code agent.
No private credentials, private routes, or unpublished repository data are included.
Kimi Code releases and model limits: https://www.kimi.com/code/docs/en/release-notes.html
Reuters report on the subscription pause: https://www.reuters.com/legal/transactional/chinas-moonshot-pauses-kimi-subscriptions-amid-hot-demand-ipo-push-2026-07-20/
Canonical source: https://www.xbstack.com/en/ai/tools-lab/kimi-k3-real-astro-project-test/

GPT-5.6 Real-World Test: What Changed in an Astro, Content, and Analytics Workflow

xn — Thu, 23 Jul 2026 03:40:54 +0000

GPT-5.6 Real-World Test: What Changed in an Astro, Content, and Analytics Workflow

Search performance review, updated July 22, 2026: Search Console currently has data through July 19. This page recorded 47 impressions and an average position of 12.6 in the current 28-day opportunity window. Only 26 of 28 metric dates have returned, so this update uses impressions and position to confirm that the page is close to the first two result pages; it does not calculate a new CTR from an incomplete daily window.

The original workflow snapshot was captured on July 10, 2026. The 218-item content inventory, the site-wide 16 clicks from 794 impressions, and the historical 404 summary describe that earlier test. They are not the site's July 22 totals. Historical evidence, current opportunity data, and incomplete metric dates remain separate throughout this article.

Five questions this test had to answer

The workflow was designed around five practical questions:

Can GPT-5.6 understand and safely modify an existing Astro and React content project?
Does it improve technical content production, or merely produce a more complete report?
Can it analyse aggregate GSC and GA4 data without inventing page-level attribution?
How should Sol, Terra, Luna, max, and ultra be routed in everyday production?
Which actions can a coding agent perform automatically, and which still require explicit approval?

Who is this article for?

Independent developers and engineering teams evaluating AI tools in real repositories.
Site owners and growth engineers using AI for content operations, SEO, Search Console, and GA4 analysis.
Developers building coding agents who need explicit permission, evidence, and production-safety boundaries.

The conclusion: it closes more of the loop, but it still needs a human stop condition

The most useful change I observed was continuity across a long, mixed workflow. GPT-5.6 had to read an Astro content system, respect existing routes and frontmatter, compare the proposed topic against a published inventory, recalculate Search Console and GA4 metrics, write the article, and run local verification. The value was not a better isolated answer. It was the ability to keep earlier constraints active while moving from research to files, data, and validation.

That does not make it an autonomous project owner. The task was bounded: no navigation changes, no production deployment, no credential access, no destructive file operations, and no database migration. The model could plan and execute inside the allowed scope, but the author still decided which conclusions were supported, which sections should be deleted, and whether the final change was safe to publish.

The central tension is simple: a model that is more willing to finish the job also needs clearer permission boundaries. In content work, over-completion produced a 25-heading report instead of a readable technical essay. In analytics, correct arithmetic could still become an unsupported causal story if the input dimensions were too coarse. In code, a valid change would still be unacceptable if it exceeded the authorized file or deployment scope.

My 87/100 score applies only to this XBSTACK workflow. It is not a general model ranking, a latency benchmark, or a claim that GPT-5.6 beats every alternative on every task.

Test design: three real tasks and explicit boundaries

I did not use puzzle questions or a synthetic demo repository. I placed GPT-5.6 inside XBSTACK's existing Astro content workflow and asked it to complete three connected tasks. The acceptance criteria were project artifacts, recalculated data, and local verification—not how persuasive the chat response sounded.

I kept three deep tasks:

Task	Inputs	Success criteria	Evidence
Real project modification	Astro, React, Node SSR, content collections, dynamic routes, and build scripts	Find the existing content path, preserve the information architecture, and pass local verification	File diff, route, build log, generated HTML
Content review	A 218-item inventory, existing articles, writing rules, and SEO/GEO requirements	Avoid a duplicate search intent, separate official facts from observations, and produce valid metadata	Draft history, metadata, internal links, manual edits
Data analysis	GSC daily exports and a GA4 404 page-title summary	Recalculate the metrics, state the supported conclusion, and stop where the dimensions are insufficient	Formula, date range, metric definition, unsupported claims removed

This was a production-shaped test, not a controlled model laboratory. It involved file reads, official-document verification, local edits, several rounds of revision, and build checks. GPT-5.6 could inspect the repository and edit the specified article, but it was not authorized to deploy, delete production resources, migrate credentials, alter navigation, or write to a production database.

I also did not run a rigorous GPT-5.5 A/B test. There were no repeated trials under the same API account, prompt, cache state, network path, and tool permissions, and I did not capture a complete latency and token ledger. This article therefore makes no personal claim about a speed multiplier or a universal win rate. Official benchmarks remain external evidence, not my test result.

The analytics inputs had limits as well. The original GSC export was an aggregate snapshot, and the GA4 material contained 404 page titles and views rather than the original failing URLs. The model could verify the arithmetic and identify concentration, but it could not support page-level attribution or a causal explanation without the missing dimensions.

The updates that actually changed how I work

The changes that matter here are not a larger parameter count. They are the Sol, Terra, and Luna tiers, Programmatic Tool Calling, prompt caching, and the max and ultra runtimes. Together, they make model routing, tool-result preprocessing, and failure-cost management part of the application architecture instead of an afterthought.

How to choose Sol, Terra, and Luna?

OpenAI's public positioning gives each model a different capability, speed, and price envelope. This review did not run a controlled latency benchmark across accounts, regions, prompts, cache states, and repeated trials, so the earlier draft's precise latency ranges, "strong/medium/weak" tool ratings, and fixed usage percentages have been removed.

Model	Official role	Input price	Output price	My workflow decision
Sol	Highest-capability tier for complex work	$5 / 1M tokens	$30 / 1M tokens	Reserve it for cross-file changes, deep research, and high failure-cost tasks
Terra	Balanced quality and cost for everyday work	$2.50 / 1M tokens	$15 / 1M tokens	The default candidate for routine development, editing, and data explanation
Luna	Faster, lower-cost tier for lighter work	$1 / 1M tokens	$6 / 1M tokens	Use it for classification, extraction, routing, and low-risk batch processing

The final column is an XBSTACK routing decision, not an OpenAI recommendation for every application. A production router still needs its own success rate, human rework time, cache behavior, and failure-cost data.

max and ultra are not the fourth or fifth models. max allows individual models to spend more time on inference, explore solutions, run checks, and revise results; ultra is a multi-agent parallel model. The official default is that four agents handle different workflows separately, which are then aggregated by the main agent. Simply put, Max is one person thinking several rounds of work, Ultra is about splitting tasks into several people to complete simultaneously.

If Ultra is used for this work, it can be broken down into four branch lines: official document verification, project structure check, content inventory analysis, and GSC/GA4 recalculation. But if you change a title, extract a few fields, or format data, there's no need to enable four agents. Multi-agent entities increase the upper limit and completion speed of complex tasks, while also consuming more tokens; When tasks can't be disassembled, they only create repetitive work.

Another change that deserves more attention from developers is Programmatic Tool Calling. Traditional tool calls often push a large number of intermediate results back into the context before the model judges the next step. GPT-5.6 can write and run lightweight programs within the Responses API, filtering, grouping, and aggregating tool results first, retaining only the truly important content.

For example, when handling 10,000 logs, a more reasonable process is not to send each log to the model individually, but to first group them by status code, count repeat paths, extract abnormal samples, and then let the model determine the cause and priority. The program is responsible for stable computation, while the model handles uncertainty judgment. This is more practical for website logs, financial reports, MCP, multi-tool agents, and n8n workflows than "the model can write a script."

Real project test: did it read the rules before editing files?

In this bounded task, GPT-5.6 read the content collection, Tools Lab routing, and frontmatter conventions before it edited the article. It used the existing directory and route, did not invent a new collection, and did not change navigation. Those inspectable actions are stronger evidence than a generic claim that it "understood the project."

The instruction was not simply "write a GPT-5.6 article." It first had to determine whether the topic duplicated an existing page, identify the correct collection and route, preserve the frontmatter contract, and only then edit the file.

The resulting content placement was:

content file: src/content/ai/gpt56-test.md
section: tools-lab
hub: tools-lab
subcategory: tools-lab
series: ai-tools-lab
page route: /ai/tools-lab/gpt56-test/

The path itself is simple. The important part is what happened next: the task continued through the content ledger, Astro build, target route, sitemap, Pagefind, canonical URL, JSON-LD, and generated HTML. In that historical build snapshot, the page pre-rendered successfully, Pagefind indexed 820 static HTML pages, the structured data parsed, and machine-oriented metadata did not leak into the visible article. Those checks belong to the July 10 test snapshot; they are not the site's current global page count.

in XBSTACK Astro projects

Continuity was the strongest part of the project task. Repository rules read at the beginning still constrained the later file edit and verification steps. An abstract requirement such as "GEO must not pollute the visible article" was translated into concrete checks around hideStructuredBlocks, frontmatter, JSON-LD, the layout, and the generated page head.

This does not prove that GPT-5.6 can independently maintain an arbitrary production system. The change was narrow and reversible. I did not test a database migration, a large refactor, an automated deployment, or a live rollback. The defensible conclusion is narrower: when a repository already has clear rules, a bounded file scope, and testable outputs, GPT-5.6 can absorb a substantial amount of reading, locating, editing, and verification work.

Content test: strong evidence collection, weak editorial hierarchy

The content task exposed a different failure mode: over-structuring. GPT-5.6 was good at retrieving facts, checking metadata, and making sure every requirement appeared somewhere. That same behavior produced a draft with 25 H2 headings. It was complete in a checklist sense and weak as a continuous technical essay.

The historical inventory contained 218 published entries, including 134 files in the AI collection and 96 entries with category: ai. Those are different dimensions: one describes storage, the other a business category. The model kept them separate and correctly placed the article in AI Tools Lab rather than Agent, LangGraph, or general Notes.

The useful work included duplicate-intent checking, metadata alignment, canonical and internal-link planning, and separating official specifications from observations. The weak point was editorial judgment. Test scope, data definitions, inventory, GSC, GA4, GEO, build evidence, benchmarks, pricing, security, and audience were each turned into separate sections. Most facts survived review, but the reading experience resembled a delivery report.

I had to merge sections, remove repeated conclusions, delete unsupported precision, and rebuild the article around one tension: GPT-5.6 can close more of a complex workflow, but the stronger the execution capability, the more important human acceptance and permission boundaries become.

That is the division of labor I would keep. The model handles evidence collection, recalculation, inventory checks, a first structure, metadata, and repetitive validation. The author decides which evidence matters, what to remove, what order serves the reader, and whether the final article still reflects an accountable point of view.

Data test: correct arithmetic is not the same as correct attribution

The arithmetic was easy. The real test was whether the model would stop before turning an aggregate ratio into a causal explanation. It correctly recalculated the historical GSC CTR and the distribution of views across three 404 page titles. It did not have the original failing URLs, referrers, unique users, or redirect chain, so those inputs could not support a page-level root cause.

The first calculation was:

CTR = 16 ÷ 794 × 100%
    = 2.0151%

Rounded to two decimals, the historical site-wide CTR was 2.02%. The supported conclusion was limited: Google was already showing the site, but much of that visibility occurred at weak average positions and produced few clicks. That does not prove that the titles alone were the problem.

However, it does not directly attribute the problem to "the title not being attractive enough." A weighted average ranking of 34.19 means that many impressions occur in lower positions in the search results, which in itself lowers the CTR. A more prudent course of action should be to first break down the data by page and query, identify pages with an average ranking of 8–30 that already have a certain number of impressions, and then check the title, meta description, and search intent. Pages ranked beyond the thirtieth position with few impressions should not have their titles frequently changed solely because of low CTR.

The second data set came from the GA4 Page titles and screens report for April 11 through July 9, 2026. Three explicit 404 titles recorded 523, 57, and 25 views, for a total of 605. The currently used "No Man's Land | 404 - XBSTACK" title accounted for 523 of those views.

523 ÷ 605 × 100% = 86.4463%

Rounded to one decimal, the share was 86.4%. It only means that most views across these three named 404 titles were attached to the current title. It does not mean that 86.4% of broken URLs came from one page type, one route migration, or /tags/. The export did not contain those dimensions.

What really tests the model here is not division, but whether it can stop at the data boundary in time. The current exports only have page titles and view counts, without triggering the original page path of 404, so it is impossible to further determine whether these visits come from tabs, historical article addresses, case values, URL encoding, or external old links. 605 is just the total number of views for three categories of 404 titles, not 605 unique visitors or 605 unique invalid URLs.

Combining the content inventory, GSC, and GA4 produced a clearer action order. First export the original 404 paths and sources, then fix the concentrated broken destinations. In parallel, improve pages that already have impressions and average positions near the first two result pages. New AI content should pass a stricter duplicate-intent check, while weaker sections should be expanded only with real material. The model did not choose the site's strategy; it separated supported actions from questions that still required more data.

Official benchmarks are context, not project evidence

Official benchmark gains are useful background, but they are not the acceptance test for this article. I used local build results, recalculated metrics, preserved constraints, and the amount of manual correction as the evidence. A benchmark can explain why a capability may have improved; it cannot prove that a particular repository change, analytics conclusion, or publishing decision was correct.

Review	GPT-5.6 Sol	GPT-5.6 Sol Ultra	GPT-5.5
Terminal-Bench 2.1	88.8%	91.9%	85.6%
BrowseComp	90.4%	92.2%	84.4%
OSWorld 2.0	62.6%	Not listed	47.5%
SWE-Bench Pro	64.6%	Not listed	59.4%
DeepSWE v1.1	72.7%	Not listed	67.0%

The benchmark pattern is directionally consistent with the workflow I observed: the model was most useful when browsing, repository reading, tool use, and repeated verification had to remain connected. That is still an interpretation, not a private reproduction of the benchmark.

The official table also shows that GPT-5.6 is not first on every evaluation, and each benchmark has its own environment and success criteria. I did not run a controlled GPT-5.5 comparison, so the supported claim is limited to a better experience with long-task continuity—not a measured percentage gain in overall ability.

Benchmarks explain why a capability may be worth testing. The project decides whether it is useful. In this review, a passing build, a correctly bounded analytics conclusion, and the amount of manual restructuring mattered more than a small difference on an unrelated leaderboard.

Pricing and model routing: daily work does not need Sol everywhere

The published prices make a single-model strategy difficult to justify. A router should consider not only token price, but also task failure cost, expected human rework, input stability, and whether cached context can be reused. The cheapest request is not cheaper if it regularly produces a result that must be redone.

Model	Input / 1M tokens	Output / 1M tokens
GPT-5.6 Sol	$5	$30
GPT-5.6 Terra	$2.50	$15
GPT-5.6 Luna	$1	$6

GPT-5.6 also supports explicit prompt cache breakpoints and a cache lifecycle of at least 30 minutes. Cache writes are charged at 1.25 times the input price of uncached inputs, while cache reads continue to enjoy a 90% discount on input prices. For workflows containing a large number of stable project rules, duplicate documents, or fixed tool descriptions, caching strategies directly affect costs.

This test took place in the ChatGPT/Codex workflow, and I didn't have a complete API token bill, so I couldn't give "the exact amount cost to complete this article." If there is no real billing record, only the API unit price is listed, without falsifying the cost per task.

My actual route would be:

Tagging, classification, formatting, simple extraction  → Luna
Routine editing, summaries, standard data explanation   → Terra
Cross-file changes, deep research, high failure cost    → Sol
One difficult task that needs deeper checking            → Sol + max
A task that cleanly splits into independent workstreams  → ultra / multi-agent

Terra is more suitable as the default setting for daily use. Many content organization tasks, general development, and routine Agent tasks do not require paying Sol's output price. Sol should be reserved for work with high failure costs, cross-file or cross-tool tasks, and results that need thorough review. Ultra is also not a free quality button; when tasks cannot be naturally divided, multiple Agents will repeatedly read materials, perform reasoning, and consume tokens.

The failures and limits that were actually observed

I re-audited the earlier version of this section and removed three examples that did not have preserved logs or a direct official source: an Astro API error, a fabricated-looking prompt-cache threshold error, and a credential-search shell error. A review cannot promote a plausible incident into an observed one merely because it sounds technically possible.

Failure 1: a complete draft that read like a project report

The first draft tried to satisfy every requirement as a separate section. A roughly 6,500-Chinese-character article ended up with 25 H2 headings. The problem was not missing information. It was the absence of editorial hierarchy. The model optimized for checklist coverage; the author needed a single argument, a deliberate pace, and evidence placed where it changed the decision.

Failure 2: correct arithmetic could still become an unsupported attribution

The historical GSC CTR and the 404 title-view share were recalculated correctly. The input still lacked the dimensions required to explain the root cause. The model could say that 523 of 605 views belonged to one 404 title. It could not say those views came from one route, one migration, one platform, or one class of broken URL.

The useful behavior in an analytics task is often a stop condition: "this calculation is supported; the attribution is not." A model that always returns a complete explanation can be less useful than one that explicitly identifies the next data export required.

Failure 3: no controlled A/B means no personal speed claim

I did not run GPT-5.6 and GPT-5.5 repeatedly under the same API, account, prompt, cache state, network path, and tool permissions. I also did not preserve a full token and latency ledger. The article can describe a better experience with long-task continuity. It cannot claim a measured speedup, win rate, or cost reduction.

The previous version included precise first-token latency ranges and fixed model-usage percentages without test logs. Those figures have been removed. Deleting unsupported precision improves the review more than adding another benchmark table.

Limit 4: the task ran inside explicit permissions

GPT-5.6 was not allowed to deploy, push, delete files, change navigation, migrate credentials, or write to production data. It completed a bounded local task. That result does not prove that an unsupervised coding agent should control an arbitrary production environment.

OpenAI's GPT-5.6 System Card also discusses a somewhat greater tendency than GPT-5.5 to exceed the user's original intent in a small number of agentic coding cases, while noting that the absolute rates remain low. That finding supports permission design; it does not justify claiming that the model will inevitably overreach.

Greatest Risk: The more it wants to get things done, the more it needs to limit what it can do

The risk is not limited to incorrect code. A coding agent can produce technically valid code and still perform an unauthorized action. That changes the safety question from "is the patch correct?" to "was this action allowed, reversible, and independently verified?"

Permissions should follow action risk, not a single on/off switch

I use four permission levels. Read-only inspection, repository search, public-document research, and plan generation are allowed by default. Scoped local edits and test commands are allowed only in named directories with a visible diff. Deletion, overwrite, dependency changes, database migrations, push, deployment, external messages, and public publishing require separate approval. Credential discovery, cross-environment token copying, bypassing safety checks, and altering audit records are prohibited.

The same principle applies to sub-agents: parallel workers must not inherit more authority than the parent task, and their operations must return to the same audit trail. Multi-agent execution can reduce elapsed time, but it also increases the number of actions that need to be observable.

Data analysis needs explicit stop conditions

I now define both the allowed conclusion and the stopping point before asking for an analytics explanation. Aggregate page metrics can support prioritisation; they cannot identify a page-level cause. A 404 title report can show concentration; it cannot identify the original broken URL. A high Direct share can justify better campaign tagging; it cannot prove which untagged platform caused the visits. When a required dimension is missing, the output should name the next export instead of completing the story with a plausible guess.

Evidence has three different owners

I now separate every major statement into three categories. Official facts—model roles, prices, benchmarks, product features, and safety findings—must point to OpenAI material. Project evidence—file paths, diffs, build output, route checks, recalculated metrics, and generated HTML—must come from the local workflow. Routing choices and editorial decisions are author judgments, not universal product claims.

This separation prevents a common review failure: writing an official benchmark, one successful project run, and a personal preference in the same authoritative voice. A reader should be able to tell whether a claim came from the vendor, this test, or my operating context.

A file write is not the end of a project test

For this Astro task, the verification chain was longer than "the markdown file exists." The content path had to match the collection, the frontmatter had to parse, the route had to generate, the build had to pass, the canonical URL and structured data had to be correct, the sitemap and Pagefind output had to include the page, and machine-oriented metadata had to remain outside the visible article.

Any broken link in that chain would change the status from complete to partial. This is why a polished answer or a successful file edit is not enough evidence for a coding-agent review. The acceptance criteria must be specified before the model starts, and each completed check needs an artifact.

What the human editor changed

The final article is not the model's untouched output. I merged the 25-heading structure, removed repeated conclusions, separated historical and current data, changed the model name to the official GPT-5.6 form, corrected the 86.4% interpretation, and deleted unsupported latency ranges, tool-capability ratings, cache errors, and credential errors.

The model handled a large amount of retrieval, organization, recalculation, file work, and validation. The human work was evidence review, deletion, sequencing, and deciding how far each conclusion could go. Hiding those edits would overstate the model's contribution and make the test impossible to learn from.

Reproducing the method

The exact result will change with the repository, model service, and data window, but the method is reproducible. Use a real repository, define the allowed files and prohibited actions, provide a mixed task with a measurable output, preserve the prompt and plan, record the diff and test commands, and include at least one task where the correct behavior is to stop for missing evidence or permission.

For analytics, preserve the date range, dimensions, metric definitions, and formulas. For content, preserve the first draft and the human-edited version. The goal is not to reproduce the same score. It is to reproduce the evidence format: what went in, what changed, what failed, how the result was verified, and which conclusions remain out of scope.

Why I updated this page instead of publishing a second GPT-5.6 review

For the 28-day period ending July 17, the page had 46 impressions, 3 clicks, and an average position of 12.8. The sample is too small for a causal SEO claim, but it shows that Google already associates this URL with the relevant intent. Publishing another page with the same core query would split signals and create a duplicate-intent problem.

The update therefore keeps the route and canonical URL, improves the title and description, adds evidence and boundaries, and removes weak claims. I will review indexing after seven days, query growth after fourteen days, and page-level impressions, clicks, CTR, and average position after twenty-eight days. Short-term volatility will be recorded, not promoted as proof that the rewrite succeeded.

Release acceptance needs its own stop conditions

The article does not pass merely because the prose is finished. Publication stops if a core number cannot be recalculated, a screenshot exposes a local path or credential, the English page is only an unreviewed literal translation, the canonical or schema fails, or the build and article gates do not pass. An untested production action remains unverified, and an undeployed change cannot be described as live. These conditions prevent a timely review from becoming a less trustworthy one.

2026-07-19 Update: Compare It with the Kimi K3 Real-Project Test

I later tested Kimi K3 on the same type of Astro content-architecture problem. Kimi K3 correctly identified Collection ownership, dynamic routing, and legacy URL risks in its first pass, but its final recommendation initially tried to hide a valid warning through an exception; it withdrew that recommendation only after receiving additional evidence. The result reinforces the conclusion of this article: the model's greatest value is reducing investigation and review time, but routing, canonical URLs, redirects, and production changes still require human approval. See the full process in Kimi K3 Real-Project Test: Strong Cross-File Analysis, but Final Decisions Still Need Human Review.

Final decision: use more of the workflow, not more authority

GPT-5.6 earned a larger role in my research, repository reading, content inventory, data recalculation, drafting, and local verification workflow. The reason is continuity: it connected work that would otherwise be split across browsing, code inspection, spreadsheets, metadata, and build commands without discarding most of the earlier constraints.

It did not replace editorial judgment, data acceptance, business priority, or permission decisions. The 25-heading draft showed that complete coverage can still produce poor reading. The analytics task showed that correct math can still support an incorrect story. The permission analysis showed that a technically valid action can still be unauthorized.

I would route low-risk batch work to Luna, most daily production work to Terra, and genuinely complex or expensive-to-fail tasks to Sol. Max is for one difficult task that benefits from deeper checking. Ultra is useful only when the work can be split into independent branches with clear aggregation criteria.

Production writes, deletion, deployment, credentials, database changes, cloud resources, and public publishing remain human-approved. The practical upgrade is not that GPT-5.6 should receive every task and every permission. It is that more of the workflow can be delegated while the stop conditions become explicit.

FAQ

Is GPT-5.6 good for content creation?

Yes, especially for evidence collection, duplicate-intent checks, structural drafts, metadata, internal links, and revision passes. Its tendency toward complete coverage can produce too many sections and a report-like rhythm, so the author still needs to control evidence selection, sequence, voice, and factual boundaries.

Can GPT-5.6 Analyze Search Console and GA4 Data?

It can recalculate metrics, identify concentration, compare priorities, and turn the available dimensions into an action list. In this test it verified 16/794 = 2.02% and 523/605 = 86.4%. Without page, query, referrer, or original broken-URL data, it could not support page-level attribution or a causal claim.

How to Choose Between GPT-5.6 Sol, Terra, and Luna?

Use Sol for cross-file development, deep research, long tasks, and work with a high failure cost. Use Terra as the everyday default for development, editing, and routine agent work. Use Luna for classification, extraction, routing, format conversion, and high-volume low-risk tasks.

Is GPT-5.6 Ultra a More Powerful Standalone Model?

No. Ultra is a multi-agent parallel mode, with four Agents by default dividing tasks and then summarizing them. It can increase the upper limit and completion speed of complex tasks, but consumes more tokens; simple tasks are usually not worth activating.

How Much Faster Is GPT-5.6 Than GPT-5.5?

This article did not conduct rigorous personal A/B testing, so it cannot provide speed multipliers. The token, latency, and success rate improvements announced by the official and partners can only be used as external references and cannot be directly equated with the actual performance of all projects.

Is GPT-5.6 suitable for directly taking over production projects?

No. Start read-only, show the plan before execution, restrict directories and tools, approve risky actions separately, inspect the diff, run verification, and keep a rollback point. Deletion, dependency changes, deployment, credentials, database changes, and production data writes require explicit approval.

Will the GEO information in this article affect the reading of ordinary users?

Machine query terms and structured summaries will not be additionally displayed. XBSTACK uses hideStructuredBlocks: true to disable automatically visible queries, summaries, and audience modules; TechArticle, FAQPage, Dataset, BreadcrumbList, keywords, and references are written into the page header via JSON-LD and Meta, but the main text still only has one set.

Official Information

Keep reading

Originally published on XBSTACK.

DEV Community: xn

Google ADK state_delta Not Applied on Resume: Runner.run_async Reproduction and Workaround

Short answer

Why this is easy to misdiagnose

What the ADK 2.6.2 source path shows

Do not use a fake new_message as the default fix

Temporary workaround I verified locally

Where this bug matters most

Is every Google ADK version affected?

Minimal troubleshooting checklist

FAQ

Why does run_async() succeed while the state remains unchanged?

Does sending a new_message fix it?

Is SessionService.append_event() the official fix?

Is the bug model-specific?

Final recommendation

References

Reproduction assets

OpenAI Responses API Stream Abort: Fix No tool call found

OpenAI Responses API: Why Stream Abort Causes No tool call found for function call output

1. What the 400 Error Actually Means

2. Official Status as of August 5, 2026

3. Stream Visibility Is Not a Commit Token

4. The Local Lab and Its Evidence Boundary

5. Four Scenarios, One Critical Invariant

6. Fast Triage for Production Incidents

Step 1: Confirm the Conversation and call_id belong to the same turn

Step 2: Read Conversation Items

Step 3: Identify how the stream ended

Step 4: Check whether the tool already produced a side effect

Step 5: Choose the recovery path

7. A Production Tool-Call State Machine

What the state record should store

8. A Safer Python Control Skeleton

9. Recovering Browser Disconnects and Proxy Timeouts

10. Regression Tests That Are Worth Keeping

11. Common Mistakes and Final Decision

Frequently Asked Questions

Why can the client have a call_id while Conversation Items are empty?

Is waiting for response.completed enough?

What if the tool executed but the call_id is missing?

Is this the same as the OpenAI Agents SDK RunState persistence issue?

Does the local lab prove the OpenAI server implementation?

OpenAI Agents SDK Duplicate Tool Names: Why the Later Tool Wins

OpenAI Agents SDK Duplicate Tool Names: Why the Later Tool Wins

The smallest collision

Offline test environment

Layer one: the SDK validator does not reject the conflict

Layer two: both duplicate tools remain visible to the model

Layer three: local dispatch becomes last-wins

Why silent last-wins is more dangerous than a 400

Temporary fix: validate uniqueness before model dispatch

Fix the public names

Use namespaces for larger registries

Where the gate should run

Agent factory unit tests

Plugin registration

Multi-tenant configuration

Clone and dynamic append paths

Release gates

Name uniqueness is only the first registry check

Three approaches to avoid

Waiting for the provider 400

Treating list order as configuration

Checking only __name__

Regression test after an upstream fix

Relationship to the RunState article

Conclusion

References

Why LangGraph State Disappears After Run Cancellation: Streaming, Checkpoints and Recovery

Why LangGraph State Disappears After Run Cancellation: Streaming, Checkpoints and Recovery Consistency

Test design, environment, and explicit boundaries

The official persistence model: checkpoints are super-step state

Experiment 1: the UI saw three chunks and the checkpoint stored none

Experiment 2: completed graph steps restored all three visible chunks

What sync, async, and exit actually change

interrupt() is a recoverable pause, not ordinary cancellation

Production repair: separate visible progress, recoverable state, and business truth

Do not turn every token into a graph node

Regression tests that prove the repair

Do not use a fake `new_message` as the default fix

Why does `run_async()` succeed while the state remains unchanged?

Does sending a `new_message` fix it?

Is `SessionService.append_event()` the official fix?

Checking only `name`

What `sync`, `async`, and `exit` actually change

`interrupt()` is a recoverable pause, not ordinary cancellation

Does `durability="sync"` prevent state loss on cancellation?

Does `interrupt()` resume at the exact Python line?

Why did `exit` preserve three chunks in the stepwise cancellation test?

Breakage 2: `response.messages` is not a stable database contract

`chunkMs` and Cloudflare 524 are different timeout layers