Tool Errors Are Not Retries: LangChain vs OpenAI Agents SDK vs Microsoft Agent Framework

A failed tool call is not automatically a retryable failure.

A timeout may be safe to replay. A validation error usually needs corrected arguments. A permission denial should often stop the run. Treating all three as “try again” is how agents become slow, expensive, and occasionally unsafe.

This article compares the documented error-handling primitives in LangChain, the OpenAI Agents SDK, and Microsoft Agent Framework. The goal is not to crown a universal winner. It is to make the recovery policy explicit before an agent reaches production.

The evaluation criteria

I compared each framework on five questions:

1. Where is the failure intercepted? Tool middleware, model/runtime retry, or outer agent middleware?
2. Can the system distinguish transient from permanent errors?
3. What does the model receive after failure? A sanitized tool message, an exception, or a fallback response?
4. How are retries bounded? Attempts, backoff, jitter, and policy hooks matter more than the word “retry.”
5. How much application code is needed? A powerful primitive still has an operational cost.

Test conditions and limits: this is a documentation comparison performed on July 20, 2026. I reviewed the current official documentation and examples linked in the Sources section. I did not run a common benchmark, measure latency, or compare model quality. “Best fit” below means best fit for the documented control surface, not a measured ranking.

Short answer

If your priority is...	Start with...	Why
Separate tool-error presentation from retry policy	LangChain	Dedicated ToolErrorMiddleware and ToolRetryMiddleware can be composed and ordered explicitly.
Policy-driven retries for model/API calls	OpenAI Agents SDK	Retry decisions can inspect normalized error facts, attempt count, stream state, and provider advice.
Centralized graceful degradation around an agent	Microsoft Agent Framework	Middleware can catch exceptions, replace results, and implement fallback or retry behavior around execution.
A portable production policy	Any of the three, with your own error taxonomy	The framework primitive is not a substitute for classifying failures by side effect and recoverability.

That table is a routing aid, not a leaderboard.

1. LangChain: explicit composition at the tool boundary

LangChain’s current middleware docs make an important distinction: ToolErrorMiddleware does not automatically retry failed calls. It turns handled failures into controlled messages that the model can use, while ToolRetryMiddleware handles retry attempts separately.

That separation is useful because a tool can fail in two very different ways:

• Transient: a 502, timeout, or temporary rate limit may be worth retrying.
• Correctable: invalid arguments or a missing required field should be shown to the model so it can repair the call.
• Permanent or unsafe: an authorization failure, policy denial, or non-idempotent partial write should not be blindly replayed.

The middleware order also carries meaning. LangChain’s example places retry middleware inner, with on_failure="error", so exhausted exceptions reach the outer tool-error handler. In plain English: retry first, then convert the final failure into a model-visible result.

from langchain.agents import create_agent
from langchain.agents.middleware import (
    ToolErrorMiddleware,
    ToolRetryMiddleware,
)


def present_tool_error(exc, request):
    if isinstance(exc, ValueError):
        return "The tool arguments were invalid. Correct the arguments and try once more."
    return "The tool failed after bounded retries. Do not repeat it without changing the plan."

agent = create_agent(
    model="your-model",
    tools=[search_service],
    middleware=[
        ToolRetryMiddleware(
            max_retries=2,
            on_failure="error",
        ),
        ToolErrorMiddleware(
            on_error=present_tool_error,
            tools=["search_service"],
        ),
    ],
)

The practical strength here is locality. The tool failure is handled where it occurs, and the application can choose which tools receive the policy. The tradeoff is that you must understand middleware ordering and decide which exception classes are retryable. The framework will not infer your tool’s side effects for you.

2. OpenAI Agents SDK: policy-driven retries for model calls

The OpenAI Agents SDK exposes a different center of gravity. Its retry reference describes runner-managed retries for model calls, with configurable maximum retries and backoff. Retry policy callbacks receive context such as the exception, attempt number, stream state, normalized error facts, and provider advice.

That gives an application a useful basis for decisions such as:

• retry network errors and timeouts;
• honor a provider’s Retry-After guidance;
• retry selected HTTP statuses;
• avoid retrying aborts or unsafe replays;
• treat an error after streaming has started more cautiously than a request that never produced a response.

This is a strong fit when the failure you need to control is the model/API request itself. It is not the same thing as automatically making a failed business operation safe to repeat. If a tool already created a resource before the connection dropped, a second tool call may duplicate the resource even if the model request is retryable.

A good design is to keep the SDK’s model retry policy narrow, then put idempotency keys and tool-specific compensation in the tool layer:

# Sketch, not a drop-in policy for every provider.
def retry_model_call(ctx):
    err = ctx.normalized
    if err.is_abort or (err.is_network_error is False and err.status_code not in {429, 500, 502, 503, 504}):
        return False
    if ctx.attempt >= 2:
        return False
    return True

The snippet is intentionally a sketch because the exact policy type and provider adapter should follow the SDK version you pin. The documented concept is the important part: retry eligibility is a policy decision based on normalized facts, not a blanket exception handler.

3. Microsoft Agent Framework: middleware for graceful degradation

Microsoft Agent Framework’s exception-handling guidance puts the control at the agent middleware layer. The examples catch exceptions around agent execution and can return a replacement response, log the failure, or add retry and fallback behavior.

That makes it a natural fit for teams that want a centralized safety net around several agents or function tools. For example, a middleware layer can:

1. attach a correlation ID;
2. catch a timeout from a downstream service;
3. return a user-safe explanation instead of raw exception details;
4. record the original exception for operators;
5. route to a fallback agent or degraded capability.

The tradeoff is scope. An outer middleware can see that an agent run failed, but it may not know whether the failed tool had already performed a side effect. If you need tool-specific retry rules, put those rules closer to the tool or function boundary and make the tool idempotent before adding automatic replay.

A useful pattern is to separate recovery for the user experience from replay of the operation:

async def safe_run(inner_agent, messages, context):
    try:
        return await inner_agent.run(messages, context=context)
    except TimeoutError:
        record_failure(context, kind="timeout")
        return "The service timed out. No further write was attempted; check status before retrying."
    except PermissionError:
        record_failure(context, kind="permission_denied")
        return "This action is not permitted for the current identity."

The first response can be graceful without claiming that replay is safe.

The policy that matters more than the framework

Before enabling retries, classify every tool operation along two axes:

Error class	Safe to replay?	Agent response
Timeout before a request is accepted	Usually, if the client can prove no side effect occurred	Retry with bounded exponential backoff.
429 or 5xx from an idempotent read	Usually	Honor Retry-After when present; cap attempts.
Invalid arguments	No automatic replay with identical arguments	Return a concise, sanitized correction message.
Permission or policy denial	No	Stop and surface the boundary.
Network loss after a write may have committed	Unknown	Check operation status or use an idempotency key before replay.
Tool returned malformed data	Not blindly	Validate, log, and either repair through a separate path or fail closed.

This is also where observability belongs. Record the tool name, operation class, attempt number, error class, duration, whether a side effect was possible, and the final disposition. Do not put secrets or full sensitive payloads into the model-visible error message.

I covered trace-driven regression testing in Stop Replaying Coding-Agent Bugs by Hand. The same idea applies here: save representative failures and assert not just that the run completes, but that the agent chooses the right recovery path.

What to do now

1. Inventory your tools and mark each one as read-only, idempotent write, non-idempotent write, or approval-gated.
2. Define a small error taxonomy: transient, correctable, denied, unknown-side-effect, and malformed-result.
3. Enable automatic retries only for the first category and selected idempotent reads.
4. Return sanitized, actionable tool errors to the model. Never expose stack traces or credentials.
5. Add a hard attempt cap and backoff jitter. Log every retry and every final failure.
6. Test a dropped connection after a write. If the system cannot determine whether the write committed, make status lookup or idempotency keys part of the tool contract.
7. Re-run those cases in CI using captured traces, as described in Turn Traces Into Regression Tests.

Candid limitations

Framework documentation shows the available control surfaces, not the reliability of your downstream services or the quality of a model’s recovery decision. Defaults and APIs can change, especially in fast-moving agent SDKs, so pin versions and re-check the linked references during upgrades. None of these frameworks can prove that an arbitrary external write is safe to replay. That requires an application-level contract.

Sources

• LangChain prebuilt middleware: tool errors and retries
• OpenAI Agents SDK retry reference
• OpenAI Agents SDK overview
• Microsoft Agent Framework exception handling

Discussion

Which failure has caused more pain in your agent system: a retryable outage, a bad tool argument, or a write whose commit status was unknown? What recovery contract did you end up adopting?