Why Your AI Agent Can't Execute Its Own Plan: Bridging the Gap Between Local LLM Intelligence and Real-World Software Reliability

Originally published on tamiz.pro.

You prompt your agent to orchestrate a multi-step workflow. It generates a beautifully reasoned plan. Then it fails on step three. Or eight. Or quietly produces wrong output that no one notices until it's too late.

This isn't a prompt engineering problem. It's an architecture problem — one rooted in the fundamental mismatch between probabilistic language models and deterministic software systems. Understanding why agents fail to execute their own plans is the prerequisite to building ones that don't.

This article dissects the technical failure modes, traces them through the agent stack, and explores architectural patterns that close the gap between LLM-grade reasoning and production-grade execution reliability.

The Core Mismatch: Probabilistic Reasoning vs. Deterministic Execution

Before diagnosing the failure modes, we need to be precise about what's actually happening when an agent "executes a plan."

A modern agentic system has two conceptual layers that are conflated in practice:

1. The reasoning layer — the LLM generates a plan, decomposes a task, reasons through constraints. This is probabilistic: the model samples tokens based on learned distributions.
2. The execution layer — code runs, APIs are called, data is written, side effects occur. This is deterministic: the same inputs must produce the same outputs, or the system is broken.

The LLM lives entirely in the reasoning layer. When we say an agent "executes its plan," what's really happening is that the LLM generates text that a software harness interprets as instructions. The reliability of the whole system is bounded by whichever of these two layers is weaker.

In practice, the execution layer is where things collapse. And they collapse in predictable, categorizable ways.

Failure Mode 1: The Plan Itself Is Non-Deterministic

An LLM-generated plan is a sequence of natural language instructions. When you say "the agent will execute this plan," you're implicitly assuming the plan is executable by something other than another LLM.

But here's the problem: the plan is opaque.

# What the LLM generates as a "plan":
plan = [
    {"step": 1, "action": "query user database", "params": "active users last 30 days"},
    {"step": 2, "action": "aggregate metrics", "params": "count by region"},
    {"step": 3, "action": "format report", "params": "PDF with charts"},
    {"step": 4, "action": "send to stakeholder", "params": "weekly digest list"}
]

Step 3 says "format report" with a param of "PDF with charts." That's not a program. That's a description of intent. The execution engine needs to:

• Know which charting library to use
• Decide which metrics map to which chart types
• Handle missing data gracefully
• Render to PDF
• Style the output

Every one of these decisions is a potential point of failure. The LLM that generated the plan doesn't actually understand any of this — it's predicting the next reasonable word in a sequence. The execution engine that does need to understand it is a separate system, often hand-written, and almost always incomplete.

The fix: Plans must be expressed in a formal intermediate representation (IR), not natural language. This means the LLM generates structured output that maps to concrete, typed actions:

// Formal plan representation
interface ExecutablePlan {
  steps: ExecutionStep[];
  constraints: PlanConstraints;
  validation: ValidationRules[];
}

interface ExecutionStep {
  id: string;
  action: ActionType; // Enum, not string
  inputs: TypedSchema; // JSON Schema validated
  dependencies: string[]; // DAG, not implicit ordering
  retryPolicy: RetryConfig;
  timeout: Duration;
}

enum ActionType {
  DATABASE_QUERY,
  API_CALL,
  FILE_WRITE,
  EMAIL_SEND,
  // ... exhaustively enumerated
}

The LLM fills in parameters, but the shape of execution is constrained by the type system. This eliminates the semantic gap between "what the plan says" and "what the code does."

Failure Mode 2: State Drift Between Planning and Execution

Even when the plan is well-formed, a silent killer is state drift — the world changes between when the plan is generated and when a step is executed.

Consider this sequence:

T=0s    : Agent plans → "fetch user X's data from API endpoint /v2/users"
T=5s    : API endpoint /v2/users is deprecated; /v3/users is now live
T=6s    : Agent executes step 1 → 404 error, plan halts

Or worse — a silently wrong result:

T=0s    : Agent plans → "calculate total revenue from transactions table"
T=5s    : A deployment changes the schema; a new `currency` column appears
T=6s    : Agent executes → queries sum(revenue) but gets mixed currencies
T=7s    : Agent reports $1.2M revenue to stakeholders → wrong by 3x

The LLM has no awareness of these state transitions. It generates a plan based on its training data and whatever context you provided. It doesn't know the schema changed yesterday. It doesn't know the API version you meant.

The fix: Agents need a state awareness layer that validates plan assumptions against current reality before execution:

class PlanValidator:
    def __init__(self, state_probe: StateProbe):
        self.probe = state_probe

    def validate_plan(self, plan: ExecutablePlan) -> ValidationReport:
        for step in plan.steps:
            # Check that referenced schemas still exist
            schema = self.probe.get_schema(step.inputs)
            if not schema.matches(step.params.schema):
                raise SchemaMismatchError(step.id, schema, step.params.schema)

            # Check that endpoints are reachable
            if step.action == ActionType.API_CALL:
                health = self.probe.check_endpoint(step.params.url)
                if health.status != "healthy":
                    self.flag_risk(step.id, f"Endpoint degraded: {health.status}")

        return ValidationReport(completed=True, warnings=self.warnings)

This turns plan execution from a blind leap into a verified execution. The agent still generates plans naturally, but they're validated against the live system state before any side effect occurs.

Failure Mode 3: Error Recovery Is Not Planable

LLMs are excellent at generating plans. They're terrible at handling deviations from plans — because error recovery requires situational awareness that the planning process doesn't carry forward.

When an agent encounters an unexpected error, it has several options:

• Retry with the same parameters (likely to fail again)
• Retry with modified parameters (requires understanding why it failed)
• Skip the step and continue (requires knowing the step is non-critical)
• Abort and escalate (requires knowing the whole plan's critical path)
• Regenerate a revised plan from scratch (computationally expensive)

The LLM, operating in a stateless request-response loop, has no memory of the original plan's intent beyond what's in the context window. It doesn't know which steps are on the critical path. It doesn't know whether a failure is transient or permanent. It makes a best-guess decision based on whatever context happened to be in the prompt.

The fix: Implement structured error handling as a first-class component of the agent architecture:

class ErrorRecoveryEngine:
    """
    Separates error handling logic from the LLM's planning logic.
    Uses deterministic rules and bounded LLM calls for recovery.
    """

    RECOVERY_STRATEGIES = {
        "timeout": ["retry_with_backoff", "skip_with_log", "abort"],
        "validation_error": ["retry_with_corrected_params", "ask_for_clarification", "abort"],
        "auth_failure": ["retry_with_refreshed_token", "abort"],
        "dependency_unavailable": ["retry_after_delay", "use_fallback", "abort"],
    }

    def handle(self, step: ExecutionStep, error: ExecutionError, context: PlanContext) -> RecoveryDecision:
        strategy_type = self._classify_error(error)
        strategies = self.RECOVERY_STRATEGIES.get(strategy_type, ["abort"])

        # Deterministic first-pass filtering
        viable = [s for s in strategies if self._is_viable(step, s, context)]

        # Bounded LLM call for nuanced decisions
        if len(viable) > 1:
            decision = self._llm_select_recovery(step, error, viable, context)
        else:
            decision = RecoveryDecision(action=viable[0])

        return decision

    def _is_viable(self, step: ExecutionStep, strategy: str, ctx: PlanContext) -> bool:
        """Deterministic checks — no LLM involved."""
        if strategy == "retry_with_backoff" and step.retry_count >= step.retry_policy.max_attempts:
            return False
        if strategy == "use_fallback" and not step.has_fallback:
            return False
        if strategy == "skip_with_log" and step.is_critical_path:
            return False
        return True

The key insight: don't ask the LLM to solve everything. Use deterministic logic for structural decisions (can we retry? have we exhausted retries? is this critical?) and reserve LLM calls for genuinely ambiguous situations where contextual judgment is needed. Each LLM call should be bounded — few tokens, focused question, short response.

Failure Mode 4: The Context Window Is a Lying Memory

Your agent's context window is its short-term memory. And it's a bad one.

At any given point, the context window contains:

• The original user request
• The generated plan
• Previous step results
• Tool outputs
• System prompts
• Few-shot examples (if any)

As the plan executes, this window grows. At some point, it hits the token limit. What gets truncated? Usually, it's the oldest messages — which often includes the original plan and the reasoning that produced it.

So the agent is now executing step five, but it can no longer see why it chose the approach in step one. It's making decisions in a vacuum, optimizing for local correctness rather than global coherence.

This is the amnesic agent problem: the agent literally cannot remember its own rationale as it progresses through a long plan.

The fix: Explicit external memory management — the plan and its rationale must be persisted outside the context window:

class AgentMemory:
    """
    External memory store decoupled from the LLM context window.
    Persists plan state, rationale, and execution history.
    """

    def __init__(self, store: KVStore):
        self.store = store

    def save_plan_context(self, run_id: str, context: PlanContext):
        """Persist full context for retrieval at any step."""
        self.store.set(f"plan:{run_id}", context, ttl=3600)

    def retrieve_relevant_context(self, run_id: str, current_step: int, query: str) -> ContextSnapshot:
        """
        Retrieve only the context relevant to the current decision point.
        Uses embedding similarity to avoid loading everything.
        """
        plan = self.store.get(f"plan:{run_id}")

        # Load rationale for nearby steps, not all steps
        window = range(max(0, current_step - 2), min(len(plan.steps), current_step + 1))
        relevant_steps = [plan.steps[i] for i in window]

        return ContextSnapshot(
            original_intent=plan.intent,
            relevant_steps=relevant_steps,
            recent_outcomes=plan.executed_steps[-3:],
            current_query=query
        )

This gives you two benefits: first, the agent always has access to its original intent; second, you can control what gets loaded into the context window at each step, keeping it lean and focused.

Failure Mode 5: Tool Use Is Under-Specified

Most agent frameworks treat tool use as a simple function-calling interface:

@tool(description="Search the knowledge base")
def search_knowledge_base(query: str) -> str:
    results = kb.search(query)
    return format_results(results)

The LLM sees the tool name and description and decides when to call it. But the description is always a natural language approximation of what the tool actually does. There's a semantic gap between "search the knowledge base" and the actual Elasticsearch query being constructed, the pagination logic, the relevance scoring, the error handling.

When the LLM calls a tool with slightly wrong parameters, the result is wrong. When it calls the tool at the wrong time, the plan derails. When the tool returns an error the LLM doesn't recognize, the agent loops or hallucinates a fix.

The fix: Formal tool contracts — every tool must declare its preconditions, postconditions, and error semantics:

interface ToolContract {
  name: string;
  description: string;

  // What must be true before calling
  preconditions: precondition[];

  // What the caller can expect after successful execution
  postconditions: postcondition[];

  // Exhaustive error taxonomy with recovery guidance
  errors: ErrorCode[];

  // Max latency the caller should expect
  latencySLA: Duration;

  // Whether results are cached (idempotent?)
  caching: CachingPolicy;
}

interface ErrorCode {
  code: string;
  message: string;
  recovery: RecoveryStrategy; // "retry", "abort", "ask_user", etc.
  hint: string; // What the LLM should try differently
}

This transforms tools from black boxes into verifiable components. The agent's execution engine can check preconditions before calling, interpret errors against a known taxonomy, and apply deterministic recovery strategies rather than hoping the LLM figures it out.

Failure Mode 6: No Feedback Loop Between Execution and Planning

Perhaps the most subtle failure mode: agents don't learn from execution.

A plan is generated, executed, and if it fails, the LLM is shown the error and asked to continue. But the LLM doesn't retain anything from this failure. The next time a similar plan is generated, it makes the same mistake. The system has no memory of what went wrong, no model of its own failure modes, no way to improve.

This is especially damaging because LLMs have a well-documented sycophancy problem — they tend to double down on their initial reasoning rather than self-correct when presented with contradictory evidence. When an agent fails at step 3 and you feed the error back, the LLM might acknowledge the error but then proceed to make the same type of error at step 4.

The fix: Build execution feedback into the planning pipeline as a first-class loop, not an afterthought:

class ExecutionFeedbackLoop:
    """
    Captures execution outcomes and feeds them back to improve planning.
    Operates at two levels: online (per-session) and offline (cross-session).
    """

    def __init__(self, session_store: SessionStore, improvement_engine: ImprovementEngine):
        self.session = session_store
        self.improve = improvement_engine

    def process_outcome(self, plan: ExecutablePlan, outcome: ExecutionOutcome):
        # Online: update the current plan with what we learned
        adjusted_plan = self._adjust_plan(plan, outcome)

        # Offline: accumulate patterns for systemic improvement
        self.improve.record_f failure_pattern(outcome)

        return adjusted_plan

    def _adjust_plan(self, plan: ExecutablePlan, outcome: ExecutionOutcome) -> ExecutablePlan:
        """
        Apply deterministic adjustments based on execution results.
        Not an LLM call — just rule-based plan modification.
        """
        adjusted = copy.deepcopy(plan)

        if outcome.step_failed:
            failed_step = outcome.failed_step

            # Add explicit error handling to the failed step
            failed_step.error_handling = self._infer_error_handler(failed_step, outcome)

            # Add dependency on the failed step succeeding
            for dependent in adjusted._find_dependents(failed_step.id):
                adjusted._add_dependency(dependent.id, failed_step.id)

        return adjusted

The key distinction: the online loop makes adjustments to the current plan using deterministic rules (add retry, add error handling, add dependencies). The offline loop accumulates failure patterns across sessions and uses them to improve the planner itself — perhaps by fine-tuning, perhaps by improving system prompts, perhaps by building a failure-mode database that the planner consults.

Architectural Pattern: The Executive Layer

All of these failure modes share a common root: the LLM is being asked to do something it's architecturally unsuited for — bridging the gap between high-level intent and low-level execution.

The solution is an executive layer — a software component that sits between the LLM (the planner) and the execution environment (the tools, APIs, databases). Its responsibilities are:

1. Translation — converting LLM-generated plans into executable, typed instructions
2. Validation — checking plans against current system state before execution
3. Orchestration — managing the execution DAG, dependencies, and concurrency
4. Error handling — applying deterministic recovery strategies
5. Observability — logging every decision, failure, and recovery action

interface ExecutiveLayer {
  /** Receive a plan from the LLM, validate, and execute */
  execute(plan: LLMGeneratedPlan): AsyncGenerator<ExecutionEvent>;

  /** Check if the current system state supports the plan's assumptions */
  validateAssumptions(plan: LLMGeneratedPlan): ValidationResult;

  /** Recover from a failure without restarting the entire plan */
  recover(step: FailedStep, error: ExecutionError): RecoveryPlan;

  /** Log the full execution trace for debugging and improvement */
  trace(): ExecutionTrace;
}

This layer is not an LLM. It's deterministic software. It doesn't need to be smart — it needs to be correct. The LLM provides the creativity and adaptation. The executive layer provides the reliability and accountability.

What This Means for Your Agent Architecture

If you're building AI agents today, you're almost certainly under-instrumented. The gap between "the agent planned this" and "the agent executed this correctly" is where your bugs live. Here's where to focus:

Immediate fixes (this week):

• Add structured validation of plans before execution starts
• Log every tool call with its inputs, outputs, and latency
• Implement basic retry logic with exponential backoff
• Pin tool descriptions to formal schemas, not natural language

Medium-term (this sprint):

• Build the executive layer as a separate component
• Add preconditions/postconditions to all tools
• Implement the execution feedback loop for online corrections
• Create a deterministic error taxonomy for all failure modes

Longer-term (this quarter):

• Cross-session failure pattern accumulation
• Automated plan refinement based on historical success rates
• Formal verification of plan feasibility against system contracts
• Separation of planning and execution into independent deployment units

Frequently Asked Questions

Q: Should I use a smaller model for the execution layer?
A: No — the executive layer shouldn't use an LLM at all. It should be deterministic code. If you find yourself needing an LLM inside your execution loop, you've likely pushed responsibility onto the model that should belong to your software architecture. The LLM belongs in planning and in genuinely ambiguous error resolution, not in the hot path of execution.

Q: How do I handle plans that legitimately need LLM judgment during execution?
A: Limit these to bounded, isolated calls with clear input/output contracts. When the execution engine encounters an ambiguous error state, it should collect all relevant context, make a single focused LLM call (not a free-form conversation), and treat the response as a recommendation — not a command. The executive layer should always be able to override or reject the LLM's suggestion.

Q: Isn't all of this over-engineering? Why can't the LLM just do it?
A: Because the LLM can't. Not reliably. Not repeatedly. Not at scale. The same properties that make LLMs flexible — probabilistic token generation, context window limitations, lack of persistent state, no native understanding of side effects — are the same properties that make them unreliable as execution engines. You could keep pushing prompt engineering further, but you'll hit a wall. The wall is the difference between saying something and doing something. Closing that gap requires engineering, not prompting.