Your AI Agent Graph Compiles. That Doesn't Mean It's Correct.

Pavel Ravvich — Tue, 27 Oct 2026 20:38:01 +0000

Your AI Agent Graph Compiles. That Doesn't Mean It's Correct.

Why TypeScript can't catch certain bugs in multi-agent workflows — and what static graph analysis can do about them.

TypeScript catches a remarkable number of mistakes before an application ever runs. Incompatible types, incorrect arguments, missing properties — the compiler handles these pretty well.

But when building complex AI workflows, there's a problem that TypeScript alone can't solve:

Every component in a graph can be correctly typed while the graph itself is structurally incorrect.

I'm not talking about LLM hallucinations, unreliable prompts, or model nondeterminism.

I'm talking about ordinary engineering bugs caused by how execution paths connect.

Let's look at a concrete example.

1. When Types Are Correct but the Graph Isn't

Imagine a contract-processing system. Before publication, each contract must pass a compliance review.

The system also supports a fast-track route that skips part of the normal processing.

The workflow looks like this:

                       Start
                         |
                       Triage
                         |
                 +-------+-------+
                 |               |
             Normal route    Fast track
                 |               |
          Compliance Review      |
                 |               |
                 +-------+-------+
                         |
                  PublishContract

At first glance, this seems reasonable.

Except there's a problem.

PublishContract requires a compliance approval, but the fast-track route skips the component that produces it.

In other words, there is a reachable execution path where required input data is never created.

What does TypeScript think about this?

2. Why TypeScript Misses the Bug

Let's use a conventional LangGraph StateGraph.

First, define the shared state:

import {
  Annotation,
  StateGraph,
  START,
  END,
} from "@langchain/langgraph";

interface ContractDraft {
  contractId: string;
  summary: string;
  fastTrackRequested: boolean;
}

interface ComplianceApproval {
  reviewer: string;
  certificateId: string;
}

const State = Annotation.Root({
  draft: Annotation<ContractDraft>(),
  approval: Annotation<ComplianceApproval | undefined>(),
  publishedRecord: Annotation<string | undefined>(),
});

The approval field is declared as ComplianceApproval | undefined.

That makes sense. At the beginning of the workflow, the compliance review hasn't happened yet.

Now let's add the publication node:

.addNode("publishContract", (state) => {
  const cert =
    state.approval!.certificateId.toUpperCase();

  return {
    publishedRecord:
      `PUBLISHED:${state.draft.contractId}:${cert}`,
  };
})

The non-null assertion operator (!) tells TypeScript that we are certain the value exists.

But that certainty has nothing to do with the actual structure of the graph.

Now consider the conditional transition:

.addConditionalEdges("triage", (state) =>
  state.draft.fastTrackRequested
    ? "publishContract"
    : "complianceReview"
)

If fastTrackRequested is true, execution reaches publishContract without producing approval.

TypeScript accepts the code.

At runtime, accessing certificateId on undefined throws.

Of course, we could remove the non-null assertion and handle the missing value explicitly. That would prevent this particular exception, but it wouldn't fix the architectural problem: the workflow still allows publication without the required compliance review.

The Fundamental Limitation

TypeScript understands types and local control flow within functions.

But it doesn't prove that an arbitrary sequence of nodes in an external execution graph will produce every value required by downstream components.

To establish that guarantee, we need to know more than the type of a value.

We need to know whether that value is available on every execution path that reaches its consumer.

That's a dataflow analysis problem.

3. From Component Types to Graph Verification

In a conventional application, we might write:

function publish(
  draft: ContractDraft,
  approval: ComplianceApproval
): PublishedContract

TypeScript verifies that both arguments are present at a call site.

But in a graph-based system, the producer of ComplianceApproval and its consumer may be separated by conditional branches, loops, and parallel execution paths.

Typing individual components is no longer enough.

We need to answer a more difficult question:

For every reachable execution path, are all required inputs guaranteed to exist when a component runs?

This is one of the problems I set out to address with AGBoss, an open-source TypeScript framework built on top of LangGraph.

AGBoss components declare their input and output DTOs explicitly.

Here's an example from the project's executable test suite. An action renders an invoice using an order request and a coupon discount:

@Action({
  name: "render-invoice-action",
  description: "Renders final invoice",
  input: {
    request: CheckoutRequestDto,
    coupon: CouponDiscountDto,
  },
  output: InvoiceSummaryDto,
})
export class RenderInvoiceAction
  implements ActionHandler<RenderInvoiceAction> {

  async execute(
    input: InputOf<RenderInvoiceAction>
  ): Promise<OutputOf<RenderInvoiceAction>> {
    // Generate the invoice
  }
}

The component explicitly requires two inputs: an order request and a calculated coupon.

Now imagine a workflow with two routes. VIP customers receive a promotional discount, while standard customers proceed directly to invoice generation.

start(MissingInputBrokenStart)
  .returns(CheckoutRequestDto)
  .toRouter(TierBrokenRouter)
  .chooseOne(
    route(ApplyVipCouponAction)
      .returns(CouponDiscountDto)
      .toAction(RenderInvoiceAction),

    route(RenderInvoiceAction),
  )

Look at the second route.

It invokes RenderInvoiceAction without producing the required CouponDiscountDto.

The DTOs are valid. The component contracts are valid. But the graph contains a structural error.

4. Catching the Problem Before Execution

AGBoss includes a static graph analyzer, invoked with:

npx agboss check

It analyzes reachable execution paths and tracks which data is available to each component.

For our example, it reports:

[input.missing-on-path]

The diagnostic identifies the required coupon input of render-invoice-action.

This is a different level of verification.

Instead of waiting for a particular combination of inputs to trigger a failure in production, we can detect the structural problem before starting the application.

And because this is static analysis, it doesn't require calling an LLM, spending tokens, or relying on nondeterministic model responses.

What Does the Fix Look Like?

We need to ensure that every route reaches a component whose declared inputs are actually available.

One solution is to use separate invoice actions for VIP and standard orders:

start(MissingInputFixedStart)
  .returns(CheckoutRequestDto)
  .toRouter(TierRouter)
  .chooseOne(
    route(ApplyVipCouponAction)
      .returns(CouponDiscountDto)
      .toAction(RenderInvoiceAction)
      .returns(InvoiceSummaryDto)
      .toFinish(MissingInputFixedFinish),

    route(RenderStandardInvoiceAction)
      .returns(InvoiceSummaryDto)
      .toFinish(MissingInputFixedFinish),
  )

Now the VIP branch produces a coupon before rendering the invoice, while the standard branch uses an action that doesn't require one.

Both branches produce InvoiceSummaryDto.

We've fixed the dataflow structure, rather than merely adding another runtime check.

5. Missing Data Is Only the Beginning

While working on this problem, I realized that checking whether a DTO exists is only the first step.

Complex workflows have several other interesting classes of structural bugs.

Stale Data After Revision Loops

Imagine an agent drafts a document, another agent approves it, and a third requests changes.

The document is revised, but the previous approval remains in the workflow state.

The approval exists. Its type is correct. But it belongs to an earlier version of the document.

AGBoss detects this class of problem with:

[input.outdated-on-path]

Conflicting Parallel Branches

Suppose two branches run concurrently.

One modifies a financial calculation, while the other audits the previous version.

When the branches join, the audit result may no longer correspond to the current data.

The analyzer can report:

[launch.branch-outdates-sibling]

Uncompensated Side Effects

Consider an order workflow that reserves inventory, charges a customer, and confirms the purchase.

If the payment fails and an error route reaches the workflow finish without releasing the inventory, the reservation may remain active.

AGBoss supports declared compensating actions and checks compensation coverage on error paths.

The corresponding diagnostic is:

[flow.uncompensated-action]

These are different problems, but they share a common principle:

The correctness of a graph cannot be inferred from the correctness of its individual nodes.

6. Reproduce the Failures Yourself

I wanted these claims to be reproducible rather than just theoretical.

So I created a separate repository of executable examples:

AGBoss Examples — 02-path-scope-proof

It covers all four scenarios.

Each includes:

A broken LangGraph implementation that passes TypeScript compilation.
A broken AGBoss workflow with an expected static diagnostic.
A corrected AGBoss implementation.
Automated tests that verify the expected behavior.

After installing the example's dependencies, you can run:

npm run check
npm test
npm run demo

The scenarios are designed to run deterministically and offline, without external LLM API calls.

The goal isn't simply to display a convincing error message.

It's to reproduce a concrete failure class and verify that the analyzer detects it.

7. What Static Analysis Cannot Guarantee

Static analysis doesn't magically prove that an entire application is correct.

It won't tell you whether a discount was calculated correctly, whether a legal document complies with regulations, or whether an LLM produced a useful answer.

It also relies on developers declaring component contracts and data dependencies accurately.

But it can verify structural properties that would otherwise require extensive testing.

And the more complex a graph becomes, the more valuable that distinction is.

With three nodes and two transitions, manual inspection is relatively easy.

With dozens of nodes, conditional routes, loops, parallel branches, and nested workflows, the number of possible execution paths can grow rapidly.

Testing every combination independently becomes increasingly impractical.

Static analysis doesn't replace testing.

It helps eliminate entire classes of errors before testing even begins.

Final Thoughts

In conventional software engineering, we've long accepted that certain bugs are better caught at build time than at runtime.

I think AI-agent systems deserve the same approach.

LLMs introduce nondeterminism. That's unavoidable.

But the presence of nondeterministic components doesn't mean the rest of the system has to be unpredictable.

Graph structure, component contracts, data availability, and execution paths are engineering problems.

And many of them can be verified before the first model call.

That's the idea behind AGBoss, the open-source TypeScript framework I'm building on top of LangGraph, with dependency injection, declarative components, and static graph verification.

I'm curious how other developers approach this.

Have you encountered bugs where every node was correctly typed, but the workflow failed only on a particular execution path? How did you catch them?

GitHub: https://github.com/agbossjs/agboss

Executable examples: https://github.com/agbossjs/agboss-examples/tree/main/02-path-scope-proof

Documentation: https://agboss.dev

Tags: #typescript #ai #architecture #opensource

TypeSafe's Jev: Independent Benchmark Against LLMs (with code)

Pavel Ravvich — Sun, 27 Sep 2026 08:34:27 +0000

I built an independent benchmark to test TypeSafe's Jev model against GPT-4, Claude, and Gemini on classification tasks.

Jev is a different kind of model — instead of generating text, it outputs probabilities for given answer choices. This makes it particularly interesting for:

Intent routing in AI agents
Guardrails and safety checks
Low-latency classification pipelines

The benchmark covers spam detection, sentiment analysis, and topic classification tasks with full reproducible code.

Read the full article + code on Medium:
https://medium.com/@pravvich/typesafes-jev-beyond-the-hype-an-independent-benchmark-8bdc1c99d000GitHub repo:
https://github.com/PavelRavvich/jev-bench

Connect with me on LinkedIn:
https://www.linkedin.com/in/pavel-ravvich/

DEV Community: Pavel Ravvich

Your AI Agent Graph Compiles. That Doesn't Mean It's Correct.

Your AI Agent Graph Compiles. That Doesn't Mean It's Correct.

1. When Types Are Correct but the Graph Isn't

2. Why TypeScript Misses the Bug

The Fundamental Limitation

3. From Component Types to Graph Verification

4. Catching the Problem Before Execution

What Does the Fix Look Like?

5. Missing Data Is Only the Beginning

Stale Data After Revision Loops

Conflicting Parallel Branches

Uncompensated Side Effects

6. Reproduce the Failures Yourself

7. What Static Analysis Cannot Guarantee

Final Thoughts

TypeSafe's Jev: Independent Benchmark Against LLMs (with code)