ruixuan jiang

Posted on Sep 25

Your Agent's Free-Text Output Is an API You Never Designed

#ai #llm #architecture #programming

Most agent demos fail in the same place, and it is not the model.

It is the boundary where the model's output leaves the model and enters your program. Up to that point it is a string. Strings do not have a schema.

So you end up with code like this:

const reply = await llm.complete(prompt);

// please work
if (reply.toLowerCase().includes("approve")) {
  await merge();
} else {
  await requestChanges();
}

This works until the model writes "I would not approve this yet" and your substring check matches approve. Now a change that should have been blocked got merged.

The hidden interface

Any time you parse meaning out of generated text, you have declared an API. You just did not write it down.

That interface has properties you probably did not intend:

It is not versioned. Change the prompt and the contract silently changes.
It is not validated. There is no error when the shape is wrong, only a wrong branch.
It cannot be logged as a decision. You can log a paragraph, but you cannot log "the model chose X from a set of three."

The fix is not a better prompt. Prompt engineering narrows the failure rate; it does not remove the parser.

Ask for a decision, not a paragraph

An agent step usually needs two different outputs, and they should not be the same output:

A description for humans — what it noticed, what surprised it, what it is unsure about.
A decision for the program — a value the host can validate and branch on.

The decision should come from a closed set that your code already understands:

type ReviewDecision = {
  status: "pass" | "review" | "fail";
  confidence: "low" | "medium" | "high";
  findings: Array<{ file: string; note: string; severity: "info" | "warn" | "block" }>;
};

Now the failure modes separate cleanly:

What broke	Where to look
`status` came back as `"probably fine"`	Contract violation — reject before acting
`status` was valid but wrong	Judgment problem — improve evidence or prompt
`status` was right but the wrong branch ran	Implementation bug in your own code

Without the typed boundary, all three look like "the AI did something weird," and you have no way to tell them apart.

Validate before you act

The important part is not the schema. It is that the schema is checked before anything changes.

const ALLOWED = new Set(["pass", "review", "fail"]);

function assertDecision(value: unknown): ReviewDecision {
  if (typeof value !== "object" || value === null) {
    throw new Error("Decision is not an object");
  }
  const d = value as Record<string, unknown>;
  if (typeof d.status !== "string" || !ALLOWED.has(d.status)) {
    throw new Error(`Illegal status: ${String(d.status)}`);
  }
  if (!Array.isArray(d.findings)) {
    throw new Error("findings must be an array");
  }
  return {
    status: d.status as ReviewDecision["status"],
    confidence: d.confidence === "high" || d.confidence === "medium" ? d.confidence : "low",
    findings: d.findings as ReviewDecision["findings"],
  };
}

A thrown error is a feature here. It is a loud, recoverable failure that happens before a merge, a payment, or a deploy — instead of a silent wrong branch.

This is the pattern behind structured judgment tools, like the Choice and Score interfaces exposed over MCP in this case: the model supplies the judgment, the typing layer fixes its shape, and the calling program gets a value it can compare or gate on instead of prose it has to interpret.

Keep the evidence with the decision

A typed result is not automatically trustworthy. status: "pass" with no support is just a shorter guess.

So the decision record should carry:

the inputs the model actually saw (or a hash and a reference)
the allowed output space
the chosen value
the evidence behind it
an identifier and a timestamp

That gives you a replayable record. When a bad decision ships, you can tell whether the right evidence was missing, the wrong rule was applied, or the validation layer was too loose.

What this does not fix

Typing the output does not:

make the model correct
make a subjective label objective
replace tests, permissions, review, or rollback
turn an experiment into a production control

It solves one narrow problem: the host no longer has to guess a control signal out of decorative prose.

That is worth a lot. It is much easier to reason about a system where you can see the allowed choices, the selected choice, the evidence, and the resulting action.

A short checklist

Before an agent step is allowed to change state, I want answers to these:

What is the model allowed to decide?
What is the program allowed to execute?
Are those represented separately?
Can an invalid decision be rejected before any action runs?
Is the evidence visible without reading the whole transcript?
Is there a pause or rollback path when confidence is low?

If those are all "no," the demo can still look impressive. It is just not yet an instrument.

Disclosure: I maintain JevCases, an independent index of Jev use cases and experiments. It is not affiliated with TypeSafe.

DEV Community