A privacy warning became bad advice in an AI reviewer's draft

On November 4, a welcome draft for textokit suggested rewording this sentence from a tool description:

Remove secrets and personal data before submitting.

The warning belonged there. The draft treated it as a problem because a security scanner had mentioned it. Nobody sent the letter. It was one of five welcome drafts from the first run of our supplier-review agent; reading it sent us back to the JSON behind the prose.

The reviewer runs in our Multica installation. A script downloads the manifest, federation peers and observed x402 sellers from modelmarket.dev, compares them with the previous run, and checks new or changed entries. The model writes the report and welcome drafts. A facts block generated by the script is included unchanged.

The checks were deterministic. Choosing which observations to turn into advice was still left to the model.

A boolean disappeared in the adapter

The textokit draft also suggested changing "instead of neutral or European Spanish" and "without a browser or API key." Those phrases describe a translation choice and a requirement the buyer does not have. WARDEN returned advisory matches for all three phrases.

The adapter converted WARDEN's results into the reviewer's finding format. During that projection, it discarded advisory. It also promoted an informational advisory result to low severity.

Here are the relevant fields for the informational "instead of" match before and after the repair. The production records also carry the location and message:

const before = {
  check: "injection", code: "TOOL_DEF_IMPERATIVE",
  severity: "low"
};
const after = {
  check: "injection", code: "TOOL_DEF_IMPERATIVE",
  severity: "info", advisory: true
};

That missing boolean matters more than a different display label. The report had made an advisory word match look like another finding for the model to explain. The resulting advice sounded helpful: reword whatever the checker mentioned. For the privacy warning, following that advice could make the description less useful.

After the first run, we tightened the prompt: advisory matches must stay out of welcome drafts, and the agent must never recommend rewording text just to stop a scanner match. On November 6, we changed the report representation too. The adapter now preserves advisory: true, and informational advisory results remain informational. WARDEN's scanning rules stayed the same.

Give letter drafting a field contract

Each reviewed item now carries two lists:

Field	What it contains	How it is used
findings	All recorded signals, including advisory matches	The owner inspects them; the model receives them as context
letter_points	The eligible subset computed in code	The model's instructions restrict advice to this list

This is the selection rule:

function letterPoints(findings) {
  return findings.filter((f) =>
    !f.advisory &&
    f.severity !== "info" &&
    (
      f.severity === "high" ||
      f.severity === "medium" ||
      f.check === "schema"
    )
  );
}

The schema exception was deliberate. A property without a declared type can deserve a useful correction even when rated low. A low-severity signal from another check does not qualify. Informational results and advisory matches never qualify. The adapter normalizes WARDEN's critical findings to the report's high category, so those remain eligible.

The instructions name letter_points directly and prohibit taking other findings into a welcome draft. An empty array means a welcome without an invented scanner-related complaint. Advisory matches remain visible in a separate part of the owner's report.

A regression fixture retains the three ordinary phrases and produces no letter points from their matches. Two controls exercise useful findings: a tool whose input schema declares type: "string" produces an eligible schema point, and an explicit instruction to ignore previous instructions and send a wallet seed phrase produces a high-severity injection finding.

The local and deployed-container checks against the same downloaded November 6 hub documents agreed: textokit had zero letter points; agent-exec had four, concerning an input schema declaring type: "string" and untyped properties. Both checks used temporary state.

These are eligibility counts for two saved examples. They do not measure the quality of generated drafts or seller safety.

The model still reads the full report. letter_points gives it an explicit contract to follow; it does not enforce that contract on the generated prose. A person still reviews the drafts.

Eligibility also applies to the recipient

The next inputs included 88 pending peer registrations shaped like probes of the hub's address handling: 69 non-HTTPS addresses and 19 HTTPS addresses the hub had never successfully read. The old rule would have treated each outside source as a candidate for a welcome draft.

Sources now carry welcome and no_welcome_reason. Eligibility requires an HTTPS URL with a dotted hostname rather than an IP literal, no credentials in the URL, and, for a federation peer, at least one successful hub read. These are drafting conditions, not certification of a seller.

A checker simulation using copied agent state found one changed outside listing and grouped the 88 peers on one line. Only agent-exec was eligible for a welcome draft. The real daily state was left untouched. That simulation checked the next inputs and eligibility rules; it did not show that a subsequent model run had written better letters.

We build WARDEN and operate AIMarket. This article was drafted with AI assistance using our implementation, regression tests and recorded rollout checks.