DEV Community: Tang Haoran

Proving Your AI Agent Rules Hold for Every Input — Not Just the Ones You Tested

Tang Haoran — Fri, 04 Sep 2026 04:21:39 +0000

Proving Your AI Agent Rules Hold for Every Input — Not Just the Ones You Tested

Here's a question your auditors will eventually ask: "This decision the agent made — on what basis?"

LLMs are probabilistic. Ask twice, get two answers. If your enterprise delegates approvals, refunds, or access decisions to an agent and the only thing standing between it and a bad call is a prompt, you don't have governance — you have a probability distribution with a job title.

The industry is converging on an answer: the LLM handles understanding; rules handle the verdict. Put a deterministic rule engine in front of the model, and let it decide what the agent may and may not do, no matter what the model says.

But here's the uncomfortable part: how do you know the rules themselves are deterministic?

Most rule engines back that claim with unit tests. And unit tests prove exactly one thing: the inputs you tested behave correctly. They say nothing about the inputs you didn't test.

This post is about closing that gap — with three open-source projects that attack the problem at three different levels:

Layer	Project	The question it answers
Language	ERDL	"Can we express the rule unambiguously?"
Tests	erdl-vectors	"Do independent implementations agree, byte for byte?"
Proof	erdl-formal	"Does it hold for every input?"

Together they move "deterministic" from a claim to a measurement — and, in the limit, to a proof.

Layer 1 — ERDL: a language where "deterministic" is the point

ERDL (Entity-Rule Definition Language) is a declarative rule format for AI agent behavior governance. The core idea is a when → then decision expressed in plain YAML:

protocol: "erdl/v2"
version: "2.1.0"
metadata:
  name: "refund-guard"
  decision: ALLOW
rules:
  - name: "SEC-001-refund-limit"
    priority: 10
    when:
      logic: AND
      conditions:
        - field: "tool.name"
          operator: eq
          value: "issue_refund"
        - field: "tool.args.amount"
          operator: gt
          value: 5000
    then: REQUEST_HUMAN
    message: "Refund amount over 5000, human approval required"

Three things make this different from "just YAML config":

A single semantic tree. Every rule compiles to one of a 34-node expression tree — the same tree whether you wrote it in the Simple projection (30 operators), the Expression projection, or a decision table. Three ways to author; one way to mean.
A precise evaluation semantics. The tree is evaluated under a set of named constraints (E1–E12) that pin down the fuzzy parts of real-world rules: fixed-point decimal arithmetic for money (scale=14, half-even rounding — so 0.1 + 0.2 can't drift), three-valued logic for missing fields (a missing field folds to false, so nothing fail-opens), empty-quantifier folding (all([]) is false), and NFC string normalization.
Auditable by construction. Every evaluation produces a hashable, chainable Decision Object — the audit record of which rule fired, on what input, with what context.

The reference implementation ships as an npm package:

import { loadErdlFile, Evaluator } from '@openoba/erdl'

const { rules, metadata } = loadErdlFile('refund.erdl.yaml')
const result = new Evaluator().evaluate(rules, {
  tool: { name: 'issue_refund', args: { amount: 8000 } },
  'metadata.decision': metadata.decision,
})
console.log(result.decision) // 'REQUEST_HUMAN'

But a language is only as trustworthy as the claim "every implementation of it agrees." That's where the second layer comes in.

Layer 2 — erdl-vectors: trust is measured, not endorsed

erdl-vectors is a cross-implementation verification benchmark: 301 frozen test vectors that don't belong to any single implementation.

The mechanism is deliberately adversarial to hand-waving:

A neutral spec. Vectors are generated from the spec alone, with answers stored in a physically isolated file (.gitignored) so nobody can "pass" by reading the oracle.
First-principles verification. A runner must re-implement JCS (RFC 8785) and SHA-256 from scratch — no json-canonicalize, no SDK — then recompute every Decision Object hash byte-for-byte.
A canary for honesty. One vector (K01) is generated by a deliberately broken implementation. A correct runner must report it as a mismatch. A runner that skips independent recomputation and just echoes the expected answer gets caught on the spot.

The audit layer (78 vectors) is now verified byte-for-byte by two independent third-party runners — one in Go (norviq-go), one in Python (concordia-python, by Erik Newton of Concordia) — each matching 107/107 canonical bytes.

The principle behind all of this is captured in the repo's own line: "Neutrality isn't declared — it's measured." The registry records who, on what date, passed how many vectors — nothing more. Nobody gets an endorsement; the numbers speak for themselves.

This matters because it answers the question "is the spec right, or does the reference implementation just agree with its own generator?" Only when multiple unrelated implementations, built from the spec text alone, converge byte-for-byte do you have evidence that the standard itself is sound.

Layer 3 — erdl-formal: from "tested" to "proven"

Here's the thing about test vectors, even 301 of them: they are samples. Vectors prove the cases you chose to include. They can never prove the cases you didn't.

erdl-formal is the layer that closes that gap. It compiles ERDL's expression kernel into SMT (via Z3) and proves properties over all inputs — not a sample, the entire space.

A single assertion proves two things at once:

from erdl_formal.field_contracts import FieldContract, Schema
from erdl_formal.properties import always_denies

schema = Schema()
schema.add(FieldContract(field="file_cls", type="int"))
schema.add(FieldContract(field="op_cls", type="int"))

# when: file_cls > op_cls  →  DENY
rule = ["gt", ["field", "file_cls"], ["field", "op_cls"]]

assert always_denies(rule, schema,
                     premises=["file_cls", "op_cls"],
                     missing_field="op_cls")

Behind that one line, Z3 searches the space of all integers for a violating input. If it finds one, you get a concrete counterexample you can replay against the real engine. If it finds none — UNSAT — the property holds for every input, and the proof is complete.

What can it prove?

never-errors — evaluation never throws
always-denies — hit means block (including fail-closed: a missing field can't bypass the rule)
always-allows / subsumption / equivalence / disjointness
override-soundness — an override can only relax DENY→ALLOW, never tighten toward a less safe state
ring-respect and emergency-shortcut — ERDL-specific semantics that Cedar/OPA don't even model

The three "ERDL-specific" properties are the interesting ones: they're not generic policy properties, they're guarantees about this language's money, time, aggregation, quantifier, and decision-object semantics — the parts that make ERDL an enterprise rules kernel rather than a generic policy DSL.

A note on scope, because honesty builds trust: erdl-formal proves the expression kernel — the full 34-node tree and the E1–E12 constraints. Document structure, gloss rendering, and integration patterns are covered by the vectors and by engineering verification, not by SMT. It's a precise claim, and the precision is what makes it worth something.

Why all three — and not just the proof?

Because they answer different questions, and each one makes the next credible:

ERDL gives the language a canonical meaning — without it, there's nothing to prove about.
erdl-vectors proves that meaning is reproducible — that independent implementations, from the spec alone, converge byte-for-byte.
erdl-formal proves that meaning is safe — that the semantics hold over all inputs, not just the sampled ones.

A language without vectors is "trust my implementation." Vectors without a language are just a benchmark for something no one uses. Proof without vectors is a proof of a semantics only you implemented — which is a proof about your code, not the standard.

Layered together, they're the difference between "our rules engine is deterministic" (a claim) and "here is the language, here is the byte-for-byte agreement, here is the proof" (an audit trail).

Why this matters now: A2A is coming

The urgency isn't just about single agents. As agent-to-agent (A2A) protocols grow, agents will start delegating decisions to each other — one agent approves, another acts, a third records. In that world, cross-implementation trust can't rest on bilateral agreements between vendors. It has to rest on something any independent party can verify.

That's the standardization path this stack is built for: three independent implementations, one open spec, no single owner. Every new independent runner is a brick in the trust infrastructure for the agent economy.

Try it

ERDL engine — npm install @openoba/erdl · spec · MIT
erdl-formal — pip install erdl-formal · repo · Apache-2.0
erdl-vectors — repo · Apache-2.0 · open call for independent runners: implement JCS + SHA-256 from the spec, verify all 78 audit vectors, and get recorded in the registry.

The 223 expression-layer vectors are still waiting for their first independent runner. If you want to prove a standard rather than endorse one — the repo is open.

Determinism isn't declared. It's tested. And in the limit, it's proven.

rulsynor-core v1.0: An AI Agent You Can Audit

Tang Haoran — Sat, 08 Aug 2026 06:51:27 +0000

rulsynor-core v1.0: An AI Agent You Can Audit

How do you trust an AI Agent in production? Not "trust" as in I think it'll do the right thing. Trust as in cryptographic proof that every decision was correct, traceable, and independently verifiable.

Today: rulsynor-core — open-source Guard engine for AI Agents. MIT license. On npm.

30-Second Demo


bash
npx @openoba/rulsynor-core --tool=exec --cmd="rm -rf /"
🛡️ Decision: DENY
📝 Reason: Destructive command blocked.
🧾 Recorded: sha256:8274b0... (tamper-evident)
🧭 Alternative: Use the read tool to inspect first.
Not just "no." The agent learns why and what to do instead.

What It Does
Guard: Evaluates every tool call before execution — ring-sorted, sub-millisecond. 30 preset rules + write your own.
Audit: Every decision produces a 25-field cryptographic record. JCS + SHA-256. Chain-linked.
Verify: Anyone can verify audit records with zero SDK. Just JCS + SHA-256. No rulsynor needed.
What Makes It Different
Most AI safety tools are prompt-based. "Be careful" isn't governance. rulsynor-core gives you:

7 decision types — ALLOW, DENY, CORRECT, QUARANTINE, REQUEST_HUMAN, NOTIFY, EMERGENCY_HALT
Cryptographic audit chain — not "we logged it," mathematical proof
Third-party verified — 101 cross-implementation vectors, 13/13 passed by Concordia
MIT licensed — zero framework dependencies
Rules in Plain Language
复制
# "If the agent runs rm -rf, block it"
name: block-destructive-rm
when:
  conditions:
    - field: context.tool.name
      operator: eq, value: exec
    - field: context.tool.args.command
      operator: matches, value: rm\s+-rf
then:
  decision: DENY
  instruction: Destructive command blocked.
Any LLM can translate English descriptions into ERDL rules. The compiler validates everything before loading.

5-Minute Integration
import { Evaluator, GuardStateManager,
  loadPresetRules, toCompiledRules }
from '@openoba/rulsynor-core';

const rules = toCompiledRules(loadPresetRules());
const evaluator = new Evaluator(new GuardStateManager());

const result = evaluator.evaluate(
  { toolName, toolArgs, sessionId, agentId },
  rules
);
// result.decision → ALLOW | DENY | CORRECT | ...
Works with LangChain, MCP, custom ReAct loops. Same API everywhere.

Roadmap
 30 preset rules, 20 operators
 JCS+SHA-256 audit trail
 Third-party verified
 LangGraph integration guide
 Community rule marketplace
GitHub · npm

"LLM vendors deliver exceptional intelligence. We deliver accountability."

Built at OpenOBA. MIT.

I proposed a rules layer for the A2A protocol. Here's why.

Tang Haoran — Sun, 05 Jul 2026 13:04:42 +0000

The AI agent protocol stack is taking shape fast. Two standards are now widely adopted:

MCP (Anthropic → Linux Foundation): Agent ↔ Tools. 97M+ monthly downloads.
A2A (Google → Linux Foundation): Agent ↔ Agent. 150+ organizations in production.

Both are solid protocols. But they expose a question that every engineer deploying agents hits eventually:

Who decides what an agent shouldn't do?

The current answer: Prompts

You are a responsible agent. Do not execute dangerous commands.
Follow security policies. Respect data boundaries.

Any engineer who's actually deployed an agent knows how this ends. The prompt is a suggestion. The LLM is a probability engine. It doesn't understand "forbidden" — it understands "low probability."

That's fine for a chatbot. It's not fine for banking, healthcare, or production infrastructure.

What's needed is a mechanism that sits before the tool call and says "no" deterministically — not as an LLM judgment, but as a protocol-level enforcement.

What I proposed in the A2A community

Last week, I opened a discussion on the A2A repo (#2031) proposing an extension to the Agent Card: a way for agents to declare their behavioral rules at the protocol level.

The reactions confirmed what I suspected: this is a real gap.

A respected contributor from the community (chopmob-cloud / AlgoVoi) responded with a sharp distinction that materially improved the proposal: a trust score is reputation, but compliance requires evidence. A number like 850 says "this agent is generally trusted." A recomputable, content-addressed record — hash(rule_version + inputs + verdict) — says "this specific decision was permitted, and you can verify it yourself with no keys and no issuer contact."

That distinction is now baked into the spec. Reputation scores are advisory. Compliance is per-decision, recomputable, no-trust-required.

ERDL in 30 seconds

ERDL (Entity-Rule Definition Language) is a declarative rules standard. One YAML file. Placed in the agent's workspace. Evaluated by a deterministic engine before every tool call.

# agent.erdl.yaml

rules:
  - name: "no-dangerous-commands"
    priority: 100
    when:
      logic: AND
      conditions:
        - field: "tool.name"
          operator: eq
          value: "exec"
        - field: "tool.args.command"
          operator: match
          value: "(delete|format|drop table)"
    then: BLOCK
    message: "Dangerous command intercepted"

  - name: "required-human-approval"
    priority: 45
    when:
      logic: AND
      conditions:
        - field: "action.impact_tier"
          operator: gte
          value: "high"
        - field: "action.domain"
          operator: in
          value: ["employment", "credit", "healthcare"]
    then: REQUEST_HUMAN
    message: "High-impact decision requires human approval"

The agent cannot bypass this. It's not a prompt. It's a gate.

Where ERDL fits

A2A  (Google)   Agent ↔ Agent   Communication
ERDL (OpenOBA)  Rules & Governance   ← This
MCP  (Anthropic) Agent ↔ Tool   Tools

It doesn't replace either. It complements them.

MCP says what tools the agent can use
A2A says who the agent can talk to
ERDL says what the agent should and shouldn't do

The evidence-first architecture

This is the part I'm most proud of, and it came directly from community feedback.

Every ERDL decision generates an evidence record:

rule_name: "no-dangerous-commands"
rule_version: 1.3.2
inputs: { tool: "exec", command: "drop table users" }
eval_tree:
  - tool.name == "exec" → true
  - command MATCH "(delete|format|drop table)" → true
  - AND → true
verdict: BLOCK
timestamp: 2026-07-05T09:23:11.452Z
receipt: sha256(canonicalize(above))

Any third party — a regulator, a partner, an auditor — can re-derive the receipt from the record alone. No keys, no issuer contact, no "trust me."

This is the difference between saying "our agent is safe" and proving it.

What we have so far

Spec v1.0: English + Chinese, MIT license
Reference implementation: npm install @openoba/erdl, 197 tests, 0 failures
Agent Card extension proposal: under active discussion on A2A #2031
Landing page: openoba.github.io/erdl-landing

What we need

This isn't a solo project. A protocol standard only works if people use it, critique it, and contribute to it.

If you've deployed AI agents and hit the "how do I stop it from doing X" problem — I'd love to hear about your experience. What guardrails did you build? What broke?

If you think a rules layer in the agent protocol stack makes sense — or doesn't — I'd welcome the debate.

GitHub: github.com/openoba/erdl
A2A Discussion: RFC #2031
Contact: support@openoba.com

Posted by haoran-tang-ch · OpenOBA

The AI Agent Protocol Stack is Missing a Layer — Here's the Third One

Tang Haoran — Sat, 04 Jul 2026 13:07:10 +0000

MCP connects agents to tools. A2A connects agents to agents. Both are excellent. Neither answers one question:

How do agents know what they're ALLOWED to do?

Cisco Research proposed a layered agent protocol architecture in 2025 (arXiv:2511.19699):

L8 (Agent Communication Layer) → Standardized message envelopes, speech-act performatives
L9 (Agent Semantic Negotiation Layer) → "does not exist today" (direct quote from the paper)

L8 is being implemented by MCP and A2A. L9 had no answer — until now.

ERDL: The Third Layer

Entity-Rule Definition Language is an open standard for agent behavioral rules. It sits between the agent's reasoning and its tool execution:

MCP → tools. A2A → communication. ERDL → rules.

Three layers. One complete agent protocol stack.

What ERDL brings

Capability	Status
11 operators including `within` and `rate`	Complete
16 deterministic action types	Complete
Execution Rings (Ring 0–3)	Complete
Guardian Agent model	Complete
SafeExpr engine (zero code injection)	Complete
Hot reload without restart	Complete
Structured audit trails	Complete
Snapshot + rollback	Complete

How it works


yaml
rule: block_privileged_commands
priority: 0
description: "Block dangerous shell commands"
when:
  field: "tool.command"
  operator: in
  value: ["sudo", "rm -rf"]
then:
  action: BLOCK
  message: "Privileged commands require human approval"
**One YAML file. When an agent tries to call a blocked tool, ERDL's Action Guard intercepts the call before it reaches the tool. Not after. Not as a suggestion. The engine evaluates conditions, matches rules, and returns a deterministic verdict.

The agent never touches sudo. Period.

OWASP Top 10 (2026) — How ERDL Maps
OWASP Risk  ERDL Mitigation
R1: Excessive Agency    Time windows + rate limiting
R2: Prompt Injection    Guard rules run before LLM output
R3: Supply Chain    Agent BOM with rule audit trail
R4: Data Leakage    Field-level BLOCK rules
R5: Overreliance    Guardian Agent oversight
R6: Multi-Agent Collusion   Cross-agent audit chains
EU AI Act (effective August 2, 2026)
ERDL maps directly to transparency and human oversight requirements. Audit records export to OCSF format, compatible with SIEM systems.

The Stack is Now Complete
MCP — tools (Anthropic, Linux Foundation)
A2A — communication (Google, Linux Foundation)
ERDL — rules (OpenOBA, MIT)
Landing Page: openoba.github.io/erdl-landing
Spec v1.0 (English): spec/index-en.md
GitHub: github.com/OpenOBA/ERDL
MIT License. RFC open.

What do you think — does the agent protocol stack need a dedicated rules layer? Let's discuss.**