Weir - deterministic unit tests for AI agents (no LLM)

IdoGol24 — Tue, 25 Aug 2026 20:15:04 +0000

Your agent's logs can say "success" while the run was quietly hijacked - untrusted content at step 2 steering a tool call at step 6 - and the industry's default answer is pointing a second LLM at the first one.

I built Weir to be the deterministic opposite: it reads the OpenTelemetry traces you already have and answers two questions:

How much of your agent's behavior your telemetry can actually prove
Whether a forbidden flow happened (shown as a witness path you can fail CI on)

Same input, byte-identical answer, no LLM in the loop, nothing leaves your machine.

Try it out:

pip install weir-scan && weir gauge --sample

License: Apache-2.0

IdoGol24 / weir

Weir - unit tests for your agents

Weir is a CI gate for AI agents.

It reads the OpenTelemetry traces your agent already emits and fails the build when sensitive data reaches a sink it should not reach.

Weir asks a structural question:

Did sensitive data flow through the agent to a sink it should never have reached?

Your agent already answers that question in the traces it emits. Weir reconstructs the session graph, tracks taint through it, and shows the evidence node by node.

flowchart LR
  A["traces your agent<br/>already emits"] --> B{"weir gauge"}
  B -->|"coverage too low"| C["names the exact<br/>instrumentation switch"]
  C -.->|"flip it, re-run"| B
  B -->|"coverage sufficient"| D{"weir scan"}
  D -->|"no forbidden flow"| E["exit 0"]
  D -->|"forbidden flow"| F["exit 1 + witness path<br/>n2 → n3 → n4 → n5 → n6"]

Try it in two minutes

pip install weir-scan
weir gauge your-export.jsonl   # or: weir gauge

…

View on GitHub

The roadmap is completely in the open. I'd genuinely like to know where it breaks on your exports!

DEV Community: IdoGol24

Weir - deterministic unit tests for AI agents (no LLM)

IdoGol24 / weir

Weir - unit tests for your agents

Weir - unit tests for your agents

Did sensitive data flow through the agent to a sink it should never have reached?

Try it in two minutes