I built a system to catch unreliable AI agents. In my own evaluation, it missed the worst one.

Ali Zahid Raja — Mon, 17 Aug 2026 10:04:10 +0000

My framework exists to identify unreliable transmitters in a multi-agent pipeline. In my own evaluation, the grade-recovery loop missed the highest-fault narrator in the set.

The single worst actor. The exact thing the system is for.

I put that in the abstract of the paper rather than in an appendix, and this post is about why — plus the two other results I had to label inconclusive, and what I think all of it says about how we evaluate systems like this.

The problem

Modern multi-agent knowledge systems don't retrieve knowledge so much as accumulate it. A claim passes through chains of autonomous transformations: one agent retrieves, another summarizes, a third synthesizes, a fourth writes the answer. By the time it reaches a user, that claim has been through four hands and the chain is gone.

We have decent tooling for recording what happened — execution traces, tool calls, evidence links. And source-reliability estimation is a mature field on its own; truth discovery and reputation systems have been around for years.

What's missing is the thing that joins them: graded, per-domain reliability for each transmitter, attached to a specific claim's transmission chain, with rules for what to do when the chain is incomplete, how to aggregate across different kinds of transformation, and where the resulting claim should be routed.

"This claim came from somewhere" isn't provenance. "This claim came through these four transmitters, this one has a poor record on this specific domain, the chain is missing a step, therefore quarantine" is provenance.

Where the method comes from

Classical Islamic hadith science faced a structurally identical problem and worked on it for centuries: how do you decide whether knowledge transmitted through a chain of human narrators should be accepted?

What they built is, in engineering terms, remarkably close to what we need:

Isnad — a complete transmission chain attached to every single claim. Not optional metadata. A claim without a chain is a different class of object.
Rijal — systematic, documented grading of each narrator's integrity and precision. Not reputation-by-vibes. Criteria, applied consistently, recorded, contestable.
Weakest-link evaluation — a chain is graded by its worst member, not its average. Averaging hides the bad node, which is precisely the node you care about.
Corroboration through independent chains — multiple separate routes to the same claim strengthen it.
Matn criticism — the content is evaluated independently of the chain's quality. A perfect chain carrying an absurd claim is still an absurd claim.

That last one is the piece most modern provenance work collapses. Chain quality and content quality are different axes, and if you fuse them into one score you lose the ability to distinguish "trustworthy source, wrong claim" from "sketchy source, correct claim." Those need different handling.

So the paper transfers that methodology: a formal mapping from hadith-science concepts to multi-agent pipelines, a relational schema for claim chains and a graded narrator registry, a decision matrix combining chain grade with content criticism, and serve/review/quarantine routing on the output.

The evaluation

20,000 claims extracted from real physics textbooks. Not synthetic data, not a toy set.

Two things held up. Weakest-link quarantine worked — grading a chain by its worst transmitter correctly caught claims that averaging would have waved through. And independent-chain corroboration worked — claims reached by genuinely separate routes were more reliable, as the theory predicts.

One thing partially failed, and it's the interesting one.

The grade-recovery loop is supposed to recover each transmitter's reliability grade from observed behaviour — you watch what an agent does over time and back out how much it should be trusted. In the evaluation, it missed the highest-fault narrator.

Think about what kind of failure that is. It isn't uniform degradation, where the system is a bit noisy everywhere and you discount accordingly. It's a system that behaves plausibly across the distribution and then fails on the extreme case that it exists to catch. That's worse than a system that fails visibly, because you'd trust it. You'd look at the grades, see nothing alarming, and ship.

Two analyses came back inconclusive. The one worth naming is a matched-coverage comparison against a reference content critic — the framework couldn't reach comparable coverage, so the comparison couldn't be made fairly. Not a positive result, not a negative one. Genuinely undetermined.

Why report any of this

Because the alternative is that somebody else finds it.

There's a version of research writing where limitations are damage to be minimized — reported at the smallest defensible volume, in the last section, in the passive voice. I understand the incentive. I also think it's a bad trade, for a reason that's purely practical rather than moral:

The failure modes are the most useful part of the paper for anyone deciding whether to build on it. A method that works on a defined slice is genuinely valuable if you know where the edges are. The same method with the edges hidden is a trap, and it becomes the next person's expensive discovery.

There's also a specific thing I'd argue for: "inconclusive" should be a normal category. Right now most work has two buckets — it worked, or it isn't mentioned. That's a lie by omission at the field level, and it means every honest negative result gets quietly reclassified as a positive one somewhere between the experiments and the abstract. Two of my analyses didn't resolve. Saying so cost me nothing except the stronger version of a claim I hadn't earned.

What generalizes to anyone building evaluation

Three things I'd take away even if you never touch provenance:

Average-case metrics hide tail failures, and tail failures are usually the point. My system is for catching the worst actor. Any metric averaged across all transmitters would have looked fine while missing exactly the case that matters. Ask what your metric does at the extreme, not at the mean.

If your ground truth and your system share a component, you're partly measuring agreement with itself. Same model on both sides, same heuristic, same person's judgement at both ends — the number that comes back measures consistency, and consistency is much easier to achieve than correctness. The diagnostic question: if my system were systematically wrong in one specific way, would this evaluation catch it? If you can't name the failure mode and point at the part of the eval that flags it, there's circularity in there somewhere.

Separate the axes you're tempted to fuse. Chain quality and content quality are different questions. One score is easier to report and strictly less useful.

The open problem

The critique I've heard most, and the one I'd most like help with, is chain independence.

Corroboration assumes that two independent chains supporting the same claim is stronger evidence than one. Reasonable — until you ask what "independent" means when every agent calls the same base model, retrieves from an overlapping corpus, and inherits the same biases from pretraining.

Two chains that look structurally separate can share a hidden root. When they do, their agreement isn't corroboration. It's correlated failure wearing corroboration's clothes — and it presents as more convincing than a single chain, not less. That's the dangerous shape.

I don't have a clean answer, and I don't think the multi-agent literature has one either. If you've worked on correlated failure in ensembles or retrieval, I'd genuinely like to talk.

Try it

pip install isnad

MIT licensed. It's a research prototype backed by a published evaluation — usable, actively developed, and honest about where it stops working.

Paper: arxiv.org/abs/2607.24117
Code: github.com/alizahidraja/isnad

Issues and disagreement welcome. Several of the open issues in that repo came from people who turned up to argue, and the project is better for all of them.

DEV Community: Ali Zahid Raja