Why most AI fails at IDOR (and how AMAS fixes it with causal reasoning)

mr_miou — Mon, 25 May 2026 15:42:49 +0000

The problem no one talks about

Large language models are great at pattern matching.

Show them enough “vulnerable” examples, and they learn the words – not the reason.

That’s why they struggle with logical vulnerabilities like IDOR, privilege escalation, and workflow bypasses.

These attacks depend on who the actor is, who owns the resource, and where the trust boundary sits – not just on the presence of keywords like “authorization”.

Enter AMAS

AMAS (AI Multi‑Agent Security Analysis System) is a security reasoning substrate.

It doesn’t teach the model to memorise patterns.

It teaches it to understand causality.

Instead of “this is an IDOR”, AMAS models:

Identity & Session – who is the actor? who is the owner?
Enforcement failures – which security control was missing or broken?
Temporal transitions – what changed before and after the request?
Causal graphs – how does a chain of events lead to a breach?

How it works (in a nutshell)

Ingest real‑world CVEs – streaming NVD feeds, filter by relevance (access control, auth, business logic).
Convert each CVE into a causal graph – extract actors, resources, trust boundaries, and missing controls.
Mix with high‑quality synthetic data – generated by a semantic mutation engine that changes mechanics (infrastructure, auth model, async behaviour), not just wording.
Quality control – deduplication, domain coherence checks, repetition validation.
Export – train.jsonl ready for fine‑tuning, plus detailed manifests.

Everything is deterministic (same seed → same dataset) and scalable (handles 200k+ CVEs).

Results (so far)

After fine‑tuning a Mistral‑7B model on AMAS‑generated data:

✅ Generalisation – same vulnerability, different wording → correctly identified.
✅ Negative tests – secure login requests → no false alarms.
✅ Domain shift – secure scenarios in fintech, e‑commerce, healthcare → all classified correctly.

Duplication dropped from 52% to <10%, and domain corruption was eliminated in the latest pipeline.

Next steps

Open‑source release (GitHub) – complete pipeline, docs, examples.
Research paper (causal reasoning + identity‑aware training).
Pre‑trained model weights.

Why share this now?

Because the industry keeps building pattern‑matching scanners.

We need reasoning engines.

If you’re into AI security, causal ML, or synthetic data – let’s connect.

I’ll drop the repo link here as soon as it’s public.

Comments and questions are very welcome. Let’s make AI actually understand security.