Your compliance tool saying you're "95% ready" is a liability, not a feature

Momodu Kamara-Kolleh — Wed, 07 Oct 2026 12:10:39 +0000

I've spent the last stretch building a SOC 2 readiness tool, and the most useful thing I learned has nothing to do with dashboards. It's about a quiet failure mode that kills deals — and why a lot of compliance tools are built to cause it.

The failure mode

Picture a seed-stage SaaS. They land a great enterprise lead. The lead's security team sends the inevitable email: "Share your SOC 2 report, or your current readiness status."

The founder opens their shiny compliance tool. It says: "You're 95% compliant." Great. They forward that.

Then the security team asks one specific question:

"What's your access-review cadence, and how do you evidence it?"

And the answer the founder trusts was generated, not grounded. The model paraphrased a control it half-remembered and invented a definition that's subtly wrong. The reviewer — who reads these every week — catches it in about five seconds.

Now the founder doesn't look 95% ready. They look like they don't understand their own posture. The deal stalls a quarter. Sometimes it dies.

That "95%" wasn't a feature. It was a liability dressed up as reassurance.

Why LLM compliance tools do this

LLMs are fluent and confident, and a compliance control is exactly the kind of precise, citeable text they paraphrase badly. Ask a general model "what does this control require?" and it will happily produce something that sounds right. Most of the time nobody checks.

But compliance has an adversarial reader built in: the buyer's security team. Their entire job is to catch the gap between what you claim and what you can evidence. An ungrounded, confident answer is the single easiest thing for them to flag.

The fix is boring: grounding

The fix isn't a smarter model or a prettier dashboard. It's a rule:

Every answer about a control must quote the real criterion, or say it doesn't have it. Never guess.

Concretely: every AI response is injected with the actual criterion text from the framework files on disk. The model's job is constrained — quote the source, or abstain. If the control isn't in the grounding set, the honest output is "I don't have that control's text," not a confident paraphrase.

control = frameworks.lookup(control_id)   # real text, from disk
if control is None:
    return f"I don't have the criterion for {control_id}."

# system prompt: answer ONLY from the provided criterion text;
# if the question goes beyond it, say so explicitly.
return model.respond(question, grounding=control.text)

It feels almost too simple. But "boring and correct" is exactly what you want in the one place where a confident mistake costs a customer their trust in you.

Why this matters more than features

Compliance tools love to compete on breadth — more integrations, more frameworks, more automation. Useful, but none of it is the thing a buyer's security team actually tests. What they test is whether your answers hold up under one specific question. An ungrounded tool fails the only exam that matters, no matter how many checkboxes it has.

The honest part

A readiness tool is not an audit. It doesn't make you SOC 2 certified — a licensed CPA firm still performs the examination and issues the report. Any tool (mine included) that implies otherwise is doing the same overpromising I just described.

I'm building this solo, from Freetown, Sierra Leone. It's called Attestlane and it's live if you want to poke at it (use fake company data — don't connect real systems just to test). But I wrote this because the grounding lesson holds whether you use my tool, a competitor's, or a spreadsheet:

If your compliance answers can't cite their source, don't send them to a security team.

If you've been through a first SOC 2, I'd genuinely like to hear the part that actually hurt.

I built a discovery layer for AI agents because mine kept getting lost

Momodu Kamara-Kolleh — Sat, 25 Jul 2026 23:54:41 +0000

Everyone's building AI agents right now. Almost nobody's building the boring layer underneath them: how do agents find each other?

I hit this in my own stack. I run a few MCP servers one for search, one for my database, one exposing my product's API. Their addresses lived in a JSON config I edited by hand. When I added a fourth, the file got messy. When one went down, I found out because something downstream broke and I debugged backwards. There was no single source of truth for what agents existed or whether they were alive.

MCP gave agents a common language. There's still no discovery layer. That felt like having HTTP without a service registry.

So I built agentreg. It's a self-hosted service registry for agents one Go binary:

agentctl serve &
agentctl register search-agent -c search -e http://localhost:3000
agentctl find search # discover by capability, not a hardcoded URL
agentctl list # everything, with live health

It health-checks every registered agent on a heartbeat, so a dead one shows up as unhealthy before it takes something down with it. That single feature has already saved me a debugging session.

A fair question I keep getting: "isn't this just service discovery, like Consul?" Close, but the query is different. Consul answers where is this service? a host and port. Agents need what can this thing do, is it healthy, and can I trust it? That's capability-first, and it's why the registry is agent-shaped rather
than micro service-shaped.

I'm being upfront about the stage: this is v0.1, I built it solo, and it does discovery + health and nothing more yet. No federation, no auth, no trust verification — though the code has a pluggable verifier seam for exactly that later. It's MIT, self-hosted, no cloud, no account.

Install:

brew install mkk2026/tap/agentreg
Repo: https://github.com/mkk2026/agentreg

If you run more than a couple of MCP servers, I'd genuinely like to know whether this solves a real annoyance for you or whether I'm a year early. The most useful thing you can tell me is whether you're still running it a week after you try it. That's the only feedback that counts.