Tenable Is Reviewing Community Agent Components. Signing the Index Is the Other Half.

Tenable and OpenAI announced the CyberAgents Exchange AI Inspector this week: a security review process for AI agents, skills, MCP servers and multi-agent playbooks submitted to Tenable's CyberAgents Exchange. It pairs OpenAI's GPT cyber models for frontier assessment with Tenable One AI Exposure for skills inspection and a review pass by Tenable researchers. The Exchange opened in August as an open-source registry and already lists more than 100 community-submitted components after a SWARM build event at Black Hat USA. Availability is expected this month. Tenable CPO Eric Doerr framed it plainly: "Agentic AI will only reach its potential in the enterprise if security teams can trust the components being introduced into their environments."

That is the right problem. Reviewing what a component does is the expensive half, and it is good to see someone spending real researcher time on it rather than shipping a scanner and calling it governance.

The cheap half is the one registries skip, and it is the one that makes a review mean anything.

A review is a claim about specific bytes

Reviewing a skill produces a verdict attached to an artifact at a moment in time. For that verdict to survive the trip to an operator's machine, the registry has to pin the bytes the reviewer saw, sign the index that carries the pin, and the client has to refuse to install anything that fails either check. Without that, you have published an opinion about one tarball while users download another.

This is boring supply-chain plumbing, and it is entirely mechanical, which is why it should land before the interesting work rather than after it.

What our catalogue pins

The Pilot app store index lives in the open at catalogue/catalogue.json in the Pilot Protocol repo. As of current main it carries 31 entries, 12 published by us and 15 by outside vendors including Sixtyfour, Miren, Telepat, Bowmark AI, AgentPhone, InsForge and Upfile. Four entries carry no vendor string, one of which is a tombstone kept for install continuity rather than a listing.

Four things are pinned per entry:

1. bundle_sha256 over the tarball, plus a per-platform bundles map with its own hash for each os/arch (26 of the 31 entries have one).
2. metadata_sha256 over the app's detail document, so a substituted store listing is rejected and pilotctl appstore view falls back to the teaser.
3. publisher, the app's ed25519 key. All 31 entries carry one and all 31 are distinct.
4. The index itself, via a detached base64 ed25519 signature in catalogue.json.sig, verified against the key compiled into internal/catalogtrust.

That last one is checkable by anyone in a handful of lines of Go. Clone the repo, read publicKeyB64 out of internal/catalogtrust/catalogtrust.go, and run ed25519.Verify over the committed catalogue.json bytes against the signature file. It returns true, and flipping a single byte of the JSON returns false. pilotctl does exactly this fetch-and-verify before trusting any entry, and refuses an unsigned, missing-signature or tampered catalogue outright.

The publisher pin keeps working after install. In cmd/daemon/main.go the daemon hands the app-store service catalogtrust.PublicKey() along with the per-app publisher pins read out of the signature-verified catalogue, on a two-second rescan interval, with the comment right there in the file stating the rule: an unpinned app fails closed.

What our catalogue does not do

It carries no security verdict. The catalogueEntry struct in cmd/pilotctl/appstore_catalogue.go has exactly fifteen fields, and none of them is a review, an audit result, or a scan status. The reviews slot in the per-app metadata schema is reserved and null everywhere: appstore_metadata.go parses it and never writes it. The pilotctl review command that does exist takes a 1-to-5 rating and free text from users, routes it through consent-gated telemetry, and makes no security claim at all.

So Tenable is building the layer we left as a reserved null field, and provenance is the layer most agent registries have not built. Neither substitutes for the other. A signed bundle from an unreviewed publisher is still a bundle nobody read, and a reviewed component delivered over an unsigned index is a verdict about bytes nobody is checking.

Then it runs

Both halves stop at the install boundary. After that, the component is a process with network reach, and the question moves to who is calling it and what it did.

Our request-envelope design in docs/SIGNATURE-VERIFICATION.md handles the first: a node signs a canonical pilot-req-v1 string over its own address, a timestamp, a nonce, a body hash and an audience identifier. The audience field binds the signature to one recipient so it cannot be replayed elsewhere, the freshness window is 300 seconds, and unknown node, reaped node, bad signature and expired key all return the same {"valid": false} so the endpoint is not an enumeration oracle over the address space.

The second question is more mundane and gets ignored more often. When a newly installed component runs a migration, a build, or a long tool call, someone usually wants to watch it without being handed a shell on the box. That is what shell.online does: shell --read-only python train.py turns a running process into a view-only browser link whose access mode is fixed at session creation, with read-only input rejected at the Worker so no amount of poking at the page or the WebSocket frames upgrades it to interactive. The command and the PTY never leave your machine. If you want to see how the read-only path is enforced rather than take our word for it, the CLI and Worker sources are public.

The same distribution discipline shows up there: every shell.online release publishes a canonical SHA256SUMS manifest, and the installer verifies the binary checksum before it writes anything and prints the digest it verified.

Review the component. Pin the bytes. Verify the caller. Watch the process. The middle two are almost free, and today's announcement is a good reminder that the expensive one only pays off if you did them.