DEV Community

Imran Siddique
Imran Siddique

Posted on

My quickstart ends in FAIL, and that is the correct answer

If you run the cMCP quickstart on your laptop, the last command prints FAIL.

I want to explain that before you hit it, because the reflex is to assume the tutorial is broken. It is not. That output is the whole argument.

What the ten minutes actually do

pip install cmcp-runtime
Enter fullscreen mode Exit fullscreen mode

MIT, on PyPI, no account, no sales call, no special hardware. You write a Cedar policy. You point an agent at a tool. You then fire an action your own policy forbids, something like a record with PII leaving on a call to Salesforce.

It returns 403 before the call is forwarded. Not after. Not logged and alerted on. The request does not reach the tool, and the decision is made at the boundary rather than inside the model that wanted to make the call. That distinction is the entire reason the project exists: a model may propose an action, but it cannot be the authority that permits it.

Then you finalise the session and get a TRACE claim, and you verify it yourself:

cmcp verify claim.json
Enter fullscreen mode Exit fullscreen mode

The part I want you to look at

In dev mode the verifier prints something close to this:

  • schema: pass
  • signature: pass
  • policy bundle hash: pass
  • tool catalog hash: pass
  • attestation freshness: pass
  • audit chain integrity: pass
  • hardware attestation: fail

Overall result: FAIL (partially_verified).

Six checks pass. One cannot run, because your laptop has no TEE, so there is no hardware measurement to compare against. And the tool refuses to round that up.

It would be trivial to make this print VERIFIED with a footnote. Most things in this space do exactly that. I think it is the single most damaging habit in the category, because it teaches operators that a green result is a green result, and then a green result arrives one day from a system that measured nothing at all.

Unmeasured is not empty. A check that could not be performed and a check that passed are different facts, and any evidence format that collapses them is worse than no evidence, because it launders an absence into an assurance. If you take one thing from the quickstart, take that, and go look at whether your own pipeline can tell the difference.

Run the same claim on real TEE hardware and the last line passes and the overall result changes. Nothing else about the record changes. That is the point: the record is honest about its own coverage in both cases.

What I actually want from you

The happy path working is not interesting. I already know the happy path works.

What I want is the edges:

  • a policy that should have fired and did not
  • a record that does not reconcile against the policy bundle it claims
  • anything that behaves differently from what the spec says it does
  • a verification result you can argue is wrong in either direction, false pass or false fail

Find one and I will publish it with the fix, or with the open issue if there is no fix yet, and I will name you if you want the credit. I have written up limits in my own spec before and I would rather do it again than find out from a customer.

There is a real one already on the table, so I am not pretending the surface is clean: TRACE verifies against the issuer's own key, which means it detects later tampering well and says much less than it sounds about whether the issuer was honest at write time. Independent verification needs an anchor outside the issuer. That is unsolved, and it is not the only thing here that is.

Ten minutes. Then tell me what broke.

https://agentrust-io.com/quickstart/?source=devto-imran

One question if you want to answer only one: did it unblock something you cannot ship today?

Top comments (0)