I built a public, cryptographically signed test registry for AI agents (student project)

Hi everyone! 👋

I’m a final-year BTech CSE student.

The Problem I Noticed
As AI agents start doing real tasks (like sending emails or accessing files), we currently just have to trust the developer's word that the agent is safe from being tricked or hijacked. There is no simple, public way to verify an agent's actual safety record.

My Attempt at a Solution: A2A-G
I built a small, open-source project called A2A-G to explore if we could make agent safety public and testable.

Here is in short how it works:

1. Grey Badge: An unverified self-report of what the agent claims to access. (To be clear: this is just a claim, not verified by us).
2. The Choice: After the Grey Badge is issued, the owner is simply asked if they want to earn a Blue Badge. Testing is completely optional.
3. Mock Sandbox: If they choose to test, they must provide a safe, consented mock sandbox environment. We never test live production systems.
4. The Test: We send 18 OWASP hijacking prompts, 3x each, randomized. An LLM judges if it blocked or complied.
5. Blue Badge: 80%+ blocks earns a Blue Badge. Failures are published openly too.
6. The Proof: Every result is cryptographically signed (Ed25519) and hash-chained, so the history cannot be faked or hidden.

Honest Limitations
I want to be completely transparent: this is an early, solo-built attempt. The testing methods aren't perfect yet, and it doesn't solve all AI security problems. I simply built it to learn if public, verifiable safety records are practical.

I’d Love Your Feedback

I would deeply appreciate any honest technical feedback, opinions, or critique on the mock sandbox approach.

🔗 Live Site: https://a2ag-website.vercel.app/
💻 GitHub: https://github.com/Annora998/a2ag-website

Thank you for reading!

Comments

[Pushpendra Agrawal]

the grey/blue badge split is the right call, most of these projects fail by forcing every agent through the same test before anyone trusts the signal. curious what happens when an agent passes at 80% then a dependency update changes its behavior, does the blue badge get revoked automatically or does it just sit there stale until someone notices?

[a998]

To be honest, right now the Blue badge doesnt auto revoke if a dependency updates. It just waits for the 90-day expiry, so yeah, it can definitely sit stale in the meantime.

I'd love to add webhook triggers later.
But doing true continuous monitoring means running the LLM judge in the background constantly. Whether it's paying for API calls or running local models, the compute costs add up really fast.
Since I'm just a solo student building this on my own, I don't really have the budget to cover those ongoing background costs right now. That's why I kept it to on demand tests for this version.