I Built an AI Agent Risk Auditor for Agentic Repositories

James Bridges — Sun, 17 May 2026 00:53:07 +0000

GitHub repo: https://github.com/jbridges82/hermes-guard

Live demo: https://hermes-guard.vercel.app

What I Built

I built Hermes Guard, a local-first risk auditor for repositories touched by AI agents.

The core question behind the project is simple:

As AI agents start touching real repositories, who audits the agents?

Hermes Guard scans a local repository for risky AI-agent automation patterns and produces evidence-backed Markdown and JSON reports. It also includes a polished React/Vite dashboard for reviewing the scan results.

The MVP focuses on the repo-level files that often shape agent behavior:

AGENTS.md instructions.
Prompt files.
GitHub Actions workflows.
MCP server and tool permission config.
Shell scripts that run agent-generated commands.
Example environment files that normalize unsafe secret handling.

That means Hermes Guard is not trying to inspect only application logic. It is looking at the automation layer around the agent.

For example:

approval_mode: none
allow: "*"

Those two lines are not an application bug. They are an automation risk. They tell a reviewer something important about the repo's agent posture.

Demo

Live demo: https://hermes-guard.vercel.app

The dashboard presents the scan as a security review workspace:

A left panel for the scanned repo tree.
A center panel for Hermes-style activity.
A main panel for risk posture, severity counts, and finding cards.

The demo repo is intentionally synthetic and safe. It contains fake credentials only, clearly marked as fake placeholders.

Code

GitHub repo: https://github.com/jbridges82/hermes-guard

The scanner writes:

reports/hermes-guard-report.json
reports/hermes-guard-report.md

A typical finding includes:

ID and title.
Severity.
File path and line number.
Evidence snippet.
Why it matters.
Recommended fix.

That evidence-first design matters. A reviewer should not have to trust the tool. They should be able to verify the finding immediately.

My Tech Stack

React + Vite frontend.
Tailwind CSS.
Node.js scanner logic.
Rule-based risk engine.
Local filesystem scanning.
Markdown and JSON report output.
Vercel deployment for the dashboard.

How I Used Hermes Agent

Hermes Guard is Hermes-inspired and Hermes-ready, but the MVP does not claim live Hermes Agent execution.

I designed the scanner around the kinds of repository risks that matter when an agentic system can plan, use tools, and touch real code. The project includes a clean adapter boundary at:

scanner/hermesAdapter.js

Right now, that adapter clearly reports stub mode:

Hermes Agent runtime is not connected. Hermes Guard is running deterministic local rules only.

That was intentional. I wanted the submission to stay honest: deterministic local scanning works today, and a future Hermes runtime can connect through the adapter without changing the scanner/reporting pipeline.

Safe By Design

Hermes Guard is designed to be safe to run against a repository before any optional agent runtime is connected.

Hermes Guard never executes repository code.
The scanner is read-only: it walks files, reads text, applies deterministic rules, and writes reports.
The architecture is local-first, so repository contents do not need to leave the developer machine.
No cloud dependency is required for the MVP.
The included demo repo contains intentionally fake credentials only.

What It Detects

The current deterministic rule engine looks for:

Fake or exposed API-key-like strings.
ignore previous instructions prompt-injection text.
Unrestricted shell execution.
approval_mode: none.
allow: "*" wildcard access.
curl or wget piped into a shell.
GitHub Actions secrets used with broad agent automation.
MCP or tool permission config that appears overly broad.

What I Learned

The interesting part of this project was not writing regular expressions. It was deciding what an "agentic repository" actually means.

A repo is no longer just source code. It can also be a set of permissions, prompts, tools, and automation pathways. Once an agent can act on that repo, those files become part of the system's security boundary.

That boundary needs review.

What I Would Build Next

The next version of Hermes Guard would add:

SARIF output for GitHub code scanning.
Pull request diff scanning.
Rule suppressions with reviewer justification.
Live dashboard wiring to the local scanner output.
Policy profiles for strict, balanced, and advisory scans.
Real Hermes Agent integration through the existing adapter once a runtime is available.

Closing

AI agents make repositories more capable. They also make repositories more complex.

Hermes Guard is a small, local-first step toward making that complexity auditable. It gives developers a way to ask, before the agent acts:

What is this repository allowing an AI agent to do?

That is the question I think every agentic repo will need to answer.

DEV Community: James Bridges