DEV Community

MLisanti_Dev
MLisanti_Dev

Posted on

Repo-audit-agent — AI-powered GitHub repository auditor built with Hermes Agent

#hermesagentchallenge #devchallenge #agents

This is a submission for the Hermes Agent Challenge: Build With Hermes Agent

What I Built

repo-audit-agent helps developers perform fast first-pass repository reviews using Hermes Agent.

Point it at a public GitHub repository and Hermes Agent performs a first-pass review of the visible repository content, then generates a structured Markdown report with tech stack detection, code quality observations, a risk register, and an improvement roadmap.

A faster starting point for human review. No copy-pasting code into a chat window. Just one command:

python3 audit.py https://github.com/NousResearch/hermes-agent
Enter fullscreen mode Exit fullscreen mode

And Hermes Agent does the rest.

The Problem It Solves

Every developer knows the situation: you inherit a codebase, evaluate an open source dependency, or onboard a new project. Before you can do anything useful, you need a quick technical assessment — what's the stack, what are the risks, what needs fixing first.

Traditionally this takes hours. With repo-audit-agent and Hermes Agent, it takes minutes.

Sample Output

Here's a real report generated by Hermes Agent on the hermes-agent repository itself:

# Technical Audit Report: NousResearch/hermes-agent

Generated: 2026-05-29 21:07 UTC
Tool: repo-audit-agent v1.0.0 powered by Hermes Agent

## Executive Summary

The hermes-agent repository is a substantial, well-structured project 
primarily developed in Python with significant TypeScript contributions, 
indicating a complex application with a web-based interface...

## Tech Stack

- Primary: Python 46.7%, TypeScript 7.9%, TSX 2.0%
- Config: YAML, TOML, JSON, Docker
- Infrastructure: Bash, Systemd, Makefile

## Code Quality Score: 7/10

Strong documentation coverage (23.7% comment ratio) and a well-developed 
feature set. Areas for improvement include unknown/duplicate file categories 
and cross-language integration complexity.

## Risk Register (Top 5)

| # | Risk | Severity |
|---|------|----------|
| 1 | Dependency sprawl across 5+ languages | Medium |
| 2 | Documentation drift risk | Medium |
| 3 | Performance bottlenecks in Python core | Medium |
| 4 | Cross-language integration complexity | Medium |
| 5 | Security vulnerabilities in external tools | High |

## Improvement Roadmap (Top 5)

1. Automated dependency scanning across all ecosystems
2. Performance profiling for critical Python components
3. Refactor unknown/duplicate file categories
4. Comprehensive E2E testing across frontend and backend
5. Streamline CI/CD pipeline and build process
Enter fullscreen mode Exit fullscreen mode

Real output from a real repository, generated by Hermes Agent and intended as a starting point for human technical review.

Demo

Running an Audit 

# Install Hermes Agent
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

# Configure your LLM (Gemini free tier works great)
hermes setup  # Select Google AI Studio

# Clone and run
git clone https://github.com/MaurizioLisanti/repo-audit-agent
cd repo-audit-agent
python3 audit.py https://github.com/NousResearch/hermes-agent
Enter fullscreen mode Exit fullscreen mode

What Happens Under the Hood

When you run audit.py, Hermes Agent:

  1. Receives a detailed audit prompt with the target repository URL
  2. Plans its analysis strategy autonomously
  3. Uses its browser tool to navigate to the GitHub repository
  4. Fetches the README, file structure, and codebase information
  5. Reasons about code quality, risks, and improvement priorities
  6. Generates a structured Markdown report

The report is saved to ./reports/audit_<repo>_<timestamp>.md.

Architecture 

repo-audit-agent
│
├── audit.py                  ← CLI entry point
│   ├── build_audit_prompt()  ← Instructs Hermes Agent
│   ├── run_hermes_audit()    ← Invokes Hermes Agent
│   └── save_report()         ← Saves Markdown report
│
└── reports/                  ← Generated audit reports
Enter fullscreen mode Exit fullscreen mode

Code

GitHub Repository: https://github.com/MaurizioLisanti/repo-audit-agent

The core of the tool is how it communicates with Hermes Agent:

def run_hermes_audit(repo_url: str, max_turns: int = 15) -> str:
    """
    Invoke Hermes Agent to perform the repository audit.

    Hermes Agent uses its agentic capabilities (web browsing, tool use,
    multi-step reasoning) to fetch and analyze the repository.
    """
    prompt = build_audit_prompt(repo_url, owner, repo_name)

    result = subprocess.run(
        [
            "hermes", "chat",
            "--query", prompt,
            "--quiet",
            "--max-turns", str(max_turns),
        ],
        capture_output=True,
        text=True,
        timeout=300,
    )

    return result.stdout
Enter fullscreen mode Exit fullscreen mode

The key insight: Hermes Agent's --max-turns parameter controls how many tool-calling iterations it can use. For large repositories, increasing this gives Hermes Agent more room to explore and analyze.

My Tech Stack

  • Hermes Agent — core agentic engine (planning, tool use, reasoning)
  • Python 3.11 — CLI wrapper and report handling
  • Google Gemini 2.5 Flash — LLM provider via Hermes Agent
  • AWS EC2 — Ubuntu server on a t2.micro free-tier instance
  • GitHub — repository hosting and output

How I Used Hermes Agent

Hermes Agent is not a wrapper around this tool — it IS the tool.

Agentic Capabilities Used

1. Multi-step Planning
When given the audit prompt, Hermes Agent doesn't just send one API call. It breaks the task into sub-steps: fetch the repository, read the README, analyze the file structure, identify the tech stack, assess quality, generate the report. This planning happens autonomously.

2. Tool Use — Web Browsing
Hermes Agent uses its built-in browser tool to navigate to the GitHub repository URL and fetch real content. It reads the actual repository structure, not a cached or synthetic version.

3. Multi-step Reasoning
Hermes Agent synthesizes what it finds — file counts, language percentages, documentation coverage — into a coherent technical assessment with a justified quality score and prioritized recommendations.

4. Structured Output Generation
The prompt instructs Hermes Agent to produce output in a specific Markdown structure. Hermes Agent follows the template reliably, making the output machine-readable and consistent across different repositories.

Why Hermes Agent Was the Right Fit

I chose Hermes Agent for three reasons:

  • Open source and self-hosted agent runtime: I run Hermes Agent on my own AWS EC2 instance, while LLM calls are routed through the configured provider. In this demo I used Google Gemini via Hermes Agent.
  • Real tool use: Hermes Agent actually browses the repository. This is not a prompt that asks a language model to "imagine" analyzing a repo — Hermes fetches real data.
  • Composable: The hermes chat -q interface makes it trivial to integrate Hermes Agent into any Python script or pipeline.

Real-World Application

I built this tool for a real need: I maintain several Italian public-sector repositories (fatturapa-mcp-server, sdi-ops-monitor, conto-termico-gse, GaraAI) and needed a fast way to audit their technical health before client presentations.

repo-audit-agent + Hermes Agent gives me a structured first-pass technical review in minutes, which I can then validate and refine manually.

Limitations

repo-audit-agent generates AI-assisted first-pass technical reviews. It does not replace manual code review, security testing, dependency scanning, or production readiness assessment. The generated findings should be validated by a human engineer before being used for business or security decisions.

Repository: https://github.com/MaurizioLisanti/repo-audit-agent

Built with Hermes Agent by Nous Research

Top comments (0)