DEV Community: nujovich

A plugin for Observability + Budget Guardrails built with Hermes Agent

nujovich — Sun, 31 May 2026 16:38:17 +0000

Challenge Entry for the Hermes Agent Challenge

🚀 What Problem Does This Solve?

AI agent deployments often suffer from two critical blind spots:

Cost visibility — you discover a $500 OpenAI bill at the end of the month with no clue which cron jobs or sessions caused it
Budget control — runaway loops or expensive model choices can drain your account before you notice

hermes-telemetry solves both by giving you real-time observability and automatic budget enforcement for Hermes Agent.

🎯 Why This Plugin Matters

Every production AI system needs observability and cost control. This isn't just a nice-to-have — it's essential infrastructure.

Before this plugin, Hermes users had no way to:

Track spending per cron job or messaging platform
Set budget limits that actually pause runaway processes
Compare cost efficiency between different models
Get real-time cost alerts before hitting billing limits

Now they can manage their AI spend like a modern SaaS — with dashboards, alerts, and automatic circuit breakers.

✨ Key Features

Real Usage Data (Not Estimates)

Captures actual token counts and costs returned by providers like OpenRouter, OpenAI, and Anthropic. No guesswork.

Multi-Level Budget Enforcement

Soft warnings at 80% of budget
Hard tool blocks at 100% (prevents new API calls)
Cron job pauses for automated workflows
Scope-specific limits (global, per-cron-job, per-platform)

Rich Analytics via Slash Commands

/stats — session performance, tool usage, cost breakdowns
/stats cron week — cron job cost comparison across time
/stats providers — which providers return real vs estimated data
/budget — current spending vs limits with visual indicators

Zero Model Awareness

Pure observability layer — captures everything through hooks without affecting model behavior or adding latency.

📊 Screenshots

Session Analytics (`/stats`)

Budget Status (`/budget`)

Cron Job Cost Comparison (`/stats cron week`)

Provider Analysis (`/stats providers`)

🧪 Proof of Concept: Real Data

I tested the plugin with three different models to validate pricing accuracy and budget enforcement:

Model	Cost per Test Run	Budget Behavior
`owl-alpha` (free)	$0.00	No limits triggered
`claude-sonnet-4-6`	$0.31	Soft warning at $0.001 limit
`claude-opus-4-7`	$2.23	Hard pause enforced ✅

Budget enforcement works. When I set a $0.001 daily limit and ran a cron job, it correctly paused at $0.18 spending. When I raised the limit to $2.00, jobs resumed normally.

Real provider data. OpenRouter returned actual token counts (Est% = 0%), not estimates. The plugin correctly captured and priced these.

🏗️ Technical Implementation

Hook Pipeline Architecture

on_session_start → pre_api_request → ★ post_api_request → post_tool_call
                                     │
                                     ▼
                               [capture usage]
                                     │
                                     ▼
pre_llm_call (budget check) → pre_tool_call (tool gate) → SQLite storage

Data Layer

SQLite WAL database — efficient, local, no external deps
Custom pricing.yaml — override provider rates for accurate cost calculation
budget.yaml configuration — flexible limits (daily/monthly, global/scoped)
94 comprehensive tests — full coverage of edge cases and enforcement logic

Provider Compatibility

Works with any provider that follows the Hermes Agent provider interface:

✅ OpenRouter (tested with real usage data)
✅ OpenAI (pricing table included)
✅ Anthropic (pricing table included)
✅ Custom providers (via pricing.yaml overrides)

🎯 Production Ready

This isn't a demo — it's production infrastructure. The plugin includes:

Error handling — graceful fallbacks when providers return no usage data
Hot-reload — update budgets via /budget set without restart
Concurrent safety — SQLite WAL mode handles multiple sessions
Memory efficiency — hook pipeline adds negligible overhead
Comprehensive logging — debug telemetry issues with structured logs

🚀 Installation & Usage

1. Install

cd ~/.hermes/plugins
git clone https://github.com/nujovich/hermes-telemetry.git
# Add 'hermes-telemetry' to plugins.enabled in config.yaml
# Restart gateway: hermes gateway restart

2. Configure Budget (Optional)

# Set daily budget
hermes> /budget set global daily 5.00

# Check status  
hermes> /budget

3. Monitor Usage

# Session stats
hermes> /stats

# Cron job breakdown
hermes> /stats cron week

# Provider analysis
hermes> /stats providers

That's it. The plugin immediately starts capturing usage data for all sessions and cron jobs.

🏆 Why this is a win-win

This plugin solves a universal need in AI systems — cost visibility and control. Every Hermes Agent deployment, from personal automation to enterprise cron jobs, benefits from this infrastructure.

It's not just useful, it's essential. Without budget controls, a misconfigured cron job with an expensive model can cost hundreds of dollars overnight. This plugin prevents that.

Real-world tested. I built, deployed, and validated this with actual usage data across multiple providers and models. It's not a concept — it's working infrastructure that saves money and provides operational insight.

Community impact. This sets a standard for observability in the Hermes ecosystem. Other plugin authors can build on these patterns, and users get immediate operational confidence.

📋 Technical Details

Repository: https://github.com/nujovich/hermes-telemetry
Documentation: Complete README with architecture, configuration, and troubleshooting
Tests: 94 passing tests covering all major functionality
License: MIT
Dependencies: PyYAML only (for config files)

👨‍💻 About the Author

I'm Nadia Ujovich.

I understand the operational challenges of running AI systems at scale, and I built this plugin to solve the observability gap I see in every deployment.

This plugin makes Hermes Agent production-ready for cost-conscious deployments. It's the infrastructure piece that every serious AI system needs, but few teams build themselves.

Give your agents the observability they deserve. Try hermes-telemetry today.

Made with ☕ for the Hermes Agent ecosystem

Building an Autonomous MCP Lead Generation System with Hermes Agent

nujovich — Mon, 25 May 2026 21:43:58 +0000

How I built a daily research pipeline with Hermes Agent that finds SMEs with legacy APIs, scores them on MCP readiness, and pre-audits their public API surface — all running autonomously on my own infrastructure.

The Problem: Finding Companies That Need MCP

MCP (Model Context Protocol) is the emerging standard for connecting legacy APIs, databases, and manual workflows to AI agents. But finding companies that actually need MCP integration — and that have the budget and readiness to act — is a manual, time-consuming sales process.

I'm constantly thinking about where to focus my help and services. My problem was simple: **I needed a way to find prospects daily without spending hours on LinkedIn and Reddit.

Enter Hermes Agent.

What I Built

A fully autonomous MCP Lead Generation & Pre-Audit System that runs daily on my own infrastructure:

Every day at 10:00 AM, Hermes Agent researches LinkedIn, Reddit, GitHub, Product Hunt, and tech blogs looking for SMEs that show signs of systems fragmentation
It scores each prospect against a 13-point MCP Readiness Checklist
For high-scoring prospects, it pre-audits their public API documentation to identify MCP integration opportunities
It emails me a structured report with company name, contact info, score, and specific MCP tools that could be built

The result is a daily pipeline of warm, pre-vetted leads — each one with a concrete MCP action plan ready to go.

The MCP Readiness Checklist

I designed a 13-question scoring system based on four dimensions:

1. Systems Fragmentation (3 pts)

Uses 5+ SaaS tools that don't talk to each other
Has proprietary/legacy internal APIs or siloed databases
Uses Zapier/Make for integrations (bonus if complaining about cost)

2. Manual Work & Friction (4 pts)

Someone copies/pastes data between systems regularly
Workflows that "should be automated" but aren't
Data conflicts between systems
A "data person" doing undocumented manual magic

3. Scale & Growth (3 pts)

Each new client adds manual work hours
Wants automation but no devs available
Planning 50%+ growth in the next 12 months

4. Budget & Readiness (3 pts)

Already spending €500+/month on SaaS
Has budget for consulting (€20-40k)
Has a CTO or VP Ops champion

8+ points = High priority. 5-7 = Warm. 0-4 = Skip.

How Hermes Agent Runs the Pipeline

The system is a Hermes Agent cron job — a self-contained prompt that the agent executes on schedule. Here's how it works:

Phase 1: Parallel Research

Hermes Agent launches parallel research tasks across multiple sources:

delegate_task(tasks=[
  {"goal": "Search LinkedIn for operations managers
            at SMEs mentioning 'tool sprawl'..."},
  {"goal": "Scan Reddit r/SaaS, r/smallbusiness
            for integration pain points..."},
  {"goal": "Check ProductHunt and tech blogs
            for companies with obvious API needs..."},
])

Phase 2: Scoring & Analysis

Each prospect is evaluated against the 13-point checklist. The agent reads company websites, checks their API docs, looks for GitHub repos, and builds a complete profile.

Phase 3: Pre-Audit (The Secret Weapon)

For companies with public API documentation or GitHub repos, the agent performs a deeper analysis:

API surface mapping — What endpoints exist? Auth model? Data formats?
MCP tool identification — Which endpoints would make good MCP tools?
Complexity estimation — Easy / Medium / Hard to build MCP wrappers
Security considerations — What needs attention before building

This transforms a cold lead into a warm prospect with a concrete MCP proposal ready to present.

Real Results: Two Pre-Audited Companies

On its first run, the system found and pre-audited two LATAM tech companies:

1. Cobre (Fintech, Colombia) — Score: 9/13

Signal: API-first payments platform with fragmented client systems.

The agent found their API documentation, analyzed ~50 REST endpoints across 18 categories (accounts, payments, FX, checkouts), and identified 5 specific MCP tools that could be built:

cobre_crear_contraparte — Register beneficiaries by country/rail
cobre_ejecutar_pago_local — Execute SPEI/ACH/Bre-B payments
cobre_crear_cotizacion_fx — Get FX quotes for cross-border
cobre_ejecutar_pago_internacional — Execute international payments
cobre_consultar_estado_pago — Track payment status

Key finding: Cobre already has an MCP server via Apidog — but a custom MCP server would be more powerful and fill their SDK gap (no Python/Node.js SDKs exist).

2. Truora (Identity Verification, Colombia) — Score: 7/13

Signal: API-first identity verification with ~120 endpoints across 5 API families.

The agent mapped their entire API surface: Checks API (background checks), Digital Identity (document/facial verification), WhatsApp Engagement (chatbots), Business Rules Engine, and Validator Suite. Suggested 5 MCP tools:

truora_check_background — Create background checks per country
truora_get_check_result — Get verification results with scores
truora_create_identity_process — Start digital identity validation
truora_send_whatsapp_message — Send WhatsApp via templates
truora_list_checks — Query verification history

Key finding: Truora only has iOS SDKs — no Python, Node.js, or Go tooling. An MCP server would fill a critical gap in their developer ecosystem.

The Architecture

The heart of the system is a self-contained Hermes Agent cron job prompt. Here's the structure:

# Create the cron job
hermes cron create \
  --schedule "0 10 * * *" \
  --prompt "research prospects..."

The prompt instructs Hermes Agent to:

Research — Use browser and web tools to find SMEs across LinkedIn, Reddit, GitHub, and blogs
Score — Apply the 13-point MCP Readiness Checklist
Pre-audit — For READY TO AUDIT companies, visit their API docs and GitHub repos
Compile — Format everything into a structured plain-text report
Deliver — Email via himalaya template send

cat << 'MAILDELIM' | himalaya template send
From: me@email.com
To: me@email.com
Subject: 🔌 MCP Prospects — [DATE]

═══════════════════════════════════════
1. AUDIT NOW — Company Name
   🔧 PRE-AUDIT: READY TO AUDIT
   API Docs: https://docs.example.com/
   ...
   🛠️ Suggested MCP tools:
   • tool_name — what it does
═══════════════════════════════════════
MAILDELIM

Why Hermes Agent Was the Right Tool

I chose Hermes Agent for three reasons:

Autonomous research — The agent can browse, read, analyze, and make decisions without me in the loop. It's not a script — it reasons about what it finds.
Tool ecosystem — Hermes Agent's built-in browser, web search, and terminal tools let it navigate real websites, parse API docs, and interact with GitHub — just like a human researcher would.
Self-hosted — This runs on my own infrastructure. No API costs, no data leaks to third parties. The prospect data stays mine.
Cron jobs — The built-in scheduler means the system runs itself. I wake up to a fresh lead report every morning.

What's Next

This system is already running daily. My roadmap includes:

Building the actual MCP servers — Taking the pre-audit output and building the MCP servers for the highest-priority prospects
Adding more research sources — AngelList, Crunchbase, and Y Combinator companies
Automated outreach — Integrating with LinkedIn DM or email to send personalized MCP proposals
Refining the scoring model — Learning from which leads convert and adjusting the checklist weights

Try It Yourself

If you're running Hermes Agent, you can build something similar in a few hours:

Define your scoring criteria
Write a research prompt (the agent handles the execution)
Create a daily cron job
Sit back and watch the leads roll in

The hardest part isn't the technology — it's knowing what signals to look for. That's where your domain expertise comes in.

Built with Hermes Agent — the open-source agentic system that runs on your own infrastructure.

🔗 **Full code and prompts:* github.com/nujovich/hermes-mcp-lead-gen*

Follow me on LinkedIn

hermesagentchallenge #devchallenge #agents #mcp #opensource