DEV Community: Orvi Das

Hermes Agent ran overnight and I woke up to a $47 bill — so I built a kill-switch

Orvi Das — Tue, 26 May 2026 21:05:23 +0000

This is a submission for the Hermes Agent Challenge

What I built

It was a Tuesday. I gave Hermes Agent a research task before bed: "analyse the top open-source agent frameworks and write a comparison report." Reasonable task. Maybe 10 minutes of work. I'd check the output in the morning.

I woke up to a $47 bill and a 34-page report that no one asked for.

Hermes had hit a tricky subtask around 2am, retried with different approaches, gone deeper on each one, and kept going, because that's what it's supposed to do. Autonomous agent. Autonomy is the feature. The problem is that autonomy doesn't come with a receipt until after you've already paid.

I spent that morning looking for a way to give Hermes a hard spending limit. Not a dashboard alert at $40 that I'd miss while sleeping. A hard stop that fires before the API call, not after. I didn't find one, so I built it.

baar-core is a budget-aware proxy that sits between Hermes and the real LLM providers. Every call Hermes makes goes through a kill-switch first. When a call would push spend past the cap, it gets 402 Payment Required. The provider is never contacted. Cost: $0.00.

from baar.integrations.hermes import BaarHermesSession

with BaarHermesSession(budget=1.00) as session:
    reply = session.run_task("Research the top 5 open-source agent frameworks")
    print(reply)
    print(f"Spent ${session.spent:.4f} of $1.00")

# It cannot spend $1.01. Not $1.005. $1.00 is the ceiling.

Demo

Code

GitHub: github.com/orvi2014/Baar-Core

pip install baar-core[vercel] hermes-agent

Tech stack

Component	Role
Python 3.10+	Core library
Hermes Agent	The agentic runtime being budget-capped
LiteLLM	Unified provider interface + live pricing data
FastAPI + uvicorn	Local OpenAI-compatible proxy server
SQLite (WAL mode)	Persistent spend store, concurrent-safe
pytest	606 tests, all passing

How I used Hermes Agent

Hermes doesn't stop. It plans, tool-calls, reflects, retries until the task is done or you kill the process. That's the whole point of it, and also what caused the $47 bill.

I couldn't change how Hermes works internally, but it lets you point its provider config at any OpenAI-compatible endpoint. So I built one: a local proxy that speaks the OpenAI API and intercepts every LLM call before it leaves the machine.

BaarHermesSession(budget=1.00)
  ├── BaarHermesProxy.start()    ← uvicorn on 127.0.0.1:8080, daemon thread
  └── hermes subprocess          ← HERMES_HOME → temp config pointing to proxy

Each Hermes LLM turn:
  POST /v1/chat/completions → baar proxy (local, no network)
    └── BAARRouter
          ├── pre-flight budget check    → over limit? 402. Zero API calls made.
          ├── complexity routing         → simple task → cheap model, hard task → big model
          └── real provider call via LiteLLM

Hermes thinks it's talking to a provider. Every tool-call, retry, and reflection step goes through the check first.

Getting the timing right took a few attempts. Most cost tracking records spend when the response arrives — by then you've already paid. baar estimates the cost of each call, atomically reserves that amount, makes the call, then reconciles the real cost. Two concurrent Hermes turns can't both pass the check and jointly overshoot the cap, because the reservation step is atomic.

It also routes to cheaper models as budget runs down

The routing layer scores each request for complexity and picks the model accordingly. Low-complexity turns go to the cheap model, high-complexity turns go to the big one. As the budget runs low, the threshold shifts:

Budget 0–30%:   complexity > 0.50 → big model
Budget 60–80%:  complexity > 0.75 → big model
Budget 95%+:    almost everything → small model

A $1.00 session doesn't just cut off at $1.00. It gets cheaper per turn as the budget depletes. The agent keeps working, it just costs less toward the end.

Alerts, because a silently dead session is also bad

Waking up to a session stuck at $0.999 since 2am, waiting on a cap that already fired, is almost as annoying as the $47 bill. So I added thresholds:

from baar import BAARRouter, BudgetWindow, Alert

def warn_at_80(info):
    print(f"⚠️  {info['utilization']*100:.0f}% of daily budget used — "
          f"${info['remaining']:.4f} remaining")

router = BAARRouter(
    budget=5.00,
    window=BudgetWindow.DAILY,   # resets at midnight UTC, no cron needed
    alerts=[
        Alert(threshold=0.8, callback=warn_at_80),
        Alert(threshold=0.95, callback=lambda _: send_slack("Hermes at 95% — check it")),
    ],
)

BudgetWindow.DAILY resets at midnight UTC. Each day gets its own bucket. Historical spend is preserved so you can audit any past session, and the alert re-arms automatically when the new window opens.

A policy engine, for when a single number isn't enough

If you're running Hermes on behalf of multiple users, you need rules. A free tier user hitting gpt-4o at 60% budget utilization is a problem waiting to happen. An enterprise user getting downgraded to gpt-4o-mini is a different kind of problem.

from baar.core.policy import Policy, Rule

policy = Policy(rules=[
    # Free tier users: force cheap model past 50% spend
    Rule(when={"plan": "free", "utilization": ">= 0.5"}, then="force_small"),

    # Never use big model past 70% budget for anyone
    Rule(when={"utilization": ">= 0.7"}, then="force_small"),

    # Enterprise users always get the big model
    Rule(when={"plan": "enterprise"}, then="force_big"),
])

router = BAARRouter(budget=5.00, policy=policy)

Rules are first-match-wins. You thread user metadata per call from your application layer. System facts like real utilization always override caller context, so users can't spoof their own budget status.

When a block rule fires, baar raises PolicyViolation, distinct from BudgetExhausted. Both carry a facts dict with exactly which rule matched and why.

The audit log

Every Hermes turn is logged:

for step in session.log.steps:
    print(
        f"Step {step.step_num:2d} | {step.decision.model:<20} | "
        f"${step.cost:.6f} | {step.latency_ms:6.0f}ms | "
        f"{step.decision.reason}"
    )

Step  1 | gpt-4o-mini          | $0.000023 |   412ms | complexity=0.31 → small
Step  2 | gpt-4o               | $0.000891 |  1823ms | complexity=0.78 → big
Step  3 | gpt-4o-mini          | $0.000019 |   388ms | complexity=0.28 → small
Step  4 | gpt-4o-mini          | $0.000021 |   401ms | [POLICY FORCE_SMALL] complexity=0.71
...
Total: $0.003847 of $1.00 (0.38% used)

forced_by_budget on each step tells you whether the model downgrade was a budget constraint or a policy decision.

One more thing: a supply chain issue we caught mid-build

While shipping v0.7.0 we found CVE-2026-33634, a supply chain compromise in litellm==1.82.7 and 1.82.8. Since baar-core depends on LiteLLM, any user installing without this fix would pull in the compromised version.

Two defences: an install-time constraint (!=1.82.7,!=1.82.8) so pip never resolves to those versions, and a runtime check that raises at BAARRouter construction if the bad version is already installed. If you have it, baar won't start.

The $47 bill was the useful part of that Tuesday. Turns out "iterate until done" is not a plan when you're paying per iteration and you're asleep.

GitHub: github.com/orvi2014/Baar-Core — pip install baar-core[vercel] hermes-agent

I Use AI to Build. I Don't Let It Think for Me.

Orvi Das — Sun, 17 May 2026 00:45:01 +0000

I have been building software with AI tools for about two years now. I ship with Claude Code every day. And I still do not vibe code. Here is why that distinction matters more than people think.

The first time I watched someone vibe code, I felt the same thing I feel watching someone drive with their knees. Technically possible. Impressive for about thirty seconds. And then just a matter of time.

I use AI every single day. I am not writing this from some purist position where I compile my own tools and distrust anything generated by a machine. I use Claude to write boilerplate, draft logic, suggest patterns I would have spent an hour looking up. It has made me faster in the ways that were boring to be slow in.

But I do not let it think for me.

There is a difference and it matters more than almost anything else I have learned in the last two years of building with these tools.

What Vibe Coding Actually Is

Vibe coding is not just "using AI to help write code." That is a category error that people make to either defend or attack it. Using AI to help write code is just programming now. That is what the tools are for.

Vibe coding is something specific: it is prompting without understanding, accepting without reading, and shipping without testing. It is the workflow where the developer's job becomes describing what they want and clicking approve.

The output looks like software. It passes the smell test. It runs. And then, three weeks later, something quietly breaks in production and you spend an afternoon staring at code you do not actually understand, written by a model that does not remember writing it.

I have seen this happen to smart people. I have started to do it myself on late nights when I was tired and the model was confident. It is seductive because the short loop feels productive. You say a thing, the code appears, it works. The feedback is immediate and positive.

The cost is invisible until it is not.

The Line I Draw

My workflow goes in one direction: I understand first, then I use AI to execute faster.

That means I write the unit test before I ask the model for the implementation. Not because I am rigorous by nature — I am not — but because writing the test forces me to know what I actually want. It forces me to think about edge cases before I have code that creates attachment to a specific approach. It forces me to have a definition of done that exists outside my head.

When I hand that context to the model, the output is different. Not because the model is smarter — it is the same model but because I am asking a specific, bounded question instead of a vague, open-ended one. The difference between "write me a function that handles payments" and "write me a function that takes a Stripe webhook payload, validates the signature, extracts the event type, and returns a typed result with this shape" is the difference between code that kind of works and code that actually does the thing.

Then I read what comes back. All of it. Even when it is long. Especially when it is long.

This sounds obvious. It is not practiced as much as it sounds.

What AI Is Actually Good At

The honest list of where AI makes me dramatically faster:

Boilerplate that I know the shape of. If I know I need a repository pattern with these five methods, I can describe it precisely and get it in thirty seconds instead of fifteen minutes. I understand what it should look like before I ask. The AI just types faster than I do.

Surfacing options I had not thought of. I will describe a problem and ask what patterns exist for solving it, not for the model to pick one, but so I have a more complete menu. Then I decide. The model does not know my codebase, my constraints, or my risk tolerance.

Catching things I missed. After I write something, I ask the model to review it — specifically to look for edge cases, error paths I did not handle, security issues I glossed over. It finds real things. Not always, but often enough that I have made it a habit.

Writing tests for logic I just wrote. Once the implementation is done and I understand it, I will have the model write additional test cases. It is good at thinking of inputs I did not try.

What it is not good at: deciding what to build, deciding how to architect something that will need to scale, or writing code I am not equipped to review. When I catch myself in that last situation, I stop and learn the thing first.

The Senior Developer Problem

There is a version of this conversation that gets framed as: AI will replace junior developers but senior developers are safe because they can guide it.

I do not think this is quite right, and I think believing it creates a complacency that is more dangerous than the replacement question.

The thing that makes a senior developer valuable is not primarily the ability to generate correct code. It is the ability to know which code should not exist, which abstractions will turn into debt, which requirements are wrong before you build them. That judgment comes from having been wrong enough times to develop taste.

AI does not have taste. It has pattern completion. It will write you a technically correct solution to the wrong problem with the same confidence it writes a technically correct solution to the right one. It cannot tell the difference.

If you are not developing the judgment — because you are outsourcing the thinking to the model — you are not building toward senior. You are extending the period where you do not yet know what you do not know.

Why This Is Not About Being Anti-AI

I am not arguing for slowing down or using fewer tools. I am arguing for staying in the driver's seat of your own work.

The people I have watched get the most out of these tools are the ones who get more done, not the ones who get more generated. The difference is that they know what done means before they start, and they verify they reached it before they ship.

I write unit tests first. Then integration tests against real systems, not mocks — I learned the hard way that mocked tests can pass while the actual integration is broken. Then end-to-end tests with Playwright for the paths users actually take. And I read everything the model gives me before I commit it.

That workflow is slower than vibe coding for the first hour. It is faster than vibe coding over any meaningful timescale.

The AI handles the typing. I stay responsible for the thinking.

That is the only arrangement I trust.