I built the team of AI agents at my company using Hermes - here's what actually happened

The Inspiration

I've always been drawn to exploring new AI tools, and somewhere along the way I stumbled onto the idea of self-learning agents. It's a simple concept on paper — an agent that gets better at a task the more it does it — but genuinely fascinating once you start digging into how it actually works in practice. That curiosity led me to Hermes, and I spent a while going through its documentation to understand what it was actually capable of.

The idea stayed on the back burner until my company ran an internal hackathon, where the brief was simple: build something cool. That was my excuse to finally get hands-on. I put together a Discord agent running locally on Hermes — nothing production-grade, just enough to prove the concept could work.

The Hands-On Experiment

The hackathon demo landed well enough that my seniors asked me to take it from "cool local prototype" to something the company could actually rely on. That's where the real work started.

The first wall I hit: Hermes didn't have any built-in support for deploying directly to Azure. So I built my own path — packaging the agent into a Docker image and writing a deployment script around it.

But before I touched deployment, I wanted to get the agent's job right, not just the infrastructure around it. My first real target was a sales agent, and rather than relying purely on the sales skills already floating around online, I sat down with someone from our co-working space who actually works in sales and asked them to walk me through their real process — how they research a lead, how they qualify it, how they follow up. I took that and shaped it into a skill built specifically around how my company actually sells, rather than a generic template. Somewhat unexpectedly, I picked up a decent amount of practical sales knowledge along the way.

Meet the First Hire: Ben

With the skill built, I installed it locally on Hermes and connected it to Slack to test it end-to-end. I named this first agent Ben — the company's first AI hire, built (and hired) by me.

Once Ben was working, I moved him into production: the same Docker image approach, hosted on Azure App Service. Under the hood, Hermes was running on Azure AI Foundry models — GPT-5.5 in my case, though Hermes supports a range of other model providers too.

For connecting Ben (and later agents) to external tools, I used Composio, which meant managing a single API key instead of juggling credentials for every tool the agents needed to touch.

Under the Hood: How Ben Actually Finds and Reaches Leads

To make the sales workflow concrete — say I ask Ben to find five new leads at FMCG brands similar to Innocent Drinks. Here's what happens next:

1. Search — Ben uses Tavily to search the web and identify matching companies and the right point of contact at each.
2. Store — Every lead gets written into a Notion database, with fields for name, email, LinkedIn profile link, a drafted LinkedIn message, a drafted email, an email status, and a few other supporting details.
3. Human in the loop — Nothing goes out automatically. Someone — usually me or the CEO — reviews the drafted messages in Notion and flips the email status from "drafted" to "ready" once they're happy with it.
4. Send — Once marked ready, Ben picks it up and sends the email via Composio.
5. The one manual step — LinkedIn simply doesn't allow automated messaging, and getting approved for the kind of API access that would permit it is a long, uncertain process. So that step stays manual by necessity: the drafted message sits ready in Notion, and a person sends it by hand.

The human checkpoint on the email side was a deliberate choice too — it keeps a person in control of what actually gets sent to a real prospect, while still letting Ben handle all the research and drafting legwork upfront.

Watching What the Agents Do: Langfuse

Once an agent is running unattended in production, "it seems to be working" isn't good enough — you need to actually see what it's doing and what it's costing. Hermes has built-in support for Langfuse, so observability was mostly a matter of enabling the plugin and adding a thin wrapper around my code to surface per-agent cost data on the Langfuse dashboard.

Keeping the Agents on a Budget

Cost control turned out to be the trickiest part of the whole project. Azure does offer some API-level controls for this, but they're not as straightforward as they sound. Azure OpenAI doesn't have a built-in way to enforce a hard monthly spending cap on its own — the standard Tokens-Per-Minute and Requests-Per-Minute quotas only throttle the rate of requests, they don't track or cap total spend directly. To actually enforce a hard budget, you'd need to layer on something like Azure API Management's token-limit policy, which can cap token usage over a set period (hourly, daily, weekly, and so on) and returns an error once that quota is used up — or wire together Azure Budgets with an Automation Runbook that disables API access when a cost threshold is hit. Both routes work, but they add real setup and moving parts.

Given that complexity, I opted for a simpler, more direct approach at the agent layer instead:

• Trimmed the skill set — restricting or removing pre-installed skills the agents didn't need, since every enabled skill is a potential source of unplanned token usage.
• Built a custom budget-constraint skill that tracks a weekly spending cap and halts all agent operations the moment it's crossed.

Since this means continuously checking cost against the budget, there's a small monitoring overhead — but it comes out to roughly $0.02 per check based on what I've seen in the Langfuse dashboard, which is a fair trade for the peace of mind.

Growing the Team: Dexter and Alan

Once the blueprint worked for Ben, scaling it to new roles was mostly a matter of repeating the process: build the skill, ground it in how the company actually operates, connect it through Hermes, deploy the same way.

That's how the team grew to include:

• Dexter, a PM agent, built from a mix of established project-management skills and the specific rituals my team follows. Dexter takes advantage of Hermes' built-in cron job support to automatically post mid-sprint and end-of-sprint progress reports straight to our Slack channels.
• Alan, a fundraising agent, similarly built on top of existing fundraising best practices layered with our own internal process for engaging investors.

Results

A few months in, the impact has gone well beyond "interesting internal experiment":

• Our CEO now uses Alan to help secure investor meetings.
• We recently onboarded our first customer — a win that traces directly back to work done by Ben.
• Our COO relies on Ben (the sales agent) to research prospects, manage outreach, and keep track of leads.
• Dexter has quietly become part of the team's sprint rhythm, keeping everyone updated without anyone having to chase status manually.

What started as a weekend hackathon project turned into a small team of AI agents that our leadership actually depends on day to day — which is a pretty satisfying place to land.