DEV Community

Cover image for I Built an AI Ops Assistant That Replaced $2,400/month in SaaS — In 3 Days, for Under $100/month
Stephen Goldberg for Harper

Posted on • Edited on

I Built an AI Ops Assistant That Replaced $2,400/month in SaaS — In 3 Days, for Under $100/month

Last month, one of our engineers typed this into Slack:

/incident what would cause basic authentication to fail even with a properly formed authentication header when the user is present and the roles are correct

Nine seconds later — before anyone else had even read the message — Harper Eye posted a full incident analysis in the thread. It had identified the root cause (a misconfigured authorizeLocal setting in harperdb-config.yaml), cited three internal sources (our documentation, codebase, and Slack history), listed specific remediation steps, and @-mentioned the five engineers with the most expertise in authentication — by name, with clickable Slack handles.

Harper Eye responding in Slack with a full incident analysis, including situation summary, recommended steps with cited sources, @-mentioned experts, clickable channel links, and feedback buttons<br>

No one had to context-switch. No one had to dig through four different dashboards. No one had to remember which runbook lived where. The answer was right there in the channel, with clickable links to every source.

That moment crystallized something I'd been feeling for a while: the most valuable internal tool your company will ever have is one that connects your existing knowledge and puts it in the hands of every engineer, instantly.

And you don't need to buy it from a vendor. You can build it yourself.


The Hidden Cost of Scattered Engineering Knowledge

If you run an engineering team, this scenario is your daily reality:
Your knowledge is scattered across six or more systems, documentation wikis, support tickets, monitoring dashboards, chat history, code repos, and incident postmortems. When something breaks at 2 am, your on-call engineer has to manually search each one, piece together context from fragments, and hope they find the right runbook before customers start complaining.

The real cost isn't the monthly SaaS bills (though we'll get to those). It's the compounding tax on every engineer, every day:

Time to resolution goes up. A junior engineer who joined last quarter doesn't know that "replication lag on us-east" was caused by a misconfigured sharding key three months ago. That tribal knowledge lives in a Slack thread they'll never find.

Expertise walks out the door. When your senior engineer leaves, their mental model of the system — the one that connects a Datadog alert to a Confluence runbook to a specific config file — goes with them.

Duplicate work compounds silently. The same question gets answered differently by different people in different channels. There's no feedback loop. No institutional memory.

Most companies try to solve this by buying SaaS tools. An AI chatbot here, an incident management platform there, a knowledge base product, a Slack integration. Before you know it, you're paying $2,000-5,000/month across multiple vendors, and none of them actually know your systems, your architecture, your people, or your internal terminology.


What Harper Eye Actually Does (And Why It's Different)

Harper Eye is not a chatbot with a Confluence integration. It's an AI orchestrator that treats your entire internal knowledge surface as a single searchable brain.

Here's what happens when someone asks Harper Eye a question, and I mean exactly what happens, because I'll show you every line of code in this series:

1. Instant Knowledge Base Check

Before anything else, Harper Eye checks if this question (or something semantically similar) has been asked and answered before. If a verified answer exists with high confidence, it returns it instantly, no API calls, no waiting. Sub-second response.

2. Parallel Multi-Source Search

If there's no cached answer, Harper Eye searches all your data sources simultaneously, not sequentially, not one-at-a-time. In parallel:

  • Internal documentation (Confluence, wikis, runbooks)
  • Support tickets (Zendesk — what are customers actually reporting?)
  • Monitoring data (Datadog — what's alerting right now? What do host metrics show?)
  • Source code (GitHub — what does the code actually do?)
  • Chat history (Slack — what have engineers said about this topic recently?)
  • Team expertise (Who on your team has answered similar questions before?)

This parallel search typically completes in 2-4 seconds across all sources. Here's what the pipeline timing actually looks like for a real query:

Pipeline timing breakdown showing EMBEDDING 135ms, KB SEARCH 7ms, SOURCES 3.5s, CLAUDE 5.5s, 9.1s total. Data source results: Harper Docs 5 results, Zendesk 5 results, Code Knowledge 3 results, Slack History 10 results

That's a real query, not a benchmark. 135 milliseconds for the embedding, 7 milliseconds for the knowledge base search, 3.5 seconds for all six data sources in parallel, and 5.5 seconds for Claude to synthesize. Nine seconds from question to fully cited answer.

3. AI Synthesis with Claude

All that raw data gets fed to Claude with a system prompt that is deeply specific to your company. Not generic advice — your architecture, your terminology, your config files, your API endpoints. Claude synthesizes it into a structured response:

Harper Eye web UI showing a detailed answer about Plexus replication, with expert names highlighted, cited sources including docs and Slack history, and

Look at that response. It's not generic. It names the specific replication engine (Plexus), references the exact config file (harperdb-config.yaml), cites Slack conversations about a real customer deployment, and names five specific engineers to contact — all grounded in actual internal data, not LLM hallucination.

4. The Knowledge Loop (This Is the Magic)

After every response, engineers can give feedback — thumbs up or thumbs down. When someone marks an answer as helpful, it gets stored in a vector-indexed knowledge base. The next time a similar question comes in, Harper Eye returns that verified answer instantly.

When someone marks an answer as not helpful, that negative signal gets stored too. If an answer accumulates enough negative feedback, Harper Eye automatically degrades it — forcing a fresh search instead of returning stale information. Eventually, bad answers get purged entirely.

Your AI assistant gets smarter every time your team uses it. No retraining, no manual curation. Just a feedback loop that compounds institutional knowledge over time.

5. Expert Routing — It Knows Your People

This is the feature that no vendor tool offers. Harper Eye indexes your Slack conversations to build a map of which engineers answer questions about which topics. Not who's assigned to a team — who actually demonstrates expertise by helping others.

Skills & Expertise matrix showing 35 people across 17 topics with answer counts — clustering, authentication, fabric-deployment, replication, and more

That's a live skills matrix with 35 engineers across 17 topics, built entirely by mining Slack history. When an incident requires escalation, Harper Eye doesn't guess; it tells you exactly who to loop in, based on who's actually answered questions about that topic before.


The Architecture: Surprisingly Simple

Here's the part that surprised me most: the entire system is about 700 lines of core logic running on a single platform.

Architecture diagram of the Harper (Fabric) system. Entry points on the left include Slack commands (/incident, /harper-ask, @harper-eye) and a Web UI Dashboard, both feeding into the Harper core. Inside Harper, Resource Classes (HTTP endpoints) connect to an Orchestrator that performs four steps: KB Check, Parallel Fetch, Claude Synthesis, and Feedback Loop. The Orchestrator feeds into three internal stores: a Knowledge Base (HNSW), Vector Search (HNSW), and an Audit Log. At the bottom, six external connectors fan out from the core: Confluence, Zendesk, Datadog, GitHub, Slack History, and Harper Docs.

The entire application runs on Harper; it's simultaneously the application server, the database, and the vector search engine. There's no separate Postgres for storage, no Pinecone for embeddings, no Redis for caching, no Express for HTTP. One platform, one deploy target, one bill.

Here's what the stack looks like:

Layer Technology Why
Runtime + Database + Vector Search Harper (Fabric) One platform handles HTTP endpoints, data storage, and HNSW vector indexing. No infrastructure to manage.
AI Synthesis Claude (Anthropic API) Structured JSON output, handles massive context windows for multi-source analysis.
Embeddings Google Gemini 768-dimension vectors for semantic search. Free tier covers most usage.
Data Sources Direct HTTP APIs Simple fetch() calls to Confluence, Zendesk, Datadog, GitHub, and Slack APIs.
Frontend Vanilla HTML/CSS/JS Static files served by Harper. No build step, no React, no webpack.

Proof This Works: Real Numbers from Production

This isn't a prototype. Harper Eye has been running in production at Harper, serving a team of 35 engineers. Here are the real numbers:

Dashboard showing 222 total queries, 29 KB articles, 24 feedback items, 0 degraded entries, with leaderboards showing real team usage across 12+ engineers

222 queries answered. 29 knowledge base articles built automatically from positive feedback. 24 feedback items shaping future responses. 12+ engineers actively using it across the team. All of that from a system that costs us less than $100/month to run.


The Cost Breakdown That Changes the Math

Here's what a typical "AI-powered internal knowledge" stack costs from vendors:

Tool Monthly Cost What You Get
Guru / Glean / Dashworks $800-2,000 AI search across your tools
PagerDuty AIOps add-on $500-1,000 AI incident triage
Incident.io or similar $400-800 Incident management + Slack bot
Vector DB (Pinecone, etc.) $70-200 Embedding storage
Total $1,770-4,000/mo Separate tools that don't talk to each other

Here's what Harper Eye costs to run:

Resource Monthly Cost
Harper Fabric (1 instance) ~$25
Claude API (Anthropic) ~$30-50 (depends on query volume)
Gemini embeddings ~$0 (free tier)
Confluence/Zendesk/Datadog APIs $0 (included in existing subscriptions)
Total ~$55-75/mo

That's not a typo. Under $100/month to replace thousands in SaaS, with a system that's more integrated, more specific to your company, and gets smarter over time.

The gap is even wider when you factor in what vendors can't do: search your actual Slack history, know which engineers are experts on which topics, learn from your team's feedback, or speak in your company's specific terminology.


Why I Built This Instead of Buying It

I want to be direct about the motivation. I'm a technical co-founder at Harper. I could have justified buying a tool; there are plenty of good ones out there, and the budget would have been approved.

But here's the thing: every SaaS tool you buy is someone else's opinion about how your problem should be solved. It's built for the average customer, not for your specific architecture, your specific team, your specific knowledge topology.

Harper Eye knows that our replication engine is called Plexus (not NATS, that's deprecated). It knows that "Custom Functions" is a legacy term, and the current pattern is Resource Classes. It knows that when someone asks about "Fabric," they mean our managed cloud platform, not a generic concept. And when it escalates an incident, it doesn't tell you to "contact the on-call team" — it @-mentions @Kris and @David by name, because it knows from Slack data that they've answered 28 and 32 questions about clustering, respectively.

No vendor product will ever know that. And more importantly, you know things about your company that no vendor ever will. The question isn't whether AI can help your team, it's whether you want that AI to work for a vendor, or for you.

Let's build it.


What We're Building in This Series

Over the next five parts, you'll build your own version of Harper Eye from scratch. Every line of code is real, tested, and production-ready. By the end, you'll have a system that does everything you've seen in the screenshots above, running on your infrastructure, for your team.

Part 2: Foundation — Harper Setup, Schema & Your First Endpoint

Set up Harper, define your database schema with vector-indexed tables, and build your first Resource Class endpoint. By the end, you'll have a running application with a REST API.

Part 3: The AI Brain — Claude Orchestrator & Multi-Source Search

The core of the system. Build the orchestrator that searches all your data sources in parallel, feeds results to Claude, and returns structured, cited responses. This is where the real magic happens.

Part 4: Slack Integration — Slash Commands, Events & Rich Responses

Wire it into Slack so your team can use slash commands, @-mention the bot, and get beautifully formatted responses with feedback buttons — all inside the tool they already live in.

Part 5: The Knowledge Loop — Vector Search, Feedback & Self-Improvement

Build the system that makes Harper Eye smarter over time. Embedding generation, semantic similarity search, feedback collection, knowledge degradation, and expert routing.

Part 6: Ship It — Deploy to Production & What to Build Next

Deploy to Harper Fabric, walk through the real cost breakdown, and explore extensions like PagerDuty webhooks, a web dashboard, and automated knowledge capture from Slack threads.

Each part stands on its own — you'll have a working, deployed system after every article. But together, they build something that fundamentally changes how your team operates.


Prerequisites

To follow along, you'll need:

  • Node.js (v20+ recommended)
  • Harper Fabric account (free to sign up at fabric.harper.fast)
  • Anthropic API key (for Claude — console.anthropic.com)
  • Slack workspace where you can create an app
  • At least one data source to connect (Confluence, Zendesk, Datadog, or GitHub — even one is enough to start)

No prior experience with Harper is required. If you can write JavaScript and have used a REST API, you have everything you need.

Top comments (2)

Collapse
 
aleks4 profile image
Aleks

This is what makes me think the whole “SaaS is dead” conversation is not that crazy. The gap between buying a rigid tool and generating exactly what you need is shrinking fast, and it’s not hard to imagine AI companies eventually owning the full backend stack that creates systems on demand. It feels like we’re seeing the early stages of that shift.

Collapse
 
margo_hdb profile image
Margo McCabe Harper

Excited to follow along with this series!