DEV Community: Sagar Maurya

Ouroboros AI: We Made a Fake Agent Break Itself, Then Built Something That Fixes It

Sagar Maurya — Sun, 26 Jul 2026 15:15:12 +0000

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

If you're just landing on this series for the first time, here's the short version of where we've been.

In our first post, we installed a tool called SigNoz for the very first time, so we could watch what a piece of software is doing while it runs — which parts are slow, which parts fail, and why. We built a tiny fake AI agent, and just by looking at a picture of its actions laid out in order, we could instantly see that one single step was eating up 85% of the total time. That was the moment this whole thing clicked for us.

In our second post, we went further. We built a slightly smarter fake agent and asked SigNoz a real question: "does this get slower when it has more information to look through?" We didn't calculate the answer ourselves. We just told SigNoz what to group and average, and it handed us a table proving the pattern, clean as anything.

Both of those were practice. Small, safe experiments before the real hackathon clock started ticking. This post is about what we actually built once it did — and honestly, it's the post we're proudest of, because it's the one where we stopped just watching our fake agent and started building something that does something about it.

We're calling the project Ouroboros AI. Repo's here if you want to poke around: github.com/mauryasagar/ouroboros-ai

The problem that was quietly bugging us

Right after we published that second post, something about it kept nagging at us.

We'd asked SigNoz a good question and gotten a good answer: yes, more documents means a slower response, and it climbs in an almost perfectly straight line. We felt clever about it for about a day. Then we looked at what we'd actually built and realized... nothing happened next. We saw the pattern, said "yep, makes sense," and closed the browser tab. If this had been a real product used by real people, someone would still have had to notice that pattern themselves, write it up, wait for an engineer to have time, and fix it — probably days later, and only if they happened to be looking at the right dashboard at the right moment.

That bothered us more the more we thought about it. Watching something break in slow motion isn't the same as fixing it. So a simple, slightly stubborn question became the whole idea for this project:

What if the system could notice its own problem and fix itself, without a person in the middle at all?

Not "send someone a faster alert." Actually close the loop.

A few words before the story, so nothing feels confusing

We're going to use a handful of words a lot in this post. If you already know them, skip ahead. If you don't, here they are in plain terms:

Trace — a record of everything that happened during one single request, start to finish.
Span — one step inside that trace. Four things happened? Four spans.
Alert — a rule you set up that says "if X happens, tell someone."
Webhook — instead of "tell someone" meaning a text message or an email, it can mean "call this web address automatically." That's the piece that let a computer react instead of a human.
MCP (Model Context Protocol) — a standard way for an AI model to reach out and use real tools — in our case, letting an AI ask SigNoz real questions instead of us clicking around the dashboard ourselves.

That's genuinely all the vocabulary you need. Everything else is just those five ideas, wired together.

What we actually decided to build

We sat down together and sketched this out on a shared doc before writing a single line of code. Five steps, in order:

Chaos — a fake AI agent runs constantly, and every so often, on purpose, it messes up.
Traced — every single step it takes gets recorded and sent to SigNoz, live.
Alert — SigNoz watches all these traces coming in, notices the mess-up pattern, and automatically fires off a webhook — no human clicks anything.
Heal — a separate little program receives that webhook and retries the failed thing with better settings. Still no human.
Explain — an AI "Sidekick" you can literally chat with, which reads the real SigNoz data through MCP and tells you, in plain English, what just happened.

Steps 1 and 2 were a more serious version of what we'd already built in post two. Steps 3, 4, and 5 didn't exist anywhere yet. That's where basically the entire week went.

Step one: giving our fake agent something real to fail at

Our agent from post two never actually failed — it just got randomly slower or faster each time. That's fine for a screenshot, but it's useless for an alert. You can't tell a computer "notice when things feel kind of slow sometimes." You need something crisp: a clear line that gets crossed.

So we gave our agent one specific way to break: context overload. In plain terms — our agent normally looks through somewhere between 1 and 8 "documents" to answer a question. But 20% of the time, on purpose, we let it get buried under 15 documents at once, which is way more than it can handle cleanly.

def retrieve_context(query, is_chaos_mode=False):
    with tracer.start_as_current_span("retrieve_context") as span:
        # CHAOS MODE: 15 docs, NORMAL: 1-8 docs
        context_docs = 15 if is_chaos_mode else random.randint(1, 8)
        span.set_attribute("llm.context_docs", context_docs)
        time.sleep(random.uniform(0.1, 0.4))
        return f"Mock context data ({context_docs} docs)"

We picked this specific failure on purpose, not randomly. Real AI agents that search through documents actually run into this exact problem — feed them too much information at once, and they get slower, cost more, and often don't even answer better for it. We wanted our fake failure to be one that real systems actually have, not something we made up just to have a demo.

When the overload happens, we don't just log it as "a bit slow." We mark it as an actual error, with a clear message SigNoz can watch for:

if context_docs >= 15:
    logging.error(f"CONTEXT_OVERLOAD: Agent failed. Used {tokens} tokens, cost ${round(cost, 6)}, took {delay:.2f}s.")
    span.set_status(Status(StatusCode.ERROR, "Context Overload"))

One small thing from post two turned out to matter a lot here: we'd mentioned back then that just printing something to your terminal doesn't automatically send it to SigNoz — you have to specifically wire up your logs to go there too, separately from your traces. This time, we made sure that was working properly from day one, because our whole alert depended on SigNoz actually being able to see that error message, not just a slightly-slower-than-usual span.

We let this fake agent run forever in the background, quietly sending a new trace every 200 milliseconds, so there was always something fresh happening while we built the rest.

Step three and four: teaching the system to fix itself

This is the part that didn't exist in either of our earlier posts, and honestly, the part we're most excited about.

The good news is, we didn't have to build anything fancy to make SigNoz watch for the failure — it already does that with alert rules, the exact same feature we used in post one to watch for slow calls. The only difference is where the alert points. Instead of sending a Slack message or an email, we pointed it at a webhook — a plain web address that SigNoz calls automatically the moment it sees the problem.

On the other end of that web address, we built a tiny program whose only job is to sit there, wait, and fix things when it gets called:

@app.route('/webhook', methods=['POST'])
def webhook():
    data = request.json or {}
    alert_name = data.get("alertname", "Unknown Alert")
    print(f"🚨 ALERT RECEIVED FROM SIGNOZ: {alert_name}")

    result = optimized_llm_call("Auto-healed request")

    span.add_event("AUTO-HEAL SUCCESS", attributes={
        "message": "Request auto-healed with optimized parameters",
        "status": "healed"
    })
    return jsonify({"status": "healed", "result": result}), 200

The actual fix is simple on purpose — it retries the request with a smaller, safer amount of information instead of the overload amount. The part we spent real time thinking about wasn't the fix itself, though. It was this question: should the fix be invisible, or should it leave a trace of its own?

We almost let it be invisible — quietly patch things up in the background and move on. But that felt like exactly the thing we were trying to get away from in the first place: a fix that only lives in someone's head, not in the system. So we made the healer trace its own retry too, tag it clearly as an auto-heal, and record an event that says, plainly, "this got fixed automatically." Now the healing itself shows up in SigNoz right next to the failure that caused it. Nobody has to remember it happened. The system remembers for you.

Step five: teaching an AI to explain all of it in plain English

Here's where post two's biggest lesson came back around. Back then, we discovered that if you tag your data with the right detail, SigNoz can answer real questions for you — no manual math required. The bigger question that kept nagging at us afterward was: what if you didn't even need to know which button to click, or which filter to type? What if you could just ask, like you're talking to a person?

That question is what led us into MCP — the protocol that lets an AI model reach out and actually use tools, instead of just talking. Neither of us had touched MCP before this hackathon, so before writing any code, we spent a real evening just reading about how it works. That reading turned out to save us a lot of pain later, even though it didn't save us from all of it.

What we ended up building is a small chat app we call the Sidekick. You type a question, and behind the scenes:

You type a question
   → Groq (the AI model) decides which tool it needs
   → it asks SigNoz for the real data through MCP
   → the real numbers come back
   → the AI reads them and answers you in plain English

SigNoz's MCP connection offers over 40 different tools it can use — way more than we needed — so we limited our Sidekick to five: list services, get traces, get logs, get metrics, get one specific span. Simple, focused, and cheap to run.

Getting this actually working took two separate evenings of head-scratching, and both problems were the sneaky kind — nothing crashed, things just quietly didn't work.

The first problem: our AI would just never use any tools. It would answer vaguely instead of actually checking real data, no matter how directly we asked it something like "what services are running right now." We spent a while assuming the AI itself was being stubborn, tweaking our instructions to it, before we finally printed out exactly what we were sending it — and found out the tool descriptions we were passing along were completely empty. Turns out SigNoz describes its tools using a label called inputSchema, while the AI model we were using expects a label called parameters. We were only checking for one of the two. The fix was a single line of code, but finding it meant working backwards through several layers of "this should be working" first:

schema = tool.get("inputSchema") or tool.get("parameters") or {"type": "object", "properties": {}}

The second problem was stranger. The very first request would work perfectly, and then the second one would get flatly rejected — every time. It took a while to notice that the server hands you an invisible little "session ID" the first time you connect, and expects you to send it back on every request after that, like a wristband at an event. Miss that, and it treats you like a stranger who just walked in:

sid = response.headers.get("Mcp-Session-Id")
if sid:
    self.session_id = sid

On top of that, some answers came back in a slightly unusual format made for streaming data, instead of a plain, simple response — so our little connector had to learn to handle both.

None of this felt exciting while it was happening. It was two nights of adding print statements, staring at raw technical details, and slowly ruling things out one at a time. But the moment both fixes were in, the payoff was immediate. Post two's big win was clicking through a filter and a group-by to get a table. This time, the same kind of insight just... showed up, after typing a normal sentence:

"Which services are currently being monitored in SigNoz?"

{
  "answer": "The services currently being monitored are:\n- auto-healer-service — 1 trace in the last hour (avg ~494ms)\n- deep-agent-service — 688 traces in the last hour (avg ~1.63s)",
  "tools_used": ["signoz_list_services"]
}

We kept the AI's instructions short and strict on purpose — answer in a few sentences, use the real numbers you found, don't waste time asking for logins it already has. It's meant to feel like a fast, useful teammate answering a quick question, not a general chatbot rambling.

The moment it actually came together

Here's the part that felt like more than just code working.

We finally let all three pieces run at the same time and just watched. In one window, the fake agent quietly sending traces every 200 milliseconds, like it had been doing for days. In another, the Sidekick sitting there, waiting for a question. And then, in the third window — completely on its own, with nobody typing a single command — the healer suddenly printed:

🚨 ALERT RECEIVED FROM SIGNOZ

We hadn't triggered that. SigNoz had caught the overload pattern by itself, in the live stream of data, and called the webhook on its own.

That's the moment this stopped being a demo we were operating and started being a system that was actually running itself.

About thirty seconds later, we asked the Sidekick what had just happened, half-expecting to have to nudge it toward the right answer. We didn't need to. It described the overload, the automatic retry, and the corrected numbers, pulled straight from the same live data we'd watched land in SigNoz seconds earlier. That was the entire five-step loop, closing itself, start to finish, with nobody standing in the middle of it.

What we're taking away from all three posts

Post one taught us that just seeing where the time goes changes how you think about a problem. Post two taught us that tagging the right detail turns a dashboard into something you can actually have a conversation with. This project is where those two lessons stopped being separate little discoveries and turned into one real habit: instrument things not just to log what's easy right now, but to answer the question you'll actually want answered later — because that same detail is what lets an alert fire cleanly, and what lets an AI explain the whole story back to you without you writing a single extra line of code for it.

If someone asked us what we'd tell ourselves before starting this hackathon, it's that one sentence. Everything else — the failures, the debugging nights, the late realization that our first agent needed to actually break instead of just wobble — all came from following that one idea seriously enough to build around it.

Full code, the landing page, and everything else is in the repo: github.com/mauryasagar/ouroboros-ai

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

We Asked SigNoz a Question, and It Actually Answered

Sagar Maurya — Sun, 19 Jul 2026 16:51:23 +0000

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

Before this, we wrote a first post about setting up SigNoz for the very first time — installing it, sending it some data, and figuring out what all this "tracing" stuff actually means. You can also check that out on LinkedIn. If you're new to any of this, that post explains the basics in a simple way, so it's worth a quick look first.

This post is us going one step further as a team. We wanted to see if SigNoz could actually help us discover something, not just show us pretty charts. It did, and here's exactly what happened, explained simply.

First, two words you need to know

Trace — a record of everything that happened during one single request, from the moment it starts to the moment it finishes.
Span — one step inside that trace. If a request does 4 things, it has 4 spans, one for each step.

That's really all you need to follow along.

What we built

A small fake AI assistant. Instead of doing just one thing, it does four things every time someone asks it a question:

Look up some documents — pretend to search for information related to the question (a random number, somewhere between 1 and 8 documents)
Ask the AI — pretend to send the question and those documents to an AI model and get an answer back
Use a tool — pretend to use something like a calculator, a calendar, or a search tool
Clean up the answer — tidy up the final response before sending it back

We wrapped each of these four steps in code that tells SigNoz "hey, this step just started" and "hey, this step just finished." That's what creates the spans.

Looking at just one question it answered

We ran our fake assistant 30 times, and then picked one single run to look at closely in SigNoz. Here's what it showed us:

The whole thing took 1.72 seconds. Almost all of that — 1.43 seconds — was spent in the "ask the AI" step. Everything else (looking up documents, using a tool, cleaning up) barely took any time at all in comparison. You can literally see this with your eyes on the chart, no reading through code needed.

Then we asked a bigger question

Looking at one run is useful, but we wanted to know something bigger: does the AI step get slower when it has more documents to look through?

In real life, this makes sense — if you ask an AI to read through 8 documents instead of 1, it usually takes longer to answer, because there's more for it to process.

So instead of checking each of our 30 runs one by one, we asked SigNoz to do the work for us:

We told it: "only show me the AI-call steps, ignore the other three"
We told it: "now group these by how many documents each one used"
We told it: "show me the average time for each group"

And within seconds, SigNoz gave us this table:

Documents used	Average time taken
1	568ms
2	658ms
3	825ms
4	999ms
5	1147ms
6	1333ms
7	1488ms
8	1614ms

Look at that — it climbs almost perfectly evenly. Every extra document adds roughly 150 milliseconds to the response time. We built this rule into our code, so finding it wasn't a total surprise, but what genuinely impressed us was how easily SigNoz found and displayed it. We didn't write any extra code, we didn't open Excel, we didn't do any manual math. Just a couple of clicks, and the whole pattern was right there in a table.

We also checked the logs

Traces are great for timing, but sometimes you just want to read a plain sentence about what happened. That's what logs are for. We turned those on too, and got results like this:

Now every request has two views: the trace shows how long things took and in what order, while the logs show what actually happened, in plain readable sentences. Small tip if you try this yourself — just printing messages to your terminal doesn't automatically send them to SigNoz. You need a couple of extra lines of setup code to actually connect them, or nothing will show up in the Logs tab.

What we're taking away from this

The biggest thing we learned: a trace waterfall is great for spotting what's slow. But the real magic happened when we tagged our own extra detail onto a span — in our case, "how many documents did this use." That one small addition is what let us ask SigNoz a real question and get a real answer back, across all 30 runs at once, instead of guessing.

For the actual hackathon build, this is exactly the habit we want to carry forward — don't just time your code, tag it with details that'll actually matter later when something's slow and you're trying to figure out why.

Data Isn't What Your Textbook Said It Was

Sagar Maurya — Sat, 11 Jul 2026 02:13:37 +0000

A beginner-friendly breakdown of what data really is, where it comes from, and why it matters more than you think.

You woke up this morning and checked your phone.

You scrolled through Instagram. Opened WhatsApp. Maybe checked the weather. Ordered breakfast on Swiggy.

By the time you finished your morning tea, you had already generated hundreds of data points — and that was before 9 AM.

We live in a world that runs on data. But most people, including a lot of CS students, have never stopped to ask: what actually is data?

Not the textbook definition. The real one.

And that's exactly the problem — we all learned about data from a textbook. Which means we learned about it wrong.

The Textbook Lie

Ask anyone what data is and they'll say something like:

"Data is raw, unprocessed facts and figures."

Technically correct. Completely useless.

That definition tells you nothing about why data matters, where it comes from, or what makes it powerful. It's the kind of definition written for an exam, not for understanding.

So let's close the textbook and start over.

Data is Just a Record of Something That Happened

Every time something happens in the real world — a click, a purchase, a step you walked, a message you sent — there's a possibility of recording it.

When you record it? That's data.

You clicked on a product → data
Your phone counted 6,400 steps today → data
You paused a YouTube video at 2:34 → data
You left a website after 3 seconds → data (and a headache for that website's owner)

Data is just a trace that something happened. That's it.

The interesting part is what happens after you collect those traces.

Raw Data is Almost Always Ugly

Here's what nobody tells beginners: data in the real world is messy.

Imagine a Google Form that asks for someone's phone number. You'll get responses like:

9876543210
+91-9876543210
98765 43210
9876543210 (call after 6pm)

All four people gave you the same number. But to a computer, these are four completely different values.

This is what's called dirty data — and cleaning it is genuinely one of the most important (and underrated) skills in the data field.

A famous saying in data science is:

"80% of data work is cleaning. The other 20% is complaining about cleaning."

It's a joke. But not really.

The Journey of Data: From Noise to Decision

Here's a simple way to think about how data travels:

Something happens
      ↓
It gets recorded (raw data)
      ↓
It gets cleaned (processed data)
      ↓
It gets analyzed (insights)
      ↓
Someone makes a decision

Let's make this real.

Example: Swiggy wants to know why orders drop on Tuesday evenings.

They collect order timestamps → raw data
They remove duplicates, fix timezone errors → clean data
They spot that Tuesday 7–9 PM has 40% fewer orders → insight
They run a Tuesday evening discount campaign → decision

That entire chain? It all started with someone pressing "Order" on their phone.

Not All Data is Numbers

A common misconception is that data means spreadsheets full of numbers. But data comes in many forms:

Structured data — rows and columns, like a CSV or a database table. Easy for computers to process.
Unstructured data — text, images, audio, video. Harder to process, but far more common in the real world.
Semi-structured data — like JSON or XML. Has some structure, but not rigid rows and columns.

When you send a WhatsApp message, that's unstructured data. When your bank logs a transaction, that's structured data. When you fill a Google Form, that produces semi-structured data.

Most of the world's data — over 80% of it — is unstructured. Which is why fields like NLP (Natural Language Processing) and Computer Vision exist: to make sense of data that doesn't fit neatly into a table.

Your Data is Someone's Product

Here's the uncomfortable truth.

Every free app you use — Instagram, Google, YouTube — is free because you are not the customer. You're the product. More precisely, your data is.

When you like a post, skip an ad, or spend 45 minutes on Reels instead of 5, that behavior is recorded, analyzed, and used to serve you more content that keeps you on the platform longer.

This isn't conspiracy theory. It's just the data pipeline at scale.

Understanding this doesn't mean you need to delete all your apps. But it does mean you should be aware of the trade you're making — your attention and behavior in exchange for a free service.

Why Everyone Should Care About Data (Not Just Data Scientists)

You don't need to work in data to benefit from understanding it.

As a developer — you'll build systems that generate data. Understanding data helps you design better databases and APIs.
As a product person — decisions without data are just opinions. Data makes arguments.
As a user — knowing how your data is used makes you a more informed digital citizen.
As a student preparing for placements — almost every tech company today is a data company in some way. Interviews increasingly involve data thinking, even for SDE roles.

Data literacy is becoming as fundamental as being able to read and write.

So What Did Your Textbook Get Wrong?

Nothing, technically. But everything, practically.

Your textbook gave you a definition of data that would pass an exam. It didn't give you a mental model that would help you think about the world differently.

The difference between a student who "knows data" and someone who understands it is this: one memorized a definition, the other sees data everywhere they look — in the apps they use, the decisions companies make, and the systems they build.

That shift in perspective is what this blog is about. Not definitions. Mental models.

Where to Go From Here

This post was just the surface. In future posts, I'll go deeper into:

How databases store and retrieve data efficiently
What actually happens when you run a SQL query
The difference between a data analyst, data engineer, and data scientist
And some hands-on projects that helped me understand all of this better

If you're a student or someone just getting into tech — I hope this gave you a clearer mental model of what data actually is.

Because before you can work with data, you need to understand what you're actually working with.

Thanks for reading! If you found this useful, drop a reaction or share it with someone who's just getting started in tech.