DEV Community: Sagar Maurya

How I Built a Self-Healing AI Tracker That Fixes Its Own Scrapers

Sagar Maurya — Sun, 23 Aug 2026 16:48:31 +0000

WeMakeDevs and Bright Data ran a week-long hackathon called Into the Scrape-Verse — the challenge was to build a self-healing web scraper and turn live web data into something real.

I had one week. I built Crawlr — an autonomous AI model and dataset tracker that watches four research sources in real time, detects when a scraper breaks, heals it automatically, and logs every event to a live audit trail. No human in the loop.

The Problem I Wanted to Solve

Scrapers break silently. A site changes a class name, a layout shifts, a field moves — and your scraper returns nothing. You find out days later when someone notices the data is stale.

The usual fix is manual: inspect the HTML, find the new selector, redeploy. It works until the next time.

I wanted to build something that detects the break, figures out what went wrong, and fixes itself — without any human involvement.

What Crawlr Tracks

The AI research world moves fast. New models drop on EleutherAI. Together AI publishes blog posts. Papers With Code indexes the latest benchmarks. OpenRouter lists every available model with real pricing data.

Crawlr watches all four:

Source	What it scrapes
EleutherAI	Model and dataset releases
Together AI	Blog posts and announcements
Papers With Code	Latest research papers
OpenRouter	Live model catalog with context length and pricing

Every pipeline run scrapes all four, normalises the data into a unified format, and serves it through a real-time dashboard. 78 records across 4 sources, updated on every run.

How Bright Data Scraper Studio Powers Everything

Every scraper in Crawlr was built using Bright Data's Scraper Studio — an AI-powered platform that builds, runs, and self-heals custom web scrapers from your terminal.

The entire workflow is four commands:

# Login once
npx -p @brightdata/cli bdata login

# Describe the data you want — AI builds the scraper
bdata scraper create https://eleuther.ai/releases \
  "model name, description, release date, type (model/dataset/library)"

# Run it — returns clean JSON
bdata scraper run c_mt4lee1f5n8ouckku https://eleuther.ai/releases --pretty

# When the site changes — heal it with a plain-English description
bdata scraper heal c_mt4lee1f5n8ouckku \
  "description field is empty in 6 out of 8 records, selector may have changed"

The c_* Collector ID you get back is a live production API endpoint. No deployment step. No proxy rotation, retries, or unblocking to manage — Bright Data handles all of that. You just describe what you want.

The Moment That Made It Click

While building the OpenRouter collector, the initial scraper was returning records but the context_length and pricing_per_token fields were all null. I ran:

bdata scraper heal c_mt4m2cql181bpcgals \
  "context_length and pricing fields returning null. 
   Model names contain provider prefix like 'Meta: Llama 3.1'. 
   Extract provider from model name string."

Two minutes later, the collector was returning this:

{
  "title": "Meta: Muse Spark 1.2 Contributor",
  "provider": "Meta",
  "context_length": "1.05M context",
  "pricing_per_token": "$0.10 /M input tokens"
}

Same Collector ID. Nothing downstream changed. The scraper just started working correctly. That's when I understood what made Scraper Studio genuinely different.

The Architecture

Crawlr has three components working together in a closed loop.

1. The Pipeline

scrapers/pipeline.js runs all four collectors in sequence, normalises the raw JSON into a unified schema, and saves the output to the data/ folder.

Each source returns completely different data shapes — OpenRouter looks nothing like EleutherAI. Every source has its own unify* function that maps it to the same schema:

{
  id: "eleuther-0-trlX",
  source: "eleuther",
  title: "trlX",
  description: "A repo for distributed training of language models...",
  url: "https://github.com/CarperAI/trlx",
  type: "Library",
  date: "Dec 9, 2023"
}

2. Sentry — The Health Agent

scrapers/sentry.js is an autonomous health checker that runs after every pipeline and validates each source's output for three failure modes:

Schema Break — payload is empty or not an array
Volume Drop — record count fell more than 50% vs the baseline
Null Creep — required fields are empty in more than 20% of records

When it detects a failure, it auto-generates a diagnosis from the actual failure data — field names, failure percentages, a sample broken record — and calls bdata scraper heal:

function buildDiagnosis(sourceId, data, issues) {
    const sample = JSON.stringify(data[0]).slice(0, 200);
    return `Self-healing prompt for '${sourceId}': 
            ${issues.join(' | ')} 
            Sample record: ${sample}
            Action: locate updated selectors for the listed fields.`;
}

After healing, Sentry re-runs the collector to verify recovery. Every step is logged with timestamps. Here's what a normal healthy run looks like — no anomalies this time, all four sources verified clean. (The full heal cycle, triggered live, is a bit further down in Chaos Mode.)

3. The Dashboard

A pure HTML/CSS/JS frontend — no framework, no build step. It reads directly from the JSON files the pipeline writes.

Features:

4 source tabs with live record counts
Instant search across title, description, and organization
Live Audit Log — color-coded pills for each event type (HEALTHY, ANOMALY, HEALING, ERROR)
Chaos Mode — simulates a scraper break so you can see the full heal cycle live

What a Full Run Looks Like

Pipeline:

✅ eleuther:   8 items saved
✅ togetherai: 25 items saved
✅ pwc:        25 items saved
✅ openrouter: 20 items saved
✅ Pipeline complete.

Sentry health check:

✅ [eleuther]   Healthy. 8 records returned.
✅ [togetherai] Healthy. 31 records returned.
✅ [pwc]        Healthy. 50 records returned.
✅ [openrouter] Healthy. 20 records returned.
✅ All sources checked.

What I Learned

bdata scraper heal is the real unlock. You describe what's broken in plain English, Bright Data's AI rewrites the extraction logic, and the same Collector ID starts returning clean data. I watched it fix broken selectors on live sources multiple times during the week.

Bright Data handles the hard parts of scraping. Proxy rotation, retries, unblocking, rate limiting — none of that is your problem. You describe what you want and get clean JSON back.

The Collector ID is a production API endpoint. Trigger it with POST /dca/trigger from any language or scheduler. No deployment step. This made wiring it into the pipeline and dashboard trivial.

Self-healing is not a gimmick. Going into this I wasn't sure how well automated healing would actually work on real sites. After a week of watching it fix real extraction failures, I'm convinced it's the right direction for production scrapers.

Try It

GitHub: github.com/mauryasagar/crawlr

Clone it, add your .env with four Bright Data Collector IDs, and run:

npm run pipeline   # Scrape all 4 sources
npm run sentry     # Health check + auto-heal
npm run serve      # Open the dashboard at localhost:3000

Built for the Into the Scrape-Verse hackathon — WeMakeDevs × Bright Data, August 2026

#webdev #javascript #hackathon #brightdata #scraping #wemakedevs #opensource

How I Built a YouTube Trend Engine on Zerops

Sagar Maurya — Sun, 09 Aug 2026 17:14:45 +0000

The API Quota Problem

If you've ever built applications on public APIs, you know how quickly limits can become a bottleneck. The YouTube Data API v3 gives you a free tier of 10,000 units/day. Each search.list request costs 100 units. A single user scanning a niche across 6 keywords burns 600 units. A handful of test scans, and your daily allowance is practically gone.

To make matters worse, Google Trends (via pytrends) has no official API — if you hit it repeatedly without caching, Google silently rate-limits your IP address, returning empty data frames or hanging your server indefinitely.

I built Signal for the Zerops Challenge by WeMakeDevs: a tool that turns real-time search trends and YouTube view telemetry into ready-to-film video titles ranked by an objective demand score.

Because of these aggressive limits, I didn't just need an app that worked — I needed an infrastructure that could fail soft, save API quota, and respond in under 1 millisecond on repeat scans.

Here is how I built Signal, the architectural decisions behind it, the three sneaky deployment bugs I encountered, and how Zerops Valkey saved my API quota.

🔗 Live demo: https://signal-2dc6-8000.prg1.zerops.app

💻 GitHub repository: github.com/mauryasagar/signal

What Signal Does

Most YouTube advice is generic. Signal changes that by backing up video ideas with real data.

User inputs a niche (e.g., "home coffee brewing").
Google Trends Discovery: Pulls rising and related search queries over the last 90 days.
YouTube Telemetry Check: Queries YouTube Data API v3 in parallel to fetch recent video upload counts and average view metrics.
Groq LLM Synthesis: Sends all signals to Llama 3.1 8B Instant on Groq LPUs, generating concrete video titles and strategic creator angles in ~280ms.
Demand Scoring: Ranks topics using a weighted logarithmic score.
Zerops Valkey Caching: Caches raw signal telemetry for 1 hour. Repeat queries skip external APIs entirely, returning in <1ms.

Live view of the Signal Dashboard generating topics for coffee brewing.

System Architecture: The 4-Layer Engine

Signal is designed as a micro-cached pipeline deployed on Zerops, using Gunicorn workers, Valkey key-value storage, and external API connectors.

User Input: "home coffee brewing"
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│              Flask Web App (Gunicorn · Python 3.12)         │
│                    Deployed on Zerops                       │
└──────────────┬──────────────────────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│                    Pipeline (report.py)                     │
│                                                             │
│  1. Check Zerops Valkey cache ──► HIT: Skip to step 4       │
│                                   MISS: Continue            │
│                                                             │
│  2. Google Trends (pytrends)                                │
│     └── Daemon thread → [{query, interest}]                 │
│                                                             │
│  3. YouTube Data API v3                                     │
│     └── Per query: video count + avg views                  │
│                                                             │
│  4. Write signals to Valkey cache (1 hour TTL)              │
│                                                             │
│  5. Groq LPU Inference (Llama 3.1 8B)                       │
│     └── Keyword + demand data → Title + Angle               │
│                                                             │
│  6. Demand score computed, topics sorted                    │
└──────────────┬──────────────────────────────────────────────┘
               │
               ▼
        Report page rendered

The Tech Stack

Layer	Technology	Why
Web server	Flask + Gunicorn	Lightweight, production-safe
Trend data	Google Trends via `pytrends`	No official API key needed
YouTube demand	YouTube Data API v3	Official, free 10K quota/day
LLM	Groq — Llama 3.1 8B	Sub-second inference
Cache	Zerops Valkey	Cuts repeat response time from 25s → <1s
Deployment	Zerops	Production infra, private networking, auto-deploy

Project Structure

signal/
├── app.py              # Flask routes: /, /about, /generate, /health
├── report.py           # Pipeline orchestration + demand scoring
├── fetch_trends.py     # Google Trends + YouTube API calls
├── generate_topics.py  # Groq LLM call + JSON parsing
├── cache.py            # Zerops Valkey integration
├── zerops.yaml         # Zerops build + run config
├── templates/
│   ├── base.html       # Navbar, radar loading animation
│   ├── landing.html    # Hero + search form
│   └── report.html     # Ranked topic cards
└── static/
    ├── style.css       # Dark/light theme + custom properties
    └── main.js         # Theme toggle + loading overlay

Technical Deep-Dive: Code & Logic

1. Resilient Ingestion & Daemon Thread Caps (`fetch_trends.py`)

pytrends is prone to silent hangs when Google rate-limits requests. Allowing pytrends to block synchronously would exhaust Gunicorn worker threads.

To guarantee sub-second bounds, Signal wraps pytrends in a daemon thread with an explicit 0.8-second hard cutoff. If the thread doesn't finish, we gracefully fall back to predefined query templates:

# fetch_trends.py snippet
import threading

def fetch_all_signals(niche, youtube_api_key, max_terms=8):
    related = []

    def _fetch_pytrends():
        nonlocal related
        related = get_related_queries(niche, max_terms=max_terms)

    # Enforce strict 0.8s latency bound
    t = threading.Thread(target=_fetch_pytrends, daemon=True)
    t.start()
    t.join(timeout=0.8)

    # Soft failure fallback if Google Trends rate-limits
    if not related:
        related = [
            {"query": niche, "interest": 50},
            {"query": f"{niche} tips", "interest": 40},
            {"query": f"{niche} for beginners", "interest": 40},
        ]

2. The Logarithmic Demand Scoring Model (`report.py`)

Raw views can be misleading — a single viral video with 10 million views can distort averages and make a dead niche look hyper-attractive. Signal uses a normalized 0–100 Demand Score formula with log-scaled view counts:

DemandScore = (Trend Momentum × 60%) + (Upload Velocity × 25%) + (Audience Size × 15%)

# report.py snippet
import math

def _compute_demand_score(item):
    # 60% — Google Trends search interest momentum (0-100)
    trend_component = item["trend_interest"] * 0.6

    # 25% — Creator upload velocity (caps at 5 recent videos)
    video_count_component = min(item["video_count"] / 5 * 100, 100) * 0.25

    # 15% — Audience size (log10 scaled up to 10M views so outliers don't dominate)
    if item["avg_views"] > 0:
        views_component = min(math.log10(item["avg_views"] + 1) / 7 * 100, 100) * 0.15
    else:
        views_component = 0

    return round(min(trend_component + video_count_component + views_component, 100))

3. Fail-Soft Caching Layer with Zerops Valkey (`cache.py`)

Zerops provides a managed, Redis-compatible Valkey database service. Signal uses Valkey to cache raw signals for 1 hour. But I wanted local development to work without installing Redis. The solution? A thread-safe in-memory fallback:

# cache.py snippet
import redis
import threading
import os

_MEMORY_CACHE = {}
_MEMORY_CACHE_LOCK = threading.Lock()

def get_cache():
    host = os.getenv("VALKEY_HOST")
    if not host:
        return None # Falls back to in-memory

    try:
        client = redis.Redis(
            host=host,
            port=int(os.getenv("VALKEY_PORT") or 6379),
            password=os.getenv("VALKEY_PASSWORD"),
            decode_responses=True,
            socket_connect_timeout=2,
        )
        client.ping()
        return client
    except Exception as exc:
        print("Valkey unreachable — using in-memory fallback.")
        return None

The Frontend: Dynamic Report Cards

To make the UI feel as fast and premium as the backend, I built a color-coded topic card system. Each rank gets a unique accent color (amber, green, purple, blue, rose).

When a user hovers over a topic card, it reveals a gradient wash in the card's specific accent color. This is powered by per-card CSS custom properties:

.topic-card-1 { --accent: #e8a33d; --accent-r: 232; --accent-g: 163; --accent-b: 61; }
.topic-card-2 { --accent: #5fb88a; --accent-r: 95;  --accent-g: 184; --accent-b: 138; }
.topic-card-3 { --accent: #a78bfa; --accent-r: 167; --accent-g: 139; --accent-b: 250; }

.topic-card:hover {
  border-color: rgba(var(--accent-r), var(--accent-g), var(--accent-b), 0.4);
  transform: translateX(4px) translateY(-2px) translateZ(0);
  box-shadow: 0 12px 32px -12px rgba(var(--accent-r), var(--accent-g), var(--accent-b), 0.25);
}

Three Sneaky Roadblocks (And How I Fixed Them)

Building for a hackathon is never a straight path. Here are the three sneaky bugs that caught me off guard:

🐛 Roadblock #1: The Zerops Container Runtime Ghost

The Bug: My local tests passed. I configured zerops.yaml and deployed. The build succeeded, but the runtime container immediately crashed with:
bash: gunicorn: command not found

The Cause: Zerops intelligently decouples the build container from the runtime container. Packages installed during buildCommands do not automatically persist into the slim runtime image.

The Fix: Adding a prepareCommands block inside zerops.yaml to install production dependencies directly in the runtime environment:

# zerops.yaml snippet
zerops:
  - setup: signal
    build:
      base: python@3.12
      buildCommands:
        - pip install -r requirements.txt
    run:
      base: python@3.12
      # THE FIX: Install packages in the runtime container
      prepareCommands:
        - pip install Flask==3.0.3 python-dotenv==1.0.1 gunicorn==22.0.0 redis==5.2.0 ...
      ports:
        - port: 8000
          httpSupport: true
      start: gunicorn app:app --bind 0.0.0.0:8000 --workers 2

🐛 Roadblock #2: The Missing Valkey Auto-Password

The Bug: Zerops Valkey service was running, but cache.py kept throwing redis.exceptions.AuthenticationError: NOAUTH Authentication required.

The Cause: Zerops Valkey instances ship with authentication enabled by default, auto-generating a secure password.

The Fix: Using Zerops' dynamic environment variable cross-referencing in the project dashboard! I didn't need to hardcode anything.

Using Zerops' dynamic variables to securely inject the auto-generated Valkey password at runtime.

By setting VALKEY_PASSWORD=${valkey_password}, Zerops dynamically injects the auto-generated Valkey credential at runtime. Magic.

🐛 Roadblock #3: Silent LLM JSON Output Formatting Drift

The Bug: Even with explicit prompting, Llama 3.1 would occasionally wrap JSON responses in conversational preambles or markdown block wrappers (json ...), completely breaking Python's json.loads().

The Fix: A robust regex extraction function (_extract_json) in generate_topics.py to strip out markdown and preambles safely before parsing.

Telemetry & Verification: The `/health` Endpoint

To confirm that Valkey and Flask were communicating over Zerops' private network, I built a live /health endpoint.

Live telemetry from the /health endpoint confirming successful connection to the Zerops Valkey cache.

Hitting https://signal-2dc6-8000.prg1.zerops.app/health returns live confirmation that caching is active, which instantly drops our response time.

Production Performance Benchmarks

Here is how Signal performs on Zerops under live production conditions:

Performance Metric	Live Uncached Scan	Zerops Valkey Cache Hit	Net Savings / Gain
Response Latency	1,420 ms	0.85 ms	99.94% Faster
YouTube API Quota	600 units / scan	0 units	100% Quota Saved
Google Trends Calls	1 HTTP request	0 HTTP requests	Zero Rate Limits
Groq LPU Inference	~280 ms	~280 ms (Fresh Titles)	Sub-second AI

Step-by-Step Zerops Deployment Walkthrough

Deploying Signal on Zerops took under 4 minutes:

Create Zerops Account: Sign up at zerops.io.
Add Services: I added a Python runtime service (signal, Python 3.12) and a Valkey database service (valkey, Valkey 7.2 Single mode).
Configure Environment Variables: Set YOUTUBE_API_KEY, GROQ_API_KEY, VALKEY_HOST=valkey, and VALKEY_PASSWORD=${valkey_password}.
Connect GitHub Repo: Triggered the build pipeline directly from my main branch.
Enable Public Subdomain: Flipped the switch for public access on port 8000.

Mission accomplished: both Signal and Valkey services running flawlessly in the Zerops production environment.

Known Limitations

Demand score is directional, not exact. No free data source provides real search volume. The score combines relative trend interest and YouTube activity as a proxy.

pytrends is unofficial. Google Trends has no public API. The library can be rate-limited or break without warning. Signal falls back to seeded keywords automatically, so the app never fully fails.

Avg views can look inflated. Results are sorted by view count, so the top videos tend to be outlier performers — it's a ceiling signal, not a guaranteed expectation for a new channel.

Key Takeaways & Lessons Learned

Never trust external APIs to fail loud: Wrap unofficial endpoints (like pytrends) in daemon threads with hard timeout limits and soft fallbacks.
Decouple build vs runtime environments: Always configure prepareCommands in zerops.yaml for runtime dependencies like gunicorn.
Fail-soft caching is worth every line of code: Adding a thread-safe in-memory cache fallback meant local dev worked without Redis, and production degraded gracefully during network blips.
Zerops makes multi-service infra seamless: Wiring Flask to Valkey over internal networking took literally one environment variable (VALKEY_HOST=valkey).

AI Disclosure: As per hackathon rules, tools like Claude and Antigravity were used as pair-programmers to speed up boilerplate code and draft this post. The core architecture, API logic, and Zerops deployment are my own work.

Try Signal Today

🚀 Live Demo: https://signal-2dc6-8000.prg1.zerops.app
💻 GitHub Repo: github.com/mauryasagar/signal

Built for The Zerops Challenge by WeMakeDevs 2026. Thanks to the Zerops team for building an exceptional developer platform, and a massive thank you to the WeMakeDevs community for hosting this incredible hackathon!

Ouroboros AI: We Made a Fake Agent Break Itself, Then Built Something That Fixes It

Sagar Maurya — Sun, 26 Jul 2026 15:15:12 +0000

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

If you're just landing on this series for the first time, here's the short version of where we've been.

In our first post, we installed a tool called SigNoz for the very first time, so we could watch what a piece of software is doing while it runs — which parts are slow, which parts fail, and why. We built a tiny fake AI agent, and just by looking at a picture of its actions laid out in order, we could instantly see that one single step was eating up 85% of the total time. That was the moment this whole thing clicked for us.

In our second post, we went further. We built a slightly smarter fake agent and asked SigNoz a real question: "does this get slower when it has more information to look through?" We didn't calculate the answer ourselves. We just told SigNoz what to group and average, and it handed us a table proving the pattern, clean as anything.

Both of those were practice. Small, safe experiments before the real hackathon clock started ticking. This post is about what we actually built once it did — and honestly, it's the post we're proudest of, because it's the one where we stopped just watching our fake agent and started building something that does something about it.

We're calling the project Ouroboros AI. Repo's here if you want to poke around: github.com/mauryasagar/ouroboros-ai

The problem that was quietly bugging us

Right after we published that second post, something about it kept nagging at us.

We'd asked SigNoz a good question and gotten a good answer: yes, more documents means a slower response, and it climbs in an almost perfectly straight line. We felt clever about it for about a day. Then we looked at what we'd actually built and realized... nothing happened next. We saw the pattern, said "yep, makes sense," and closed the browser tab. If this had been a real product used by real people, someone would still have had to notice that pattern themselves, write it up, wait for an engineer to have time, and fix it — probably days later, and only if they happened to be looking at the right dashboard at the right moment.

That bothered us more the more we thought about it. Watching something break in slow motion isn't the same as fixing it. So a simple, slightly stubborn question became the whole idea for this project:

What if the system could notice its own problem and fix itself, without a person in the middle at all?

Not "send someone a faster alert." Actually close the loop.

A few words before the story, so nothing feels confusing

We're going to use a handful of words a lot in this post. If you already know them, skip ahead. If you don't, here they are in plain terms:

Trace — a record of everything that happened during one single request, start to finish.
Span — one step inside that trace. Four things happened? Four spans.
Alert — a rule you set up that says "if X happens, tell someone."
Webhook — instead of "tell someone" meaning a text message or an email, it can mean "call this web address automatically." That's the piece that let a computer react instead of a human.
MCP (Model Context Protocol) — a standard way for an AI model to reach out and use real tools — in our case, letting an AI ask SigNoz real questions instead of us clicking around the dashboard ourselves.

That's genuinely all the vocabulary you need. Everything else is just those five ideas, wired together.

What we actually decided to build

We sat down together and sketched this out on a shared doc before writing a single line of code. Five steps, in order:

Chaos — a fake AI agent runs constantly, and every so often, on purpose, it messes up.
Traced — every single step it takes gets recorded and sent to SigNoz, live.
Alert — SigNoz watches all these traces coming in, notices the mess-up pattern, and automatically fires off a webhook — no human clicks anything.
Heal — a separate little program receives that webhook and retries the failed thing with better settings. Still no human.
Explain — an AI "Sidekick" you can literally chat with, which reads the real SigNoz data through MCP and tells you, in plain English, what just happened.

Steps 1 and 2 were a more serious version of what we'd already built in post two. Steps 3, 4, and 5 didn't exist anywhere yet. That's where basically the entire week went.

Step one: giving our fake agent something real to fail at

Our agent from post two never actually failed — it just got randomly slower or faster each time. That's fine for a screenshot, but it's useless for an alert. You can't tell a computer "notice when things feel kind of slow sometimes." You need something crisp: a clear line that gets crossed.

So we gave our agent one specific way to break: context overload. In plain terms — our agent normally looks through somewhere between 1 and 8 "documents" to answer a question. But 20% of the time, on purpose, we let it get buried under 15 documents at once, which is way more than it can handle cleanly.

def retrieve_context(query, is_chaos_mode=False):
    with tracer.start_as_current_span("retrieve_context") as span:
        # CHAOS MODE: 15 docs, NORMAL: 1-8 docs
        context_docs = 15 if is_chaos_mode else random.randint(1, 8)
        span.set_attribute("llm.context_docs", context_docs)
        time.sleep(random.uniform(0.1, 0.4))
        return f"Mock context data ({context_docs} docs)"

We picked this specific failure on purpose, not randomly. Real AI agents that search through documents actually run into this exact problem — feed them too much information at once, and they get slower, cost more, and often don't even answer better for it. We wanted our fake failure to be one that real systems actually have, not something we made up just to have a demo.

When the overload happens, we don't just log it as "a bit slow." We mark it as an actual error, with a clear message SigNoz can watch for:

if context_docs >= 15:
    logging.error(f"CONTEXT_OVERLOAD: Agent failed. Used {tokens} tokens, cost ${round(cost, 6)}, took {delay:.2f}s.")
    span.set_status(Status(StatusCode.ERROR, "Context Overload"))

One small thing from post two turned out to matter a lot here: we'd mentioned back then that just printing something to your terminal doesn't automatically send it to SigNoz — you have to specifically wire up your logs to go there too, separately from your traces. This time, we made sure that was working properly from day one, because our whole alert depended on SigNoz actually being able to see that error message, not just a slightly-slower-than-usual span.

We let this fake agent run forever in the background, quietly sending a new trace every 200 milliseconds, so there was always something fresh happening while we built the rest.

Step three and four: teaching the system to fix itself

This is the part that didn't exist in either of our earlier posts, and honestly, the part we're most excited about.

The good news is, we didn't have to build anything fancy to make SigNoz watch for the failure — it already does that with alert rules, the exact same feature we used in post one to watch for slow calls. The only difference is where the alert points. Instead of sending a Slack message or an email, we pointed it at a webhook — a plain web address that SigNoz calls automatically the moment it sees the problem.

On the other end of that web address, we built a tiny program whose only job is to sit there, wait, and fix things when it gets called:

@app.route('/webhook', methods=['POST'])
def webhook():
    data = request.json or {}
    alert_name = data.get("alertname", "Unknown Alert")
    print(f"🚨 ALERT RECEIVED FROM SIGNOZ: {alert_name}")

    result = optimized_llm_call("Auto-healed request")

    span.add_event("AUTO-HEAL SUCCESS", attributes={
        "message": "Request auto-healed with optimized parameters",
        "status": "healed"
    })
    return jsonify({"status": "healed", "result": result}), 200

The actual fix is simple on purpose — it retries the request with a smaller, safer amount of information instead of the overload amount. The part we spent real time thinking about wasn't the fix itself, though. It was this question: should the fix be invisible, or should it leave a trace of its own?

We almost let it be invisible — quietly patch things up in the background and move on. But that felt like exactly the thing we were trying to get away from in the first place: a fix that only lives in someone's head, not in the system. So we made the healer trace its own retry too, tag it clearly as an auto-heal, and record an event that says, plainly, "this got fixed automatically." Now the healing itself shows up in SigNoz right next to the failure that caused it. Nobody has to remember it happened. The system remembers for you.

Step five: teaching an AI to explain all of it in plain English

Here's where post two's biggest lesson came back around. Back then, we discovered that if you tag your data with the right detail, SigNoz can answer real questions for you — no manual math required. The bigger question that kept nagging at us afterward was: what if you didn't even need to know which button to click, or which filter to type? What if you could just ask, like you're talking to a person?

That question is what led us into MCP — the protocol that lets an AI model reach out and actually use tools, instead of just talking. Neither of us had touched MCP before this hackathon, so before writing any code, we spent a real evening just reading about how it works. That reading turned out to save us a lot of pain later, even though it didn't save us from all of it.

What we ended up building is a small chat app we call the Sidekick. You type a question, and behind the scenes:

You type a question
   → Groq (the AI model) decides which tool it needs
   → it asks SigNoz for the real data through MCP
   → the real numbers come back
   → the AI reads them and answers you in plain English

SigNoz's MCP connection offers over 40 different tools it can use — way more than we needed — so we limited our Sidekick to five: list services, get traces, get logs, get metrics, get one specific span. Simple, focused, and cheap to run.

Getting this actually working took two separate evenings of head-scratching, and both problems were the sneaky kind — nothing crashed, things just quietly didn't work.

The first problem: our AI would just never use any tools. It would answer vaguely instead of actually checking real data, no matter how directly we asked it something like "what services are running right now." We spent a while assuming the AI itself was being stubborn, tweaking our instructions to it, before we finally printed out exactly what we were sending it — and found out the tool descriptions we were passing along were completely empty. Turns out SigNoz describes its tools using a label called inputSchema, while the AI model we were using expects a label called parameters. We were only checking for one of the two. The fix was a single line of code, but finding it meant working backwards through several layers of "this should be working" first:

schema = tool.get("inputSchema") or tool.get("parameters") or {"type": "object", "properties": {}}

The second problem was stranger. The very first request would work perfectly, and then the second one would get flatly rejected — every time. It took a while to notice that the server hands you an invisible little "session ID" the first time you connect, and expects you to send it back on every request after that, like a wristband at an event. Miss that, and it treats you like a stranger who just walked in:

sid = response.headers.get("Mcp-Session-Id")
if sid:
    self.session_id = sid

On top of that, some answers came back in a slightly unusual format made for streaming data, instead of a plain, simple response — so our little connector had to learn to handle both.

None of this felt exciting while it was happening. It was two nights of adding print statements, staring at raw technical details, and slowly ruling things out one at a time. But the moment both fixes were in, the payoff was immediate. Post two's big win was clicking through a filter and a group-by to get a table. This time, the same kind of insight just... showed up, after typing a normal sentence:

"Which services are currently being monitored in SigNoz?"

{
  "answer": "The services currently being monitored are:\n- auto-healer-service — 1 trace in the last hour (avg ~494ms)\n- deep-agent-service — 688 traces in the last hour (avg ~1.63s)",
  "tools_used": ["signoz_list_services"]
}

We kept the AI's instructions short and strict on purpose — answer in a few sentences, use the real numbers you found, don't waste time asking for logins it already has. It's meant to feel like a fast, useful teammate answering a quick question, not a general chatbot rambling.

The moment it actually came together

Here's the part that felt like more than just code working.

We finally let all three pieces run at the same time and just watched. In one window, the fake agent quietly sending traces every 200 milliseconds, like it had been doing for days. In another, the Sidekick sitting there, waiting for a question. And then, in the third window — completely on its own, with nobody typing a single command — the healer suddenly printed:

🚨 ALERT RECEIVED FROM SIGNOZ

We hadn't triggered that. SigNoz had caught the overload pattern by itself, in the live stream of data, and called the webhook on its own.

That's the moment this stopped being a demo we were operating and started being a system that was actually running itself.

About thirty seconds later, we asked the Sidekick what had just happened, half-expecting to have to nudge it toward the right answer. We didn't need to. It described the overload, the automatic retry, and the corrected numbers, pulled straight from the same live data we'd watched land in SigNoz seconds earlier. That was the entire five-step loop, closing itself, start to finish, with nobody standing in the middle of it.

What we're taking away from all three posts

Post one taught us that just seeing where the time goes changes how you think about a problem. Post two taught us that tagging the right detail turns a dashboard into something you can actually have a conversation with. This project is where those two lessons stopped being separate little discoveries and turned into one real habit: instrument things not just to log what's easy right now, but to answer the question you'll actually want answered later — because that same detail is what lets an alert fire cleanly, and what lets an AI explain the whole story back to you without you writing a single extra line of code for it.

If someone asked us what we'd tell ourselves before starting this hackathon, it's that one sentence. Everything else — the failures, the debugging nights, the late realization that our first agent needed to actually break instead of just wobble — all came from following that one idea seriously enough to build around it.

Full code, the landing page, and everything else is in the repo: github.com/mauryasagar/ouroboros-ai

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

We Asked SigNoz a Question, and It Actually Answered

Sagar Maurya — Sun, 19 Jul 2026 16:51:23 +0000

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

Before this, we wrote a first post about setting up SigNoz for the very first time — installing it, sending it some data, and figuring out what all this "tracing" stuff actually means. You can also check that out on LinkedIn. If you're new to any of this, that post explains the basics in a simple way, so it's worth a quick look first.

This post is us going one step further as a team. We wanted to see if SigNoz could actually help us discover something, not just show us pretty charts. It did, and here's exactly what happened, explained simply.

First, two words you need to know

Trace — a record of everything that happened during one single request, from the moment it starts to the moment it finishes.
Span — one step inside that trace. If a request does 4 things, it has 4 spans, one for each step.

That's really all you need to follow along.

What we built

A small fake AI assistant. Instead of doing just one thing, it does four things every time someone asks it a question:

Look up some documents — pretend to search for information related to the question (a random number, somewhere between 1 and 8 documents)
Ask the AI — pretend to send the question and those documents to an AI model and get an answer back
Use a tool — pretend to use something like a calculator, a calendar, or a search tool
Clean up the answer — tidy up the final response before sending it back

We wrapped each of these four steps in code that tells SigNoz "hey, this step just started" and "hey, this step just finished." That's what creates the spans.

Looking at just one question it answered

We ran our fake assistant 30 times, and then picked one single run to look at closely in SigNoz. Here's what it showed us:

The whole thing took 1.72 seconds. Almost all of that — 1.43 seconds — was spent in the "ask the AI" step. Everything else (looking up documents, using a tool, cleaning up) barely took any time at all in comparison. You can literally see this with your eyes on the chart, no reading through code needed.

Then we asked a bigger question

Looking at one run is useful, but we wanted to know something bigger: does the AI step get slower when it has more documents to look through?

In real life, this makes sense — if you ask an AI to read through 8 documents instead of 1, it usually takes longer to answer, because there's more for it to process.

So instead of checking each of our 30 runs one by one, we asked SigNoz to do the work for us:

We told it: "only show me the AI-call steps, ignore the other three"
We told it: "now group these by how many documents each one used"
We told it: "show me the average time for each group"

And within seconds, SigNoz gave us this table:

Documents used	Average time taken
1	568ms
2	658ms
3	825ms
4	999ms
5	1147ms
6	1333ms
7	1488ms
8	1614ms

Look at that — it climbs almost perfectly evenly. Every extra document adds roughly 150 milliseconds to the response time. We built this rule into our code, so finding it wasn't a total surprise, but what genuinely impressed us was how easily SigNoz found and displayed it. We didn't write any extra code, we didn't open Excel, we didn't do any manual math. Just a couple of clicks, and the whole pattern was right there in a table.

We also checked the logs

Traces are great for timing, but sometimes you just want to read a plain sentence about what happened. That's what logs are for. We turned those on too, and got results like this:

Now every request has two views: the trace shows how long things took and in what order, while the logs show what actually happened, in plain readable sentences. Small tip if you try this yourself — just printing messages to your terminal doesn't automatically send them to SigNoz. You need a couple of extra lines of setup code to actually connect them, or nothing will show up in the Logs tab.

What we're taking away from this

The biggest thing we learned: a trace waterfall is great for spotting what's slow. But the real magic happened when we tagged our own extra detail onto a span — in our case, "how many documents did this use." That one small addition is what let us ask SigNoz a real question and get a real answer back, across all 30 runs at once, instead of guessing.

For the actual hackathon build, this is exactly the habit we want to carry forward — don't just time your code, tag it with details that'll actually matter later when something's slow and you're trying to figure out why.

Data Isn't What Your Textbook Said It Was

Sagar Maurya — Sat, 11 Jul 2026 02:13:37 +0000

A beginner-friendly breakdown of what data really is, where it comes from, and why it matters more than you think.

You woke up this morning and checked your phone.

You scrolled through Instagram. Opened WhatsApp. Maybe checked the weather. Ordered breakfast on Swiggy.

By the time you finished your morning tea, you had already generated hundreds of data points — and that was before 9 AM.

We live in a world that runs on data. But most people, including a lot of CS students, have never stopped to ask: what actually is data?

Not the textbook definition. The real one.

And that's exactly the problem — we all learned about data from a textbook. Which means we learned about it wrong.

The Textbook Lie

Ask anyone what data is and they'll say something like:

"Data is raw, unprocessed facts and figures."

Technically correct. Completely useless.

That definition tells you nothing about why data matters, where it comes from, or what makes it powerful. It's the kind of definition written for an exam, not for understanding.

So let's close the textbook and start over.

Data is Just a Record of Something That Happened

Every time something happens in the real world — a click, a purchase, a step you walked, a message you sent — there's a possibility of recording it.

When you record it? That's data.

You clicked on a product → data
Your phone counted 6,400 steps today → data
You paused a YouTube video at 2:34 → data
You left a website after 3 seconds → data (and a headache for that website's owner)

Data is just a trace that something happened. That's it.

The interesting part is what happens after you collect those traces.

Raw Data is Almost Always Ugly

Here's what nobody tells beginners: data in the real world is messy.

Imagine a Google Form that asks for someone's phone number. You'll get responses like:

9876543210
+91-9876543210
98765 43210
9876543210 (call after 6pm)

All four people gave you the same number. But to a computer, these are four completely different values.

This is what's called dirty data — and cleaning it is genuinely one of the most important (and underrated) skills in the data field.

A famous saying in data science is:

"80% of data work is cleaning. The other 20% is complaining about cleaning."

It's a joke. But not really.

The Journey of Data: From Noise to Decision

Here's a simple way to think about how data travels:

Something happens
      ↓
It gets recorded (raw data)
      ↓
It gets cleaned (processed data)
      ↓
It gets analyzed (insights)
      ↓
Someone makes a decision

Let's make this real.

Example: Swiggy wants to know why orders drop on Tuesday evenings.

They collect order timestamps → raw data
They remove duplicates, fix timezone errors → clean data
They spot that Tuesday 7–9 PM has 40% fewer orders → insight
They run a Tuesday evening discount campaign → decision

That entire chain? It all started with someone pressing "Order" on their phone.

Not All Data is Numbers

A common misconception is that data means spreadsheets full of numbers. But data comes in many forms:

Structured data — rows and columns, like a CSV or a database table. Easy for computers to process.
Unstructured data — text, images, audio, video. Harder to process, but far more common in the real world.
Semi-structured data — like JSON or XML. Has some structure, but not rigid rows and columns.

When you send a WhatsApp message, that's unstructured data. When your bank logs a transaction, that's structured data. When you fill a Google Form, that produces semi-structured data.

Most of the world's data — over 80% of it — is unstructured. Which is why fields like NLP (Natural Language Processing) and Computer Vision exist: to make sense of data that doesn't fit neatly into a table.

Your Data is Someone's Product

Here's the uncomfortable truth.

Every free app you use — Instagram, Google, YouTube — is free because you are not the customer. You're the product. More precisely, your data is.

When you like a post, skip an ad, or spend 45 minutes on Reels instead of 5, that behavior is recorded, analyzed, and used to serve you more content that keeps you on the platform longer.

This isn't conspiracy theory. It's just the data pipeline at scale.

Understanding this doesn't mean you need to delete all your apps. But it does mean you should be aware of the trade you're making — your attention and behavior in exchange for a free service.

Why Everyone Should Care About Data (Not Just Data Scientists)

You don't need to work in data to benefit from understanding it.

As a developer — you'll build systems that generate data. Understanding data helps you design better databases and APIs.
As a product person — decisions without data are just opinions. Data makes arguments.
As a user — knowing how your data is used makes you a more informed digital citizen.
As a student preparing for placements — almost every tech company today is a data company in some way. Interviews increasingly involve data thinking, even for SDE roles.

Data literacy is becoming as fundamental as being able to read and write.

So What Did Your Textbook Get Wrong?

Nothing, technically. But everything, practically.

Your textbook gave you a definition of data that would pass an exam. It didn't give you a mental model that would help you think about the world differently.

The difference between a student who "knows data" and someone who understands it is this: one memorized a definition, the other sees data everywhere they look — in the apps they use, the decisions companies make, and the systems they build.

That shift in perspective is what this blog is about. Not definitions. Mental models.

Where to Go From Here

This post was just the surface. In future posts, I'll go deeper into:

How databases store and retrieve data efficiently
What actually happens when you run a SQL query
The difference between a data analyst, data engineer, and data scientist
And some hands-on projects that helped me understand all of this better

If you're a student or someone just getting into tech — I hope this gave you a clearer mental model of what data actually is.

Because before you can work with data, you need to understand what you're actually working with.

Thanks for reading! If you found this useful, drop a reaction or share it with someone who's just getting started in tech.

DEV Community: Sagar Maurya

How I Built a Self-Healing AI Tracker That Fixes Its Own Scrapers

The Problem I Wanted to Solve

What Crawlr Tracks

How Bright Data Scraper Studio Powers Everything

The Moment That Made It Click

The Architecture

1. The Pipeline

2. Sentry — The Health Agent

3. The Dashboard

What a Full Run Looks Like

What I Learned

Try It

How I Built a YouTube Trend Engine on Zerops

The API Quota Problem

What Signal Does

System Architecture: The 4-Layer Engine

The Tech Stack

Project Structure

Technical Deep-Dive: Code & Logic

1. Resilient Ingestion & Daemon Thread Caps (fetch_trends.py)

2. The Logarithmic Demand Scoring Model (report.py)

3. Fail-Soft Caching Layer with Zerops Valkey (cache.py)

The Frontend: Dynamic Report Cards

Three Sneaky Roadblocks (And How I Fixed Them)

🐛 Roadblock #1: The Zerops Container Runtime Ghost

🐛 Roadblock #2: The Missing Valkey Auto-Password

🐛 Roadblock #3: Silent LLM JSON Output Formatting Drift

Telemetry & Verification: The /health Endpoint

Production Performance Benchmarks

Step-by-Step Zerops Deployment Walkthrough

Known Limitations

Key Takeaways & Lessons Learned

Try Signal Today

Ouroboros AI: We Made a Fake Agent Break Itself, Then Built Something That Fixes It

The problem that was quietly bugging us

A few words before the story, so nothing feels confusing

What we actually decided to build

Step one: giving our fake agent something real to fail at

Step three and four: teaching the system to fix itself

Step five: teaching an AI to explain all of it in plain English

The moment it actually came together

What we're taking away from all three posts

We Asked SigNoz a Question, and It Actually Answered

First, two words you need to know

What we built

Looking at just one question it answered

Then we asked a bigger question

We also checked the logs

What we're taking away from this

Data Isn't What Your Textbook Said It Was

A beginner-friendly breakdown of what data really is, where it comes from, and why it matters more than you think.

The Textbook Lie

Data is Just a Record of Something That Happened

Raw Data is Almost Always Ugly

The Journey of Data: From Noise to Decision

Not All Data is Numbers

Your Data is Someone's Product

Why Everyone Should Care About Data (Not Just Data Scientists)

So What Did Your Textbook Get Wrong?

Where to Go From Here

1. Resilient Ingestion & Daemon Thread Caps (`fetch_trends.py`)

2. The Logarithmic Demand Scoring Model (`report.py`)

3. Fail-Soft Caching Layer with Zerops Valkey (`cache.py`)

Telemetry & Verification: The `/health` Endpoint