DEV Community: Sagar Maurya

I Built an Offline AI File Organizer for a Friend | Hacktoberfest 2026

Sagar Maurya — Mon, 05 Oct 2026 06:54:14 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

My friend's Downloads folder is a dumping ground: lecture notes, an invoice, code files, installers, images, all mixed together under names like doc_final and scan. Extension-based sorters put every PDF in one folder, and cloud AI tools would mean uploading her private files to someone else's server.

So I built Organizr, an offline file organizer that sorts files by what's inside them, not by their extension. It reads each file with a local Gemma model, proposes a folder and a short reason, and moves nothing until you click Apply. Every move can be undone.

Demo

The video has no voiceover (I have a cough), so it has English subtitles. The whole run is with Wi-Fi turned off.

I made a 20-file sample Downloads folder from the file names she sent me, using stand-in names because her real folder has IDs and certificates. Here is what Organizr did:

12 files sorted with confidence. For example, a C assignment went to Assignments and an invoice to Fee Receipts, based on the text inside.
8 files flagged "Please check this one": images, installers and a video, which have no readable text.
Apply moves the files, and Undo last move puts everything back.

What my friend said

I sent her the plan and the result screenshots and asked for her reaction. She replied: "this is actually cool 😮 my lecture notes and the invoice went where I'd put them.
timetable in Lecture Notes is fine too. I'd try it on my laptop..."

She said the lecture notes, the invoice and the timetable landed where she'd have put them, and that she'd try it on her laptop. She hasn't run it on her own folder yet, and she didn't comment on the files it got wrong, so that is all I'm claiming.

Where it got things wrong

Gemma put a LinkedIn strategy document and a research CSV in Lecture Notes. Both are wrong. The plan view lets you change any category before applying.
Images, installers and videos can't be read, so they're flagged for review. Earlier in testing, Gemma made up a reason for a file with no text. OCR isn't included yet.
Gemma's wording changes from run to run.

Code

mauryasagar / ai-file-organizer

Offline AI file organizer that sorts files by content, not extension, using local Gemma via Ollama. Private, undoable, no uploads.

🗁 Organizr

An offline AI file organizer that sorts your files by what's inside them, not by their extension.
It reads each document with a local open-weight model (Gemma via Ollama), proposes where it belongs,
and moves files only after you approve. Your private files never leave your laptop

Problem · Solution · How it works · Getting started · Structure · Limitations

🎃 Built for the DEV Hacktoberfest Weekend Challenge: Build for a Friend.

The Problem

Downloads folders turn into a dumping ground. This project started with a friend whose folder mixed lecture PDFs, fee receipts, assignment files, scholarship letters and random images, all with names like doc_final.pdf and scan (3).pdf.

Existing options did not fit:

Extension-based sorters put every PDF in one folder, so a fee receipt lands next to lecture notes.
Cloud AI tools would mean uploading ID scans, marksheets and bank statements to someone else's…

View on GitHub

How I Built It

Python, Streamlit, Ollama running gemma3:4b, PyMuPDF and python-docx.

scanner.py finds safe files, skipping unfinished downloads and files changed in the last 5 minutes.
extractor.py reads the first ~1000 characters of text from each file.
classifier.py sends the content (not the filename) and her list of categories to Gemma, which returns JSON with a category and a reason. Anything invalid falls back to "Other".
organizer.py builds the plan, moves files without ever overwriting or deleting, and writes an undo log.

I tested gemma3:1b and gemma3:4b on six files with misleading names. The 1B model got 4 of 6 right and the 4B model got 5 of 6, so I chose 4B. It runs on CPU on a 16 GB laptop in a few seconds per file. Removing the filename from the prompt improved the results, because the model stopped trusting names like doc_final.

Why Does Open Innovation Matter?

Her folder holds ID scans, marksheets and bank documents. With a closed API, sorting it would mean uploading all of that. With an open-weight model:

Her data stays on her laptop. The demo runs with Wi-Fi off.
It costs nothing to run after the model download, with no API keys and no per-request billing.
The model is swappable. Changing it is one line in classifier.py, which is how I compared 1B and 4B.

Prize Categories

Best Use of Gemma: the whole classifier runs on a local Gemma model (gemma3:4b) through Ollama.

Building a Student Marks Dashboard: From Data to Live Web App

Sagar Maurya — Fri, 02 Oct 2026 10:06:21 +0000

Most pandas tutorials teach you syntax. They show you .mean(), .groupby(), .apply(). But they don't show you how the pieces connect — how a CSV becomes a chart, how a chart becomes a web app, how a web app ends up live on the internet.

I wanted to see that full pipeline for myself. So I built a small project: a student marks dashboard that reads a CSV, calculates results, plots a chart, and runs as a live web app.

The data

Eight students, three subjects:

name,maths,science,english
Arjun,78,85,72
Priya,92,88,95
Rahul,45,50,40
Sneha,88,91,84
Vikram,30,35,28
Anita,67,72,70
Karan,55,60,58
Meera,95,93,97

Small and clean. That's on purpose — the goal was to understand the pipeline, not to wrestle with messy data.

Loading a CSV into a DataFrame

import pandas as pd

df = pd.read_csv("data/marks.csv")
print(df)

Output:

      name  maths  science  english
0    Arjun     78       85       72
1    Priya     92       88       95
2    Rahul     45       50       40
3    Sneha     88       91       84
4   Vikram     30       35       28
5    Anita     67       72       70
6    Karan     55       60       58
7    Meera     95       93       97

pd.read_csv() returns a DataFrame — basically a table in memory. Each row has an index (0–7) and each column is a Series.

At this point it's just numbers. No totals, no averages, no conclusions.

Creating new columns

df["total"] = df["maths"] + df["science"] + df["english"]
df["average"] = (df["total"] / 3).round(2)
print(df)

Output:

      name  maths  science  english  total  average
0    Arjun     78       85       72    235    78.33
1    Priya     92       88       95    275    91.67
2    Rahul     45       50       40    135    45.00
3    Sneha     88       91       84    263    87.67
4   Vikram     30       35       28     93    31.00
5    Anita     67       72       70    209    69.67
6    Karan     55       60       58    173    57.67
7    Meera     95       93       97    285    95.00

When you add two columns in pandas, it works row by row automatically. df["maths"] + df["science"] + df["english"] adds the three marks for each student and returns a new Series. Assigning it to df["total"] creates a new column.

.round(2) keeps the average to two decimal places.

Raw marks don't say much on their own. A total and an average turn three numbers into a single metric you can sort and compare.

When the rule defines the result

Define "Pass" as: 40 or more in every subject.

df["result"] = df[["maths", "science", "english"]].apply(
    lambda row: "Pass" if all(row >= 40) else "Fail", axis=1
)
print(df[["name", "average", "result"]])

Output:

      name  average result
0    Arjun    78.33   Pass
1    Priya    91.67   Pass
2    Rahul    45.00   Pass
3    Sneha    87.67   Pass
4   Vikram    31.00   Fail
5    Anita    69.67   Pass
6    Karan    57.67   Pass
7    Meera    95.00   Pass

Line by line:

df[["maths", "science", "english"]] selects only the three subject columns.
.apply(..., axis=1) runs a function on each row. axis=1 means row-wise.
lambda row: ... is a small anonymous function. row is one student's three marks.
all(row >= 40) returns True if every mark is 40 or more.
The ternary "Pass" if ... else "Fail" returns the label.

On this dataset, "every subject ≥ 40" and "average ≥ 40" happen to give the same answer. That's luck. A student with 95, 95, and 20 would pass on average but fail my rule.

The definition of "pass" is a decision. I made it before writing any code, and it shaped every result after.

What the data showed

Looking at the finished table:

Topper: Meera, average 95.00
Class average: 69.5
Only failure: Vikram, average 31.00
Borderline: Rahul scrapes through at 45.00

Eight rows is small, but the shape of the class is already visible. One clear topper, one clear failure, and a wide middle.

Visualizing with matplotlib

import matplotlib.pyplot as plt

plt.figure(figsize=(8, 5))
plt.bar(df["name"], df["average"], color="steelblue")
plt.title("Average Marks per Student")
plt.xticks(rotation=45)
plt.tight_layout()
plt.savefig("outputs/average_marks.png")

plt.bar() draws the bars, xticks(rotation=45) stops the names from overlapping, and savefig() writes the chart to a file. Meera's bar is the tallest. Vikram's is the shortest. That's obvious at a glance, much faster than reading the table.

Making it usable by anyone

The script works. But only I can run it, only on my laptop, only with Python installed.

Streamlit turns a Python script into a web app. Here's the full app.py:

import streamlit as st
import pandas as pd

st.title("Student Marks Analysis")

df = pd.read_csv("data/marks.csv")

df["total"] = df["maths"] + df["science"] + df["english"]
df["average"] = (df["total"] / 3).round(2)

df["result"] = df[["maths", "science", "english"]].apply(
    lambda row: "Pass" if all(row >= 40) else "Fail", axis=1
)

st.dataframe(df)

st.subheader("Average Marks Chart")
st.bar_chart(df.set_index("name")["average"])

st.title() adds the heading.
st.dataframe(df) renders the DataFrame as an interactive table — users can sort columns by clicking headers.
st.bar_chart(...) renders the chart in the browser. set_index("name") makes student names the X-axis labels.

The pandas logic is identical to the script. Streamlit just converts the output into HTML.

Running it locally:

python -m streamlit run app.py

I had to use python -m streamlit instead of just streamlit. I'd installed it correctly, but Windows didn't know where the executable was, so it kept saying "not recognized." Running it as a Python module sidesteps the PATH issue.

Deploying to the cloud

Streamlit Community Cloud is free and connects directly to GitHub. Sign in, pick the repo, select app.py, click Deploy. It installs dependencies from requirements.txt and gives you a public URL in about two minutes.

One thing went wrong. I'd edited the README on GitHub's website while also editing it locally, so git push was rejected — the remote had changes I didn't have. I fixed it with:

git pull origin main
git push

Git opened Vim to write a merge message. I'd never seen Vim before and had no idea how to get out. Took me a few minutes of searching to find that :wq saves and quits. Small thing, but it's the part I remember most clearly.

Project structure

By the end, the folders looked like this:

student-marks-analysis/
├── data/
│   └── marks.csv
├── outputs/
│   └── average_marks.png
├── src/
│   └── analysis.py
├── app.py
├── requirements.txt
└── README.md

Data in data/, scripts in src/, generated files in outputs/, the web app at the top level. When someone opens the repo, they immediately know what's where.

Dependencies:

pip install pandas matplotlib streamlit

What I'd improve

Use a real dataset. 8 rows with no missing values, no typos, no duplicates. Real data is messier, and cleaning is where half the work lives.
Add filters. A st.selectbox() for subjects or a slider for marks would make the app actually interactive.
More charts. Subject-wise comparison, or a trend over time if the data had dates.

These aren't flaws. They're the next things to learn.

Summary

The pipeline, end to end:

Load the CSV with pandas.
Transform by creating columns and applying rules.
Visualize so patterns become obvious.
Ship as a web app and deploy it.

Most tutorials cover steps 1 and 2.
Steps 3 and 4 are where a script becomes something other people can actually use.

It's a small project. But now I get how the pieces connect.

Links

Live dashboard

GitHub Repo

mauryasagar / student-marks-analysis

Interactive data analysis dashboard for student marks, built with Python, pandas, matplotlib, and Streamlit. Deployed live via Streamlit Cloud.

Student Marks Analysis

Problem

Analyze a dataset of student marks to identify the topper, calculate the class average, determine pass/fail results, and visualize performance.

Dataset

A small dataset of 8 students with marks in Maths, Science, and English.

Key Findings

Topper: Meera with an average of 95.00
Class Average: 69.5
Failed Students: 1 (Vikram)
Pass Rule: A student passes only if they score 40 or more in every subject

Chart

Live Dashboard

View Live Dashboard

To run it locally:

python -m streamlit run app.py

Tools Used

Python
pandas (data loading and analysis)
matplotlib (visualization)
Streamlit (interactive dashboard)

Project Structure

student-marks-analysis/
├── data/
│   └── marks.csv          # Raw student marks data
├── outputs/
│   └── average_marks.png  # Generated bar chart
├── src/
│   └── analysis.py        # Analysis script
├── app.py                 # Streamlit dashboard
├── .gitignore
├── LICENSE
├── README.md
└── requirements.txt

How to Run

Install dependencies:
```
pip install pandas matplotlib streamlit
```
…

View on GitHub

Three Days, One Tool, and Every Bug Was Hiding Another One — MindMap Debugger, the Full Story

Sagar Maurya — Mon, 21 Sep 2026 10:52:25 +0000

Building MindMap Debugger for AWS First Commit by #WeMakeDevs — from the first working extraction to the bugs I found the night before submission.

What it does

MindMap Debugger takes any argument or transcript you paste in, and finds two things in it:

Contradictions — two things that were said which can't both be true.
Circular reasoning — a chain of claims that loops back and ends up using itself as its own proof.

The idea came from a simple observation: reading carefully doesn't scale. A meeting transcript says the deadline is fixed. Ten minutes later someone says there isn't time to hit it. Nobody flags it because everyone's listening for their own part, not tracking the whole argument. The same thing happens in design docs, contracts, long debates — claims pile up faster than any one reader can cross-check them.

Day 1 — Getting something to work at all

I started with the basic shape: paste text in, call an LLM to extract "propositions" (claims) and "relations" (how those claims connect — supports, depends_on, contradicts), then run detection logic over the relations to find contradictions and cycles.

For the model backend, I used Groq — free, no card required, which mattered because I don't have a debit or credit card and UPI isn't accepted by AWS account signup. That also meant pivoting from the "Ship It" track (needs a real AWS account) to "Build It" (no AWS account required). To still satisfy the "use AWS-native tooling" requirement, I wrapped the Groq call in AWS's Strands Agents SDK — Strands is model-agnostic, so it happily talks to Groq through an OpenAI-compatible endpoint even though Groq isn't Bedrock.

The first real bug showed up almost immediately: I was using gpt-oss-120b, a reasoning model, and its internal chain-of-thought was leaking straight into the output instead of staying hidden. Instead of clean JSON, I'd get a wall of the model "thinking out loud" followed by the actual answer buried somewhere inside.

The fix was passing reasoning_format=hidden and reasoning_effort=high through Strands' extra_body parameter — plain response_format alone and a top-level reasoning_format both failed silently. extra_body was the only path that actually worked.

By the end of Day 1, the pipeline could take a paragraph, extract claims, detect an obvious contradiction, and show it in a rough UI. It felt like a huge milestone. It was also, I'd learn the next day, held together with tape.

Day 2 — Every bug I fixed was hiding another one

Day 2 started with a simple question: does this work reliably, or did I just get lucky once?

I ran the same input five times in a row. The number of findings bounced between 2 and 6. Same text, same prompt, same everything — wildly different output. That's when I learned Groq's serving of gpt-oss-120b is genuinely non-deterministic across calls, even with low temperature. A single extraction call just isn't trustworthy on its own.

The fix was consensus extraction: run the same extraction three times, then merge the results. In theory this smooths out the noise — if two of three runs agree on a relation, that's more trustworthy than any single run.

In practice, merging three noisy outputs together doesn't just cancel the noise. It creates new problems.

Bug: paraphrased duplicates weren't merging. The model would phrase the exact same claim slightly differently between calls — "the deadline is fixed" in one run, "the deadline cannot move" in another. My merge logic was doing exact text matching, so these became two separate propositions instead of one. I added a similarity check: if two propositions shared enough words, treat them as the same claim and merge them, keeping the higher-confidence version.

Bug: cycle detection had a blind spot. My find_cycles() function only built graph edges from depends_on relations. But a circular reasoning chain in the wild doesn't neatly alternate through one relation type — a loop might go depends_on → supports → depends_on back to the start. Because I was only looking at one edge type, I was missing real cycles that used a mix. Fixed by unioning both relation types when building the graph for cycle detection.

Bug: duplicate findings. The cycle-finding DFS could rediscover the exact same cycle starting from a different node in the loop — so a single 3-claim circular chain would get reported three times, once per starting point. Added a dedup step that collapses cycles with identical node sets, regardless of which node the search happened to start from.

By the end of Day 2, I had what looked like a solid pipeline: consensus extraction, similarity-based proposition merging, cross-relation-type cycle detection, deduped findings, and a Cedar policy gate on top (more on that below). I tested it, it looked right, and I moved on to building out the UI.

I should have kept testing longer.

Day 3 — The bugs the Day 2 fixes were hiding

This is the part of the story I almost didn't catch, and it's the most important part.

The pipeline "worked." But something felt off when testing edge cases, so instead of just looking at the UI, I went back to reading the raw console logs from pipeline.py — the same habit that had caught the Day 2 bugs.

The pipeline "worked." But something felt off when testing edge cases, so instead of just looking at the UI, the session went back to reading the raw console logs from pipeline.py — the same habit that had caught the Day 2 bugs.

Bug #1: the similarity merge was too aggressive

Take these two propositions from a witness-testimony test case:

P1: "the witness is reliable because their testimony is consistent"
P2: "their testimony is consistent because they are telling the truth"

These are about different things — P1 is about the witness's reliability, P2 is about their truthfulness — but they share a lot of common English scaffolding: "the," "is," "because," "their," "testimony," "consistent."

The old similarity formula was overlap divided by the smaller set's size. With 6 shared words and a smaller set of 8 words, that's 6/8 = 0.75 — well above the 0.7 merge threshold. The two distinct claims got collapsed into one.

The fix was switching to Jaccard similarity — overlap divided by the union of both sets, not just the smaller one:

overlap = len(words_a & words_b)
union   = len(words_a | words_b)
return overlap / union

Same two propositions: 6 / 12 = 0.5 — below threshold, correctly kept separate. The union denominator punishes cases where two sentences only look similar because they both use common connector words. Genuine paraphrases (same core claim, same key content words) still merge fine; superficially similar-but-different claims no longer do.

After this fix, a 4-sentence test input that was collapsing into 3 propositions correctly produced 4. The fake circular finding (a nonsensical two-node "reliable → reliable" self-loop) disappeared, and the real 3-claim circular chain in that same text — reliable → consistent → truthful → reliable — was still correctly detected. The fix didn't suppress findings. It made them honest.

Bug #2: the model was directionally inconsistent, and merging fabricated cycles

This one was sneakier. Testing a "bridge maintenance" sample designed to have exactly one contradiction and zero circular chains, the tool reported one contradiction — correct — plus four fake circular chains.

The console logs showed why:

Run 1: P4 depends_on P5, P5 depends_on P6
Run 2: P5 depends_on P4, P6 depends_on P5
Run 3: P4 depends_on P5, P5 depends_on P6

The model was expressing the same underlying relationship with the arrow pointing in different directions across runs. "Regular maintenance requires a budget" got labeled maintenance depends_on budget in two runs and budget depends_on maintenance in the third. Individually, none of the three runs contained a cycle — each one's relations were internally consistent. But the merge step was unioning all edges from all three runs, and once you union P4→P5 from one run with P5→P4 from another, you've created a 2-node loop that never existed in any single reasoning chain. The cycle detector, doing its job correctly, found that loop and flagged it.

The fix: after merging relations, prune reverse-direction depends_on pairs, keeping only the higher-confidence direction:

for (from_t, to_t, rel_type) in list(all_rels.keys()):
    if rel_type != "depends_on":
        continue
    reverse = (to_t, from_t, rel_type)
    if reverse in all_rels:
        if all_rels[reverse]["confidence"] > all_rels[(from_t, to_t, rel_type)]["confidence"]:
            all_rels.pop((from_t, to_t, rel_type))
        else:
            all_rels.pop(reverse, None)

Two details mattered here. First, this only applies to depends_on — a bidirectional supports relation ("A supports B and B supports A") is genuinely circular reasoning by definition, so those are left alone. Second, when both directions exist, keep whichever one the model was more confident about, rather than just picking one arbitrarily.

After this fix, the bridge sample dropped from 1 contradiction + 4 fake cycles to exactly 1 contradiction, 0 circular — correct. And the witness sample still correctly reported its real 3-claim cycle, unaffected, because that cycle used consistent-direction edges across all three runs.

That's the actual engineering payoff of this whole project, more than the UI or the graph: a merge layer that's honest about disagreeing with itself instead of quietly averaging its way into fabricated findings.

Here's what the fully-fixed pipeline looks like live in the browser, on the product-launch sample from the top of this post — real contradictions and real circular chains, no fakes mixed in:

The smaller bug: a white flash on page load

Separately, on a hard refresh, the hero section's 3D constellation would flash as a plain white box for a fraction of a second before rendering. Browsers paint a <canvas> element with a default white backing store until WebGL actually attaches and draws to it — and on a cold load, there's a small race between the DOM painting and the JS clearing the canvas.

The fix was to keep the canvas invisible until the very first real frame has rendered:

<canvas id="hero-canvas" style="opacity:0;background:transparent;transition:opacity .5s ease"></canvas>

requestAnimationFrame(() => {
  renderer.render(scene, camera);
  canvasEl.style.opacity = '1';
  animateHero();
});

Small fix, but it's the kind of "everyone who's touched WebGL knows this" gotcha that never gets written down anywhere obvious.

What I actually learned

LLMs aren't just noisy in what they say — they're noisy in which direction they say it. Two runs can both be individually correct and still disagree in a way that, if merged carelessly, manufactures a finding that never existed in either run alone. If you're combining multiple LLM calls to reduce noise, you have to explicitly check for this, because naive merging can create errors that are more confident-looking than the noise you were trying to remove.

Similarity metrics have real semantics, not just a threshold to tune. Overlap-over-smallest-set measures "how much of the smaller sentence is contained in the larger one." Jaccard similarity measures "how similar are these two sets overall, penalizing everything that's different." For deduplication, you almost always want the second one — the first one silently favors merging short, generic-sounding text with anything that shares its common words.

"The UI shows correct-looking output" is not the same as "the output is correct." The pipeline ran successfully the entire time on Day 2 and looked done. It just quietly produced wrong answers on certain inputs, and the only way to catch that was going back to raw console logs and deliberately trying to break it with adversarial test cases, instead of trusting that a clean-looking UI meant a clean-working pipeline.

Design is mostly subtraction. Not a pipeline lesson, but a real one from rebuilding the UI multiple times: every glow effect, every drop-shadow, every "make it look like the hero section" pass started too strong and ended up dialed back. The final cursor-following border glow on the dashboard cards is a 3px line with one radial gradient. Earlier versions had two stacked shadows and a masked pseudo-element and looked like a neon sign.

What the tool does, end to end

Paste text with claims in it.
Extraction runs three times through Groq via the Strands Agents SDK.
Propositions are merged by Jaccard similarity — genuine paraphrases collapse, distinct claims survive.
Relations are unioned across runs, deduped by confidence, and reverse-direction depends_on pairs are pruned so the merge can't fabricate cycles.
Contradictions are found by scanning contradicts edges.
Circular reasoning is found via depth-first search over depends_on and supports edges, with duplicate cycles collapsed.
Everything is gated through a Cedar policy: contradictions always surface regardless of confidence; circular findings only surface above 0.6 confidence.
Findings render as a 3D relation graph and a plain-language summary — no jargon, just "these two things can't both be true" and "this loops back on itself."

Repo and stack

Repo: github.com/mauryasagar/mindmap-debugger — MIT licensed, runs locally:

pip install flask strands-agents openai
python app.py

Then open http://localhost:5000.

Track: WeMakeDevs × AWS First Commit, Build It
Stack: Strands Agents SDK + Cedar (the required AWS-native tools), Groq gpt-oss-120b, Flask, Three.js

Why Strands and Cedar, specifically

This was built for the Build It track, which asks for AWS-native tooling — but I didn't want to bolt on an AWS SDK just to check a box, so here's the actual reasoning for each one.

Strands Agents SDK is the only thing standing between this project and Bedrock. I don't have a card to set up billing on a real AWS account, so Bedrock itself was off the table. Strands is model-agnostic by design, which meant I could point it at Groq's OpenAI-compatible endpoint instead and still get a real, working AWS agent framework doing the actual work of calling the model, handling the response, and giving me a structured place to add tools later if the project grows. It's not a token integration — every extraction call in the pipeline runs through it.

Cedar does the actual gating. Contradictions always surface regardless of confidence, because a false contradiction is still worth a human's attention — better to over-flag than hide something real. Circular reasoning only surfaces above 0.6 confidence, because a weak circular claim is more often noise than signal. That's a real policy decision, not a default, and Cedar is what makes it declarative and auditable instead of an if statement buried in detect.py.

Between the two, they cover the actual shape of what "Build It" is asking for: a real agent framework driving the reasoning, and a real policy layer deciding what a user sees.

GitHub Repo: github.com/mauryasagar/mindmap-debugger

Also read:
Day 1 blog (Dev.to): I hit my first contradiction before building one
Day 2 blog (Dev.to): Every bug I fixed today was hiding another one
Day 3 blog (AWS Builder Center)

Built with Claude and DeepSeek as coding partners.

I set out to build something that catches contradictions. Turns out the biggest one was mine: thinking it was finished.

Every Bug I Fixed Today Was Hiding Another One

Sagar Maurya — Sat, 19 Sep 2026 15:18:19 +0000

Building MindMap Debugger for AWS First Commit — Day 2

On Day 1, I got the first working version running — paste in an argument, and the tool reads through it and points out contradictions, powered by Groq's API. It worked. And in the process of testing it, I ran straight into a contradiction of my own, before I'd even finished building the thing meant to catch them.

Yesterday ended with a working extraction pipeline and a deceptively simple plan for today: wrap it in Strands Agents SDK, build the Cedar policy gate, wire it all together, done by evening.

That plan lasted about twenty minutes — turns out the tool wasn't the only thing finding contradictions today. My own assumptions kept getting caught out too.

Bug #1: The AI started thinking out loud inside my JSON

First task — swap the raw Groq client for Strands' Agent wrapper, since Build It's "use AWS-native tooling" requirement means the model call itself needs to route through Strands, not straight to Groq. The SDK was already installed. My extraction logic already worked. This should have been a clean drop-in replacement.

Instead, the model started returning this:

"We need to extract propositions. Two statements: 'SQLite is fast enough for our needs.' and 'the system must survive process restarts...' Could split second into two claims? ... Now check pairwise contradictions. P1: SQLite is fast enough for our needs. No contradiction. P2: System must survive process restarts. No contradiction..."

— followed, eventually, buried at the very end of several paragraphs of visible internal monologue, by the actual JSON I'd asked for.

What was actually happening: openai/gpt-oss-120b is a reasoning model. It always narrates its thinking process, and nothing in Strands' default parameters was telling it to suppress that at the API level. My JSON-extraction fallback could technically still dig the {...} block out of the noise, but it was one weird sentence away from breaking completely.

Fixing it took three attempts, and two of them were wrong in instructive ways:

Adding response_format: {"type": "json_object"} alone — no change. Still leaking.
Adding a top-level reasoning_format: "hidden" parameter — this didn't just fail quietly, it crashed outright: AsyncCompletions.create() got an unexpected keyword argument 'reasoning_format'. Strands' OpenAI-compatible wrapper validates params against a fixed list and rejects anything it doesn't recognize.
The actual fix — nest it inside extra_body instead, so Strands passes it through as a raw field straight to Groq's API rather than checking it against OpenAI's own parameter list:

params={
    "temperature": 0.2,
    "response_format": {"type": "json_object"},
    "extra_body": {
        "reasoning_format": "hidden",
    },
},

Clean JSON, first try, no rambling. First win of the day.

Bug #2: The contradiction that refused to be found

Fixing the leak didn't fix the actual reasoning underneath it. My test sentence — "SQLite is fast enough for our needs. But the system must survive process restarts, and in-memory data does not survive restarts." — contains a genuine contradiction. Using SQLite in-memory conflicts with surviving a restart. But nothing in the wording says so directly; you have to connect three separate facts to see it.

The model kept extracting all three propositions perfectly. It never once flagged the conflict.

I tried two fixes, and both taught me something by failing:

Adding a more explicit prompt rule, spelling out that a "solution" claim can contradict a "requirement" claim even with zero shared wording. Result: worse. The model went from finding one weak relation to finding none at all — the extra instruction diluted the one rule that actually mattered instead of reinforcing it.
Cranking reasoning_effort to "high", forcing more internal reasoning before answering. Result: no change whatsoever. Same weak relation, same missing contradiction.

This is the part that actually surprised me: I'd assumed harder reasoning settings would obviously help with harder reasoning problems. They didn't move the needle at all. So I stopped guessing and looked up whether this was a known thing — and it is. Multi-hop implicit contradictions, where the conflicting facts are spread across separate premises with no shared vocabulary connecting them, are a documented, genuinely hard case for large language models in general. It's not unique to this model, this SDK, or my prompt. It's a real limitation of how these models reason.

The decision that mattered more than any prompt tweak: stop fighting one adversarial sentence. Accept it as an honest, documented limitation instead of a bug to keep chasing, and pick clearer demo inputs where the model's actual strength — catching direct, explicit contradictions — gets to shine. Trying to force a model past its genuine reasoning limits, on a deadline, is a worse use of time than being honest about where those limits are.

Bug #3: Rebuilding a policy that no longer existed

Time to move to Cedar — the AWS-native policy engine at the heart of the Build It track's story. Except the actual rules I'd designed for it — which findings get shown, under what conditions — had lived entirely in a conversation that had already expired. No notes. Nothing saved. The design was just gone.

Rebuilding it meant re-deciding, from first principles, what the policy should actually reward. The real question wasn't technical — it was what earns trust in front of judges:

Direct contradictions: always surfaced, no matter the confidence score. Missing a genuine contradiction is worse than showing one that's slightly uncertain — and contradictions are the entire point of the tool.
Circular reasoning: only surfaced above 0.6 confidence. A shaky contradiction is still useful context for a reader. A shaky circularity claim just looks like the tool crying wolf. Being selective here signals judgment, not just pattern-matching.

That became the actual Cedar policy:

permit (
    principal == Service::"Detector",
    action == Action::"surface_finding",
    resource
)
when {
    resource.type == "direct_contradiction"
};

permit (
    principal == Service::"Detector",
    action == Action::"surface_finding",
    resource
)
when {
    resource.type == "circular" &&
    resource.confidence >= 0.6
};

With a Python enforcement layer (cedar_gate.py) mirroring the same logic, since standing up the full Cedar authorization engine was heavier setup than the timeline allowed — but the actual policy spec above is the real, canonical rule set, honestly documented as such rather than hidden.

First smoke test, using two contradictions (one strong, one deliberately weak) and two circular findings (one strong, one deliberately weak):

Three of four findings passed through. Both contradictions surfaced — even the weak one, exactly per policy. The strong circular finding passed. The weak one got correctly denied. Real, demonstrable policy logic — not "show everything the model said and hope it looks intentional."

Bug #4: The wall at the end of the day

Wiring apply_gate() into the pipeline was one import and one function call — the smallest change of the day. Then I ran the full pipeline against real input for the very first time: a longer, five-sentence argument about launching a product under conflicting pressure, instead of my short toy test sentence.

It broke immediately:

openai.APIError: Failed to validate JSON. Please adjust your prompt.
See 'failed_generation' for more details.

failed_generation, the field that's supposed to show you the broken output, came back completely empty. No malformed JSON to inspect. No partial text. Just a flat wall.

I raised max_tokens to rule out a truncation issue — no change, which at least told me it wasn't simply running out of room. I added debug logging to print the raw error body directly from the API response, ready to finally see what Groq was actually rejecting.

And that's where the day ended — mid-investigation, with the fix still unknown.

The twist: it wasn't the bug I thought it was

Picking the debugging back up, the actual cause turned out to have nothing to do with malformed JSON at all. It was a token budget death spiral: reasoning_effort: "high" combined with strict JSON mode was quietly consuming the model's entire output budget on hidden internal reasoning for this longer, more tangled input — leaving zero tokens left to actually write the answer. Not truncation in the way I'd assumed; raising max_tokens on its own hadn't touched it, because the problem wasn't the ceiling, it was what was eating the budget underneath it. The real fix was dropping reasoning effort to "medium" and raising the token ceiling together, at the same time.

Fixing that immediately uncovered a second, completely different bug hiding behind it: my circular-reasoning detector only ever looked for cycles built from one specific relation type. Real circular arguments, it turns out, don't always stay consistent — a claim can "depend on" another, which in turn "supports" the first, and that's just as circular as two matching relations, but my detector was blind to the mix. A one-line fix — treating both relation types as cycle-relevant — and circular reasoning started showing up for the first time all night.

Fixing that revealed a third bug: to make extraction more reliable (since a single pass sometimes missed things), I started running it multiple times and merging the results — a deliberate reliability upgrade, not a bug fix. But that upgrade introduced its own bug: the same circular reasoning chain occasionally got reported twice, because the underlying graph search could rediscover an identical cycle starting from two different entry points. Fixed with a dedup pass that collapses cycles representing the same set of claims into one.

Three real, distinct root causes, each one hiding behind the last, like debugging nesting dolls. None of them were the bug I originally thought I was chasing.

Where it stands now

Full pipeline, verified end-to-end: paste an argument in, get contradictions and circular reasoning chains out, filtered through real policy logic, rendered live in the browser. On one real test case, the system now reliably catches multiple genuine contradictions and several distinct circular reasoning chains — with zero duplicates.

Tech stack, as it stands:

Python — core pipeline
Groq API — model backend, free tier
Strands Agents SDK — AWS-native model wrapper
Cedar policy language — real policy spec, enforced faithfully
Flask — UI, confirmed working live

Four bugs today. Four different root causes. Only one of them was where I first thought to look.

Building in public for AWS First Commit.

I Hit My First Contradiction Before Building One for the First Commit Hackathon

Sagar Maurya — Fri, 18 Sep 2026 15:21:10 +0000

Building MindMap Debugger for AWS First Commit — Day 1

The setup

Three days ago I landed on an idea: a contradiction-detector that reads through a set of claims and flags where they don't add up. Then First Commit — AWS's "Bharat Builds Tour", run by WeMakeDevs, gave me a reason to finally deploy something on AWS — something I'd never actually done before.

I had 4 days. I had an idea. I had zero AWS experience deploying anything real.

Here's everything that went right, wrong, and sideways on Day 1 — before I'd written a single working line of code. The first contradiction I found had nothing to do with the project.

Problem #1: Figuring out what "built on AWS" actually means for my idea

First Commit judges on something specific: "Built on AWS" is worth real points, non-negotiable. It's not enough to have a good idea sitting behind an API call — AWS has to be doing real work in the architecture, or it reads as generic.

So the first real decision of this hackathon wasn't "what do I build" — it was how do I architect this so AWS is actually load-bearing, not decorative.

Where I landed:

MindMap Debugger (contradiction detection) — maps naturally onto structured extraction and a UI-first demo, with room to bring in AWS-native pieces like Strands and Cedar as the pipeline matures

Lesson one, before any code: know exactly what the judges are scoring, and architect toward that from the first decision — not as an afterthought.

Problem #2: I didn't have a card. AWS didn't care.

This is the part nobody warns you about.

I went to set up a real AWS account for the Ship It track — the one with live deployment on AWS. I verified on AWS Builder Center first, no problem. Then I went to create the actual AWS account for real cloud deployment, and hit this:

Turns out AWS Builder Center and an actual AWS account are two completely separate systems. I didn't know that going in. Fair enough — I went to create a real account.

Then I hit the actual wall:

AWS account signup requires a debit or credit card. I only had UPI, and that wasn't accepted at signup. And here's where I hit my first contradiction of the hackathon.

So I pivoted to Build It — the track that explicitly promises "no AWS account, no card, no bill." It's fully real, fully eligible.

Lesson two: check your hard constraints before you pick your architecture, not after.

Problem #3: If no AWS account, then no Bedrock. Then what?

Build It's tool list includes Strands Agents SDK, Cedar, PartyRock, SAM CLI + LocalStack, OpenSearch. None of them are the actual AI brain my project needed — something that reads text and reasons about whether two claims contradict each other.

I posted in the WeMakeDevs Discord asking about local model options. A community member's reply pointed me toward Groq — free tier, genuinely no card required, and fast.

Day 1's actual stack was simpler than the final architecture: just Groq, called directly. Strands Agents SDK — AWS's own, model-agnostic, so it wouldn't care that I wasn't using Bedrock — was the plan for later, once the core pipeline actually worked.

The first real code, and the first real bug

Wired up the extraction pipeline: paste text in, get structured claims and relationships out, in JSON.

First run:

A clean 404. The model name I'd used had been deprecated by Groq back in June — dead before I ever touched it. Not a mistake, just stale information colliding with a fast-moving API landscape.

Swapped in the current recommended model. Ran it again:

Real structured output. Propositions extracted, typed, ready to reason over. This was the moment the project stopped being a plan and started being a thing that actually runs.

Where Day 1 ends

By the end of today I had:

A clear, deliberate track decision (Build It, and why)
A working extraction pipeline calling a real model, no AWS account required
A first real bug, diagnosed and fixed within the hour
A strong sense of exactly which AWS-native tools (Strands, Cedar) actually belong in this architecture — and which ones (PartyRock, SAM CLI/LocalStack, OpenSearch) don't, and why forcing them in would hurt more than help

Tomorrow: the detection logic — teaching the system to catch contradictions that aren't stated explicitly, wiring the full pipeline end-to-end, and (I'm guessing, based on how Day 1 went) at least one more bug I don't know about yet.

Tech stack so far

Python — core pipeline
Groq API — model backend, free tier, no card
(Coming: Strands Agents SDK to wrap the Groq call, Cedar for policy-based filtering, Flask for the UI)

Building a Voice-First AI Reflection Journal with 3D Mindset Constellations on Google Cloud Run

Sagar Maurya — Sun, 06 Sep 2026 15:33:42 +0000

Most digital journals are fundamentally passive text storage. You type your thoughts into an empty text area, save the entry under a timestamp, and close the tab. Over time, that leaves you with an unindexed graveyard of text entries that are rarely revisited, organized, or connected.

When developers attempt to introduce AI into journaling tools, the implementation often stops at generic chatbot wrappers or ungrounded motivational slogans. What is missing is structure, cognitive continuity, and active reflection:

Low-barrier capture: When you are fatigued after hours of engineering, sitting down to type several paragraphs creates friction. Speaking aloud with natural voice dictation lowers the barrier to getting thoughts out of your head.
Socratic feedback: Instead of generic positive reinforcement, a reflective partner should identify underlying assumptions, clarify trade-offs, and ask thoughtful follow-up questions.
Cross-session memory stitching: Daily reflections shouldn't exist in silos. An intelligent journal should identify connections between today's friction and dilemmas you faced three weeks ago.
Spatial pattern synthesis: Rather than keeping thoughts locked in a flat chronological feed, visualizing recurring cognitive themes—breakthroughs, friction, growth, and decisions—as an interactive 3D universe makes mental habits immediately visible.

To explore this, I built Gemini Reflection Journal: an open, voice-first reflection application powered by Gemini 3.6 Flash, an interactive Three.js WebGL 3D constellation galaxy, and a unified full-stack architecture deployed to Google Cloud Run.

Here is what the interface looks like in practice:

Figure 1: The distraction-free reflection workspace featuring live voice dictation, reflection spark selection, Socratic multi-turn dialogue, and historical callbacks.

🏗️ System Architecture & Data Flow

The application is structured as a unified full-stack service where a React single-page frontend and an Express API proxy are served from a single container:

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                               React 18 + Vite Frontend                                 │
│       (Tailwind CSS • Lucide Icons • Web Speech API • Three.js 3D Cosmic Galaxy)       │
└───────────────────────────┬────────────────────────────────┬───────────────────────────┘
                            │                                │
                   (Firebase Auth SDK)               (Internal API Proxy)
                   (Owner-Bound Firestore Sync)      (/api/chat, /api/summarize,
                            │                         /api/socratic-callback,
                            │                         /api/mindset-constellation)
                            ▼                                │
               ┌────────────────────────┐                    ▼
               │ Google Cloud Firestore │      ┌─────────────────────────────┐
               │ (Data Isolation Rules) │      │ Node/Express on Cloud Run   │
               └────────────────────────┘      │ (GCP Secret Manager Ingest) │
                                               └──────────────┬──────────────┘
                                                              │
                                                   (@google/genai SDK)
                                                              ▼
                                               ┌─────────────────────────────┐
                                               │ Gemini Model Fallback Ladder│
                                               │ 1. gemini-3.6-flash         │
                                               │ 2. gemini-3.1-flash-lite    │
                                               │ 3. gemini-flash-latest      │
                                               │ 4. gemini-3.8-flash         │
                                               │ 5. gemini-3.7-flash         │
                                               └─────────────────────────────┘

Stack Components:

Frontend: React 18, TypeScript, Tailwind CSS, Three.js, Lucide Icons, and the browser Web Speech API.
Backend: Node.js and Express, bundled into a single self-contained CommonJS artifact (dist/server.cjs) via esbuild.
AI Engine: @google/genai TypeScript SDK with an automated circuit-breaker model fallback ladder.
Database & Identity: Google Cloud Firestore with owner-bound rules, plus Firebase Authentication (Google Sign-In with an automatic fallback to isolated guest sessions).
Secrets Management: Google Cloud Secret Manager.
Hosting & Runtime: Google Cloud Run.

🌟 Core System Capabilities

1. Hands-Free Voice Dictation & Spoken Feedback

Typing on a keyboard can interrupt quick trains of thought. The application integrates the browser's native Web Speech API for real-time speech recognition:

Continuous Transcription: Captures spoken thoughts and writes directly to the reflection buffer with silence handling.
Visual Audio Feedback: Animated speech indicators provide immediate confirmation that your microphone is capturing input without lag.
Text-to-Speech Playback: Follow-up questions generated by Gemini can be read aloud using SpeechSynthesis with synchronized play, pause, and stop controls.

Figure 2: Real-time voice dictation with animated speech waveform and audio playback of Socratic inquiries.

2. Socratic Reflection Engine

When a reflection is submitted, the backend doesn't output generic advice. The system prompt instructs Gemini to act as a thoughtful Socratic partner:

Identifies core tensions, cognitive dissonance, or hidden assumptions.
Formulates one or two focused clarifying questions.
Suggests small, concrete next actions rather than abstract platitudes.

Users can converse across multiple turns, digging deeper into a technical problem, team friction, or personal decision before concluding the session.

3. Socratic Callbacks & Cross-Session Memory Stitching

Most journaling applications treat each entry as an isolated event. Our Socratic Callback Engine (POST /api/socratic-callback) queries historical entries to bridge the gap between past and present:

Analyzes previous reflections (e.g. from 3, 7, or 14 days ago) alongside today's entry.
Detects recurring themes, unresolved tensions, and cognitive growth.
Surfaces tailored callbacks: "Three days ago, you noted feeling blocked by asynchronous review delays. In today's entry, you mentioned shipping the auth service. Did your strategy of small PRs resolve the friction, or did team dynamics shift?"

This transforms passive logging into a continuous, active feedback loop.

4. Interactive 3D Mindset Constellations (Three.js WebGL)

The centerpiece of the application is the 3D Mindset Constellation (POST /api/mindset-constellation).

When reflections are recorded, the backend prompts Gemini 3.6 Flash to analyze the text and extract cognitive nodes classified into five primary categories:

💡 Breakthroughs (moments of sudden clarity or insight)
🌱 Growth (positive momentum, new habits, skill acquisition)
⚡ Friction (blockers, fatigue, technical hurdles)
🧭 Decision-Making (trade-offs, architectural forks, prioritization)
🧘 Mindset (grounding, self-awareness, perspective shifts)

These nodes and their conceptual links are mapped into a custom Three.js WebGL scene:

Figure 3: Interactive Three.js WebGL 3D universe mapping recurring breakthroughs, friction points, and cognitive growth anchors.

🚀 Technical Breakthrough: Zero-Lag Hardware-Accelerated 2D HUD Projection

In hybrid 3D WebGL applications, displaying 2D HTML labels over 3D coordinates often suffers from noticeable lag or jitter. If you pipe 3D coordinates through React state (useState), React's asynchronous render batching creates a 1–2 frame delay behind the canvas. Furthermore, CSS transitions on left/top cause labels to drag like a trailing tail during rotations.

To achieve fluid, 60 FPS synchronization, we engineered a direct DOM projection pipeline:

Direct DOM Refs: Badges are mounted with standard React markup, but their coordinates are managed outside the virtual DOM via badgeElementsRef.
Synchronized Animation Loop: Inside requestAnimationFrame, right after renderer.render(scene, camera), coordinates are calculated via tempV.project(cam) and immediately applied to element styles using hardware-accelerated GPU transforms:

   el.style.transform = `translate3d(${Math.round(screenX)}px, ${Math.round(screenY + offsetY)}px, 0) translate(-50%, ${translateY})`;

Camera Forward Vector Dot-Product Check: Prevents inverted projections when nodes rotate behind the camera lens:

   const toNode = tempV.sub(camPos);
   if (toNode.dot(camDir) <= 0) {
     el.style.display = 'none';
     return;
   }

Dynamic Boundary Flip Logic: When a star orbits near the top edge of the canvas (screenY < 54), the badge automatically flips below the star with an upward-pointing anchor stem, eliminating clipping.
Depth-Based Z-Indexing: Badges dynamically scale their z-index based on normalized camera distance so foreground stars naturally occlude background labels.
Smooth Camera Fly-To: Clicking or searching any star triggers a spherical camera flight directly to the node, opening an inspection drawer with related reflection excerpts.

5. Structured Synthesis & Daily Sparks

Executive Synthesis: Users can generate a clean summary of their journal entry containing an executive summary, key realizations, mood/energy tags, and micro-commitments (POST /api/summarize).
Daily Reflection Sparks: A built-in prompt generator offers targeted starters across five disciplines: Stoic Premeditatio Malorum, Evening Wind-Down Review, Decision Framing, Mindfulness, and Creative Problem Solving (POST /api/daily-prompt).

Figure 4: Automated reflection synthesis generating concise executive summaries, sentiment tags, and actionable micro-commitments.

🛡️ Security Architecture & Threat Model

Personal reflections require strict data privacy and isolation. Before building the application logic, we mapped the system against five primary threat zones:

Threat Summary Table

Threat Zone	Identified Risk	Countermeasure Implemented
Input Surfaces	Malicious injection or oversized payload structures in API endpoints.	Strict schema validation, top-level JSON request deserialization, and defensive null-safe parameter guarding (`(req.body && typeof req.body === 'object') ? req.body : {}`).
Planning & Reasoning	Prompt injection attempting to divert the Socratic persona.	Strict system instruction boundaries; incoming reflections are treated strictly as plain text data, never executable instructions.
Tool & Server Execution	API credential exposure or client-side token leakage.	Complete backend API proxy architecture. The browser client never touches or stores the Gemini API key.
Memory & Database	Cross-user data access or unauthorized document read/write.	Owner-bound Firestore security rules verify `request.auth.uid == userId` on every path and sub-collection.
Secret Management	Hardcoded secrets or keys leaked through version control.	`GEMINI_API_KEY` is loaded at container runtime from Google Cloud Secret Manager. Zero secrets exist in code or repository commits.

Owner-Bound Firestore Security Rules

To enforce complete user data isolation at the database level, firestore.rules enforces that only the authenticated user can access their own document trees:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /users/{userId} {
      allow read, write: if request.auth != null && request.auth.uid == userId;

      match /interactions/{interactionId} {
        allow read, write: if request.auth != null && request.auth.uid == userId;
      }

      match /entries/{entryId} {
        allow read, write: if request.auth != null && request.auth.uid == userId;
      }
    }
  }
}

Even if client-side configuration parameters are inspected in the browser, Firestore directly rejects any query where request.auth.uid fails to match the targeted document path.

⚡ Engineering Highlight: Resilient Gemini Model Fallback Ladder

In production, external API endpoints can occasionally return transient 429 Resource Exhausted, 503 Service Unavailable, or capacity errors.

Rather than letting an API hiccup interrupt a user's train of thought, our backend helper wraps @google/genai in an automated multi-tier fallback ladder with circuit-breaker health tracking:

// server.ts - Resilient Gemini Call with Circuit-Breaker Fallback Ladder
import { GoogleGenAI } from '@google/genai';

const BASE_MODEL_LADDER = [
  'gemini-3.6-flash',       // Primary: Low latency, high reasoning quality
  'gemini-3.1-flash-lite',  // High-availability lightweight fallback
  'gemini-flash-latest',    // Dynamic alias fallback
  'gemini-3.8-flash',       // Next-gen reasoning engine
  'gemini-3.7-flash'        // Deep reasoning fallback
];

// Track degraded models to avoid hammering exhausted quotas
const modelDegradedUntil = new Map<string, number>();

function getPrioritizedModelList(): string[] {
  const now = Date.now();
  const healthy: string[] = [];
  const degraded: string[] = [];

  for (const model of BASE_MODEL_LADDER) {
    const degradedUntil = modelDegradedUntil.get(model) || 0;
    if (now < degradedUntil) {
      degraded.push(model);
    } else {
      healthy.push(model);
    }
  }
  return [...healthy, ...degraded];
}

export async function generateContentWithFallback(params: {
  contents: any;
  systemInstruction?: string;
  config?: any;
}) {
  const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
  const candidateModels = getPrioritizedModelList();
  let lastError: any = null;

  for (const model of candidateModels) {
    const isQuota = modelDegradedUntil.get(model) && 
      (modelDegradedUntil.get(model)! - Date.now() > 5 * 60 * 1000);
    if (isQuota) continue; // Skip models with active multi-hour quota exhaustion

    const maxAttempts = 2; // Allow 1 quick retry for transient demand surges
    for (let attempt = 0; attempt < maxAttempts; attempt++) {
      try {
        const response = await ai.models.generateContent({
          model,
          contents: params.contents,
          config: {
            systemInstruction: params.systemInstruction,
            ...params.config
          }
        });

        // Success: clear degraded state
        modelDegradedUntil.delete(model);
        return { response, modelUsed: model };
      } catch (err: any) {
        lastError = err;
        const status = err.status || err.statusCode || err.error?.code;

        if (status === 429) {
          // Quota exhausted: cool down model for 15 minutes
          modelDegradedUntil.set(model, Date.now() + 15 * 60 * 1000);
          break; // Cascade to next model immediately
        } else if (status === 503 || status === 500) {
          // Transient server pressure: short 30-second cooldown
          modelDegradedUntil.set(model, Date.now() + 30 * 1000);
          if (attempt === 0) continue; // Try once more before cascading
        }
      }
    }
  }
  throw lastError;
}

If the primary model experiences transient capacity pressure, the call cascades to the next tier in milliseconds, returning a valid response without breaking the user experience.

🚀 Deploying to Google Cloud Run

Deploying a full-stack container to Google Cloud Run provides several key operational advantages:

Unified Container: Both the static client assets and the Node.js Express server run in one container on port 3000.
Scale to Zero: During idle hours, Cloud Run instances scale down to zero, minimizing resource consumption.
Native Secret Manager Binding: Cloud Run mounts secrets directly into environment variables without requiring .env files in production images.

Step-by-Step Deployment Walkthrough

1. Enable Required Google Cloud APIs

gcloud services enable \
  run.googleapis.com \
  secretmanager.googleapis.com \
  firestore.googleapis.com

2. Configure Secret Manager for the Gemini API Key

# Create the secret
gcloud secrets create GEMINI_API_KEY --replication-policy="automatic"

# Set the secret value
echo -n "YOUR_GEMINI_API_KEY" | gcloud secrets versions add GEMINI_API_KEY --data-file=-

# Grant Cloud Run's service account permission to access the secret
PROJECT_NUMBER=$(gcloud projects describe $(gcloud config get-value project) --format="value(projectNumber)")

gcloud secrets add-iam-policy-binding GEMINI_API_KEY \
  --member="serviceAccount:${PROJECT_NUMBER}-compute@developer.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

3. Build & Deploy to Cloud Run

# Deploy container directly from the application source
gcloud run deploy gemini-reflection-journal \
  --source . \
  --platform managed \
  --region us-central1 \
  --allow-unauthenticated \
  --set-secrets GEMINI_API_KEY=GEMINI_API_KEY:latest

# Attach the required Cloud Run AI Challenge label for automated verification
gcloud run services update gemini-reflection-journal \
  --update-labels=dev-tutorial=cloud-run-ai-challenge \
  --region=us-central1

Once deployed, Cloud Run provides a production HTTPS URL with automatic TLS termination and scalable request routing.

📈 Key Takeaways

Sub-second latency enables natural conversational flow: Using Gemini 3.6 Flash ensures that reflection feedback returns within seconds of finishing dictation, preventing awkward conversational pauses.
Direct DOM synchronization solves hybrid 3D UI lag: Decoupling 2D HTML labels from React's virtual DOM and projecting them directly in the WebGL requestAnimationFrame loop delivers smooth 60 FPS performance without trailing or jitter.
Cross-session callbacks turn logs into insights: Connecting historical reflections with current entries via Socratic callbacks bridges cognitive patterns that users would otherwise miss.
Containerized serverless simplifies full-stack delivery: Running a bundled React + Express service in Cloud Run eliminates the need to coordinate separate hosting environments, API gateways, or complex reverse proxies.

Submitted as part of the Google Cloud Run AI Challenge.

#AccelerateAIwithCloudRun #GoogleCloud #CloudRun #Gemini #GeminiAI #GoogleGenAI #Firestore #Firebase

How I Built a Self-Healing AI Tracker That Fixes Its Own Scrapers

Sagar Maurya — Sun, 23 Aug 2026 16:48:31 +0000

WeMakeDevs and Bright Data ran a week-long hackathon called Into the Scrape-Verse — the challenge was to build a self-healing web scraper and turn live web data into something real.

I had one week. I built Crawlr — an autonomous AI model and dataset tracker that watches four research sources in real time, detects when a scraper breaks, heals it automatically, and logs every event to a live audit trail. No human in the loop.

The Problem I Wanted to Solve

Scrapers break silently. A site changes a class name, a layout shifts, a field moves — and your scraper returns nothing. You find out days later when someone notices the data is stale.

The usual fix is manual: inspect the HTML, find the new selector, redeploy. It works until the next time.

I wanted to build something that detects the break, figures out what went wrong, and fixes itself — without any human involvement.

What Crawlr Tracks

The AI research world moves fast. New models drop on EleutherAI. Together AI publishes blog posts. Papers With Code indexes the latest benchmarks. OpenRouter lists every available model with real pricing data.

Crawlr watches all four:

Source	What it scrapes
EleutherAI	Model and dataset releases
Together AI	Blog posts and announcements
Papers With Code	Latest research papers
OpenRouter	Live model catalog with context length and pricing

Every pipeline run scrapes all four, normalises the data into a unified format, and serves it through a real-time dashboard. 78 records across 4 sources, updated on every run.

How Bright Data Scraper Studio Powers Everything

Every scraper in Crawlr was built using Bright Data's Scraper Studio — an AI-powered platform that builds, runs, and self-heals custom web scrapers from your terminal.

The entire workflow is four commands:

# Login once
npx -p @brightdata/cli bdata login

# Describe the data you want — AI builds the scraper
bdata scraper create https://eleuther.ai/releases \
  "model name, description, release date, type (model/dataset/library)"

# Run it — returns clean JSON
bdata scraper run c_mt4lee1f5n8ouckku https://eleuther.ai/releases --pretty

# When the site changes — heal it with a plain-English description
bdata scraper heal c_mt4lee1f5n8ouckku \
  "description field is empty in 6 out of 8 records, selector may have changed"

The c_* Collector ID you get back is a live production API endpoint. No deployment step. No proxy rotation, retries, or unblocking to manage — Bright Data handles all of that. You just describe what you want.

The Moment That Made It Click

While building the OpenRouter collector, the initial scraper was returning records but the context_length and pricing_per_token fields were all null. I ran:

bdata scraper heal c_mt4m2cql181bpcgals \
  "context_length and pricing fields returning null. 
   Model names contain provider prefix like 'Meta: Llama 3.1'. 
   Extract provider from model name string."

Two minutes later, the collector was returning this:

{
  "title": "Meta: Muse Spark 1.2 Contributor",
  "provider": "Meta",
  "context_length": "1.05M context",
  "pricing_per_token": "$0.10 /M input tokens"
}

Same Collector ID. Nothing downstream changed. The scraper just started working correctly. That's when I understood what made Scraper Studio genuinely different.

The Architecture

Crawlr has three components working together in a closed loop.

1. The Pipeline

scrapers/pipeline.js runs all four collectors in sequence, normalises the raw JSON into a unified schema, and saves the output to the data/ folder.

Each source returns completely different data shapes — OpenRouter looks nothing like EleutherAI. Every source has its own unify* function that maps it to the same schema:

{
  id: "eleuther-0-trlX",
  source: "eleuther",
  title: "trlX",
  description: "A repo for distributed training of language models...",
  url: "https://github.com/CarperAI/trlx",
  type: "Library",
  date: "Dec 9, 2023"
}

2. Sentry — The Health Agent

scrapers/sentry.js is an autonomous health checker that runs after every pipeline and validates each source's output for three failure modes:

Schema Break — payload is empty or not an array
Volume Drop — record count fell more than 50% vs the baseline
Null Creep — required fields are empty in more than 20% of records

When it detects a failure, it auto-generates a diagnosis from the actual failure data — field names, failure percentages, a sample broken record — and calls bdata scraper heal:

function buildDiagnosis(sourceId, data, issues) {
    const sample = JSON.stringify(data[0]).slice(0, 200);
    return `Self-healing prompt for '${sourceId}': 
            ${issues.join(' | ')} 
            Sample record: ${sample}
            Action: locate updated selectors for the listed fields.`;
}

After healing, Sentry re-runs the collector to verify recovery. Every step is logged with timestamps. Here's what a normal healthy run looks like — no anomalies this time, all four sources verified clean. (The full heal cycle, triggered live, is a bit further down in Chaos Mode.)

3. The Dashboard

A pure HTML/CSS/JS frontend — no framework, no build step. It reads directly from the JSON files the pipeline writes.

Features:

4 source tabs with live record counts
Instant search across title, description, and organization
Live Audit Log — color-coded pills for each event type (HEALTHY, ANOMALY, HEALING, ERROR)
Chaos Mode — simulates a scraper break so you can see the full heal cycle live

What a Full Run Looks Like

Pipeline:

✅ eleuther:   8 items saved
✅ togetherai: 25 items saved
✅ pwc:        25 items saved
✅ openrouter: 20 items saved
✅ Pipeline complete.

Sentry health check:

✅ [eleuther]   Healthy. 8 records returned.
✅ [togetherai] Healthy. 31 records returned.
✅ [pwc]        Healthy. 50 records returned.
✅ [openrouter] Healthy. 20 records returned.
✅ All sources checked.

What I Learned

bdata scraper heal is the real unlock. You describe what's broken in plain English, Bright Data's AI rewrites the extraction logic, and the same Collector ID starts returning clean data. I watched it fix broken selectors on live sources multiple times during the week.

Bright Data handles the hard parts of scraping. Proxy rotation, retries, unblocking, rate limiting — none of that is your problem. You describe what you want and get clean JSON back.

The Collector ID is a production API endpoint. Trigger it with POST /dca/trigger from any language or scheduler. No deployment step. This made wiring it into the pipeline and dashboard trivial.

Self-healing is not a gimmick. Going into this I wasn't sure how well automated healing would actually work on real sites. After a week of watching it fix real extraction failures, I'm convinced it's the right direction for production scrapers.

Try It

GitHub: github.com/mauryasagar/crawlr

Clone it, add your .env with four Bright Data Collector IDs, and run:

npm run pipeline   # Scrape all 4 sources
npm run sentry     # Health check + auto-heal
npm run serve      # Open the dashboard at localhost:3000

Built for the Into the Scrape-Verse hackathon — WeMakeDevs × Bright Data, August 2026

#webdev #javascript #hackathon #brightdata #scraping #wemakedevs #opensource

How I Built a YouTube Trend Engine on Zerops

Sagar Maurya — Sun, 09 Aug 2026 17:14:45 +0000

The API Quota Problem

If you've ever built applications on public APIs, you know how quickly limits can become a bottleneck. The YouTube Data API v3 gives you a free tier of 10,000 units/day. Each search.list request costs 100 units. A single user scanning a niche across 6 keywords burns 600 units. A handful of test scans, and your daily allowance is practically gone.

To make matters worse, Google Trends (via pytrends) has no official API — if you hit it repeatedly without caching, Google silently rate-limits your IP address, returning empty data frames or hanging your server indefinitely.

I built Signal for the Zerops Challenge by WeMakeDevs: a tool that turns real-time search trends and YouTube view telemetry into ready-to-film video titles ranked by an objective demand score.

Because of these aggressive limits, I didn't just need an app that worked — I needed an infrastructure that could fail soft, save API quota, and respond in under 1 millisecond on repeat scans.

Here is how I built Signal, the architectural decisions behind it, the three sneaky deployment bugs I encountered, and how Zerops Valkey saved my API quota.

🔗 Live demo: https://signal-2dc6-8000.prg1.zerops.app

💻 GitHub repository: github.com/mauryasagar/signal

What Signal Does

Most YouTube advice is generic. Signal changes that by backing up video ideas with real data.

User inputs a niche (e.g., "home coffee brewing").
Google Trends Discovery: Pulls rising and related search queries over the last 90 days.
YouTube Telemetry Check: Queries YouTube Data API v3 in parallel to fetch recent video upload counts and average view metrics.
Groq LLM Synthesis: Sends all signals to Llama 3.1 8B Instant on Groq LPUs, generating concrete video titles and strategic creator angles in ~280ms.
Demand Scoring: Ranks topics using a weighted logarithmic score.
Zerops Valkey Caching: Caches raw signal telemetry for 1 hour. Repeat queries skip external APIs entirely, returning in <1ms.

Live view of the Signal Dashboard generating topics for coffee brewing.

System Architecture: The 4-Layer Engine

Signal is designed as a micro-cached pipeline deployed on Zerops, using Gunicorn workers, Valkey key-value storage, and external API connectors.

User Input: "home coffee brewing"
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│              Flask Web App (Gunicorn · Python 3.12)         │
│                    Deployed on Zerops                       │
└──────────────┬──────────────────────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│                    Pipeline (report.py)                     │
│                                                             │
│  1. Check Zerops Valkey cache ──► HIT: Skip to step 4       │
│                                   MISS: Continue            │
│                                                             │
│  2. Google Trends (pytrends)                                │
│     └── Daemon thread → [{query, interest}]                 │
│                                                             │
│  3. YouTube Data API v3                                     │
│     └── Per query: video count + avg views                  │
│                                                             │
│  4. Write signals to Valkey cache (1 hour TTL)              │
│                                                             │
│  5. Groq LPU Inference (Llama 3.1 8B)                       │
│     └── Keyword + demand data → Title + Angle               │
│                                                             │
│  6. Demand score computed, topics sorted                    │
└──────────────┬──────────────────────────────────────────────┘
               │
               ▼
        Report page rendered

The Tech Stack

Layer	Technology	Why
Web server	Flask + Gunicorn	Lightweight, production-safe
Trend data	Google Trends via `pytrends`	No official API key needed
YouTube demand	YouTube Data API v3	Official, free 10K quota/day
LLM	Groq — Llama 3.1 8B	Sub-second inference
Cache	Zerops Valkey	Cuts repeat response time from 25s → <1s
Deployment	Zerops	Production infra, private networking, auto-deploy

Project Structure

signal/
├── app.py              # Flask routes: /, /about, /generate, /health
├── report.py           # Pipeline orchestration + demand scoring
├── fetch_trends.py     # Google Trends + YouTube API calls
├── generate_topics.py  # Groq LLM call + JSON parsing
├── cache.py            # Zerops Valkey integration
├── zerops.yaml         # Zerops build + run config
├── templates/
│   ├── base.html       # Navbar, radar loading animation
│   ├── landing.html    # Hero + search form
│   └── report.html     # Ranked topic cards
└── static/
    ├── style.css       # Dark/light theme + custom properties
    └── main.js         # Theme toggle + loading overlay

Technical Deep-Dive: Code & Logic

1. Resilient Ingestion & Daemon Thread Caps (`fetch_trends.py`)

pytrends is prone to silent hangs when Google rate-limits requests. Allowing pytrends to block synchronously would exhaust Gunicorn worker threads.

To guarantee sub-second bounds, Signal wraps pytrends in a daemon thread with an explicit 0.8-second hard cutoff. If the thread doesn't finish, we gracefully fall back to predefined query templates:

# fetch_trends.py snippet
import threading

def fetch_all_signals(niche, youtube_api_key, max_terms=8):
    related = []

    def _fetch_pytrends():
        nonlocal related
        related = get_related_queries(niche, max_terms=max_terms)

    # Enforce strict 0.8s latency bound
    t = threading.Thread(target=_fetch_pytrends, daemon=True)
    t.start()
    t.join(timeout=0.8)

    # Soft failure fallback if Google Trends rate-limits
    if not related:
        related = [
            {"query": niche, "interest": 50},
            {"query": f"{niche} tips", "interest": 40},
            {"query": f"{niche} for beginners", "interest": 40},
        ]

2. The Logarithmic Demand Scoring Model (`report.py`)

Raw views can be misleading — a single viral video with 10 million views can distort averages and make a dead niche look hyper-attractive. Signal uses a normalized 0–100 Demand Score formula with log-scaled view counts:

DemandScore = (Trend Momentum × 60%) + (Upload Velocity × 25%) + (Audience Size × 15%)

# report.py snippet
import math

def _compute_demand_score(item):
    # 60% — Google Trends search interest momentum (0-100)
    trend_component = item["trend_interest"] * 0.6

    # 25% — Creator upload velocity (caps at 5 recent videos)
    video_count_component = min(item["video_count"] / 5 * 100, 100) * 0.25

    # 15% — Audience size (log10 scaled up to 10M views so outliers don't dominate)
    if item["avg_views"] > 0:
        views_component = min(math.log10(item["avg_views"] + 1) / 7 * 100, 100) * 0.15
    else:
        views_component = 0

    return round(min(trend_component + video_count_component + views_component, 100))

3. Fail-Soft Caching Layer with Zerops Valkey (`cache.py`)

Zerops provides a managed, Redis-compatible Valkey database service. Signal uses Valkey to cache raw signals for 1 hour. But I wanted local development to work without installing Redis. The solution? A thread-safe in-memory fallback:

# cache.py snippet
import redis
import threading
import os

_MEMORY_CACHE = {}
_MEMORY_CACHE_LOCK = threading.Lock()

def get_cache():
    host = os.getenv("VALKEY_HOST")
    if not host:
        return None # Falls back to in-memory

    try:
        client = redis.Redis(
            host=host,
            port=int(os.getenv("VALKEY_PORT") or 6379),
            password=os.getenv("VALKEY_PASSWORD"),
            decode_responses=True,
            socket_connect_timeout=2,
        )
        client.ping()
        return client
    except Exception as exc:
        print("Valkey unreachable — using in-memory fallback.")
        return None

The Frontend: Dynamic Report Cards

To make the UI feel as fast and premium as the backend, I built a color-coded topic card system. Each rank gets a unique accent color (amber, green, purple, blue, rose).

When a user hovers over a topic card, it reveals a gradient wash in the card's specific accent color. This is powered by per-card CSS custom properties:

.topic-card-1 { --accent: #e8a33d; --accent-r: 232; --accent-g: 163; --accent-b: 61; }
.topic-card-2 { --accent: #5fb88a; --accent-r: 95;  --accent-g: 184; --accent-b: 138; }
.topic-card-3 { --accent: #a78bfa; --accent-r: 167; --accent-g: 139; --accent-b: 250; }

.topic-card:hover {
  border-color: rgba(var(--accent-r), var(--accent-g), var(--accent-b), 0.4);
  transform: translateX(4px) translateY(-2px) translateZ(0);
  box-shadow: 0 12px 32px -12px rgba(var(--accent-r), var(--accent-g), var(--accent-b), 0.25);
}

Three Sneaky Roadblocks (And How I Fixed Them)

Building for a hackathon is never a straight path. Here are the three sneaky bugs that caught me off guard:

🐛 Roadblock #1: The Zerops Container Runtime Ghost

The Bug: My local tests passed. I configured zerops.yaml and deployed. The build succeeded, but the runtime container immediately crashed with:
bash: gunicorn: command not found

The Cause: Zerops intelligently decouples the build container from the runtime container. Packages installed during buildCommands do not automatically persist into the slim runtime image.

The Fix: Adding a prepareCommands block inside zerops.yaml to install production dependencies directly in the runtime environment:

# zerops.yaml snippet
zerops:
  - setup: signal
    build:
      base: python@3.12
      buildCommands:
        - pip install -r requirements.txt
    run:
      base: python@3.12
      # THE FIX: Install packages in the runtime container
      prepareCommands:
        - pip install Flask==3.0.3 python-dotenv==1.0.1 gunicorn==22.0.0 redis==5.2.0 ...
      ports:
        - port: 8000
          httpSupport: true
      start: gunicorn app:app --bind 0.0.0.0:8000 --workers 2

🐛 Roadblock #2: The Missing Valkey Auto-Password

The Bug: Zerops Valkey service was running, but cache.py kept throwing redis.exceptions.AuthenticationError: NOAUTH Authentication required.

The Cause: Zerops Valkey instances ship with authentication enabled by default, auto-generating a secure password.

The Fix: Using Zerops' dynamic environment variable cross-referencing in the project dashboard! I didn't need to hardcode anything.

Using Zerops' dynamic variables to securely inject the auto-generated Valkey password at runtime.

By setting VALKEY_PASSWORD=${valkey_password}, Zerops dynamically injects the auto-generated Valkey credential at runtime. Magic.

🐛 Roadblock #3: Silent LLM JSON Output Formatting Drift

The Bug: Even with explicit prompting, Llama 3.1 would occasionally wrap JSON responses in conversational preambles or markdown block wrappers (json ...), completely breaking Python's json.loads().

The Fix: A robust regex extraction function (_extract_json) in generate_topics.py to strip out markdown and preambles safely before parsing.

Telemetry & Verification: The `/health` Endpoint

To confirm that Valkey and Flask were communicating over Zerops' private network, I built a live /health endpoint.

Live telemetry from the /health endpoint confirming successful connection to the Zerops Valkey cache.

Hitting https://signal-2dc6-8000.prg1.zerops.app/health returns live confirmation that caching is active, which instantly drops our response time.

Production Performance Benchmarks

Here is how Signal performs on Zerops under live production conditions:

Performance Metric	Live Uncached Scan	Zerops Valkey Cache Hit	Net Savings / Gain
Response Latency	1,420 ms	0.85 ms	99.94% Faster
YouTube API Quota	600 units / scan	0 units	100% Quota Saved
Google Trends Calls	1 HTTP request	0 HTTP requests	Zero Rate Limits
Groq LPU Inference	~280 ms	~280 ms (Fresh Titles)	Sub-second AI

Step-by-Step Zerops Deployment Walkthrough

Deploying Signal on Zerops took under 4 minutes:

Create Zerops Account: Sign up at zerops.io.
Add Services: I added a Python runtime service (signal, Python 3.12) and a Valkey database service (valkey, Valkey 7.2 Single mode).
Configure Environment Variables: Set YOUTUBE_API_KEY, GROQ_API_KEY, VALKEY_HOST=valkey, and VALKEY_PASSWORD=${valkey_password}.
Connect GitHub Repo: Triggered the build pipeline directly from my main branch.
Enable Public Subdomain: Flipped the switch for public access on port 8000.

Mission accomplished: both Signal and Valkey services running flawlessly in the Zerops production environment.

Known Limitations

Demand score is directional, not exact. No free data source provides real search volume. The score combines relative trend interest and YouTube activity as a proxy.

pytrends is unofficial. Google Trends has no public API. The library can be rate-limited or break without warning. Signal falls back to seeded keywords automatically, so the app never fully fails.

Avg views can look inflated. Results are sorted by view count, so the top videos tend to be outlier performers — it's a ceiling signal, not a guaranteed expectation for a new channel.

Key Takeaways & Lessons Learned

Never trust external APIs to fail loud: Wrap unofficial endpoints (like pytrends) in daemon threads with hard timeout limits and soft fallbacks.
Decouple build vs runtime environments: Always configure prepareCommands in zerops.yaml for runtime dependencies like gunicorn.
Fail-soft caching is worth every line of code: Adding a thread-safe in-memory cache fallback meant local dev worked without Redis, and production degraded gracefully during network blips.
Zerops makes multi-service infra seamless: Wiring Flask to Valkey over internal networking took literally one environment variable (VALKEY_HOST=valkey).

AI Disclosure: As per hackathon rules, tools like Claude and Antigravity were used as pair-programmers to speed up boilerplate code and draft this post. The core architecture, API logic, and Zerops deployment are my own work.

Try Signal Today

🚀 Live Demo: https://signal-2dc6-8000.prg1.zerops.app
💻 GitHub Repo: github.com/mauryasagar/signal

Built for The Zerops Challenge by WeMakeDevs 2026. Thanks to the Zerops team for building an exceptional developer platform, and a massive thank you to the WeMakeDevs community for hosting this incredible hackathon!

Ouroboros AI: We Made a Fake Agent Break Itself, Then Built Something That Fixes It

Sagar Maurya — Sun, 26 Jul 2026 15:15:12 +0000

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

If you're just landing on this series for the first time, here's the short version of where we've been.

In our first post, we installed a tool called SigNoz for the very first time, so we could watch what a piece of software is doing while it runs — which parts are slow, which parts fail, and why. We built a tiny fake AI agent, and just by looking at a picture of its actions laid out in order, we could instantly see that one single step was eating up 85% of the total time. That was the moment this whole thing clicked for us.

In our second post, we went further. We built a slightly smarter fake agent and asked SigNoz a real question: "does this get slower when it has more information to look through?" We didn't calculate the answer ourselves. We just told SigNoz what to group and average, and it handed us a table proving the pattern, clean as anything.

Both of those were practice. Small, safe experiments before the real hackathon clock started ticking. This post is about what we actually built once it did — and honestly, it's the post we're proudest of, because it's the one where we stopped just watching our fake agent and started building something that does something about it.

We're calling the project Ouroboros AI. Repo's here if you want to poke around: github.com/mauryasagar/ouroboros-ai

The problem that was quietly bugging us

Right after we published that second post, something about it kept nagging at us.

We'd asked SigNoz a good question and gotten a good answer: yes, more documents means a slower response, and it climbs in an almost perfectly straight line. We felt clever about it for about a day. Then we looked at what we'd actually built and realized... nothing happened next. We saw the pattern, said "yep, makes sense," and closed the browser tab. If this had been a real product used by real people, someone would still have had to notice that pattern themselves, write it up, wait for an engineer to have time, and fix it — probably days later, and only if they happened to be looking at the right dashboard at the right moment.

That bothered us more the more we thought about it. Watching something break in slow motion isn't the same as fixing it. So a simple, slightly stubborn question became the whole idea for this project:

What if the system could notice its own problem and fix itself, without a person in the middle at all?

Not "send someone a faster alert." Actually close the loop.

A few words before the story, so nothing feels confusing

We're going to use a handful of words a lot in this post. If you already know them, skip ahead. If you don't, here they are in plain terms:

Trace — a record of everything that happened during one single request, start to finish.
Span — one step inside that trace. Four things happened? Four spans.
Alert — a rule you set up that says "if X happens, tell someone."
Webhook — instead of "tell someone" meaning a text message or an email, it can mean "call this web address automatically." That's the piece that let a computer react instead of a human.
MCP (Model Context Protocol) — a standard way for an AI model to reach out and use real tools — in our case, letting an AI ask SigNoz real questions instead of us clicking around the dashboard ourselves.

That's genuinely all the vocabulary you need. Everything else is just those five ideas, wired together.

What we actually decided to build

We sat down together and sketched this out on a shared doc before writing a single line of code. Five steps, in order:

Chaos — a fake AI agent runs constantly, and every so often, on purpose, it messes up.
Traced — every single step it takes gets recorded and sent to SigNoz, live.
Alert — SigNoz watches all these traces coming in, notices the mess-up pattern, and automatically fires off a webhook — no human clicks anything.
Heal — a separate little program receives that webhook and retries the failed thing with better settings. Still no human.
Explain — an AI "Sidekick" you can literally chat with, which reads the real SigNoz data through MCP and tells you, in plain English, what just happened.

Steps 1 and 2 were a more serious version of what we'd already built in post two. Steps 3, 4, and 5 didn't exist anywhere yet. That's where basically the entire week went.

Step one: giving our fake agent something real to fail at

Our agent from post two never actually failed — it just got randomly slower or faster each time. That's fine for a screenshot, but it's useless for an alert. You can't tell a computer "notice when things feel kind of slow sometimes." You need something crisp: a clear line that gets crossed.

So we gave our agent one specific way to break: context overload. In plain terms — our agent normally looks through somewhere between 1 and 8 "documents" to answer a question. But 20% of the time, on purpose, we let it get buried under 15 documents at once, which is way more than it can handle cleanly.

def retrieve_context(query, is_chaos_mode=False):
    with tracer.start_as_current_span("retrieve_context") as span:
        # CHAOS MODE: 15 docs, NORMAL: 1-8 docs
        context_docs = 15 if is_chaos_mode else random.randint(1, 8)
        span.set_attribute("llm.context_docs", context_docs)
        time.sleep(random.uniform(0.1, 0.4))
        return f"Mock context data ({context_docs} docs)"

We picked this specific failure on purpose, not randomly. Real AI agents that search through documents actually run into this exact problem — feed them too much information at once, and they get slower, cost more, and often don't even answer better for it. We wanted our fake failure to be one that real systems actually have, not something we made up just to have a demo.

When the overload happens, we don't just log it as "a bit slow." We mark it as an actual error, with a clear message SigNoz can watch for:

if context_docs >= 15:
    logging.error(f"CONTEXT_OVERLOAD: Agent failed. Used {tokens} tokens, cost ${round(cost, 6)}, took {delay:.2f}s.")
    span.set_status(Status(StatusCode.ERROR, "Context Overload"))

One small thing from post two turned out to matter a lot here: we'd mentioned back then that just printing something to your terminal doesn't automatically send it to SigNoz — you have to specifically wire up your logs to go there too, separately from your traces. This time, we made sure that was working properly from day one, because our whole alert depended on SigNoz actually being able to see that error message, not just a slightly-slower-than-usual span.

We let this fake agent run forever in the background, quietly sending a new trace every 200 milliseconds, so there was always something fresh happening while we built the rest.

Step three and four: teaching the system to fix itself

This is the part that didn't exist in either of our earlier posts, and honestly, the part we're most excited about.

The good news is, we didn't have to build anything fancy to make SigNoz watch for the failure — it already does that with alert rules, the exact same feature we used in post one to watch for slow calls. The only difference is where the alert points. Instead of sending a Slack message or an email, we pointed it at a webhook — a plain web address that SigNoz calls automatically the moment it sees the problem.

On the other end of that web address, we built a tiny program whose only job is to sit there, wait, and fix things when it gets called:

@app.route('/webhook', methods=['POST'])
def webhook():
    data = request.json or {}
    alert_name = data.get("alertname", "Unknown Alert")
    print(f"🚨 ALERT RECEIVED FROM SIGNOZ: {alert_name}")

    result = optimized_llm_call("Auto-healed request")

    span.add_event("AUTO-HEAL SUCCESS", attributes={
        "message": "Request auto-healed with optimized parameters",
        "status": "healed"
    })
    return jsonify({"status": "healed", "result": result}), 200

The actual fix is simple on purpose — it retries the request with a smaller, safer amount of information instead of the overload amount. The part we spent real time thinking about wasn't the fix itself, though. It was this question: should the fix be invisible, or should it leave a trace of its own?

We almost let it be invisible — quietly patch things up in the background and move on. But that felt like exactly the thing we were trying to get away from in the first place: a fix that only lives in someone's head, not in the system. So we made the healer trace its own retry too, tag it clearly as an auto-heal, and record an event that says, plainly, "this got fixed automatically." Now the healing itself shows up in SigNoz right next to the failure that caused it. Nobody has to remember it happened. The system remembers for you.

Step five: teaching an AI to explain all of it in plain English

Here's where post two's biggest lesson came back around. Back then, we discovered that if you tag your data with the right detail, SigNoz can answer real questions for you — no manual math required. The bigger question that kept nagging at us afterward was: what if you didn't even need to know which button to click, or which filter to type? What if you could just ask, like you're talking to a person?

That question is what led us into MCP — the protocol that lets an AI model reach out and actually use tools, instead of just talking. Neither of us had touched MCP before this hackathon, so before writing any code, we spent a real evening just reading about how it works. That reading turned out to save us a lot of pain later, even though it didn't save us from all of it.

What we ended up building is a small chat app we call the Sidekick. You type a question, and behind the scenes:

You type a question
   → Groq (the AI model) decides which tool it needs
   → it asks SigNoz for the real data through MCP
   → the real numbers come back
   → the AI reads them and answers you in plain English

SigNoz's MCP connection offers over 40 different tools it can use — way more than we needed — so we limited our Sidekick to five: list services, get traces, get logs, get metrics, get one specific span. Simple, focused, and cheap to run.

Getting this actually working took two separate evenings of head-scratching, and both problems were the sneaky kind — nothing crashed, things just quietly didn't work.

The first problem: our AI would just never use any tools. It would answer vaguely instead of actually checking real data, no matter how directly we asked it something like "what services are running right now." We spent a while assuming the AI itself was being stubborn, tweaking our instructions to it, before we finally printed out exactly what we were sending it — and found out the tool descriptions we were passing along were completely empty. Turns out SigNoz describes its tools using a label called inputSchema, while the AI model we were using expects a label called parameters. We were only checking for one of the two. The fix was a single line of code, but finding it meant working backwards through several layers of "this should be working" first:

schema = tool.get("inputSchema") or tool.get("parameters") or {"type": "object", "properties": {}}

The second problem was stranger. The very first request would work perfectly, and then the second one would get flatly rejected — every time. It took a while to notice that the server hands you an invisible little "session ID" the first time you connect, and expects you to send it back on every request after that, like a wristband at an event. Miss that, and it treats you like a stranger who just walked in:

sid = response.headers.get("Mcp-Session-Id")
if sid:
    self.session_id = sid

On top of that, some answers came back in a slightly unusual format made for streaming data, instead of a plain, simple response — so our little connector had to learn to handle both.

None of this felt exciting while it was happening. It was two nights of adding print statements, staring at raw technical details, and slowly ruling things out one at a time. But the moment both fixes were in, the payoff was immediate. Post two's big win was clicking through a filter and a group-by to get a table. This time, the same kind of insight just... showed up, after typing a normal sentence:

"Which services are currently being monitored in SigNoz?"

{
  "answer": "The services currently being monitored are:\n- auto-healer-service — 1 trace in the last hour (avg ~494ms)\n- deep-agent-service — 688 traces in the last hour (avg ~1.63s)",
  "tools_used": ["signoz_list_services"]
}

We kept the AI's instructions short and strict on purpose — answer in a few sentences, use the real numbers you found, don't waste time asking for logins it already has. It's meant to feel like a fast, useful teammate answering a quick question, not a general chatbot rambling.

The moment it actually came together

Here's the part that felt like more than just code working.

We finally let all three pieces run at the same time and just watched. In one window, the fake agent quietly sending traces every 200 milliseconds, like it had been doing for days. In another, the Sidekick sitting there, waiting for a question. And then, in the third window — completely on its own, with nobody typing a single command — the healer suddenly printed:

🚨 ALERT RECEIVED FROM SIGNOZ

We hadn't triggered that. SigNoz had caught the overload pattern by itself, in the live stream of data, and called the webhook on its own.

That's the moment this stopped being a demo we were operating and started being a system that was actually running itself.

About thirty seconds later, we asked the Sidekick what had just happened, half-expecting to have to nudge it toward the right answer. We didn't need to. It described the overload, the automatic retry, and the corrected numbers, pulled straight from the same live data we'd watched land in SigNoz seconds earlier. That was the entire five-step loop, closing itself, start to finish, with nobody standing in the middle of it.

What we're taking away from all three posts

Post one taught us that just seeing where the time goes changes how you think about a problem. Post two taught us that tagging the right detail turns a dashboard into something you can actually have a conversation with. This project is where those two lessons stopped being separate little discoveries and turned into one real habit: instrument things not just to log what's easy right now, but to answer the question you'll actually want answered later — because that same detail is what lets an alert fire cleanly, and what lets an AI explain the whole story back to you without you writing a single extra line of code for it.

If someone asked us what we'd tell ourselves before starting this hackathon, it's that one sentence. Everything else — the failures, the debugging nights, the late realization that our first agent needed to actually break instead of just wobble — all came from following that one idea seriously enough to build around it.

Full code, the landing page, and everything else is in the repo: github.com/mauryasagar/ouroboros-ai

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

We Asked SigNoz a Question, and It Actually Answered

Sagar Maurya — Sun, 19 Jul 2026 16:51:23 +0000

By Sagar Maurya & Disha Sonowal — WeMakeDevs SigNoz Hackathon

Before this, we wrote a first post about setting up SigNoz for the very first time — installing it, sending it some data, and figuring out what all this "tracing" stuff actually means. You can also check that out on LinkedIn. If you're new to any of this, that post explains the basics in a simple way, so it's worth a quick look first.

This post is us going one step further as a team. We wanted to see if SigNoz could actually help us discover something, not just show us pretty charts. It did, and here's exactly what happened, explained simply.

First, two words you need to know

Trace — a record of everything that happened during one single request, from the moment it starts to the moment it finishes.
Span — one step inside that trace. If a request does 4 things, it has 4 spans, one for each step.

That's really all you need to follow along.

What we built

A small fake AI assistant. Instead of doing just one thing, it does four things every time someone asks it a question:

Look up some documents — pretend to search for information related to the question (a random number, somewhere between 1 and 8 documents)
Ask the AI — pretend to send the question and those documents to an AI model and get an answer back
Use a tool — pretend to use something like a calculator, a calendar, or a search tool
Clean up the answer — tidy up the final response before sending it back

We wrapped each of these four steps in code that tells SigNoz "hey, this step just started" and "hey, this step just finished." That's what creates the spans.

Looking at just one question it answered

We ran our fake assistant 30 times, and then picked one single run to look at closely in SigNoz. Here's what it showed us:

The whole thing took 1.72 seconds. Almost all of that — 1.43 seconds — was spent in the "ask the AI" step. Everything else (looking up documents, using a tool, cleaning up) barely took any time at all in comparison. You can literally see this with your eyes on the chart, no reading through code needed.

Then we asked a bigger question

Looking at one run is useful, but we wanted to know something bigger: does the AI step get slower when it has more documents to look through?

In real life, this makes sense — if you ask an AI to read through 8 documents instead of 1, it usually takes longer to answer, because there's more for it to process.

So instead of checking each of our 30 runs one by one, we asked SigNoz to do the work for us:

We told it: "only show me the AI-call steps, ignore the other three"
We told it: "now group these by how many documents each one used"
We told it: "show me the average time for each group"

And within seconds, SigNoz gave us this table:

Documents used	Average time taken
1	568ms
2	658ms
3	825ms
4	999ms
5	1147ms
6	1333ms
7	1488ms
8	1614ms

Look at that — it climbs almost perfectly evenly. Every extra document adds roughly 150 milliseconds to the response time. We built this rule into our code, so finding it wasn't a total surprise, but what genuinely impressed us was how easily SigNoz found and displayed it. We didn't write any extra code, we didn't open Excel, we didn't do any manual math. Just a couple of clicks, and the whole pattern was right there in a table.

We also checked the logs

Traces are great for timing, but sometimes you just want to read a plain sentence about what happened. That's what logs are for. We turned those on too, and got results like this:

Now every request has two views: the trace shows how long things took and in what order, while the logs show what actually happened, in plain readable sentences. Small tip if you try this yourself — just printing messages to your terminal doesn't automatically send them to SigNoz. You need a couple of extra lines of setup code to actually connect them, or nothing will show up in the Logs tab.

What we're taking away from this

The biggest thing we learned: a trace waterfall is great for spotting what's slow. But the real magic happened when we tagged our own extra detail onto a span — in our case, "how many documents did this use." That one small addition is what let us ask SigNoz a real question and get a real answer back, across all 30 runs at once, instead of guessing.

For the actual hackathon build, this is exactly the habit we want to carry forward — don't just time your code, tag it with details that'll actually matter later when something's slow and you're trying to figure out why.

Data Isn't What Your Textbook Said It Was

Sagar Maurya — Sat, 11 Jul 2026 02:13:37 +0000

A beginner-friendly breakdown of what data really is, where it comes from, and why it matters more than you think.

You woke up this morning and checked your phone.

You scrolled through Instagram. Opened WhatsApp. Maybe checked the weather. Ordered breakfast on Swiggy.

By the time you finished your morning tea, you had already generated hundreds of data points — and that was before 9 AM.

We live in a world that runs on data. But most people, including a lot of CS students, have never stopped to ask: what actually is data?

Not the textbook definition. The real one.

And that's exactly the problem — we all learned about data from a textbook. Which means we learned about it wrong.

The Textbook Lie

Ask anyone what data is and they'll say something like:

"Data is raw, unprocessed facts and figures."

Technically correct. Completely useless.

That definition tells you nothing about why data matters, where it comes from, or what makes it powerful. It's the kind of definition written for an exam, not for understanding.

So let's close the textbook and start over.

Data is Just a Record of Something That Happened

Every time something happens in the real world — a click, a purchase, a step you walked, a message you sent — there's a possibility of recording it.

When you record it? That's data.

You clicked on a product → data
Your phone counted 6,400 steps today → data
You paused a YouTube video at 2:34 → data
You left a website after 3 seconds → data (and a headache for that website's owner)

Data is just a trace that something happened. That's it.

The interesting part is what happens after you collect those traces.

Raw Data is Almost Always Ugly

Here's what nobody tells beginners: data in the real world is messy.

Imagine a Google Form that asks for someone's phone number. You'll get responses like:

9876543210
+91-9876543210
98765 43210
9876543210 (call after 6pm)

All four people gave you the same number. But to a computer, these are four completely different values.

This is what's called dirty data — and cleaning it is genuinely one of the most important (and underrated) skills in the data field.

A famous saying in data science is:

"80% of data work is cleaning. The other 20% is complaining about cleaning."

It's a joke. But not really.

The Journey of Data: From Noise to Decision

Here's a simple way to think about how data travels:

Something happens
      ↓
It gets recorded (raw data)
      ↓
It gets cleaned (processed data)
      ↓
It gets analyzed (insights)
      ↓
Someone makes a decision

Let's make this real.

Example: Swiggy wants to know why orders drop on Tuesday evenings.

They collect order timestamps → raw data
They remove duplicates, fix timezone errors → clean data
They spot that Tuesday 7–9 PM has 40% fewer orders → insight
They run a Tuesday evening discount campaign → decision

That entire chain? It all started with someone pressing "Order" on their phone.

Not All Data is Numbers

A common misconception is that data means spreadsheets full of numbers. But data comes in many forms:

Structured data — rows and columns, like a CSV or a database table. Easy for computers to process.
Unstructured data — text, images, audio, video. Harder to process, but far more common in the real world.
Semi-structured data — like JSON or XML. Has some structure, but not rigid rows and columns.

When you send a WhatsApp message, that's unstructured data. When your bank logs a transaction, that's structured data. When you fill a Google Form, that produces semi-structured data.

Most of the world's data — over 80% of it — is unstructured. Which is why fields like NLP (Natural Language Processing) and Computer Vision exist: to make sense of data that doesn't fit neatly into a table.

Your Data is Someone's Product

Here's the uncomfortable truth.

Every free app you use — Instagram, Google, YouTube — is free because you are not the customer. You're the product. More precisely, your data is.

When you like a post, skip an ad, or spend 45 minutes on Reels instead of 5, that behavior is recorded, analyzed, and used to serve you more content that keeps you on the platform longer.

This isn't conspiracy theory. It's just the data pipeline at scale.

Understanding this doesn't mean you need to delete all your apps. But it does mean you should be aware of the trade you're making — your attention and behavior in exchange for a free service.

Why Everyone Should Care About Data (Not Just Data Scientists)

You don't need to work in data to benefit from understanding it.

As a developer — you'll build systems that generate data. Understanding data helps you design better databases and APIs.
As a product person — decisions without data are just opinions. Data makes arguments.
As a user — knowing how your data is used makes you a more informed digital citizen.
As a student preparing for placements — almost every tech company today is a data company in some way. Interviews increasingly involve data thinking, even for SDE roles.

Data literacy is becoming as fundamental as being able to read and write.

So What Did Your Textbook Get Wrong?

Nothing, technically. But everything, practically.

Your textbook gave you a definition of data that would pass an exam. It didn't give you a mental model that would help you think about the world differently.

The difference between a student who "knows data" and someone who understands it is this: one memorized a definition, the other sees data everywhere they look — in the apps they use, the decisions companies make, and the systems they build.

That shift in perspective is what this blog is about. Not definitions. Mental models.

Where to Go From Here

This post was just the surface. In future posts, I'll go deeper into:

How databases store and retrieve data efficiently
What actually happens when you run a SQL query
The difference between a data analyst, data engineer, and data scientist
And some hands-on projects that helped me understand all of this better

If you're a student or someone just getting into tech — I hope this gave you a clearer mental model of what data actually is.

Because before you can work with data, you need to understand what you're actually working with.

Thanks for reading! If you found this useful, drop a reaction or share it with someone who's just getting started in tech.