Agent configs and billing exports often contain secrets: API keys, account IDs, internal service names. So when I built Arup Banerjee Labs, I set one rule first: whatever you paste stays in your browser.
The result is three small, free tools that turn a paste into a scorecard.
The constraint that shaped the suite
Most "analyze your config" tools want you to upload a file or grant OAuth access to your cloud org. That's a big ask when all you want is a quick sanity check. A static site that parses and scores locally removes the trust problem: there's no backend to leak to, because there's no backend.
Everything is hosted on GitHub Pages, and the source is public: https://github.com/ultimatixarup/arup-banerjee-labs
Tool 1: Agent Health Checker
Paste an AI agent config (JSON, YAML, or plain text) and get a reliability/privacy scorecard. It looks for the things demos tend to hide:
- retries and backoff
- timeouts
- tracing hooks
- secrets sitting in the config
- tool-auth notes and injection-prone tool sinks
- evals/tests
- a single LLM with no fallback
https://ultimatixarup.github.io/arup-banerjee-labs/agent-health-checker/
Tool 2: Cloud Bill Smell Detector
Paste a cost CSV, invoice export, or billing JSON that you exported from your own account, and get an educational smell scorecard: idle-looking lines, missing tags, transfer-heavy patterns, and similar heuristics.
Explicit non-goals: it never connects to AWS, GCP, or Azure, and it only looks at text you paste.
https://ultimatixarup.github.io/arup-banerjee-labs/cloud-bill-smell/
Tool 3: Observability Gap Finder
Paste OpenTelemetry, Prometheus, or Splunk-style config snippets, or a service inventory YAML, and get a gaps scorecard: missing SLIs, services without tracing, no cardinality guards.
https://ultimatixarup.github.io/arup-banerjee-labs/observability-gap-finder/
What "heuristic" means here
These are prioritization aids, not audit stamps. They'll produce false positives, and they won't catch everything. The goal is a fast "what should I look at first?" before something hits production.
Verify the privacy claim yourself
Open your browser's DevTools, switch to the Network tab, paste something, and run a score. Your paste should never leave the page. If you ever see it go out, that's a bug and I want to know.
What's next
I'm collecting feedback on wrong and missing checks. If one of these flags something silly, or misses something obvious in your stack, open an issue on the repo or leave a comment here.
Try the suite: https://ultimatixarup.github.io/arup-banerjee-labs/
Arup Kumar Banerjee · Little Elm, Texas
Questions? Message my free help bot on Telegram: t.me/ArupLabsHelpBot_bot
Personal project built on my own time. Not affiliated with or endorsed by my employer; views are my own.
Top comments (17)
the devtools network tab test is the strongest part of this. "we never see your data" is usually a vibe, making it verifiable is what makes it real.
of the three, the agent health checker is the one i'd reach for first. secrets-in-config and injection-prone tool sinks are the checks that actually bite people, the rest is nice to have.
one thing i'd think about: if the feedback loop for wrong checks lives in github issues, someone will eventually paste a config with a live key into a "this flagged nothing" report. the privacy story is airtight on the tool side, the report side is the leakier surface.
Thanks Om, this is really useful. Agreed that secrets-in-config and injection-prone tool sinks are the checks that bite. And you're right, the report side is the leakier surface. I'm going to add a clear "redact keys and tokens before you paste" warning to the GitHub issue template so nobody drops a live key into a report.
nice, glad that landed. the template warning is exactly the fix i'd have suggested, catches people at the point of paste instead of after. though i'd bet the next leakier-than-expected surface is people pasting their scorecards into a chatbot to "explain my bill". one hole at a time.
Ha, good call. The scorecard Markdown is meant to stay with you — pasting it into a chatbot to explain a bill reopens the privacy hole. One hole at a time.
yeah, we're converging on the same fixes. the annoying bit is the ordering: people do the careful local thing, then paste the raw scorecard into a chatbot for "explain my bill". the template warning catches github reports, a one-liner in the scorecard itself would catch the artifact that actually travels.
Yep — the issue template catches the report path, but the Markdown export is what people actually carry around. I'll put a one-liner on the scorecard itself so the warning travels with the artifact.
Shipped. Copy Markdown on all three tools now opens with a one-line note to redact secrets, account IDs, hostnames and customer data before sharing or pasting into a chatbot. Thanks for pushing on this, Om.
love that it shipped. a one-line nudge right where the export happens beats a warning paragraph people scroll past.
Agreed — right at the export is where people actually look. Glad it landed.
the fact it shipped as a one-liner instead of growing into a redaction pipeline is what i liked. next question is whether anyone reads it before pasting the markdown into a chat window.
Ha, that's the honest open question. Keeping it to one line was the bet: short enough that people might actually read it before they paste.
honest answer to the open question: don't ask anyone, measure it. the scorecards that come back through your issue template are the sample. if more of them arrive redacted than before the nudge, the line is doing its job. behavioural metric, no survey needed.
That's a better answer than mine. The reports coming back through the template are the sample, and redacted vs. not is a clean signal. Noted, thanks Om.
glad it was useful. the shipping speed on this whole thread has been fun to watch.
Thanks Om. Having you poke at it in public made it easy to move fast. Appreciate you sticking with the thread.
the speed is half the fun honestly, easy to follow along when things keep moving. keep shipping in public.
Thanks Om — having you follow along in public is half of why it's fun to keep shipping.