DEV Community: Siddhartha bhattarai

Our AI Agent Looked Fine. SigNoz Showed Us It Was Not.

Siddhartha bhattarai — Sun, 19 Jul 2026 06:37:19 +0000

Team: The Badminton Philosophers

The three of us met on a college badminton court 6 months ago, started talking about AI, Cryptography and security between games, and never really stopped. That is where the name comes from. We are building an AI agent project for the Agents of SigNoz hackathon and we gave ourselves one day to explore SigNoz properly before the sprint started.

We built a small agent to explore with. It was supposed to answer questions by searching a knowledge base, doing a quick calculation if numbers were involved, and returning a clean answer. Simple enough that we could focus on the observability rather than the agent itself.

It ran. It returned answers. Everything looked fine.

Everything was not fine.

Setting up SigNoz first

Before we ran anything we needed SigNoz up. The current self-host path goes through Foundry, their CLI that provisions the whole stack from one config file.

curl -fsSL https://signoz.io/foundry.sh | bash

Create casting.yaml:

apiVersion: v1alpha1
kind: Installation
metadata:
  name: signoz
spec:
  deployment:
    flavor: compose
    mode: docker

Deploy:

foundryctl cast -f casting.yaml

Verify:

docker ps

Five containers need to show healthy. ClickHouse is the telemetry store. Postgres is the metastore. There is a ClickHouse keeper, the OTel collector, and the SigNoz app. Then http://localhost:8080 opens the UI.

One warning the documentation person caught before the setup person made this mistake: Foundry generates compose files into a folder called pours and owns them completely. Do not edit those files by hand. The next cast run overwrites every change. All configuration goes in casting.yaml.

We instrumented the agent with OpenTelemetry and OpenInference, sent it twenty test queries, and watched the spans land in SigNoz. The Services page showed our agent. Response times looked reasonable. No errors flagged anywhere.

We thought we were done with setup and about to move on to exploring features.

Then we opened the first trace.

The trace told a different story

The agent was taking between four and nine seconds to respond. We had assumed that was the LLM being slow. Reasonable assumption. LLMs are slow sometimes.

The spans told a different story. The agent.run span showed 188ms total. The llm.call underneath it took 188ms as well, meaning almost all the time was in the LLM. The tool.calculator span was 0.1ms, barely registering. But it was being called repeatedly and silently failing on malformed inputs.

The LLM call was taking about 800 milliseconds. Fast, actually. The thing eating all the time was the calculator tool. It was being called three and four times per query in some cases, each call taking over a second, and it was failing silently about half the time and just retrying without logging anything visible.

We would never have seen this from the outside. The agent returned an answer every time. The answers looked plausible. The response time was annoying but not obviously broken. From the surface everything looked like a slow but working system.

The trace showed a broken system that happened to produce output anyway.

Opening the logs made it worse

The setup person clicked through to the logs correlated with one of those traces. In SigNoz you can jump from a trace directly to the log lines that fired during that exact span. Same interface, one click, no timestamp hunting across separate tools.

The logs showed the calculator tool receiving inputs like "what is 4 times the result of the previous step" as literal strings. The agent was not resolving the reference before passing it. The tool was trying to parse "the result of the previous step" as a number, failing, and retrying with a slightly different phrasing that also failed, and eventually either getting lucky or returning a wrong answer confidently.

The LLM was fine. The tool was receiving garbage and trying to work with it anyway.

Pranay Prateek, co-founder and CEO of SigNoz, wrote in their funding announcement that many teams are stuck using only basic logs and metrics and cannot reach modern tooling like distributed tracing, often because of expensive pricing and compatibility constraints. That sentence landed differently after seeing what a trace revealed that logs alone never would have.

Without the trace we would have spent time tuning the LLM prompt trying to get better answers. The prompt was not the problem. The trace showed us that in about thirty seconds.

Metrics showed us the pattern across all twenty runs

We had been looking at individual traces. The metrics view showed us the same failure happening systematically.

The calculator tool was erroring about forty percent of the time across the full run. Not randomly. It clustered on queries that involved referencing a previous result. The LLM was producing a specific type of malformed input consistently and the tool was consistently failing on it.

The metrics made the pattern visible. The trace had shown us one instance. The metrics showed us it was not an edge case.

Dashboards and alerts: catching this next time before it happens

We built a dashboard tracking tool error rate and agent response time together so we could see them move in relation to each other.

Then we set an alert for tool error rate above twenty percent sustained for two minutes. If this happens again in a real deployment we will know about it before users start noticing the slow and wrong answers.

Ankit Nayan, co-founder and CTO, has been driving OpAMP support into SigNoz, a newer OpenTelemetry sub-protocol that lets you reconfigure the Collector dynamically without container restarts. The direction the platform is heading is toward an observability stack that adapts at runtime rather than at redeploy time, which matters more as the shape of what you are monitoring changes constantly.

The thing we found while looking for something else

While the setup person was running all of this, the person reading documentation found something in the Foundry config reference that made us stop what we were doing.

SigNoz ships an optional MCP server. MCP stands for Model Context Protocol. It lets an AI agent query SigNoz directly in plain language and get answers from real telemetry. Enable it by updating casting.yaml and rerunning cast:

apiVersion: v1alpha1
kind: Installation
metadata:
  name: signoz
spec:
  deployment:
    flavor: compose
    mode: docker
  mcp:
    spec:
      enabled: true

foundryctl cast -f casting.yaml

Check it is alive:

curl -fsS localhost:8000/livez && echo " OK"

We had been planning to write a polling loop that called the SigNoz REST API every sixty seconds to pull span data into our detection agent. The documentation person sent a message to the group that said something like "wait, does this mean we do not need any of that."

We do not need any of that. Our agent can ask SigNoz what it needs to know directly. The two-day integration layer we had planned does not exist anymore.

The person watching the SigNoz YouTube channel found this working in their Noz AI debugging assistant. A recent video showed Noz generating a plain language reliability report from a full week of telemetry automatically. Instead of manually comparing dashboards across time ranges, you ask a question and get an answer.

For a hackathon building AI agents on top of SigNoz, that is worth knowing before you plan your architecture.

What we would have built without traces

We would have spent the first day tuning the LLM prompt. We would have made the system message more explicit about not referencing previous results. We would have iterated on a few variations and run evaluations. Eventually we might have stumbled onto the fact that the calculator was receiving bad input, but we would have arrived there through guesswork, not evidence.

The trace showed us the actual failure in thirty seconds. Not approximately where things were slow. Not that errors were happening. Exactly which tool, exactly which span, exactly what input it received, exactly how many times it retried.

Hiro Tamada, a Founding Engineer at Kernel building AI agent infrastructure, made a point in a video that we kept coming back to during this day. He said that teams make the mistake of leaving metrics and dashboards until the end of a feature cycle and that the ones who start instrumentation on day one get to use traces as a continuous debugging tool rather than scrambling to add visibility after something breaks in production.

We built a broken agent and SigNoz showed us it was broken in the first thirty minutes. If we had shipped this to real users first we would have gotten confused complaints about wrong answers and spent days guessing at the cause.

One technical thing worth knowing

We initially sent data as unstructured log messages attached to spans rather than structured attributes with typed values. Querying unstructured text through the SigNoz Query Builder is painful compared to filtering on clean typed fields. Use structured span attributes from the start. Type your keys and values. The Query Builder is significantly more useful when the data underneath it is clean.

What we are building with all of this

We are building an agent that watches other AI agents for behavioral drift. Not resource spikes. Not error rates. Whether the agent is still producing semantically similar outputs week over week, because model updates and RAG database changes can shift how your agent behaves without changing a single line of code or triggering a single infrastructure alert.

The detection agent asks SigNoz which spans look semantically different from baseline. SigNoz answers through MCP. The agent scores the drift and pushes it back as a custom metric. When the score drops below threshold an alert fires.

We found the approach because we spent a day inside SigNoz before writing project code. The broken calculator agent is what gave us the idea. Watching traces reveal a failure that looked like success from the outside is what made us want to build a system that does that watching continuously and automatically.

We will show you what we built after the sprint.

— The Badminton Philosophers

Resources: SigNoz docs at signoz.io/docs, OpenTelemetry Python SDK at opentelemetry.io, SigNoz YouTube at youtube.com/@signoz

Getting Selected for the LFX Mentorship, and What I've Learned So Far About OpenSSL

Siddhartha bhattarai — Tue, 07 Jul 2026 21:07:57 +0000

I still remember checking my email that morning and seeing the subject line something like "Congratulations LFX Mentorship." I read it twice before it actually landed. Out of everyone who applied I got in, to work on CBOMKit, a project I'd only vaguely understood a few weeks earlier when I first read about post-quantum cryptography and thought wait, all of our current encryption is going to be broken someday? Also a huge thanks to Aditya koranga(my mentor for choosing me).

That was genuinely one of the best days I've had this year. Not because mentorships themselves are rare, if you look around there's plenty. It felt good because this one was specific. I'd been reading about PQC on my own out of pure curiosity not because a class told me to and suddenly I had a mentor, a real codebase and a real problem sitting in front of me instead of just articles.

This post is about what that problem actually is, what I've picked up so far and honestly what I still don't fully understand. There's a lot.

What I Thought OpenSSL Was and What It Actually Is

Going into this I genuinely thought OpenSSL was one thing. Like one library one job. Turns out it's four separate pieces that just happen to travel together. libcrypto which does the actual math (hashing digital signatures ciphers) libssl which is built on top of libcrypto to run the actual TLS and DTLS handshakes (this is the part that lets something like Zoom encrypt a live call) a command line tool and the EVP API which I now understand is the modern interface most serious codebases should be using partly because it was built with post-quantum support in mind. I didn't know any of this three weeks ago.

There's also BoringSSL which I'd heard of but never really understood. It exists because of Heartbleed back in 2014 a vulnerability that showed just how much of the internet was quietly resting on one under resourced codebase. Google forked OpenSSL after that so they could keep a leaner more aggressively updated set of primitives and just drop support for the old less secure stuff that OpenSSL still has to carry around for backward compatibility. Chrome runs on BoringSSL now. So does Bun apparently even though Google originally meant it to stay internal.

And Node.js for what its worth still wraps OpenSSL directly and the most recent releases actually do support post quantum algorithms natively through OpenSSL 3.5 and up. I only found that out because someone asked during a call and I had to go check.

Where This Actually Shows Up

The thing that made this click for me wasn't the theory it was seeing where OpenSSL quietly sits inside stuff I already use. Nginx uses it for TLS termination and for handling Server Name Indication which is the thing that let's one server host certificates for a bunch of different websites. PostgreSQL uses it to encrypt traffic between the database and whatever's talking to it and there's an extension called pgcrypto that uses OpenSSL directly for encryption inside the database itself. JWT libraries lean on OpenSSL for HMAC signing and verification.

None of this was surprising once I saw it. It was just never something I'd thought about before. Its the kind of thing you use everyday and never once wonder whats actually happening underneath.

The Part That Took Me a While to Actually Appreciate

Here's the thing I didn't get at first. Java's crypto APIs are clean. You call Cipher.getInstance("AES") and that's basically the whole story one call one clear answer. OpenSSL is nothing like that. The same operation can be spread across a struct here a macro there and an initialization call three lines down sometimes in a completely different function.

The first time this actually hit me was hearing that our detection module might miss a chunk of real usage simply because the information isn't sitting in one obvious place. And turns out that's not a hypothetical its a real currently open question. Nobody on the team can tell you right now exactly how much of OpenSSL's api surface we actually cover. That used to bother me a little when I first heard it. Now its basically become my job for the next two weeks and honestly I like that better.

There's also a more boring practical problem underneath all this. OpenSSL 3.6 restructured its own header files in a way that broke naive path resolution so the C/C++ side of things was literally failing to find headers it needed. That's being fixed by redirecting to the systems OpenSSL headers which sounds simple written out like that but took real debugging to land on.

Where Things Stand Right Now

C/C++ support is integrated into sonar-cryptography. There's an OpenSSL detection module already and it has what were calling meaningful coverage meaning it clearly works for the rules that exist. What it doesnt have yet is confirmed full coverage of everything OpenSSL exposes. Unit tests exist too but again only for the rules that have actually been written so far so test coverage is really just tracking rule coverage right now nothing more.

We're also deliberately not supporting older OpenSSL api versions at the moment. That's a real decision not something that got missed. Keeping the scope tight matters more right now than trying to cover everything at once.

My Actual Job Right Now

My first task sounds almost embarrassingly simple when you say it out loud. Find real open source C/C++ projects that use OpenSSL and actually test our plugin against them.

It sounds easy until you realize the point isn't just running a scanner and glancing at whatever comes out. Nginx PostgreSQL curl OpenSSH HAProxy Apache httpd Redis OpenVPN Postfix, these are all candidates and the goal isn't to grab five projects that all do the exact same TLS handshake pattern. It's to find ones that stretch different corners of the api, X.509 parsing here HMAC there EVP ciphers somewhere else DTLS in another. For each one I need to note what it actually uses legacy api or EVP roughly how big it is and why its actually a useful test rather than just a familiar name.

Small practical thing I already learned the hard way sort of, clone with a single branch instead of mirroring the whole repo. Someone mirrored OpenSSL's own repository earlier in this project and it took over thirty minutes just to clone. A single branch ideally shallow too avoids that entirely.

Once I have that list the actual work starts. Run the plugin against each project then go through the codebase myself and grep for the real OpenSSL calls, EVP_* SSL_CTX_* RSA_* HMAC_*, and compare that by hand against what the tool actually reported. Where it agrees fine. Where it doesnt thats a gap and I need to be the one who notices it not assume someone already checked. Im planning to track all of this in a simple table project name what OpenSSL apis it uses what got detected and notes on anything missing. Its slow unglamorous work. But its also the first point in this whole project where "coverage" stops being a vague comfortable word and turns into an actual number.

After that once the OpenSSL detection PR is merged on Shubham's side I move on to wiring things into cbomkit-lib properly making sure it pulls in the updated C/C++ module resolves the right parser as a dependency and can actually feed C/C++ source files through the fake in memory server setup it uses (which from what I understand is genuinely more involved for C/C++ than it was for Java or Python since C/C++ needs extra build context like compile commands and include paths just to be parsed correctly at all).

Alongside all of that Im also setting up a full SonarQube deployment through Docker installing the plugin properly and running it against one of the same cataloged projects Nginx or curl most likely. That's a genuinely separate path from cbomkit-lib even though they share the same detection engine underneath so both need to actually be checked on their own rather than assumed to behave the same way.

What I Don't Know Yet

I don't know yet how many gaps we'll find once I actually start running this against real projects next week. Could be a handful. Could be a lot. My honest guess based on how scattered OpenSSL's configuration patterns are is that its more than we'd like.

There's also a longer term goal sitting in the back of my mind now. My mentor sent me a paper recently BF-CBOM from a group at the University of Bern that basically ran multiple CBOM generators against the same codebases and found they barely agreed with each other at all. Almost no overlap. Its a slightly uncomfortable finding if your building one of these tools but its also exactly the kind of honest specific result we'd like to eventually be able to contribute ourselves. There arent many papers out there yet on CBOM tooling specifically and that feels like an actual gap worth trying to fill by the end of this mentorship not just a nice idea to mention.

For now though my job is smaller and more concrete than any of that. Find the projects. Run the tests. Write down what's actually true not what we assumed was true.

I'll write about it either way.

QWEN Cloud Hackathon - #1 Technical Deep dive on what I am building

Siddhartha bhattarai — Sun, 14 Jun 2026 18:07:41 +0000

The Problem: A week ago I went to my grandmother who is suffering from demantia. At first I thought it was normal at this second but i wanted to know the cause so i looked up in google and saw that over 55 million patentis are navigating a world where they wake up and cannot recognize their own children's faces. They miss critical life spanning medications and suffer catastrophic falls when unmonitored.

We cannot solve a dynamic, high stakes human crisis with a single, static LLM prompt or a standard chatbot wrapper. If a healthcare AI hallucinates a medication dosage or fails to recognize a family caregiver, the consequences are life threatening.

That's why we are introducing Smriti - Building a 4-Agent Elderly Care system with QwenVL, MemoAssistant and persevere thinking.

The Solution - 4 Specialized QwenNative Agents working together:

Agent1: - Vision Agent (Qwen-VL) -> Responsible For Recognition, medicine label reading, hazard detection.It is optimized to perform high speed facial detection, medicine label reading, and spatial hazard analysis (such as identifying a water spill on the floor or an open stove burner).

Agent2: - Memory Agent(MemoAssistant): Persistent KeyValue Storage Across sessions.The Memory Agent uses MemoAssistant to manage persistent key-value storage across sessions. It caches family profiles, authorized medical staff, and historical routines so the system doesn't have to re-evaluate static profiles on every frame loop.

Agent3: - Guardial Agent(Qwen 3.6 Max + Persevere Thinking) - Hallunication blocking with reasoning traces.It utilizes Qwen's trillion-parameter MoE architecture and forces a reasoning trace calculation using the preserve_thinking parameter. This layer acts as a strict hallucination blocker by forcing the model to explicitly evaluate safety weights and confidence before executing actions.

MOE STRUCTURE

Internal Reasoning for Guardrail Agent:

Analyzing Vision Agent input: Unidentified individual detected in bedroom at 23:15.
Cross-referencing Memory Agent database: No matching family token or caregiver schedule found.
Checking risk matrix: Midnight proximity to patient bed presents high hazard potential.
Evaluating validation metrics: Match confidence is 12.5%.
Action: Threshold (85%) unmet. Halt direct patient voice output; route to Caregiver Agent.

Agent4: - When the Guardrail Agent detects an anomaly or drops below the 85% confidence score, it forwards the state payload to the Caregiver Agent via the Model Context Protocol (MCP). This pushes live notifications, image clips, and the system's reasoning logs directly to a web-based dashboard for immediate human approval.

How Agents Communicate - Vision -> Memory -> Guardrail -> Response in under 2 seconds.

One of the reason to build this - Only in reddit there are like 70K poeple in the subreddit channel of dementia.

Medo — A vibecoding agent better than Loveable, Replit, Emergent you have never heard of and it’s ongoing hackathon

Siddhartha bhattarai — Mon, 04 May 2026 16:39:14 +0000

BuiltWithMeDo

FreshStart: Building a Global Student Support Network with MeDo
What Inspired Me
6.9 million international students move to a new country every year. Each one starts from zero. No roadmap. No local friends. No one to answer simple questions like "How do I find a room that won't eat my entire budget?" or "What documents do I actually need before getting my visa?"

I asked over 100 students a simple question: Would you be interested in a single platform for immigrants and students to help you learn about your college, surroundings, paperwork requirements, and connect with people already overseas?

82.5% said yes. 11.7% said yes but only with AI integrated.

That's 94.2% of students wanting exactly what FreshStart is building. So I stopped asking and started building.

The name "FreshStart" comes from exactly what every international student needs – a fresh start in a new country. And "Dai" comes from the Nepali word for elder brother – someone who guides you through unfamiliar territory. That's exactly what this AI assistant is meant to be.

What I Learned
Building FreshStart taught me more than any classroom ever could.

First, vibecoding is the future. MeDo allowed me to go from idea to working product without spending months on boilerplate code. I described what I wanted, and MeDo helped me build it. One person. One platform. One working application that serves thousands of students.

Second, students want AI, not just information. The survey data was clear – 94.2% want this platform, and over 12% specifically demanded AI integration. Generic checklists and static FAQ pages aren't enough anymore. Students want personalized guidance that understands their specific situation.

Third, fallback systems are essential. When building with free API tiers, rate limits are inevitable. I learned to build intelligent fallback responses that use localStorage data – so Dai always provides helpful answers even when API calls fail. A system that gracefully degrades is better than a system that crashes.

Fourth, community is the killer feature. The peer connector wasn't originally the main focus, but early feedback showed that connecting with students already overseas is just as valuable as getting AI advice. Real humans who have been through the journey – that's something no LLM can replace.

How I Built It
FreshStart was built entirely on MeDo, one of the most powerful vibecoding platforms available today.

The Tech Stack:

Frontend: React with TypeScript and Tailwind CSS

Backend: Supabase for database and authentication

Plugins: LLM Plugin, ExchangeRate Plugin, WebSocket Plugin

Deployment: MeDo's native deployment system

The 3 Plugins That Make FreshStart Work:

LLM Plugin: Powers Dai's conversational intelligence. It remembers each student's real data – name, destination city, moving date, budget, and checklist progress – and gives personalized answers, not generic responses. Built with Groq's free tier and automatic fallback to Gemini.

ExchangeRate Plugin: Converts budgets between 161 currencies in real time. Uses the Frankfurter API with 24-hour caching to minimize calls.

WebSocket Plugin: Enables live peer connections. When one student completes a visa application, everyone heading to the same city gets an instant notification. Built for real-time messaging and collaborative checklist updates.

The system follows a simple flow: User input → Context building (from localStorage) → Plugin routing → Fallback handling (if API limits reached) → Personalized response.

Due to limited API credits during the hackathon, the LLM plugin operates within free tier limits. But intelligent fallback responses use each student's real checklist, budget, and deadline data from localStorage – so Dai always provides specific, personalized answers. The complete architecture is production ready.

The Challenges I Faced
Challenge 1: API Rate Limits

The biggest technical challenge was staying within free tier limits. Groq gives 10,000 calls per day, which sounds like a lot until you're testing aggressively. I solved this by building a fallback system that serves checklist, budget, and deadline data directly from localStorage when API calls fail. Now Dai never returns an empty error – only helpful answers.

Challenge 2: Currency Confusion

Students from different countries were confused by USD-only budgets. A student from Nepal had no idea if $2000 was enough for Boston. I integrated the ExchangeRate Plugin to show budgets in local currency with daily breakdowns. Now everyone understands exactly what they can afford.

Challenge 3: Making AI Actually Useful

Most AI chatbots are generic and useless. "I'm here to help!" – help with what? I solved this by giving Dai access to each student's real data. Dai knows your name, your city, your moving date, your budget, and exactly what tasks you still need to complete. Ask "Am I on track?" and Dai tells you the truth with specific numbers and deadlines.

Challenge 4: Building Alone

Building a full-stack application alone is hard. But MeDo's vibecoding approach meant I could focus on product decisions instead of fighting with boilerplate code. What would have taken months with a traditional stack took weeks with MeDo.

Challenge 5: Proving It's Not Just Another App

Students have seen hundreds of "helpful" apps that don't actually help. The survey data (94.2% wanting this) validated the idea. Early user feedback confirmed we're solving real problems. The challenge now is execution – and we're just getting started.

What's Next
Real-time WebSocket messaging so students can chat instantly with peers in their city and with students already overseas. Deeper AI personalization that learns each student's unique needs over time. A global student support network that doesn't exist anywhere else.

94.2% of students want this. The data is clear. The need is urgent. And now the tool exists.

Try FreshStart
FreshStart is live. Some features are still being improved, and we're working on them daily. But the core experience works.

Try the room finder. Test the budget planner. Ask Dai about your checklist. Connect with a peer.

This is just the beginning.