DEV Community: Neha Prasad

Traccia vs. Maxim: Observe Agent Quality or Enforce Agent Bounds?

Neha Prasad — Thu, 16 Jul 2026 07:10:52 +0000

You're shipping AI agents to production. You need visibility. Maybe guardrails. Maybe proof for compliance.

Two names keep coming up: Maxim and Traccia.

This isn't a "winner takes all" comparison. These tools sit at different layers of the agent lifecycle — and the teams that confuse them often buy the wrong thing.

Maxim helps you simulate, evaluate, and observe agent quality.

Traccia helps you observe agents, enforce policy at the agent boundary, and prove what happened.

Enforce, not just observe. That's the distinction this article is about.

Why this comparison matters now

Production AI moved fast:

Agents went from demos → customer-facing workflows
"It worked in the playground" stopped being enough
Finance asks about token spend. Legal asks about evidence. Eng asks why Agent B failed at 2am.

You need quality confidence before ship and runtime control after ship.

Maxim and Traccia address overlapping words — tracing, observability, safety — with different philosophies.

At a glance

Dimension	Maxim	Traccia	Edge
Layer of the stack	Simulate → evaluate → observe quality	Runtime observability & control plane	Complementary
Visibility	Multi-agent visual traces, live debugging	OTel tracing, lineage, per-agent dashboards	Parity
Intelligence (cost)	Cost/latency in observability views	Sampling-accurate cost + anomaly detection	Traccia
Agent-boundary control	Online evaluators + safety alerts	`@govern` + platform policies	Traccia
Guardrail posture	Toxicity / RAI evaluators on live traffic	3-tier detection proving controls fired	Different approach
EU evidence from traces	Vendor certs	Article-mapped evidence packs from OTel spans	Traccia
Simulation / prompt IDE	Native strength	Roadmap	Maxim
Developer SDK	Python, TS, Java, Go + webhooks	Python & TypeScript OTel auto-instrumentation	Maxim (breadth)

Visibility: eval-linked traces vs agent telemetry

Maxim

Strength: production debugging tied to quality workflows.

Visual multi-agent traces
Live issue tracking
Online evaluations on generations, tool calls, and retrievals

Great when your question is: "Why did this conversation score badly?"

Traccia

Strength: operational agent telemetry.

Per-agent tracing with errors, latency, throughput
Multi-step decision lineage and tool-call graphs
Import-time auto-instrumentation for major LLM stacks
W3C OTLP to Traccia Cloud or any OpenTelemetry backend

Great when your question is: "What did this agent do, how much did it cost, and can I export it anywhere?"

from traccia import init, observe

init()

@observe(as_type="agent")
def run(prompt: str) -> str:
    return call_llm(prompt)

Takeaway: Both give you traces. Maxim optimizes for eval-linked debugging. Traccia optimizes for OTel-native ops telemetry.

Intelligence: cost as a production signal

Maxim surfaces cost and latency alongside eval scores — useful for optimization loops tied to quality.

Traccia treats cost as a first-class control signal:

Token-level cost per agent and model
Metrics that stay accurate under trace sampling
Historical recomputation across a 2,500+ model pricing registry
Cost anomaly detection

Why sampling-accurate cost matters: if you sample 10% of traces, naive cost dashboards lie. Spend Cap policies need trustworthy numbers — independent of sample rate.

That's Intelligence feeding Control.

Control: two enforcement philosophies

This is the core fork.

Maxim — evaluate and alert on live traffic

Online evaluators and safety alerts: toxicity checks, RAI scorers, regression alerts on production conversations.

That's quality control on outputs and traffic patterns — not necessarily a hard gate before every agent invocation.

from maxim import Maxim  # illustrative

maxim = Maxim({"api_key": "..."})
# Traces + online evaluators land in Maxim observability views

Posture: Detect → score → alert → human review.

Traccia — policies + `@govern` at the agent boundary

Control is embedded in the application path:

Platform policies: Spend Cap, Retry Protection, Duration Limit, Token Limit, Error Rate
@govern checks agent status before invocation
hard_block raises AgentBlockedError — function body never runs
Soft blocks warn and continue

from traccia import init, govern
from traccia.governance import AgentBlockedError

init(api_key="...", endpoint="https://api.traccia.ai/v2/traces")

@govern(agent_id="onboarding-agent", fail_open=False)
def run_agent(user_msg: str) -> str:
    return agent.run(user_msg)

Posture: Gate → block or allow → prove controls fired.

Guardrail detection (Explicit / Provider-native / Heuristic) proves controls existed on a run. @govern enforces the next run.

Certification: vendor trust vs trace depth

Capability	Maxim	Traccia
Production trace debugging	Visual multi-agent traces	Per-agent lineage + registry
Online safety evaluators	Native strength	Guardrail findings + policies
EU AI Act evidence from traces	Not a primary module	Integrity-hashed packs from OTel traces
FRIA drafts (Art. 27)	Not a primary module	Wizard → downloadable JSON
Governance Hub	Human eval pipelines	Registry, reviews, incidents, evidence export
Simulation / prompt IDE	Playground++, scenarios at scale	Roadmap

Maxim's SOC 2 / ISO 27001 / HIPAA / GDPR posture covers Maxim as a vendor.

Traccia's Certification pillar is depth on your application: governance enrichment on spans, FRIA draft wizard, disclosure() trails, article-mapped evidence packs from live telemetry.

Where Maxim leads

Choose Maxim when the bottleneck is quality confidence across the development lifecycle:

Large-scale agent simulation (thousands of scenarios)
Prompt IDE (Playground++) with versioning and no-code collaboration
Rich evaluator library + human-in-the-loop pipelines
Cross-functional UX for product and design teams
Bifrost LLM gateway for routing needs

Home turf: pre-production quality, simulation, collaboration.

Where Traccia leads

Choose Traccia when agents are live and you need observe → limit → prove:

Developer-native visibility with per-agent ops dashboards
Sampling-accurate cost intelligence powering Spend Cap policies
@govern hard blocks and platform policies at the agent boundary
Guardrail posture as evidence that controls fired
EU AI Act evidence packs from the same OTel stream
OpenTelemetry-first — no proprietary trace lock-in

Home turf: production runtime control and certification.

The bottom line

Choose Maxim if…

You need to simulate, evaluate, and iterate on agent quality — including no-code collaboration and human review — across the development lifecycle.

Choose Traccia if…

You need to enforce agent bounds in production: visibility and cost intelligence on OpenTelemetry, control via policies and @govern, and certification evidence from the same spans.

Choose both if…

You're mature enough to separate concerns:

Maxim → quality loops, simulation, eval pipelines pre- and post-ship
Traccia → runtime enforcement, cost control, audit evidence in production

They're complementary layers — not duplicates.

Discussion

Where does your team feel the most pain today?

[ ] Pre-production eval and simulation
[ ] Production tracing and debugging
[ ] Cost control and spend caps
[ ] Compliance evidence from agent runs
[ ] All of the above (welcome to 2026)

Tell me in the comments I'll share how teams typically sequence these tools.

Resources

LangSmith vs Traccia: Observe vs Enforce in Production AI Agents

Neha Prasad — Wed, 15 Jul 2026 05:43:38 +0000

LangSmith vs Traccia: Observe vs Enforce in Production AI Agents

Updated July 2026

LangSmith helps you debug and ship agents in the LangChain ecosystem.

Traccia helps you observe agents across frameworks, enforce policy at the agent boundary, and prove what happened.

Enforce, not just observe.

This isn't a "which tool is better" post. It's a stack-layer question: agent engineering vs runtime control plane.

TL;DR

Dimension	LangSmith	Traccia
Stack layer	Agent engineering & trace-first debugging	Runtime observability & control plane
Visibility	Nested LC/LG traces, Insights clustering	OTel tracing, lineage, per-agent dashboards
Cost intelligence	Workflow cost dashboards	Sampling-accurate cost + anomaly detection
Agent-boundary control	Online evals + alerts	`@govern` + platform policies (hard block)
Multi-framework	Good / improving	OTel-first, framework-agnostic
Offline evals / prompt hub	Native strength	Roadmap
EU evidence from traces	Enterprise regions	Article-mapped evidence packs from OTel spans

Choose LangSmith if you live in LangChain/LangGraph and need debugging + eval velocity.

Choose Traccia if you ship across frameworks and need operational limits + audit-ready evidence on OpenTelemetry.

Introduction

LangSmith is LangChain's agent engineering platform: deep observability for chains, tools, and agent trajectories (especially LangChain / LangGraph), production monitoring with cost and latency dashboards, online evaluators, and expanding deployment tooling.

Traccia is the developer runtime control plane built on four pillars:

Visibility → Intelligence → Control → Certification

Instrument once with OpenTelemetry. Attribute cost accurately under sampling. Define operational policies. Gate agents with @govern. Export evidence from the same spans — without locking you to a single framework.

Visibility: LangChain-Native Traces vs Agent Telemetry

LangSmith's strength

Zero-config tracing for LangChain / LangGraph apps:

Nested runs and tool calls
Thread-level debugging
Insights clustering for failure modes

If your agents are LC-native, this is best-in-class debugging UX.

Traccia's strength

Operational telemetry across any stack:

Per-agent tracing (errors, latency, throughput)
Multi-step decision lineage and tool-call graphs
Import-time auto-instrumentation for OpenAI, Anthropic, LangChain, CrewAI, OpenAI Agents SDK
W3C OTLP to Traccia Cloud or any OpenTelemetry backend


python
from traccia import init, observe

init()

@observe(as_type="agent")
def run(prompt: str) -> str:
    return call_llm(prompt)

Your AI agent just blew $500. Now what?

Neha Prasad — Fri, 03 Jul 2026 10:52:58 +0000

A few weeks ago, a founder told me his customer-service AI agent quietly burned through $1,200 in a weekend. The logs showed every API call. Not one flag. Not one alert. Just a silent billing massacre.

That’s why Traccia exists.

The Problem Nobody Talks About

We’re past “ChatGPT demos.” Teams are shipping AI agents that can email clients, update databases, issue refunds, and call other APIs. That’s great—until something goes wrong.
Logs tell you what happened. They don’t tell you:
Whether the agent should have done it at all
How much it’s costing in real money
If it’s breaking company policies or regulatory rules
If your agent can spend money or touch customer data, you’ve already graduated from “let’s just add logging.” You need governance.

What Traccia Actually Does

Traccia is an open-source platform that monitors your AI agents and—more importantly—governs them. It’s built by Algen, and I work on it as a Developer Advocate Engineer (so yes, I’m biased, but bear with me).

You add a couple of lines to your agent code. Suddenly you can see:

Every step your agent took (traces)
How many tokens it ate and what it cost
Whether it triggered any guardrails
And if it broke any policies you set

And here’s the kicker: you can stop it mid‑flight. Enforce spending caps. Block risky tool calls. Require human approval for sensitive actions. Not just “alert me when it’s too late.”

Why Now?

Because production AI agents are running loose, and most monitoring tools are still stuck in passive mode. LangSmith will show you a beautiful trace of your agent lighting $200 on fire. Traccia will snuff the match.

A Real Example

Imagine a support agent that can:

Answer billing questions
Issue refunds through a process_refund tool
Escalate to a human

Without governance, it could:

Loop endlessly and rack up LLM costs

Refund the wrong customer $500

Skip the escalation step entirely

With Traccia, you set policies:

Max $2 LLM spend per conversation

Refunds over $50 → human approval required

Guardrail check before any tool runs

When something goes sideways, Traccia blocks the action, logs it, and gives you a full audit trail. Finance doesn’t scream. Compliance is happy. You sleep better.

How It Compares

LangSmith – Great for debugging chains and evals. Shows you what happened. No governance.

TraceRoot – Focuses on debugging agentic failures. Good for RCA, not runtime control.

Traccia – Combines observability with active policy enforcement. It’s the “control plane” missing from most stacks.

Getting Started Is Stupidly Simple

bash
pip install traccia
Then in your agent code:

python
from traccia import init, observe
init() # auto-patches OpenAI, Anthropic, LangChain, etc.

@observe()
def run_agent(query):
return agent.run(query)
That’s it. Traces, costs, guardrails, and governance—all live.

Open Source, Real Transparency

GitHub: https://github.com/traccia-ai/traccia-py

Apache 2.0 license. It’s also listed in the OpenAI Agents SDK docs as an external tracing integration—one of the few, and the first built by an Indian team.

Let’s Talk

If you’re running agents in production (or about to), try Traccia. Break it. Send feedback. Star the repo. I’m building this in the open, and I’d love to hear what you think — especially the horror stories. Because we’ve all got them.

I’ve shipped 150+ PRs and built AI agents in a day - but I still can’t get a job

Neha Prasad — Mon, 22 Jun 2026 11:14:33 +0000

I'm writing this because I don't know what else to do.
This is Neha. I am a software engineer. I have been coding for years, contributing to open source, building autonomous AI agents, and shipping production code into frameworks that are used by thousands of companies. But today I see an empty bank account and a calendar that tells me I have less than two weeks to figure something out.
This is not a tale of woe. It’s a job application, but not the sort you are used to.

What I’ve done in reality

I am not a CS graduate from a renowned university. I don’t have an FAANG internship on my CV. What I do have is a trail of pull requests merged that speak for themselves.
150+ pull requests merged into core AI infrastructure (Mastra, LlamaIndex, LiteLLM, PostHog, Next.js, OpenHands, etc.)
Tracked down the bug in the streaming state machine and serialization layer for Mastra's silent agent freeze when using Anthropic’s Programmatic Tool Calling and shipped a fix across 21 packages.
Built DigiNav AI, an autonomous compliance agent that manages multi-week government filing processes for Indian businesses, featuring a LangGraph state machine, hybrid DPI integration, and human-approval gates.
Built CompliScore, a compliance health scanner, in one day - Next.js, Groq, real-time streaming, and polished UI. “Exactly what the Indian ecosystem "needs" - validated and lived by a CA-turned-founder backed by a VC.
Created Agent Blueprint, a visual AI agent builder (similar to ComfyUI for LangGraph) that is open source, drag-and-drop, and exports runnable Python code. Constructed in a week.
Currently building ExplainAI, a layer that makes AI agent decisions transparent to non-technical stakeholders, with bias flags and a “Human Override” button.
I ship quick. I understand AI agents at the framework level and above. I don't wait for permission to build what matters.

The reality of the job search

I’ve applied to 100+ companies in the past couple of months. YC companies, VC-backed scaleups, and open-source companies. I've DM'ed founders cold, made PRs to get noticed, and posted my work on Twitter and LinkedIn. I’ve had a few calls, some positive comments, and even a founder said my work is “just what India needs." But no offers yet.
I hear people say remote jobs are too competitive. I should move to Bangalore. I need a network. I know all that stuff. But I also know I can build things that most developers can't, and I do it faster than almost anyone.
Well, here I am, writing this article, hoping the right person reads it. Maybe a CTO, founder, hiring manager will see my work and think: “We need someone who can ship like this."

What I’m looking for:

I am looking for a remote full-stack or front-end engineering position to continue building AI-native products. I am open to contract work, full-time positions, or even a paid trial project. I'm on European and US hours. I offer
Strong experience with TypeScript, React, Next.js, Node.js, Python, and FastAPI.
AI agent orchestration in the real world (LangChain, LangGraph, Mastra).
Proven ability to ship fast – entire products in days, critical fixes in hours.
The hunger of the man who has everything to prove.
If you're building an AI product, an agent platform, a developer tool, or anything that needs to be shipped fast and clean – I'm your engineer.

Let’s chat

My Github is here: https://github.com/nehaprasad-dev
Portfolio: https://neha-portfoliooo.vercel.app/
Live compliance scanner: compliscore-nu.vercel.app
Twitter/X https://x.com/nehaaaa_6
Email ID: nehaprasad27118@gmail.com

I know I’m not the only one in this position, I’m not the only one in this boat. If you're another dev grinding through the job search, I see you. Let's keep building. Let’s keep on shipping!
If you're hiring full‑stack AI engineers who ship fast, I'm available for remote contract and full‑time work. [Link to portfolio / DM me.]

I’m Building a Visual AI Agent Builder - Because I’m Tired of Debugging Invisible State Machines

Neha Prasad — Tue, 02 Jun 2026 08:20:16 +0000

Every developer I know who builds AI agents has the same ritual: they stare at LangGraph code, draw mental maps, and pray the agent doesn’t silently freeze mid‑loop. I’ve been that person more times than I can count—including once when I tracked a streaming freeze in Mastra down to a missing synthetic tool‑call chunk.

The problem isn’t that we don’t know how to write agent logic. It’s that we have no simple, visual way to see the agent before it runs.

So I’m building Agent Blueprint.

What is it?
A drag‑and‑drop editor where you connect nodes like LLM Call, Tool Execution, Condition, and Human Approval — and export a working LangGraph Python script with one click.

Why now?
LangGraph usage is exploding, but the developer experience is still raw. We deserve a visual canvas that feels as natural as ComfyUI does for image workflows. I’ve merged 150+ PRs in the frameworks that run modern agents (Mastra, LlamaIndex, LiteLLM), so I’m building this from the inside out — with real understanding of where agents break.

What’s next?
I’m documenting the whole build in public — raw updates, architecture decisions, and a live demo this week. If you’ve ever lost an evening debugging a state graph, follow along. And if you want to influence what it becomes, drop your biggest agent‑debugging pain in the comments.

https://www.linkedin.com/feed/update/urn:li:activity:7467474353769701377/
https://x.com/nehaaaa_6/status/2061705090357215682

CompliScore a free compliance health check for Indian startup founders.

Neha Prasad — Mon, 01 Jun 2026 07:00:27 +0000

What I Built
CompliScore - a free compliance health check for Indian startup founders.

Type any company name and get a score out of 100, a list of overdue filings (GSTR-3B, MCA annual returns, notices), a penalty estimate, and a plain-English action plan — in under a minute, no login required.

For this challenge, I rebuilt the AI layer around Hermes Agent. Instead of a single LLM prompt, a reasoning agent plans an investigation, calls deterministic compliance tools, and writes a prioritized report — with a collapsible Agent investigation panel so you can see every plan step and tool call.

Compliance is a natural fit for agentic work: a company with overdue GST needs a filing-calendar deep-dive; one with active notices needs triage; a clean profile needs a light touch. One prompt can't adapt to all three. An agent that picks tools based on what it finds produces tighter, grounded reports.

Note: CompliScore uses fictional demo data only — it does not connect to government portals.

Demo
Live app: https://hermes-scout.vercel.app
(Replace with your actual Vercel URL if it's different.)

Try these scans:

Mumbai Chai Co. — high risk (overdue GST, MCA, multiple notices)
Razorpay — clean profile, score 100
When Hermes is enabled, expand Agent investigation under the AI action plan to see the tool-call trace.

Code
Repository: https://github.com/nehaprasad-dev/hermes-scout

File Role
lib/agent/orchestrator.ts
Plan → tool-call → observe loop
lib/agent/tools.ts
Four compliance tools + validating dispatcher
lib/agent/hermes-client.ts
OpenAI-compatible Hermes client
components/agent-trace.tsx
Collapsible trace UI
My Tech Stack
Next.js 16 (App Router) + TypeScript
Tailwind CSS v4 + Framer Motion
Hermes Agent — OpenAI-compatible /chat/completions with function calling
Groq (llama-3.3-70b-versatile) — fallback when Hermes is unavailable
Vitest — 36 automated tests
Vercel — production deploy
How I Used Hermes Agent
Hermes is the primary reasoning engine when enabled. Every scan runs:

computeHealth() → Hermes agent loop → aiSummary + agentTrace
↓ on failure
Groq → static fallback
Planning: The agent decides which tools to call based on the company profile — not a fixed script.

Tool use: Four deterministic tools so scores are never hallucinated:

Tool Purpose
score_company
Score, risk level, pending tasks
estimate_penalty
GST / MCA / notice breakdown
filing_calendar
Statutory deadlines (90-day horizon)
classify_notices
Severity labels for pending notices
Multi-step reasoning: Hermes reads tool results and writes a founder report — summary, biggest risk, prioritized action plan, and filing calendar — in plain English.

Transparency: Each run returns an agentTrace (plan + tool calls) shown in a collapsible UI panel — no secrets or URLs exposed.

Reliability: Three-tier fallback (Hermes → Groq → static) so scans never break, even when the agent endpoint is down.

Hermes fit this project because it's self-hostable, tool-calling native, and runs over a standard OpenAI-compatible API — while CompliScore's scoring engine stays the source of truth in lib/scoring.ts.

Tags: #hermesagent #ai #nextjs #typescript #startup #opensource #showdev

# DEV Submission Build With Hermes Agent

Neha Prasad — Mon, 01 Jun 2026 06:50:47 +0000

Submission Template

Challenge: Build With Hermes Agent

Project: CompliScore AI compliance health checks for Indian startups

Repo/live-demo: https://github.com/nehaprasad-dev/hermes-scout

What I built

CompliScore gives Indian startup founders a compliance score out of 100 in under a minute - overdue GST filings, MCA returns, penalty exposure, and a plain-English action plan.

The upgrade for this challenge: I replaced the one-shot Groq summary with a Hermes Agent reasoning loop that plans an investigation, calls deterministic compliance tools, and writes a prioritized report - with a collapsible agent trace so judges can see the agentic work.

Why an agent loop fits here

Compliance analysis is conditional. A company with overdue GST needs a filing-calendar deep dive; one with active notices needs notice triage; a clean company needs a light touch. A single prompt guesses all of this at once. An agent that calls tools based on what it finds produces tighter, grounded reports.

Hermes Agent integration

Scan → computeHealth (deterministic score)
     → Hermes Agent loop (plan → tool calls → report)
     → agent trace in UI
     ↓ on failure
     Groq one-shot → static fallback

Four tools exposed to Hermes (scores never hallucinated):

Tool	Purpose
`score_company`	Canonical score, risk level, pending tasks
`estimate_penalty`	GST / MCA / notice penalty breakdown
`filing_calendar`	GSTR-3B, GSTR-1, MCA deadlines (90-day horizon)
`classify_notices`	Severity labels for pending government notices

The agent runs over Hermes's OpenAI-compatible /chat/completions API with function calling — self-hostable via vLLM, LM Studio, Ollama, etc.

Transparency: Every successful agent run returns an agentTrace — plan steps, tool names, compact result previews — rendered in a collapsible panel under the AI action plan.

Reliability: Three-tier fallback (Hermes → Groq → static). Scans never break.

Tech stack

Next.js 16 (App Router), TypeScript, Tailwind v4
Hermes Agent (OpenAI-compatible tool-calling loop)
Groq fallback (llama-3.3-70b-versatile)
Vitest — 35 tests covering tools, orchestrator, and fallback paths

Try it

Visit [live demo URL]
Scan Mumbai Chai Co. (high risk — overdue GST, MCA, notices)
Expand Agent investigation to see the tool-call trace (when Hermes is enabled)
Scan Razorpay for a clean profile

How Hermes Agent is at the heart of this project

This is not a thin wrapper around a single prompt. The scan route calls runComplianceAgent(), which:

Sends the structured health profile + tool schemas to Hermes
Runs a bounded loop (plan → tool call → observe → reason)
Records each step in AgentTrace
Returns the final founder-facing report

The deterministic scoring engine (lib/scoring.ts) stays the source of truth — Hermes reasons about structured facts rather than inventing numbers.

Key files:

lib/agent/orchestrator.ts — agent loop
lib/agent/tools.ts — tool definitions + dispatcher
lib/agent/hermes-client.ts — OpenAI-compatible client
components/agent-trace.tsx — trace UI panel

Production setup

On Vercel (recommended — avoids heavy local LLM inference):

GROQ_API_KEY=your_key          # fast, reliable summaries
HERMES_ENABLED=false             # or true + a public Hermes URL (not localhost)
LEAD_WEBHOOK_URL=your_webhook    # optional

Set HERMES_ENABLED=true only when you have a publicly reachable Hermes endpoint. localhost:8000 works for local dev with a running Hermes server; it cannot work on Vercel.

See DEPLOY.md in the repo for full instructions.

What I learned

Tool design matters more than prompt length — keeping tools bound to pre-computed health data prevents score hallucination
Agent transparency (the trace panel) makes agentic work legible to users and judges
Graceful fallback is non-negotiable for production — Hermes → Groq → static means the product never feels broken

Fixing a Frustrating Bug in LiteLLM Guardrails

Neha Prasad — Tue, 26 May 2026 13:21:00 +0000

You set up custom regex patterns in a Content Filter guardrail, save it, then later toggle the mode (mask ↔ block) or make a small edit - and suddenly all your patterns are gone.
Super annoying.

What was happening?
In LiteLLM’s guardrail settings, the patterns and blocked words were only being sent in the update request if they had changed. So if you only toggled the mode or edited something else, the patterns got wiped out on save.

What I fixed
I updated the logic so that patterns and blocked words are always included when updating a guardrail - whether they changed or not.
Now your custom patterns stay safe no matter what you edit.
PR: https://nehacodes.hashnode.dev/fixing-a-frustrating-bug-in-litellm-guardrails

Lesson Learned

In update operations, never assume existing data will stay unless you explicitly send it. It’s safer to always send the full current state for important fields.

Small fixes like this make developer tools much more reliable and less frustrating to use.

I Built a Compliance Health Scanner for Indian Startups in 24 Hours - Here’s What I Learned

Neha Prasad — Tue, 26 May 2026 12:55:41 +0000

I built this in a day. Not because it was easy, but because I wanted to prove something: Indian founders don't need another complicated dashboard. They need someone to tell them, in plain English, what's broken and what to fix first.

Last weekend, I sat down and built CompliScore – a free 60-second compliance health scanner for Indian startups. You type your company name, and it gives you:

A Compliance Score (out of 100)

A list of pending GST & MCA filings

A penalty risk level (Low / Medium / High)

An AI‑generated action plan written in simple language

🔗 Live Demo: compliscore-nu.vercel.app

Why I Built This

I'm Neha, a software engineer from Mumbai. I've been deep in the compliance automation space for months, building DigiNav AI – an autonomous agent that will eventually handle actual GST filings, incorporation, and audits end‑to‑end.

But the big product takes weeks to build. I needed something today that could help founders right now and also serve as a live portfolio piece. So I stripped everything down to the single most useful action: Show me my risks.

I've also contributed 150+ PRs to core AI agent frameworks (Mastra, LlamaIndexTS, LiteLLM, PostHog) – so I understand both the compliance domain and the AI orchestration under the hood.

What I Learned While Building It

Founders don't care about compliance data. They care about consequences. Most tax-tech tools dump a table of due dates. That's useless. What a founder actually wants to know is:

Am I about to get a penalty?

How much will it cost?

What's the ONE thing I should do right now?

I designed the scanner to answer exactly that. The score and risk badge give an instant emotional signal (green = relief, red = danger), and the AI‑written action plan gives a clear next step in 150 words.

2. Speed builds trust.

The scanner uses mock data for now (real GST/MCA integration is in progress), but the UI flow and the AI logic are production‑ready. I built it in <8 hours using Next.js, Tailwind, shadcn/ui, and Groq for the AI summary. When you ship fast and share publicly, people notice. Within 48 hours of launching, I got an inbound message from a VC‑backed founder building in the exact same space.

3. Personal projects are the strongest resume

.
My GitHub account was flagged due to a 2FA loss, and I lost direct access to my old profile (150+ PRs). But this scanner, built in a day and deployed publicly, has done more for my credibility this week than my entire contribution graph did in a year. Recruiters and founders want to see live proof, not just a heatmap.

Tech Stack (for the curious)
Frontend: Next.js 15 (App Router), Tailwind CSS, shadcn/ui, Framer Motion

AI Layer: Groq API (Llama 3.3 70B) – generates the plain‑English action plan

Data Engine: Static mock data with a scoring algorithm that calculates compliance health based on overdue filings, missing returns, and pending notices

Deployment: Vercel (free tier, instant deploys)

What's Next

I'm currently building the full autonomous agent (DigiNav AI) that will connect to actual GSTN, MCA, and DigiLocker APIs and execute real filings. The scanner is the first step – a free tool that demonstrates the UX I want the full product to have.

If you're a founder and want a real, manual compliance health scan for your business, I offer those for ₹5,000 – delivered in 2 hours. DM me here or on Twitter/X / LinkedIn.

If you're building in the tax‑tech or AI agent space, I'd love to connect and compare notes. Drop a comment or reach out directly.

My links:

Portfolio: https://neha-portfoliooo.vercel.app/

GitHub: https://github.com/nehaprasad-dev

Live Scanner: https://compliscore-nu.vercel.app/

I Built a Compliance Health Scanner for Indian Startups in 24 Hours - Here’s What I Learned

Neha Prasad — Tue, 26 May 2026 12:55:41 +0000

Last weekend, I sat down and built CompliScore – a free 60-second compliance health scanner for Indian startups. You type your company name, and it gives you:

A Compliance Score (out of 100)

A list of pending GST & MCA filings

A penalty risk level (Low / Medium / High)

An AI‑generated action plan written in simple language

🔗 Live Demo: compliscore-nu.vercel.app

Why I Built This

I've also contributed 150+ PRs to core AI agent frameworks (Mastra, LlamaIndexTS, LiteLLM, PostHog) – so I understand both the compliance domain and the AI orchestration under the hood.

What I Learned While Building It

Founders don't care about compliance data. They care about consequences. Most tax-tech tools dump a table of due dates. That's useless. What a founder actually wants to know is:

Am I about to get a penalty?

How much will it cost?

What's the ONE thing I should do right now?

2. Speed builds trust.

3. Personal projects are the strongest resume

Tech Stack (for the curious)
Frontend: Next.js 15 (App Router), Tailwind CSS, shadcn/ui, Framer Motion

AI Layer: Groq API (Llama 3.3 70B) – generates the plain‑English action plan

Data Engine: Static mock data with a scoring algorithm that calculates compliance health based on overdue filings, missing returns, and pending notices

Deployment: Vercel (free tier, instant deploys)

What's Next

If you're a founder and want a real, manual compliance health scan for your business, I offer those for ₹5,000 – delivered in 2 hours. DM me here or on Twitter/X / LinkedIn.

If you're building in the tax‑tech or AI agent space, I'd love to connect and compare notes. Drop a comment or reach out directly.

My links:

Portfolio: https://neha-portfoliooo.vercel.app/

GitHub: https://github.com/nehaprasad-dev

Live Scanner: https://compliscore-nu.vercel.app/

How I Added Multi-Turn Image Generation Support to LlamaIndex

Neha Prasad — Thu, 14 May 2026 19:06:59 +0000

The agent could generate an image once, but when you asked it to modify or create variations - it had no idea what image you were talking about. The conversation had no memory of the previous image.

That broke a lot of interesting multi-turn creative workflows.

Context

While contributing to LlamaIndexTS (the TypeScript version of LlamaIndex), I noticed that image generation tools only worked for single-turn interactions. There was no clean way to reference a previously generated image in follow-up messages. This was especially painful when building agents that iterate on visuals - like creating logos, editing images, or generating multiple versions.

The Investigation

I started by reproducing the issue. The tool was calling OpenAI’s image generation API correctly the first time, but the response didn’t preserve any identifier for the generated image. Later messages had no context about which image to modify.

After digging through the tool calling flow, response parsing logic, and how messages were being stored, I found that the image_id returned by OpenAI wasn’t being extracted or passed forward in the conversation history.

The Solution

I added support for image_id across the board:

Added an image_id parameter to image generation tools

Enhanced response parsing to properly extract and store the image_id from OpenAI responses

Updated message options and tool configurations so subsequent requests can reference previous images

Created a working example showing the full multi-turn image generation workflow

The PR got merged smoothly: feat: multi-turn image generation support #2106

Lessons Learned

Streaming + tool calling can get tricky when the API returns important metadata (like image IDs) that isn’t part of the main content. Always check what fields the model actually returns, not just what you expect. Small details in response parsing can unlock much bigger capabilities.

What about me?
I love diving deep into agent frameworks and fixing core interaction loops. If you're building AI agents (especially ones involving images, tools, or complex workflows) and need help shipping fast - feel free to reach out.

DEV Community: Neha Prasad

Traccia vs. Maxim: Observe Agent Quality or Enforce Agent Bounds?

Why this comparison matters now

At a glance

Visibility: eval-linked traces vs agent telemetry

Maxim

Traccia

Intelligence: cost as a production signal

Control: two enforcement philosophies

Maxim — evaluate and alert on live traffic

Traccia — policies + @govern at the agent boundary

Certification: vendor trust vs trace depth

Where Maxim leads

Where Traccia leads

The bottom line

Choose Maxim if…

Choose Traccia if…

Choose both if…

Discussion

LangSmith vs Traccia: Observe vs Enforce in Production AI Agents

LangSmith vs Traccia: Observe vs Enforce in Production AI Agents

TL;DR

Introduction

Visibility: LangChain-Native Traces vs Agent Telemetry

LangSmith's strength

Traccia's strength

Your AI agent just blew $500. Now what?

That’s why Traccia exists.

I’ve shipped 150+ PRs and built AI agents in a day - but I still can’t get a job

I’m Building a Visual AI Agent Builder - Because I’m Tired of Debugging Invisible State Machines

CompliScore a free compliance health check for Indian startup founders.

# DEV Submission Build With Hermes Agent

Submission Template

What I built

Why an agent loop fits here

Hermes Agent integration

Tech stack

Try it

How Hermes Agent is at the heart of this project

Production setup

What I learned

Tags

Fixing a Frustrating Bug in LiteLLM Guardrails

I Built a Compliance Health Scanner for Indian Startups in 24 Hours - Here’s What I Learned

Why I Built This

What I Learned While Building It

2. Speed builds trust.

3. Personal projects are the strongest resume

What's Next

I Built a Compliance Health Scanner for Indian Startups in 24 Hours - Here’s What I Learned

Why I Built This

What I Learned While Building It

2. Speed builds trust.

3. Personal projects are the strongest resume

What's Next

How I Added Multi-Turn Image Generation Support to LlamaIndex

Traccia — policies + `@govern` at the agent boundary