DEV Community

Cover image for How I Built an AI Security Auditor That Streams AST Fixes (151 Users in 3 Days)
Kien Tran
Kien Tran

Posted on

How I Built an AI Security Auditor That Streams AST Fixes (151 Users in 3 Days)

Most Static Application Security Testing (SAST) tools give you a massive wall of text. They flag OWASP vulnerabilities, but leave you searching line-by-line to manually apply fixes.

To solve this, I spent the last few weeks building BugZ—an autonomous security auditor designed to analyze code and generate immediate, actionable fixes.


🛠️ The Core Concept & How It Works

Instead of dumping static logs, BugZ parses AST nodes and streams real-time reasoning directly to the UI. Once a flaw is detected, it auto-generates a downloadable Git .patch file so you can fix your code instantly.

# Apply a fix generated by BugZ in one command
git apply security-fix.patch
Enter fullscreen mode Exit fullscreen mode

⚡ The Tech Stack

  • Frontend: Next.js 14 (App Router) + Tailwind CSS
  • Database & Real-Time State: Convex
  • AI Engine: Gemini Pro

🚀 Early Traction (First 3 Days)

Building in public and focusing on a genuine developer pain point brought in awesome early validation:

  • 📈 151 unique visitors/users
  • 3 GitHub stars
  • 💰 $0 spent on ads

💡 What I Learned Building Solo

  1. Streaming UI is Essential: Handling real-time AST parsing streams directly to the frontend required fine-tuning Server-Sent Events (SSE) alongside React Suspense boundaries to keep latency low.
  2. Convex for Live State Sync: Instead of polling a traditional REST DB, Convex made syncing live AST execution states between the parser engine and the client seamless.

🔗 Try It Out & Give Feedback

Run a free audit on your repositories:
👉 bugz-ai.vercel.app

I'd love your feedback! Test it on your repos, roast the UI/UX, or let me know what features you'd like to see next in the comments below! 👇

Top comments (4)

Collapse
 
peterbuildssecure profile image
Peter

The generated patch is where I'd put the strongest trust boundary. Finding a risky AST pattern and producing a safe semantic change are two different claims.

I'd make every patch carry the rule ID, exact source span, evidence for the finding, expected security invariant, and tests that demonstrate the vulnerable case fails after the change. Apply it in a disposable checkout, run the project's existing tests plus a targeted negative test, then show the resulting diff rather than treating git apply succeeding as validation.

There is also a repository-data question worth making explicit: which files leave the user's environment, whether Git history is included, how secrets are redacted, and how long source and model traces are retained. A security scanner often receives the most sensitive version of a repository, including configuration and code paths that are intentionally absent from public builds.

For evaluation, I'd track fix acceptance separately from fix correctness: applied unchanged, edited before merge, rejected, caused a regression, and actually removed the exploit path. Visitor count validates interest; those outcomes validate the security claim.

Collapse
 
kientndev profile image
Kien Tran

Thanks a ton for this feedback, Peter! This is incredibly insightful and hits right at the core of making AI security tooling actually trustworthy in production.

A few quick thoughts on your points:

Patch Validation & Test Verification: I love the idea of running generated patches in a disposable checkout against existing test suites before surfacing the diff. Adding a targeted regression test alongside the patch is a fantastic trust anchor—definitely adding this to the roadmap to validate patch correctness.

Data Boundaries & Privacy: Right now, AST parsing and analysis are processed in-memory during the session without permanently retaining raw source code or secrets. Making these data boundaries and retention policies explicitly clear on the UI/docs is a super important callout.

Validation Metrics: Completely agree—visitor counts only prove initial curiosity. Long-term, tracking "fix acceptance rate" and "regression-free merges" is the real metric for BugZ's security value.

Really appreciate you taking the time to write such a detailed breakdown!

Collapse
 
peterbuildssecure profile image
Peter

That sounds like the right direction. One detail on the metrics: track attempted fixes as the denominator, including patches users reject or validation blocks. “Regression-free merges” can look artificially strong if failed patches disappear before merge.

I’d separate detection precision, patch acceptance, validation-pass rate, and post-merge regression rate. Also record whether the regression test failed before the patch and passed afterward; otherwise an agent can add a test that never exercised the vulnerability.

Collapse
 
kientndev profile image
Kien Tran

Bug Z Repo link! github.com/kientndev/BugZ
Bug Z Project link! bugz-ai.vercel.app (no domain yet sorry :)