DEV Community: VibeDoctor.io

I scanned the most famous AI coding repos on GitHub. Here's what I found

VibeDoctor.io — Sat, 11 Apr 2026 22:15:35 +0000

I built VibeDoctor, a scanner for AI-generated code. It looks for the stuff generic scanners miss - hallucinated imports, XSS patterns specific to AI output, N+1 queries in agent loops, memory leaks from copy-pasted React hooks.

Once I had it working, I had to know: how does the code from the tools everyone is using actually hold up? So I pointed it at three of the biggest AI coding repos on GitHub.

The leaderboard

open-lovable (Firecrawl): 50/100, 190 issues, 24 critical
devika: 66/100, 81 issues, 16 critical
bolt.new (StackBlitz): 64/100, 77 issues, 0 critical security findings

Not a single one cleared 70. The tools millions of people are using to ship production apps are themselves scoring "poor" to "fair" on the patterns they generate.

Let me show you the specific things that stood out.

open-lovable: 22 XSS vectors in one repo

open-lovable is Firecrawl's open-source Lovable clone. It's a great project and I like the team. But the scanner found 22 separate XSS vectors, mostly dangerouslySetInnerHTML and direct innerHTML assignments across ASCII animation components, tooltips, and the main generation page.

It also found 4 unbounded while loops with .push() inside the apply-ai-code route. No break, no limit, no max iterations. If the AI response ever loops, the Node process runs out of memory.

And one file, app/generation/page.tsx, is 3,958 lines long. A single function, AISandboxPage, is 3,888 of those lines. This is what AI-assisted development produces when nobody stops to refactor.

Report: https://app.vibedoctor.io/shared/fb2ce6f3-c32f-4f6a-9af4-e43fb0ac698b

devika: the agent loop is doing N+1 queries

devika was a viral AI developer agent that hit 18k stars in its first week. The scanner found 5 separate N+1 query patterns inside src/agents/agent.py and src/agents/runner/runner.py. Every iteration of the reasoning loop fires off individual database queries instead of batching. On a long agent run this is brutal.

It also has a hardcoded api_key="no..." in src/llm/lm_studio_client.py:12. Placeholder or not, that pattern gets copied into forks and nobody notices when someone swaps in a real key.

Zero test files for 86 source files. The requirements.txt has 33 unpinned dependencies. No lock file.

Report: https://app.vibedoctor.io/shared/62e9432d-0786-4a00-9ff4-243571b8bb22

bolt.new: clean code, dirty dependencies

Here's the plot twist. bolt.new scored best of the three on the code itself. Zero critical security findings in their own code. Good structure, reasonable function lengths, proper Remix patterns.

But their package.json is pinning react-router@6.24.1, which has four separate CVEs against it right now including path traversal and SSR XSS. Plus svelte@4.2.18 (three CVEs), nanoid@3.3.6, jsondiffpatch@0.6.0. None of these are bolt.new's fault, they're supply chain. But they ship with every user's generated project until someone updates.

This is the hardest problem in vibe coding and nobody is solving it: the AI tool generates clean code, pins an old dependency it was trained on, and every app built with it inherits the CVE.

Report: https://app.vibedoctor.io/shared/77e28919-a345-4743-b982-cf943295fed2

The meta-point

AI coding tools are trained on code, then they generate code in the patterns of that training data. When the reference implementations have XSS, N+1s, god files, and stale dependencies, every app built with them starts from that baseline. Generic scanners don't know to look for the AI-specific patterns. Human reviewers don't read 3,888-line functions.

Something has to catch this before it ships. Right now almost nothing does.

Firecrawl, StackBlitz, and the devika team are shipping at a pace most of us can't match. None of this is a dunk. It's a pattern worth naming because the apps built with these tools inherit it.

Try it on your own code

VibeDoctor is free for your first scan. Connect your GitHub, pick a repo, get a report like the ones above. Signup required, scan is free: https://vibedoctor.io

If you're from Firecrawl, StackBlitz, or the devika team and want to chat about any specific finding, 911@vibedoctor.io. Happy to be wrong on specifics and I'll update the post.

I scanned my vibe-coded SaaS with every tool I could find. Here's what they all missed.

VibeDoctor.io — Tue, 31 Mar 2026 11:08:13 +0000

I build fast. Like most founders using Bolt, Lovable, and Cursor -
I ship first and think later.

Last month I pushed 40+ commits to my SaaS.
I had no idea what was actually breaking with each one.

Not until I built something to tell me.

The problem with vibe coding at speed

When you're prompting an AI to build your app, you're not reading
every line it writes. Nobody is. That's the point.

But here's what happens in practice:

Commit 1: AI adds auth. Looks fine.
Commit 7: AI refactors a helper. Accidentally exposes an API route.
Commit 23: AI installs a package. It has 3 known CVEs.
Commit 31: AI adds logging. Now you're logging user emails to console.

You don't see any of this. Your users might.

What I tried first

I ran my app through the usual suspects:

Lighthouse - Told me my performance score. Useful, but it's
a snapshot. Doesn't tell me what commit caused the regression.

Snyk - Great for dependency CVEs. Misses everything else.

GitHub Dependabot - Only catches known CVE packages. Silent
on everything AI introduces structurally.

Manual PR review - I'm a solo founder. Who am I reviewing with?

None of these answered the one question I actually cared about:

What did my last push break?

What I actually needed

After every commit, I want to know:

Did this push introduce a new security issue?
Did my score go up or down vs the last scan?
What are the top 3 things I should fix right now?

Not a 200-line SAST report. Not a generic "you have 47 warnings."
Just - what changed, what broke, what do I fix first.

What VibeDoctor showed me

I built VibeDoctor (vibedoctor.io) to answer exactly this.

Here's what a real scan of my own app surfaced:

Security: 3 Anthropic API keys and a Stripe token committed
to test files. Low severity because they're test files - but
still embarrassing.

Performance: LCP 4.3s. TTI 8.3s. My own landing page
was loading like it's 2009.

Code Health: 1,575 total issues. 71 Critical. 2 Blockers.

Vibe Coding Health Score: 7/100. CRITICAL.

That last one stings because it's my own product.

But the part that changed how I work?

The push scan.

Every time I push a commit, VibeDoctor runs automatically and
shows me a before/after:

Score before push: 71 | Score after push: 64
New issues introduced: 4 | Fixed: 1
Still open: 847

That's the thing no other tool was giving me. Not just
"here's your health score" - but "here's what THIS push did."

Why this matters for vibe coders specifically

When you're using Bolt or Lovable, you're not the one writing
the code. You're directing it.

That means bugs don't look like bugs. They look like features.

AI doesn't hallucinate in obvious ways. It halluccinates in
subtle ones - importing packages that don't exist, leaving
console.logs with sensitive data, building SQL queries from
raw user input.

These aren't the things Lighthouse catches. These aren't even
things a senior dev reviewer always catches on a fast PR.

You need something scanning specifically for what AI coding
tools tend to get wrong.

The 5 things AI coders miss most often

After scanning dozens of vibe-coded repos, here's what comes
up constantly:

Hardcoded secrets - API keys, tokens, passwords in source code
Hallucinated imports - packages the AI invented that don't exist
Exposed API routes - endpoints with no auth that AI forgot to protect
N+1 queries - database calls inside loops that will destroy you at scale
Dependency CVEs - AI picks popular packages, not necessarily safe ones

If you're building on Bolt, Lovable, Cursor, or v0 - run a scan
before you show it to anyone.

Try it

VibeDoctor.io is free to scan. Sign up at vibedoctor.io, connect
your GitHub repo, and get your score in under 5 minutes.

If your vibe coding health score is above 60, I'll be genuinely
impressed.

Most aren't.