DEV Community

Yuvraj Angad Singh
Yuvraj Angad Singh

Posted on

Your AI Wrote the Code. Who's Checking It?

#python #javascript #ai #opensource

I review a lot of PRs at work. Over the last year, I started noticing patterns in AI-generated code that kept showing up. Same five or six things, every time.

Empty catch blocks. as any sprinkled everywhere. Comments that just restate what the code does. Hardcoded API keys. except: pass in Python. The code works, it passes tests, but it's the kind of stuff you'd flag in a review and ask someone to fix.

The numbers back this up. CodeRabbit found AI-generated PRs have 1.7x more issues than human PRs. Veracode says 45% of AI code samples contain security vulnerabilities.

ESLint catches syntax issues. But nobody's catching the behavioral patterns that AI tools leave behind. So I built one.

vibecheck

24 rules across JS/TS and Python. Zero config. Runs offline. Regex-based, so it's fast.

npx @yuvrajangadsingh/vibecheck .
Enter fullscreen mode Exit fullscreen mode 
  src/api/routes.ts
    12:5    error  no-hardcoded-secrets   Hardcoded secret detected
    45:3    error  no-empty-catch         Empty catch block swallows errors
    89:1    warn   no-console-pollution   console.log left in production code

  src/utils/db.ts
    34:5    error  no-sql-concat          SQL query built with string concatenation

  4 problems (3 errors, 1 warning)
  2 files with issues out of 47 scanned (0.8s)
Enter fullscreen mode Exit fullscreen mode

No API keys, no cloud, no LLM calls. It's regex rules that match the patterns AI tools tend to produce.

What it actually catches

Here's the stuff I kept flagging in code reviews:

The silent failure:

try:
    response = api.fetch(user_id)
except:
    pass
Enter fullscreen mode Exit fullscreen mode

Your API call fails and nobody ever knows. vibecheck flags no-bare-except and no-pass-except.

The "I'll type it later":

const data = response.json() as any;
Enter fullscreen mode Exit fullscreen mode

AI tools love as any. It shuts up the type checker but defeats the entire point of TypeScript. vibecheck flags no-ts-any.

The useless comment:

// initialize the counter
let counter = 0;
Enter fullscreen mode Exit fullscreen mode

Your linter doesn't care about this. vibecheck does. no-obvious-comments catches comments that just repeat what the code already says.

The hardcoded key:

const API_KEY = "sk-proj-abc123def456";
Enter fullscreen mode Exit fullscreen mode

This one's obvious but it keeps happening. no-hardcoded-secrets matches common API key patterns.

Diff mode

This is the part I use most. Instead of scanning your entire codebase, scan only the lines you just changed:

vibecheck --staged .
Enter fullscreen mode Exit fullscreen mode

Drop it in a pre-commit hook and it only checks what you're about to commit:

# .git/hooks/pre-commit
npx @yuvrajangadsingh/vibecheck --staged .
Enter fullscreen mode Exit fullscreen mode

In CI, it runs on PR diffs so you're not drowning in warnings from old code.

"Isn't this just ESLint?"

No. ESLint catches syntax and style. vibecheck catches the patterns that come from how AI tools generate code. Your linter won't flag a catch block that only does console.error(err) without rethrowing. It won't flag # type: ignore without a specific error code. It won't flag a function that's 120 lines long because the AI didn't know when to stop.

They're complementary. Run both.

Install 

# no install needed
npx @yuvrajangadsingh/vibecheck .

# or install globally
npm install -g @yuvrajangadsingh/vibecheck

# standalone binary (no Node required)
curl -fsSL https://github.com/yuvrajangadsingh/vibecheck/releases/latest/download/vibecheck-darwin-arm64 -o vibecheck
chmod +x vibecheck
Enter fullscreen mode Exit fullscreen mode

GitHub: github.com/yuvrajangadsingh/vibecheck

Open to feedback on what rules to add next. If you keep flagging the same thing in AI-generated PRs, I probably want to hear about it.

Top comments (0)