If you're using Copilot, Cursor, or ChatGPT to ship code faster, you already know the upside. They save time, tackle boilerplate, and help you thin...
For further actions, you may consider blocking this person and/or reporting abuse
One thing I've noticed is that AI rarely creates completely new security bugs it tends to reproduce the same insecure patterns consistently. That makes deterministic guardrails (AST analysis, policy checks, secret scanning, auth validation) much more valuable than relying on another LLM to "review" the code afterward.
We've seen similar patterns while building AI-powered applications at IT Path Solutions, especially around generated auth flows and database access. The most reliable approach has been treating AI output as untrusted code that must pass the same CI/CD security gates as human-written code. AI speeds up implementation, but secure defaults still have to come from engineering discipline, not the model.
That framing — treat AI output as untrusted code — is exactly it. I keep coming back to that when people ask why not just use another LLM to review it. A rule that says "this string starts with sk- and is 40 characters, reject it" is a much harder guarantee than asking a model to judge whether something looks like a secret.
The auth masking pattern is the one that worries me most honestly. The code looks correct at a glance — there's a try/catch, it's handling errors, what's the problem. The problem is the catch is calling next() which means a thrown auth error silently becomes a passed auth check. That specific mistake is almost always AI. A human writing auth from scratch just doesn't make it.
auth-masking is the rule I'd most want to get right and also the hardest to keep quiet. A catch block that calls next() is a legit pattern in plenty of Express middleware that isn't auth, so how does the rule decide a given handler is an auth boundary? Is it name-based (matching auth/verify/token in the function), or does the AST trace where the token actually gets checked?
Right now it's name-based — matches verifyToken, jwt.verify, authenticate etc. inside the try block. You're right that's the weak point, a catch calling next() is valid in plenty of non-auth middleware and the rule can't tell the difference yet. Cross-file tracing would fix it properly but that needs a full project graph — v1.x territory. For now it errs toward flagging and leans on hallint-disable for legitimate cases.
The tool may be useful, but the “AI-specific” framing feels more like marketing than a technical distinction. Most of these are ordinary security mistakes that existing static-analysis tools already target. I have geniune problem with this: ESLint is portrayed unfairly. Its plugin ecosystem and other static-analysis tools already detect many of these patterns.
The interesting question is not whether hallint can catch the carefully chosen examples shown here, but whether it catches more real vulnerabilities with fewer false positives than the existing alternatives. A benchmark would say much more than the list of rules.
And whats with the : Using an LLM to review unreliable LLM-generated code, that only adds another uncertain layer. Also some rules look extremely noisy. An async function does not always need its own catch, http:// is sometimes valid, and public routes intentionally lack authentication.
This will help you : dev.to/jennapederson/you-can-build...
Fair, the ESLint comparison was lazy. Its plugin ecosystem covers some of this and I undersold that.
The honest pitch is hallint is useful if you're not already running a well-configured security plugin stack — which in practice is most teams. If you are, the delta is smaller.
On the LLM layer — fully agree, deterministic rules are a harder guarantee. The LLM is opt-in and only ever explains a finding, never gates on one.
The benchmark point is the one I can't argue with. False positive rate on real codebases is unknown and that gap matters.
To be honest, if a person is building an app and is not having a basic ESLint or some sort of guidelines, chances are he is never going to use another tool.
Having managed teams with young developers, everyone tends to build their own thing. Some just ask the LLM models to scan for api keys and what not (there are agent skills available that does a more systematic checks and have an inbuilt checklist that is more thorough).
"The honest pitch is hallint is useful if you're not already running a well-configured security plugin stack — which in practice is most teams" : I would like some stats or source on this statement, as this seems like a more of an opinion than a fact!!
The useful distinction here is that AI-generated bugs cluster around patterns a style linter was never designed to care about. Catching swallowed errors, hardcoded secrets, or over-trusting input before code review is exactly the kind of gate that belongs in CI next to tests, not in a security retrospective.
The hard part is usually rule credibility over time. False positives decide whether a tool becomes part of the default pipeline or gets bypassed after the first noisy week.
If you can preserve a compact "why this rule fired" receipt per finding, human review also gets much faster. Curious how you are thinking about that tradeoff.
Yeah, false positives are what kill tools like this. The first noisy week is basically a make-or-break moment — if engineers start seeing it as "that thing that cries wolf," it gets added to the ignore list and never comes back.
The public route allowlist and inline suppression are my current answer to that, but honestly the real test is running it against enough real codebases to see which rules earn their keep and which ones are too aggressive. That's why I'm building rule fire rate tracking — suppression rate is probably the most honest signal for "this rule is annoying people."
On the receipt idea — that's basically what the LLM explanation layer is trying to be. Each finding can carry a short plain-English note about why that specific snippet triggered it. Less "rule X fired" and more "this API key pattern matches known provider formats." Still early but that's the direction.
The framing that AI-written bugs are a distinct failure class matches what I've seen: they're not random, they're the modal completion pattern for a prompt (e.g. always the naive SQL concat, always the same missing auth check). What I'd want to know from hallint is whether the rule set was grounded on a labeled corpus of LLM-generated CVEs or hand-curated from experience, because the former is what makes the detector generalize past the frameworks you personally tested. Also how are you thinking about false positives on hand-written code that happens to look modal?
Hand-curated from patterns I kept seeing across Express and common Node.js codebases. No labeled corpus yet, that's a real gap.
The modal completion framing is what makes early hand-curation defensible though — the patterns are consistent enough to enumerate without a huge dataset. But it does mean the rules are biased toward frameworks I personally tested.
On false positives from hand-written code that looks modal — no good answer yet. Suppression and the public route allowlist are escape hatches more than solutions. Rule fire rate tracking is on the roadmap partly for this reason, suppression rate is probably the most honest signal for whether a rule is earning its place.
If you have pointers to a labeled LLM CVE corpus, genuinely interested.
I don't see anything new here which the regular linters does not achieve already
If you're already running eslint-plugin-security well-configured, probably not much new here. The target is teams that aren't.