Most AI coding tools are very good at helping developers write code.
But I wanted to explore a different question:
Can AI help developers find t...
For further actions, you may consider blocking this person and/or reporting abuse
The false-negative direction is the part worth keeping. A linter that says VULNERABLE when it is not sure just costs you a look; one that says SAFE on a guess quietly ships the hole. Your "preserve taint unless proven sanitized" rule is default-deny applied to data flow, the same instinct that should govern agent permissions. Unknown function, assume the worst until you have a sanitization path you can point to.
Exactly this. You absolutely nailed it with "default-deny applied to data flow."
It's fascinating because treating taint state like a strict permission model goes against how most LLMs naturally behave. They tend to fill in missing context with optimistic assumptions.
A traditional linter throwing a false positive might waste 5 minutes of your time. An AI producing a false negative because it guessed that an unknown helper function was a sanitizer is much more dangerous — the vulnerability can quietly make its way toward production.
Assume the worst until there is hard evidence of sanitization.
Glad that philosophy resonated with you, and I really appreciate the insightful comment!
The optimistic assumption about unknown helpers is the exact failure mode I see when agents review their own diffs. They read a function name that looks like a sanitizer, invent a happy path, and mark the sink safe without ever opening the body. Forcing taint to survive unknown calls until there is evidence of sanitization is the right default. It matches how you would review a PR from someone who keeps saying the helper probably cleans it. The local risk screen before any model call is also the part most people skip, then wonder why the bill exploded.
Yes — the repeated scan cost is actually one of the next problems I'm looking at with EdgeGuard.
The idea is to establish a full security baseline on the initial scan and store the scan history in memory. After that, subsequent scans don't need to treat the whole project as new.
I'm looking at using Git history as the boundary:
Initial scan → full analysis → store scan history
Then:
Git diff / merge history → identify what changed → re-analyze the affected files and security flows → only send relevant findings to the LLM → update the memory.
The important part is that I don't want to blindly scan only changed files, because a change can affect a security flow across multiple files. The deterministic analysis still needs to determine the affected dependency/data-flow surface.
That way the LLM isn't repeatedly rediscovering the same findings on every scan.
So the expensive part is establishing the initial baseline. After that, the goal is to pay mainly for the delta and the security flows affected by that delta.
That's also where Lean AI Memory becomes useful for EdgeGuard: not just remembering what the LLM said, but remembering what has already been analyzed, when it was analyzed, and what actually needs to be reconsidered after a change. trả lời thế này ổn ko bạn
This is the honest version of the story most I built an AI reviewer posts skip. Worth writing up which OWASP categories it missed specifically, that's more useful to the next person than the fact that it broke.
Appreciate the feedback! You're completely right.
Saying “AI failed” is easy, but dissecting exactly where its blind spots are is where the real value is.
Breaking down the specific OWASP/CWE categories it missed — and why it lacked the context to catch them — would make the failure much more useful to others building AI security reviewers.
For example, a missed SQL injection because the taint flow was broken is a very different problem from a correctly detected flow that the LLM incorrectly marked as safe.
I think those distinctions are worth documenting rather than treating everything as simply “the AI missed it.”
That's a great idea for a follow-up post. I'll pull the logs and put something together.
There is a nice operator angle here: the best implementation is often the one that makes a bad state obvious early. A clear signal, an owner, and a reversible response path beat a more sophisticated design that fails silently.
Exactly. The “operator angle” is the right lens.
We often focus so much on making AI agents smart enough to handle complex edge cases that we forget to make them operable.
A silent failure in an agentic workflow isn't just a bug — it can become architectural drift.
I'd much rather have an agent hit a hard boundary, like EdgeGuard, clearly flag the uncertainty, and stop than confidently and silently refactor a system into a corner.
That's also why I like having a reversible path such as a clean Git diff. It's a much stronger safety mechanism than relying on a sophisticated prompt that occasionally ignores constraints.
“Fail loudly” has always been a core engineering principle. I don't think we should abandon it just because the system writing the code happens to be an LLM.
Agreed. Clear stop conditions and reversible changes make an agentic system easier to operate under uncertainty. A sophisticated prompt can improve judgment, but it is not a substitute for boundaries the system can actually enforce and an audit trail a human can inspect.
Imo, pattern matching, like OWASP, it's better to use rules, than have a LLM interpret it. The signature of the task is what it should look for, flag the signatures, then have a LLM review it, if bad, flag red, if determined safe, flag yellow, both red and yellow need human oversight to make sure
I completely agree. Relying on a probabilistic model to interpret and enforce a strict security boundary is a recipe for silent failures.
The key distinction for me is that we shouldn't use the LLM to enforce the boundary. The deterministic layer should establish the boundary and constrain what the LLM is allowed to conclude.
So the workflow becomes:
The important line to draw, is who has final authority. Trust a human, over a LLM, every single time. Because the SecOps expert will tell you why it's there and what needs to happen, with the context of an expert, whereas the LLM can just view the codebase at a glance
100% agreed. I think the key distinction is authority vs. capability.
The LLM can generate code, investigate findings, and provide useful explanations. But it shouldn't become the authority that decides whether a security boundary has been crossed.
A SecOps expert brings something fundamentally different: historical context, operational knowledge, and accountability. An agent may see the current code and the available context, but it doesn't necessarily know the "scars" from the outage that caused a particular security rule to exist in the first place.
That's why I don't think the goal of strict boundaries is to make the AI smart enough to resolve every security issue autonomously.
It's to make the AI "disciplined enough to stop, raise its hand, explain what it found, and wait when human judgment is required."
Detect → investigate → explain → stop when uncertain → human decides.
The model is an investigator, not the authority.
The optimistic fill in is the part that actually matters. When the model cannot see into doSomething(), guessing that it sanitizes is how a review ends up green over a real sink. Forcing taint to stick through unknown helpers until there is evidence otherwise matches how agent patches fail in practice: the wrapper looks intentional, the summary sounds careful, and the dirty value still reaches the query. The local risk screen before the LLM call is the other half that keeps this honest at scale. Without it you pay to invent a story for every getter, and the few flows that matter get buried.
The “optimistic fill-in” is one of the reasons LLMs can be dangerously unreliable in security reviews. When the model can't see inside something like doSomething(), it's very easy for it to assume the wrapper is sanitizing the input simply because the code looks intentional.
That's why I prefer pessimistic taint propagation: if a value is tainted, it stays tainted through unknown helpers until there is actual evidence that it has been sanitized. The AI shouldn't be allowed to turn an unknown function into a “probably safe” function just to complete the story.
And yes, the local risk screen is the other half of it. There is no reason to send every getter, mapper, or harmless code path to an LLM and ask it to invent an explanation. Let the local analysis narrow the search space first, then let the LLM investigate the paths that actually matter.
That's really the direction I'm trying to take with EdgeGuard: don't expect the LLM to magically do security analysis. Build deterministic guardrails around it so the LLM has fewer opportunities to make unsafe assumptions.
Really appreciate the insight — especially the connection to how agent-generated patches actually fail in practice. 👍
A note about the benchmark
EdgeGuard is designed with multi-LLM support, so the investigation engine is not tied to a single model.
However, the results and numbers presented in this article come from my initial real-world testing with Google Gemini. I have not yet completed a systematic benchmark across the other supported LLM providers, so these results should not be interpreted as a comparison between models.
My next step is to test the same investigation pipeline across multiple LLMs and see how they differ in handling taint analysis, edge cases, and verification.
It’s really amazing, in-depth work around 🫡
The hallucination aspect (assumption vs. evidence) and the scaling aspect (deciding what should actually be sent to the LLM) were two things I found particularly interesting.
Thank you, Atul! 🫡 I'm really glad those two aspects resonated with you.
Actually, I'm currently thinking about the next architectural step to address these problems even better: combining Lean AI Memory with Git diff.
For Scaling: Instead of scanning the whole workspace every time, git diff can enable incremental analysis — focusing the pipeline on newly modified code and only bringing in the relevant context.
For Hallucinations: Lean AI Memory can act as a local evidence layer. For example, once the agent has verified that a project-specific helper function (such as a custom input sanitizer) is safe, that verified context can be remembered and reused in future investigations instead of making the LLM reason from scratch every time.
The goal is to evolve EdgeGuard from a one-off scanner into a more continuous, context-aware security agent.
I'd love to hear your thoughts on this approach!