AI coding assistants make it incredibly easy to generate code and open a pull request.
The problem is that security review doesn't automatically become faster just because code generation does.
So I wanted to test something simple:
What happens if AI-generated code is treated like any other pull request and automatically security-scanned before it is merged?
I built a small GitHub Action called AI Code Guard to test that.
The goal wasn't to create another generic AI code reviewer.
The goal was to catch concrete security problems in the changed code and make every finding traceable to actual evidence.
The test
I created two test cases:
- a vulnerable version containing intentionally unsafe code
- a safe version with the problems fixed
Then I opened the vulnerable code as a pull request and let the GitHub Action run automatically.
The scanner checks the pull-request diff for things such as:
- hardcoded credentials and secrets
- SQL and other injection patterns
- dangerous shell commands
- unsafe workflow permissions
- risky dependency patterns
- other deterministic security problems
There is also an optional AI review for issues that require more context, such as broken authorization or missing validation.
What happened
The vulnerable pull request produced 2 CRITICAL findings.
One of the findings identified a possible hardcoded credential and pointed to the relevant location in the changed code.
That distinction matters.
A security tool telling you:
“There might be a vulnerability somewhere.”
is much less useful than:
“Here is the exact code that caused this finding.”
The result is structured around the finding, its evidence, explanation, recommendation, and confidence.
The evidence requirement
This is the part I care about most.
For AI-generated findings, AI Code Guard requires the model to provide an exact snippet from the diff as evidence.
If the claimed evidence isn't actually present in the code that was sent to the model, the finding is discarded.
In other words:
No evidence, no report.
The reason is simple: security tools lose their value quickly if developers repeatedly receive findings they cannot verify.
A scanner that constantly cries wolf eventually gets ignored.
Then I fixed the vulnerable code
After correcting the issues, I ran the pull request again.
The result:
Risk: NONE
5 checks passed
No issues found in the changed code.
So the basic loop became:
AI-generated / changed code
↓
Pull Request
↓
AI Code Guard
↓
CRITICAL finding
↓
Exact evidence
↓
Fix
↓
Scan again
↓
Clean PR
What AI Code Guard actually does
The free deterministic layer requires no API key.
It checks the changed code for things such as:
- hardcoded secrets
- dangerous commands
- risky GitHub Actions permissions
- suspicious dependency patterns
- common injection patterns
The optional AI layer can look for more context-dependent problems.
Importantly, the AI analysis is scoped to the pull-request diff rather than sending the entire repository to the model.
Why I built it
AI-assisted development is moving extremely quickly.
The interesting security question isn't only:
“Can AI write code?”
It's also:
“How do we automatically check what it just wrote before we merge it?”
I wanted something small enough to install directly into a GitHub repository and useful enough to run on every pull request.
That's what this experiment became.
Try it
AI Code Guard is open source and available as a GitHub Action.
For public repositories, the deterministic checks are free and require no API key.
Repository:
https://github.com/sarzho33-design/AI-CODE-GUARD
If you use AI coding assistants and already have pull requests running through GitHub Actions, I'd be particularly interested in seeing what it catches on real projects.
Top comments (0)