GHAS is the one people end up on by default rather than by choice. It is right there in the repo, you flip it on, CodeQL starts scanning, job done. That convenience is the whole pitch, and it is also the trap.
Where GHAS is good
Let me be fair, because GHAS is not a weak tool. CodeQL is properly good. It does real semantic analysis, actual dataflow and taint tracking, not just pattern matching. Of all the scanners in these comparisons it is the one with the most serious engine under it. If you are all in on GitHub and you want something native that does more than grep for patterns, it is a reasonable thing to have turned on.
Where it falls down in practice
CodeQL is only as good as the queries written for it, and writing good custom queries is genuinely hard, so most teams just run the default pack and never touch it again.
So you get strong analysis pointed at a generic question, which means it is great at the vuln classes GitHub wrote queries for and quiet on everything else, especially anything specific to your own business logic. It also basically assumes you live entirely inside GitHub, and the moment you are across GitLab or Azure or a mix, the "it is already there" advantage evaporates.
The bigger thing
This is the same point that runs through all of these. GHAS finds and hands you a list. You still own the triage, you still write the fix, you still open the PR. The convenience is in the scanning being there, not in the work being done.
GHAS: scan -> here is your list -> the rest is your afternoon
Kolega: scan -> generate fix -> test in sandbox -> open PR -> you review and merge
We scan, generate the fix, test it in a sandbox, and open the PR for you to review. Different job.
The receipts
RealVuln is our open benchmark: 676 real vulnerabilities across 26 production repositories, plus 120 false positive traps to catch tools that flag everything to inflate recall.
RealVuln
- 676 real vulnerabilities
- 26 production repos
- 120 false positive traps
- fully open source
We benchmarked against the serious engines, including the frontier models, not just the easy targets, and you can run your own setup against it and check. The point of making it open is that nobody has to believe the marketing.
So which one
This is not "GHAS bad." It is the strongest default on this list. It is just that having it switched on because it came free with the repo is not the same as actually being covered, and "we have GHAS enabled" tends to be where security thinking stops rather than starts. Worth knowing the difference before you tell a customer you are secure.
Full breakdown and the benchmark: https://kolega.dev/compare/github-advanced-security/
Top comments (0)