DEV Community

Allen Brooks
Allen Brooks

Posted on • Originally published at lionshead.digital

The security checks in every Lionshead PR

At enterprise scale, a breach is a bad quarter. You have a legal team to coordinate disclosure. A disaster-recovery plan you drill annually. A PR team to control the narrative. A security team to quarantine, investigate, and triage the incident. A bank account big enough to absorb regulatory fines, class-action settlements, and the customer churn that follows.

At solo scale, you have none of those. A real breach of user data will almost certainly end the product. Not "hurts the brand," not "sets us back a quarter." Ends it.

So you breach yourself first.

The security stack in every Lionshead PR is an automated self-breach. Every merge attempt is scanned by tools a mid-sized security team would run against production code weekly. If any of them find something, the merge blocks. The vulnerability never reaches production, and the version of me that would have caused an incident never gets the chance.

Here's what actually runs on every PR against every Lionshead product.

Trivy filesystem — vulnerability, secret, and license scanning across the whole repo. Runs on every PR, no path gate. Catches dependency CVEs, accidental hardcoded credentials, license conflicts. Configured via trivy.yaml at the repo root; fails on any CRITICAL or HIGH severity. Trivy is Aqua Security's open-source scanner and it's the single best free vulnerability scanner I've used.

Trivy IaC (Terraform) — the same tool, second mode, against Terraform files. Catches insecure infrastructure defaults (public buckets, permissive IAM roles, unencrypted resources). Path-gated: only runs when terraform/** files change. Same severity gate.

Gitleaks — secret scanning across the entire git history, not just the current diff. Catches API keys, tokens, private keys, connection strings that someone (probably me) committed years ago and later deleted. The secret is still in the history and still exploitable. Runs on every PR. Uses the default ruleset unless the repo drops a .gitleaks.toml override.

Checkov — a second IaC security scanner, run alongside Trivy IaC for defense in depth. Checkov and Trivy IaC use different rulesets and different heuristics; they catch different classes of problem. Path-gated on terraform/**. Skip lists for known-false-positive rules go in .checkov.yaml.

OWASP ZAP baseline — active DAST against the PR's Vercel preview URL. When a repo produces a preview deploy, ZAP hits the deployed site with a baseline security scan. Catches missing security headers, cookie configuration issues, common injection vectors. Fails on any WARN+ alert; per-rule suppressions live in the repo's .zap/rules.tsv.

Actionlint — GitHub Actions workflow linting. Not strictly security, but security-adjacent: catches syntax errors, invalid contexts, and permission-scope typos that would silently over-scope your workflow's access. Path-gated on .github/workflows/**.

Shellcheck — shell script static analysis. Catches quoting errors, command-injection patterns, and the classic "unquoted variable in an rm command" bug that has ended more careers than any single vulnerability. Path-gated on *.sh files.

Hadolint — Dockerfile linting. Catches insecure image bases, missing USER directives, and other Dockerfile-shape issues that expand blast radius on container escape. Path-gated on Dockerfile*.

Action pins check — a Lionshead-authored guardrail that validates every third-party GitHub Action referenced in workflow files is pinned to a full commit SHA, not a moving tag. Prevents a supply-chain attack against an action's tag from silently propagating into Lionshead's CI. Fails PRs that add or update actions without a SHA pin.

Infracost + OPA cost gate — not security in the CVE sense, but security in the "my card gets charged $10K/month by mistake" sense. Runs on every Terraform PR, computes the cost delta, and pauses the PR on a GitHub Environment approval gate when the delta exceeds $50/month. I covered this in more detail in the last post; mentioned here because it lives in the same workflow.

Two more layers run outside the PR gate.

Scheduled security audit — every product repo runs lionshead-security-audit.yml daily against the default branch. It runs npm audit (or the language-appropriate equivalent) plus a full Trivy filesystem scan. Findings open or update a single security:dependency tracking issue per repo; clean runs close any open issue. The point is to catch CVEs disclosed after code was merged, not just what a PR introduces. The majority of real-world CVEs are disclosed against code already in production; a PR-time scan alone misses them.

Renovate — dependency-update automation. The ci-standards Renovate preset applies to every product repo via a one-line renovate.json. Renovate opens PRs to bump dependencies, pin action SHAs, and update base container images. Its github-actions manager keeps the pinned reusable workflow SHAs current, which is how new products cloned from template-product stay on the latest ci-standards workflow without manual intervention. Weekly schedule keeps the noise bounded.

Three things are on the roadmap but not shipping yet.

Semgrep (SAST) — the security baseline mandates Semgrep OSS for static-analysis coverage as a free-and-open-source substitute for GitHub Advanced Security. Rollout across all products is in progress.

OWASP ZAP as a scheduled cron against dev — today, ZAP runs at PR time against the preview URL. The plan is to also run ZAP against the shared dev environment weekly to catch drift between preview builds and the environment that lives longer.

Pre-commit hooks — a Husky-driven local hook stack that runs the most annoying failure modes (a Gitleaks pre-commit scan, at minimum) before code even gets to the PR. Removes a full round-trip when you catch a mistake before pushing. On the shortlist to ship next.

The whole stack costs zero in licensing. Every tool above is open source. The cost is time to wire them up and ongoing time to tune false-positive suppressions. That cost is measured in hours, once, not months. What it buys is the difference between "my product survives" and "my product ends" in the scenario where I fat-finger a Terraform PR at 11pm.

That is the trade at solo scale. It happens to look like the trade at enterprise scale for exactly the same reason.


Update, 16 July 2026: what it cost, and what it caught

Someone on Hacker News asked how often these checks actually hold up my PRs. I didn't know, which bothered me, so I went and counted every PR check this stack has ever run: 2,864 of them across nine repos. 286 failed, call it 10%.

The failures aren't spread out, and that's the part I didn't expect. They arrive in tight clusters of three or four days, and every cluster sits on top of a setup event. The biggest was late April, when I pulled these checks out of Vesper and into the shared ci-standards workflow. Eighteen Infracost failures over four days while I shook out credential and fetch-depth bugs, then ten weeks of silence. Another cluster in May when Crisper adopted the same workflow. Another in July when I bootstrapped some skeleton repos that are parked until Vesper ships. Between the clusters it's quiet for months. The repos with the worst failure rates are the ones that have only ever been in setup and have no steady state to average against.

So the cost isn't a tax on every PR. It's a bill that comes due when I migrate something, and it's mostly me misconfiguring my own plumbing rather than the scanners crying wolf.

The other surprise: tests, not security, are what actually block me. Failing tests account for roughly half of every PR check failure I've ever had. All the security tooling combined is well under a quarter.

What the security tools have caught, honestly: not much. Trivy found a real CVE in a transitive form-data dependency and I bumped it. Checkov found a real IAM policy attached to a user instead of a role. It also flagged AKIAEXAMPLEEXAMPLE00 in a .env.example, and read a Terraform format("postgresql://%s:%s@...") string as hardcoded credentials when the password was a random_password reference. Which is to say it caught the correct implementation and would have sailed straight past a bad one.

And then, while I was writing this very update, the pull request carrying it went red. Trivy flagged CVE-2026-54466, rated CRITICAL, in websocket-driver 0.7.4, reached through @firebase/database. It had nothing to do with anything I'd changed. It was disclosed against code I merged weeks ago, and the PR-time scan is simply where I happened to meet it. Bumped to 0.7.5, green again, about six minutes of work.

So: two real findings in four months three real findings in four months, one of them CRITICAL, and the third arrived while I was typing the sentence that said there were only two.

That's still a thin haul if you measure it the way a security team measures things when it needs headcount approved. But that was never the trade. The premium is a few hours per migration and zero dollars, and what it covers is the scenario where I fat-finger a Terraform PR at 11pm and end the product. At that price I don't need it to catch much. I need it to catch the one. It just caught one while I was mid-sentence arguing it rarely does.

A correction and a clarification to the post above, while I'm here.

The post says the tuning cost is "measured in hours, once, not months." Hours is right. Once is wrong. It's hours per migration, and every new repo keeps rediscovering the same false positives because I've written a slightly different allowlist in each one instead of fixing template-product so the next repo inherits it. That's on me, not the tooling. For the full effect: publishing this update tripped Gitleaks on the AKIAEXAMPLEEXAMPLE00 two paragraphs up, and I fixed it by writing yet another per-repo allowlist. Which tells you exactly how much I've learned.

On ZAP: what runs at PR time today is the baseline passive scan. Full active DAST is on the roadmap, not shipping yet.

What's next

Next in the CI/CD sub-series: the preview environment pattern that ZAP runs against. Vercel plus a fresh Neon Postgres branch per PR, how the reviewer gets a real running app talking to a real database on every PR, and why that costs almost nothing.

Top comments (0)