Most SaaS teams still pentest once a year. Almost none of them ship code once a year.
That mismatch is the actual security problem worth talking about, more than any single vulnerability class. Security gets validated at one cadence. Risk gets introduced at another. The distance between those two cadences has a name: the deployment velocity gap.
What the gap actually is
The deployment velocity gap is the time between a security-relevant change hitting production and that change actually being evaluated by a penetration test.
In an annual model, that gap can run for months. A system might be genuinely secure the day the assessment wraps, but every deploy after that day creates new, untested surface area: new endpoints, an updated auth flow, a new third-party integration, a reconfigured piece of infrastructure. None of it was in scope for the test that already happened. All of it is in scope for the breach that hasn't happened yet.
This isn't a knock on pentesting as a discipline. It's a scoping problem. A penetration test evaluates a moment in time, and it evaluates a defined scope, both of which shrink in relative value as deployment frequency rises.
You can put a number on your own exposure with something this simple:
def deployment_velocity_gap(test_interval_days, weekly_deployments):
"""
test_interval_days: 365 = annual, 90 = quarterly, 30 = monthly
weekly_deployments: how many times code ships per week
"""
average_gap_days = test_interval_days / 2
total_deployments = (test_interval_days / 7) * weekly_deployments
risk_level = (
"CRITICAL" if average_gap_days > 90 else
"HIGH" if average_gap_days > 45 else
"MEDIUM" if average_gap_days > 14 else
"LOW"
)
return {
"average_exposure_window_days": average_gap_days,
"max_exposure_window_days": test_interval_days,
"untested_deployments_per_cycle": total_deployments,
"risk_level": risk_level,
}
Run it against a team shipping three times a week on an annual test cadence and you get a 182.5-day average exposure window and 156 untested deployments per cycle: CRITICAL. Drop the interval to monthly and the same team lands at a 15-day average window and roughly 13 untested deployments: MEDIUM. The application didn't get safer. The window just got shorter.
Continuous pentesting isn't "scan on every commit"
Worth being precise here, because the term gets diluted fast: continuous penetration testing does not mean pointing an automated scanner at the app on every push and calling it a pentest. Scanners find known patterns at volume. A pentest investigates how a weakness can actually be chained and exploited in the context of a real, running application, with defined scope, authorization, controlled exploitation, and human oversight over the findings.
What continuous testing does mean in practice:
- Monthly testing — a recurring assessment at a materially shorter interval than annual.
- Sprint-cadence testing — assessments aligned to the development cycle itself.
- Targeted testing — auth changes, new APIs, payment flows, and infra changes trigger a focused assessment on top of the recurring one.
- Integrated coverage — pentesting sits alongside IDE, PR, and CI/CD controls rather than replacing them.
There's no single "correct" frequency. The right cadence is a function of deployment velocity, data sensitivity, regulatory obligation, and how fast the team can actually act on a finding once it exists.
Scoping a sprint by risk, not by calendar
The more useful version of "test every sprint" is test every sprint proportionally to what changed. A simple risk-scoring pass over merged PRs gets you most of the way there:
class SprintSecurityTestingProgram:
def classify_test_depth(self, risk_score: int) -> str:
if risk_score > 100:
return "FULL_DEPTH — auth changes require complete auth chain review"
elif risk_score > 50:
return "TARGETED_DEEP — multiple security-relevant changes"
elif risk_score > 20:
return "TARGETED_STANDARD — specific components need focused testing"
else:
return "LIGHTWEIGHT — automated testing sufficient"
def identify_priority_areas(self, changed_components: dict) -> list:
priorities = []
if changed_components["authentication"]:
priorities.append({
"area": "Authentication",
"priority": 1,
"test_focus": "JWT validation, session management, MFA bypass, brute force",
})
if changed_components["authorization"]:
priorities.append({
"area": "Authorization",
"priority": 2,
"test_focus": "RBAC, IDOR, cross-tenant access, role bypass",
})
if changed_components["data_access"]:
priorities.append({
"area": "Data Access Layer",
"priority": 3,
"test_focus": "SQL/NoSQL injection, ownership filter presence",
})
return sorted(priorities, key=lambda x: x["priority"])
Auth and authz changes get full-depth review every time. Everything else gets scoped by what it actually touches. That's the difference between a program that scales and one that turns into indiscriminate noise.
The economics people usually get wrong
The lazy comparison is "one annual invoice" versus "a subscription." That comparison undercounts the annual model and overcounts the continuous one, because it ignores retesting, remediation effort, false-positive waste, emergency response, and the actual expected cost of a breach sitting inside a long exposure window.
| Cost category | Annual model | Continuous model | Delta |
|---|---|---|---|
| Direct testing cost | $25K–$50K | $48K–$72K/yr (sub) | +$10K–$25K |
| Retest cost | $8K–$15K | Included | -$12K |
| Engineering remediation | $19.2K (8 findings × 3 days) | $14.4K (12 findings × 1.5 days) | -$4.8K |
| False positive waste | $9.6K (30% FP rate) | $1.6K (5% FP rate) | -$8K |
| Emergency response | $17.5K (35% probability) | $4K (8% probability) | -$13.5K |
| Expected breach cost | $24.7K (180-day window) | $1.6K (14-day window) | -$23K |
| Total TCO | ~$104K | ~$80K | -$24K |
Illustrative numbers for a $10M ARR company shipping weekly, 15 engineers at $100/hr, 12% annual breach probability, $500K average breach cost, but the shape of the comparison holds more broadly: the direct testing line goes up, almost everything downstream of it goes down, and the breach-cost line is usually the biggest single swing because it scales directly with how long the exposure window is.
The expected-breach-cost math is worth writing out, because it's the part most TCO comparisons skip:
exposure_window_days_continuous = 14 # sprint cadence
adjusted_breach_probability = breach_probability_annual * (
exposure_window_days_continuous / 365
)
expected_breach_cost = adjusted_breach_probability * avg_breach_cost
Shrink the exposure window, shrink the probability term, shrink the expected cost. It's linear and it's easy to underweight if you're only looking at the invoice.
Picking a cadence: a decision framework, not a rule
Four questions, roughly in order of how much weight they should carry:
How often do you deploy? Weekly or more and annual testing is covering under 10% of your deployments by construction. Monthly or less and annual/semi-annual is probably fine.
What data do you handle? PII at scale, payment data, or health data pushes you toward quarterly-minimum or continuous regardless of deploy cadence, because breach impact and regulatory exposure are both high.
What's the regulatory environment? PCI DSS, SOC 2, HIPAA, and ISO 27001 all have their own expectations for testing evidence and cadence. Continuous testing can supplement that evidence; it doesn't automatically substitute for a compliance-mandated assessment, and that distinction matters when an auditor is asking.
Can you actually act on findings? A one-person security team that can triage in real time can run continuous. A team with no dedicated security function and no plan for continuous will just accumulate an ignored backlog, which is worse than not having found the issues at all.
Rough mapping:
| Deployment frequency | Recommended model |
|---|---|
| Less than monthly | Annual / semi-annual |
| Monthly to bi-weekly | Quarterly minimum |
| Weekly or more | Sprint-based or continuous |
SLAs are what keep a continuous program from collapsing
Continuous testing produces continuous findings, and without a severity-tiered SLA, that turns into noise the engineering team eventually starts ignoring.
| Severity | CVSS | Acknowledge | Remediate | Retest |
|---|---|---|---|---|
| Critical | 9.0–10.0 | 4h | 48h | within 24h of fix |
| High | 7.0–8.9 | 24h | 7d | within 48h of fix |
| Medium | 4.0–6.9 | 72h | 30d | within sprint |
| Low | 0.1–3.9 | 1 week | 90d | next quarterly |
Why continuous programs actually die
Most don't fail because the testing was bad. They fail for one of four operational reasons:
- Finding fatigue without triage. Everything gets flagged with the same urgency, engineers tune it out. Fix: CVSS-based SLAs, a security champion per team doing first-pass triage, a weekly standup instead of ad-hoc tickets.
- Testing is time-based instead of change-based. A calendar cadence misses whatever shipped between checkpoints. Fix: scope each cycle off actual PR/change data, not the calendar (this is what the sprint risk-scoring class above is for).
- Surface monitoring without ownership. New subdomains and endpoints get flagged and nobody owns the follow-up. Fix: a rotation that owns the alert queue, with an explicit SLA (72h is reasonable) for investigating new surface.
- Compliance-minimum thinking. The program quietly reverts to "just enough for the auditor." Fix: report on breach-probability and incident-cost terms at the board level, not just pass/fail testing status, so the program's ROI stays visible.
Metrics that actually tell you if it's working
| Metric | Annual baseline | Continuous target |
|---|---|---|
| Mean time to detection | ~180 days | <14 days |
| Mean time to remediation | ~45 days (batch quarterly) | <7 days |
| Vulnerability escape rate | ~25% (found by someone else first) | <5% |
| SLA compliance rate | ~60% | >95% |
| False positive rate | ~40% | <10% |
| Attack surface coverage | ~70% | >95% |
None of these should be optimized in isolation. The point of tracking them together is to see whether the gap between "change ships" and "change gets validated" is actually shrinking over time, not just whether a test happened.
Where AI actually changes the equation
The reason traditional pentest firms can't support a monthly cadence isn't unwillingness, it's structural: a human consultant works a target sequentially over one to two weeks, and that timeline doesn't compress just because you want it to.
What changes the math is running large numbers of specialized exploit agents in parallel against a target instead of one consultant working through it linearly, and carrying codebase context forward between engagements instead of starting cold every time, which is closer to how an attacker who's already been inside your codebase (via source access, leaked credentials, or a prior foothold) would actually operate. That combination, source-aware testing plus accumulated context, is what makes a 48-hour full-engagement turnaround and unlimited free retesting operationally sustainable instead of a pricing gimmick.
If you want to see how that model works end to end: CodeAnt AI's continuous penetration testing platform.
The actual takeaway
Annual pentesting isn't wrong. It's a point-in-time answer to a problem that, for most SaaS teams now, is continuous. The question worth asking isn't "did we pass our last pentest," it's "how long does a newly introduced risk sit untested before anyone looks at it." If that number is measured in months, the testing cadence and the deployment cadence have drifted apart, and that drift is the actual attack surface.
For the cost-side deep dive behind the TCO numbers above, see CodeAnt's guide to penetration testing costs. For the operational checklist on choosing a recurring testing vendor, see PTaaS provider SLAs: what to look for.
Top comments (0)