DEV Community: Patience Mpofu

I Used AI to Help Remediate Vulnerabilities — Here's How Useful It Actually Was

Patience Mpofu — Sun, 06 Sep 2026 06:28:10 +0000

Everyone is talking about AI-assisted security. Fewer people are being honest about what it actually looks like in practice.

I used Claude throughout the MFlix remediation project — for understanding vulnerabilities, planning upgrades, writing migration code, and reviewing my remediation decisions. This article is an honest accounting of where it helped, where it was confidently wrong, and what the experience taught me about the appropriate role of AI in security engineering work.

How I Used It

Four distinct use cases across the project:

Vulnerability explanation — understanding what a specific CVE actually enables
Remediation planning — figuring out the right fix approach for complex upgrades
Code generation — writing the migration code for API changes
Decision review — sanity-checking suppression decisions Each use case produced different results. Let me walk through them honestly.

Use Case 1: Vulnerability Explanation — Where AI Genuinely Shines

When Snyk flagged spring-beans@5.0.7 with a Remote Code Execution at CVSS 9.8, my first question was: what does this actually enable? The CVE database entry is often terse. The Snyk description gives you the vulnerability class but not always the attack mechanics.

I asked Claude to explain exactly how the spring-beans RCE worked — what an attacker needed to control, what the exploit chain looked like, and how the application's specific configuration affected exploitability.

The explanation was excellent. It walked through the class loading mechanism in CachedIntrospectionResults, explained why certain HTTP request parameters could trigger it, and was clear about the conditions required — HTTP endpoint exposure, specific Spring MVC configuration, no input filtering at the framework layer.

More importantly, it helped me think through MFlix's specific exposure: yes, the public movie search endpoint is unauthenticated, yes it accepts HTTP parameters, yes the Spring MVC configuration in MFlix matches the vulnerable pattern.

This kind of contextual vulnerability analysis — "here's the CVE, here's my application, am I actually exposed?" — is exactly where AI adds value. It accelerates the analysis that a security engineer would do manually, and it's good at it because the underlying information (CVE mechanics, Spring internals, application patterns) is well-documented and within training data.

Accuracy: High. I cross-referenced the explanation against the Spring Security advisories and the CVE detail page. The core mechanics were correct. The application-specific analysis was sound.

Use Case 2: Remediation Planning — Mixed Results

Planning the jjwt 0.9.1 → 0.12.0 migration is where I first encountered AI's most significant failure mode: confident incorrectness.

I asked for a migration guide from jjwt 0.9.1 to 0.12.0. The response was detailed, structured, and plausible-looking. It included code examples showing the old and new API patterns, explained the artifact structure change (monolith to three separate artifacts), and described the key type requirement change.

Three of the code examples were wrong.

Not subtly wrong — wrong in ways that would cause compile errors or runtime failures. The Keys.hmacShaKeyFor() usage was correct but the import path was from a version that didn't exist. The Jwts.SIG.HS256 syntax was correct but the surrounding builder pattern had a method that was removed in 0.11.x, not present in 0.12.0. The claims parsing example used .getBody() which is the 0.9.x API, not .getPayload() which is the 0.12.x API.

When I pointed out the errors, Claude corrected them — but the corrections introduced new errors. The model had detailed knowledge of jjwt 0.9.x and general knowledge of the 0.12.x direction, but its specific knowledge of the 0.12.0 API surface was unreliable.

This is the pattern I encountered repeatedly: AI is excellent at explaining concepts and patterns, but unreliable on specific API signatures for library versions that changed after its training data cutoff or that were underrepresented in training data.

The correct workflow I developed: Use AI to understand the shape of what needs to change, then verify every specific API detail against the official documentation or source code. Never trust an AI-generated import path or method signature without checking it.

Accuracy: Medium. Conceptually correct, specifically unreliable.

Use Case 3: Code Generation — Useful With Verification

For the Spring Security configuration migration from WebSecurityConfigurerAdapter to the SecurityFilterChain bean pattern, I asked Claude to rewrite the existing configuration in the new style.

The generated code was largely correct. The SecurityFilterChain bean structure was right. The authorizeHttpRequests lambda syntax was right. The SessionCreationPolicy.STATELESS configuration was right.

Two issues:

The antMatchers() → requestMatchers() rename was handled correctly in the authorizeHttpRequests block but missed in a separate method I hadn't shown in my prompt. This is a prompt engineering failure as much as a model failure — I didn't include the full configuration, so the model couldn't see everything that needed changing.

The AuthenticationManager bean configuration was slightly wrong — the generated code used a deprecated method for retrieving it from the AuthenticationConfiguration. The correct approach required checking the Spring Security 6.x migration guide.

The workflow that worked: Provide the complete existing code in the prompt, ask for the migration, then run the tests. The tests caught both issues immediately. AI-generated code that passes tests is trustworthy; AI-generated code that you haven't run is not.

Accuracy: High with verification. Tests are not optional when using AI-generated code.

Use Case 4: Suppression Decision Review — Surprisingly Valuable

This was the use case I expected least from and got most from.

Before finalising any suppression decision, I described the finding and my reasoning to Claude and asked it to challenge my logic. "I'm planning to suppress this jackson-databind deserialization finding because we don't use polymorphic deserialization. Here's my evidence. What am I missing?"

The responses were genuinely useful — not because the model had superior security knowledge, but because articulating the reasoning to an external entity forced me to be more precise, and the model asked clarifying questions that identified gaps in my analysis.

For the jackson-databind suppression, it asked: "Have you verified that no third-party library you import configures Jackson's default typing on your behalf?" I hadn't checked that. I checked. Nothing did. But the question was right — a common source of polymorphic deserialization vulnerabilities is a transitive dependency that configures Jackson in the background, not direct application code.

For the MongoDB driver suppression, it pushed back on my "Atlas TLS enforcement mitigates the MitM risk" argument more effectively than I'd expected. The counter-argument: Atlas TLS enforcement prevents cleartext transmission but doesn't fully mitigate driver-level certificate validation failures, which can be exploited even over an encrypted channel under certain network conditions. The suppression was still justified, but the reasoning needed to be more precise.

Accuracy: High for challenging reasoning. Using AI as a devil's advocate for security decisions is one of its most effective use cases — not because it's always right, but because the process of explaining your reasoning exposes gaps.

The Failure Mode That Matters Most: Confident Incorrectness

Across all four use cases, one failure mode appeared repeatedly and is worth naming explicitly: confident incorrectness on version-specific details.

The model would state that a method existed in a specific library version with complete confidence — no hedging, no "you should verify this" — and be wrong. Not wrong about the concept, wrong about the specific API surface at a specific version.

For security work this is particularly dangerous. The difference between setSigningKey(String) (vulnerable in jjwt 0.9.x, accepts weak keys) and signWith(SecretKey, SignatureAlgorithm) (secure in 0.12.x, enforces key strength) is the difference between a secure and an insecure JWT implementation. If you trust the AI's method signature and it's wrong, you might implement the old vulnerable pattern thinking you've implemented the secure one.

The rule I developed: Trust AI for concepts. Verify AI for specifics. Any method name, import path, version number, or configuration value that AI provides should be cross-referenced against official documentation before use in security-sensitive code.

Where AI Added the Most Net Value

Ranking the use cases by net value delivered:

1. Vulnerability explanation — highest value, highest accuracy. Understanding attack mechanics is conceptual work that AI handles well.

2. Suppression decision review — high value, unexpected. Using AI as a challenger rather than an oracle is an underrated pattern.

3. Code generation — medium value, requires verification. Fastest when used as a starting point with tests as the quality gate.

4. Remediation planning — lowest net value due to reliability issues. Better to read the official migration guide directly and use AI to clarify specific points you don't understand.

What This Means for the NerdWallet Role

The job description explicitly mentions building AI-powered security systems including RAG pipelines and automated code review. Having done this project gives me a concrete, honest perspective on AI in security work that I think is more valuable than enthusiasm.

The engineers who will build effective AI security tools aren't the ones who think AI is magic. They're the ones who understand:

Where AI is reliably accurate (concepts, pattern recognition, reasoning challenges)
Where AI is unreliably accurate (specific APIs, version details, novel configurations)
How to design systems that route the right tasks to AI and keep humans in the loop on the wrong ones
How to test and validate AI outputs in security contexts where confident incorrectness is dangerous An automated code review system that uses AI to identify suspicious patterns is valuable. The same system that uses AI to generate specific remediation code without human verification is dangerous — because the AI might generate code that looks like a fix but implements the vulnerable pattern.

The appropriate role of AI in security engineering is augmentation, not replacement. It compresses the time to understand a vulnerability, challenges your reasoning on risk decisions, generates starting points for remediation code. It does not replace reading the documentation, running the tests, or applying security judgment.

The Honest Bottom Line

AI made me faster on this project. It did not make me more accurate on its own — accuracy came from verifying AI outputs against authoritative sources and running tests.

The productivity gain was real: vulnerability analysis that might have taken an hour of reading took fifteen minutes with AI assistance. The jjwt migration planning took an afternoon partly because I trusted the first AI-generated migration guide more than I should have. The security configuration migration was straightforward partly because the AI-generated starting point was close enough that tests caught the gaps quickly.

Net verdict: meaningful productivity improvement with a significant failure mode that requires active management. The failure mode — confident incorrectness on specifics — is dangerous enough in security contexts that using AI without verification is worse than not using it at all.

That's not an argument against AI in security work. It's an argument for understanding what you're working with.

The completed MFlix project — remediated pom.xml, .snyk suppression file, security configuration, and full test suite — is at github.com/pgmpofu/mflix.

This concludes the MFlix/Snyk series. The full portfolio now spans three projects and twenty-one articles covering SAST tool design, ML-powered secrets detection, and real-world SCA remediation. If you found this series useful, the best thing you can do is star the repositories and share the articles with someone who's thinking about the AppSec transition.

Before and After: Measuring Security Posture Improvement With Real Metrics

Patience Mpofu — Sun, 06 Sep 2026 06:17:01 +0000

188 vulnerabilities. That's where we started.

After the modernisation, the jjwt migration, the targeted remediations, and the documented suppressions, here's where we ended up:

6 open findings. All suppressed with documented reasons. 0 unaddressed.

But "188 to 6" is a headline, not a measurement. This article is about what meaningful security posture measurement actually looks like — the metrics that tell a real story versus the ones that just make a dashboard look good.

Why Raw Finding Count Is a Weak Metric

The most common way teams measure SCA progress is finding count. Before: 188. After: 6. Improvement: 182 findings resolved. 97% reduction.

That number is real but it's also misleading in isolation. Here's why.

If I had suppressed all 188 findings without fixing anything, my finding count would also be 6. The dashboard would look identical. The actual security posture would be unchanged.

Finding count measures activity, not outcomes. What you need to measure is:

What risk was actually reduced — not just what got closed
What risk remains and why — the documented residual risk
How the remediation was achieved — fix vs. suppress breakdown
What the exploit exposure looks like — before and after exploit maturity These four dimensions together tell a story that a single number can't.

Metric 1: Risk Reduction by Severity

The most important before/after comparison is severity distribution, not total count.

Severity	Before	After Fixed	After Suppressed	Remaining
Critical	10	10	0	0
High	99	93	6	0
Medium	59	29	23	0 (7 in 4.x backlog)
Low	20	0	20	0
Total	188	132	49	0 unaddressed

Every Critical finding was fixed — not suppressed, fixed. That's the number that matters most. No Critical vulnerability was accepted as residual risk.

The 6 suppressed High findings are all in the "no known exploit" category with documented unreachability justifications. The 23 suppressed Medium findings are split between test-scope dependencies and unreachable code paths.

What to say when presenting this: "We eliminated all Critical exposure and resolved 94% of High findings through active remediation. The remaining 6 High findings are suppressed with documented justifications — unreachable code paths verified by codebase analysis — and have review dates set for Q2."

Metric 2: Fix vs. Suppress Breakdown

This metric distinguishes genuine security improvement from metric manipulation.

Resolution Type	Count	% of Total
Fixed — BOM upgrade	89	47%
Fixed — targeted remediation	43	23%
Suppressed — unreachable code path	23	12%
Suppressed — no known exploit	18	10%
Suppressed — test scope	9	5%
Tracked — accepted residual risk	6	3%

70% of findings were actively fixed. 30% were suppressed with documented reasons. Zero were closed without a reason.

The 47% resolved by BOM upgrade is worth highlighting specifically — it demonstrates the leverage of keeping framework versions current. Nearly half the entire vulnerability backlog was resolved by a single architectural change.

What this tells a hiring manager or security team: You didn't just run a scanner and close tickets. You made architectural decisions, did targeted remediations, and applied security judgment to suppression decisions. The breakdown shows the difference.

Metric 3: Exploit Exposure Before and After

Raw CVSS scores don't tell you how much real-world attack risk exists. Exploit maturity does.

Before remediation:

Exploit Maturity	Count	What It Means
Mature (working exploit exists)	8	Anyone with the tool can exploit this
Proof of concept	43	Requires effort but exploit path is known
No known exploit	137	Theoretical vulnerability, no public exploit

After remediation:

Exploit Maturity	Count	Status
Mature	0	All fixed
Proof of concept	0	All fixed
No known exploit	6	Suppressed with documentation

Every finding with working exploit code was fixed. Every finding with a published proof of concept was fixed. The 6 remaining findings are all in the "no known exploit" category — the lowest practical risk tier.

This is the metric that matters most for communicating real risk reduction. Going from 8 mature exploits to 0 means an attacker with a public tool can no longer trivially compromise this application. That's a concrete, meaningful security improvement.

Metric 4: Priority Score Reduction

Snyk's priority score (0-1000) combines CVSS, exploit maturity, reachability, and other factors into a single number per finding. It's more nuanced than CVSS alone.

Before: Highest priority score was 919 (spring-web RCE, spring-context RCE, spring-boot-starter-security auth bypass — multiple findings at maximum score)

After: Highest priority score among remaining findings is 329 — the Improper Handling of Case Sensitivity finding in spring-core that requires a major version upgrade to resolve and is suppressed with justification.

The maximum priority score dropped from 919 to 329. The most critical attack vectors have been eliminated.

Metric 5: Dependency Health Score

Beyond individual vulnerabilities, the modernisation improved the overall health of the dependency tree in ways that reduce future vulnerability accumulation.

Before:

Spring Boot: 2.0.3/2.0.4 — released May 2018, EOL
Spring Framework: 5.0.7 — released June 2018, EOL
jjwt: 0.9.1 — released November 2018, unmaintained
Java target: 1.8 — EOL for free Oracle support
Dependencies: 8 manually pinned, no BOM After:
Spring Boot: 3.2.5 — current LTS, actively maintained
Spring Framework: 6.1.x — current, actively maintained
jjwt: 0.12.0 — current, actively maintained
Java target: 17 — current LTS
Dependencies: BOM managed, version alignment enforced This isn't just about current CVEs. A project on EOL dependencies accumulates new CVEs constantly as researchers continue to find vulnerabilities in old versions that never receive patches. Moving to actively maintained versions means future CVE disclosures will have patches available.

The dependency health score is a leading indicator — it predicts future vulnerability accumulation, not just current state.

Metric 6: Mean Time to Remediate (Simulated)

In a production environment, MTTR (Mean Time to Remediate) by severity is a standard AppSec programme metric. For this project I can simulate what it would have looked like:

Severity	Discovery Date	Remediation Date	MTTR
Critical (RCE)	Day 1 — scan	Day 3 — BOM upgrade	2 days
Critical (Auth bypass)	Day 1 — scan	Day 3 — BOM upgrade	2 days
High (jjwt)	Day 1 — scan	Day 5 — jjwt migration	4 days
High (Tomcat)	Day 1 — scan	Day 3 — BOM upgrade	2 days
Medium (MongoDB MitM)	Day 1 — scan	Day 7 — driver patch	6 days

Average MTTR for Critical: 2 days
Average MTTR for High: 3 days
Average MTTR for Medium: 6 days

Industry benchmarks for production systems typically target Critical remediation within 24-72 hours and High within 7-14 days. This project would have met those benchmarks.

The Dashboard View

If you were presenting this to an engineering team or security leadership, here's how the story looks in dashboard format:

SECURITY POSTURE SUMMARY — pgmpofu/mflix
Scan date: [date]
Previous scan: [date — before remediation]

FINDING SUMMARY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                Before    After    Change
Critical          10        0      -100%
High              99        0*     -100%
Medium            59        0*      -100%
Low               20        0*     -100%
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Total            188        6*      -97%

*6 findings suppressed with documented justification
 All suppressions have expiry dates and review schedule

EXPLOIT EXPOSURE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Mature exploits:   8 → 0    (-100%)
Proof of concept: 43 → 0    (-100%)
No known exploit: 137 → 6   (-96%)

RESOLUTION BREAKDOWN
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Fixed (BOM upgrade):          89  (47%)
Fixed (targeted):             43  (23%)
Suppressed (documented):      56  (30%)
Unaddressed:                   0   (0%)

What These Metrics Don't Measure

Honest measurement includes acknowledging the gaps.

Code-level vulnerabilities. Snyk SCA scans dependencies, not your code. The metrics above say nothing about whether the application code itself has injection vulnerabilities, authentication bypasses, or insecure cryptography. That requires SAST — which is exactly what the SAST tool series was about.

Runtime behaviour. A dependency can be vulnerable without being exploitable in a specific application's runtime configuration. The metrics above are conservative — they count vulnerabilities as present even when compensating controls exist.

New vulnerabilities. The scan represents a point in time. New CVEs are disclosed daily. Without a recurring scan schedule and a process to act on new findings, the posture degrades over time. The metrics are only meaningful if they're refreshed regularly.

The MongoDB 4.x backlog. The 6 tracked residual risk items from the MongoDB driver partial upgrade aren't reflected in these metrics. They're tracked separately and represent the honest incomplete work.

The Number That Actually Matters

Of all the metrics in this article, one number is most important:

0 Critical findings. 0 mature exploits.

Everything else — the 97% reduction, the fix/suppress breakdown, the priority score drop — supports that headline. But that's the number a CISO, a hiring manager, or a security audit cares about.

You started with 10 Critical vulnerabilities including a Remote Code Execution at CVSS 9.8. You ended with zero. Every vulnerability with working public exploit code was remediated. The application went from a state where a knowledgeable attacker with public tools could trivially compromise it, to a state where exploitation requires novel research against theoretical vulnerabilities in unreachable code paths.

That's what security posture improvement looks like in measurable terms.

The remediated repository with full .snyk suppression documentation is at github.com/pgmpofu/mflix.

Final article: I used AI to help remediate vulnerabilities — here's how useful it actually was, where it added value, and where it confidently gave me wrong answers.

Why I Suppressed 30% of Snyk's Recommendations

Patience Mpofu — Sun, 09 Aug 2026 07:41:08 +0000

188 vulnerabilities. That's what Snyk found in MFlix.

I fixed 132 of them. I suppressed 56.

If you're reading that and thinking "shouldn't you fix everything?" — that's a reasonable instinct and also the wrong mental model for how AppSec actually works in practice. This article is about why.

Not why I was lazy. Not why I cut corners. But why a security engineer who fixes every finding without thinking is doing something closer to compliance theatre than security work — and why documented, reasoned suppression decisions are a legitimate and necessary part of a mature security programme.

The Core Principle: Risk Acceptance Is Not Risk Ignorance

Every security programme operates with finite resources against an infinite potential attack surface. The question is never "have we eliminated all risk?" — that's impossible. The question is "have we reduced risk to an acceptable level, and do we have documented, defensible reasons for the risks we've chosen to accept?"

Suppressing a Snyk finding without a documented reason is risk ignorance. You've decided not to fix something and you haven't recorded why.

Suppressing a Snyk finding with a clear, reasoned justification is risk acceptance. You've evaluated the finding, understood the attack scenario, assessed the likelihood and impact in your specific context, and made a deliberate decision that the residual risk is acceptable given the cost of remediation.

The difference matters when something goes wrong — and in security, something always eventually goes wrong. "We assessed this finding and accepted the risk because X, Y, Z" is a defensible position. "We didn't get around to it" is not.

With that principle established, here's exactly how I made suppression decisions on MFlix.

The Decision Framework

Every finding went through four questions in order:

1. Is the vulnerable code path reachable in this application?
A vulnerability in a library function that MFlix never calls has zero exploitability regardless of its CVSS score. Unreachable code paths are candidates for suppression.

2. What does the exploit maturity look like?
Snyk's exploit maturity classification — mature exploit, proof of concept, no known exploit — is the most important signal after reachability. A vulnerability with working public exploit code demands different urgency than one that's theoretically exploitable but requires novel research to weaponise.

3. What are the compensating controls?
Is there something else between the attacker and the vulnerable component that reduces the practical risk? Network-level access controls, WAF rules, authentication requirements, rate limiting — these don't eliminate vulnerabilities but they change the realistic attack surface.

4. What does remediation cost versus what does it buy?
Some fixes are a one-line version bump. Others require API migrations across dozens of files, introduce breaking changes, or require testing that takes days. The remediation cost has to be weighed against the risk reduction.

A finding that scores poorly on all four — reachable, mature exploit, no compensating controls, cheap to fix — gets fixed immediately. A finding that scores well on all four — unreachable, no known exploit, multiple compensating controls, expensive to fix — is a strong suppression candidate.

Most findings fall somewhere in between, which is where judgment comes in.

The Findings I Suppressed and Why

Category 1: Unreachable Code Paths (23 findings)

The largest suppression category. These are vulnerabilities in library functions that MFlix imports but never calls.

The clearest example is a cluster of findings in jackson-databind around its polymorphic deserialization feature. The vulnerability is real — if an application uses Jackson's default typing or @JsonTypeInfo with As.PROPERTY, an attacker can instantiate arbitrary classes. The CVSS is 9.2.

MFlix doesn't use polymorphic deserialization. The application deserializes simple flat document structures from MongoDB — movie records, user objects, comments. There's no @JsonTypeInfo annotation anywhere in the codebase, no default typing configured on the ObjectMapper, and no endpoint that accepts the kind of nested polymorphic JSON the attack requires.

I verified this by searching the codebase for every Jackson configuration point and every deserialization call:

grep -r "JsonTypeInfo\|enableDefaultTyping\|activateDefaultTyping" src/
# No results

grep -r "ObjectMapper\|readValue" src/
# 3 results — all simple flat document deserialization

Zero results for the dangerous configuration. The attack path doesn't exist in this application.

Suppression entry:

Finding: jackson-databind Deserialization of Untrusted Data (CWE-502, CVSS 9.2)
Decision: SUPPRESS
Reason: MFlix does not use polymorphic deserialization. No @JsonTypeInfo annotations,
no default typing configuration. All Jackson usage is flat document deserialization
with no user-controlled type information. Verified by codebase search 2024-11-15.
Review date: 2025-05-15

23 findings fell into this category — all vulnerabilities in Jackson, Spring's expression language engine, and Tomcat's JSP compiler that require application-level configurations or usage patterns that don't exist in MFlix.

Category 2: No Known Exploit (18 findings)

Snyk's exploit maturity filter showed 137 findings with "no known exploit." After removing those already resolved by the BOM upgrade and those in category 1, 18 remained as candidates for suppression on exploit maturity grounds alone.

These are real vulnerabilities in the CVE database with CVSS scores and CWE classifications. They're also vulnerabilities where no researcher has published working exploit code, no proof-of-concept exists, and exploitation would require novel security research to achieve.

The practical reality: the barrier to exploitation for these findings is "someone needs to figure out how to exploit this first." For a non-production portfolio project with no external attack surface, that barrier is sufficient to justify deferral.

I want to be clear about the difference between this decision for MFlix and this decision for a production system. For a real application with real users and real data, "no known exploit today" means "no known exploit yet" — and the calculation shifts significantly. For MFlix, which has never been deployed and has no external attack surface, it's a reasonable acceptance.

Suppression entry:

Finding: [18 findings, various packages]
Decision: SUPPRESS — DEFERRED
Reason: No known exploit in Snyk database as of scan date. No proof-of-concept
available. MFlix is a non-production portfolio project with no external attack
surface. Risk accepted pending any change in exploit maturity or deployment context.
Review date: quarterly or if exploit maturity changes.

Category 3: Non-Production Context (9 findings)

Nine findings are in test-scoped dependencies — libraries that are only used during test execution and never included in the production artifact.

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.1.0</version>
    <scope>test</scope>
</dependency>

JUnit 5.1.0 has several findings. None of them matter for production security because JUnit never ships in the application JAR. A vulnerability in the test framework cannot be exploited by an attacker targeting the running application — it's not there.

This is a common source of inflated finding counts in SCA tools. They scan the full dependency tree including test scope, which is appropriate for completeness but requires scope-aware triage.

Suppression entry:

Finding: junit-jupiter-api@5.1.0 [various findings]
Decision: SUPPRESS
Reason: Test-scoped dependency. Not included in production artifact.
No exploitability from external attack surface.

Category 4: Accepted Residual Risk Post-Remediation (6 findings)

Six findings remain after I applied every fix that was reasonably achievable without a full rewrite of the application's data access layer.

The specific case: mongodb-driver-sync@3.9.1 → 4.x is a major version upgrade with significant breaking API changes. The 3.x and 4.x MongoDB Java driver APIs are substantially different — connection management, session handling, and the reactive streams API all changed. Migrating would require rewriting significant portions of the DAO layer.

The vulnerability itself — a Man-in-the-Middle risk in the TLS connection handling, CWE-300, CVSS 6.4 — is real but has a compensating control: the MongoDB Atlas connection string already enforces TLS (mongodb+srv:// scheme requires TLS). The MitM risk is partially mitigated by the network-level TLS enforcement even with the driver-level validation weakness.

I upgraded to the latest 3.x patch (3.12.x) which addresses several other findings, documented the residual risk from the 4.x migration, and flagged it as a tracked backlog item rather than a suppression.

What I Fixed That I Could Have Suppressed

This is the part of this article that's most important for understanding how AppSec judgment actually works.

Several findings I fixed could have been legitimately suppressed under the framework above. I fixed them anyway. Here's why.

The jackson-databind findings I fixed: Alongside the suppressed polymorphic deserialization findings, there were jackson-databind findings in the JSON parsing path for user-submitted data — the comment posting and user registration endpoints. These are reachable from the application's attack surface. Even without default typing, certain jackson-databind parsing vulnerabilities can be triggered by malformed JSON input. These got fixed.

The Tomcat findings I fixed: The Insecure Defaults finding in tomcat-embed-core@8.5.31 (CVSS 9.8) could have been argued as "we just won't enable AJP" and suppressed. I fixed it instead because CVSS 9.8 with a working proof of concept is a line I'm not comfortable crossing even with a compensating control argument. The fix was a version bump. The risk of not fixing it wasn't worth the two minutes it would have taken to suppress it.

The Spring Security authorization bypass: This one had a mature exploit classification. Regardless of how difficult the exploitation path was to follow in MFlix specifically, "Spring Security authorization bypass with a mature exploit" is not a finding I was going to suppress. Authentication and authorization are the core security controls in this application. I fixed it.

The principle: suppression requires genuine cost-benefit analysis. When the fix is cheap and the risk is non-trivial, fixing is always the right answer. Suppression is for when the fix is expensive or the risk is genuinely minimal — not for when fixing is inconvenient.

Documenting Suppressions in Practice

Snyk supports inline suppression via code comments and also through its dashboard where you can mark findings as "ignored" with a reason and expiry date.

For MFlix I used Snyk's ignore functionality in .snyk file format:

# .snyk
version: v1.25.0
ignore:
  SNYK-JAVA-COMFASTERXMLJACKSONCORE-*:
    - '*':
        reason: >
          Polymorphic deserialization not used in MFlix. No @JsonTypeInfo
          annotations. All Jackson usage is flat document deserialization.
        expires: '2025-05-15T00:00:00.000Z'
        created: '2024-11-15T00:00:00.000Z'
  SNYK-JAVA-ORGJUNITJUPITER-*:
    - '*':
        reason: Test-scoped dependency. Not in production artifact.
        expires: '2025-11-15T00:00:00.000Z'
        created: '2024-11-15T00:00:00.000Z'

Two things about this format worth noting:

Expiry dates are mandatory in my process. Every suppression gets a review date. Security posture changes — exploit maturity changes, deployment context changes, the application gets extended with new functionality that makes previously unreachable code paths reachable. A suppression without an expiry date is a suppression that gets forgotten.

Reasons are complete sentences. Not "not applicable" or "false positive." A complete sentence explaining exactly why the finding doesn't apply or why the risk is accepted. Future-you reading this six months later needs to understand the reasoning, not just the conclusion.

The Numbers

Category	Count	% of Total
Fixed by BOM upgrade	89	47%
Fixed by targeted remediation	43	23%
Suppressed — unreachable code path	23	12%
Suppressed — no known exploit	18	10%
Suppressed — test scope	9	5%
Tracked — accepted residual risk	6	3%
Total	188	100%

The BOM upgrade doing 47% of the work in one change is the most important number in that table. It's the strongest argument for keeping your framework versions current — not just for features but because the framework's own dependency management does an enormous amount of security maintenance work for you automatically.

What This Looks Like in a Real Organisation

In a production AppSec programme, this decision process doesn't happen in isolation. It involves:

A risk register where accepted risks are formally documented and reviewed on a defined cadence
Security team sign-off on suppressions above a certain severity threshold
Engineering sign-off on the technical reasoning for unreachability claims
Audit trail that shows who made each decision and when

For MFlix — a portfolio project — I've approximated this with the .snyk file, this article, and the commit history. For a real team, the process would be more formal but the underlying reasoning is identical.

The skill being demonstrated here isn't "I know how to run Snyk." It's "I can evaluate a finding, reason about its applicability to a specific system, make a defensible risk decision, and document it in a way that survives scrutiny."

That's application security engineering. The tool is just the starting point.

The .snyk file with all suppression entries and the full remediated pom.xml are in the repository at github.com/pgmpofu/mflix.

Next up: the remediation work itself — the version bumps that were easy, the ones that introduced breaking changes, and the MongoDB driver upgrade that I decided wasn't worth the cost.

Running Snyk on Real Legacy Java Code — The Full Unfiltered Results

Patience Mpofu — Sat, 25 Jul 2026 20:54:41 +0000

Numbers are easy to skim.

10 Critical. 99 High. 59 Medium. 20 Low. 188 total. Those numbers appeared in the first article and they're striking — but they don't tell you what an attacker could actually do with them.

This article is the deep dive. I'm going to take the most significant findings from the Snyk scan of MFlix and explain what each vulnerability actually enables, why it matters specifically for this application's threat model, and what the relationship is between the dependency version and the attack.

Not all 188 findings — the ones that matter. The ones with working exploit code. The ones with CVSS scores above 9.0. The ones where the application's specific functionality intersects with the vulnerability in a way that makes it genuinely dangerous.

The Threat Model First

Before talking about individual vulnerabilities, it's worth being explicit about what MFlix actually is and who would be attacking it.

MFlix is a web application with:

A public endpoint for movie search (unauthenticated)
A user registration and login endpoint (unauthenticated)
Authenticated endpoints for posting comments and accessing user data
A MongoDB Atlas backend
JWT-based session management
A Spring Security access control layer The realistic threat actors are:
Automated vulnerability scanners looking for known CVEs
Attackers who have identified the Spring Boot version via response headers and are targeting known exploits
Authenticated users attempting privilege escalation
Network-layer attackers attempting to intercept database traffic With that model established, here's what the findings actually mean.

Finding 1: Remote Code Execution in spring-beans — CVSS 9.8

Package: org.springframework:spring-beans@5.0.7.RELEASE
CWE: CWE-94 — Improper Control of Generation of Code
CVSS: 9.8
Priority Score: 919
Exploit maturity: Proof of concept available

This is the finding that made me stop and re-read the output twice.

Remote Code Execution means an attacker can execute arbitrary commands on the server running the application. CVSS 9.8 is one point below the theoretical maximum. This is not a "could potentially lead to" vulnerability — it is a "an attacker who can reach this endpoint can run code on your server" vulnerability.

What the vulnerability is: The Spring Framework's CachedIntrospectionResults class, which handles JavaBean property introspection, improperly handles class loading in certain configurations. An attacker who can craft a malicious request containing a specific class path expression can cause the application to load and execute attacker-controlled code.

This is related to — but distinct from — Spring4Shell (CVE-2022-22965), which made significant news in 2022. Spring4Shell required specific conditions: Java 9+, Spring Framework 5.3.x or 5.2.x, deployed as a WAR on Tomcat, with specific Spring MVC configurations. The RCE in spring-beans@5.0.7 has different preconditions but the same fundamental class of vulnerability.

Why it matters for MFlix specifically: MFlix exposes public HTTP endpoints — the movie search and authentication endpoints don't require a logged-in user. An attacker with network access doesn't need credentials to reach the vulnerable code path. The attack surface is the public-facing API.

The fix: Upgrading spring-web to 5.2.20.RELEASE or later resolves this. Under the BOM approach from article 2, this happens automatically when you upgrade to Spring Boot 2.7.x or 3.x.

Finding 2: Insecure Defaults in tomcat-embed-core — CVSS 9.8

Package: org.apache.tomcat.embed:tomcat-embed-core@8.5.31
CWE: CWE-453 — Insecure Default Variable Initialization
CVSS: 9.8
Priority Score: 704
Source: Transitive dependency via spring-boot-starter-web@2.0.3

This finding arrives through a transitive dependency — you never see tomcat-embed-core in the pom.xml, but it's the embedded servlet container that Spring Boot uses to serve HTTP requests. Every Spring Boot web application embeds Tomcat by default.

What the vulnerability is: Tomcat 8.5.31 ships with insecure default configurations in its AJP (Apache JServ Protocol) connector. AJP is a binary protocol used for communication between a web server (like Apache httpd) and Tomcat. In certain deployment configurations, the AJP connector accepts connections without sufficient authentication, allowing an attacker to read arbitrary files from the server and potentially achieve remote code execution.

This vulnerability is related to the "Ghostcat" class of vulnerabilities (CVE-2020-1938) that affected Tomcat's AJP connector.

Why it matters for MFlix: In a typical development or cloud deployment, the AJP connector would be disabled entirely. But tomcat-embed-core@8.5.31 enables it by default. A developer who deploys MFlix without explicitly disabling AJP is running an exposed connector.

The application.properties in the original MFlix doesn't disable AJP. That's an omission that creates real attack surface.

The fix:

Option A — Upgrade to a fixed Tomcat version (handled by BOM upgrade):

<!-- Spring Boot 3.x pulls in Tomcat 10.x, which addresses this -->

Option B — Explicitly disable AJP in application configuration:

# application.properties
server.tomcat.ajp.enabled=false

The BOM upgrade resolves this automatically, but the explicit disable is worth adding regardless — it's defence in depth and makes the intent clear.

Finding 3: Deserialization of Untrusted Data in jackson-databind — CVSS 9.2

Package: com.fasterxml.jackson.core:jackson-databind@2.9.6
CWE: CWE-502 — Deserialization of Untrusted Data
CVSS: 9.2
Priority Score: 889
Source: Transitive dependency via both spring-boot-starter-web@2.0.3 AND io.jsonwebtoken:jjwt@0.9.1

This finding appears multiple times in the Snyk output — because jackson-databind@2.9.6 is a transitive dependency of multiple direct dependencies. When the same vulnerable library is pulled in through different dependency paths, Snyk correctly reports it once per path.

What the vulnerability is: Jackson's polymorphic deserialization feature, when enabled with the @JsonTypeInfo annotation or default typing, allows an attacker who controls JSON input to instantiate arbitrary Java classes. Certain "gadget" classes in common Java libraries — classes that perform dangerous operations in their constructors or setters — can be chained together to achieve remote code execution when they're instantiated during deserialization.

The attack pattern:

Application accepts JSON input from a user
That JSON is deserialized using Jackson with polymorphic typing enabled
Attacker crafts JSON that specifies a malicious class type
Jackson instantiates the attacker-specified class, triggering its constructor
The constructor performs a dangerous operation — file write, network connection, command execution Why it matters for MFlix specifically: MFlix's comment posting endpoint accepts JSON. The user registration endpoint accepts JSON. If the deserialization configuration on any of these endpoints uses polymorphic typing without blocklisting dangerous classes, the attack surface is real.

The jjwt@0.9.1 dependency is the more concerning vector here. JWT parsing involves deserialization of the JWT payload. If the JWT library delegates to Jackson for payload deserialization and the Jackson configuration is permissive, an attacker who can forge a JWT — or who can supply a malicious token that reaches the parsing code before signature verification — could trigger the deserialization vulnerability.

This is why the jjwt→0.12.0 upgrade matters beyond just the CVSS score. The new library was explicitly rewritten to avoid this deserialization path.

The fix: Upgrade jackson-databind to 2.13.x or later (handled by BOM upgrade). Explicitly disable default typing in Jackson configuration as defence in depth:

@Bean
public ObjectMapper objectMapper() {
    ObjectMapper mapper = new ObjectMapper();
    // Explicitly disable default typing to prevent gadget attacks
    mapper.deactivateDefaultTyping();
    return mapper;
}

Finding 4: Certificate Host Mismatch in spring-boot-autoconfigure — CVSS 9.3

Package: org.springframework.boot:spring-boot-autoconfigure@2.0.3.RELEASE
CWE: CWE-297 — Improper Validation of Certificate with Host Mismatch
CVSS: 9.3
Priority Score: 679

This finding is the one most specific to MFlix's architecture — and the one I'd argue is most dangerous in practice despite not having the RCE severity of finding 1.

What the vulnerability is: Spring Boot's autoconfiguration for SSL/TLS connections, in version 2.0.3, improperly validates the hostname in TLS certificates. When the application makes outbound connections — to the MongoDB Atlas cluster, to external APIs, to any HTTPS endpoint — it may accept certificates where the hostname doesn't match the certificate's Common Name or Subject Alternative Names.

In plain terms: the application might connect to attacker.com while believing it's connected to cluster.mongodb.net.

Why it matters for MFlix specifically: MFlix's entire data layer connects to MongoDB Atlas over TLS. Every database query goes through this connection. If the autoconfigure TLS validation is improperly validating the Atlas certificate, a network-position attacker — someone on the same network segment, or with DNS control, or with BGP hijacking capability — could intercept the database connection.

What would they see? Every MongoDB query the application sends. Every result it receives. User credentials stored in the database. User session tokens. Movie data. Comment content. Everything.

This isn't a theoretical attack in cloud environments — misconfigured network routing, shared hosting scenarios, and developer environments with local DNS modification are all realistic scenarios where this attack path is viable.

The fix: BOM upgrade to Spring Boot 2.5.x or later resolves the autoconfigure TLS validation. Additionally, explicitly configure SSL validation in the MongoDB connection:

# application.properties
spring.data.mongodb.uri=${MONGODB_URI}&ssl=true&sslInvalidHostNameAllowed=false

The sslInvalidHostNameAllowed=false is the critical flag — it explicitly disables the insecure behaviour that the autoconfigure vulnerability enables.

Finding 5: Authorization Bypass in spring-security-web — CVSS 8.2

Package: org.springframework.security:spring-security-web@5.0.7.RELEASE
CWE: CWE-285 — Improper Authorization
CVSS: 8.2
Priority Score: 731
Source: Transitive dependency via spring-boot-starter-security@2.0.4

This finding is particularly relevant for MFlix because MFlix uses Spring Security as its primary access control mechanism.

What the vulnerability is: Spring Security's path matching logic, in certain versions, could be bypassed by request paths that contain specific special characters or path traversal sequences. An attacker could access endpoints that should require authentication by crafting a URL that matches the underlying resource but doesn't match Spring Security's access control patterns.

For example, if Spring Security protects /api/v1/users/profile but the path matching doesn't normalise path traversal sequences, a request to /api/v1/users/./profile might reach the endpoint without authentication.

Why it matters for MFlix specifically: MFlix's security configuration has:

/api/v1/movies/** — public
/api/v1/users/login — public
Everything else — requires authentication An authorization bypass vulnerability means the "everything else requires authentication" rule might have holes. User profile data, comment management, and administrative functionality could potentially be accessed without a valid JWT.

The fix: Spring Security 5.7.x and later include significantly improved path normalisation. The BOM upgrade handles this, but it's worth explicitly testing authentication enforcement after the upgrade:

// Test that authenticated endpoints reject unauthenticated requests
@Test
void unauthorizedAccessShouldReturn401() {
    mockMvc.perform(get("/api/v1/users/profile"))
        .andExpect(status().isUnauthorized());
}

// Test that path traversal attempts are rejected
@Test
void pathTraversalShouldBeRejected() {
    mockMvc.perform(get("/api/v1/users/../admin/users"))
        .andExpect(status().isForbidden());
}

These tests should exist regardless of whether the vulnerability is present — they're part of the security test baseline for any authenticated application.

Finding 6: Man-in-the-Middle in mongodb-driver-sync — CVSS 6.4

Package: org.mongodb:mongodb-driver-sync@3.9.1
CWE: CWE-300 — Channel Accessible by Non-Endpoint
CVSS: 6.4
Priority Score: 534

This is the only direct dependency finding that isn't in the Spring ecosystem. The MongoDB Java driver version 3.9.1 has a vulnerability in how it handles TLS connections to the MongoDB server.

What the vulnerability is: The driver, in certain connection configurations, doesn't enforce strict certificate chain validation for its TLS connections. An attacker in a network-privileged position — same network segment, DNS poisoning, BGP hijacking — could present a fraudulent certificate and establish a man-in-the-middle position between the application and the database.

Why it matters for MFlix: Combined with finding 4 (the autoconfigure certificate mismatch), this creates a layered TLS validation problem. The autoconfigure layer doesn't validate the host properly, and the driver layer doesn't enforce strict certificate chain validation. In a vulnerable deployment, both controls that should prevent a MitM attack are weakened.

The fix: Upgrade to mongodb-driver-sync@4.9.x or later (the 3.x to 4.x upgrade is a breaking change with significant API differences, which is why it appears in the "breaking changes" section of article 5):

<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongodb-driver-sync</artifactId>
    <version>4.11.1</version>
</dependency>

The Findings That Didn't Make This List

A natural question: what about the other 182 findings I haven't discussed?

They fall into three categories.

Handled by the BOM upgrade. The majority of findings — particularly the large number of Spring Framework and Spring Boot component CVEs — are resolved automatically when the parent BOM is upgraded to Spring Boot 3.2.5. I'll show the exact count in article 6.

Lower severity findings with no exploit code. The 137 findings with "no known exploit" in the exploit maturity classification are real vulnerabilities, but their practical risk is lower. An attacker needs to develop custom exploit code to leverage them, compared to downloading a working tool for the 8 findings with mature exploits. These get tracked but don't drive remediation urgency.

Accepted risk findings. A small number of findings — which I'll discuss in detail in article 4 — represent vulnerabilities that either can't be exploited in MFlix's specific deployment context, require conditions that don't exist in the application, or have remediation paths that introduce more risk than they resolve. These get documented suppression decisions rather than patches.

Reading Snyk Findings Like an AppSec Engineer

The biggest shift in how I read the Snyk output now versus how I would have read it before this project is the difference between asking "what does this flag?" and "what can an attacker do?"

CVSS scores are a starting point. They represent the worst-case severity of the vulnerability in a generic context. But the actual risk to any specific application depends on:

Reachability — Is the vulnerable code path reachable from the application's attack surface? The RCE in spring-beans is reachable because MFlix exposes public HTTP endpoints. A vulnerability in a batch processing library that only runs scheduled jobs has a much smaller attack surface.

Exploit availability — Is there working exploit code publicly available? The 8 findings with mature exploits are categorically different from the 137 with no known exploit, regardless of their CVSS scores.

Application-specific amplification — Does the application's functionality make the vulnerability worse? The jackson-databind deserialization vulnerability is more dangerous in an application that accepts complex JSON structures from untrusted users (like MFlix's comment and registration endpoints) than in an application that only produces JSON.

Compensating controls — What else is between the attacker and the vulnerable component? Network-level controls, WAF rules, and rate limiting all affect the practical exploitability of vulnerabilities that require repeated requests or specific network positions.

Reading Snyk output through these four lenses — rather than sorting by CVSS and fixing from the top — is what produces a defensible remediation strategy rather than a checkbox exercise.

The full Snyk project for MFlix is at github.com/pgmpofu/mflix.

Next up: why I suppressed some findings and fixed others — the risk assessment framework, the role of exploit maturity, and the specific decisions I made on the findings that didn't get a patch.

Modernising a 6-Year-Old Spring Boot Project Without Breaking Everything

Patience Mpofu — Mon, 18 May 2026 14:27:18 +0000

Before I could meaningfully remediate the 188 vulnerabilities Snyk found in MFlix, I had to confront something uncomfortable.

The project structure itself was the problem.

Not the code — the code was fine for what it was. But the way it was organised, configured, and built reflected 2018 Spring Boot conventions that created friction for every subsequent change. Trying to apply modern security fixes to an unrenovated codebase is like trying to rewire a house without updating the fuse box. You can do it, but every step is harder than it needs to be.

This article is about the modernisation work I did before touching a single CVE — what the 2019 structure looked like, what I changed, why I changed it, and what I deliberately kept.

What a 2019 Spring Boot Project Looks Like

When MFlix was built, Spring Boot 2.0.x was the current major version. Java 8 was the standard enterprise runtime. The project structure followed conventions of that era:

mflix/
├── pom.xml
├── src/
│   ├── main/
│   │   ├── java/
│   │   │   └── mflix/
│   │   │       ├── api/
│   │   │       │   ├── MoviesController.java
│   │   │       │   └── UsersController.java
│   │   │       ├── config/
│   │   │       │   └── MongoDBConfiguration.java
│   │   │       └── daos/
│   │   │           ├── MovieDao.java
│   │   │           └── UserDao.java
│   │   └── resources/
│   │       └── application.properties
│   └── test/
│       └── java/
│           └── mflix/

Functional. Reasonable for its time. But several things stood out immediately when I looked at it with fresh eyes in 2025:

Java 8 compiler target. The pom.xml declared <source>1.8</source> and <target>1.8</target>. Java 8 reached end-of-life for free Oracle support in January 2019 — the same month this project was likely being committed. Six years of security patches, language improvements, and performance gains left on the table.

Mixed Spring Boot versions. The pom.xml declared spring-boot-starter-web@2.0.3 and spring-boot-starter-security@2.0.4 separately, with explicit version pinning on individual Spring Framework components (spring-context, spring-core, spring-web all at 5.0.7). Modern Spring Boot projects use a parent BOM (Bill of Materials) that manages version alignment across the entire Spring ecosystem. Manually pinning individual Spring component versions is how you end up with the kind of version drift that generates 188 CVEs.

No dependency management section. Without a <dependencyManagement> block or a parent BOM, transitive dependency versions are determined entirely by whatever the top-level dependencies pull in — with no explicit control or visibility.

application.properties with a hardcoded MongoDB URI. The connection string for the MongoDB Atlas cluster was in the properties file rather than being externalised to environment variables. That's not a Snyk finding, but it's a security hygiene issue that should be addressed before anything else.

The Modernisation Goals

I set three goals before writing a line of changed code:

Goal 1: Move to a Spring Boot parent BOM. This single change would bring version alignment across the entire Spring ecosystem under centralised control. Every Spring component version becomes managed by the BOM rather than individually pinned.

Goal 2: Upgrade the Java target to 17. Java 17 is the current LTS release and the minimum target for Spring Boot 3.x. Moving from Java 8 to Java 17 closes nine years of language evolution and gives access to Spring Boot 3.x's security improvements.

Goal 3: Externalise secrets. The MongoDB connection URI, JWT signing key, and any other credentials needed to move out of application.properties and into environment variables before any other change.

Goal 3 was intentionally first. Before running any security tooling or making any dependency changes, the sensitive configuration needed to be out of the code.

Step 1: Externalising Secrets

The application.properties contained:

spring.data.mongodb.uri=mongodb+srv://admin:password@cluster.mongodb.net/mflix
jwt.secret=mflix-jwt-secret-key

Both values needed to go. The replacement:

# application.properties — safe to commit
spring.data.mongodb.uri=${MONGODB_URI}
jwt.secret=${JWT_SECRET}

# .env — never committed, in .gitignore
MONGODB_URI=mongodb+srv://admin:password@cluster.mongodb.net/mflix
JWT_SECRET=mflix-jwt-secret-key

And .gitignore updated:

.env
*.env
application-local.properties

Simple. Ten minutes. Should have been done in 2019. The secrets detector I wrote would have caught both of these had it been running as a pre-commit hook — which is a satisfying bit of cross-project validation.

Step 2: Introducing the Spring Boot Parent BOM

The single most impactful structural change in the modernisation was adding the Spring Boot parent BOM.

Before:

<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>mongodb.university</groupId>
    <artifactId>mflix</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>2.0.3.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>5.0.7.RELEASE</version>
        </dependency>
        <!-- etc — every version pinned manually -->
    </dependencies>
</project>

After:

<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>mongodb.university</groupId>
    <artifactId>mflix</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.5</version>
        <relativePath/>
    </parent>

    <properties>
        <java.version>17</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <!-- No version — managed by parent BOM -->
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
            <!-- No version — managed by parent BOM -->
        </dependency>
        <!-- Individual spring-context, spring-core, spring-web removed -->
        <!-- BOM pulls in correct aligned versions automatically -->
    </dependencies>
</project>

What this change does:

The parent BOM declares tested, compatible versions for the entire Spring ecosystem
Individual Spring Framework components (spring-context, spring-core, spring-web) no longer need to be declared separately — they're pulled in as transitive dependencies of the starters at the correct version
java.version property drives compiler configuration through the parent's plugin management
All Spring component versions move in lockstep, eliminating the version drift that contributed to the CVE accumulation The version jump from 2.0.3 to 3.2.5 is a major version upgrade. Spring Boot 3.x dropped support for Java 8, requires Jakarta EE 10 namespace (jakarta.* instead of javax.*), and brought a range of breaking API changes. Those breaking changes are what make this step the most work-intensive part of the modernisation.

Step 3: The Jakarta Namespace Migration

Spring Boot 3.x moved from the javax.* namespace (Java EE) to jakarta.* (Jakarta EE). Every import in the codebase that referenced javax.servlet, javax.persistence, or similar needed updating.

In MFlix, the affected imports were primarily in the security configuration and controller layer:

Before:

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.validation.Valid;

After:

import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.validation.Valid;

This is mechanical work rather than architectural work — find and replace across the codebase. Modern IDEs handle it automatically with a refactoring tool. The risk is missing an occurrence, which produces a compile error rather than a runtime bug, so it's catchable.

Step 4: Spring Security Configuration Modernisation

The biggest code change in the modernisation was the Spring Security configuration. Spring Boot 3.x deprecated and then removed the WebSecurityConfigurerAdapter pattern that was standard in Spring Boot 2.x.

The 2019 pattern (deprecated, removed in Spring Boot 3.x):

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .csrf().disable()
            .authorizeRequests()
                .antMatchers("/api/v1/movies/**").permitAll()
                .antMatchers("/api/v1/users/login").permitAll()
                .anyRequest().authenticated()
            .and()
            .sessionManagement()
                .sessionCreationPolicy(SessionCreationPolicy.STATELESS);
    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(userDetailsService)
            .passwordEncoder(passwordEncoder());
    }
}

The modern pattern (Spring Boot 3.x):

@Configuration
@EnableWebSecurity
public class SecurityConfig {

    @Bean
    public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception {
        http
            .csrf(csrf -> csrf.disable())
            .authorizeHttpRequests(auth -> auth
                .requestMatchers("/api/v1/movies/**").permitAll()
                .requestMatchers("/api/v1/users/login").permitAll()
                .anyRequest().authenticated()
            )
            .sessionManagement(session -> session
                .sessionCreationPolicy(SessionCreationPolicy.STATELESS)
            );

        return http.build();
    }

    @Bean
    public AuthenticationManager authenticationManager(
            AuthenticationConfiguration config) throws Exception {
        return config.getAuthenticationManager();
    }
}

The functional change is minimal — the same security rules apply. The structural change is significant: instead of extending an abstract class and overriding methods, the configuration is composed through beans with a fluent lambda-based API. The new pattern is cleaner, more testable, and aligns with Spring's component model more naturally.

Two specific API changes worth noting:

antMatchers() → requestMatchers() — The method rename is straightforward but easy to miss because both compile without error in certain configurations; only the runtime behaviour differs.

authorizeRequests() → authorizeHttpRequests() — This change has security implications beyond naming. authorizeHttpRequests() uses the newer AuthorizationManager API which short-circuits earlier in the request processing chain and is more consistent in its behaviour across different dispatcher types.

Step 5: JWT Library Migration

The jjwt library had its own breaking change to address. io.jsonwebtoken:jjwt@0.9.1 — which Snyk gave a priority score of 889 and found 58 fixable issues in — underwent a major API restructuring between 0.9.x and 0.12.x.

Before (jjwt 0.9.1 API):

String token = Jwts.builder()
    .setSubject(userId)
    .setIssuedAt(new Date())
    .setExpiration(expiration)
    .signWith(SignatureAlgorithm.HS256, secret)
    .compact();

Claims claims = Jwts.parser()
    .setSigningKey(secret)
    .parseClaimsJws(token)
    .getBody();

After (jjwt 0.12.0 API):

String token = Jwts.builder()
    .subject(userId)
    .issuedAt(new Date())
    .expiration(expiration)
    .signWith(secretKey, Jwts.SIG.HS256)
    .compact();

Claims claims = Jwts.parser()
    .verifyWith(secretKey)
    .build()
    .parseSignedClaims(token)
    .getPayload();

The key change — beyond the fluent API restructuring — is how the signing key is handled. jjwt 0.9.1 accepted a raw String as a signing key, which is a known security weakness. A short or low-entropy string could be brute-forced if an attacker obtained a token. jjwt 0.12.0 requires a proper SecretKey object, which enforces minimum key length requirements at the API level.

// 0.9.1 — accepts any string, no validation
.signWith(SignatureAlgorithm.HS256, "weak")

// 0.12.0 — requires proper SecretKey, enforces minimum length
SecretKey key = Keys.hmacShaKeyFor(
    Decoders.BASE64.decode(base64EncodedSecret)
);
.signWith(key, Jwts.SIG.HS256)

This is an example of a library upgrade that isn't just a security patch — it's a security design improvement. The new API makes it harder to write insecure code, not just patching a specific vulnerability.

What I Deliberately Kept

Not everything needed to change.

The DAO layer. The MongoDB operations in MovieDao and UserDao use the Java driver directly with proper parameterised queries. The code is correct, readable, and doesn't need to be rewritten just because the framework version changed. Unnecessary refactoring introduces risk without benefit.

The API structure. The REST endpoint design in MoviesController and UsersController is sound. RESTful conventions, appropriate HTTP status codes, clear URL structure. These don't need to change to fix security issues.

The test suite. The JUnit 5 tests — updated to junit-jupiter-api@5.10.x from 5.1.0 — largely passed after the namespace migration. Keeping the tests as close to their original form as possible gave me confidence that the modernisation hadn't changed the application's behaviour.

The guiding principle: change what needs to change for security and maintainability. Don't rewrite what doesn't need to be rewritten.

The Modernisation Diff — What Actually Changed

Summarising the changes as a before/after:

Area	Before	After
Java version	1.8 (Java 8)	17
Spring Boot	2.0.3–2.0.4 (manually pinned)	3.2.5 (BOM managed)
Spring Framework	5.0.7 (manually pinned)	6.1.x (BOM managed)
jjwt	0.9.1	0.12.0
Security config	`WebSecurityConfigurerAdapter`	`SecurityFilterChain` bean
Namespace	`javax.*`	`jakarta.*`
Secrets	Hardcoded in properties file	Environment variables
Dependency versions	8 manually pinned	BOM managed

What the Modernisation Did to the Snyk Results

Running Snyk after the modernisation — before doing any targeted vulnerability remediation — already moved the numbers significantly.

The BOM upgrade to Spring Boot 3.2.5 automatically resolved a large portion of the Spring-related CVEs because the BOM pulled in patched versions of Spring Framework, Tomcat, and Jackson. The jjwt upgrade to 0.12.0 cleared all 58 of its fixable issues in a single version change.

I'll show the full before/after numbers in article 6. For now, the preview: the modernisation alone — without any targeted CVE remediation — reduced the total finding count substantially. This illustrates an important point about legacy Java dependency management: often the most effective security intervention isn't patching individual CVEs, it's getting the project onto a supported version of its primary framework and letting the framework's dependency management do the heavy lifting.

The modernised repository is at github.com/pgmpofu/mflix on the modernised branch.

Next up: the full unfiltered Snyk results — a deeper look at the most significant findings, what each vulnerability actually enables an attacker to do, and why the RCE in spring-beans deserved the attention it got.

I Dusted Off a 6-Year-Old Java Project and Ran Snyk Against It — Here's What I Found

Patience Mpofu — Mon, 18 May 2026 14:24:28 +0000

The README said "implementing security best practices."

That line has been sitting in the pgmpofu/mflix repository since 2019. A MongoDB-backed movie browsing application with user registration, authentication, JWT-based sessions, and full CRUD operations. Built as part of a MongoDB University course. Described, in my own words, as implementing security best practices.

I ran Snyk against it last week.

188 vulnerabilities. 10 Critical. 99 High. 59 Medium. 20 Low.

Every single one fixable. None of them there when I wrote that README.

This is the first article in a series about what happens when you apply modern software composition analysis to a Java project that hasn't been touched in six years — what Snyk found, what I fixed, what I chose not to fix, and what the before-and-after security posture actually looks like in measurable terms.

What the Project Is

MFlix is a Spring Boot Java application backed by MongoDB. The core functionality:

Movie search — basic and complex queries against a MongoDB collection
User registration and authentication
JWT-based session management using io.jsonwebtoken:jjwt
Comment posting on movie entries
Analytical reporting against the movie dataset
Spring Security for access control It's not a toy. It has real authentication flows, real database operations, and real dependency complexity. The pom.xml has eight direct dependencies spanning Spring Boot, Spring Security, the MongoDB Java driver, and the JWT library.

That dependency tree is where the story starts.

The Dependency Snapshot — What Was In the pom.xml

Before running anything, here's exactly what the project declared as of the last commit in 2019:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>2.0.3.RELEASE</version>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
    <version>2.0.4.RELEASE</version>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot</artifactId>
    <version>2.0.4.RELEASE</version>
</dependency>

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-context</artifactId>
    <version>5.0.7.RELEASE</version>
</dependency>

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-core</artifactId>
    <version>5.0.7.RELEASE</version>
</dependency>

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-web</artifactId>
    <version>5.0.7.RELEASE</version>
</dependency>

<dependency>
    <groupId>io.jsonwebtoken</groupId>
    <artifactId>jjwt</artifactId>
    <version>0.9.1</version>
</dependency>

<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongodb-driver-sync</artifactId>
    <version>3.9.1</version>
</dependency>

Eight direct dependencies. All declared at versions that were current in mid-2018 to mid-2019. All untouched since.

The Java compiler target is 1.8 — Java 8, which reached end of life for free Oracle support in January 2019. The project has been running on a deprecated runtime configuration since approximately the month it was committed.

Running the Scan

Setup is straightforward. With Snyk installed and authenticated:

cd mflix
snyk test --all-projects

Snyk reads the pom.xml, resolves the full dependency tree including transitive dependencies, and cross-references every package version against its vulnerability database.

The result appeared in seconds. The dashboard view told the story immediately:

10 Critical · 99 High · 59 Medium · 20 Low

Total: 188 fixable vulnerabilities. Zero with no supported fix.

That last number — zero unfixable — is actually significant. Every single vulnerability Snyk found has a known fix available. I'll come back to what that means for the remediation strategy.

The Finding That Stopped Me Cold

Before getting into the full breakdown, one finding deserves immediate attention.

org.springframework:spring-context@5.0.7.RELEASE — Remote Code Execution. CVSS 9.8. Priority Score 919.

The vulnerability is in spring-beans@5.0.7.RELEASE via CWE-94 — improper control of code generation. An attacker who can reach the application can, under certain conditions, execute arbitrary code on the server.

CVSS 9.8. That's not a theoretical risk category. That's one point below the maximum possible score. That's the kind of finding that, in a production system, triggers an emergency change control process at 11pm on a Friday.

MFlix was never deployed to production. The attack surface was always zero. But the finding is real — the vulnerability exists in the version of spring-beans that ships with spring-context@5.0.7, and it would be exploitable if the application were running and accessible.

This is the finding that "security best practices" in the README was supposed to prevent. It didn't — not because of negligence, but because security best practices in 2019 didn't include the CVE that was disclosed after the code was written.

The Full Breakdown by Dependency

Here's what Snyk found grouped by the dependency it originated from, with priority scores and headline vulnerability types:

Critical Severity (Priority Score 919)

org.springframework:spring-web@5.0.7.RELEASE
11 direct issues, 8 transitive issues. The highest-priority dependency in the scan.

Remote Code Execution — CWE-94, CVSS 9.8 (via spring-beans)
Privilege Escalation — CWE-264, CVSS 4.4
Improper Input Validation — CWE-20, CVSS 8.6
Reflected File Download — CWE-494, CVSS 8.0
Denial of Service — CWE-400, CVSS 3.7 org.springframework:spring-context@5.0.7.RELEASE 2 direct issues, 11 transitive issues.
Remote Code Execution — CWE-94, CVSS 9.8 (via spring-beans)
Relative Path Traversal — CWE-23, CVSS 8.2
Incorrect Authorization — CWE-863, CVSS 8.7 org.springframework.boot:spring-boot-starter-security@2.0.4.RELEASE 37 transitive issues.
Authorization Bypass — CWE-285, CVSS 8.2
Reflected File Download — CWE-494, CVSS 8.0
Improper Input Validation — CWE-20, CVSS 8.6 org.springframework.boot:spring-boot-starter-web@2.0.3.RELEASE 206 transitive issues — the single dependency pulling in the most downstream vulnerabilities.
Insecure Defaults via tomcat-embed-core@8.5.31 — CWE-453, CVSS 9.8
Deserialization of Untrusted Data via jackson-databind@2.9.6 — CWE-502, CVSS 9.2
Session Fixation via tomcat-embed-core — CWE-384, CVSS 3.1
Cross-Site Scripting via tomcat-embed-core — CWE-79, CVSS 3.5 io.jsonwebtoken:jjwt@0.9.1 — Priority Score 889 63 transitive issues, 58 fixable. All fixed in a single upgrade to version 0.12.0.
Deserialization of Untrusted Data via jackson-databind@2.9.6 — CWE-502, CVSS 9.2
Allocation of Resources Without Limits via jackson-core — CWE-770, CVSS 8.x ### Critical — Certificate Validation

org.springframework.boot:spring-boot-autoconfigure@2.0.3.RELEASE

Improper Validation of Certificate with Host Mismatch — CWE-297, CVSS 9.3 This one matters specifically because MFlix handles user authentication. A TLS certificate validation bypass in the autoconfigure layer means a connection claiming to be a trusted service could potentially be impersonated without detection.

High Severity (Priority Score 649)

org.springframework.boot:spring-boot@2.0.4.RELEASE
4 direct issues, 7 transitive.

Insecure Temporary File — CWE-377, CVSS 7.8
Incorrect Authorization — CWE-863, CVSS 8.7 org.springframework:spring-core@5.0.7.RELEASE 5 direct issues.
Incorrect Authorization — CWE-863, CVSS 8.7
Improper Case Sensitivity Handling — CWE-178, CVSS 2.3 ### Medium Severity

org.mongodb:mongodb-driver-sync@3.9.1
1 direct issue.

Man-in-the-Middle — CWE-300, CVSS 6.4 The MongoDB driver finding is particularly relevant for this application. MFlix connects to a MongoDB Atlas cluster. A MitM vulnerability in the driver layer means the connection between the application and the database could potentially be intercepted.

The Transitive Dependency Problem

The number that tells the real story of legacy Java dependency management is this one:

spring-boot-starter-web@2.0.3 — 206 transitive issues.

One line in the pom.xml. One version declaration. 206 downstream vulnerabilities pulled in through the dependency chain, in packages the application never explicitly imported and whose version numbers never appeared in the build file.

This is the fundamental challenge of Software Composition Analysis in Java projects. The pom.xml has eight dependencies. The actual dependency graph has dozens of packages. Each of those packages has its own version, its own CVE history, its own patch cadence.

A developer in 2018 who wrote spring-boot-starter-web@2.0.3 made a single decision. That decision pulled in a specific version of Tomcat, a specific version of Jackson, a specific version of Spring's core libraries — all of which have accumulated vulnerabilities over six years. None of those vulnerabilities were visible at the point of the original decision.

This is why dependency scanning exists. The alternative — manually tracking CVE disclosures across every transitive dependency in your build graph — is not a realistic approach for any team at any scale.

The Exploit Maturity Breakdown

Not all 188 findings carry the same actual risk. Snyk's exploit maturity classification tells a more nuanced story:

Maturity Level	Count
Mature exploits (working exploit code exists)	8
Proof-of-concept exploits	43
No known exploit	137

Eight findings have working, publicly available exploit code. These are the ones that require the most urgent remediation — not because the vulnerability is necessarily more severe than others, but because the barrier to exploitation is effectively zero. Anyone with the exploit code and network access to the application could use it.

The 137 findings with no known exploit are real vulnerabilities — they're in the CVE database, they have CVSS scores, Snyk flags them correctly — but the practical attack risk is lower because exploitation requires custom effort rather than running a public tool.

This breakdown becomes critical for article 4, where I'll explain which findings I remediated, which I suppressed, and why the exploit maturity classification was the primary factor in that decision.

What "100% Fixable" Actually Means

One number that surprised me when I first saw the Snyk output: 188 fixable, 0 with no supported fix.

That's unusually clean. In my experience scanning production Java codebases at work, there are almost always some findings in the "no supported fix" category — typically because a vulnerable package has no patched version available, or because the fix requires changes that would break the API the application depends on.

MFlix having 100% fixable findings is partly a function of how old the dependencies are. Six years is a long time. Every major vulnerability that exists in Spring Boot 2.0.x, Spring 5.0.x, and jjwt 0.9.1 has had years to be patched in subsequent versions. The fix exists — I just have to apply it.

The question is how straightforward "applying the fix" actually is. Some of these are minor version bumps. Others are major version upgrades — Snyk recommends going from spring-boot-starter-web@2.0.3 to 2.0.6 for some fixes and all the way to 3.x for others. Major version upgrades in Spring Boot involve breaking API changes that require code modifications, not just version bumps.

That complexity is what articles 4 and 5 are about. The 100% fixability rate is the theoretical ceiling. The practical remediation story is considerably more interesting.

The Irony of the README

I want to return to that line. "Implementing security best practices."

It was accurate in 2019. I used parameterised queries for MongoDB operations. I implemented JWT authentication correctly for the era. I used Spring Security for access control. I followed the MongoDB University course's security guidance.

None of that is what Snyk found. What Snyk found is a different category of security problem entirely — not vulnerabilities in the code I wrote, but vulnerabilities in the dependencies I imported. The distinction matters because it reveals a gap in how most developers think about application security.

When developers think about writing secure code, they think about SQL injection, authentication flows, authorisation checks, input validation. These are code-level concerns. A skilled developer can learn them, apply them consistently, and write code that is largely free of them.

Software composition vulnerabilities are different. They accumulate silently. They appear in packages you didn't write and may never read. They arrive via CVE disclosures months or years after you made your dependency choices. They require an ongoing process — not a one-time skill — to manage.

The "security best practices" that prevent code-level vulnerabilities are largely different from the "security best practices" that prevent composition vulnerabilities. Both matter. Until I ran this scan, I was only thinking about one of them.

What Comes Next

Over the next six articles in this series, I'll document:

Article 2 — Modernising the project structure: wrapping the legacy code in a current Spring Boot shell, what changed, and what had to change before Snyk could even give me a useful remediation path.

Article 3 — The full unfiltered Snyk results: a deeper dive into the most significant findings with full CVE context, what each vulnerability actually enables an attacker to do, and which ones matter most for an application with user authentication and database access.

Article 4 — Why I suppressed some findings and fixed others: the risk assessment framework, the role of exploit maturity in prioritisation, and the findings I made a documented decision to accept.

Article 5 — The remediation work itself: the easy version bumps, the breaking changes that required code modification, and the one dependency upgrade that took four attempts to get right.

Article 6 — Before and after metrics: what changed, how to measure security posture improvement, and what the numbers look like when you present them to an engineering team.

Article 7 — Using AI to assist with remediation: what worked, what didn't, and the difference between AI suggestions and Snyk recommendations on the same vulnerabilities.

The repository is at github.com/pgmpofu/mflix. The pom.xml in the current state is exactly as it was in 2019. The Snyk findings are real. The remediation work is ongoing.

Next article: modernising the project — what a 2019 Spring Boot structure looks like compared to what a current one should look like, and what had to change before the security work could begin in earnest.

I Ran My ML Secrets Detector Against My Own Repositories — Here's What It Found

Patience Mpofu — Sat, 16 May 2026 03:00:54 +0000

here's a moment every security tool builder eventually faces.

You've built the scanner. You've written the rules. You've validated it against synthetic test cases and contrived examples. And then you point it at your own code — the repositories you've actually written, committed, and pushed over years of real development work.

That moment is humbling.

I ran my ML secrets detector against every personal repository I own — 11 repositories across Python, Java, Node.js, and Kotlin projects accumulated over several years of portfolio building and side projects. I'm documenting the results honestly: what it found, what was real, what was a false positive, and what the numbers actually looked like.

The Setup

Before running, I configured the scan for comprehensive coverage:

# Full repository scan including git history
python main.py scan ./repos/ \
  --include-history \
  --threshold 0.65 \
  --format all \
  --output ./scan-results/

A threshold of 0.65 rather than the default 0.70 — I wanted to see more findings, including ones that would normally sit just below the reporting threshold. For an audit of your own code, more signal is better than less.

The --include-history flag scans not just the current working tree but every commit in git history. This is the mode that makes people nervous. Whatever got committed and "fixed" later is still in the history. It's still accessible. It still needs to be addressed.

Repositories scanned: 11

Total commits scanned: 847

Total files scanned: 2,341

Scan duration: 4 minutes 23 seconds

The Raw Numbers

Severity	Findings	Confirmed Real	False Positives	False Positive Rate
CRITICAL	7	6	1	14%
HIGH	19	11	8	42%
MEDIUM	31	9	22	71%
Total	57	26	31	54%

A few things to unpack here.

The CRITICAL findings had a 14% false positive rate — one in seven was benign. That's roughly what I expected based on the test set results. The one false positive was a 32-character hex string in a variable named encryption_mode — the word "encryption" pushed the key name score high, but the value was actually a configuration mode identifier, not a key.

The HIGH findings had a 42% false positive rate. Higher than I'd like, but consistent with the nature of HIGH confidence findings — they're cases where the evidence is strong but not overwhelming. Most of the false positives in this tier were package integrity hashes in older package-lock.json files that hadn't been added to the skip list yet.

The MEDIUM findings had a 71% false positive rate. This is expected and by design. MEDIUM findings are prompts for human review, not automatic defects. Most were generic high-entropy strings in configuration files where the variable names were moderately suspicious but the values were benign.

The overall 54% false positive rate sounds alarming until you account for the lower threshold (0.65 vs. default 0.70) and the MEDIUM tier. At the default threshold, the false positive rate drops to approximately 28% — closer to the test set results.

The Real Findings: What Was Actually There

Of the 26 confirmed real findings, here's what they were. I've anonymised the specific values but documented the pattern honestly.

Finding 1–3: Test Credentials That Never Left Test Files (But Were Still Committed)

Three findings were test database credentials in integration test configuration files:

# tests/integration/test_database.py (2021 commit)
TEST_DB_PASSWORD = "integration_test_password_2021"
TEST_DB_URL = "postgresql://testuser:local_test_pass@localhost/testdb"

These were intentionally "fake" credentials — values I created specifically for local testing. But they were committed to a public repository. The classifier flagged them at 87% and 91% confidence respectively.

Are these real vulnerabilities? Technically no — a local test database password with no external access isn't a secret in the traditional sense. But they taught me something: even intentional test credentials get flagged, which means either the suppression annotation should have been there from the start, or the test configuration should have used environment variables even for local test values.

The lesson isn't that the scanner was wrong. It's that "this is only for testing" is not a reason to skip secure credential handling.

Finding 4: An Actual JWT Secret (History)

This one made my stomach drop.

CRITICAL (97%) · src/auth/config.py:23
jwt_secret = "my-jwt-signing-secret-change-this"
↳ History: commit a3f8b2c · 2020-03-14

I found a hardcoded JWT signing secret in a 2020 commit to a project that I had since "fixed" by moving to environment variables. The fix was in the current code. The secret was still in git history.

The value itself — "my-jwt-signing-secret-change-this" — is one of those values that developers write with the intention of replacing it before going anywhere near production. The comment is literally in the name. But it got committed, and committed things live in git history forever unless you rewrite it.

The project was never deployed to production with this value. But it was a public repository. Anyone who cloned it at any point in 2020 has this value. The theoretical attack surface was real even if the practical exploitation probability was low.

What I did: Rewrote the commit history using git filter-branch to remove the file containing the secret, then force-pushed. I also added a .gitignore entry for config.py files and a pre-commit hook (obviously) to catch this pattern in future.

Finding 5–8: API Keys in Old Test Scripts

Four findings were API keys in utility scripts I'd written to test integrations:

# scripts/test_sendgrid.py (2019 commit)
SENDGRID_API_KEY = "SG.abc123...xyz789"  # key has been rotated

These were real API keys at the time of commit. I confirmed with the respective providers that all four had been rotated or the accounts had been closed — so the operational risk was zero. But they were real keys that were real secrets when committed.

This is the most common pattern in real credential exposure incidents: keys that were live at the time of commit, rotated after discovery, but remain in history as evidence of the exposure. The key rotation closes the operational risk but doesn't erase the fact of the exposure.

What I did: Rotated anything still active (none were), documented the historical exposure, and rewrote history for the two repositories where the keys were in active-looking scripts. For older repositories where the scripts were clearly abandoned, I left the history intact and noted the exposure in the repository README.

Finding 9–11: Internal Service URLs With Embedded Credentials

Three findings were database and service connection strings:

# config/database.py (2022 commit)
DATABASE_URL = "postgresql://admin:password123@internal-host:5432/appdb"

None of these were production credentials — they were development environment connection strings pointing to local or development hosts. But the pattern is exactly what you see in production credential exposures, and the scanner correctly identified them as high confidence.

Two were for hosts that no longer exist. One was for a development Postgres instance that still exists but has no external network access. The operational risk was low; the pattern risk was real.

Finding 12: A Private Key Fragment in a README

The most surprising finding:

CRITICAL (99%) · README.md:47
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA...

A README containing an example private key that I'd generated specifically to demonstrate what a private key looks like in documentation. It was a real RSA private key — not a truncated fake — but generated purely for documentation purposes and never associated with any system.

The scanner correctly flagged it. The private key has never been used for anything. But it's a valid RSA private key that anyone could theoretically use to claim they found something in my repository.

What I did: Replaced the real private key in the README with a clearly truncated fake:

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA[EXAMPLE - NOT A REAL KEY]...
-----END RSA PRIVATE KEY-----

If you're writing documentation that shows what a private key looks like, never use a real generated key. Generate a fake-looking placeholder instead.

Findings 13–26: Various Confirmed Vulnerabilities

The remaining 14 confirmed findings were a mix of:

Hardcoded passwords in older Java projects using Spring with properties files committed directly
OAuth client secrets in mobile app prototype code from 2018–2019
Slack webhook URLs (which are effectively secrets — anyone with the URL can post to your channel)
Internal service tokens from a project that has since been decommissioned All were historical, all have been rotated or decommissioned. All are now either suppressed with justification or removed from history.

The False Positives: What Triggered Them

The 31 false positives clustered into four categories:

Category 1: Package Lock File Hashes (12 findings)

The most numerous false positive source. package-lock.json files contain SHA-512 integrity hashes for every dependency:

"integrity": "sha512-abc123def456..."

These are high-entropy strings in a file that often has keys named integrity. The key name risk for "integrity" is 0.0 in my vocabulary, which should push these below threshold — and at the default 0.70 threshold, most don't appear. At 0.65, several edge cases squeaked through.

Fix: Added package-lock.json, yarn.lock, and *.lock to the global skip list.

Category 2: UUID Values With Moderately Sensitive Variable Names (8 findings)

session_token = "550e8400-e29b-41d4-a716-446655440000"
auth_correlation_id = "7c9b2de1-3f4a-8b5c-2d1e-9f8a7b6c5d4e"

"Session token" and "auth correlation ID" score moderately high on key name risk. UUIDs have moderate entropy. The combination pushed these above 0.65.

Fix: Added correlation_id, session_id, request_id, and similar terms to the explicitly benign vocabulary with a score of 0.0.

Category 3: Example Values in Documentation (7 findings)

Markdown files and READMEs containing example code snippets:

Set your API key:

python
API_KEY = "your-api-key-here"

python

"your-api-key-here" is low entropy and obviously a placeholder. The scanner correctly passes it. But other examples used more realistic-looking values:

API_KEY = "aK9mP2xL8vR3qT7nY5wZ1bJ4cH6dF0eI"

The variable name is high risk, the entropy is high, and no pattern matches — 78% confidence. False positive, but an understandable one.

Fix: Added .md and .rst files to a lower-confidence mode (threshold raised to 0.90 for documentation files) rather than skipping them entirely — real secrets do appear in committed documentation.

Category 4: High-Entropy Configuration Values (4 findings)

Configuration values that are long and random-looking but aren't secrets:

CACHE_KEY_PREFIX = "app_v2_prod_cache_2024_r3f8b2"
CORRELATION_HEADER = "X-Request-ID-v2-production-shard-3"

These are deterministic, human-readable configuration values that happen to be long and contain alphanumeric characters. Low false positive risk in most codebases but they appeared in mine.

Fix: These are the hardest category to address systematically. The suppression annotation is the right tool — add # secrets-ignore with a note that the value is a configuration constant.

What the History Scan Revealed That the Current Scan Didn't

Scanning history found 9 findings that don't appear in the current codebase — secrets that have been "fixed" but remain in git history. This is the most important capability of the history scanner and the most overlooked.

Findings in current code: 17
Findings only in history: 9
Total unique findings: 26

The 9 historical-only findings represent credentials that a developer committed, noticed (or was told about), and removed from the current code — but never removed from history. From a security perspective, these are live exposures. The credential exists in a public repository's history. Anyone who cloned the repository at any point has it.

The remediation for historical findings is harder than current findings:

Option 1: Rotate the credential. If the credential is still active, rotate it immediately. The historical exposure is already done — rotation closes the operational risk.

Option 2: Rewrite git history. Using git filter-branch or the newer git filter-repo, you can rewrite history to remove the file or commit containing the secret. This requires force-pushing, which is disruptive if other people have cloned the repository.

Option 3: Make the repository private. If the repository is public and the historical exposure is significant, making it private while history is cleaned up is a reasonable interim step.

Option 4: Document and accept. For decommissioned systems and rotated credentials with no active risk, documenting the historical exposure in the repository README and marking the findings as suppressed is acceptable. Not ideal, but pragmatic for old secrets with no active attack surface.

The Honest Assessment

Running the scanner against my own repositories was a genuinely useful exercise that I'd recommend to anyone building security tooling.

What worked well:

CRITICAL findings were high precision — 6 out of 7 were real
The history scanner found things I'd genuinely forgotten about
The scan was fast enough that 11 repositories in 4 minutes felt reasonable
The output was actionable — I knew exactly what to fix and where What needs improvement:
The HIGH finding false positive rate of 42% is too high for a production tool targeting real organisations. It would erode trust in a team context
The package-lock.json skip list should have been in place from the start — that's a known false positive source that I didn't anticipate fully
The threshold calibration needs work — 0.70 feels too conservative for CRITICAL findings and not conservative enough for HIGH findings The finding that most surprised me: The JWT secret in history. Not because finding it surprised me — that's exactly what the history scanner is for. Because I had genuinely forgotten it was there. I "fixed" the issue in 2020 by moving to environment variables and closed the mental file. The history scanner reopened it.

That's the value proposition of history scanning in one sentence: it finds the things you fixed but didn't actually fix.

What to Do If You Want to Run This Against Your Own Repos

Start with current code only, at the default threshold:

python main.py scan ./your-repo --threshold 0.70 --format terminal

Triage every CRITICAL finding before looking at anything else. Then work through HIGH. Treat MEDIUM as informational unless something catches your eye.

Once you've cleaned up the current state, run the history scan:

python main.py scan ./your-repo --include-history --threshold 0.70

Be prepared for findings you've forgotten about. Have a decision framework ready for each one: rotate, rewrite history, or document and accept.

The scan itself is the easy part. The remediation decisions are where the real work is.

The full tool, including the history scanner and all configuration options, is at github.com/pgmpofu/secrets-detector.

If you run it against your own repositories and find something interesting — or find a false positive pattern I haven't handled — open an issue. The tool gets better from real-world feedback, and real-world feedback only comes from people running it on real code.

Blocking Secrets Before They Hit the Repository: Building a Pre-Commit Hook With ML

Patience Mpofu — Sat, 16 May 2026 02:54:18 +0000

here are two places you can catch an exposed secret.

After it's in the repository — in a CI/CD pipeline scan, a periodic audit, or a breach notification from a security researcher who found it in your public history. Or before it ever gets there — at the moment of git commit, when the developer is still at their keyboard and the fix takes thirty seconds.

The second option is better in every dimension. Earlier detection means lower remediation cost. A blocked commit means no credential rotation required, no incident response, no git history rewriting. The developer who gets stopped at commit understands immediately what they did and why — the context is fresh, the fix is obvious.

The challenge is UX.

A pre-commit hook that's too slow gets disabled. A hook that generates too many false positives gets disabled. A hook that doesn't explain itself gets disabled and complained about on Slack. A hook that developers trust — that's fast, precise, and tells them exactly what it found and why — stays enabled and actually prevents exposures.

This article is about building a pre-commit hook that developers will actually leave on.

What the Hook Needs to Do

Before writing a line of code, I defined what a good pre-commit secrets hook looks like from the developer's perspective.

Speed. The hook runs on every commit. If it adds more than two or three seconds, developers will notice and resent it. On a typical feature branch with a handful of changed files, the scan needs to complete in under two seconds.

Scope. The hook should scan staged content — only the files about to be committed — not the entire repository. Scanning everything on every commit is unnecessary and slow.

Signal clarity. When the hook blocks a commit, the developer needs to know immediately: which file, which line, what variable, why it was flagged. "Secret detected" with no context is useless. "HIGH confidence (94%): api_key = "sk-proj-abc123..." in config/settings.py line 47 — matches OpenAI key format" is actionable.

Suppression path. Developers need a documented, low-friction way to handle false positives. The hook can't be a hard wall with no escape — that's how hooks get disabled entirely.

Non-destructive. The hook never modifies files. It either passes silently or blocks and explains. That's it.

Architecture: Scanning Staged Content

The first architectural decision is what to scan. There are two options:

Option A: Scan the working tree — the files as they currently exist on disk, including unstaged changes.

Option B: Scan the staged content — exactly what git diff --cached shows, which is what will actually be committed.

Option B is correct. Scanning the working tree means flagging things the developer hasn't committed and may never intend to commit. That's noise. Scanning staged content means flagging exactly what's about to enter the repository — which is the precise intervention point.

def get_staged_content() -> dict[str, str]:
    """Get the staged content for all modified/added files."""
    staged_files = {}

    # Get list of staged files
    result = subprocess.run(
        ["git", "diff", "--cached", "--name-only", "--diff-filter=ACM"],
        capture_output=True, text=True
    )

    filenames = result.stdout.strip().split('\n')

    for filename in filenames:
        if not filename:
            continue

        # Get staged content (not working tree content)
        content_result = subprocess.run(
            ["git", "show", f":{filename}"],
            capture_output=True, text=True
        )

        if content_result.returncode == 0:
            staged_files[filename] = content_result.content_result.stdout

    return staged_files

The --diff-filter=ACM flag limits to Added, Copied, and Modified files — not deletions. Scanning deleted file content would generate findings for secrets that are being removed, which is the wrong direction.

The Scan Loop: From Staged Content to Findings

The hook extracts string literal assignments from each staged file and passes them through the ML classifier:

def scan_staged_files(staged_content: dict[str, str], threshold: float = 0.7):
    findings = []

    for filepath, content in staged_content.items():
        # Skip binary files, lock files, and known safe extensions
        if should_skip_file(filepath):
            continue

        lines = content.split('\n')

        for line_num, line in enumerate(lines, 1):
            # Skip lines with suppression annotation
            if '# secrets-ignore' in line or '# nosec' in line:
                continue

            # Extract (key_name, value) pairs from string assignments
            assignments = extract_string_assignments(line)

            for key_name, value in assignments:
                if len(value) < 8:  # Skip very short strings
                    continue

                features = extract_features(value, key_name)
                confidence = model.predict_proba([features])[0][1]

                if confidence >= threshold:
                    findings.append({
                        "file": filepath,
                        "line": line_num,
                        "key_name": key_name,
                        "value_preview": value[:20] + "..." if len(value) > 20 else value,
                        "confidence": confidence,
                        "severity": confidence_to_severity(confidence)
                    })

    return findings

A few implementation details worth highlighting:

should_skip_file() excludes file types that generate systematic false positives: package-lock.json, yarn.lock, *.sum (Go module checksums), *.min.js (minified JavaScript), binary file extensions, and image files. These are maintained in a skip list rather than being hardcoded into the scan logic, so teams can extend it for their specific false positive patterns.

Value preview truncation. The finding reports only the first 20 characters of the flagged value, with ... truncation. Showing the full value in terminal output creates a secondary exposure — if someone is screen sharing when the hook fires, the secret shouldn't appear in full in the terminal.

Minimum length of 8. Strings shorter than 8 characters are almost never secrets. This eliminates a class of false positives from short configuration values and reduces scan time on files with many string literals.

The Output: Making Findings Actionable

The most important UX decision in the hook is what to show when a finding is blocked. I went through four iterations of the output format before settling on one that developers responded well to.

Iteration 1 (too terse):

BLOCKED: Secret detected in config/settings.py

Developers immediately asked: "What secret? Where exactly? What should I do?"

Iteration 2 (better but still vague):

BLOCKED: Possible secret at config/settings.py:47

Still not enough context. Developers had to open the file and count to line 47 to understand what was flagged.

Iteration 3 (too verbose):

[SECRETS DETECTOR] 
==========================================
COMMIT BLOCKED — POTENTIAL SECRET DETECTED
==========================================
File: config/settings.py
Line: 47
Variable: api_key
Value (truncated): sk-proj-abc123...
Confidence: 94%
Severity: CRITICAL
Matched Pattern: OpenAI API key format (sk-proj-*)
Feature contributions:
  - key_name_risk: 0.90 (HIGH)
  - shannon_entropy: 5.82 (HIGH)
  - pattern_openai_key: 1.00 (MATCH)
  - repetition_ratio: 0.94 (HIGH)

To suppress this finding, add '# secrets-ignore' to line 47
To bypass this check entirely (NOT RECOMMENDED): git commit --no-verify
==========================================

This is technically complete but overwhelming. Developers in flow state don't want to read a report. They want to know: what, where, what to do.

Final version (what shipped):

🔴 Secrets Detector — Commit Blocked

  CRITICAL (94%) · config/settings.py:47
  api_key = "sk-proj-abc123..."
  ↳ Matches OpenAI key format · High entropy · Sensitive variable name

  To suppress false positive: add  # secrets-ignore  to line 47
  To use env vars instead:    export API_KEY="your-key"
                              then  api_key = os.environ["API_KEY"]

1 finding blocked this commit. Fix the issue or suppress with justification.

The final format answers the three questions developers actually have in two seconds of reading: what is it (OpenAI key), where is it (file and line), what do I do (env var example or suppression). The feature contributions are available in verbose mode (--verbose) but don't appear by default.

The emoji is intentional. 🔴 provides an immediate visual signal in terminals that support it, and degrades gracefully to plain text in terminals that don't.

Handling Multiple Findings

When multiple findings exist, the output stacks them:

🔴 Secrets Detector — Commit Blocked

  CRITICAL (96%) · src/database.py:12
  DB_PASSWORD = "Tr0ub4dor&3"
  ↳ High-risk variable name · Matches human-chosen password pattern

  HIGH (78%) · src/config.py:34
  internal_token = "prod-service-backend-2019"
  ↳ Moderate-risk variable name · Low entropy but sensitive context

2 findings blocked this commit. Fix all issues before committing.

Findings are sorted by confidence descending — the most certain findings appear first, which is where the developer's attention should go.

The commit is blocked if any finding exceeds the threshold, not just the highest-confidence one. A batch of MEDIUM confidence findings is still a blocked commit. If all findings are genuine false positives, they should all be suppressed with justification — not just the top one.

The Suppression UX

The suppression path needs to be low-friction but not invisible. If suppressing a false positive is too hard, developers will use git commit --no-verify to bypass the hook entirely — which defeats the purpose.

The designed flow:

# Developer encounters a false positive:
# file_integrity_hash = "d8e8fca2dc0f896fd7cb4cb0031ba249"  ← flagged

# They add the annotation with a justification:
# MD5 hash for file integrity check only — not a credential
file_integrity_hash = "d8e8fca2dc0f896fd7cb4cb0031ba249"  # secrets-ignore

# Commit proceeds normally on next attempt

The # secrets-ignore annotation is visible in code review. A reviewer can see that a suppression was added and evaluate whether the justification is reasonable. This is the governance layer — suppressions can't happen silently.

The hook also respects the SECRETS_DETECTOR_THRESHOLD environment variable, which allows individual developers to adjust their personal threshold without modifying shared configuration:

# Developer who wants to see more findings (lower threshold)
SECRETS_DETECTOR_THRESHOLD=0.55 git commit -m "wip"

# Developer who wants fewer false positives (higher threshold)
SECRETS_DETECTOR_THRESHOLD=0.85 git commit -m "feature: payment flow"

This flexibility matters for adoption. Some developers will want to see everything; others will want a tighter filter. Forcing everyone to the same threshold is a source of friction.

Installation: Making Setup Frictionless

A hook that's hard to install never gets installed. The setup needs to be one command:

# Using pre-commit framework (recommended)
pip install pre-commit
echo "repos:
- repo: https://github.com/pgmpofu/secrets-detector
  rev: v1.0.0
  hooks:
  - id: secrets-detector
    args: [--threshold, '0.7']" > .pre-commit-config.yaml
pre-commit install

Or manual installation for teams not using the pre-commit framework:

# Copy hook to git hooks directory
cp hooks/pre-commit .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

The pre-commit framework approach is preferable for teams because it version-pins the hook, makes it part of the repository configuration (.pre-commit-config.yaml is committed), and automatically installs on git clone for new team members. The manual approach works for individual use.

What Happens at `git commit --no-verify`

This is the escape hatch that can't be removed. Git's --no-verify flag bypasses all hooks, and there's nothing a hook can do to prevent it.

The right response to this is not technical — it's cultural and procedural.

In a team setting, git commit --no-verify should require a comment in the commit message explaining why the hook was bypassed. This can be enforced through CI/CD: a pipeline step that checks whether any commit in a PR used --no-verify and requires a justification in the commit message if so.

# In GitHub Actions
- name: Check for hook bypasses
  run: |
    git log --oneline origin/main..HEAD | while read line; do
      hash=$(echo $line | cut -d' ' -f1)
      msg=$(git log --format=%B -n 1 $hash)
      if git log --format=%B -n 1 $hash | grep -q "no-verify bypass"; then
        echo "Documented bypass found in $hash"
      fi
    done

The goal is to make --no-verify traceable, not to make it impossible. A developer in a genuine emergency who needs to commit right now and deal with the secret later should be able to do that — but there should be a record of the decision.

Measuring Hook Effectiveness

After the hook has been running for a few weeks, three metrics tell you whether it's working:

Bypass rate. What percentage of commits use --no-verify? A bypass rate above 10% suggests the hook is generating too many false positives or too much friction. Investigate which developers are bypassing most frequently and why.

Suppression rate. What percentage of findings are suppressed rather than fixed? High suppression rates indicate either noisy rules or developers treating suppression as the default response. Review suppressions in code review and push back on suppression-without-justification.

Secrets found in CI despite the hook. If your CI pipeline also runs a secrets scan and finds things the pre-commit hook didn't catch, those are false negatives worth understanding. Each one is an opportunity to improve the hook's coverage.

The hook is not a complete solution — it's the first line of defence. CI scanning is the second. Periodic full history scanning is the third. Each layer catches what the previous one misses.

The Broader Point: Shift Left Has a UX Requirement

"Shift left" — catching security issues earlier in the development lifecycle — is the right strategy. Every study on the economics of security defects confirms that earlier detection means lower remediation cost.

But shift left only works if the shifted controls are actually used. A pre-commit hook that developers disable after the first false positive has shifted nothing. A CI gate that gets bypassed in every release has shifted nothing.

The investment in UX — the careful output format, the clear suppression path, the fast scan, the explainable findings — is not cosmetic. It's what determines whether the security control actually operates or sits dormant in the repository while credentials quietly accumulate in git history.

Security controls that developers trust are security controls that get used. That's the only metric that matters.

The pre-commit hook implementation is in hooks/pre-commit at github.com/pgmpofu/secrets-detector.

Last article in the series: I ran the secrets detector against my own repositories — here's what it actually found, the false positives I encountered, and what the real-world numbers looked like.

Training on Synthetic Data: How to Build an ML Security Tool Without Touching Real Leaked Secrets

Patience Mpofu — Sat, 16 May 2026 02:51:27 +0000

Before I wrote a single line of model training code, I made a decision that constrained everything that followed.

I would not train on real leaked credentials.

The alternative was straightforward. GitHub's public commit history contains millions of accidentally committed secrets — API keys, passwords, connection strings, private keys — that have been scraped, indexed, and catalogued by security researchers. Datasets of this material exist. Using them as positive training examples would produce a model trained on exactly the kind of data it needs to recognise.

I chose not to do that. And the reasoning is more nuanced than "it felt wrong."

This article is about why I made that choice, how I built synthetic training data that avoids the problem, what the tradeoffs are, and what the broader principle is for anyone building ML security tooling.

Why Real Leaked Credentials Are a Problematic Training Source

The obvious objection to using real leaked secrets is ethical: those credentials belong to real people and organisations. Even if the data is technically public — visible in a GitHub commit, indexed by search engines — using it for commercial or portfolio purposes raises questions about consent and purpose.

But the ethical argument alone isn't the strongest one. The stronger arguments are practical.

Legal Ambiguity

The legal status of scraping and using publicly accessible but unintentionally published credentials is genuinely unclear across jurisdictions. In some interpretations of computer fraud and data protection law, accessing and storing leaked credentials — even for research purposes — could constitute unauthorised access to data or improper processing of personal information.

The GDPR position on this is particularly murky. Credentials are often linked to personal accounts. Processing personal data, even publicly accessible personal data, requires a lawful basis. "I needed it to train my model" is not a lawful basis.

I'm not a lawyer and this isn't legal advice. But I am someone building a tool I intend to put on GitHub with my name on it. "The legal status of my training data is unclear" is not a position I wanted to be in.

Data Quality Problems

Leaked credential datasets have severe quality problems that make them worse training data than they might appear.

Temporal distribution shift. Key formats change over time. GitHub PATs changed format in 2021 from a 40-character hex string to a structured format with a ghp_ prefix. AWS has introduced new key formats. An older leaked credentials dataset would train the model on formats that no longer exist while underrepresenting current formats.

Survivorship bias. The credentials that get scraped and catalogued are the ones that were detected and revoked. Harder-to-detect secrets — generically named variables, low-entropy human-chosen passwords — are systematically underrepresented in public leaked credential datasets precisely because they're harder to find.

Label noise. Not every string in a "leaked credentials" dataset is actually a sensitive credential. Test keys, example values, documentation snippets, and deliberately fake keys appear throughout. Cleaning a scraped dataset to get reliable labels is a substantial manual effort.

Negative example scarcity. A dataset of leaked credentials is purely positive examples. You still need high-quality negative examples — high-entropy strings that aren't secrets — to train a classifier that distinguishes secrets from benign values. These need to be generated separately anyway.

Synthetic data generation, done carefully, avoids all of these problems. You control the format distribution, the label quality, and the class balance precisely.

Reusability and Sharing

A tool trained on synthetic data can be shared freely. The training code can be published. The data generation methodology can be documented. Other researchers can reproduce, audit, and improve the approach.

A tool trained on scraped real credentials has a provenance problem the moment someone asks "where did your training data come from?" Publishing that training data would mean republishing the leaked credentials. Not publishing it means the model can't be fully reproduced or audited.

Reproducibility matters in security tooling specifically because trust matters. A secrets detector that you can't audit end-to-end is a secrets detector you're taking on faith.

How I Generated Synthetic Training Data

The synthetic data generator in trainer.py produces two classes of examples: secrets (label=1) and benign high-entropy strings (label=0).

Generating Positive Examples (Secrets)

For known-format secrets, I generate values that match the structural properties of real secrets without being real secrets:

def generate_aws_access_key():
    """Generate synthetic AWS access key format"""
    chars = string.ascii_uppercase + string.digits
    suffix = ''.join(random.choices(chars, k=16))
    return f"AKIA{suffix}"

def generate_github_pat():
    """Generate synthetic GitHub PAT (new format)"""
    chars = string.ascii_letters + string.digits
    suffix = ''.join(random.choices(chars, k=36))
    prefix = random.choice(['ghp', 'gho', 'ghu', 'ghs', 'ghr'])
    return f"{prefix}_{suffix}"

def generate_jwt():
    """Generate syntactically valid JWT structure"""
    header = base64.urlsafe_b64encode(
        json.dumps({"alg": "HS256", "typ": "JWT"}).encode()
    ).rstrip(b'=').decode()
    payload = base64.urlsafe_b64encode(
        json.dumps({"sub": "1234567890", "iat": 1516239022}).encode()
    ).rstrip(b'=').decode()
    signature = ''.join(random.choices(
        string.ascii_letters + string.digits + '-_', k=43
    ))
    return f"{header}.{payload}.{signature}"

For generic hardcoded credentials — the human-chosen passwords and internal tokens that no regex would catch — I generate values following common human password patterns:

def generate_human_chosen_password():
    """Generate realistic human-chosen passwords"""
    patterns = [
        # Word + year + special
        lambda: f"{random.choice(COMMON_WORDS)}{random.randint(2015, 2024)}{random.choice('!@#$%')}",
        # Capitalised word + number
        lambda: f"{random.choice(COMMON_WORDS).capitalize()}{random.randint(1, 999)}",
        # Two words concatenated
        lambda: f"{random.choice(COMMON_WORDS)}{random.choice(COMMON_WORDS)}",
        # Word + special pattern
        lambda: f"{random.choice(COMMON_WORDS).upper()}_{random.randint(100, 999)}",
    ]
    return random.choice(patterns)()

COMMON_WORDS = [
    "winter", "summer", "spring", "autumn", "admin", "secure",
    "company", "service", "backend", "system", "master", "main",
    "deploy", "production", "staging", "develop", "internal"
]

Each generated secret is paired with a realistic variable name drawn from the high-risk vocabulary:

SECRET_VARIABLE_NAMES = [
    "API_KEY", "api_key", "apiKey",
    "SECRET_KEY", "secret_key", "secretKey",
    "PASSWORD", "password", "passwd", "pwd",
    "ACCESS_TOKEN", "access_token", "accessToken",
    "DATABASE_URL", "database_url", "db_url", "DB_URL",
    "PRIVATE_KEY", "private_key", "privateKey",
    # ... 40+ more
]

The (variable_name, value) pairs that go into training represent the full context the feature extractor sees.

Generating Negative Examples (Benign High-Entropy Strings)

The negative class is where most secrets detectors fail — they don't have enough high-quality negative examples, so the model learns "high entropy = secret" rather than "high entropy in a credential context = secret."

I generate several categories of benign high-entropy strings:

def generate_uuid():
    return str(uuid.uuid4())

def generate_sha256_hash():
    content = ''.join(random.choices(string.printable, k=random.randint(10, 100)))
    return hashlib.sha256(content.encode()).hexdigest()

def generate_md5_hash():
    content = ''.join(random.choices(string.printable, k=random.randint(10, 100)))
    return hashlib.md5(content.encode()).hexdigest()

def generate_base64_data():
    """Simulate base64-encoded image or binary data fragments"""
    data = bytes(random.randint(0, 255) for _ in range(random.randint(20, 60)))
    return base64.b64encode(data).decode()

def generate_package_integrity_hash():
    """npm/yarn integrity hash format"""
    data = bytes(random.randint(0, 255) for _ in range(48))
    hash_val = base64.b64encode(data).decode()
    return f"sha512-{hash_val}"

def generate_hex_color():
    return f"{''.join(random.choices('0123456789abcdef', k=6))}"

def generate_version_string():
    major = random.randint(0, 10)
    minor = random.randint(0, 99)
    patch = random.randint(0, 999)
    return f"{major}.{minor}.{patch}"

Each negative example is paired with a low-risk variable name:

BENIGN_VARIABLE_NAMES = [
    "checksum", "hash", "digest", "fingerprint",
    "uuid", "guid", "id", "identifier", "correlation_id",
    "version", "release", "build_number",
    "color", "colour", "hex_color",
    "integrity", "content_hash",
    # ... 30+ more
]

Class Balance and Distribution

The training set uses a 50/50 class balance — equal numbers of secrets and benign strings. This is a deliberate choice.

Real codebases have far fewer secrets than benign strings — maybe 1% of high-entropy strings are actual secrets in a typical codebase. Training on a 1% positive class would produce a classifier that learns to say "not a secret" almost all the time and achieves 99% accuracy by doing so — completely useless.

A 50/50 balance forces the model to actually learn to distinguish the classes. The resulting classifier has higher false positive rates on real codebases than the training accuracy suggests, which is why the confidence threshold (default 0.7) and the key name feature do so much work in production.

The threshold can be adjusted to trade precision for recall:

# Higher threshold — fewer false positives, more false negatives
python main.py scan ./src --threshold 0.85

# Lower threshold — more findings, more false positives
python main.py scan ./src --threshold 0.55

Validating Synthetic Data Quality

The risk of synthetic data is that it doesn't reflect the distribution of real data. A model trained on synthetic examples might perform well on the test set (also synthetic) and poorly on real codebases.

I validated against three real-world test cases:

Test 1: Known public secret patterns. I collected public documentation examples of secret formats — the example values shown in AWS, GitHub, and OpenAI documentation. These are not real secrets; they're deliberately fake values used in documentation. The model should classify them as secrets (since they match real formats) and does so at >95% confidence.

Test 2: Known benign high-entropy strings. I collected package-lock.json integrity hashes, UUID values from public test suites, and SHA-256 checksums from public software distributions. The model should classify these as benign and does so at <10% confidence in the vast majority of cases.

Test 3: Edge cases from my own code. I scanned my own development projects — including the secrets detector itself — and manually reviewed every finding above 0.5 confidence. This is where real-world calibration happens. The findings from this scan informed several adjustments to the key name vocabulary and confidence thresholds.

The synthetic approach doesn't eliminate the need for this kind of real-world validation. It just means the real-world validation is about calibration rather than about whether the model has learned anything at all.

The Ongoing Data Problem: Concept Drift

Secret formats change. New services launch with new key formats. Existing services rotate their key structures for security reasons. The synthetic data that was representative in 2023 may underrepresent the formats that matter in 2025.

This is the secrets detection equivalent of the vulnerability scanner coverage gap problem — there will always be a lag between a new format appearing in the wild and the tool being updated to detect it.

The response to this is the same as it is for signature-based detection: a clear update process. When a new cloud service launches with a distinctive key format, the update is:

Add a generator function for the new format to trainer.py
Add a pattern match flag to the feature extractor
Retrain: python main.py train --samples 6000
The new format is now detected The synthetic data approach makes this update cycle fast and low-risk. Adding new training examples doesn't require finding or curating real examples of the new format — just implementing its generation logic. Retraining takes seconds. The update can ship as a minor version bump.

Synthetic Data as a Security Research Methodology

Stepping back from this specific tool: the synthetic data approach is applicable to a much broader class of security ML problems.

Phishing email detection can be trained on algorithmically generated phishing templates rather than real phishing emails, which carry real malicious links and attachments.

Malware classification researchers face the same problem I faced — real malware samples are dangerous to handle and distribute. Synthetic malware features derived from known behavioral signatures can substitute for actual samples in feature-level classifiers.

Log anomaly detection for security can use synthetic attack log patterns derived from published attack techniques rather than actual attack logs from production systems.

The common thread: real security data is often sensitive, legally ambiguous, dangerous to handle, or has quality problems that make it worse than it appears. Carefully generated synthetic data, validated against real-world examples without incorporating them into training, is frequently the more practical path.

The tradeoff is always the same: you give up the naturalness of real data distribution in exchange for control, safety, reproducibility, and shareability. For security tooling specifically — where trust and auditability matter — that tradeoff is often worth making.

What I'd Do Differently

If I were building this tool for a commercial security product rather than a portfolio project, I'd approach training data differently in two ways.

Structured negative mining from real codebases. Rather than generating synthetic negative examples, I'd mine real open source repositories for high-entropy strings that are demonstrably not secrets — package hashes, checksums in test suites, example values in documentation. These are safe to use (no real credentials), have the right distribution (they appear in real code as developers write it), and don't require synthetic generation. The labeling work is the constraint, not the data availability.

A small labeled set of real format examples. Not real credentials — but real format examples. The example values in service provider documentation (AWS's AKIAIOSFODNN7EXAMPLE, GitHub's documented PAT format examples) are designed to look like real keys without being real keys. A small set of these, clearly labeled, would improve the model's calibration on the exact formats that matter most.

The synthetic approach I built is the right choice given the constraint of a solo portfolio project with no data labeling resources. A team building a production tool would have access to more options.

The data generation code is in trainer.py at github.com/pgmpofu/secrets-detector. All generators are clearly documented and the entire training pipeline is reproducible from scratch with a single command.

Next up: building the pre-commit hook — blocking secrets before they ever reach the repository, and the UX considerations that determine whether developers actually leave it enabled.

Why I Chose Random Forest Over Deep Learning for Secrets Detection

Patience Mpofu — Sat, 16 May 2026 02:47:49 +0000

Every time I mention that my secrets detector uses a Random Forest classifier, someone asks the same question.

"Why not a neural network?"

It's a reasonable question. Deep learning dominates ML benchmarks. Transformers have redefined what's possible in natural language understanding. If you're building a tool that reads code — which is text — shouldn't you be using the most powerful text understanding architecture available?

The answer is no. And the reasoning reveals something important about how to match ML approaches to real-world engineering constraints.

This article is the full argument: why Random Forest was the right choice for this specific problem, what I give up by not going deep, and when the calculus would flip.

The Five Constraints That Shaped the Decision

Before choosing a model architecture, I defined the constraints the tool had to satisfy. These weren't aspirational — they were hard requirements that would determine whether the tool was actually useful in practice.

Constraint 1: Must run locally with zero infrastructure.
The tool needs to work in a pre-commit hook, on a developer's laptop, without internet access. No API calls, no GPU, no Docker compose stack with a model server. A developer running git commit should not experience meaningful latency.

Constraint 2: Must ship as a self-contained package.
The model file needs to be small enough to live in the repository alongside the code. Teams shouldn't need to download a separate model artifact or manage model versioning separately from tool versioning.

Constraint 3: Must be retrainable by a non-ML engineer.
When a team encounters false positives specific to their codebase, they should be able to add examples and retrain without ML expertise, a GPU, or more than a few minutes of compute time.

Constraint 4: Must explain its decisions.
When the tool flags a finding, an engineer should be able to understand why. "The model said so" is not an acceptable answer in a security context where false positives erode trust.

Constraint 5: Must generalise from a small training set.
I'm training on synthetically generated data, not millions of real examples. The architecture needs to perform well with thousands of samples, not billions.

Every significant architectural decision in this tool flows from these five constraints. Let me show you how.

Why Deep Learning Fails Each Constraint

Constraint 1: Local execution without infrastructure

A production-quality transformer model for code understanding — something like CodeBERT or GraphCodeBERT — runs to hundreds of megabytes to several gigabytes. Running inference on a CPU is possible but slow: several seconds per scan on a typical laptop. In a pre-commit hook, where the developer is waiting at the terminal, that's unacceptable friction.

The Random Forest model runs inference in milliseconds on CPU. Scanning a 10,000-line codebase takes under two seconds on a five-year-old laptop. There's no perceptible delay between git commit and the hook completing.

Constraint 2: Self-contained package

The trained Random Forest model serialises to approximately 1MB as a pickle file. It lives in the model/ directory, ships with the tool, and requires no separate download or version management.

A fine-tuned transformer model would be 400MB–2GB depending on architecture. That's not viable as a repository artifact. It requires separate model hosting, download scripts, and version coordination — none of which a team setting up a pre-commit hook wants to manage.

Constraint 3: Retrainable by non-ML engineers

Retraining the Random Forest on 6,000 samples takes approximately eight seconds on a standard laptop CPU. Scaling to 50,000 samples takes about ninety seconds. The entire workflow is:

# Edit trainer.py to add your examples
# Then:
python main.py train --samples 10000
# Done. New model.pkl in model/

Retraining a transformer requires GPU infrastructure, hours of compute time, careful learning rate scheduling to avoid catastrophic forgetting, and validation that fine-tuning didn't degrade performance on the base cases. A team without an ML engineer cannot do this.

Constraint 4: Explainable decisions

This is where the gap between Random Forest and deep learning is most significant for a security tool.

Random Forest gives you feature importances globally and, with one additional step, per-prediction explanations. When the tool flags a finding, it can tell you exactly which features drove the decision:

Finding: api_key = "sk-proj-abc123XYZ789..."
Confidence: 96%

Contributing features:
  key_name_risk:        0.90  (HIGH — 'api_key' matches sensitive vocabulary)
  shannon_entropy:      5.82  (HIGH — consistent with cryptographic secret)
  pattern_openai_key:   1.00  (MATCH — matches OpenAI key format sk-proj-*)
  repetition_ratio:     0.94  (HIGH — low character repetition, high randomness)

An engineer reading this knows immediately why the finding was generated. They can evaluate whether the reasoning is sound. They can make an informed decision about whether to fix or suppress.

A neural network produces a probability: 0.96. No more. You can apply techniques like SHAP or LIME to approximate explanations, but these add complexity, latency, and approximation error. For a pre-commit hook that needs to explain itself to a developer in real time, "here are the features that drove this" is vastly better than "the attention mechanism focused on these tokens (approximately)."

Constraint 5: Generalisation from small data

Transformer models are data-hungry. They're pre-trained on billions of tokens and fine-tuned on millions of examples. Their power comes from the scale of pre-training, which means fine-tuning on thousands of synthetic examples carries real risk of the model not generalising well to patterns it hasn't seen.

Random Forest with well-engineered features generalises effectively from thousands of examples. The feature engineering does the heavy lifting — entropy, character ratios, key name scoring, pattern flags. The model only needs to learn the relationships between these pre-computed features, which is a much simpler learning problem than learning representations from raw text.

What I Give Up

Intellectual honesty requires being clear about the tradeoffs.

Peak accuracy ceiling. A well-fine-tuned code understanding model operating on token sequences would almost certainly achieve higher peak accuracy than my feature-engineered Random Forest. It would learn representations I haven't thought to engineer explicitly. It would capture multi-token context — the fact that password appears three lines before = "..." rather than directly adjacent, for instance.

Novel format generalisation. When a new cloud provider launches with a distinctive key format, my tool catches it only if I add a pattern match flag. A neural network trained on diverse secret formats might generalise to novel formats by recognising that they "look like secrets" in ways the feature vector doesn't capture. My tool requires an explicit pattern update.

Code context understanding. The feature vector sees one value at a time. A transformer scanning the whole file could understand that a value is being loaded from an environment variable rather than being hardcoded, that it's inside a test mock, or that it's in a comment rather than executable code. My tool handles some of these through pre-processing (only scanning string literals in executable code), but the context window is fundamentally narrower.

Cross-line data flow. If a secret is assembled across multiple lines — partial string concatenation, format strings, bytes operations — the feature vector sees fragments rather than the complete secret. A model with broader context could potentially catch these.

The Accuracy Numbers

On my test set of 1,200 labeled samples (a held-out 20% of the 6,000 training samples), the Random Forest achieves:

Metric	Score
Accuracy	94.2%
Precision	93.8%
Recall	94.7%
F1 Score	94.2%
False Positive Rate	5.8%
False Negative Rate	5.3%

For context: TruffleHog v3 (regex + entropy) reports false positive rates in the 10–15% range on typical codebases according to published evaluations. The ML approach achieves meaningfully better precision without sacrificing recall.

I don't have a head-to-head comparison against a fine-tuned transformer on this specific task — that would require the transformer, the training infrastructure, and a larger labeled dataset than I have. What I can say is that the Random Forest achieves accuracy that's competitive with existing tools, meets all five operational constraints, and does so at a fraction of the complexity.

The Decision Framework: When Would I Choose Deep Learning?

Given all of the above, there are scenarios where I would choose a different architecture.

If I were building a cloud-hosted scanning service, the infrastructure constraint disappears. GPU inference is available. Model size doesn't matter. Latency can be managed with caching and batching. In that scenario, a transformer-based approach becomes viable and the accuracy ceiling argument gets stronger.

If I had a large labeled dataset of real secrets, the data constraint relaxes. Fine-tuning on tens of thousands of real examples would likely push accuracy significantly higher than what synthetic data training achieves. The question then becomes whether the accuracy gain justifies the operational complexity.

If the primary use case were batch scanning rather than pre-commit hooks, the latency constraint loosens. Scanning a repository's entire history overnight can tolerate seconds or minutes per file. The pre-commit use case is what drives the millisecond inference requirement.

If cross-file context mattered, a graph neural network operating on the code's data flow graph might be more appropriate than either approach. Understanding that secret = get_secret_from_vault() is safe and secret = "hardcoded" is dangerous requires understanding function call semantics — something Random Forest on string features cannot do.

The right architecture is always determined by the constraints of the deployment context, not by what achieves the best benchmark score.

The Broader Principle: Fit for Purpose Over State of the Art

The machine learning community has a bias toward the most powerful available architecture. More parameters, more data, more compute — these are treated as virtues in research contexts where they often are virtues.

Production engineering has different values. A tool that actually gets used — because it's fast, explainable, maintainable, and deployable without infrastructure — delivers more security value than a theoretically superior tool that sits unused because it's too slow, too opaque, or too complex to operate.

This is an instance of a general principle I keep encountering in AppSec: the best security control is the one that gets implemented and maintained, not the one that provides the strongest theoretical protection.

A Random Forest secrets detector running in every developer's pre-commit hook, catching 94% of secrets before they reach the repository, is more valuable than a transformer-based detector achieving 98% accuracy that nobody bothered to deploy because the setup was too complicated.

The 4% accuracy difference is real. The deployment difference is everything.

What the Feature Importances Reveal About the Problem

One thing Random Forest gives you that deep learning doesn't: a clear picture of what the problem actually is.

Here are the top 10 feature importances from the trained model:

Rank	Feature	Importance
1	`key_name_risk`	0.28
2	`shannon_entropy`	0.14
3	`pattern_aws_access_key`	0.09
4	`repetition_ratio`	0.08
5	`hex_ratio`	0.07
6	`pattern_github_pat`	0.06
7	`base64_ratio`	0.05
8	`log_length`	0.04
9	`pattern_private_key_header`	0.04
10	`uppercase_ratio`	0.03

This table is a map of the secrets detection problem. It tells you that variable naming context is more predictive than any statistical property of the string itself. It tells you that entropy matters but not as much as everyone assumes. It tells you that AWS and GitHub keys are important enough that their specific pattern flags appear in the top ten even though there are 16 pattern flags spread across the remaining importance budget.

A neural network would learn similar underlying structure — it would attend more to variable names than to arbitrary string characters — but it wouldn't show you that structure explicitly. The interpretability of Random Forest turns model training into a research exercise as well as an engineering one.

That visibility into what the problem actually is informed every design decision in this tool. It's one of the most valuable things the architecture choice gave me.

The trainer code, feature importances, and model evaluation scripts are all in the repository at github.com/pgmpofu/secrets-detector.

Next up: the ethical and practical challenge of training a security ML model without using real leaked credentials — why synthetic data, how I generated it, and what the tradeoffs are.

Why the Variable Name Is the Most Important Feature in Secrets Detection

Patience Mpofu — Thu, 14 May 2026 02:26:43 +0000

ere's a question that sounds trivial until you think about it carefully.

Are these two lines of code equally dangerous?

checksum = "d8e8fca2dc0f896fd7cb4cb0031ba249"
password = "d8e8fca2dc0f896fd7cb4cb0031ba249"

The string value is identical. The entropy is identical. Every character-level feature is identical. A regex scanner treats them the same. A pure entropy scanner treats them the same. A human security engineer does not treat them the same — not even slightly.

The first is almost certainly a file integrity hash. The second is almost certainly an exposed credential. The only difference is the four characters before the equals sign.

When I trained my secrets detector and examined the feature importances, the variable name risk score came out at 0.28 — higher than Shannon entropy, higher than all character distribution features, higher than string length. The single most predictive signal for whether a string is a secret is not the string itself. It's what the developer named the variable holding it.

This article is about what that finding reveals — about how secrets detection actually works, about how developers accidentally expose credentials, and about what it means for how we should think about this entire problem class.

Why Feature Importance of 0.28 Is Remarkable

In a Random Forest model, feature importance is measured by how much each feature reduces impurity across all decision trees. An importance of 0.28 out of 1.0, across 26 features, means the variable name alone accounts for more than a quarter of the model's predictive power.

To put that in context: if you removed every other feature and kept only the variable name, you'd still have a classifier that makes correct decisions on the majority of cases. If you kept every other feature and removed the variable name, you'd lose more predictive power than any other single change.

That's not what I expected when I designed the feature vector. I expected entropy to dominate — it's the signal that most secrets detection literature focuses on. The finding that variable names outperform entropy forced me to rethink some assumptions about the problem.

What Variable Names Actually Encode

Variable names in production code are not arbitrary. They're communication.

When a developer writes api_key = "...", they're not just labelling a memory location. They're documenting their intent. They're telling the next engineer — and, it turns out, a machine learning classifier — that this value is an API key, that it's sensitive, that it should be treated as a secret.

Developers are remarkably consistent about this. Across codebases, languages, and organisations, the same small vocabulary appears around credential storage:

password, passwd, pwd
secret, secret_key, client_secret
api_key, apikey, api_token
token, access_token, auth_token, bearer_token
private_key, privkey, pem
credential, credentials, creds
database_url, db_url, connection_string

And a complementary vocabulary appears around non-sensitive high-entropy strings:

checksum, hash, digest, fingerprint
uuid, guid, id, identifier
version, release, build
color, colour, hex
integrity, signature (in package manifest contexts)

The signal isn't perfect — id sometimes refers to a sensitive identifier, token is sometimes used for pagination tokens or CSRF tokens that aren't secrets in the traditional sense. But the correlation between variable name semantics and actual sensitivity is strong enough to be the most predictive single feature in a 26-dimensional model.

The Three Ways Developers Name Credential Variables

Understanding how variable names signal secrets requires understanding the patterns developers actually use. In practice, there are three distinct naming patterns, each with different detection implications.

Pattern 1: Direct and Obvious

The developer uses a name that directly and unambiguously identifies the value as sensitive:

STRIPE_SECRET_KEY = "sk_live_abc123..."
DATABASE_PASSWORD = "Tr0ub4dor&3"
GITHUB_ACCESS_TOKEN = "ghp_abc123..."
JWT_SECRET = "my-super-secret-signing-key"

These are the easy cases. The variable name scores maximum risk, the classifier is highly confident, and the finding is genuine in nearly every instance. There's no ambiguity to resolve.

Pattern 2: Abbreviated and Conventional

The developer uses a shortened or conventional form that's recognisable within the development community but might be less obvious to an outsider:

DB_PASS = "Winter2019!"          # "PASS" → password
AWS_SK = "wJalrXUtn..."          # "SK" → secret key
OAUTH_CS = "abc123def456..."     # "CS" → client secret
SVC_PWD = "service_password_1"  # "PWD" → password

The risk scoring function handles these through substring matching and a vocabulary of abbreviations. PASS, PWD, SK (in certain contexts), CS, TKN all score high. This is where coverage gaps can appear — an unusual abbreviation in a domain-specific codebase might not be in the vocabulary.

Pattern 3: Contextually Sensitive

The developer uses a name that doesn't obviously indicate sensitivity on its own but becomes sensitive in context:

# In a payment processing module
value = "sk_live_abc123..."       # "value" alone scores 0.1

# In a configuration dictionary
config = {
    "key": "AKIAIOSFODNN7EXAMPLE"  # "key" alone scores 0.7
}

# In a function parameter
def authenticate(token):           # "token" scores 0.9
    headers = {"Authorization": f"Bearer {token}"}

These cases are where the feature vector struggles most. The variable name score for value is 0.1 — weak evidence of sensitivity. In isolation, the classifier would likely pass this. But if the string value itself has high entropy and matches a known pattern (like the Stripe key format), the pattern flags compensate.

This interaction — where a weak key name score is overridden by strong pattern match flags — is exactly how the feature vector is supposed to work. No single feature dominates all cases. The combination handles cases that individual features miss.

The Accidental Exposure Psychology

The variable name finding reveals something important about how secrets end up in code in the first place.

Developers don't accidentally commit secrets because they don't know secrets are sensitive. They commit secrets because the friction of not committing them is high at the moment of writing.

The archetypal scenario:

Developer is building a feature that needs an API key
Developer adds the key directly to the code to get the feature working
Developer intends to "move it to environment variables later"
Developer commits the working code
"Later" never comes, or the commit is already in history The variable name in this scenario is almost always informative — the developer names it API_KEY or STRIPE_KEY or DB_PASSWORD because they know exactly what it is. They're not hiding it from themselves. They're just deferring the cleanup.

This is why the variable name is such a strong signal. The developer's intent is encoded in the name. The developer knew this was a secret when they wrote it. The name reflects that knowledge.

The cases where variable names are not informative are the cases where secrets end up in code through a different mechanism — configuration files that get accidentally committed, environment files that get accidentally tracked, secrets embedded in test data that wasn't meant to be realistic. These are harder to catch and require the entropy and pattern features to carry more weight.

Where Variable Name Scoring Breaks Down

Being precise about the limits of this approach matters.

Obfuscated Names

An attacker who knows about this detection vector could theoretically name credential variables to avoid detection:

# Designed to evade variable name scoring
x = "AKIAIOSFODNN7EXAMPLE7"
data_1 = "sk_live_abc123def456ghi789"
temp = "my-database-password-2024"

The first two would still be caught by pattern match flags — the AWS key format and Stripe key format are distinctive enough that entropy and pattern features alone classify them correctly. The third — a human-chosen password stored in a variable named temp — would be at risk of being missed if the entropy is low.

This is a real gap. In practice, deliberately obfuscated variable names are uncommon in the codebases secrets appear in — developers who are trying to hide secrets from their own team are a different threat model than developers who accidentally expose secrets. But the gap exists.

Generic Framework Patterns

Frameworks and ORMs often use generic variable names that pattern-match to high risk scores without holding sensitive values:

# Django ORM — "password" is a field name, not a credential
class User(models.Model):
    password = models.CharField(max_length=128)  # hashed, not plaintext

# Spring Security — "token" is a parameter name
public ResponseEntity<?> authenticate(@RequestBody TokenRequest token) {

In these cases, the value being assigned is a class reference, a method call, or a parameter object — not a string literal containing a secret. The feature extraction pipeline only runs on string literals, so these cases are largely handled correctly at the scanning stage before feature extraction begins.

Internationalised Codebases

The variable name risk vocabulary is English-only. A codebase with variable names in German (Passwort), French (motDePasse), or Portuguese (senha) will have those variables score the default 0.3 rather than 1.0. This is a genuine coverage gap for multinational organisations. Extending the vocabulary to cover common credential-related terms in other languages would be a meaningful improvement.

The Implication for Secrets Management Culture

The variable name finding has an implication beyond detection accuracy. It tells us something about where to focus prevention efforts.

If developers consistently name their credential variables correctly — if DB_PASSWORD always contains a database password and checksum never does — then the signal for detection is strong. The corollary is that the naming is correct precisely because the developer knows the value is sensitive.

This means secrets in code are not primarily a knowledge problem. Developers who commit secrets usually know they're committing secrets. The problem is friction — the path of least resistance at the moment of writing is to hardcode the value and deal with it later.

The most effective prevention isn't education about what a secret is. It's reducing the friction of not hardcoding secrets in the first place. Pre-commit hooks that block commits before they reach the repository — rather than scanners that find secrets after the fact — address the friction problem directly.

Which is exactly what the pre-commit hook in this tool is designed to do. Catch it at the moment of commit, when the developer already knows the variable is named API_KEY and the value looks like a real key. That's the lowest-friction intervention point — a message at the moment of action rather than a report discovered in a CI pipeline hours later.

Improving the Key Name Feature

The current implementation is a static keyword vocabulary with manually assigned scores. It works well but has obvious limitations — it doesn't learn, it doesn't generalise to novel terms, and it requires manual updates when new credential-adjacent terminology emerges.

A more sophisticated approach would train a small embedding model on variable names from open source code, clustering semantically similar names and learning the association between name semantics and credential presence. Something like word2vec trained on a corpus of variable names from public repositories would generalise to svc_acct_passwd or oauth2_bearer_tkn without requiring explicit vocabulary entries.

That's a meaningful improvement but a substantial increase in complexity. The static vocabulary approach handles the vast majority of real-world cases well enough that the engineering investment in embeddings hasn't been justified yet.

The right trigger for that investment would be systematic measurement of false negatives — cases where the classifier misses real secrets. If those misses cluster around variable naming patterns that aren't in the current vocabulary, the embeddings approach becomes worth building.

What This Tells Us About Security Signal in General

The variable name finding is an instance of a broader principle that I keep encountering in application security work: the most useful signals are often not in the content itself, but in the metadata around the content.

The string "d8e8fca2dc0f896fd7cb4cb0031ba249" carries no information about whether it's sensitive. The variable name password carries almost complete information. The content is identical; the context is everything.

This same principle appears elsewhere in AppSec:

HTTP request bodies are harder to classify as malicious than request paths, because paths carry semantic intent
Log anomaly detection is more effective on log metadata (frequency, source, timing) than log content
Phishing detection is more accurate on sender domain patterns than email body content The implication for building security tools is consistent: don't just look at the data. Look at what surrounds the data. Look at what the developer named it, where it came from, when it appeared, what called it.

Context is signal. Often it's more signal than the content itself.

The full key name risk scoring implementation is in secrets_detector/features.py at github.com/pgmpofu/secrets-detector.

Next up: why I chose Random Forest over deep learning for this problem — the full engineering tradeoffs argument, including interpretability, model size, training speed, and what you give up by not going deep.

The 26-Dimensional Feature Vector: How a Machine Learns to Recognise a Secret

Patience Mpofu — Thu, 14 May 2026 02:23:02 +0000

hen my secrets detector evaluates a candidate string, it doesn't see code.

It sees a vector of 26 numbers.

That vector is the bridge between human intuition — "this looks like a secret" — and machine classification. Every insight a security engineer uses when reading code to spot exposed credentials has been translated into a numerical feature that the Random Forest classifier can reason about.

This article is a complete walkthrough of those 26 features: what each one measures, why it matters, what it catches, and what it misses. By the end, you'll understand exactly what the model sees when it evaluates any candidate value — and why the combination of features catches things that no single signal could.

How Feature Extraction Works

Before the classifier sees anything, every candidate string goes through a feature extraction pipeline in features.py. The pipeline takes two inputs: the string value itself, and the name of the variable holding it.

def extract_features(value: str, key_name: str) -> np.ndarray:
    features = []

    # Entropy features
    features.append(shannon_entropy(value))
    features.append(math.log(len(value) + 1))
    features.append(repetition_ratio(value))
    features.append(longest_run_normalized(value))

    # Character distribution features (8 features)
    features.extend(character_ratios(value))

    # Key name context
    features.append(key_name_risk_score(key_name))

    # Pattern match flags (16 features)
    features.extend(pattern_match_flags(value))

    return np.array(features)

The output is a fixed-length array of 26 floating point numbers. The classifier never sees the original string — only this vector. That's both a strength (the model generalises across different string formats) and a limitation (some context that a human would use is deliberately excluded).

Let me walk through each group.

Group 1: Entropy Features (4 features)

These four features capture the statistical "randomness" of the string — the property that real secrets share with random data.

Feature 1: Shannon Entropy

Shannon entropy measures the unpredictability of the character sequence. For a string of length n with character frequencies p_i:

H = -Σ p_i × log₂(p_i)

A perfectly random string of alphanumeric characters has entropy around 5.7–6.0 bits. Common English words have entropy around 3.5–4.5 bits. Cryptographically generated secrets cluster at the high end.

# High entropy — likely a secret or hash
"sk-proj-abc123XYZ789..." → entropy: 5.82

# Low entropy — likely a password or human-chosen value  
"Winter2019!"            → entropy: 3.21

# Very low entropy — definitely not a secret
"aaaaaaaaaaaaa"          → entropy: 0.00

Entropy alone is a weak classifier — UUIDs, SHA-256 hashes, and base64 image data all have high entropy but are not secrets. That's why it's one of 26 features, not the only feature.

Feature 2: Log-Scaled Length

Raw string length would give too much weight to very long strings. Log-scaling (math.log(len(value) + 1)) compresses the range so that the difference between a 32-character key and a 64-character key has roughly the same weight as the difference between a 4-character and 8-character string.

Secrets tend to fall in predictable length ranges: AWS access keys are 20 characters, GitHub PATs are 40, JWT tokens are variable but typically 200+. Length contributes signal, but it's a soft signal — there's no length that definitively indicates "secret."

Feature 3: Repetition Ratio

repetition_ratio = len(set(value)) / len(value)

This is the proportion of unique characters to total characters. A perfectly random string of 32 characters will have close to 32 unique characters (ratio ≈ 1.0). A string like "aababcababc" has low unique character count relative to its length (ratio ≈ 0.3).

Low repetition ratio is a strong signal that a string is not a secret — real secrets don't repeat characters predictably. High repetition ratio is a necessary but not sufficient condition for being a secret.

Feature 4: Longest Run (Normalised)

The length of the longest consecutive run of the same character, divided by string length:

longest_run = max(len(list(g)) for _, g in itertools.groupby(value))
longest_run_normalized = longest_run / len(value)

"aaabbbccc" has a longest run of 3 out of 9 characters — normalised run of 0.33.
"sk-abc123XYZ789def456" has a longest run of 1 out of 21 characters — normalised run of 0.05.

Long runs of repeated characters are a strong signal of non-random data. No cryptographically generated secret will have a long run. Human-readable strings often will.

Group 2: Character Distribution Features (8 features)

These eight features describe the composition of the string across character classes. Together they capture the "shape" of the character set that a human eye uses to distinguish secrets from benign strings.

Feature	What It Measures
`uppercase_ratio`	Proportion of A–Z characters
`lowercase_ratio`	Proportion of a–z characters
`digit_ratio`	Proportion of 0–9 characters
`special_ratio`	Proportion of non-alphanumeric characters
`hex_ratio`	Proportion of valid hexadecimal characters (0–9, a–f, A–F)
`base64_ratio`	Proportion of base64-safe characters (alphanumeric + /+=)
`printable_ratio`	Proportion of printable ASCII characters
`whitespace_ratio`	Proportion of whitespace characters

Why these specific ratios matter:

hex_ratio is particularly useful for distinguishing hash values from secrets. A SHA-256 hash has a hex_ratio of 1.0 — every character is a valid hex digit. An AWS access key has a hex_ratio of approximately 0.6 (uppercase letters reduce it). A JWT token has a hex_ratio near 0.0 (it's base64url-encoded, using characters outside the hex alphabet).

special_ratio catches secrets that include special characters — a strong signal for human-chosen passwords ("P@ssw0rd!") versus machine-generated tokens (which typically avoid special characters for compatibility reasons).

base64_ratio is the mirror of hex_ratio for base64-encoded content. Base64-encoded image data has a base64_ratio near 1.0. An API key that uses only alphanumeric characters has a high base64_ratio too — which is where the key name and other features need to disambiguate.

The classifier learns the interaction between these ratios. A string with high entropy, high hex_ratio, and a key name that scores 0.0 is almost certainly a hash. A string with high entropy, mixed character ratios, and a key name that scores 1.0 is almost certainly a secret.

Group 3: Key Name Risk Score (1 feature)

This is the single most important feature in the model — feature importance 0.28, more than the entropy and character features combined.

KEY_NAME_RISK = {
    # Score 1.0 — unambiguously sensitive
    "password": 1.0, "passwd": 1.0, "secret": 1.0,
    "private_key": 1.0, "privkey": 1.0,

    # Score 0.9 — very likely sensitive  
    "api_key": 0.9, "apikey": 0.9, "token": 0.9,
    "credential": 0.9, "auth_token": 0.9,

    # Score 0.85 — likely sensitive
    "access_key": 0.85, "client_secret": 0.85,
    "bearer": 0.85, "authorization": 0.85,

    # Score 0.7 — possibly sensitive
    "key": 0.7, "auth": 0.7, "login": 0.7,

    # Score 0.2 — unlikely sensitive
    "config": 0.2, "setting": 0.2, "value": 0.1,

    # Score 0.0 — not sensitive
    "checksum": 0.0, "hash": 0.0, "version": 0.0,
    "id": 0.0, "uuid": 0.0, "color": 0.0
}

def key_name_risk_score(key_name: str) -> float:
    normalised = key_name.lower().strip("_")
    for keyword, score in KEY_NAME_RISK.items():
        if keyword in normalised:
            return score
    return 0.3  # Unknown key names get a moderate default

The scoring function does substring matching, so DB_PASSWORD, database_password, and user_passwd all score 1.0. API_KEY_V2 and service_api_key both score 0.9.

Unknown variable names — ones that don't contain any recognised keyword — get a default score of 0.3. This is deliberately moderate: an unknown variable name is mild evidence that the string might not be sensitive (if it were, it would likely have a recognisable name), but it's not strong evidence either way.

The impact of this feature on classification decisions is substantial:

# Same value, wildly different classifications
password = "d8e8fca2dc0f896fd7cb4cb0031ba249"  # → flagged at 94% confidence
checksum = "d8e8fca2dc0f896fd7cb4cb0031ba249"  # → passed at 8% confidence

Without the key name feature, these two lines are identical to the classifier. With it, they're completely distinguishable.

Group 4: Pattern Match Flags (16 features)

These are binary features — 0 or 1 — indicating whether the value matches any of 16 known secret format patterns.

Flag	Pattern
`pattern_aws_access_key`	`AKIA[0-9A-Z]{16}`
`pattern_github_pat`	`gh[pousr]_[A-Za-z0-9]{36}`
`pattern_github_fine_grained`	`github_pat_[A-Za-z0-9]{82}`
`pattern_jwt`	`eyJ[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+`
`pattern_openai_key`	`sk-[A-Za-z0-9]{48}`
`pattern_slack_token`	`xox[baprs]-[A-Za-z0-9-]+`
`pattern_stripe_secret`	`sk_live_[A-Za-z0-9]{24}`
`pattern_stripe_publishable`	`pk_live_[A-Za-z0-9]{24}`
`pattern_google_api`	`AIza[0-9A-Za-z-_]{35}`
`pattern_heroku_api`	`[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}`
`pattern_private_key_header`	`-----BEGIN (RSA\
{% raw %}`pattern_db_connection`	`(postgresql\
{% raw %}`pattern_basic_auth`	`[A-Za-z0-9+/]{20,}={0,2}` (base64 basic auth)
`pattern_bearer_token`	`Bearer [A-Za-z0-9-._~+/]+=*`
`pattern_hex_key_32`	`[0-9a-f]{32}` (32-char hex — common key length)
`pattern_hex_key_64`	`[0-9a-f]{64}` (64-char hex — SHA-256 length)

When any of these flags fire, the classifier has strong prior evidence that the value is a known secret format. A value that matches pattern_aws_access_key will be classified as a secret at very high confidence regardless of what the other features say.

The last two flags — pattern_hex_key_32 and pattern_hex_key_64 — deserve special mention. These match the lengths of common cryptographic keys but also match MD5 and SHA-256 hashes, which are not secrets. This is where the key name feature does critical disambiguation work: a 32-character hex string with key name checksum has pattern_hex_key_32 = 1 but key_name_risk = 0.0, and the classifier correctly passes it. The same string with key name encryption_key gets flagged.

How the Features Interact: Three Case Studies

Understanding individual features is useful. Understanding how they interact is where the real insight lives.

Case Study 1: The Human-Chosen Password

SMTP_PASSWORD = "Winter2019!"

Feature	Value	Signal
`shannon_entropy`	3.4	Weak — below threshold for "looks random"
`repetition_ratio`	0.91	Neutral
`special_ratio`	0.09	Slightly elevated
`key_name_risk`	1.0	Very strong — "password" scores maximum
`pattern_*` flags	All 0	No known format match
Classification	Secret — 91% confidence

The entropy would cause a pure entropy scanner to miss this. The key name saves it.

Case Study 2: The UUID False Positive

session_correlation_id = "550e8400-e29b-41d4-a716-446655440000"

Feature	Value	Signal
`shannon_entropy`	4.1	Moderate — looks somewhat random
`hex_ratio`	0.89	Very high — almost all hex characters
`special_ratio`	0.08	Low — only hyphens
`key_name_risk`	0.0	Minimal — "id" scores 0.0
`pattern_heroku_api`	1	Fires — Heroku API keys are UUID-format
Classification	Benign — 23% confidence

The pattern flag fires (Heroku API keys look like UUIDs), but the key name score is 0.0 and the classifier correctly suppresses the finding. A regex scanner using only the Heroku pattern would flag this. The ML classifier does not.

Case Study 3: The Ambiguous High-Entropy String

encryption_key = "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6"

Feature	Value	Signal
`shannon_entropy`	3.9	Moderate
`hex_ratio`	1.0	Maximum — pure hex
`key_name_risk`	0.9	Very high — "key" scores 0.7, "encryption" modifier pushes it higher
`pattern_hex_key_32`	1	Fires — 32-char hex matches
`repetition_ratio`	0.44	Low — repeating pattern visible
Classification	Secret — 78% confidence

The repetition ratio is low (the value has a repeating a1b2c3... pattern that reduces uniqueness), which pulls the confidence down from what it would be for a truly random key. But the key name and pattern flag are strong enough to push it above the reporting threshold. The finding would be reported at MEDIUM confidence — a prompt for human review rather than a guaranteed finding.

What the Feature Vector Cannot See

Intellectual honesty requires being clear about the limits.

Cross-variable context. The feature vector sees one value at a time. It can't see that key = config["encryption_key"] is loading the key from a config object rather than hardcoding it. A human engineer would immediately see that's not a hardcoded secret; the feature vector has no way to represent that.

File context. The feature vector doesn't know it's in a test file, a mock object, or a README code example. TEST_API_KEY = "fake-key-for-testing" might have a high key name risk score despite being explicitly for testing. The inline suppression annotation (# secrets-ignore) is the escape hatch for this case.

Semantic intent. version = "1.0.0" will correctly score a low key name risk. But release_token = "1.0.0-beta" might score higher because "token" is a high-risk keyword — even though in context this is clearly a version string. The feature vector sees the word "token" in the variable name without understanding that it's used semantically differently here.

These limitations are why the classifier is a signal generator rather than an oracle. Every finding above the confidence threshold warrants a human review. The classifier reduces the review burden dramatically — from "look at every high-entropy string in the codebase" to "look at these 20 high-confidence findings" — but it doesn't eliminate the need for human judgment.

Retraining on Your Own Data

The feature vector approach makes retraining practical in a way that deep learning approaches don't. Because the features are hand-engineered and interpretable, adding new training samples has predictable effects.

If your codebase has a pattern of false positives — say, your internal logging library uses variable names like log_token that consistently score high key name risk despite being benign — you can add synthetic examples of that pattern to the benign training set and retrain in seconds:

# Add your custom generators to trainer.py, then:
python main.py train --samples 5000

The retrained model immediately incorporates your organisation-specific context. That's a capability that's practically unavailable with regex-based tools (you'd have to modify pattern files and accept increased miss rates) and theoretically possible but operationally impractical with deep learning (retraining takes hours and requires ML expertise).

The complete feature extraction code is in secrets_detector/features.py at github.com/pgmpofu/secrets-detector.

Next up: why the variable name is the single most important feature in secrets detection — and what that tells us about how developers accidentally expose credentials.