What a vulnerability database can't tell you
Every dependency scanner works the same way: take the packages in your lockfile, look each one up in a vulnerability database, report the matches. That is useful, and it has a hard limit. It can only report what someone has already found, written up and published.
A whole class of risk never gets there. A well-maintained library with no CVE can still be where your service breaks, because of what your code hands it:
- a URL from a request passed to an HTTP client, reaching an internal address,
- an uploaded archive handed to a parser that expands it without limit,
- a user-supplied path joined into a file operation,
- input that becomes a shell command three packages down the chain.
Nothing is wrong with the library in any of these. So nobody will ever file an advisory, and every scanner that looks at packages will keep saying "safe".
We wanted to find that risk anyway. So we built vdb harden, which follows the call instead of the name: it abstracts your code locally (no source leaves your machine), joins it with your lockfile and with a summary of where each package version sends its arguments, and traces input from your code, through the dependency chain, to whatever dangerous operation it reaches. No CVE needed; the path is decided from the code. On our own service it led us to a way to stall the API with two file uploads that no advisory will ever describe. (That story, and why following the call is hard, are written up separately.)
It worked, which created a second problem.
Too many true findings
On one of our own projects, 82 packages deep, the analysis came back with 1,249 paths from the code to something dangerous. Every one was a fair finding. None of them told anyone what to do on Monday morning. A list of paths is not an order of work.
So we built the layer on top. It answers three questions:
- How risky is this project, as one number you can track?
- When something changes upstream, does it matter to us?
- What should we fix first, and what will that buy?
One graph for every way in
The decided paths become edges in an attack graph: call sites, packages and sinks as nodes. The key change is what counts as an entry point. Not only request handlers. Six kinds go into the same graph, on the same scale:
| Entry | Example |
|---|---|
| Input to your code | a request argument, a CLI argument, an environment variable, stdin |
| A package name that does not exist | an AI agent added a dependency it invented |
| A newly registered package | a name that appeared on the registry days ago |
| An install script | a package that runs code when it is installed |
| A tool server's scope | an MCP server that declares shell or file access |
| A published CVE | weighted by whether it is known to be exploited |
So "an agent added a dependency that does not exist" and "an exploited advisory is reachable from your upload handler" land in one ranking, not in two tools that never talk to each other. And that is where vulnerability data comes back in: as one kind of entry among six, not the whole picture.
Scoring a path, and a project
A path's score multiplies four things:
- The entry's threat probability. A request argument is attacker-controlled by definition; an environment variable mostly is not. A CVE on the known-exploited list counts in full; otherwise its EPSS score is used.
- A confidence per hop, lowered wherever the engine had to assume propagation.
- A risk factor per package, from registration age, release history, a tool server's recent scope expansion.
- The sink's impact: command execution and deserialization at the top; file paths and network requests below; regexes, decompression and global state lower still.
Paths combine into a project score the way independent risks do (noisy-OR).
The part we care about most is coverage. If half your packages have no summary yet, a low score means "we couldn't see", not "you're safe". So the score comes with the share of the graph that was analysed, and when that share is too low it becomes an interval: decided paths give the lower bound; assuming every undetermined path reaches the worst sink gives the upper.
Real output, for a one-file Flask app with five pinned dependencies:
$ vdb harden app.py --manifest requirements.txt
5 dependencies on the graph · 3 analysed · 2 without a summary yet · taint reached 3
Project risk: 0.65–0.75 (interval — analysis incomplete; 3 of 5 dependencies have a summary, 4 undetermined path(s))
low 0.32 code_input request > requests@2.34.2 > net_request in app.py
high 0.25 code_input param > stdlib > cmd_exec in app.py
low 0.16 code_input request > requests@2.34.2 > urllib3@2.7.0 > net_request in app.py
low 0.06 cve PYSEC-2026-4175 > urllib3@2.7.0 > net_request in app.py
low 0.06 cve PYSEC-2026-4177 > urllib3@2.7.0 > net_request in app.py
low 0.06 cve PYSEC-2026-4176 > urllib3@2.7.0 > net_request in app.py
undetermined: 4 path(s) end in a package without a summary — most often idna@3.11 (2), certifi@2026.7.22 (2)
A request reaching an HTTP client, a parameter reaching a shell, and three advisories on the pinned urllib3, in one list, on one scale.
Keeping it current
The graph is stored and re-scored when an advisory is published, a missing summary gets built, the registry changes, or a tool server's scopes move. Each event re-scores only the paths it touches, and you're mailed only when your project's score actually moved, not for every CVE in every package you include.
We learned one rule the hard way: an engine upgrade re-decided paths for projects analysed days earlier, the difference looked like new findings, and people got mail about nothing. A new version of our analysis is not a change in your code, so it now re-baselines silently.
What to fix first
Every candidate fix is applied to the graph counterfactually (upgrade, harden one call site or all that reach a sink, remove a dependency, change execution policy), costed (version jump, broken constraints, call sites to edit), and ranked by risk reduction per cost. The ranking ends with the combination that reaches a target.
$ vdb harden app.py --manifest requirements.txt --remediate
Remediation ranking — risk 0.65–0.75, target 0.20
# action Δrisk cost value closes
1 harden app.py:6 0.26 0.20 0.219 2 (worth 0.43)
2 egress-allowlist on net_request sinks 0.17 0.40 0.125 0
3 remove requests 0.19–0.26 0.80 0.106 2 (worth 0.43)
4 harden app.py:9 0.12 0.20 0.097 1 (worth 0.25)
5 upgrade urllib3 2.7.0 → 2.8.0 0.12–0.16 0.40 0.083 7 (worth 0.31)
...
Combination: #1 → 0.39–0.55, #2 → 0.32–0.51, #4 → 0.09–0.34 (reaches the target; 3 step(s), cost 0.80)
Hardening one request handler leads: one edit closes both paths through requests. Upgrading urllib3 closes more paths but buys less. Three steps reach the target.
Five popular open-source projects
Our own code is not a fair test, so we ran it on open-webui, AutoGPT's platform backend, the full-stack FastAPI template, paperless-ngx and saleor. These measure our analysis, not those projects; we report no vulnerabilities in them.
Three things broke:
Derived values scored as if an attacker typed them. In cmd = "convert " + name, only explicitly read values had an origin, so cmd fell back to "attacker input". Now origins flow through assignments, and a value with no trail is weighted at half a request's. The FastAPI template went from "1.00, saturated" to 0.26–0.52 with one real path.
Every large project scored 1.0. Noisy-OR saturates once a few hundred paths are open. So the score now also carries open paths, risk mass (summed path scores: 7.5 for paperless-ngx, 109 for AutoGPT's backend) and the highest path per sink class, and the report leads with those when saturated.
The ranking chased noise. Under saturation, rounding-level changes in the score were outranking a candidate that closes 180 paths for one edit. Below a noise threshold, candidates are now ranked by the risk mass they close per cost.
Our own projects taught the rest: ranking 86 candidates over 1,249 paths went from 310 s to 0.4 s by re-scoring only touched paths; four of seven projects couldn't be analysed until unpinned requirements became "pin this to unlock analysis"; and two --exclude patterns, for a backup and a vendored copy, cut 9,336 call sites to 3,952.
What the score is not
- Not a probability of being breached. It's an ordering: this project against its past, this fix against that one. The weights are written-down judgements, not measurements.
- Only as wide as the analysis. That's what the coverage figure and the interval are for.
- Saturated on large projects. Read open paths and risk mass there.
Vulnerability data tells you what is already known about your packages. Following the call is our attempt at the part that will never be known that way, and the score is how a list of those findings becomes a plan.
Top comments (0)