CVE reachability with an LLM and a code graph

Asking a language model whether a CVE affects your code is easy. Deciding when not to believe the answer is the hard part.

On one real project we started with a hundred critical CVEs and went through them by hand. Seven or eight out of ten could not be reached from our code. That ratio comes from one codebase, so treat it as an anecdote. Anyone who runs a scanner knows the effect anyway: when everything is red, nobody opens the dashboard, and each CVE costs a developer an afternoon of reading advisories and grepping code.

vex-engine is the pipeline I built to give that afternoon back. It runs on GitLab CI, reads each CVE with an LLM on Amazon Bedrock, looks for it in the code with the graph that OWASP dep-scan builds, and commits a CycloneDX VEX verdict to the repository. Dependency-Track then shows only the CVEs that need a person.

There are two ways to get this wrong. A false positive costs a developer some time. A false negative hides a real vulnerability behind a confident not_affected, and preventing that is the reason for most of the design. So I follow one example through the pipeline, a Pillow CVE in an image decoder, and at every stage I ask the same question: could this stage hide a real vulnerability, and what stops it?

The code, the CI template and the lab app are on GitLab. Every snippet below comes from that repository.

The architecture in one paragraph

On every push, Trivy reads requirements.txt and writes a CycloneDX SBOM. Dependency-Track matches each component against NVD, GitHub and OSV and returns the CVEs. In parallel, OWASP dep-scan 6.3 reads the source and builds a code property graph with its atom engine. The analysis then takes one CVE at a time through three modules:

• Strategist (LLM call): reads the advisory and writes a search plan.
• Locator (no model): queries the graph.
• Auditor (second LLM call): judges the evidence.

The verdict goes into vex.json. Exploitable CVEs open one GitLab issue per component, and people close issues with labels that flow back into the VEX.

Why a regex scanner was not enough

The first version also had three phases, but the middle one was a Scanner. The Strategist wrote search patterns, the Scanner ran them as regular expressions over the files that imported the library, and the Auditor received thirty lines above and below every match.

Noise went down. Then it met this:

config = yaml.safe_load(open("settings.yml"))   # safe
data   = yaml.load(request.body)                # vulnerable

A text search finds both lines. The Auditor gets two blocks of code with no idea where request.body comes from or who calls the surrounding function. It has to guess, and on a bad day it guesses not_affected.

I replaced the Scanner with a Locator that queries a code graph. The Scanner stayed in the codebase with two smaller jobs, covered below. The division of labour stayed the same, and the rest of the post builds on it: the LLM reads the CVE at the start and judges facts at the end. The graph does the looking.

Step 1: turn the CVE text into a search plan with Bedrock

A CVE is written for people. "Heap overflow in the TIFF decoder when processing crafted images" does not tell you which function in your code triggers it, and no database does either. NVD used to add structured data such as scores and affected configurations to every entry; in April NIST announced it would stop enriching every CVE. Reading an advisory and mapping it to code used to need someone with security experience, every time.

The Strategist does that reading. It receives the CVE description and the package name and returns a plan as strict JSON. For CVE-2017-18342 (PyYAML before 5.1, yaml.load() can execute arbitrary code on untrusted data) the plan looks like this, with illustrative values:

{
  "trigger_type": "api_call",
  "vulnerable_symbols": ["yaml.load", "yaml.load_all"],
  "conditions": "the argument comes from untrusted input and no SafeLoader is passed",
  "safe_alternatives": ["yaml.safe_load", "yaml.safe_load_all"],
  "import_aliases": ["yaml"],
  "attack_vector": "a YAML document with a python/object tag executes code when loaded"
}

Each field has a consumer downstream. The Locator uses vulnerable_symbols and import_aliases (Pillow is imported as PIL, PyYAML as yaml). The Auditor uses conditions, attack_vector and safe_alternatives.

This is the first place a real vulnerability can disappear. If the Strategist names the wrong function, the graph looks for the wrong thing. This part of the prompt guards against it:

- vulnerable_symbols: ALL the PUBLIC functions/classes an application may call that
  lead to the vulnerable code, fully qualified from the TOP-LEVEL import name.
  If the bug is inside an internal decoder or C module (e.g. TiffDecode.c), name
  the public entry points that reach it (e.g. "Image.open" for image decoders).
  If it is in a class used internally by convenience functions, list those too
  (e.g. a bug in requests.Session redirects: "requests.get", "requests.post", ...).
  Missing an entry point hides a real vulnerability, so when unsure list more, not fewer.

In the Pillow example the bug lives in a C decoder that no application calls directly. Applications call Image.open, which reaches the decoder when it opens a TIFF. The prompt asks the Strategist to make that jump, from where the bug is to where your code touches it.

The plan also says how the bug fires:

• api_call: the application must call a specific function.
• framework_implicit: the library acts on its own. A CVE in Django's multipart parser fires on every upload before your view runs, and your code never names the parser.
• config: a setting turns the bug on.

The Strategist picks the type from the CVE text, and the type decides which question the Locator asks.

Nothing uses the answer before validation. The JSON must parse, the trigger type must be one of the three values, and symbol names are normalised. If Bedrock is unreachable, a keyword heuristic produces a fallback plan and the pipeline marks it as heuristic, because a guessed plan must never clear a CVE. Plans are cached per run, so each CVE/package pair goes to the model once.

A wrong symbol list is the cheapest way to hide a vulnerability. Ask for more entry points, never fewer, and record where each plan came from.

Step 2: match symbols against the dep-scan code graph

The Locator uses no model. It reads the graph that OWASP dep-scan builds with atom, which combines three views of the same code. The syntax tree knows that Image in your file is Pillow's module. The call graph knows who calls whom. The data flow knows where a value travels.

dep-scan exports the graph as two JSON slices. usages lists every call with its arguments and line. reachables lists flows that start from external input, such as an HTTP request. The CI job pins the image, because the slice format changes between atom versions, and writes a marker file only when the slices are complete:

vex_reachability:
  stage: analyze
  image:
    name: ghcr.io/owasp-dep-scan/dep-scan:v6.3.0
    entrypoint: [""]
  allow_failure: true
  script:
    - |
      if depscan -i "${VEX_APP_SRC}" -o "${CI_PROJECT_DIR}/depscan-out" -t python \
        --reachability-analyzer SemanticReachability \
        --explanation-mode EndpointsAndReachables \
        --vdb-scope app --no-banner \
        && [ -s depscan-out/python-usages.slices.json ]; then
        touch depscan-out/.complete
      fi

dep-scan exits 0 even when it produces no slices. Partial slices would report "library not used" for every file that failed to parse, and the analysis would close CVEs it never looked at. Without .complete, the Locator treats the graph as missing.

Matching is one small function. atom writes resolved calls such as PIL.py:<module>.Image.open. The Locator splits them into segments and compares from the end:

def symbol_matches(qualified: List[str], symbol: str) -> bool:
    sym = [s for s in _norm(symbol).replace("()", "").split(".") if s]
    if not sym or len(sym) > len(qualified):
        return False
    return qualified[-len(sym):] == sym

PIL.Image.open matches [pil, image, open]. yaml.load matches yaml.py:<module>.load and never safe_load, because the last segment differs. That fixes the case that sank the Scanner.

Pin the dep-scan image and check that the slices exist. A graph that failed silently looks exactly like a codebase that never uses the library.

Step 3: route each CVE by what the graph finds

The graph can give four answers, and each has a fixed next step.

1. The vulnerable function is called

For every call site the Locator collects the file and line, the origin of each argument, the external-input flows that cross the enclosing function, every path from an entry point to the call, and the source of each function on those paths. All of it goes to the Auditor. This is the Pillow case: Image.open is called at line 17 of upload_avatar.

2. The library is used, but not through the vulnerable function

This looks like good news, and it is the outcome most likely to hide a real problem, because the Strategist can still miss an entry point. Take a CVE in the redirect handling of requests.Session. The Strategist names requests.Session. The application only does this:

r = requests.get(user_supplied_url)

requests.get builds a Session internally, so the vulnerable code runs. Stopping at "not called" would be wrong. Before trusting it, the Locator runs three checks, one for each way "not called" could be false:

Check	Fails when	Why it matters
The symbols came from the LLM	Bedrock was down and the keyword heuristic produced them	a guessed list proves nothing
They are real, callable APIs	the plan names TiffDecode.c, a private _imaging module or a decoder plugin	no application calls a C file, so "not called" is trivially true
A text search agrees	grep finds ImageMath.eval( where the graph saw nothing	the graph has a hole: an unparsed file, an alias, a getattr

Any failure sends the CVE to in_triage. If all three pass, the Auditor receives every use of the library and one question: does any of these calls reach the vulnerable code internally?

3. The bug fires without a call

For framework_implicit and config there is no symbol to look for. If the library is imported, the Locator collects every place the application uses it and the Auditor decides whether the vulnerable feature is in use. On the lab app it sees upload_avatar reading request.FILES, so multipart uploads are in use.

If the library is not imported anywhere, the CVE goes straight to in_triage. A server, a plugin or a string in the settings can load a package without any import in your code.

4. The graph is missing

If dep-scan failed or the language is not Python, the old Scanner runs. If it finds uses, the Auditor judges. If it finds nothing, the CVE goes to in_triage, because without the graph "not found in the text" proves nothing.

The relationship between the graph and the regex comes down to one rule:

If the graph says "called", believe it. If it says "not called" or "not imported", the text has to agree. When they disagree, doubt wins.

Step 4: when the library is never imported, rebuild the tree from PyPI

For an api_call CVE, "never imported" sounds conclusive. It isn't, because the graph only sees your code. If Django uses sqlparse internally, a sqlparse CVE can be reached through Django, and the graph never looks inside Django.

An SBOM generated from requirements.txt has no dependency tree. The Locator rebuilds one from the requires_dist metadata on PyPI, breadth-first from the SBOM's components. On the lab app that is 34 packages in about two seconds.

PyPI cannot tell you whether Django calls the vulnerable function in sqlparse. It only tells you that Django declares a dependency on it. That is enough to know when a CVE must stay open:

Library	PyPI tree	Verdict
jinja2	direct, nothing else requires it	not_affected if no text search finds an import
sqlparse	required by Django	in_triage
private package	PyPI returns 404	in_triage

The jinja2 row is the only case the Locator closes on its own, without calling the LLM. Before it does, a regex looks for any import of the package, including dynamic ones through importlib or __import__.

Optional extras count as dependents, but the walk does not traverse them. Following every extra makes every library transitive (Pillow → docs extra → sphinx → jinja2) and nothing could ever be closed. The known trade-off: a package reached only through another package's extra, such as requests[security] → pyOpenSSL → cryptography, can look direct.

The graph sees your code. PyPI sees what your dependencies declare. Neither one is allowed to clear a CVE on a guess.

Step 5: let the Auditor judge evidence

The Auditor receives the Strategist's plan, the CVE description and a block of evidence built from the slices. The prompt tells it what kind of material it is reading:

EVIDENCE FROM THE CODE GRAPH (OWASP dep-scan). For every call site of the
vulnerable API you get: the arguments' origin, the external-input flows that
cross the enclosing function, every call path from an entrypoint, and the code
of every function on those paths. These are facts from data-flow analysis, not
guesses: the vulnerable API IS called at these lines.

For each call site decide:
1. Can attacker-controlled data (HTTP request, uploaded file, message, CLI arg)
   reach the call through one of the paths? Follow the arguments' origin.
2. Are the trigger conditions met (arguments, options, file format, settings)?
3. Is there validation or a safe alternative that blocks the attack?
The CVE is "affected" if at least one call site is exploitable.

A rule engine can't answer these questions. Is there validation before the call? Is the argument a constant or an upload? Does the file format match the CVE? Is yaml.load called with a SafeLoader? A person answers in seconds when looking at the right snippet. The graph's job is to put that snippet, and the path that leads to it, in front of the model.

The evidence is capped so prompts stay small: at most 8 call sites per CVE, 5 paths per call site, call depth 8, and 80 lines per function. That is enough to follow a request from a Django view to a library call in a typical web app.

The Auditor answers with strict JSON: a verdict that is exactly affected or not_affected, a confidence between 0 and 1, a reasoning that cites the path, and the call sites it considers exploitable.

The graph decides what the model sees. The model decides what it means.

What the Auditor cannot verify

In case 2 above (library used, vulnerable function not called) the Auditor is asked whether requests.get reaches Session internally. Nothing in the evidence answers that. The engine passes no library source, no documentation and no web search, so the model answers from training data.

For widely used libraries such as requests, Pillow, PyYAML and Django, that is usually right: the mapping between public functions and internal classes is stable and well documented. For a niche package, or a release newer than the model's training cutoff, it can be wrong. A smaller model knows less than a larger one.

Two things contain this. The confidence threshold sends an unsure answer to in_triage, which is what usually happens with a library the model barely knows. And every reasoning is written into the VEX and committed, where a reviewer can read and challenge it. A model that is confidently wrong passes the threshold; only the review catches it.

The next step is to replace memory with facts: have atom also build a graph of the installed library and show the Auditor the real path from requests.get to Session. vex-engine does not do this yet.

Step 6: fail closed

The final rule does not depend on the model:

# Fail-closed: None (doubt, error, low confidence) -> in_triage, stays visible
if is_reachable is None:
    state = "in_triage"
else:
    state = "exploitable" if is_reachable else "not_affected"

A verdict with confidence of 0.8 or more (VEX_MIN_CONFIDENCE) becomes exploitable or not_affected. Everything else becomes in_triage and stays on the dashboard for a person: a malformed answer, a lower confidence, a timeout, a missing package URL from Dependency-Track.

One failure mode only shows up over time. An LLM is not deterministic, and on an ambiguous CVE it can flip between not_affected and exploitable on every run, closing and reopening the same issue. The bot counts its own flips. After two (VEX_MAX_AUTO_FLIPS) it stops touching the issue, leaves it open and labels it vex::needs-human. A human decision resets the counter.

This rule is what makes a smaller model safe to use. I ran the whole pipeline with a 12B open model through Ollama on a laptop. It produced more in_triage verdicts than the Bedrock model and no hidden errors.

Worst case, vex-engine leaves you with the dashboard you already had.

One CVE, end to end

Back to the Pillow decoder CVE on the lab app (old Pillow, PyYAML and Django, with insecure code written on purpose).

The Strategist returns trigger_type: api_call with PIL.Image.open among the symbols, because that is the public function that reaches the decoder. The plan came from the model, and the symbol is a real API.

The Locator finds Image.open in the graph and builds the evidence. Simplified:

Call site 1: Image.open @ vuln_demo.py:17 (in upload_avatar)
Vulnerable symbol: PIL.Image.open
Arguments: img_file <- request.FILES['avatar']
Call paths (entrypoint -> call):
  <module> --(line 56)--> upload_avatar --(line 17)--> Image.open
Code of the functions involved:
  def upload_avatar(request):
      ...
      img_file = request.FILES['avatar']
      with Image.open(img_file) as img:

The Auditor does not have to find the call or trace the data. It only decides whether an attacker controls that file: it is a user upload, and nothing validates it before the decoder runs. Verdict affected, confidence above the threshold, call site vuln_demo.py:17.

The result is a VEX entry marked exploitable and a GitLab issue for the Pillow component with the file, the line, the origin of the argument and the path from the entry point. The developer who opens it can go straight to the fix.

A full run on the lab app looks like this. The app is vulnerable by design, so the exploitable share is far higher than in a real codebase:

Of 80 CVEs, 40 are closed with a written reason, 6 stay visible for a person, and 34 exploitable ones are grouped into two issues, one for Pillow and one for Django.

Write verdicts to vex.json and let people close the loop

Every verdict is written to vex.json in the application's repository, using standard CycloneDX fields, and committed with [skip ci] only when something changed:

{
  "id": "CVE-XXXX-YYYY",
  "analysis": {
    "state": "not_affected",
    "justification": "code_not_reachable",
    "response": ["will_not_fix"],
    "detail": "Vulnerability CVE-XXXX-YYYY analyzed by LLM Analyzer (python) on 2026-10-20. <Auditor reasoning>"
  }
}

A Dependency-Track trap

In the uploaded VEX, affects[].ref must stay the Dependency-Track project UUID (the BOM's metadata.component.bom-ref). If you rewrite it with the component's purl, Dependency-Track returns HTTP 200 and silently discards the verdict. The only trace is Unable to locate affected element in its logs. vex-engine keeps the purl in a separate property, vex:component-purl.

Closing labels

Issues open only for exploitable CVEs, one per component. People close them with a label that says why, and the pipeline turns the label into a VEX decision:

Label	VEX written	Expiry	What the bot does
vex::fixed	nothing forced, it's a claim	none	next analysis checks the graph; still reachable → issue reopens
vex::false-positive	false_positive	90 days	skipped until expiry, then back to in_triage
vex::mitigated	not_affected + protected_by_mitigating_control	90 days	same
vex::risk-accepted	exploitable + will_not_fix	90 days	stays visibly exploitable

If VEX_APPROVERS is set and someone else closes the issue, the bot reopens it. Two vex:: labels at once count as a conflict. A person reopening an issue revokes the decision. When a library is upgraded and its CVEs leave the SBOM, the issue closes by itself.

A decision that never expires becomes a blind spot. Ninety days is a setting. The expiry itself is the point.

Keep the analysis current with three triggers

A pipeline that runs only on push misses new CVEs on code nobody touches, and misses the decisions people make on issues. vex-engine runs on three triggers:

• Push or manual run. The full chain, from SBOM to verdict. On push only CRITICAL and HIGH CVEs go to the model (VEX_MODE=critical).
• Issue webhook. Closing or reopening a security issue, or adding a vex:: label to a closed one, runs vex_issue_sync --issues-only: no scan, no LLM, Dependency-Track updated in about a minute.
• Schedules. A nightly run syncs issues and expires decisions with no LLM. A weekly full analysis (VEX_MODE=full) catches new CVEs on unchanged code.

Two jobs write vex.json, so they share a GitLab resource group and run one at a time. Set its process mode to oldest_first to keep them in order. A project enables everything by including one template:

include:
  - project: 'demo-security/vex-engine'
    file: 'ci-template.yml'
    ref: main

stages: [scan, upload-bom, analyze, sync-vex]

What it costs on Amazon Bedrock

The LLM is called in two places, the Strategist and the Auditor, and only during an analysis. The SBOM, Dependency-Track, the graph, the webhook and the nightly sync make no model calls. The Strategist runs once per CVE in scope; the Auditor runs only for CVEs the graph does not settle on its own.

The default model is Amazon Nova 2 Lite. The task is narrow and structured (read text, return JSON), so a small model is enough. Switching model is one variable, and the same prompts run locally through Ollama, in which case no code leaves the machine:

LLM_PROVIDER=bedrock                         # or: ollama
BEDROCK_MODEL_ID=eu.amazon.nova-2-lite-v1:0  # default
OLLAMA_MODEL=gemma4:12b                      # local option
VEX_MIN_CONFIDENCE=0.8

A run costs cents on Bedrock and nothing locally. The real infrastructure cost is the graph: atom runs on the JVM and the dep-scan image is about 3.8 GB, so cache it on your runners.

What the analysis can't see

Each of these ends in in_triage, never in not_affected:

• Malware without a CVE. Out of scope.
• Bugs deep inside a C extension. The graph only sees the Python that calls into it.
• Argument-level data flow. Python data flow is tracked at function level: the graph knows that external input crosses a function, not which argument carries it. That's why the final judgement stays with the Auditor.
• Private packages. PyPI doesn't know them, so they can't be placed in the dependency tree.

Checklist

Component	Configuration	What it guarantees
Strategist prompt	all public entry points, more when unsure	the graph looks for every way in
Fallback plans	marked as heuristic	a guessed plan never clears a CVE
dep-scan job	pinned image, .complete marker	a failed graph is treated as missing
Symbol matching	segment suffix match	yaml.load never matches safe_load
"Not called" checks	LLM origin, real API, regex agreement	a hole in the graph ends in triage
PyPI tree	direct leaf only, extras not traversed	transitive libraries stay visible
Auditor evidence	8 sites, 5 paths, depth 8, 80 lines	small prompts with the right snippet
Verdict rule	VEX_MIN_CONFIDENCE=0.8, fail closed	doubt never hides a CVE
Anti-oscillation	VEX_MAX_AUTO_FLIPS=2, vex::needs-human	an unstable verdict goes to a person
VEX upload	keep affects[].ref as the project UUID	Dependency-Track keeps the verdicts
Labels	approvers, 90-day expiry	decisions are attributed and revisited
Writers	shared resource group	concurrent jobs don't drop decisions

Try it

Everything here ran with OWASP dep-scan 6.3.0, Dependency-Track 4.13 and Amazon Nova 2 Lite on Bedrock, on Python projects. The slice format changes between atom versions, so expect the matching code to move.

To run it on your own project: clone vex-engine, point VEX_APP_SRC at your source, add the include: block above to your .gitlab-ci.yml, and start with LLM_PROVIDER=ollama if you'd rather keep code on your machine for the first run. The lab app in the repo is the fastest way to see all four routing cases.