DEV Community

Ayi NEDJIMI
Ayi NEDJIMI

Posted on

Building a Dependency Vulnerability Scanner for Python Projects

#python #security #devops #tutorial

Most Python projects accumulate dozens of third-party packages before anyone thinks to audit them. Each pinned version is a potential CVE waiting to be exploited. Knowing which packages carry known vulnerabilities — and being able to act on that programmatically — is not optional in 2026.

This post shows you how to build a dependency vulnerability scanner from scratch: parse requirements.txt, query the OSV (Open Source Vulnerabilities) API, and generate an actionable report you can wire into any CI/CD pipeline.

Why Not Just Use pip-audit or Safety?

Tools like pip-audit and safety are reasonable starting points, but both have real friction in practice:

  • pip-audit resolves packages against an installed environment — useless when you want to scan a lockfile in a container build stage without installing anything first.
  • safety requires a paid plan for anything beyond basic checks, and its CVE database can lag behind OSV.
  • Both produce output in formats that resist integration into custom pipelines (Slack alerts, Jira tickets, SBOM feeds).

Building your own scanner means controlling the data source, the output format, and the integration points. The OSV API is free, maintained by Google's open-source security team, and covers the PyPI ecosystem comprehensively.

Step 1 — Parse the Requirements File

We start by reading requirements.txt and extracting package names with their pinned versions:

import re
from dataclasses import dataclass
from pathlib import Path
from typing import Optional

@dataclass
class Dependency:
    name: str
    version: Optional[str]

def parse_requirements(filepath: str) -> list[Dependency]:
    deps = []
    for line in Path(filepath).read_text().splitlines():
        line = line.strip()
        # Skip comments, blank lines, and pip options (-r, --index-url, etc.)
        if not line or line.startswith("#") or line.startswith("-"):
            continue
        # Match name==version, name>=version, name[extra]==version, etc.
        match = re.match(
            r"^([A-Za-z0-9_\-\.]+)(?:\[[^\]]+\])?\s*([=><!\^~]+)\s*([0-9][^\s,;#]*)?",
            line
        )
        if match:
            name = match.group(1)
            op = match.group(2)
            # Only treat pinned versions (==) as authoritative for CVE matching
            version = match.group(3) if op == "==" else None
            deps.append(Dependency(name=name, version=version))
    return deps
Enter fullscreen mode Exit fullscreen mode

Unpinned ranges (requests>=2.25.0) still get collected — we query OSV without a version to surface all known vulnerabilities, then flag them as "unpinned" in the report.

Step 2 — Query the OSV API

OSV exposes a simple POST endpoint that accepts a package name, ecosystem, and optionally a version:

import httpx
import time
from typing import Any

OSV_ENDPOINT = "https://api.osv.dev/v1/query"

def query_osv(dep: Dependency) -> list[dict[str, Any]]:
    payload: dict[str, Any] = {
        "package": {
            "name": dep.name,
            "ecosystem": "PyPI",
        }
    }
    if dep.version:
        payload["version"] = dep.version

    try:
        resp = httpx.post(OSV_ENDPOINT, json=payload, timeout=10)
        resp.raise_for_status()
    except httpx.HTTPStatusError as e:
        print(f"[warn] OSV returned {e.response.status_code} for {dep.name}")
        return []
    except httpx.RequestError as e:
        print(f"[warn] Network error querying {dep.name}: {e}")
        return []

    return resp.json().get("vulns", [])

def scan_dependencies(
    deps: list[Dependency], delay: float = 0.1
) -> dict[str, list[dict]]:
    results: dict[str, list[dict]] = {}
    for dep in deps:
        vulns = query_osv(dep)
        if vulns:
            results[dep.name] = vulns
        time.sleep(delay)  # stay well under OSV's public rate limit
    return results
Enter fullscreen mode Exit fullscreen mode

The delay defaults to 100 ms — conservative enough for 200-package requirements files without being annoying. OSV's public rate limit sits around 10 req/s; we stay well below it.

Step 3 — Parse and Rank the Findings

Raw OSV responses are verbose. We extract CVE IDs, severity, and the earliest fixed version:

SEVERITY_ORDER = {"CRITICAL": 0, "HIGH": 1, "MEDIUM": 2, "LOW": 3, "UNKNOWN": 4}

def cvss_to_severity(score: float) -> str:
    if score >= 9.0:
        return "CRITICAL"
    elif score >= 7.0:
        return "HIGH"
    elif score >= 4.0:
        return "MEDIUM"
    elif score > 0:
        return "LOW"
    return "UNKNOWN"

def extract_finding(pkg_name: str, vuln: dict, pinned: bool) -> dict:
    aliases = vuln.get("aliases", [])
    cve_ids = [a for a in aliases if a.startswith("CVE-")]

    severity = "UNKNOWN"
    for sev in vuln.get("severity", []):
        if sev.get("type") in ("CVSS_V3", "CVSS_V4"):
            try:
                numeric = float(sev.get("score", "").split("/")[0])
                severity = cvss_to_severity(numeric)
            except (ValueError, IndexError):
                pass

    fixed_versions = []
    for affected in vuln.get("affected", []):
        for rng in affected.get("ranges", []):
            for event in rng.get("events", []):
                if "fixed" in event:
                    fixed_versions.append(event["fixed"])

    return {
        "package": pkg_name,
        "pinned": pinned,
        "osv_id": vuln.get("id"),
        "cve_ids": cve_ids,
        "severity": severity,
        "summary": vuln.get("summary", ""),
        "fixed_in": sorted(set(fixed_versions)),
    }

def generate_report(
    scan_results: dict[str, list[dict]], pinned_map: dict[str, bool]
) -> list[dict]:
    findings = []
    for pkg, vulns in scan_results.items():
        for vuln in vulns:
            findings.append(extract_finding(pkg, vuln, pinned=pinned_map.get(pkg, False)))
    return sorted(findings, key=lambda f: SEVERITY_ORDER.get(f["severity"], 5))
Enter fullscreen mode Exit fullscreen mode

Step 4 — CLI Entry Point with CI Gate 

import json, sys, argparse

def main():
    parser = argparse.ArgumentParser(
        description="Scan a Python requirements file for known CVEs via OSV"
    )
    parser.add_argument("requirements", help="Path to requirements.txt or requirements.lock")
    parser.add_argument("--output", choices=["json", "text"], default="text")
    parser.add_argument(
        "--fail-on",
        choices=["critical", "high", "medium"],
        default=None,
        help="Exit with code 1 if any finding is at or above this severity"
    )
    args = parser.parse_args()

    deps = parse_requirements(args.requirements)
    pinned_map = {d.name: d.version is not None for d in deps}
    print(f"[*] Scanning {len(deps)} packages ({sum(pinned_map.values())} pinned)...", file=sys.stderr)

    scan_results = scan_dependencies(deps)
    report = generate_report(scan_results, pinned_map)

    if args.output == "json":
        print(json.dumps(report, indent=2))
    else:
        if not report:
            print("[+] No vulnerabilities found.")
        for f in report:
            cves = ", ".join(f["cve_ids"]) or "no CVE alias"
            fixed = ", ".join(f["fixed_in"]) or "no fix available yet"
            label = "(pinned)" if f["pinned"] else "(UNPINNED)"
            print(f"[{f['severity']}] {f['package']} {label}")
            print(f"  {f['osv_id']} | {cves}")
            print(f"  {f['summary']}")
            print(f"  Fixed in: {fixed}\n")

    if args.fail_on and report:
        threshold = {"critical": 0, "high": 1, "medium": 2}[args.fail_on]
        blocking = [f for f in report if SEVERITY_ORDER.get(f["severity"], 5) <= threshold]
        if blocking:
            print(f"[!] {len(blocking)} finding(s) at or above '{args.fail_on}'. Failing build.", file=sys.stderr)
            sys.exit(1)

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Running it looks like:

$ python vuln_scan.py requirements.txt --fail-on high
[*] Scanning 47 packages (47 pinned)...
[CRITICAL] jinja2 (pinned)
  GHSA-h5c8-rqwp-cp95 | CVE-2024-56201
  Jinja2 allows an attacker to execute arbitrary code via crafted templates
  Fixed in: 3.1.5

[HIGH] cryptography (pinned)
  GHSA-3ww4-gg4f-jr7f | CVE-2024-26130
  NULL pointer dereference in PKCS12 serialization
  Fixed in: 42.0.4

[!] 2 finding(s) at or above 'high'. Failing build.
Enter fullscreen mode Exit fullscreen mode

Integrating into GitHub Actions

The exit code makes CI integration a one-liner. Here is a complete job definition:

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install httpx
      - name: Export JSON report
        run: python vuln_scan.py requirements.txt --output json > vuln-report.json
      - name: Fail on HIGH or CRITICAL
        run: python vuln_scan.py requirements.txt --fail-on high
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: vuln-report
          path: vuln-report.json
Enter fullscreen mode Exit fullscreen mode

The JSON artifact lets you post findings to Slack, create Jira issues, or feed a dashboard — without depending on a third-party integration. Running the scanner twice (once to export, once to gate) is intentional: it keeps the exit-code logic clean without mixing it into the report generation.

One practical note: unpinned dependencies produce noisy results because OSV returns all historical CVEs, not just ones affecting a specific version range. Lock every dependency (pip freeze > requirements.txt or use pip-compile from pip-tools). Pinning is the foundation of any dependency hygiene program — if you want a broader checklist that covers this alongside other Python application security controls, our free security hardening checklists include a dedicated Python track.

The Takeaway

The full scanner is about 130 lines of Python with a single runtime dependency (httpx). It handles the cases off-the-shelf tools miss:

  • Scans lockfiles without installing packages into an environment
  • Integrates cleanly into custom pipelines via JSON output and exit codes
  • Uses a free, authoritative data source (OSV) with no rate-limit anxiety at typical project sizes
  • Distinguishes pinned (precise CVE match) from unpinned (all-vulns, noisy) packages

Extend it with SBOM output, Slack webhooks, or a SQLite cache to avoid re-querying packages that haven't changed since the last scan. The skeleton is intentionally minimal so you can add exactly what your pipeline needs — nothing more.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Top comments (0)