DEV Community: Alessio

Auditing MCP Configurations and AI Agent Rules for Security Risks

Alessio — Sun, 01 Nov 2026 21:39:14 +0000

Modern developer workflows increasingly rely on AI agent configurations and Model Context Protocol (MCP) servers. Tools like Claude Desktop, Cursor, and Windsurf execute local commands, connect to remote services, and read workspace instructions from files like .cursorrules or claude_desktop_config.json.

However, these configuration files can introduce serious security vulnerabilities into your environment:

Plaintext API keys and tokens committed to configuration files or exposed in local environments.
Unpinned runtime commands (such as executing npx or uvx packages without fixed versions), allowing upstream package updates to execute unreviewed code on your machine.
Overly broad filesystem access, such as giving an MCP server read and write permissions to root (/), home directories (~), or entire user profiles.
Unauthenticated remote endpoints that expose local agent tooling to network manipulation.
Instruction injection and hidden Unicode characters (such as zero-width spaces or bidirectional overrides) embedded within repository agent instruction files.

agentconfig_audit_mcp_and_rules_scanner.py is an audit script designed to inspect MCP configuration files and agent instruction rulebooks to flag these specific security issues.

Install & Usage

The tool is self-contained and relies entirely on Python's standard library (argparse, json, pathlib, re, unicodedata, dataclasses, and shlex). You do not need to run pip install for any third-party dependencies.

Installation

Download the script file and save it to your workspace as agentconfig_audit_mcp_and_rules_scanner.py. Ensure you have Python 3.8 or later installed.

Basic Usage

You can display the built-in command options by invoking Python directly:

python agentconfig_audit_mcp_and_rules_scanner.py --help

To audit an entire workspace for both MCP configurations and rule files (such as .cursorrules, agents.md, and markdown prompts), pass the directory path to the script:

python agentconfig_audit_mcp_and_rules_scanner.py .

You can also target specific standard system locations where tools like Claude Desktop or Cursor store their MCP definitions:

# Auditing a specific configuration file
python agentconfig_audit_mcp_and_rules_scanner.py ~/.cursor/mcp.json

How It Works

The audit tool inspects two main surfaces: structured JSON MCP server configurations and plaintext rule/instruction documents.

1. Detecting Broad Filesystem Access and Unpinned Packages

When parsing mcpServers configurations, the scanner evaluates command arguments against dangerous system roots and verifies if package runners include immutable version specifications:

# Check Overly Permissive Filesystem Paths
if isinstance(args, list):
    for arg in args:
        if not isinstance(arg, str):
            continue
        expanded = os.path.expanduser(arg)
        for pat in BROAD_PATH_PATTERNS:
            if re.match(pat, arg.strip()) or re.match(pat, expanded.strip()):
                findings.append(
                    Finding(
                        rule_id="MCP-BROAD-FILESYSTEM-ROOT",
                        title=f"Overly Broad Filesystem Access in '{server_name}'",
                        severity="CRITICAL",
                        file_path=str(file_path),
                        line_number=None,
                        description=(
                            f"Server '{server_name}' is assigned an expansive filesystem path '{arg}'. "
                            "Agents interacting with this server obtain arbitrary read/write access to user "
                            "home directories or root."
                        ),
                        evidence=f"arg: {arg}",
                        remediation="Scope the server access specifically to a dedicated subproject folder.",
                    )
                )

The script monitors paths matching expressions like ^/$, ^/home/?$, ^/Users/?$, ^~$, or root drives on Windows (^[A-Za-z]:\\?$). If an MCP server is configured to mount those locations, it raises a CRITICAL finding.

Similarly, the scanner evaluates whether npx or uvx executions pin their targets. If an argument lacks @<version> or ==<version>, the tool flags MCP-UNPINNED-SERVER to prevent automatic downstream execution of unpinned packages.

2. Identifying Hardcoded Secrets in Environment Blocks

The tool loops through every env key-value pair in each configured server. If a value contains known credential structures (such as GitHub personal access tokens, AWS keys, Slack tokens, or private key headers), or if keys containing TOKEN, KEY, or SECRET hold hardcoded strings longer than 11 characters, it generates an alert:

# Check Plaintext Credentials in 'env'
if isinstance(env, dict):
    for env_key, env_val in env.items():
        if not isinstance(env_val, str):
            continue
        # Skip template variables like ${VAR} or $VAR
        if (env_val.startswith("${") and env_val.endswith("}")) or env_val.startswith("$"):
            continue

        for regex, label in SECRET_REGEXES:
            if re.search(regex, env_val) or (
                any(s in env_key.upper() for s in ("TOKEN", "KEY", "SECRET", "PASSWORD", "AUTH"))
                and len(env_val) >= 12
            ):
                findings.append(
                    Finding(
                        rule_id="MCP-PLAINTEXT-SECRET",
                        title=f"Exposed Secret in Server '{server_name}' Env",
                        severity="CRITICAL",
                        file_path=str(file_path),
                        line_number=None,
                        description=(
                            f"Environment variable '{env_key}' contains a hardcoded plaintext secret ({label})."
                        ),
                        evidence=f"{env_key} = {env_val[:4]}...{env_val[-4:] if len(env_val) > 8 else ''}",
                        remediation="Extract credentials to the system shell environment or a secure local keyring and reference via variable expansion.",
                    )
                )
                break

If the value uses variable expansion syntax such as ${MY_SECRET} or $MY_SECRET, the scanner treats it as externalized and skips it.

3. Scanning Agent Rules for Hidden Unicode and Injection Directives

Files like .cursorrules, agents.md, and prompt.txt can contain hidden characters that alter model behavior without being visible in standard text editors. The scanner checks each line for:

Zero-width spaces (\u200B), non-joiners (\u200C), joiners (\u200D), and byte order marks (\uFEFF).
Bidirectional overrides (such as \u202E), which can visually invert displayed text while preserving a different underlying instruction.
Explicit instruction-override heuristics, such as phrases like ignore all previous instructions, do not follow safety guidelines, or injected XML tags like <system-instruction>.

Limitations

While this tool highlights high-risk patterns, it has specific design limitations based directly on its codebase:

Static Regex Heuristics: Secret detection and injection detection rely strictly on static regular expressions. A credential format not listed in SECRET_REGEXES or an obfuscated prompt injection string that avoids common phrasing will not be caught.
No Dynamic Variable Resolution: The scanner checks if an environment variable starts with $ or ${...} to avoid false positives. It does not inspect whether the parent shell environment securely populates those variables.
Hardcoded File Target Names: During workspace discovery, discover_agent_files() only checks for a predefined set of filenames (such as .cursorrules, agents.md, claude.md, and .rules extensions). Rules stored under custom or unexpected file extensions will be skipped unless explicitly passed as target paths.
JSON-Only MCP Config Support: The MCP configuration scanner assumes valid JSON formatting. It does not parse YAML, TOML, or alternative configuration formats used by non-standard orchestrators.

FAQ

Does the scanner transmit any found tokens or file contents to an external server?

No. The script runs entirely on your local machine using standard Python modules. It performs no network calls, uploads no telemetry, and outputs findings strictly to your local standard output stream.

Why does the script flag my npx or uvx command as high severity?

Commands like npx @modelcontextprotocol/server-filesystem download and run the latest published version by default. If that package is modified or compromised upstream, your agent environment executes that code automatically. The scanner requires pinning a specific version tag (e.g., npx @modelcontextprotocol/server-filesystem@0.1.0) to pass without a finding.

Can the scanner automatically redact secrets or remove invisible characters from my files?

No. The scanner is an auditing tool, not an in-place mutation tool. It reports the rule ID, offending file path, line evidence, and suggested remediation, leaving actual file modifications to the developer.

Full version and the rest of the toolkit: AgentConfig Audit MCP and rules scanner on PyForge. Product page: https://py-forge.com/products/agentconfig-audit-mcp-and-rules-scanner

How to Audit AI Crawler Traffic and Identify Spoofed Bots in Logs

Alessio — Sat, 31 Oct 2026 21:34:29 +0000

Unmanaged web scraping by artificial intelligence agents can consume server bandwidth, increase egress costs, and skew operational analytics. While legitimate crawlers identify themselves via their request headers, malicious scrapers often spoof those same headers to bypass rules or scrape protected assets under the guise of well-known automated systems.

Analyzing access logs manually across multiple servers is inefficient. You need a systematic way to parse server logs, match known crawler signatures, estimate hosting expenses tied to crawler bandwidth, and verify whether the IP addresses sending those headers actually belong to the claimed crawler networks.

The script aibot_ledger_crawler_log_reporter.py addresses this issue directly. It inspects web server access logs, identifies automated AI crawlers, performs Forward-Confirmed Reverse DNS lookups to catch spoofed clients, checks paths against your robots exclusion file, and generates server-level mitigation templates.

Install & Usage

The script relies entirely on standard modules included with Python, such as urllib.robotparser, socket, gzip, and json. You do not need to install any third-party libraries using pip.

To get started, save the source code into a file named aibot_ledger_crawler_log_reporter.py.

You can view the command-line help interface by running:

python aibot_ledger_crawler_log_reporter.py --help

Because the core processing logic is encapsulated in the AIBotLedger class, you can also import and execute the ledger directly within your own administrative scripts:

from aibot_ledger_crawler_log_reporter import AIBotLedger

# Initialize the ledger with custom bandwidth pricing and DNS verification
ledger = AIBotLedger(cost_per_gb=0.09, verify_dns=True)

# Ingest standard or compressed log files
ledger.ingest_file("/var/log/nginx/access.log")
ledger.ingest_file("/var/log/nginx/access.log.1.gz")

# Print the formatted terminal summary
print(ledger.generate_terminal_summary())

How It Works

The tool is organized around log parsing, pattern matching, DNS validation, and rules checking.

1. Dual Log Parsing and Compression Handling

The ingest_file function checks the file extension. If the target file ends in .gz, it transparently reads the stream via Python's built-in gzip module.

Log entries are processed by parse_log_line, which detects both standard Combined log entries (typical for Nginx and Apache) and structured JSON log lines (typical for Caddy):

def parse_log_line(line):
    """Parses standard Combined format or Caddy JSON logs."""
    line = line.strip()
    if not line:
        return None

    # Try Caddy JSON log format
    if line.startswith("{") and line.endswith("}"):
        try:
            data = json.loads(line)
            req = data.get("request", {})
            headers = req.get("headers", {})
            ua_list = headers.get("User-Agent", [""])
            ua = ua_list[0] if isinstance(ua_list, list) and ua_list else str(ua_list)

            ip = req.get("remote_ip") or req.get("client_ip") or ""
            if ":" in ip and not ip.startswith("["):
                # Handle host:port
                ip = ip.rsplit(":", 1)[0]

            return {
                "ip": ip,
                "time": data.get("ts", ""),
                "method": req.get("method", "GET"),
                "path": req.get("uri", "/"),
                "status": int(data.get("status", 200)),
                "bytes": int(data.get("size", 0) or 0),
                "ua": ua,
            }
        except Exception:
            pass

    # Try Combined / Nginx / Apache format
    m = COMBINED_LOG_REGEX.match(line)
    if m:
        d = m.groupdict()
        b = d["bytes"]
        bytes_val = int(b) if b.isdigit() else 0
        return {
            "ip": d["ip"],
            "time": d["time"],
            "method": d["method"],
            "path": d["path"],
            "status": int(d["status"]),
            "bytes": bytes_val,
            "ua": d["ua"],
        }
    return None

2. Signature Detection

The script maintains an internal collection of bot definitions named BOT_PROFILES. It matches user-agent values against regular expressions corresponding to entities such as GPTBot, ClaudeBot, Bytespider, CCBot, Google-Extended, PerplexityBot, Cohere-AI, and Diffbot. Each entry maps the signature to a classification category (such as Live Browsing, Dataset Mining, or Aggressive Scraper) and assigns legitimate root domains used for verification.

3. Spoof Detection with Forward-Confirmed Reverse DNS (FCrDNS)

Anyone can configure an HTTP client to send User-Agent: GPTBot. To prove authenticity, the script uses Forward-Confirmed Reverse DNS via verify_fcrdns:

def verify_fcrdns(ip, expected_domains, timeout=2.0):
    """Forward-Confirmed Reverse DNS (FCrDNS) check.

    Returns: 'VERIFIED', 'SPOOFED', 'UNVERIFIED', or 'SKIPPED'.
    """
    if not expected_domains:
        return "SKIPPED"
    if ip in _DNS_CACHE:
        return _DNS_CACHE[ip]

    # Don't resolve private or special addresses
    if ip.startswith(("127.", "10.", "192.168.", "172.16.", "172.17.", "172.18.", "172.19.", "172.20.", "172.21.", "172.22.", "172.23.", "172.24.", "172.25.", "172.26.", "172.27.", "172.28.", "172.29.", "172.30.", "172.31.")):
        result = "SKIPPED"
        _DNS_CACHE[ip] = result
        return result

    orig_timeout = socket.getdefaulttimeout()
    try:
        socket.setdefaulttimeout(timeout)
        try:
            hostname, _, _ = socket.gethostbyaddr(ip)
        except (socket.herror, socket.gaierror, socket.timeout, OSError):
            result = "SPOOFED"
            _DNS_CACHE[ip] = result
            return result

        hostname_lower = hostname.lower()
        domain_matched = any(hostname_lower.endswith(domain.lower()) for domain in expected_domains)
        if not domain_matched:
            result = "SPOOFED"
            _DNS_CACHE[ip] = result
            return result

        try:
            forward_ips = socket.gethostbyname_ex(hostname)[2]
            if ip in forward_ips:
                result = "VERIFIED"
            else:
                result = "SPOOFED"
        except (socket.gaierror, socket.timeout, OSError):
            result = "SPOOFED"

    finally:
        socket.setdefaulttimeout(orig_timeout)

    _DNS_CACHE[ip] = result
    return result

This verification process consists of three distinct checks:

It queries the reverse DNS record for the IP address to obtain a hostname.
It verifies that the returned hostname ends with one of the legitimate domain suffixes configured for that bot.
It performs a forward lookup on that hostname to ensure that the originating IP address is present in the resolved address list.

If any check fails, the record is flagged as SPOOFED and recorded for auditing.

4. Cost Calculations and Defense Export

By accumulating byte counts associated with verified and unverified crawlers, AIBotLedger multiplies total gigabytes transferred by the cost_per_gb parameter to provide an estimate of hosting bandwidth charges.

It also includes standalone helper functions—such as generate_nginx_snippets and generate_robots_snippet—to output firewall maps, rate-limiting directives, and exclusion lines targeted at the specific crawler names encountered in your logs.

Limitations

While practical for audits, the current code has several built-in constraints:

Synchronous DNS Resolution: The verify_fcrdns function runs synchronously using Python's standard socket module. Although lookups are saved to an in-memory dictionary cache (_DNS_CACHE), processing very large uncompressed logs containing thousands of unique IP addresses will experience delays due to network timeouts.
Rigid Log Formats: The parser handles standard Apache/Nginx Combined log patterns and Caddy JSON logs. If your server outputs custom access log layouts missing status codes, request sizes, or user-agent fields, lines will fail to match and will be silently ignored.
Hardcoded Profile List: The tool identifies only the twelve specific agents defined inside BOT_PROFILES. Unlisted crawlers, custom research bots, or newer AI agents will not be cataloged unless you manually add new entries to the profile list.
Robots Rule Interpretation: Compliance tracking relies on urllib.robotparser. It does not interpret non-standard directives like Crawl-delay or vendor-specific path expansions that are outside standard robot exclusions.

Frequently Asked Questions

Does running this script require elevated root privileges?

No. The script only reads log files from disk and performs standard outbound DNS resolution queries. As long as your operating system user has read permissions for the target log files, root or administrative access is not required.

Can the script process logs stored in .gz archives alongside plain text logs?

Yes. The ingest_file method inspects the file name. If it detects a .gz suffix, it uncompresses and reads the file stream via Python's gzip module without requiring prior extraction on disk.

Why does an IP show up as spoofed even if the user agent looks valid?

A request is marked as spoofed when the client supplies a known AI crawler user agent, but its IP address either has no reverse DNS entry, resolves to a domain outside the provider's authorized domain list, or fails forward verification when matching the hostname back to the originating IP address.

Full version and the rest of the toolkit: AIBot Ledger crawler log reporter on PyForge. Product page: https://py-forge.com/products/aibot-ledger-crawler-log-reporter

Guarding CI Pipelines with Dependency Release Age Gates

Alessio — Fri, 30 Oct 2026 21:27:33 +0000

When an attacker compromises a package maintainer's credentials or publishes a typo-squatted package to PyPI or npm, the malicious release is immediately visible to automated tooling. Continuous integration pipelines, automated upgrade bots, and routine builds that pull the newest pinned dependencies will fetch and execute that code right away.

In most cases, malicious releases are identified and yanked within a few days once security monitors and maintainers notice the compromise. However, if your automated builds pull new releases the moment they appear, your environment acts as the test bed during that dangerous early window.

A dependency release age gate solves this by enforcing a cooldown period. By auditing dependency manifests and lockfiles to verify that every pinned package has existed in the public registry for a minimum number of days, you introduce a buffer period where bad releases can be caught before your build consumes them.

The Problem

Development workflows frequently update lockfiles through automated bots or manual upgrades. When dependencies update immediately to a release published only hours earlier, the build takes on unvetted supply chain exposure.

Common threats during this early window include:

Compromised developer accounts publishing malicious minor or patch updates.
Fast package takeovers where existing package names are hijacked.
Accidental breakages or incomplete releases that get replaced shortly after publishing.

To mitigate this without completely halting updates, pipelines need an automated verification step: checking the upload timestamp of pinned dependencies against a cooldown policy. If an update is too young, the pipeline should flag it and recommend an older, verified version that meets the cooldown requirement.

Install & Usage

The tool is implemented as a self-contained Python script. It relies exclusively on the Python standard library, requiring no external packages from package managers.

Installation

Download the script file and place it in your project repository under the name depcooldown_dependency_release_age_gate.py.

The script runs on standard Python. If you are running Python 3.11 or later, it uses the built-in tomllib module to read TOML files. If you run an earlier Python release, it uses an internal fallback parser to process package definitions from lockfiles.

Command-Line Usage

To inspect the supported command-line options:

python depcooldown_dependency_release_age_gate.py --help

To run an audit against a Python requirements.txt file:

python depcooldown_dependency_release_age_gate.py requirements.txt

You can audit an npm lockfile by passing the path to your package-lock.json:

python depcooldown_dependency_release_age_gate.py package-lock.json

The script can also inspect modern Python lockfiles and manifests, such as poetry.lock, uv.lock, and pyproject.toml.

How It Works

The tool works through three main stages: parsing manifest files, fetching upload timestamps from registries, and evaluating ages against a cutoff window.

1. Extracting Pinned Dependencies

The tool inspects the structure of the input file to detect whether it belongs to the Python or npm ecosystem. It extracts package names and strictly pinned versions.

Here is the implementation from parse_requirements_txt:

def parse_requirements_txt(content: str) -> Dict[str, str]:
    """Extract strictly pinned dependencies (name==version) from requirements.txt."""
    pinned: Dict[str, str] = {}
    for raw_line in content.splitlines():
        line = raw_line.strip()
        if not line or line.startswith("#") or line.startswith("-"):
            continue
        # Strip trailing comments and environment markers
        if " #" in line:
            line = line.split(" #", 1)[0].strip()
        if ";" in line:
            line = line.split(";", 1)[0].strip()

        # Match name[extras]==version
        match = re.match(r"^([A-Za-z0-9_.\-]+)(?:\[[^\]]*\])?\s*==\s*([A-Za-z0-9_.\-+]+)$", line)
        if match:
            pkg_name = match.group(1).lower().replace("_", "-")
            version = match.group(2).strip()
            pinned[pkg_name] = version
    return pinned

For npm, it reads package-lock.json and extracts versions from the packages dictionary (used in modern lockfiles) or the legacy dependencies block. For Python project manifests, it reads dependencies from pyproject.toml, poetry.lock, or uv.lock.

2. Fetching Release Timestamps

The RegistryClient queries public registry metadata endpoints using standard library urllib.request.

For npm packages, it inspects the registry JSON metadata and reads the time object, which maps each released version to an ISO-8601 upload timestamp.
For PyPI packages, it reads the releases object and extracts the upload_time_iso_8601 or upload_time entry associated with the release files.

The client stores fetched metadata in an in-memory dictionary and can serialize it to a local JSON cache file to prevent redundant network lookups across runs.

3. Evaluating Cooldown Violations

The audit core compares the release timestamp against the required age window:

def audit_dependency(
    ecosystem: str,
    package: str,
    version: str,
    client: RegistryClient,
    cooldown_days: int,
    reference_time: datetime.datetime,
) -> AuditResult:
    """Audit a single pinned package version against the cooldown window."""
    history = client.fetch_release_history(ecosystem, package)
    cutoff = reference_time - datetime.timedelta(days=cooldown_days)

    if not history or version not in history:
        return AuditResult(
            package=package,
            ecosystem=ecosystem,
            current_version=version,
            upload_time=None,
            age_days=None,
            is_violation=False,
            suggested_version=None,
            reason="Unverified (version metadata not found in registry)",
        )

    upload_dt = parse_iso_timestamp(history[version])
    age = (reference_time - upload_dt).total_seconds() / 86400.0

    if upload_dt > cutoff:
        # Pinned version is in the cooldown window! Find newest safe release
        eligible_versions: List[Tuple[Tuple[int, int, int, str], str]] = []
        for ver_str, ts_str in history.items():
            try:
                candidate_dt = parse_iso_timestamp(ts_str)
                if candidate_dt <= cutoff:
                    parsed_v = parse_semver(ver_str)
                    eligible_versions.append((parsed_v, ver_str))
            except Exception:
                continue

If a package version was uploaded after the cutoff date, audit_dependency marks it as a violation. It then loops through all historical releases in the registry metadata, identifies those uploaded prior to the cutoff, sorts them using parse_semver, and returns the newest safe version as a recommendation.

Limitations

Before deploying the script into your automation, keep the following structural limitations in mind:

Strict Pinning Requirement: The parser only evaluates exact equality pins (==). Flexible ranges such as >=1.2.0, ~=2.1, or caret dependencies in manifests are ignored unless resolved into a concrete lockfile like poetry.lock or package-lock.json.
Simplified Version Parsing: Version comparison relies on parse_semver, which splits versions into major, minor, patch, and prerelease components. Complex PEP 440 specifications (such as post-releases, epoch numbers, or local versions) are parsed as zero-fallbacks or plain integers rather than strictly compliant PEP 440 objects.
Minimal TOML Fallback: When running on Python versions older than 3.11 without tomllib, the script falls back to a basic line-by-line parser. While this fallback handles standard [[package]] entries in Poetry and uv lockfiles, complex or nested TOML structures may fail to parse.
Registry Unavailability: If the registry metadata endpoint is unreachable, timed out, or blocked by a proxy, the tool returns an unverified status for that dependency rather than aborting.

FAQ

Can this tool be run in an environment without internet access?

Yes. The RegistryClient supports an offline mode and local JSON cache files. If metadata was previously fetched and cached to disk, the script evaluates dependencies directly against the stored JSON cache without performing network calls.

How does the script handle npm scoped packages?

The script handles scoped npm packages (for example, @scope/package) by preserving the @ character while percent-encoding the path separator when querying the npm registry endpoint.

What happens if a brand-new package has no versions meeting the cooldown threshold?

If a package is entirely new and none of its released versions were published before the cutoff threshold, the tool flags the pinned version as a cooldown violation and leaves the suggested version empty, indicating that no safe historical release exists in the registry.

Full version and the rest of the toolkit: DepCooldown dependency release age gate on PyForge. Product page: https://py-forge.com/products/depcooldown-dependency-release-age-gate

How to Scrub API Keys from Local AI Coding Agent Transcripts

Alessio — Wed, 28 Oct 2026 10:08:37 +0000

Autonomous AI coding agents and terminal assistants have transformed everyday software engineering workflows. Tools like Claude Code, Cursor, Aider, Open Interpreter, and Gemini CLI read workspaces, execute commands, and help debug issues directly in development environments.

However, during active development sessions, developers routinely paste credentials, pass environment variables, or allow agents to inspect configuration files containing secrets. Even when those commands finish, the underlying transcripts and session logs are retained indefinitely on your local machine.

If your workstation is accessed, or if agent history folders are synced, backed up, or shared during collaboration, sensitive authentication material can be exposed. The AgentTrail Scrub tool inspects local AI agent session logs, identifies credentials using regular expressions and Shannon entropy, and helps with redacting them while keeping file backups.

The Problem: Persistent Credentials in Local AI History

Most developers do not realize how much sensitive state AI developer tools store on disk. When you run an agent or use an editor plugin, the tool writes comprehensive logs to record the conversation, tool execution calls, and terminal responses.

Common locations where local history accumulates include:

~/.claude/projects and ~/.claude/history for Claude Code sessions.
workspaceStorage folders under user configuration paths for Cursor.
~/.aider session logs.
~/.gemini logs for terminal-based Gemini tools.
~/.openinterpreter and ~/.codex local caches.

When an AI tool prints the output of an environment dump or inspects an uncommitted file, raw strings such as Anthropic keys, OpenAI tokens, AWS secret keys, GitHub personal access tokens, and JSON Web Tokens (JWTs) are embedded directly into session records on disk.

Simply clearing your shell history or adding .env to .gitignore does not touch these assistant logs. Cleaning these records manually across deeply nested directories is tedious and error-prone.

Install & Usage

The tool is self-contained and implemented entirely in Python with zero third-party library dependencies. It uses standard library modules such as argparse, pathlib, re, shutil, math, and collections. You do not need to install additional packages through package managers.

To install the tool, download the source file directly and save it to your machine as agenttrail_scrub_ai_transcript.py.

You can view all available command-line arguments and configuration options by running:

python agenttrail_scrub_ai_transcript.py --help

To try it safely on synthetic transcripts first, run the built-in demo:

python agenttrail_scrub_ai_transcript.py --demo

A typical flow is scan first, then redact (backups go to ~/.agenttrail/backups unless you pass --backup-dir):

python agenttrail_scrub_ai_transcript.py --scan
python agenttrail_scrub_ai_transcript.py --redact

Retention cleanup is a dry run by default; old transcripts are only deleted when you add --apply-retention:

python agenttrail_scrub_ai_transcript.py --retention-days 30
python agenttrail_scrub_ai_transcript.py --retention-days 30 --apply-retention

Because the script runs in user space using your standard Python interpreter, you can run it on macOS, Linux, or Windows environments where your AI coding assistants reside.

How It Works

The cleaner combines agent directory discovery, targeted pattern matching, entropy filtering, safe redaction, and optional retention cleanup.

1. Target Directory Discovery

Rather than forcing you to look up where each individual tool stores session state, the script inspects well-known workspace and history paths:

def discover_agent_transcript_dirs() -> List[Tuple[str, Path]]:
    """Locate common transcript, history, and workspace caches of AI coding agents."""
    home = Path.home()
    app_data = Path(os.environ.get("APPDATA", home / "AppData" / "Roaming"))

    candidates = [
        ("Claude Code Projects", home / ".claude" / "projects"),
        ("Claude Code History", home / ".claude" / "history"),
        ("Claude Code Root", home / ".claude"),
        ("Cursor Workspaces (Linux/macOS)", home / ".config" / "Cursor" / "User" / "workspaceStorage"),
        ("Cursor Workspaces (macOS AppSupport)", home / "Library" / "Application Support" / "Cursor" / "User" / "workspaceStorage"),
        ("Cursor Workspaces (Windows)", app_data / "Cursor" / "User" / "workspaceStorage"),
        ("Cursor Global Storage", home / ".cursor"),
        ("Gemini CLI", home / ".gemini"),
        ("Gemini Config", home / ".config" / "gemini"),
        ("Aider History", home / ".aider"),
        ("OpenAI Codex / CLI", home / ".codex"),
        ("Open Interpreter", home / ".openinterpreter"),
        # ... further candidate paths trimmed for brevity
    ]

    discovered = []
    for label, path in candidates:
        try:
            if path.exists() and path.is_dir():
                discovered.append((label, path.resolve()))
        except (PermissionError, OSError):
            continue
    return discovered

The tool identifies which directories exist on your machine, then gathers files ending in formats commonly used for logs and chats, such as .json, .jsonl, .log, .txt, and .md.

2. Pattern Matching with Shannon Entropy Filtering

Simple regular expressions frequently trigger false alarms when matching generic token assignments. To distinguish real cryptographic keys from low-randomness dummy values (such as test_token_12345), the tool calculates the Shannon entropy of candidate strings.

def calculate_shannon_entropy(data: str) -> float:
    """Compute the Shannon entropy of a string."""
    if not data:
        return 0.0
    entropy = 0.0
    length = len(data)
    counts = Counter(data)
    for count in counts.values():
        p_x = count / length
        if p_x > 0:
            entropy -= p_x * math.log2(p_x)
    return entropy

High-confidence signatures (like AWS access key IDs or structured GitHub tokens) have explicit prefix formats and can match without an entropy check. More generic strings, such as .env variable assignments, must exceed configured entropy thresholds before they are treated as valid secret hits.

3. Overlap Suppression and Safe In-Place Redaction

When an API key matches multiple detection patterns, the scanner suppresses overlapping spans to avoid duplicate notifications. When modifying a file, it preserves a backup before overwriting contents:

def redact_file(file_path: Path, hits: List[SecretHit], backup_dir: Path) -> Tuple[bool, Optional[Path]]:
    """Create a backup and rewrite the file with redacted tokens."""
    if not hits:
        return False, None

    try:
        safe_rel = str(file_path.resolve()).lstrip("/\\").replace(":", "_")
        target_backup = backup_dir / safe_rel
        target_backup.parent.mkdir(parents=True, exist_ok=True)
        shutil.copy2(file_path, target_backup)

        with open(file_path, "r", encoding="utf-8", errors="replace") as f:
            content = f.read()

        unique_secrets = sorted(list({h.raw_secret for h in hits}), key=len, reverse=True)

        for secret in unique_secrets:
            replacement = f"[AGENTTRAIL_REDACTED_{mask_secret(secret)}]"
            content = content.replace(secret, replacement)

        with open(file_path, "w", encoding="utf-8") as f:
            f.write(content)

        return True, target_backup
    except (PermissionError, OSError):
        return False, None

The redaction logic sorts detected secrets by length in descending order before performing replacements. This prevents shorter substrings of a key from being partially replaced before a complete key string is processed.

Limitations

Before incorporating this script into automated cleaning routines, keep these limitations in mind:

Large File Skip Limit: The scan_file function skips any file exceeding 100 MB. Very large log files or aggregated session archives above this threshold will not be processed.
Line-by-Line Text Handling: The script scans files using line-based string reading (open(..., "r", encoding="utf-8", errors="replace")). While .sqlite and .db file extensions are collected, treating database files as plain text can miss secrets split across database blocks, and modifying them via direct string replacement can damage database file formatting.
Specific Provider Coverage: Secret detection relies on the patterns defined in SECRET_PATTERNS. Keys from unlisted services or custom corporate authentication tokens will not be recognized unless they conform to generic key-value assignments.
Overlapping Match Suppression: When a regex matches an offset range already claimed by another pattern, subsequent matches within that character span are skipped.

Frequently Asked Questions

Does the script upload transcripts or telemetry to remote servers?

No. The script uses only local Python standard libraries and performs all directory traversal, scanning, and file rewrites locally on your machine.

What happens to my original files when redaction runs?

Before modifying any file containing a secret, redact_file copies the original file into a specified backup directory, mirroring the source path so you can review or restore the original content if needed.

How does the tool avoid flagging placeholder tokens in configuration files?

The detection engine evaluates minimum Shannon entropy thresholds on generic token matches. Repetitive strings, common words, and short sequences are discarded because their calculated randomness falls below the required threshold for cryptographic secrets.

AgentTrail Scrub is a single Python file that helps with finding and redacting secrets in local AI agent transcripts: AgentTrail Scrub on PyForge

How to Audit Python Environments for Malicious .pth Files

Alessio — Tue, 27 Oct 2026 10:08:09 +0000

Python startup mechanisms offer convenience for package setup, but they also present subtle opportunities for unwanted persistence. When Python boots, it inspects directory trees for path configuration files (.pth), as well as user and site customizer scripts. If an untrusted dependency or a compromised package drops files into these locations, code can execute every time the interpreter launches.

Auditing an environment requires inspecting these startup hooks safely. If you attempt to inspect a compromised environment by running Python scripts that import modules from it, you risk executing the very payloads you are attempting to analyze.

The PthGuard tool address this problem. It is a standalone, dependency-free scanner designed to perform static read and checksum verification against Python environments without importing or executing target files.

The Problem: Stealth Startup Hooks in Python

Python path configuration files typically contain directory paths added to sys.path. However, the Python interpreter supports a feature where lines beginning with import are evaluated directly during interpreter startup.

Attackers and compromised packages can exploit this behavior by embedding inline Python commands inside .pth files. For instance, a single .pth file can decode strings, spawn subprocesses, or establish network connections before your application code runs. Similar issues affect sitecustomize.py and usercustomize.py, which execute automatically whenever Python initialises if they reside within the module search path.

Additionally, standard package managers track installed files and checksums inside a .dist-info/RECORD file. Tampered packages might alter existing modules or add rogue unrecorded scripts directly to the package directory. Detecting these modifications requires verifying on-disk files against the signed or recorded standard hashes without running code from the packages themselves.

Install & Usage

The scanner relies exclusively on standard library modules (argparse, base64, csv, hashlib, json, os, pathlib, re, sys, and tempfile). It does not require any third-party libraries or installation frameworks.

Step 1: Save the File

Save the scanner code locally as pthguard_python_backdoor_scanner.py.

Step 2: Run the Scanner Under Safe Interpreter Flags

When running any security audit against an existing installation, you should isolate your running Python interpreter. Use the -I (isolated mode) and -S (skip site initialization) flags:

python -I -S pthguard_python_backdoor_scanner.py /path/to/target/site-packages

The -I flag tells Python to ignore environment variables such as PYTHONPATH and prevent the user site-packages directory from being loaded into memory. The -S flag disables the automatic import of the site module, preventing the interpreter from parsing rogue .pth files in your own scanning environment.

You can inspect specific directories, isolated virtual environments, or individual candidate files:

python -I -S pthguard_python_backdoor_scanner.py ~/.local/lib/python3.11/site-packages

How It Works

The scanner evaluates targets through purely static checks and checksum calculations, avoiding dynamic loading.

1. Pattern Matching on `.pth` Files

The scanner reads path files line by line using standard text parsing. It rejects execution and scans each line against common execution indicators:

PTH_DANGEROUS_PATTERNS = [
    (re.compile(r"^\s*import\s+[a-zA-Z0-9_]+", re.MULTILINE), "Direct code execution via 'import' statement"),
    (re.compile(r"exec\s*\(", re.IGNORECASE), "Executable 'exec()' call found"),
    (re.compile(r"eval\s*\(", re.IGNORECASE), "Dynamic 'eval()' call found"),
    (re.compile(r"base64\.(?:b64decode|decode)", re.IGNORECASE), "Base64 payload decoding detected"),
    (re.compile(r"subprocess\.", re.IGNORECASE), "Subprocess / command execution attempt"),
    (re.compile(r"os\.system\s*\(", re.IGNORECASE), "Direct OS shell execution"),
    (re.compile(r"__import__\s*\(", re.IGNORECASE), "Obfuscated __import__() call"),
    (re.compile(r"urllib\.request|urllib2|requests|socket", re.IGNORECASE), "Network / egress socket invocation"),
]

Any .pth file containing executable expressions or raw import directives triggers a critical alert with the exact line number and character sample.

2. Checking Customizer Scripts

The scanner checks for sitecustomize.py and usercustomize.py files in the target path. These files are matched against expressions associated with credential access (such as references to environment variables or key files), base64 decoding routines, socket creation, and process execution.

3. RECORD Integrity and Tamper Checks

Python wheels follow the PEP 376 format, which stores relative file paths, cryptographic hashes, and byte counts inside dist-info/RECORD. The scanner reads these records via the standard csv reader and computes SHA-256 hashes matching the standard format:

def calculate_file_hash(self, filepath: pathlib.Path) -> Tuple[str, int]:
    """Calculates sha256 in url-safe base64 unpadded (PEP 376 RECORD standard) and byte size."""
    hasher = hashlib.sha256()
    size = 0
    try:
        with open(filepath, "rb") as f:
            while chunk := f.read(65536):
                hasher.update(chunk)
                size += len(chunk)
        b64_hash = base64.urlsafe_b64encode(hasher.digest()).rstrip(b"=").decode("ascii")
        return f"sha256={b64_hash}", size
    except (OSError, PermissionError) as e:
        return f"ERR:{e}", 0

The script performs three checks:

Path Traversal Protection: Ensures that the relative paths inside RECORD do not resolve outside the target site-packages directory.
Hash and Size Validation: Confirms that on-disk files match the unpadded base64 SHA-256 hash listed in RECORD. Mismatches indicate altered code.
Unrecorded File Detection: Scans the .dist-info directory for unexpected files that are not referenced in the manifest.

4. Persistence File Locations

The scanner evaluates known operating system scratchpads and persistence paths (such as /tmp/.pth, /var/tmp/.pth, and autostart directory hooks) to determine if staging artifacts exist outside normal module trees.

Limitations

Understanding what static tools cannot do is essential for a thorough audit:

Regular Expression Evasion: The scanner uses pattern matches to detect suspicious operations. Custom variable indirection, character concatenation, or atypical encodings that do not match the compiled expressions will not be flagged by the pattern rules alone.
Untrusted RECORD Manifests: Hash validation verifies that files on disk match the RECORD file. If a completely malicious package was constructed from the start with matching hashes in its own RECORD, the hashes will validate cleanly. Hash checking only flags modifications made after initial package installation.
No Dynamic Execution Analysis: The tool does not execute files in an instrumented sandbox. Code paths, dynamically constructed calls, and network communication that occur during regular program flow are outside the scope of static source inspection.
Scope of Unrecorded Files: The tool checks for untracked files located directly inside the .dist-info directories. It does not perform a recursive inventory of every neighboring package subfolder to discover untracked helper scripts unless they are specifically mentioned in a RECORD manifest.

Frequently Asked Questions

Why run the scanner with `python -I -S`?

The -I and -S options isolate the interpreter. If the Python environment you are using to launch the scanner shares paths with an infected directory, running without these flags could cause Python to load malicious .pth or sitecustomize.py files before the scanner even starts.

Does the script automatically delete infected files?

No. The script explicitly avoids mutating or deleting files on disk. Instead, it aggregates findings into structured reports and produces quarantine instructions, leaving administrative review and remediation decisions entirely to the user.

Can I run the scanner offline?

Yes. The scanner operates completely offline. It relies exclusively on local file access, internal regular expressions, local checksum operations, and a bundled offline list of known bad releases. It does not query remote package indexes or network services.

Full version and the rest of the toolkit: PthGuard Python backdoor scanner on PyForge. Product page: https://py-forge.com/products/pthguard-python-backdoor-scanner

Find the slow span in your traces with a 600-line Python script

Alessio — Wed, 21 Oct 2026 00:35:34 +0000

You have a pile of exported spans, a p99 that jumped, and no time to click through a tracing UI one trace at a time. What you want is a quick answer to two questions: which operations are slower than their own normal, and which span in each bad trace is to blame?

TraceSift is a single Python file (standard library only) that answers that from a JSON file of spans.

Try it without your own data

# Generate 50 mock traces (about 200 spans) and analyze them
python3 tracesift.py --generate-mock --mock-count 50 -o spans.json
python3 tracesift.py -i spans.json

Real output from that run:

Total Ingested Spans: 206 across 50 traces
Total Anomalies Detected: 13
Root Causes Isolated: 9
------------------------------------------------------------------------------
TOP ROOT CAUSES IDENTIFIED:
  [ROOT_ERROR] Trace: trc-228bb000f -> Culprit: database-cluster::UPDATE inventory_qty (20.1ms, depth=2)
  [LATENCY_BOTTLENECK] Trace: trc-266da0018 -> Culprit: edge-gateway::GET /api/v1/checkout (808.3ms, depth=0)
------------------------------------------------------------------------------
TOP COST OPTIMIZATION OPPORTUNITIES:
  edge-gateway -> GET /healthz: vol=5.34% noise=95% rec_sample=1% | Saving: $0.79/mo

(--demo runs a built-in self-test and demonstration instead.)

What the input looks like

A flat JSON list of spans (or an object with a spans list). Field names are matched loosely, so trace_id/traceId, span_id/spanId, parent_span_id/parentId, service/serviceName, operation/name and duration_ms (or duration_us, duration_ns) all work:

[
  {"trace_id": "t1", "span_id": "a", "parent_span_id": null,
   "service": "edge-gateway", "operation": "GET /api/v1/checkout",
   "duration_ms": 127.4, "error": false},
  {"trace_id": "t1", "span_id": "b", "parent_span_id": "a",
   "service": "database-cluster", "operation": "UPDATE inventory_qty",
   "duration_ms": 20.1, "error": true}
]

It does not read Jaeger, Zipkin or OTLP export files natively. If your backend exports nested formats, flatten them to one object per span first (a few lines of jq or Python).

How it decides

Anomalies: spans are grouped by (service, operation). For each group it computes a robust mean and standard deviation, and a span is flagged when it is above that group's p95 and its z-score is at or above the threshold (default 2.5, change it with -z). Comparing each operation with itself avoids "the DB call is slower than the cache call" noise.
Root cause per bad trace: if the trace has failed spans, the culprit is the deepest failed span in the call tree (where the error started, not the gateway that reported it). Otherwise it is the longest span in the trace.
Sampling suggestions: health checks are suggested at 1%, stable error-free operations at 10%, and operations with more than 5% errors are never downsampled.

Reports for CI or a PR

python3 tracesift.py -i spans.json -z 3 --format markdown -o trace-report.md
python3 tracesift.py -i spans.json --format json -o trace-report.json

Limitations (read these)

The dollar "savings" are a rough estimate, not a bill: it assumes about 600 bytes per span, scales your sample to a monthly volume with a fixed multiplier, and uses --cost-per-gb (default $0.65). Use it to rank operations, not to promise numbers.
It works on a file you give it; it is not a live collector or a tracing UI.
Small samples make z-scores unreliable: with only a handful of spans per operation, expect few or no anomalies.

TraceSift is on PyForge: get TraceSift for $49, or get it together with every other PyForge tool, including future ones, with the All-Access Pass.

How to Find What's Bloating Your Deploy Zip (and Strip It)

Alessio — Mon, 19 Oct 2026 23:33:52 +0000

You zip a project folder to upload it to a server or a serverless function, and the archive is a lot bigger than the code you wrote. Usually the extra weight comes from files that never run in production: __pycache__ folders, .map source maps, old .log files, .git leftovers, and the same asset copied twice.

PackSlim is a small Python script that finds those files, shows how much space each kind takes up, and writes a cleaned zip, tar.gz or folder with only the remaining files. This tutorial shows what it really does, what its output looks like on a sample project, and where it can catch you out.

Install: one file, standard library only

PackSlim is a single .py file. It imports only the Python standard library (argparse, fnmatch, hashlib, zipfile, tarfile and a few others), so you don't need pip install. It uses the walrus operator, so you need Python 3.8 or newer.

Save the script as packslim.py and check the options:

python packslim.py --help

usage: PackSlim [-h] [-o OUTPUT] [-f {zip,tar.gz,dir}]
                [-p {standard,aggressive,python,node}] [-x EXCLUDE]
                [--no-dedup] [--dry-run] [--downloads DOWNLOADS]
                [--egress-cost EGRESS_COST] [--json] [--demo]
                [source]

If you want to try it before pointing it at real code, python packslim.py --demo creates a packslim_demo_sandbox/ folder with fake files and packs it into packslim_demo_output.zip.

Step 1: dry run on your project

I made a small sample project: a Flask-style app with a __pycache__ file, a 586 KB source map, a server log, a duplicated minified JS file, a .github/ workflow, a .pytest_cache/ and a .env. Then I ran an analysis only, with --dry-run so nothing gets written:

python packslim.py myapp --dry-run -x ".env*"

This is the real output (banner removed):

[*] Target Directory  : /tmp/ps/myapp
[*] Active Preset     : STANDARD
[*] Duplicate Purging : ENABLED (SHA-256)
[*] Scanning file hierarchies...

================================================================================
METRIC                                        | OPTIMIZATION VALUE
--------------------------------------------------------------------------------
Source Artifacts Scanned                      | 16 files
Original Footprint                            | 1.06 MB
Production Retained Footprint                 | 195.46 KB (8 files)
Total Dead Bloat Eliminated                   | 888.70 KB (8 files)
Redundant Duplicate Copies Pruned             | 1 duplicates (195.31 KB)
Overall Bundle Reduction Ratio                | 81.97% payload reduction
--------------------------------------------------------------------------------

=== BLOAT CATEGORY BREAKDOWN ===
Category                            | Count      | Payload Eliminated
------------------------------------------------------------------------
Build Logs & Dumps                  | 1          | 78.12 KB
VCS Artifacts                       | 2          | 15.00 B
Custom Excludes                     | 1          | 9.00 B
Caches & Bytecode                   | 2          | 29.30 KB
Source Maps & Debug Symbols         | 1          | 585.94 KB
Exact Duplicate                     | 1          | 195.31 KB
------------------------------------------------------------------------

The breakdown answers the question "what is bloating my bundle?" by category. In this sample, the source map and the duplicate JS file account for most of the extra weight.

Below that, PackSlim prints a "financial ROI estimate". Treat it as a rough calculation, not a measurement (see the "Limitations" section).

Step 2: write the slim archive

Add -o to write the output. The format is zip by default; -f tar.gz and -f dir are the other choices.

python packslim.py myapp -x ".env*" -o build/myapp-slim.zip

[OK] Optimized Package Ready: /tmp/ps/build/myapp-slim.zip
[OK] Final Output Footprint: 196.33 KB

The zip that came out contains:

main.py
requirements.txt
LICENSE
README.md
app/core.py
app/utils.py
static/app.min.js
tests/test_core.py

Your code, the minified JS and the metadata files are kept. The cache, the log, the source map, the .git/.github leftovers, the .env (excluded with -x) and the second copy of app.min.js are left out.

How it works

The scan is a plain os.walk. For each file, the name and every part of its relative path are checked against glob groups, and the first matching group decides the category:

for grp, patterns in self.pattern_groups.items():
    if self._matches_any(f, rel_p, patterns):
        is_bloat = True
        category = grp
        reason = f"Matched filter group '{grp}'"
        break

Matching runs against each path part, so a directory pattern like __pycache__* or .git* catches every file below that directory.

Files that don't match any pattern are hashed with SHA-256. The first copy of a file is kept, and any later file with the same content is marked "Exact Duplicate". --no-dedup turns this off.

Packaging just copies the files that aren't flagged:

keep_files = [f for f in files if not f.is_bloat]

The zip and tar.gz outputs use compression level 9.

Presets

standard: VCS folders (.git*, .svn*, .hg*), Python and tool caches, *.log/*.dump, source maps and .pdb, swap and PID files.
aggressive: everything in standard, plus tests and fixtures, docs and all *.md, *.txt and LICENSE* files, and design files (.psd, .fig, .sketch, .ai).
python: Python caches, *.egg-info, venv/.venv, .env*, test caches, .vscode/.idea.
node: npm/yarn caches, *.tsbuildinfo, *.map, docs, test files.

Presets don't stack. -p python doesn't remove .log, .map or .git files, and standard doesn't remove .env. To fill the gaps, use -x (you can repeat it), as in -x ".env*" above.

Using it in CI

--json prints the same statistics as structured data. The exit code is 0 whether or not bloat was found, so if you want a build to fail, check the report yourself. For example, to fail when the source tree contains a source map:

python packslim.py myapp --dry-run --json -x ".env*" > packslim-report.json
python - <<'PY'
import json, sys
report = json.load(open("packslim-report.json"))
breakdown = report["statistics"]["bloat_breakdown"]
maps = breakdown.get("Source Maps & Debug Symbols", {}).get("count", 0)
if maps:
    sys.exit(f"{maps} source map(s) found in the source tree")
print("bundle clean")
PY

On the sample project, this printed 1 source map(s) found in the source tree and exited with status 1.

Limitations (read before shipping)

The output must be outside the source folder. PackSlim refuses (exit code 2) if -o is the source folder, sits inside it, or is a parent of it. With -f dir, an existing output folder is replaced.
No per-file report and no "keep" list. The dashboard and the JSON only show counts by category. To see exactly what was removed, write with -f dir to a new folder and diff it against the source.
aggressive removes files you may need: requirements.txt (every *.txt), LICENSE, README.md, and any path part starting with test.
Duplicates are removed, not linked. If your code loads both copies by path, the second path will be missing.
The ROI numbers are an estimate. The egress figure is removed bytes × --downloads × --egress-cost. CI build count (1200/month) and network speed (120 Mbps) are fixed in the code. Set the two CLI values to your own numbers, or ignore that section.
.gitignore and .github/ count as "VCS Artifacts", which is fine for a deploy bundle but surprising the first time.

FAQ

Does it change my project folder?
No. The source is only read. The output has to be a path outside the source folder; with -f dir, an existing output folder is replaced.

Does it shrink the code itself, like minifying?
No. It works on whole files: each one is either kept or left out. It doesn't edit file contents.

Can I use it on a JavaScript project?
Yes. Matching is by file name, so -p node (or standard for .map and log files) works on any folder. It doesn't read package.json or resolve imports.

What does --demo do?
It creates ./packslim_demo_sandbox with sample files and writes ./packslim_demo_output.zip in the current folder.

Full version and the rest of the toolkit: PackSlim on PyForge. Product page: https://py-forge.com/products/packslim

Or get all 23 tools with the All-Access Pass for $99.

Detect Breaking OpenAPI Changes in CI with a Zero-Dependency Python Diff

Alessio — Mon, 19 Oct 2026 16:53:17 +0000

Breaking an API usually doesn't happen on purpose. Someone renames a field in a shared User schema, makes an optional request field required, or drops an old endpoint. The OpenAPI file changes by a few lines, the PR looks harmless, and a mobile client that's three versions behind starts failing. A plain text diff of openapi.json can't tell you which changes break clients. A semantic diff can.

SpecDiff is a single-file Python tool (standard library only) that compares two OpenAPI JSON documents and sorts every difference into BREAKING, WARNING or INFO. It knows direction: removing a field from a response breaks clients, while making a request field required breaks callers. With --fail-on-breaking, it exits with code 2 so CI can block the merge. Everything below was run locally.

Two versions of a spec

openapi.main.json is a small Users API with GET /users/{id} (returns #/components/schemas/User), POST /users (accepts NewUser) and DELETE /users/{id}/avatar.

The PR version, openapi.pr.json, does four things:

removes email from the User component schema;
makes name required in NewUser;
marks DELETE /users/{id}/avatar as deprecated;
adds GET /users.

Running it

python3 specdiff.py -b openapi.main.json -t openapi.pr.json --no-color --fail-on-breaking
echo "exit $?"

[SpecDiff Contract Verification Report]

Summary: 2 BREAKING | 1 WARNINGS | 1 INFO

[BREAKING] Property -> paths./users/{id}.GET.responses.200.application/json.schema.properties.email
         Property 'email' removed from response payload
[WARNING]  Deprecation -> paths./users/{id}/avatar.DELETE
         Operation DELETE /users/{id}/avatar has been marked as deprecated
[INFO]     Operation -> paths./users.GET
         HTTP Method GET added to /users
[BREAKING] Constraint -> paths./users.POST.requestBody.content.application/json.schema.required[name]
         Existing property 'name' marked as REQUIRED
exit 2

Both breaking changes were made inside components/schemas, not in the path definitions. SpecDiff follows the local $ref pointers (#/components/...) and reports each finding at the endpoint where clients will notice it, which is the useful place for a reviewer.

A purely additive change (a new optional avatar_url field and a new GET /users) passes:

Summary: 0 BREAKING | 0 WARNINGS | 2 INFO

[INFO]     Operation -> paths./users.GET
         HTTP Method GET added to /users
[INFO]     Property -> paths./users/{id}.GET.responses.200.application/json.schema.properties.avatar_url
         Property 'avatar_url' added (optional)
exit 0

What it checks

From the rules in the source:

Area	BREAKING	Non-breaking
Endpoints / methods	endpoint or method removed	endpoint or method added (INFO), operation deprecated (WARNING)
Parameters	parameter removed, new required parameter, optional → required	new optional parameter
Request body	body now required, media type removed, request property made required	media type added
Responses	status code removed, response media type removed, response property removed	new status code or media type, new optional property
Schemas	data type changed (`string` → `integer`)	—

Reports for PRs and tooling

-f markdown writes a report you can post as a PR comment:

**Status**: ❌ CONTRACT BREACH DETECTED

| Metric | Count |
| :--- | :--- |
| **Breaking Changes** | 2 |
| **Warnings / Deprecations** | 1 |
| **Informational** | 1 |

### 🚨 Breaking Changes
| Category | Target Path | Issue Description |
| :--- | :--- | :--- |
| `Constraint` | `paths./users.POST.requestBody...required[name]` | Existing property 'name' marked as REQUIRED |
| `Property` | `paths./users/{id}.GET.responses.200...properties.email` | Property 'email' removed from response payload |

-f json gives a summary plus one object per finding, with the old and new value:

{
  "summary": {"total_diffs": 4, "breaking": 2, "warning": 1, "info": 1},
  "divergences": [
    {"severity": "BREAKING", "category": "Property",
     "path": "paths./users/{id}.GET.responses.200.application/json.schema.properties.email",
     "message": "Property 'email' removed from response payload",
     "old_value": "email", "new_value": null}
  ]
}

In CI

Compare the spec on the PR branch with the one on main:

- name: API contract check
  run: |
    git fetch origin main --depth=1
    git show origin/main:openapi.json > /tmp/openapi.main.json
    python3 tools/specdiff.py -b /tmp/openapi.main.json -t openapi.json \
      --fail-on-breaking --no-color

If there's a breaking change the step fails with exit 2. Without --fail-on-breaking the report is informational and the exit code is 0. When a breaking change is intentional (a new major version), skip the check for that PR or compare against the new version's baseline.

Try it without your own spec

python3 specdiff.py --demo

The demo runs the engine against two built-in sample specs, so you can see the output format first.

Limitations

JSON only. For YAML specs, convert first, e.g. yq -o=json openapi.yaml > openapi.json.
Only local references (#/components/...) are resolved. Specs split over several files need bundling into one first (most OpenAPI tooling has a "bundle" command).
oneOf / anyOf / allOf compositions and enum value changes aren't analysed.
Constraint changes like maxLength, pattern or minimum aren't classified.
It reports changes. It doesn't check that your implementation matches the spec.

FAQ

OpenAPI 3.0 or 3.1? It reads the standard paths / components structure that both share. Swagger 2.0 (definitions) isn't the target.

Dependencies? None. Python 3 and the standard library.

Can I hide colours in logs? Yes, --no-color.

SpecDiff is a one-time purchase that includes the source and README: get SpecDiff ($19). More details are on the SpecDiff product page.

Find Your API's p99 Latency Spikes with a Zero-Dependency Python Benchmark

Alessio — Mon, 19 Oct 2026 04:41:52 +0000

Your API's median latency looks fine, but users keep reporting the occasional request that hangs for a second. Averages hide that. Percentiles show it, and you don't need k6, Locust or a JVM to measure them. ApiBench is one Python file, standard library only, that fires concurrent requests at an endpoint and reports p50/p90/p95/p99. It exits non-zero when your p99 or error-rate budget is blown, so it can gate a CI job.

All the numbers in this post come from runs on my machine against a small local test server. They show how the tool behaves; they aren't benchmarks of anything you run.

Install and first run

Unzip the download and check it runs (any recent Python 3, nothing to pip install):

python3 apibench.py --demo -n 100 -c 5

--demo starts a built-in mock server on 127.0.0.1:8765 that sleeps 15 ms per request, and benchmarks it:

 Completed Reqs:    100 in 0.332s
 SLA Gatekeeper:    [PASS]
 Throughput:        301.55 req/sec (14.36 KB/s)
 Success Rate:      100/100 (100.00%)
 Median (p50):         16.17 ms
 p99:                  18.06 ms  (Target SLA: < 250.0 ms)

The handler sleeps 15 ms, the median is 16 ms, and the p99 sits right next to it: about a millisecond of client and loopback overhead on top of the work. That's what a healthy tail looks like. The rest of this post is about the unhealthy kind.

A real investigation: where does a 1-second p99 come from?

I wrote a minimal JSON endpoint with Python's ThreadingHTTPServer and hit it with 500 requests from 20 workers, with a 50 ms p99 budget:

python3 apibench.py http://127.0.0.1:9000/health -n 500 -c 20 --sla-p99 50 --json r.json --markdown r.md
echo $?

 Completed Reqs:    500 in 1.236s
 SLA Gatekeeper:    [FAIL]
 Throughput:        404.58 req/sec (4.74 KB/s)
 Success Rate:      500/500 (100.00%)

--- Latency Profile (Wall Time) ---
 Min:                   0.90 ms
 Mean:                 28.71 ms
 Median (p50):          1.72 ms
 p90:                   2.98 ms
 p95:                   4.86 ms
 p99:                1023.13 ms  (Target SLA: < 50.0 ms)
 Max:                1212.12 ms
 StdDev:              164.09 ms

Exit code: 2. Every request succeeded, the median is under 2 ms, and yet the slowest 1% take about a second. A mean of 28 ms describes no request that actually happened.

A delay of about one second with no errors is the signature of TCP SYN retransmission. When a server's listen backlog is full, the kernel drops the new connection attempt, and the client retries it after the initial retransmission timeout of one second. socketserver defaults to request_queue_size = 5, and ApiBench opens a fresh connection for each request, so 20 workers overflow it. To test that theory, I changed one line in the server:

class S(http.server.ThreadingHTTPServer):
    request_queue_size = 128

Same command, same machine:

	backlog 5 (default)	backlog 128
Throughput	390.41 req/s	3696.94 req/s
p50	1.82 ms	4.91 ms
p95	6.61 ms	5.92 ms
p99	1029.44 ms	8.87 ms
Max	1237.26 ms	13.77 ms
Exit code	2	0

(This table comes from a second pair of runs, so the backlog-5 column differs slightly from the output above.)

The median went up a little, because the server now accepts more connections at once and they compete for the CPU. The tail collapsed, though, and throughput went up almost tenfold because clients stopped sleeping in retransmission. That's why you gate on p99 and not on the average.

Gating CI on latency and errors

The exit code is the whole integration:

0: p99 ≤ --sla-p99 (default 250 ms) and error rate ≤ 1%
2: either budget was exceeded
1: bad invocation, such as a missing URL

python3 apibench.py "$STAGING_URL/health" -n 300 -c 10 --sla-p99 200 --markdown apibench.md

Any status outside 200–399 counts as an error. I ran it against an endpoint that returns a random 503 about 10% of the time:

 Success Rate:      269/300 (89.67%)
 Error Rate:        10.33%
 p99:                  15.59 ms  (Target SLA: < 250.0 ms)

--- Status Code Breakdown ---
 Status 200  ->    269 ( 89.7%)
 Status 503  ->     31 ( 10.3%)

--- Failure Diagnostics ---
  - [31x] HTTP Status 503

Latency passed, errors didn't: exit 2. A slow endpoint fails the other way. Against a handler that sleeps 300 ms, p99 came out at 304.99 ms against a 250 ms budget, and the exit code was again 2.

--json writes every metric (latency_p99_ms, error_rate_pct, status_codes, overall_sla_passed and so on), which is useful for tracking trends between builds. --markdown writes a table you can post as a PR comment.

Other options: -m POST -d '{"q":1}' for a request body, -H 'Authorization: Bearer …' (repeatable) for headers, and -t for the per-request timeout. HTTPS works with the default certificate verification.

How it works

Each request is timed around a raw http.client call, so the measurement covers connect, send, the server's work and reading the full body:

start_time = time.perf_counter()
conn = http.client.HTTPConnection(host, port, timeout=self.config.timeout_seconds)
conn.request(self.config.method, path, body=body_bytes, headers=self.config.headers)
resp = conn.getresponse()
content = resp.read()
latency = (time.perf_counter() - start_time) * 1000.0

A ThreadPoolExecutor with -c workers runs -n of these. Percentiles use linear interpolation between the closest ranks rather than picking the nearest sample:

k = (len(sorted_values) - 1) * (percentile / 100.0)
f = math.floor(k); c = math.ceil(k)
if f == c:
    return sorted_values[int(k)]
return sorted_values[int(f)] * (c - k) + sorted_values[int(c)] * (k - f)

The gate is just p99 <= sla_p99_ms and error_rate <= sla_max_error_rate.

Limitations

A new connection for every request. There's no keep-alive, so the numbers include the TCP handshake (and the TLS handshake on HTTPS). That's realistic for clients that don't pool connections, but pessimistic compared with a browser or a pooled HTTP client.
Threads, not an event loop. It's fine for tens or low hundreds of concurrent workers. If you need tens of thousands of virtual users, use a distributed load tester.
Fixed request count, not a duration or a ramp. Each run sends -n identical requests, with no scenario scripting.
The "Efficiency Score" and cost projection in the report are simple formulas based on your SLA and the --cost-per-million you pass in. Use the percentiles and the exit code to make decisions.

FAQ

Can I benchmark production with it? Technically yes, but every request is real traffic. Point it at staging or a health endpoint, and keep -n and -c modest.

Why do my numbers differ from wrk or k6? Those tools reuse connections. ApiBench doesn't, so its latencies include the connection setup.

Does it need anything installed? No, only the Python standard library.

ApiBench is a one-time purchase that includes the single-file source and README: get ApiBench ($29). Details are on the ApiBench product page.

Mask PII in Logs, CSV and JSON with Python (and Keep the Data Joinable)

Alessio — Sun, 18 Oct 2026 16:35:08 +0000

Somebody needs a copy of production data to debug an issue, or a support ticket export has to go to a contractor, or your application logs are about to be shipped to a third-party service. Each time, emails, card numbers and phone numbers go along unless you strip them first. DataMask is a single Python file, standard library only, that masks that kind of PII in plain text, CSV and JSON. With a secret key it can replace values with stable tokens, so the masked data can still be joined and counted.

Everything below was run locally with the shipped datamask.py. The outputs are copied as-is.

What it detects

The built-in rules are regular expressions for:

Type	Example input	Default output
Email	`jane.doe@example.com`	`j******e@example.com`
Card (16 digits, with or without separators)	`4111 1111 1111 1111`	`**--**-1111`
US SSN	`123-45-6789`	`[SSN_REDACTED]`
North American phone	`415-555-0132`	`(415) ***-0132`
IPv4	`192.168.10.24`	`192.168.0.0`
`key=value` secrets (`api_key`, `secret`, `token`) with a value of 16+ chars	`api_key=sk_test_51Hx…`	`[GENERIC_SECRET_REDACTED]`

Masking free text (logs, tickets, notes)

printf 'Contact jane.doe@example.com or call +1 (415) 555-0132.\nCard 4111 1111 1111 1111, SSN 123-45-6789, IP 192.168.10.24\n' > in.txt
python3 datamask.py -i in.txt -k demo-key

Contact j******e@example.com or call +(415) ***-0132.
Card ****-****-****-1111, SSN [SSN_REDACTED], IP 192.168.0.0

The defaults keep enough to stay useful: the email domain, the last four digits of the card, the phone's area code and the /16 of the IP. To mask everything with one strategy, pass --strategy:

--strategy REDACT
Contact [EMAIL_REDACTED] or call +[PHONE_REDACTED].
Card [CREDIT_CARD_REDACTED], SSN [SSN_REDACTED], IP [IPV4_REDACTED]

--strategy TOKENIZE
Contact TOK_16978AC475DBF78A or call +TOK_407080C9ACCD2486.
Card TOK_7B8FCD949B6A4A3A, SSN TOK_39D12AE6849AA1EB, IP TOK_5052DA002CECEAC6

It also reads stdin and writes stdout, so it fits in a pipe:

tail -f app.log | python3 datamask.py -k "$DATAMASK_KEY" >> app.masked.log

Tokens you can still join on

TOKENIZE isn't random. It's an HMAC-SHA256 of the value under your key, truncated to 16 hex characters:

def tokenize(self, value: str) -> str:
    token = hmac.new(self.secret_key, value.encode("utf-8"), hashlib.sha256).hexdigest()
    return f"TOK_{token[:16].upper()}"

The same input with the same key always gives the same token, whether you pass the key with -k or through the DATAMASK_KEY environment variable:

echo "ada.park@example.com" | python3 datamask.py -k team-secret --strategy TOKENIZE
TOK_58D0726A2F40DC76
echo "ada.park@example.com" | DATAMASK_KEY=team-secret python3 datamask.py --strategy TOKENIZE
TOK_58D0726A2F40DC76

So an analyst can still count orders per customer, or join the masked users table to the masked orders table, without ever seeing an address. Without a key, DataMask generates a random one for each run, so tokens won't match across runs. Keep the key in your secrets manager, not next to the data.

CSV with per-column rules

For structured files you choose a strategy per column in a small JSON file:

{"email": "TOKENIZE", "name": "REDACT", "card": "PARTIAL", "phone": "PARTIAL"}

cat users.csv
id,name,email,phone,card,notes
1,Ada Park,ada.park@example.com,415-555-0101,4242 4242 4242 4242,VIP since 2021
2,Luis Ortega,luis@example.org,212-555-0199,5555-5555-5555-4444,called from 10.0.4.17

python3 datamask.py -f csv -i users.csv --rules rules.json -k team-secret

id,name,email,phone,card,notes
1,[CONFIDENTIAL],TOK_58D0726A2F40DC76,(415) ***-0101,****-****-****-4242,VIP since 2021
2,[CONFIDENTIAL],TOK_9E2DBA2E53D1D11F,(212) ***-0199,****-****-****-4444,called from 10.0.0.0

Columns without a rule still go through the text scanner, which is why the IP buried in notes came out as 10.0.0.0. The CSV is processed row by row with csv.DictReader, so large files don't need to fit in memory.

JSON, at any depth

echo '{"user":{"id":7,"email":"ada.park@example.com","note":"reach me at 415-555-0101"},"orders":[{"card":"4242424242424242"}]}' > ev.json
python3 datamask.py -f json -i ev.json -k team-secret --rules rules.json

{
  "user": {
    "id": 7,
    "email": "TOK_58D0726A2F40DC76",
    "note": "reach me at (415) ***-0101"
  },
  "orders": [
    {
      "card": "****-****-****-4242"
    }
  ]
}

The rules match on key names at any depth, and every other string value is scanned. Numbers and booleans are left alone. Notice that the email token is identical to the one in the CSV: same key, same value, same token.

Using it from Python

from datamask import DataMaskEngine, MaskingStrategy

engine = DataMaskEngine(hmac_key="team-secret")
print(engine.mask_text("refund to jane.doe@example.com"))
# refund to j******e@example.com
clean = engine.mask_structure(payload, {"email": MaskingStrategy.TOKENIZE})

That makes it easy to wire into a logging.Filter or a serializer before data leaves the process.

How it works

Each rule is a compiled regex, a default strategy and an optional format-preserving replacer. mask_text applies them in order:

for rule in self.rules:
    strategy = strategy_override or rule.default_strategy
    if strategy == MaskingStrategy.PARTIAL and rule.custom_replacer:
        sanitized = rule.pattern.sub(rule.custom_replacer, sanitized)
    elif strategy == MaskingStrategy.TOKENIZE:
        sanitized = rule.pattern.sub(lambda m: self.tokenize(m.group(0)), sanitized)
    ...

mask_structure walks dicts and lists recursively. Keys with a rule get that strategy; other strings go through mask_text.

Limitations

It matches patterns, it doesn't understand language. Names and street addresses in free text aren't detected. "Ada Park" stays as it is unless it sits in a column or key you've given a rule. Use field rules for structured data.
The phone and SSN rules are North American formats. Other countries' numbers and 15-digit Amex cards aren't covered by the default rules.
The secret rule looks for api_key=, secret: and token= style pairs and Bearer <token> values (for example Authorization: Bearer … headers), with tokens of 16+ characters. Other credential formats need a custom rule.
HASH is a plain SHA-256 with no key. For low-entropy values such as phone numbers, use TOKENIZE (keyed) instead.

FAQ

Is this GDPR/HIPAA compliance? No tool makes you compliant by itself. DataMask helps with data minimisation and pseudonymisation; whether that's sufficient depends on your data and your process.

Does it need dependencies? No, it's standard library only.

Can I add my own patterns? Yes. The rules are a list of PIIRule(name, sensitivity, pattern, default_strategy, custom_replacer) objects in the source, so you can append your own (an internal customer-ID format, for example).

DataMask is a one-time purchase that includes the source and README: get DataMask ($19). More details are on the DataMask product page.

Test Retries and Timeouts for Real: an HTTP Fault-Injection Proxy in Python

Alessio — Sun, 18 Oct 2026 04:26:57 +0000

Your retry logic, timeouts and circuit breakers only get tested when a dependency actually misbehaves, and that usually happens in production. A fault-injection proxy lets you cause that on purpose. Put it between your service and the dependency, then tell it to add 1.5 s of latency, return a 503 on 30% of requests, or hang up mid-request.

ChaosProxy is a small reverse proxy written in Python on aiohttp that does this. Rules are managed at runtime over a JSON API, so you can turn faults on and off from a test script without restarting anything. Everything below was run locally.

Setup

pip install aiohttp multidict
python3 chaosproxy.py --upstream http://127.0.0.1:5700 --port 8780 --host 127.0.0.1

Point your client at http://localhost:8780 instead of the real service. With no rules, requests pass straight through, including method, query string, body and headers (hop-by-hop headers are filtered out). For this post the upstream is a tiny API that returns an order:

curl -s localhost:8780/orders/1001
# {"id": "1001", "status": "paid"}

Fault 1: latency

curl -s -X POST localhost:8780/__chaos/rules -H 'Content-Type: application/json' -d '{
  "rule_id": "slow-orders",
  "path_pattern": "^/orders/",
  "fault_type": "latency",
  "params": {"delay_seconds": 1.5, "jitter_seconds": 0.3}
}'

{"status": "created", "rule": {"rule_id": "slow-orders", "path_pattern": "^/orders/", "method": "*", "probability": 1.0, "fault_type": "latency", "params": {"delay_seconds": 1.5, "jitter_seconds": 0.3}, "enabled": true}}

curl -s -o /dev/null -w "%{http_code} in %{time_total}s\n" localhost:8780/orders/1001
# 200 in 1.634716s

The request still reaches the upstream and returns 200, just late. That's how you find out whether your client's 1 s timeout really fires, and what the user sees when it does.

Remove the rule when you're done:

curl -s -X DELETE localhost:8780/__chaos/rules/slow-orders
# {"status": "deleted", "rule_id": "slow-orders"}

Fault 2: intermittent errors

probability makes a rule fire on a fraction of matching requests, and method limits it to one HTTP verb:

curl -s -X POST localhost:8780/__chaos/rules -H 'Content-Type: application/json' -d '{
  "rule_id": "flaky", "path_pattern": "^/orders/", "method": "GET",
  "probability": 0.3, "fault_type": "http_error",
  "params": {"status_code": 503, "body": {"error": "upstream unavailable"}}
}'
for i in $(seq 1 20); do curl -s -o /dev/null -w "%{http_code} " localhost:8780/orders/$i; done

503 200 200 200 200 200 200 503 200 200 200 200 200 200 503 200 503 200 503 200

5 of 20, close to the 30% asked for. This is the realistic case for testing retries with backoff. Failed requests return your JSON body with the status you chose, and they never reach the upstream.

Fault 3: dropped connections and truncated bodies

curl -s -X POST localhost:8780/__chaos/rules -H 'Content-Type: application/json' \
  -d '{"rule_id":"drop","path_pattern":"^/orders/9$","fault_type":"connection_drop"}'
curl -s localhost:8780/orders/9; echo "curl exit $?"
# curl exit 52   (empty reply from server)

payload_corrupt fetches the real response and then truncates it ("strategy": "truncate") or flips random bytes (any other strategy value):

curl -s -X POST localhost:8780/__chaos/rules -H 'Content-Type: application/json' \
  -d '{"rule_id":"trunc","path_pattern":"^/orders/7$","fault_type":"payload_corrupt","params":{"strategy":"truncate"}}'
curl -s localhost:8780/orders/7
# {"id": "7", "s

That's exactly what a JSON parser hates. Good for checking that your client reports a clear error instead of crashing.

Seeing what happened

curl -s localhost:8780/__chaos/rules     # active rules
curl -s localhost:8780/__chaos/stats

{"requests_total": 23, "requests_proxied": 18, "faults_injected": 6,
 "fault_breakdown": {"latency": 1, "http_error": 5, "payload_corrupt": 0, "connection_drop": 0}}

Every injected fault is also logged:

[WARNING] [ChaosProxy] INJECTING FAULT [HTTP_ERROR] via rule [flaky] on GET /orders/1

Using it in an automated test

Because rules are just HTTP calls, a test can switch a fault on, assert on behaviour, and clean up:

import requests

CHAOS = "http://localhost:8780/__chaos/rules"

def test_checkout_survives_slow_payments():
    requests.post(CHAOS, json={"rule_id": "slow-pay", "path_pattern": "^/payments",
                               "fault_type": "latency", "params": {"delay_seconds": 3}})
    try:
        r = requests.post("http://localhost:3000/checkout", json={"cart": 1}, timeout=10)
        assert r.status_code == 202          # we expect async fallback, not a 500
    finally:
        requests.delete(f"{CHAOS}/slow-pay")

How matching works

Each rule has a regex path_pattern matched against the path plus query string, a method (* by default), a probability between 0 and 1, and an enabled flag. Rules are checked in the order they were created, and the first match wins. Paths under /__chaos are reserved for the control API and never forwarded.

Limitations

The control API has no authentication. Anyone who can reach the port can add rules. The default bind address is 0.0.0.0, so pass --host 127.0.0.1 (as above) or keep it on an isolated test network. Don't expose it publicly.
Rules live in memory. Restarting the proxy clears them, so re-create them from your test setup.
One upstream per proxy instance. Run several instances on different ports for several dependencies.
payload_corrupt reads the whole upstream response into memory before corrupting it, so it isn't meant for very large downloads. Normal forwarding streams.
HTTP only. There's no TCP-level chaos (packet loss, bandwidth limits) and no TLS termination. Put it on the plain-HTTP side of your stack.

FAQ

Does it change responses when no rule matches? No. Status, headers and body are relayed as received, including compressed bodies.

What's the default upstream? https://httpbin.org. Always pass --upstream for real use.

Python version? Python 3 with aiohttp and multidict.

ChaosProxy is a one-time purchase that includes the source and README: get ChaosProxy ($29). More details are on the ChaosProxy product page.

A Tamper-Evident Audit Log in Python and SQLite: Hash Chains, Verification and Exports

Alessio — Sat, 17 Oct 2026 16:19:41 +0000

Sooner or later someone asks "who changed this, and can you prove the log wasn't edited?" If your audit trail is a normal table, the honest answer is no. Anyone with write access to the database can UPDATE a row, and nothing will look different afterwards.

A hash chain is the usual way to make edits detectable. Each record stores the SHA-256 of its own contents plus the hash of the record before it. Change any field in any row and its hash no longer matches. Delete a row and the next record points at a hash that isn't there anymore.

This post walks through AuditLog, a single-file Python tool (standard library only) that keeps that kind of chain in SQLite. Everything below was run locally against the shipped file. Only the IDs and timestamps will differ on your machine.

Recording events

Each event has an actor, an action, a resource, a status and arbitrary JSON metadata:

python3 auditlog.py --db ledger.db record \
  --actor user_42 --action document.sign \
  --resource doc_981 --resource-type contract \
  --metadata '{"ip": "10.0.0.7"}'

{
  "event_id": "9577713c-7243-4e0f-835c-b787e096d48c",
  "timestamp": "2026-10-20T00:21:20.788253+00:00",
  "actor_id": "user_42",
  "actor_type": "user",
  "action": "document.sign",
  "resource_id": "doc_981",
  "resource_type": "contract",
  "status": "SUCCESS",
  "metadata": {"ip": "10.0.0.7"},
  "previous_hash": "0000000000000000000000000000000000000000000000000000000000000000",
  "event_hash": "aa10ac7caf7b5bd18f5af000f4189d899a2d636e8f56672c0042b3d76ff93150"
}

The first record links to an all-zero genesis hash. I added two more: a service creating an invoice, and a denied privilege escalation:

python3 auditlog.py --db ledger.db record --actor svc_billing --actor-type service \
  --action invoice.create --resource inv_2026_114
python3 auditlog.py --db ledger.db record --actor user_7 --action role.grant \
  --resource rbac:admin --status DENIED --metadata '{"reason": "not_in_group"}'

--status accepts SUCCESS, FAILURE or DENIED.

Listing and filtering

python3 auditlog.py --db ledger.db list

Retrieved 3 events:
[2026-10-20T00:21:20.788253+00:00] user_42 -> document.sign on doc_981 [SUCCESS] (Hash: aa10ac7caf7b...)
[2026-10-20T00:21:20.843074+00:00] svc_billing -> invoice.create on inv_2026_114 [SUCCESS] (Hash: cb076a346079...)
[2026-10-20T00:21:21.189529+00:00] user_7 -> role.grant on rbac:admin [DENIED] (Hash: bf448e23db4f...)

--actor, --action and --limit narrow it down. For example, list --actor user_7 returns only the denied grant.

Verifying the chain and catching an edit

python3 auditlog.py --db ledger.db verify; echo "exit $?"

[PASS] Cryptographic audit chain verified. Zero tampering detected.
exit 0

Now play the insider and make the denied request look approved, straight in SQLite:

UPDATE audit_trail SET status='SUCCESS' WHERE actor_id='user_7';

[FAIL] Audit chain verification failed! Tampering detected:
  - Tampered Payload at sequence 2 (Event ID: 7039abbf-...). Stored: bf448e23db4f886c..., Computed: 72319ad9d48b1a54...
exit 2

The exit code is the useful part. 0 means intact and 2 means tampered, so verify can run as a nightly cron or CI job that alerts on anything non-zero. Deleting a row from the middle or the start is caught too, and reported as a Broken Link because the next record's previous_hash no longer matches.

How the hash is built

The hash covers a canonical JSON serialisation of every field, including the previous hash:

def compute_hash(self) -> str:
    payload = {
        "event_id": self.event_id, "timestamp": self.timestamp,
        "actor_id": self.actor_id, "actor_type": self.actor_type,
        "action": self.action, "resource_id": self.resource_id,
        "resource_type": self.resource_type, "status": self.status,
        "metadata": self.metadata, "previous_hash": self.previous_hash,
    }
    # serialised with sorted keys, then SHA-256

Sorted keys keep the bytes stable, so the same event always gives the same hash, on any machine.

Appending is done in one SQLite write transaction (BEGIN IMMEDIATE). The tool reads the current tip and inserts the new record before another writer can get in. I ran 8 processes writing 40 events each into the same file at the same time. The result was 320 rows, and verify passed. On this box, 500 sequential writes from Python took about 0.3 s.

Using it from Python

from auditlog import AuditLogger

audit = AuditLogger("ledger.db")
audit.record("user_42", "user", "invoice.refund", "inv_2026_114", "invoice",
             "SUCCESS", {"amount_cents": 4900})
ok, problems = audit.verify_integrity()

Exporting for a review

python3 auditlog.py --db ledger.db export --out audit.csv --format csv
python3 auditlog.py --db ledger.db export --out audit.jsonl --format jsonl

Both include previous_hash and event_hash, so a reviewer can re-check the chain without your database.

Limitations

Tamper-evident, not tamper-proof. The hashes are plain SHA-256 with no secret key. Someone with write access to the file and this source code could rewrite a row and recompute every hash after it. The standard defence is anchoring: copy the latest event_hash somewhere the database admin can't edit, like a daily line in a separate log, a ticket or an object-lock bucket. Then compare it during review.
Deleting the newest records isn't detectable from the chain alone, because nothing points forward to them. Anchoring the tip solves this as well.
It's SQLite, so all writers must share one database file on a local disk. It isn't a multi-node log service.
There's no built-in date-range filter on list. Use the export, or query the audit_trail table directly.

FAQ

Does this make me SOC 2 / ISO 27001 compliant? No tool does that by itself. It gives you an append-only audit trail with integrity checks and exports, which is evidence auditors commonly ask for.

Can I add my own fields? Put them in metadata. It's part of the hash.

What Python version? Python 3.9+ and the standard library. There's nothing to pip install.

AuditLog is a one-time purchase that includes the source and README: get AuditLog ($29). More details are on the AuditLog product page.

DEV Community: Alessio

Auditing MCP Configurations and AI Agent Rules for Security Risks

Install & Usage

Installation

Basic Usage

How It Works

1. Detecting Broad Filesystem Access and Unpinned Packages

2. Identifying Hardcoded Secrets in Environment Blocks

3. Scanning Agent Rules for Hidden Unicode and Injection Directives

Limitations

FAQ

How to Audit AI Crawler Traffic and Identify Spoofed Bots in Logs

Install & Usage

How It Works

1. Dual Log Parsing and Compression Handling

2. Signature Detection

3. Spoof Detection with Forward-Confirmed Reverse DNS (FCrDNS)

4. Cost Calculations and Defense Export

Limitations

Frequently Asked Questions

Guarding CI Pipelines with Dependency Release Age Gates

The Problem

Install & Usage

Installation

Command-Line Usage

How It Works

1. Extracting Pinned Dependencies

2. Fetching Release Timestamps

3. Evaluating Cooldown Violations

Limitations

FAQ

Can this tool be run in an environment without internet access?

How does the script handle npm scoped packages?

What happens if a brand-new package has no versions meeting the cooldown threshold?

How to Scrub API Keys from Local AI Coding Agent Transcripts

The Problem: Persistent Credentials in Local AI History

Install & Usage

How It Works

1. Target Directory Discovery

2. Pattern Matching with Shannon Entropy Filtering

3. Overlap Suppression and Safe In-Place Redaction

Limitations

Frequently Asked Questions

Does the script upload transcripts or telemetry to remote servers?

What happens to my original files when redaction runs?

How does the tool avoid flagging placeholder tokens in configuration files?

How to Audit Python Environments for Malicious .pth Files

The Problem: Stealth Startup Hooks in Python

Install & Usage

Step 1: Save the File

Step 2: Run the Scanner Under Safe Interpreter Flags

How It Works

1. Pattern Matching on .pth Files

2. Checking Customizer Scripts

3. RECORD Integrity and Tamper Checks

4. Persistence File Locations

Limitations

Frequently Asked Questions

Why run the scanner with python -I -S?

Does the script automatically delete infected files?

Can I run the scanner offline?

Find the slow span in your traces with a 600-line Python script

Try it without your own data

What the input looks like

How it decides

Reports for CI or a PR

Limitations (read these)

How to Find What's Bloating Your Deploy Zip (and Strip It)

Install: one file, standard library only

Step 1: dry run on your project

Step 2: write the slim archive

How it works

Presets

Using it in CI

Limitations (read before shipping)

FAQ

Detect Breaking OpenAPI Changes in CI with a Zero-Dependency Python Diff

Two versions of a spec

Running it

What it checks

1. Pattern Matching on `.pth` Files

Why run the scanner with `python -I -S`?