DEV Community

jaryn
jaryn

Posted on

Reproduce a Secret Leak From a pytest Log You Fed to a Model

#security #ai #python #devops

The job is red. Chat is already open.

A teammate pastes the entire GitHub Actions log into a model and types "why did test_login fail?" Line 847 is not a puzzle. It is a credential. sqlalchemy.exc.OperationalError: password authentication failed for user "ci_app" sits on top of a truncated DSN: postgresql://ci_app:s3cret-from-gha@db.internal:5432/app. Nobody exploited the runner. The prompt did.

That is the invariant I want enforced before any model call: if the payload still matches a secret pattern, the request does not leave the workstation. Not "the model will be careful." Not "we trust this vendor today." A failed gate.

I am not reporting a CVE. This is a lab walkthrough with positive and negative fixtures. Treat every command below as a template you run locally, not as evidence I found a live leak.

The trust boundary that actually moved

Ask yourself a blunt question. Where did the secret stop being CI configuration and start being model input?

It was not the database. It was not even the Actions log until someone copied it. The boundary moved at paste. If you wired an agent to gh run view --log, it moved at tool invocation.

CI runner  --log artifact-->  developer workstation
                                  |
                                  |  unredacted paste / tool read
                                  v
                             prompt payload  -->  model API or shared server
                                  |
                                  v
                       provider logs, traces, queues you do not control
Enter fullscreen mode Exit fullscreen mode

Three assets sit on the wrong side of that arrow if you are sloppy: live credentials, internal hostnames, and customer-shaped data hiding in exception messages. The model is an untrusted processor. Act like it.

What never belongs in a prompt

I keep a deny list next to the editor. Not a vibe. A list.

  1. Connection strings, API tokens, Authorization headers, and anything that looks like AKIA, ghp_, xoxb-, or a PEM block.
  2. WAF or auth deny payloads that still contain cookies, session IDs, or raw request bodies.
  3. Production stack traces with SQL, LDAP filters, or home-directory paths that encode usernames.
  4. Agent traces that dump tool arguments. Those files are often worse than the original log.

Would you email that file to a vendor you have never contracted? If the answer is no, it is not a prompt either.

Lab fixture: fail the build when the prompt still has a DSN

Pinned for this write-up: Python 3.12 and pytest 8.3. Unexecuted until you run it. I want a gate boring enough to live in CI.

Directory layout:

prompt_gate/
  fixtures/negative_pytest.log
  fixtures/positive_pytest.log
  redact_gate.py
  test_redact_gate.py
Enter fullscreen mode Exit fullscreen mode

1. Negative fixture

This is the failing request I want to reproduce every time someone "just pastes the log."

# fixtures/negative_pytest.log
E   sqlalchemy.exc.OperationalError: (psycopg2.OperationalError)
E   connection to server at "db.internal" port 5432 failed: FATAL:  password authentication failed for user "ci_app"
E   [SQL: SELECT 1]
E   DSN=postgresql://ci_app:s3cret-from-gha@db.internal:5432/app
Enter fullscreen mode Exit fullscreen mode

2. Positive fixture

Same failure. No secret.

# fixtures/positive_pytest.log
E   sqlalchemy.exc.OperationalError: (psycopg2.OperationalError)
E   connection to server at "[REDACTED_HOST]" port 5432 failed: FATAL:  password authentication failed for user "[REDACTED_USER]"
E   [SQL: SELECT 1]
E   DSN=[REDACTED_DSN]
Enter fullscreen mode Exit fullscreen mode

3. Gate script

Regex is not DLP. I know. It is a regression fixture for the exact leak I keep seeing in pytest output.

#!/usr/bin/env python3
"""Fail if a prompt payload still looks like a secret. Lab fixture, not a product scanner."""
from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

PATTERNS = [
    ("dsn", re.compile(r"(?i)(?:postgres(?:ql)?|mysql|mongodb)://[^\s]+")),
    ("pem", re.compile(r"-----BEGIN (?:RSA )?PRIVATE KEY-----")),
    ("bearer", re.compile(r"(?i)authorization:\s*bearer\s+\S+")),
    ("github_pat", re.compile(r"ghp_[A-Za-z0-9]{20,}")),
    ("aws_access", re.compile(r"AKIA[0-9A-Z]{16}")),
]

def scan(text: str) -> list[str]:
    hits = []
    for name, rx in PATTERNS:
        if rx.search(text):
            hits.append(name)
    return hits

def main() -> int:
    p = argparse.ArgumentParser()
    p.add_argument("path")
    args = p.parse_args()
    text = Path(args.path).read_text(encoding="utf-8", errors="replace")
    hits = scan(text)
    if hits:
        print(f"FAIL {args.path}: unredacted patterns {hits}", file=sys.stderr)
        return 2
    print(f"PASS {args.path}")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

4. Tests that prove both directions

Positive must pass. Negative must fail. If both pass, the gate is theater.

from pathlib import Path
import subprocess
import sys

ROOT = Path(__file__).resolve().parent
GATE = ROOT / "redact_gate.py"

def run(name: str) -> int:
    return subprocess.run(
        [sys.executable, str(GATE), str(ROOT / "fixtures" / name)],
        check=False,
    ).returncode

def test_negative_fixture_fails():
    assert run("negative_pytest.log") == 2

def test_positive_fixture_passes():
    assert run("positive_pytest.log") == 0
Enter fullscreen mode Exit fullscreen mode

Expected evidence when you run it:

python redact_gate.py fixtures/negative_pytest.log
# FAIL fixtures/negative_pytest.log: unredacted patterns ['dsn']
# exit 2

python redact_gate.py fixtures/positive_pytest.log
# PASS fixtures/positive_pytest.log
# exit 0

pytest -q test_redact_gate.py
# ..  [100%]
Enter fullscreen mode Exit fullscreen mode

If the negative fixture ever starts returning 0, you did not improve the developer experience. You deleted the alarm.

Wire it in front of the model, not after

A gate that runs after the chat returns is a diary. Put it on the path that builds the prompt.

# Unexecuted CI sketch. Pin your own runner image.
set -euo pipefail
python redact_gate.py "$PROMPT_FILE"
# only then: your agent or curl to a model endpoint
Enter fullscreen mode Exit fullscreen mode

I also scan agent trace dumps. Why? Because tool-calling agents love to echo arguments. A "helpful" cat .env in a trace file is the same leak with extra JSON.

find . \( -name '*.trace.json' -o -name 'agent-log-*.txt' \) | while read -r f; do
  python redact_gate.py "$f"
done
Enter fullscreen mode Exit fullscreen mode

Tradeoff, stated plainly: this will false-positive on demo DSNs in docs, and it will false-negative on wrapped lines and custom token formats. That is acceptable for a regression fixture. It is not acceptable as your only control.

Prevent, detect, recover

Layer Prevent Detect Recover
Pre-prompt gate Refuse payloads that match PATTERNS CI job exit 2 on the negative fixture Do not send; rewrite the log
Secret store Short-lived CI creds, no long-lived DSN in logs gitleaks or trufflehog on the same file Rotate the leaked secret the same hour
Agent harness Deny tools that read .env, id_rsa, kubeconfigs Review trace files before anyone shares them Wipe the conversation and the workspace disk
Shared or free server Synthetic data only Egress allowlist Rebuild the box; assume disk was hostile

Notice what is missing from that table. "Ask the model if the log looks sensitive." That is not a control. The model is the destination you are protecting against.

Where a free model and a free server actually help

I use cheap, throwaway capacity to exercise the gate. Not to hold production logs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source AI development platform. Operator-supplied, and current as of this draft: it offers free model access and a free server option. I am not going to invent model names, token quotas, hardware SKUs, or how long a server lasts. Those change. The invariant does not.

The useful workflow is narrow. Spin a scratch workspace. Drop in negative_pytest.log. Confirm the gate fails. Confirm a redacted prompt is the only thing that would ever be sent. Then throw the workspace away. If you cannot describe the delete step, you are not in a lab. You are in a second production.

Do not put a real DATABASE_URL on a free server to "see what the model says." That inverts the whole point. If you want a disposable box to run this fixture against synthetic logs, the open-source project is one place to try that loop.

Who should not use this approach

Regex will miss custom tokens, base64 blobs, and secrets split across wrapped lines. If you handle regulated data, this fixture is a unit test, not your DLP program. Keep models off that data entirely.

Also skip this if your agent already has unrestricted filesystem tools and you refuse to deny them. The gate on PROMPT_FILE will not save you from cat ~/.aws/credentials two steps later. Fix the tool policy first.

I will not claim this reproduces a vendor-side retention bug. I have no evidence for that here. The leak I can reproduce is on our side of the paste buffer.

The boundary question

Which invariant belongs in CI, and which layer should enforce it?

For me the CI invariant is mechanical: negative fixture fails, positive fixture passes, prompt file is scanned before the network call. The enforcement layer is the agent harness and the runner, not the model's terms of service.

If your answer is "we will be careful when we paste," you already know how this story ends. The log is still red. The DSN is still in the buffer. Are you going to send it anyway?

Top comments (0)