Build a CI Log Redaction Gate Before Incident Chat Hits a Model

You know the 2 a.m. deploy that dies on docker push? Someone in the incident channel will say it. "Paste the CI log into the model. It will find the registry error faster than we will."

I opened the artifact anyway. Line 184 was not a registry error. It was echo "DEPLOY_TOKEN=$DEPLOY_TOKEN" from a debug step that never got reverted. The value was still live. Would you have spotted it before the prompt left your laptop?

That is the failure. Not a generic AI scare. A trust boundary moved, and nobody put a gate on it.

The boundary you actually crossed

A CI log is not a stack trace. It is a serialized execution trace from a privileged runner. The runner already saw deploy tokens, registry passwords, cloud OIDC assertions, and every echo a human added under stress.

Three hops. Three different owners.

flowchart LR
  Runner["CI runner\n(privileged env)"] -->|stdout / annotations| Store["Log backend\nGitHub, GitLab, Jenkins"]
  Store -->|browser, gh, curl| Human["Engineer clipboard\nor agent file-read"]
  Human -->|prompt / tool result| Model["Model context\nremote or self-hosted"]

Ask the ugly question. If the model runs on a server you control, does that make the paste safe? No. It only changes who can read the prompt later. The token still entered a context window, a swap file, and maybe a debug transcript.

What must never enter the prompt

I treat a CI log as hostile input until a gate says otherwise. Hostile does not mean malware. It means the file was produced by a job that was allowed to see secrets you are not allowed to republish.

Do not send these classes of lines. Not to a SaaS chat box. Not to a self-hosted assistant. Not to an MCP tool that "summarizes the failure."

1. Deploy and registry secrets: DEPLOY_TOKEN=, docker login, _PASSWORD=, npm_, ghp_, github_pat_.
2. Cloud long-lived keys and session material: AKIA, ASIA, aws_secret_access_key, STS-looking blobs.
3. Bearer and OIDC leftovers: Authorization: Bearer, ghs_, GitHub Actions id-token JSON.
4. Embedded credentials in URLs: https://user:pass@, x-access-token:.
5. Masking failures: values that look like *** on one line and plaintext on the next because add-mask ran too late.

Is a compile error fair game? Usually. Is a failed kubectl apply fair game? Only after you strip kubeconfig fragments and --token flags. Different layer. Same rule: if the runner needed a secret to produce the line, the model does not.

Fixture layout

I want a regression gate I can run on a laptop, in CI, and in front of any assistant. No live GitHub org. No production tokens. Canaries only.

ci-log-gate/
  gate.py
  fixtures/
    positive_compile.log
    negative_deploy_token.log
    negative_docker_login.log
  expected/
    positive.exit  -> 0
    negative.exit  -> 1

Label up front: this is an unexecuted template as published. Run it yourself. Do not take my word for the exit codes. I am not claiming I found a vulnerability in GitHub Actions, GitLab, or anyone's cluster. I am claiming the paste path has no invariant unless you add one.

Pinned runtime: Python 3.12, stdlib only (re, sys, pathlib). No extra packages. No network.

Step 1 — write the two logs you will actually test

Positive fixture. A real failure, zero secrets.

# fixtures/positive_compile.log
::group::Build
npm error code ELIFECYCLE
npm error errno 1
npm error app@1.4.2 build: `tsc -p tsconfig.json`
src/auth.ts(88,12): error TS2339: Property 'exp' does not exist on type 'unknown'.
##[error]Process completed with exit code 1.
::endgroup::

Negative fixture. A canary, not a credential. If this file ever appears in a model log, you rotate the canary and you treat the transcript as compromised.

# fixtures/negative_deploy_token.log
::group::Push
+ echo DEPLOY_TOKEN=canary-rotate-me-not-real
DEPLOY_TOKEN=canary-rotate-me-not-real
+ docker login ghcr.io -u robot --password-stdin
Error response from daemon: denied: denied
::endgroup::

Second negative, different shape, same class.

# fixtures/negative_docker_login.log
Evaluating ssh-agent...
##[debug]login with token ghp_CANARY_NOT_A_REAL_TOKEN_0000
remote: Invalid username or password.
fatal: Authentication failed for 'https://ghcr.example.invalid/org/app.git/'

Why two negatives? Because one regex that only catches DEPLOY_TOKEN= will miss ghp_. Gates that pass a single demo are theater.

Step 2 — fail closed in about sixty lines

#!/usr/bin/env python3
"""gate.py — refuse CI logs that still look like secret material."""
from __future__ import annotations

import re
import sys
from pathlib import Path

# Conservative. Prefer false positives on incident paste.
RULES: list[tuple[str, re.Pattern[str]]] = [
    ("deploy_token_assign", re.compile(r"DEPLOY_TOKEN\s*=\s*\S+", re.I)),
    ("github_pat", re.compile(r"\b(?:ghp|gho|ghu|ghs|ghr|github_pat)_[A-Za-z0-9_]+")),
    ("aws_access_key", re.compile(r"\b(?:AKIA|ASIA)[A-Z0-9]{16}\b")),
    ("bearer_header", re.compile(r"Authorization\s*:\s*Bearer\s+\S+", re.I)),
    ("url_userinfo", re.compile(r"https://[^\s/@:]+:[^\s/@]+@", re.I)),
    ("docker_password_stdin", re.compile(r"docker\s+login\b.*--password-stdin", re.I)),
    ("npm_token", re.compile(r"\bnpm_[A-Za-z0-9]{20,}\b")),
    ("x_access_token", re.compile(r"x-access-token:[^\s]+", re.I)),
]


def findings(text: str) -> list[tuple[str, int, str]]:
    hits: list[tuple[str, int, str]] = []
    for i, line in enumerate(text.splitlines(), start=1):
        for name, pat in RULES:
            if pat.search(line):
                hits.append((name, i, line[:200]))
    return hits


def main(argv: list[str]) -> int:
    if len(argv) != 2:
        print("usage: gate.py <log>", file=sys.stderr)
        return 2
    path = Path(argv[1])
    text = path.read_text(encoding="utf-8", errors="replace")
    hits = findings(text)
    if hits:
        print(f"FAIL {path} ({len(hits)} hit(s))")
        for name, line_no, snippet in hits:
            print(f"  [{name}] L{line_no}: {snippet}")
        return 1
    print(f"PASS {path}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Expected evidence, after you run it:

python3.12 gate.py fixtures/positive_compile.log
# PASS fixtures/positive_compile.log  ; echo $?
# 0

python3.12 gate.py fixtures/negative_deploy_token.log
# FAIL ... [deploy_token_assign] L3 ...
# FAIL ... [docker_password_stdin] L4 ... ; echo $?
# 1

python3.12 gate.py fixtures/negative_docker_login.log
# FAIL ... [github_pat] L2 ... ; echo $?
# 1

If the positive fixture ever fails, you tightened a rule too far. If a negative fixture ever passes, the gate is a lie. That is the whole test plan.

Step 3 — put the gate in front of the paste, not after the model

I do not want a human to remember this. Memory is not a control.

# Unexecuted wrapper. Point it at a downloaded run log, not at prod.
redact_or_die() {
  local src="$1"
  python3.12 gate.py "$src" || {
    echo "refusing to copy $src into clipboard or prompt" >&2
    return 1
  }
  # Only if you still need a human paste path:
  # xclip -selection clipboard < "$src"
}

Optional GitHub CLI path, also unexecuted, also fail-closed:

# gh run view 123456789 --log > /tmp/run.log
# python3.12 gate.py /tmp/run.log || rm -f /tmp/run.log

If an agent can cat the log file, the gate belongs in the tool wrapper, not in a style guide. A prompt that says "please do not read secrets" is not an invariant. A non-zero exit is.

What a free server buys you — and what it does not

I needed a place to iterate the gate without feeding a raw incident log to a random public chat box. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source AI development platform; the operator-supplied facts I am using are free model access and a free server option. That is the entire product claim here. No model names. No quota. No hardware story.

Why does that matter for this fixture? Because the evaluation question is a trust-boundary question, not a feature-matrix question.

Destination	Shrinks provider blast radius?	Stops secrets in the prompt?	Still writes transcripts / swap?
Public chat paste	No	No	Yes, on someone else's disk
Remote hosted agent with workspace tools	No	No, unless you wrap tools	Yes
Assistant on a server you control	Yes, for the provider hop	No	Yes, on your disk
No model until gate.py exits 0	N/A	Yes, for this class of leak	Nothing to write

A free server is useful when you want the model inside a domain you can wipe. It is useless as a substitute for redaction. Self-hosted is a different trust domain. It is not a different physics.

If you already run a local runtime, use that for the same loop. The gate does not care which assistant you were about to paste into. It cares whether line 184 still contains a token.

Prevent / detect / recover

Layer	Prevent	Detect	Recover
Runner	Delete debug echo of secrets. Mask before first print. Prefer OIDC over long-lived PATs.	Job diff review: new echo $, set -x near login, curl -H Authorization.	Rotate the secret the job used. Invalidate the run log if your platform allows it.
Log store	Retention + access control on job logs. Disable public reuse of failed-run artifacts.	Alert on gate.py hits when logs land in an internal bucket.	Purge the artifact. Treat downstream copies as live.
Human paste	Wrapper refuses clipboard copy on exit 1.	Canary token unique to the incident channel.	If the canary appears in a model transcript, rotate and assume prompt-cache exposure.
Agent / MCP	Tool allowlist cannot cat **/*.log until the gate passes.	Tool-result scanner using the same RULES.	Drop the session. Wipe local transcripts on the server you control.
CI of the gate	Negative fixtures must fail. Pin Python 3.12.	gate.py on its own fixtures in PRs.	If a negative goes green, block merges. The invariant died.

Notice what is missing. There is no row that says "trust the model to ignore the secret." That row does not exist.

Limitations — who should not use this

This gate is a regex fence around a text log. That is a feature and a warning.

Do not use it as your only control if you are in a regulated environment that already requires a DLP product, customer-key encryption, or legal hold on CI artifacts. Regex will miss binary build logs, gzip-wrapped runner archives, multiline PEM blocks split every 64 characters, and secrets that were base64'd twice. It will also false-positive on documentation that discusses docker login --password-stdin. Good. Fail closed on incident paste. Tune later on a corpus you can actually share.

Do not use canary strings that resemble production token prefixes in a repo that publishes logs. canary-rotate-me-not-real is boring on purpose. Do not paste a live ghp_ into the negative fixture "just to be realistic." Realistic is how tokens escape.

Do not skip rotation because "it was only a self-hosted model." Disk, core dumps, and admin access still exist. If the line crossed into the context window, the secret's remaining lifetime is an incident-response question, not a product-tier question.

Which invariant belongs in CI?

Two invariants. They do not live on the same layer.

Invariant A: no job log that still matches RULES is eligible for clipboard, MCP, or agent file-read. Enforce that next to the assistant, or you will keep losing to 2 a.m. Slack advice.

Invariant B: the gate itself cannot go green on fixtures/negative_*.log. Enforce that in your CI, on every change to gate.py.

A is a runtime boundary. B is a regression boundary. If you only keep B, engineers will still paste the raw file. If you only keep A, someone will "fix" a noisy regex and silently disarm the control.

So: which layer should refuse the paste in your shop — the wrapper in front of the model, or the pipeline that produced the log? If the answer is "the model will be careful," you do not have an answer yet.

If you want a self-hosted workspace to exercise this gate against an assistant instead of a public chat box, MonkeyCode is open source and currently offers free model access plus a free server option. Run the negatives first. Then decide whether the model is allowed to see the file.