Audit Prompt Egress Before a Remote Model Sees Your Internal 403

I did not mean to export the deny path.

I asked a remote model why an agent retried a 403 three times. The paste looked like a stack trace. It was not. Buried in the tool result was https://billing.svc.prod.internal:8443/v2/invoices, a request id that maps onto our edge, and a fragment of the deny body we return to bots. The model did not need any of that. I sent it anyway.

That is a trust-boundary violation, not a clever exploit. You can reproduce it with a classifier and two fixtures. Should a free coding model ever see your internal 403 body? No. Can you prove the prompt was clean before the HTTP client fired? Most harnesses cannot.

The failure sequence

Here is the order I keep seeing in agent debug sessions. None of it requires a vulnerability in the model.

1. The agent calls an internal tool. The tool returns 403 plus a routing hint.
2. The harness retries. It appends the full tool result to the next model turn.
3. A human gets impatient, copies the transcript, and pastes it into another model — often a free remote endpoint — with "explain this loop."
4. Internal hostnames, deny fingerprints, and request ids leave the VPC. Nothing in CI notices.

The invariant is blunt: error context that names an internal target must not cross into a remote model prompt. Break that, and you have sketched your topology for a party you do not operate. I am not claiming a live breach. This is a synthetic reproduction. Treat it as a regression gate, not a pentest report.

flowchart LR
  tool[Internal tool 403] --> harness[Agent harness]
  harness -->|full tool result| remote[Remote model API]
  remote --> logs[Provider-side logs]
  harness -->|classified / redacted| safe[Allowed prompt]

Look at that middle arrow. That is the control you actually own.

Threat model: prompt egress, not prompt injection

Forget jailbreaks for a minute. The interesting boundary is outbound. Injection is inbound. Different layer, different test.

Zone	What lives there	Default trust
Dev laptop / agent harness	Tool results, .env fragments, local 403 bodies	High, still leaky
Self-hosted runner	CI secrets, internal DNS	High
Remote model API, including free endpoints	Prompt, sampled output, provider logs	Untrusted for secrets
Free trial server you do not operate	Disk, process list, outbound NIC	Untrusted until isolation is proven

Assets I classify before send:

• RFC1918 literals and *.internal URLs
• Authorization, Cookie, Set-Cookie, Proxy-Authorization
• Cloud keys and PEM blocks
• WAF deny bodies and custom rule names
• Customer identifiers glued to invoice paths
• Trace ids that map 1:1 onto production requests

Who is the attacker? You. The helpful engineer. Also the agent that "includes full error context for quality." Intent does not matter. Egress does.

What the model actually needs

Ask it out loud: does the model need the hostname to explain a 403?

Usually it needs the class of failure, not the coordinate. "Upstream returned 403 after a missing scope on a mutating verb" is enough. https://billing.svc.prod.internal:8443/v2/invoices?customer=cust_9821 is a map of your billing service. Why would you ship the map?

I split prompts into three buckets:

1. Public mechanics — status codes, HTTP verbs, generic stack frames. Allowed.
2. Internal coordinates — hostnames, RFC1918, cluster names, deny-rule ids. Block.
3. Secrets — tokens, cookies, PEM, connection strings. Block, and rotate if they already left.

If you cannot classify a substring, it does not go. Ambiguity is a deny. That feels harsh until you remember the provider's log retention is not your runbook.

A 70-line egress fixture

Pinned for this write-up: Python 3.12 and pytest 8.3.3. Label this an unexecuted template until you run it on your machine. Do not treat the regexes as a WAF. They are a fail-closed tripwire for the classes above.

prompt_egress.py:

# prompt_egress.py — synthetic classifier, not a DLP product
from __future__ import annotations

import re
from dataclasses import dataclass

SECRET_RE = re.compile(
    r"(?im)(authorization\s*:\s*\S+|bearer\s+[A-Za-z0-9._\-]{12,}"
    r"|-----BEGIN (?:RSA )?PRIVATE KEY-----"
    r"|AKIA[0-9A-Z]{16})"
)
INTERNAL_RE = re.compile(
    r"(?im)(https?://[^\s]*\.internal(?::\d+)?"
    r"|https?://(?:10(?:\.\d{1,3}){3}|192\.168(?:\.\d{1,3}){2}|127\.0\.0\.1)(?::\d+)?"
    r"|\b(?:deny-rule|waf-rule|cf-ray|x-request-id)\s*[:=]\s*\S+)"
)

@dataclass(frozen=True)
class Verdict:
    action: str  # allow | block
    reasons: tuple[str, ...]


def classify_text(text: str) -> Verdict:
    reasons = []
    if SECRET_RE.search(text or ""):
        reasons.append("secret-like material")
    if INTERNAL_RE.search(text or ""):
        reasons.append("internal coordinate or deny fingerprint")
    return Verdict("block" if reasons else "allow", tuple(reasons))


def classify_messages(messages: list[dict]) -> Verdict:
    """Walk OpenAI-style messages and string tool_result payloads."""
    chunks: list[str] = []
    for msg in messages:
        content = msg.get("content")
        if isinstance(content, str):
            chunks.append(content)
        elif isinstance(content, list):
            for part in content:
                if isinstance(part, dict) and isinstance(part.get("text"), str):
                    chunks.append(part["text"])
        for key in ("tool_result", "function_call", "name"):
            val = msg.get(key)
            if isinstance(val, str):
                chunks.append(val)
    return classify_text("\n".join(chunks))


def assert_prompt_allowed(messages: list[dict]) -> None:
    verdict = classify_messages(messages)
    if verdict.action != "allow":
        raise PermissionError(f"prompt egress blocked: {verdict.reasons}")

tests/test_prompt_egress.py:

# tests/test_prompt_egress.py
import pytest
from prompt_egress import classify_messages

def test_negative_internal_403_is_blocked():
    messages = [{
        "role": "tool",
        "content": (
            "GET https://billing.svc.prod.internal:8443/v2/invoices\n"
            "HTTP/1.1 403 Forbidden\n"
            "x-request-id: req_prod_01JEXAMPLE\n"
            "deny-rule: bot-score-edge-7\n"
        ),
    }]
    v = classify_messages(messages)
    assert v.action == "block"
    assert "internal coordinate or deny fingerprint" in v.reasons

def test_positive_public_422_is_allowed():
    messages = [{
        "role": "user",
        "content": (
            "FastAPI returned 422 on POST /widgets because `name` was omitted. "
            "Explain the validation error shape. No hostnames."
        ),
    }]
    v = classify_messages(messages)
    assert v.action == "allow"
    assert v.reasons == ()

def test_negative_bearer_in_tool_result_is_blocked():
    messages = [{
        "role": "assistant",
        "content": "retrying with Authorization: Bearer FAKESECRET_w3x4y5z6a7b8c9d0e1f2",
    }]
    v = classify_messages(messages)
    assert v.action == "block"

Run it:

python -m venv .venv
source .venv/bin/activate
pip install 'pytest==8.3.3'
python -m pytest tests/test_prompt_egress.py -q

Expected: the two negative fixtures fail closed (block). The positive fixture — a public FastAPI 422 with no internal names — returns allow. If your harness cannot fail closed, you do not have a control. You have a linter you ignore.

Wire it in front of the client

Do not scan after requests.post. Scan the string you are about to send. After is forensics. Before is prevention.

# unexecuted wrapper — drop in the one function that builds `messages`
from prompt_egress import assert_prompt_allowed

def complete(messages: list[dict]) -> str:
    assert_prompt_allowed(messages)
    # only then: client.chat.completions.create(...)
    raise NotImplementedError("call your model client after the gate")

Install the gate in four steps:

1. Put assert_prompt_allowed() in the single function that builds the messages array. One chokepoint. Not twelve helpers.
2. Log reasons locally. Never log the raw blocked prompt to a cloud logger that is another trust zone. You just moved the leak.
3. Redact to classes (<internal-url>, <auth-header>) if a human still needs a model to explain the shape of the 403.
4. Keep a canary hostname (prompt-egress-canary.internal.example) in staging transcripts. If a remote provider ever echoes it, you have proof of egress. That is detect, not prevent.

Redaction that I actually use as a second pass, still fail-closed if a secret regex still hits:

def redact_for_model(text: str) -> str:
    text = SECRET_RE.sub("<redacted-secret>", text)
    text = INTERNAL_RE.sub("<redacted-internal>", text)
    return text

Redaction is not a license to send production deny text. It is how you keep a synthetic discussion about HTTP semantics. If the redacted blob is still a blow-by-blow of your edge, stop. Write a fixture instead of pasting prod.

Prevent / detect / recover

Control	Layer	Passes when	Fails when
Prevent	Harness, pre-HTTP	assert_prompt_allowed raises on internal 403 bodies	Client posts first, classifies later
Prevent	Human paste path	Editor / wrapper scans clipboard-bound transcripts	"I'll just paste into another model"
Detect	Canary hostname in staging	Provider output or your proxy log shows the canary	Canary never planted
Detect	CI on checked-in transcripts	*.internal or Authorization: in fixtures fails the job	Transcripts live only in chat UI
Recover	Secret rotation + deny-rule rename	Token/cookie/PEM rotated; rule id changed if it leaked	You treat the paste as "just debug"

Recovery is the part people skip. If a bearer token crossed, rotate it. If a deny-rule id crossed, treat it as fingerprint leakage and rename it. The model will not unsee it.

Where a free remote model still fits

I use remote free-model access for synthetic failures: public status codes, fake hosts, fixtures like orders.internal.example. I do not use it to decode production deny text. The coordinate stays in the VPC. The class of the error can leave.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source AI development platform with free model access and a free server option. That pairing is useful when you want a scratch workspace that is not your prod cluster. It does not erase the boundary. Paste a real 403 from billing.svc into that workspace and you have still exported topology. Run the classifier first. If you stand up a trial box, keep production tool results off it. Point the same assert_prompt_allowed() wrapper at whatever client you use there — that is the only ask.

Who should not use this approach

Skip this fixture if you need a contractual deletion SLA from the model provider. Regex on your laptop is not that contract. Skip it if you hoped it would replace a WAF, a DLP appliance, or a secret scanner on the repo. Different layers. Skip it if your harness streams tool results token-by-token with no message-assembly hook: you cannot classify what you never buffer. Skip it for regulated PII where the correct control is "no remote model," not "redact and hope."

Limitations are real. Homoglyphs beat naive regex. Base64-wrapped hostnames beat it too. JSON nested under tool_result.content[0].text will dodge a line-oriented scan if you only inspect messages[-1].content. Extend the walker or you are testing the demo, not the harness. I would rather ship a narrow, fail-closed gate than a 400-line "AI security platform" that nobody calls.

What belongs in CI

Here is the boundary question I want on the review: which invariant belongs in CI, and which layer should enforce it?

CI should fail if the classifier is missing from the model client, and if a checked-in transcript contains *.internal or Authorization:. The harness should enforce fail-closed at send time. The model provider will not do this for you. Why would they? They asked for context. You supplied a network diagram.

Stop shipping the deny path. Send the class of the 403. Keep the coordinate.