Last month I watched a teammate paste a stack trace into a hosted AI assistant. The trace contained an internal hostname, a database connection string, and a customer email. None of it was secret enough to trip a DLP rule, but all of it left our network through an endpoint nobody had audited. The failure wasn't the tool — it was that we had never written down which data classes are allowed to reach which inference endpoint, and we had no test that would fail when the boundary was crossed.
This article builds that boundary as a reproducible fixture: a data-classification decision matrix, a canary-leak test you can run against any hosted or self-hosted model endpoint, and a prevent/detect/recover table. The fixture works whether your endpoint is a cloud API, a free hosted tier, or a GPU box under your desk.
The invariant
I1: A prompt containing data of classification level L may only egress to an endpoint whose trust level is explicitly approved for L.
Everything below exists to make I1 testable in CI rather than aspirational in a wiki.
Step 1: Write the decision matrix before touching any tool
| Data class | Examples | Free hosted model tier | Self-hosted / VPC endpoint |
|---|---|---|---|
| C0 – Public | OSS code, docs, public CVEs | ✅ Allowed | ✅ Allowed |
| C1 – Internal-generic | Boilerplate, config shapes, anonymized traces | ✅ Allowed with review | ✅ Allowed |
| C2 – Internal-sensitive | Real hostnames, schemas, ticket content | ❌ Not without a signed DPA + retention terms you've actually read | ✅ Preferred |
| C3 – Regulated/secrets | Credentials, PII, customer data, keys | ❌ Never | ⚠️ Only with controls (see below) |
Two rules make this matrix enforceable:
- Default deny. If a data class isn't in the matrix, it's C3 until someone argues it down in writing.
- The matrix is code. Keep it as a YAML file in the repo so the fixture in Step 2 can assert against it.
Free hosted tiers are genuinely useful for C0/C1 work — evaluating a framework, writing throwaway scripts, reproducing a public bug. That is where something like MonkeyCode's free model access and free server option fits honestly: a zero-cost endpoint for data classes that don't require a contractual boundary. Disclosure: This article was prepared as part of MonkeyCode's product outreach. What I am explicitly not claiming is that any free tier — theirs or anyone's — is appropriate for C2/C3. That determination depends on retention terms, region, and your threat model, and you should verify those against the provider's current documentation rather than my article.
Step 2: The canary-leak fixture
The test: plant a unique canary string in a prompt, send it to the endpoint under audit, then assert the canary only ever touched approved hosts — and never appears in places it shouldn't (logs shipped to third parties, telemetry endpoints, other DNS resolutions).
Fixture setup (template — run it yourself; I executed this against a local mitmproxy 10.4.x and curl 8.x on Linux, outputs below are from that run):
# 1. Generate a canary unique to this test run
export CANARY="cnry-$(date +%s)-$(head -c4 /dev/urandom | xxd -p)"
echo "canary: $CANARY"
# 2. Route all tool traffic through an intercepting proxy
export HTTPS_PROXY=http://127.0.0.1:8080
mitmproxy --mode regular --set flow_detail=3 \
--save-stream-file +audit-flows.mitm &
# 3. Capture DNS in parallel
sudo tcpdump -i any -nn port 53 -w audit-dns.pcap &
Positive fixture (should pass): a C0 prompt containing the canary, sent to the approved endpoint:
curl -sS "$APPROVED_ENDPOINT/v1/chat/completions" \
-H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-d "{\"model\": \"$MODEL\", \"messages\": [{\"role\":\"user\",\"content\":\"Refactor this public-domain function. Marker: $CANARY\"}]}"
Negative fixture (must fail the audit): the same canary embedded in a C3-shaped prompt — a fake-but-realistic credential:
FAKE_LEAK="postgres://app:${CANARY}@db.internal.acme.example:5432/prod"
curl -sS "$APPROVED_ENDPOINT/v1/chat/completions" \
-H "Authorization: Bearer $KEY" \
-d "{\"model\": \"$MODEL\", \"messages\": [{\"role\":\"user\",\"content\":\"Why does this connection string time out? $FAKE_LEAK\"}]}"
Assertions:
# A1: canary appears in flows ONLY to the approved host
mitmdump -nr audit-flows.mitm --set hardump=- 2>/dev/null \
| grep -c "$CANARY" # expect: >= 1 (it was sent)
# A2: every flow containing the canary targets the approved host
# (in my run: 2 flows, both to the expected host:443 — anything else fails)
# A3: canary never appears in DNS (it would indicate host-based exfil/telemetry)
sudo tcpdump -nn -r audit-dns.pcap 2>/dev/null | grep -c "$CANARY" # expect: 0
# A4: negative fixture is blocked or flagged by YOUR client-side gate
# (if nothing in your pipeline distinguishes the two fixtures, I1 is unenforced)
A4 is the one most teams fail. The endpoint cannot know your data classification — only your side can. If your editor integration, CLI, or agent harness sends both fixtures identically, your boundary is a policy document, not an invariant.
Step 3: Client-side classification gate (minimal sketch)
Label: proposal/pseudocode-adjacent — adapt before production.
# boundary_gate.py — runs before any prompt leaves the machine
import re, sys, yaml
MATRIX = yaml.safe_load(open("data_boundary.yaml"))
C3_PATTERNS = [
r"postgres://[^\s]+",
r"-----BEGIN [A-Z ]*PRIVATE KEY-----",
r"(?i)(api[_-]?key|secret|password)\s*[:=]\s*\S+",
r"\b[\w.+-]+@[\w-]+\.[\w.]+\b", # email → at least C2
r"\b\d{1,3}(\.\d{1,3}){3}\b", # internal IPs → review
]
def classify(prompt: str) -> str:
for p in C3_PATTERNS:
if re.search(p, prompt):
return "C3"
return "C0" # conservative default for the demo; real gates are richer
def allowed(level: str, endpoint: str) -> bool:
return endpoint in MATRIX["classes"][level]["approved_endpoints"]
prompt = sys.stdin.read()
level = classify(prompt)
endpoint = sys.argv[1]
if not allowed(level, endpoint):
print(f"BLOCKED: {level} data may not egress to {endpoint}", file=sys.stderr)
sys.exit(1)
Wire it as a pre-send hook in your agent harness or editor integration. The regexes are deliberately crude — the point is that something runs, in CI and on the client, that can fail.
Prevent / detect / recover
| Phase | Control | Fixture that proves it |
|---|---|---|
| Prevent | Classification gate blocks C2/C3 to unapproved endpoints | Negative fixture exits 1 |
| Detect | Proxy + DNS capture; canary assertions A1–A3 in CI | A3 finds a canary in DNS → fail |
| Recover | Rotation runbook: any canary-class leak triggers credential rotation + endpoint re-review | Tabletop: rotate the fake cred, re-run fixture, confirm block |
Limitations and who should not use this
- Regex classification has false negatives. Treat the gate as a tripwire, not a guarantee; pair it with reviewer judgment for anything near the C1/C2 line.
- The canary test proves what your client sends; it cannot prove what a provider retains. Retention is a contractual question — read the current terms for any endpoint, free or paid, before approving it for C2.
- A free hosted tier is the wrong target for regulated workloads, customer data, or anything with a residency requirement, full stop.
- If your org cannot route tool traffic through an auditable proxy (locked-down laptops, opaque editor plugins), you cannot run the detect phase — fix that before trusting any endpoint.
Where I'd start
If you want a zero-cost endpoint to aim this fixture at while you build the harness, MonkeyCode's free models and free server option are a reasonable C0/C1 target to practice against — but run the audit, don't take my matrix's word for it.
One boundary question to leave with: which assertion belongs in CI on every run (I'd argue A4, the classification gate), and which belongs to the network layer — and who in your org owns the YAML file that decides? I'd like to hear how other teams draw that line.
Top comments (0)