jaryn

Posted on Aug 26

Audit Your Repo Before a Free Model Sees It

#security #ai #opensource #devops

Last Tuesday, a teammate pasted a stack trace into a free model endpoint. The trace carried three things it shouldn't have: an internal database hostname, a signed S3 URL, and a service-account email. The model answered in four seconds. The exposure will outlive the conversation.

Free model access is a gift. It's also a trust boundary you didn't draw.

MonkeyCode is an open-source AI development platform — a self-hostable workspace where models, agents, and your repository meet. Right now it offers free model access (10 million tokens) and a free server option, so you can run the whole workflow without provisioning your own box first. That's a genuinely useful way to evaluate the platform.

But "free" doesn't mean "no boundary." It means you need to know exactly where your data crosses a line you can't see.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The boundary, drawn badly

Here's what actually happens when you point an AI platform at your repo:

[ your repo ] --(1)--> [ MonkeyCode server ] --(2)--> [ model provider API ]
      ^                       |  ^
      |                       |  (3) logs / telemetry
      +----(4) secrets hiding in .env, git history, test fixtures

Four boundaries, four different failure modes:

Local → platform. What you upload: files, context, prompts. Most people think this is the only boundary. It isn't.
Platform → model provider. What actually leaves the server and reaches the model. This is the line everyone forgets.
Logs and telemetry. What the platform retains: prompt text, completions, metadata, IPs. You need to know whether you can inspect or delete it.
Repo contents. Secrets living in .env, docker-compose files, git history, and "temporary" test fixtures that were never temporary.

The question that matters is boundary 2. Once your code crosses it, it's inside a model provider's system. Can you live with that? If the answer is "I don't know," you're not ready to connect a free model.

What to send, what to redact, what to never send

Use this table as your starting point. It's conservative on purpose:

Category	Example	Decision
Public code	utility functions, open-source snippets	send
Internal logic	business rules, proprietary algorithms	send only if you accept provider access
Config	`.env`, compose files with credentials	never
Secrets	API keys, tokens, certificates	never
Customer data	PII, emails, IP addresses	never
Infrastructure names	hostnames, bucket names, ARNs	redact first
Stack traces	may embed paths, IDs, hostnames	redact first

The rule of thumb: if a value would let an attacker move laterally, it doesn't cross boundary 2. Full stop.

The pre-flight gate: a 30-line script

Gates beat willpower. Here's a minimal script that blocks a directory from leaving your machine until it's clean:

#!/usr/bin/env bash
# preflight.sh — refuse to send a dirty repo to any AI platform
set -euo pipefail
TARGET="${1:-.}"

echo "[1/4] scanning for secret files..."
SECRET_FILES=$(rg -l --hidden -g '!.git' \
  -e '(^|/)\.env(\..*)?$' \
  -e 'id_rsa$' -e 'id_ed25519$' \
  -e '\.pem$' -e '\.p12$' -e 'credentials\.json$' \
  "$TARGET" || true)
if [[ -n "$SECRET_FILES" ]]; then
  echo "FAIL: secret files found:"; echo "$SECRET_FILES"; exit 1
fi

echo "[2/4] scanning for key material..."
if rg -n --hidden -g '!.git' \
  -e 'AKIA[0-9A-Z]{16}' \
  -e 'ghp_[A-Za-z0-9]{36}' \
  -e '-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----' \
  "$TARGET" | grep -q .; then
  echo "FAIL: key material found"; exit 1
fi

echo "[3/4] scanning for internal hostnames..."
if rg -n --hidden -g '!.git' \
  -e '(internal|prod|staging)[.-][a-z0-9-]+\.(corp|internal|local)' \
  "$TARGET" | grep -q .; then
  echo "FAIL: internal hostnames found"; exit 1
fi

echo "[4/4] checking recent git history..."
if git -C "$TARGET" log --oneline -n 20 2>/dev/null | grep -qiE 'secret|token|password'; then
  echo "WARN: recent commits mention secrets — scrub history before sending"
fi

echo "PASS: repo is clean enough to send."

This is a gate, not a guarantee. It catches patterns, not entropy. Run gitleaks or trufflehog for deeper detection, and treat binary files and images as opaque — they can hide anything.

Three tests to prove the gate works

Positive fixture. mkdir /tmp/clean && echo 'print("hi")' > /tmp/clean/main.py && ./preflight.sh /tmp/clean. Expected: PASS.
Negative fixture. mkdir /tmp/dirty && echo 'AWS_ACCESS_KEY_ID=AKIA1234567890ABCDEF' > /tmp/dirty/.env && ./preflight.sh /tmp/dirty. Expected: FAIL with the file named.
Log test. After a real session on the free server, inspect what the platform logged. Request bodies? Prompt text? If you can't inspect or delete those logs, treat the server as a black box and send even less.

The first two tests belong in CI. The third belongs in your evaluation checklist before you trust any platform with real work.

Prevent, detect, recover

Phase	Action
Prevent	pre-flight gate in CI, secret scanning on every push, deny-list of sensitive paths
Detect	monitor egress traffic, review platform logs, alert on prompt content
Recover	rotate every secret that touched the boundary, revoke tokens, assume provider logs persist

Who should not use this approach

If your org is under a data-residency mandate, an NDA with proprietary code, or a compliance regime that forbids external processing — don't point a hosted free server at real work. Use a fully self-hosted setup with a local model, or skip model access entirely. The gate script can't save you from a policy violation; it only stops accidental leaks.

And one more thing: 10 million tokens is a trial budget, not an architecture. Verify the current terms before you build a workflow on them. If you want to kick the tires, MonkeyCode's free model access and free server are a low-cost way to do it — just run the gate first.

The boundary question

The script enforces one invariant at the repo layer: no secrets leave. But the deeper invariant lives at boundary 2 — what the server actually sends to the model provider. Which layer should enforce it? The platform, the network egress, or your pre-flight gate?

My answer: all three, because each one fails differently. The gate fails when someone disables it. Egress control fails when the traffic is encrypted and invisible. The platform fails when a new feature syncs more than you expected.

What's your answer?

DEV Community