Running Untrusted Code Safely: A Field Guide for AI and CI Pipelines

#ai #security #github #devops

Every bot that reads your pull requests is executing code it did not write. Here is how to make sure that is not the end of the story.

There is a quiet assumption baked into most developer tooling: that the code we run is code we trust. Your CI runs your tests. Your linter loads your config. Your build installs your dependencies. For years that held, because the person triggering the job and the person who wrote the code were the same person.

That assumption is now false, and a lot of teams have not noticed.

The moment you add an AI reviewer, a CI job that runs on pull requests from forks, a dependency bot, or any automation that touches contributor code, you have invited untrusted code into a trusted environment. A stranger can now open a pull request and, if you are not careful, get your infrastructure to run something on their behalf. The unsettling part is not the code in the diff. It is everything that runs around it.

This is a field guide to doing that safely. It is written for anyone building or operating a tool that reads other people's repositories, but the same principles apply to any CI pipeline that accepts pull requests from people you do not know.

The threat is bigger than the diff

When people picture untrusted code, they imagine a malicious function hidden in a pull request. That is real, but it is the least of your problems, because you are probably not executing the diff line by line. The real surface is everything your pipeline does automatically on the way to looking at that diff.

Walk through what a normal review or CI job touches:

Package installation. Running npm install, pip install, or the equivalent executes lifecycle scripts. A single postinstall hook, or one hostile transitive dependency, runs arbitrary code before a human has looked at anything.
The repository's own commands. Test runners, build steps, and task files are code. If your pipeline runs npm test on a contributor's branch, you are running whatever that contributor decided test should mean today.
Tool configuration. This is the one almost everyone misses. Linters and formatters read configuration from the repository, and that configuration can load plugins, custom rules, and local modules. A crafted linter config is a remote code execution primitive wearing a yaml costume. The tool is yours. The config it obeys is theirs.

Here is the sharp version. The attacker does not need you to run their function. They need you to run your tool against their configuration. That is a far wider door, and it is usually left unlocked.

The failure that turns a bug into a breach

A code execution bug inside an isolated sandbox is a contained incident. The same bug becomes a company-ending event when the environment running untrusted code also holds the keys to everything else.

This is the anatomy of the worst tooling breaches. An automation platform runs contributor code in an environment that, for convenience, also contains its cloud credentials, its database connection string, and, in the most painful cases, the private key of the integration that grants it access to every customer repository. One escape from the sandbox, and the blast radius is not one job. It is every tenant at once.

The lesson is not "sandbox harder." Sandboxes have escapes; assume yours will. The lesson is that the thing reading untrusted code should have nothing worth stealing and nowhere to pivot. If your isolation fails and the intruder finds an empty room with a dead network, you have had an incident worth a postmortem. If they find your master keys, you have had an extinction event. Design so that the worst realistic day is the boring one.

Seven principles that actually hold

Here is the posture I would defend in a design review. None of it is exotic. It is the discipline of assuming your isolation will eventually fail and making that failure uneventful.

Never run the repository's own scripts. No install, no test, no build, no task file from untrusted code. If you need to understand the project, parse its manifests yourself instead of executing its lifecycle. This single rule deletes the most common execution path outright.
Keep secrets out of the room. The process that touches untrusted code should not have your platform credentials in its environment, its filesystem, or its memory. Not the database password, not the signing key, not another tenant's token. What is not there cannot leak.
Scope every credential to the minimum. When the sandbox genuinely needs access, hand it the narrowest possible grant: a token scoped to the single repository, read-only, and short-lived. A token that can read one repo for ten minutes is a completely different liability from an app key that can write to a million.
Deny egress by default. Most analysis never needs the open internet, and it certainly does not need your internal services or the cloud metadata endpoint. Block outbound traffic and allow-list only what is required. This closes both data exfiltration and the classic metadata-credential theft in one move.
Treat tool configuration as untrusted input. If you run linters or analyzers, run them with plugin loading and config discovery disabled, or parse the repository-controlled config yourself rather than letting the tool obey it. Assume any file the attacker can write is an attempt to make your tools work for them.
Make the environment ephemeral, non-root, and constrained. One run, one throwaway environment, destroyed when it finishes. Run as an unprivileged user, with a read-only filesystem except for a scratch directory, and hard limits on CPU, memory, and wall-clock time. Ephemerality alone defeats an entire category of persistence.
Practice least privilege on the integration itself. Before you ever reach the sandbox, ask what your GitHub App or CI integration is even allowed to do. If it does not need write access to file contents, do not request it. The permissions you never hold cannot be abused when something downstream goes wrong.

What this looks like in practice

I care about this because I build a tool squarely in this category. ShipGuarde is an AI reviewer that reads pull requests and QAs the running app, which means it reads untrusted code every hour of every day. The posture above is not aspirational for us; it is the architecture. Each pull request is analyzed in a single-use sandbox that holds none of our platform's secrets. The only credential it ever receives is a short-lived token scoped to that one repository, read-only, and it has no route to internal services or metadata. We do not run a repository's own scripts, and we would rather parse a config than obey it. If that sandbox were fully compromised, the intruder would find one repository's readable source and a network cable that goes nowhere. That is the entire point.

I mention it not to sell you anything, but because the constraint shaped every early decision we made, and I think far more tools should be built this way. If you are evaluating any automation that touches your source, this is the posture to make it prove.

A checklist you can steal

If you take nothing else, take this. Before you let any pipeline run on untrusted code, confirm every line:

It does not execute the repository's install, test, or build.
Its environment holds no platform or tenant secrets.
Any credential it carries is repo-scoped, read-only, and short-lived.
Outbound network access is denied by default and allow-listed narrowly.
Repository tool configs cannot load plugins or arbitrary modules.
The environment is ephemeral, non-root, and resource-limited.
The integration requests the minimum permissions and nothing more.

Run down that list honestly. If you cannot check every box, you do not have a hardening problem. You have an untrusted-code problem in a hardening costume.

The takeaway

The industry spent a decade tuning pipelines for speed and convenience on the assumption that everything flowing through them was trusted. AI tooling quietly broke that assumption, and the tools built next will be judged on whether they noticed. The good news is that the safe design is not the slow one. When you stop running other people's code and refuse to hold secrets you do not need, you end up with a system that is both harder to breach and easier to reason about. For once, the secure path and the fast path are the same path. Take it.

Written from the trenches of building an AI reviewer. If your team is working through any of this, I am always happy to compare notes.