Don't Run AI-Generated Code on Your Laptop: Free Sandbox Harness

AI-generated code should be treated as untrusted input: never execute it on the machine that holds your SSH keys. This post gives you a ~60-line JavaScript harness that runs model output on a disposable remote server with a scrubbed environment, hard timeouts, and pre-written pass/fail assertions — so evaluation becomes reproducible instead of "looks right to me."

Last week I asked a model to write a small utility that "cleans up temp files older than 7 days." The output looked fine — until I read it line by line and found it happily recursing through $HOME because I had forgotten to pin a working directory in my prompt. Nothing ran. Nothing broke. But it was a useful reminder of a rule I keep violating: AI-generated code is untrusted input, and I keep executing it on my development laptop.

There's been a lot of discussion lately about what happens when agent tool boundaries fail — the OWASP Top 10 for LLM Applications lists excessive agency and insecure output handling as first-class risks. I don't have a grand theory of agent safety. What I have is a boring, reproducible workflow: generate on one side, execute on a disposable server, evaluate with a tiny harness. This post is that harness, plus an honest list of where it falls apart.

The setup: three constraints that shape everything

The workflow has three constraints I care about:

1. Generated code never executes on my development machine.
2. Every evaluation run is reproducible — same prompt, same seed of test cases, same pass/fail criteria.
3. It costs nothing to tear down and redo. If I can't afford to run the experiment ten times, I won't trust the result.

For constraints 1 and 3 I use a free remote server as the execution sandbox. I've been running this with MonkeyCode, which currently offers free model access for the generation side and a free server option for the execution side, so the whole loop stays at zero cost.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Any equivalent setup works — a throwaway VPS, a cloud free tier, a container host. The harness below doesn't depend on any specific provider; it depends on the separation between where code is generated and where it runs.

The harness: subprocess isolation plus fixed assertions

The core idea: model output goes into a file, the file runs in a subprocess with a timeout, no network-dependent behavior, a scrubbed environment, and a scratch working directory. Then a set of assertions decides pass/fail — not my eyes scanning the diff. The mechanics lean entirely on Node's built-in child_process module, so there are zero dependencies to audit.

// sandbox-run.mjs — executes one generated snippet with hard boundaries.
// Run this ON the sandbox server, never on your dev machine.
import { spawn } from "node:child_process";
import { mkdtemp, writeFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";

const TIMEOUT_MS = 5_000;

export async function runInSandbox(code) {
  const dir = await mkdtemp(join(tmpdir(), "aigen-"));
  const entry = join(dir, "snippet.mjs");
  await writeFile(entry, code);

  const result = await new Promise((resolve) => {
    const child = spawn("node", [entry], {
      cwd: dir,                        // confined working directory
      env: { PATH: process.env.PATH }, // scrubbed env: no tokens, no keys
      stdio: ["ignore", "pipe", "pipe"],
    });

    let stdout = "", stderr = "";
    child.stdout.on("data", (d) => (stdout += d));
    child.stderr.on("data", (d) => (stderr += d));

    const killer = setTimeout(() => {
      child.kill("SIGKILL");
      resolve({ status: "timeout", stdout, stderr });
    }, TIMEOUT_MS);

    child.on("close", (code) => {
      clearTimeout(killer);
      resolve({ status: code === 0 ? "ok" : "error", exitCode: code, stdout, stderr });
    });
  });

  await rm(dir, { recursive: true, force: true });
  return result;
}

And the evaluation side — a fixed table of test cases that I write before looking at any model output, so I don't bend the criteria to fit what the model produced:

// eval.mjs — one prompt, many runs, fixed assertions.
import { runInSandbox } from "./sandbox-run.mjs";

const cases = [
  { name: "sorts numbers ascending", input: [3, 1, 2], expect: [1, 2, 3] },
  { name: "handles empty array",     input: [],         expect: [] },
  { name: "does not mutate input",   input: [2, 1],     expect: "no-mutation" },
];

async function evaluate(generatedCode) {
  const wrapped = `
    import fs from "node:fs";
    const input = JSON.parse(fs.readFileSync(0, "utf8"));
    ${generatedCode}
    console.log(JSON.stringify(solution(input)));
  `;
  const results = [];
  for (const c of cases) {
    const r = await runInSandbox(wrapped);
    results.push({ case: c.name, ...r });
  }
  return results;
}

// Feed in code produced by whatever model you're evaluating.
const codeFromModel = process.argv[2]
  ? await (await import("node:fs/promises")).readFile(process.argv[2], "utf8")
  : `const solution = (a) => [...a].sort((x, y) => x - y);`;

console.table(await evaluate(codeFromModel));

Two deliberate design choices

• Assertions before generation. I write the test table first, paste the task into the model (via MonkeyCode's free model access in my runs, but any model works), save the raw output to a file, and only then run node eval.mjs output-attempt-1.js. This turns "looks right to me" into "passed 3/3 with zero mutations."
• The temp dir is the blast radius. The scrubbed env means even deliberately hostile output can't read my credentials, because there are none in scope on the sandbox box. Combined with cwd pinning, the $HOME recursion bug from my opening anecdote physically cannot escape the scratch directory.

When this harness is worth it — and when it isn't

Situation	Use the sandbox harness?	Why
Evaluating a model's output quality across many runs	Yes	Reproducible pass/fail beats vibes
Generated code touches the filesystem, shell, or network	Yes	This is exactly the failure mode that hurts
Quick syntax question, code never executed	No	Just read it; a sandbox adds nothing
Output needs real secrets or production data to be meaningful	No	Scrubbed env makes the test unrealistic — test manually in staging instead
Code needs OS-level isolation (untrusted dependencies, install scripts)	Not enough	A subprocess is not a security boundary; add a real container/VM layer

Limitations, stated plainly

• A child process is not a security sandbox. It stops accidents, not adversaries. If you're running code from an agent loop that fetched instructions from the internet, add container isolation, a read-only filesystem, and network egress rules. My harness is seatbelts, not an airbag system.
• The free tier is a convenience, not a guarantee. Free model access and free server options change over time; treat cost as zero today, design so you can swap providers tomorrow. That's why the harness has no provider-specific code in it.
• Pass/fail tables can be gamed by the model. If the task leaks into a model's training data, high scores may reflect memorization. Rotate in novel test cases — contamination is a documented problem in benchmark evaluation, as covered in NIST's guidance on AI risk management.
• This measures one narrow thing — whether output runs correctly under constraints. It says nothing about readability, maintainability, or whether you should ship it.

Who should skip this — and what to do tonight

If you review every generated line before running anything and your tasks are low-risk, the overhead isn't justified. If you already have CI-based eval infrastructure, you don't need my 60-line version. This workflow is for the gap in between: people pasting model output straight into a terminal on their main machine and hoping.

If you're in that gap, the smallest useful step isn't adopting any particular platform — it's moving execution off your laptop tonight and writing three assertions before your next prompt. Copy the two files above onto any throwaway box, run node eval.mjs against your next model output, and see what the pass/fail table tells you that your eyes didn't. If you want a zero-cost place to try that loop, the free model access and free server from MonkeyCode are what I used for the runs above; the harness itself will follow you to whatever you use next.