DEV Community

Alex Zhu
Alex Zhu

Posted on

Assign a Graduation Owner: A Wiki Playbook Before Sandbox Agent Jobs Touch Production

You are paging through Slack at 09:14 when the billing channel lights up with duplicate invoice emails. A teammate had pointed a shared agent at a free sandbox server, then reused the same production .env file. Copying those secrets felt faster than asking anyone to mint a scoped sandbox token. The agent followed its tool schema, posted a live webhook, and treated production as another demo endpoint.

Shared agents make that failure cheap to repeat, because tool calling looks the same in a demo and in a live checkout path. You cannot treat a green sandbox run as a production pass when the tools, secrets, and network paths are not actually split. You also cannot wait for a postmortem to invent an owner after the webhook has already fired. You need a named graduation owner, a one-page wiki, and a gate that fails closed.

The failure is a missing handoff, not a missing comment

Free model access and a free server option are useful when you want an isolated lane for prompt drafts, tool-schema experiments, and noisy retries. They become dangerous when that lane silently inherits production credentials, customer payloads, or write-capable endpoints. The failure is rarely the model itself; the failure is an undocumented handoff between exploring and shipping. You have probably watched a function schema look correct, watch the JSON parse, and still never ask which environment the HTTP client will hit.

A realistic API test on the sandbox still lies if the base URL, identity, and side effects are production-shaped. Clear comments in the prompt file will not save you if the runtime environment is wrong. Tool-calling tutorials make the happy path look identical across hosts, which is exactly why teams copy .env files. You need a written promotion rule before the next shared agent job leaves isolation.

You can host that isolated lane on any machine you control. Some teams use MonkeyCode here because it currently offers free model access and a free server option for exploratory agent work. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Treat those two availability claims as current product options, not as a promise about model names, quotas, hardware, or how long the offer lasts.

Name the role before you name the server

A graduation owner is the person who may say no when a sandbox agent job asks to leave isolation. They do not write every prompt, and they do not babysit every retry. They own the checklist, the secret scope, and the written rule for what promoted means on your team. If you skip the role, every engineer becomes an accidental owner at 09:14, which is how environment files travel.

If you over-scope the role, the owner becomes a bottleneck and people bypass the wiki. You want a narrow mandate: environment split, tool side-effect class, and a recorded promote-or-stop decision. Write the name on the wiki page, not in a chat thread that scrolls away during the next incident. Rotate the owner on a published cadence so the role stays a practice, not a personality.

Roles you should print on one wiki page

Keep the cast small enough that a new teammate can read it in a standup. Use job function, not heroics, and keep each handoff visible in the repository rather than in private messages.

  1. Graduation owner — Records promote or stop, and is the only person who can flip PROMOTION_STATUS=allowed for a job. They reject any run that still carries production hostnames, write tools, or customer payloads.
  2. Lane operator — Starts and stops sandbox agent jobs, and must run the environment lint before the first tool call. They never mint production tokens into the sandbox directory, even for a five-minute demo.
  3. Tool steward — Classifies each tool as read, dry-run, or write, and keeps the allowlist in version control. They block promotion while any write tool still points at an unclassified base URL.
  4. On-call reviewer — Confirms the rollback command works before a promoted job is scheduled. They own the paging path if a graduated job later touches a live side effect.

Handoff rule: the lane operator may draft, the tool steward may classify, and only the graduation owner may promote. If any of those three names is empty, the wiki page is not ready and the job stays in sandbox. Do not let a group chat vote replace the named owner when the webhook looks urgent.

Decision table you can paste beside the names

Use this table as a fail-closed filter. If a row is unknown, the answer is stop, not try it once.

Question Stay in sandbox May graduate
Where do secrets live? Sandbox-only tokens in a separate dir Separate production secret store, never copied
What can tools do? read or dry-run only write tools behind an allowlist and a human gate
Which host is called? Documented sandbox hostname Production hostname listed, reviewed, and rate-limited
What data is in context? Synthetic or scrubbed fixtures Production-like data with a retention and redaction note
Who can abort? Lane operator can kill the process On-call reviewer has a tested rollback command
Is the model lane labeled? Free or exploratory lane, not ship lane Ship lane named in the job manifest

This table is a proposal for teams that already mix exploratory agents with shipped services. Label it as unexecuted until your graduation owner fills the hostnames and secret paths for your stack. Do not treat a passing unit test as a substitute for the environment row.

A one-page wiki run you can paste today

Copy the block below into your team wiki and fill the bracketed fields before the next agent demo. Keep it to one page so people actually read it.

# Agent graduation SOP (sandbox -> production)
Owner this week: [name]
Backup: [name]
Sandbox root: [path]
Production is forbidden inside sandbox root.

## Before every sandbox job
1. Lane operator runs `make sandbox-lint`.
2. Tool steward stamps tool classes in `tools.lock.yml`.
3. Graduation owner confirms `PROMOTION_STATUS=blocked`.

## Promote request
- Job id:
- Tools requested:
- Side-effect class:
- Secret scope:
- Rollback command:
- Decision: promote / stop
- Signed by graduation owner:

## Stop conditions
- Any production hostname in sandbox configs
- Any customer payload in prompt context
- Any write tool without a dry-run twin
- Any missing rollback command
Enter fullscreen mode Exit fullscreen mode

Print the stop conditions near the job launcher, not only in an architecture doc. You want the operator to see them while the process is starting, not during the incident review. If your wiki cannot hold a one-page run, you are not ready to share agents across environments.

Numbered run for the next shared agent job

Follow these steps in order. Skipping a step is a stop, not a shortcut.

  1. Create a sandbox directory that cannot see production files. Put prompts, tool schemas, and fake fixtures under agent-sandbox/ and keep production deploy files outside that tree. Tell every operator that copying .env into this directory is an incident, not a convenience.
  2. Lint the tree for production hostnames and secret shapes. Run the check below until it exits zero. If it fails, the lane operator fixes the files before anyone talks to a model.
  3. Classify tools before the agent process starts. A search tool is not a refund tool, even when both are JSON functions with similar names. Stamp the class in tools.lock.yml so the steward and owner review the same list.
  4. Force dry-run on anything with a side effect. A webhook post, a database write, or a ticket closer must have a no-network twin. The agent may call the twin freely; the live tool stays blocked until promotion.
  5. Record the promote-or-stop decision in the repo. Chat approval is not an artifact. The graduation owner commits a small decision file, or the job remains blocked.
  6. Rehearse rollback on the sandbox host first. Kill the process, revoke the sandbox token, and confirm the dry-run log is the only residue. If rollback is theater, promotion stays forbidden.

Commands and checks you can run locally

The snippets below are labeled examples. They are not production security controls, and they will not replace a real secret manager. Adjust paths before you run them, and never paste live credentials into the wiki or the prompt.

# example: fail if sandbox configs mention production hosts or copied env files
# proposal only — extend the hostname list for your company
set -euo pipefail
ROOT="${SANDBOX_ROOT:-./agent-sandbox}"
DENY_HOSTS='api\.prod\.|billing\.internal|hooks\.stripe\.com'

if [[ ! -d "$ROOT" ]]; then
  echo "sandbox root missing: $ROOT" >&2
  exit 1
fi

if find "$ROOT" -name '.env' -o -name '*.pem' | grep -q .
then
  echo "stop: secrets or env files found under sandbox root" >&2
  exit 1
fi

if grep -REn "$DENY_HOSTS" "$ROOT" --include='*.yml' --include='*.json' --include='*.md'
then
  echo "stop: production hostname in sandbox tree" >&2
  exit 1
fi

echo "sandbox-lint ok"
Enter fullscreen mode Exit fullscreen mode

Pair the lint with a promotion gate so a green sandbox test cannot silently become a live caller. The Python example refuses to start when promotion is blocked or when a write tool has no dry-run twin.

# example promotion gate — proposal / unexecuted template
from pathlib import Path
import json
import os
import sys

BLOCKED = {"blocked", "", "no", "never"}

def load_json(path: Path) -> dict:
    if not path.exists():
        print(f"stop: missing {path}", file=sys.stderr)
        sys.exit(1)
    return json.loads(path.read_text())

def main() -> None:
    status = os.environ.get("PROMOTION_STATUS", "blocked").lower()
    tools = load_json(Path("tools.lock.yml").with_suffix(".json"))
    decision = load_json(Path("graduation-decision.json"))

    if status in BLOCKED or decision.get("decision") != "promote":
        print("stop: graduation owner has not promoted this job")
        sys.exit(1)

    for name, meta in tools.items():
        side = meta.get("class")
        if side == "write" and not meta.get("dry_run_twin"):
            print(f"stop: write tool {name} has no dry-run twin")
            sys.exit(1)
        if side == "write" and not decision.get("signed_by"):
            print("stop: write tools require a signed owner decision")
            sys.exit(1)

    print("promotion gate ok")

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Keep tools.lock.yml boring and explicit. A short classified list beats a clever registry that nobody reads during a demo.

{
  "search_docs": {"class": "read", "base_url_env": "SANDBOX_SEARCH_URL"},
  "preview_invoice": {"class": "dry-run", "base_url_env": "SANDBOX_BILLING_URL"},
  "post_webhook": {"class": "write", "dry_run_twin": "preview_invoice", "base_url_env": "PROD_BILLING_URL"}
}
Enter fullscreen mode Exit fullscreen mode

If you need a dry-run HTTP check, call a sandbox echo endpoint rather than the live webhook. Replace the hostname with your own non-production host, and keep the request body synthetic.

# example: dry-run against a sandbox echo host, not a live billing webhook
curl -sS -X POST "$SANDBOX_ECHO_URL/preview-invoice" \
  -H 'content-type: application/json' \
  -d '{"customer_id":"fixture-17","amount_cents":0}'
Enter fullscreen mode Exit fullscreen mode

What this playbook will not do

This SOP does not make a free sandbox equivalent to a hardened production control plane. Hostname linting will miss obfuscated URLs, runtime-injected hosts, and tools that build addresses from model output. A signed JSON file is an accountability artifact, not a cryptographic grant, and a determined operator can still export a production token by hand.

Do not use this approach if you are a solo hobbyist with no production side effects to protect. Do not use it if your company already has a change-advisory path that must remain the only promotion mechanism. Do not use it as a substitute for secret scanning, network policy, or a real allowlist on vendor APIs. Regulated teams that need auditable releases should keep this wiki as a local hygiene layer, not as their compliance story.

The playbook also assumes you can name a human who will be available during the job. If your agents run unattended overnight with write tools attached, assign a stop-condition practice first and keep promotion blocked. Sandbox work is for learning tool shapes and failure modes, not for quietly absorbing production traffic.

Close the loop in the repo, not in the demo

After a stopped job, leave the decision file in place so the next operator can see why promotion failed. After a promoted job, schedule a short rollback drill while the on-call reviewer is still in the channel. Update the hostname deny list whenever a new vendor API appears, because last month's lint will not catch this week's webhook.

Name the graduation owner before the next shared agent demo, and paste the one-page wiki into the repository today. If the owner field is still blank, the sandbox stays the only legal place for the job.

Top comments (0)