How to wrap Codex CLI from Python the way osFoundry does

This is the command the osFoundry desktop app runs when you click Verify sign-in for Codex. It is built by build_codex_command in src-tauri/src/providers/agent_cli.rs, the same function every real cli:codex:agent run goes through:

codex exec "Reply with the single word: ok" \
  --json \
  --skip-git-repo-check \
  --ignore-user-config \
  --ignore-rules \
  -s read-only \
  -c 'approval_policy="never"'

Going through it piece by piece:

• exec with the prompt as one positional argument. Codex has no system-prompt flag, so osFoundry builds that argument as system + "\n\n" + transcript.
• --json makes stdout one JSON object per line. That stream is the only output osFoundry reads.
• --skip-git-repo-check lets the run start in a folder that is not a repository.
• --ignore-user-config skips ~/.codex/config.toml, so the user's own MCP servers stay out of the run. Auth still loads from CODEX_HOME.
• --ignore-rules is added only if codex exec --help lists it. osFoundry asks the binary because older builds reject the flag, and an unknown flag fails the whole run.
• -s plus approval_policy="never" sets the sandbox and tells Codex nobody will be there to approve anything. An autonomous run gets -s workspace-write. One that attaches osFoundry's MCP bridge gets --dangerously-bypass-approvals-and-sandbox instead, and only inside a git worktree.

Before spawning, osFoundry also removes CODEX_API_KEY, OPENAI_API_KEY and OPENAI_BASE_URL from the child's environment. If any of them were set, the run would go to a metered API account instead of the ChatGPT sign-in.

The rest of this post is about the other end of that pipe. You have an agent CLI whose stream format is not a promise, and you need its answer, its progress and a clean failure out of it.

Why read events instead of a final-answer file?

There are two ways to get an answer out of a CLI like this:

1. Final answer to a file. Pass an output path (codex exec -o FILE; osFoundry's own module notes call it the documented file output), wait for the process to exit, then read the file. The contract is one file with one string in it.
2. Event stream on stdout. Pass --json, read line by line, and keep the items you understand.

osFoundry's current builder does the second one. The parser lives in runtime/session_event.rs (parse_codex_line) and keys on very little:

Event	Becomes
item.completed + agent_message	answer text
item.completed + reasoning	Thinking: … line
item.completed + command_execution	Using shell — <command>
item.completed + mcp_tool_call	Using <tool>… (Codex gives server and tool separately)
item.completed + file_change / todo_list	Edited <path> / Plan: 1. … 2. …
turn.completed	usage (input_tokens, cached_input_tokens, output_tokens)
turn.failed, error	failure, flagged as auth if the message looks like one

Anything else is ignored, and so is any line that is not JSON. That tolerance is what makes this workable without a stable schema. An unknown event type costs nothing. Only a renamed field in one of those seven rows can break the parser.

The price is granularity. Codex emits completed items, not token deltas. The answer appears all at once when the message is finished, which is why the app labels Codex No live stream and describes it as run-then-emit. Progress lines are different: they arrive as each item completes.

Rebuilding both on a Mac

No Codex binary is installed on the machine I used. So I wrote fake_codex.py, a stand-in that takes the same flags, prints the event types above with fixed delays, writes the final message to the -o file on exit, and can be told to hang, crash, or fail auth. It is not Codex. What it lets you measure is the wrapper's behaviour, not Codex's speed.

The stand-in's timeline: a reasoning item at 0.4 s, then a shell child (sleep 0.8), then the agent_message about 1.3 s later, then turn.completed 0.2 s after that. The answer file is written at exit.

The wrapper, wrap.py, uses only the standard library on Python 3.9.6. The core of the events strategy, trimmed:

def run_events(argv, timeout, env=None, group=True, drain_stderr=True):
    t0 = time.monotonic()
    p = subprocess.Popen(argv + ["--json"], stdin=subprocess.DEVNULL,
                         stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                         env=env, start_new_session=group)
    err = []
    if drain_stderr:
        threading.Thread(target=_drain, args=(p.stderr, err), daemon=True).start()
    lines = queue.Queue()
    threading.Thread(target=pump, args=(p.stdout, lines), daemon=True).start()

    deadline = t0 + timeout
    while True:
        left = deadline - time.monotonic()
        if left <= 0:
            os.killpg(p.pid, signal.SIGKILL) if group else p.kill()
            r["error"] = "timed out after %ss" % timeout
            break
        try:
            at, raw = lines.get(timeout=left)
        except queue.Empty:
            continue
        if raw is None:                        # stdout closed
            break
        try:
            v = json.loads(raw)
        except ValueError:
            continue                           # log line on stdout
        t = v.get("type")
        if t == "item.completed":
            item = v.get("item", {})
            if item.get("type") == "agent_message":
                r["text"] += item.get("text", "")
            else:
                r["steps"].append(item.get("type"))
        elif t == "turn.completed":
            r["usage"] = v.get("usage")
        elif t in ("turn.failed", "error"):
            r["error"] = (v.get("error") or {}).get("message") or v.get("message")
    p.wait()
    if not r["error"] and not r["text"].strip():
        r["error"] = "returned no output (%s)" % tail(err)[-200:]

The file strategy is the same spawn with -o <tmp> instead of --json, then p.wait(timeout=…), then a read of the file if it exists.

Time to first byte versus time to answer

Test setup: Apple M1, macOS 15.8.1, 20 runs per strategy, medians, timed with time.monotonic() from just before Popen (bench.py).

	events (--json)	file (-o)
first stdout byte	0.034 s	none
first completed item	0.443 s	none
answer available	2.562 s	2.824 s
process finished	2.775 s	2.824 s
successful runs	20 / 20	20 / 20

The absolute numbers come from the stand-in's sleeps. Two things in the table do not:

• With a file, nothing comes out until exit. For 2.8 s the caller has no way to tell "thinking" from "stuck". With events, the first progress line arrives at 0.44 s, and the bytes start at 0.034 s, about Python's own spawn cost (a bare python3 -c pass averaged 19 ms here).
• The file only exists after the CLI's last step. In this run that was 0.26 s after the message had already appeared on stdout. With a real agent, the gap is whatever the CLI does between finishing its message and exiting.

What does a failure look like from each side?

Same wrapper, one run per case:

Case	exit	events strategy	file strategy
crash before answering	101	error, with the stderr tail as the hint, 0.44 s	error: no answer file, 0.51 s
auth failure (turn.failed 401)	1	401 Unauthorized: please log in, flagged as auth	no answer file + stderr text
killed after the message (no turn.completed)	137	answer kept, no error	no answer file
hang past a 3 s deadline	-9	timed out after 3s at 3.004 s	timed out after 3s at 3.027 s

Two rows matter here.

Auth. The event stream tells you it was a login problem in a structured field. The file strategy only has stderr prose to guess from. osFoundry matches the message against a short list (401, unauthorized, please log in, sign in, …) and raises its own sign-in prompt.

Killed after the message. The two strategies disagree. Reading events, you have a complete agent_message and no turn.completed. Reading a file, you have nothing. osFoundry decides from what the stream said: the run fails if a turn.failed/error event arrived or if no text arrived at all, and the exit status is not part of that decision. If you would rather treat a missing turn.completed as partial, that is the place to add it.

What does a timeout actually kill?

osFoundry wraps the whole run in tokio::time::timeout: 1,800 s for an autonomous run, 300 s for the sign-in probe and other non-streaming completions. The child is spawned with kill_on_drop(true), so when the timeout drops the future, the process gets killed.

An agent CLI has children of its own, though. The stand-in starts sleep 30 as its "shell tool" and hangs on it. I checked that PID 0.3 s after the wrapper returned:

Kill	wrapper returned	tool child afterwards
p.kill() (direct child only)	3.010 s	still running
start_new_session=True + os.killpg	3.011 s	gone

Same timing, different cleanup. If the tool child holds your stdout pipe open, an EOF-driven reader also never sees EOF, which is why the loop above checks a deadline instead of waiting for readline to return.

Why drain stderr on its own thread?

These CLIs write auth failures and panics to stderr, so osFoundry pipes it and keeps a tail (about 4,000 bytes on the agent path) for the error message. A pipe has a fixed buffer, though. On this Mac a child blocked after exactly 65,536 bytes to an unread stderr (pipecap.py). Then I had the stand-in write about 200 KB of trace output before answering:

stderr handling	result
drained on a thread while stdout is read	answer at 2.56 s
read only after stdout closes	no answer, timed out after 8s

The CLI blocks on a write that nobody reads, stdout stays open, and the run is lost to the deadline. Reading stderr concurrently costs one thread.

If you wire this up yourself

• Pick the event stream over the answer file if anyone watches the run. Key on the few event types you need, and skip non-JSON lines and unknown types instead of failing on them.
• Put the system prompt in the prompt text when the CLI has no flag for it, with a plain separator, and keep earlier turns as a labelled transcript.
• Decide up front whether the exit code or the stream decides success, and what a message without turn.completed counts as.
• Give the child its own process group and kill the group on timeout, so tool processes go with it.
• Read stderr concurrently and keep a bounded tail. It carries the auth errors you will want to show.
• Strip the environment variables that would redirect a subscription login to a metered key, and probe --help for optional flags instead of keeping a version table.

The Codex row described above is in the External AI tab of the app's BYOK keys dialog.

Example: Claude Code and Codex in the External AI tab