How Long Before Your Agent Says Anything? Time 4 Coding Agent CLIs With hyperfine

• Book: Prompt Engineering Pocket Guide
• Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

---

Use --mcp-config to give Claude Code an MCP server that never answers, and a claude -p run waits for it before the first turn: up to MCP_TIMEOUT, 30 seconds by default, on v2.1.221 or later (CLI reference). Codex waits 1,000 milliseconds for optional servers while it builds its first tool list (Codex MCP docs). OpenCode gives a server 5,000 milliseconds to hand over its tools (OpenCode MCP docs).

Those are ceilings for a broken server. A healthy session is faster, and how much faster is a number only your own setup can produce.

hyperfine measures it. It runs a command many times and reports the mean and the spread, with optional warmup runs before it starts measuring. Below is one hyperfine command per layer of startup, driving each CLI's headless mode: Claude Code's -p, Codex's codex exec, GitHub Copilot CLI's -p and OpenCode's run. I haven't run them myself. The only numbers in this post are documented limits; the measured ones will be yours.

What runs before the first token

A fresh session does four things before your prompt reaches a model.

1. The process starts. If you installed Codex from npm, the codex on your PATH is a Node script that spawns the native binary (codex.js), so its start includes Node's.
2. Config and memory files load. Claude Code's docs say a plain claude -p discovers hooks, skills, custom commands, subagents, plugins, MCP servers, auto memory and CLAUDE.md "as an interactive session would." The --bare flag skips all of that "to reduce startup time" (headless docs). Codex "builds an instruction chain when it starts," from AGENTS.md files up to 32 KiB by default (AGENTS.md docs).
3. MCP servers start. Each CLI waits a different amount of time for them, per its docs (table below).
4. The model call. Network, queueing, reading the whole prefix, writing the answer.

CLI	Wait for a slow MCP server	Source
Claude Code, -p with --mcp-config	until pending servers connect, up to MCP_TIMEOUT (30 s)	CLI reference
Codex	mcp_optional_startup_grace_ms (1,000 ms) for optional servers; required ones use startup_timeout_sec (10 s)	MCP docs
OpenCode	timeout for fetching tools (5,000 ms)	MCP docs
Copilot CLI	not stated in the pages I read	MCP docs

Each run below peels off one layer.

Everything in the queue runs before the model hears your prompt.

Setup: hyperfine and an empty repo

Install hyperfine with brew install hyperfine, apt install hyperfine or cargo install --locked hyperfine (README), then check the version.

hyperfine 2.0.0, published October 7, 2026, runs benchmarked commands directly, without a shell, by default (release notes). That's what you want here: you're timing the agent, not sh. On 1.x, add -N to each hyperfine call below for the same behavior. 2.0 also adds an --env flag for environment variables, since there's no shell to read FOO=bar cmd.

hyperfine --version
mkdir -p ~/tmp/agent-timing && cd ~/tmp/agent-timing
git init -q
for c in claude codex copilot opencode; do
  printf '%s: %s\n' "$c" "$(command -v "$c")"
  "$c" --version
done

Codex refuses to run outside a git repository unless you pass --skip-git-repo-check (docs), hence the git init. The loop records which binary you're timing and its version. Paste both into your notes, because next month's release changes the numbers.

Install the MCP test server once, globally, so your timing doesn't include npx resolving a package:

npm install -g @modelcontextprotocol/server-everything

That gives you a mcp-server-everything command (package.json) that speaks stdio when started with no arguments. Its README says it "attempts to exercise all the features of the MCP protocol."

One more habit before any benchmark: run each prompt command once by hand. A CLI that stops at a sign-in screen, or exits early with an error, looks fast in a table.

Run 1: process start

hyperfine --warmup 3 --runs 30 \
  --export-markdown run1-version.md \
  'claude --version' 'codex --version' \
  'copilot --version' 'opencode --version'

--warmup runs each command before measuring, so the first read of the binary from disk stays out of the result. This is your floor: the binary loads, prints a string and exits. According to its release notes, hyperfine 2.0 reports peak memory next to wall-clock time by default, so you get both from this one command.

Run 2: one word to a full answer

hyperfine --warmup 1 --runs 10 \
  --export-markdown run2-hello.md \
  'claude -p hello' 'codex exec hello' \
  'copilot -p hello' 'opencode run hello'

That's 44 real model calls, and each one counts against your plan or your API bill. Drop to --runs 5 if that matters. Each CLI uses its default model and effort, so you're timing a CLI and a model together. To compare the CLIs alone, set each one to a similar model with its model flag.

Run 2 minus Run 1 is config discovery, the MCP servers you already have, the network and the model. To pull the local part out, time a stripped-down run next to the normal one:

CLI	Stripped-down run	What it skips
Claude Code	claude --bare -p hello	hooks, skills, plugins, MCP servers, auto memory, CLAUDE.md; needs ANTHROPIC_API_KEY (docs)
Codex	codex exec --ignore-user-config hello	$CODEX_HOME/config.toml, so your MCP servers; sign-in still works (reference)
Copilot CLI	copilot -p hello --no-custom-instructions --disable-builtin-mcps	AGENTS.md and related files, the built-in GitHub MCP server (reference)
OpenCode	opencode run --attach <url> hello against opencode serve	the boot of each run (CLI docs)

Bare mode signs in with an API key instead of your subscription, so the Claude row is an approximation. OpenCode's docs suggest the server route themselves, "to avoid MCP server cold boot times on every run":

# terminal 1
opencode serve --port 4096
# terminal 2
hyperfine --warmup 1 --runs 10 \
  'opencode run hello' \
  'opencode run --attach http://localhost:4096 hello'

Claude Code also reports its own timing in the JSON result, which jq can pull out:

claude -p hello --output-format json | jq '{
  duration_ms, duration_api_ms,
  ttft_ms, ttft_stream_ms, num_turns
}'

The Agent SDK's TypeScript reference defines the optional ttft_ms as time to first token, "measured when the first complete assistant message arrives," on successful runs only, and ttft_stream_ms as the time until the response stream opens. It lists duration_ms and duration_api_ms without defining them, so set those next to hyperfine's mean and remember that hyperfine's number is the one that includes process start and exit. Check num_turns too: if the agent ran a tool to answer "hello", you timed more than one model call. Codex's --json events carry token usage and no timing fields (docs), so for Codex hyperfine is the clock.

Run 3: one MCP server, one AGENTS.md, one server that hangs

Three small config files, one per format. mcp.json for Claude Code:

{
  "mcpServers": {
    "everything": {
      "type": "stdio",
      "command": "mcp-server-everything"
    }
  }
}

copilot-mcp.json is the same block with "type": "local" and "tools": ["*"], the format in GitHub's MCP docs. opencode-mcp.json:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "everything": {
      "type": "local",
      "command": ["mcp-server-everything"],
      "enabled": true
    }
  }
}

Then time all four with the server added:

hyperfine --warmup 1 --runs 10 \
  --export-markdown run3-mcp.md \
  'claude -p hello --mcp-config mcp.json' \
  'copilot -p hello --additional-mcp-config @copilot-mcp.json'

hyperfine --warmup 1 --runs 10 \
  --env OPENCODE_CONFIG=opencode-mcp.json \
  'opencode run hello'

codex mcp add everything -- mcp-server-everything
hyperfine --warmup 1 --runs 10 'codex exec hello'
codex mcp remove everything

Copilot's --additional-mcp-config adds a server "for this session only" and takes a file path prefixed with @ (reference). OpenCode merges OPENCODE_CONFIG between your global and project config (config docs), so nothing else changes. codex mcp add writes to your Codex config, which is why codex mcp remove follows it. On hyperfine 1.x, drop --env and write 'env OPENCODE_CONFIG=opencode-mcp.json opencode run hello', which works with or without -N.

The gap between Run 2 and Run 3 is one server starting, plus its tool definitions riding in the prompt. The token side of that has its own walkthrough in What an Empty Prompt Costs; this is the clock side.

Now copy a real AGENTS.md from a work repo into the bench folder and rerun Run 2. All four CLIs read it:

• Claude Code: when there is no CLAUDE.md or CLAUDE.local.md in the folder or above it, on v2.1.277 or later (memory docs).
• Codex: from the git root down (docs).
• OpenCode: by walking up from the current directory (rules).
• Copilot CLI: unless you pass --no-custom-instructions (reference).

A longer file is more text for the model to read before it answers. Delete the file when you're done, or it skews the next run.

Last, a server that never answers. Point a config at sleep 60:

cat > mcp-hang.json <<'EOF'
{"mcpServers": {"hang": {"type": "stdio",
  "command": "sleep", "args": ["60"]}}}
EOF
hyperfine --runs 3 \
  'claude -p hello --mcp-config mcp-hang.json'

Do the same for the others: codex mcp add hang -- sleep 60, "command": ["sleep", "60"] in an OpenCode config, "type": "local" for Copilot. The table at the top says roughly what to expect from three of them. If your numbers disagree with the docs, or a run takes the full 60 seconds (Claude Code should move on after about 30), you've found something worth an issue on that CLI's tracker.

Ten tasks a day in Hermes IDE

I build Hermes IDE, so weigh this section with that in mind. In Hermes, ⌘N (Ctrl+Shift+N on Windows and Linux) starts each task on its own hermes/<name> branch, in its own worktree, with the task as the agent's first prompt. Every agent runs its own interface in a real terminal, "exactly as it would anywhere else" (README). So every task is a fresh start of the vendor's CLI, and it pays the startup you just measured.

Hermes 2.3.4: the ⌘N New task sheet, with the "Hermes will run" preview of the claude command.

That preview is the command to put in hyperfine. Then do the arithmetic with your own table: Run 3 minus the stripped-down run, times the tasks you start in a day. If your repo commits a .mcp.json or an opencode.json, every new worktree has a copy, because it's a tracked file, and every task starts that server again.

Hermes runs the same command you would type, so these numbers don't shrink. The saving is elsewhere: on macOS and Linux, agents keep working while Hermes quits, updates or crashes, and you reattach where they left off, so closing the app doesn't make a running agent boot again.

Hermes 2.3.4: the agent picker in the ⌘N sheet, with each installed agent's version.

The picker shows the version Hermes found for each agent, which is the number to paste next to your hyperfine table. Hermes is free to download, source-available under BSL 1.1, and runs on macOS, Windows and Linux. It isn't affiliated with Nous Research, Anthropic, OpenAI, GitHub or any other agent vendor. Download it or read the source on GitHub.

Post your four rows in the comments, with versions and OS. I'd like to see how far apart they land on different machines.

---

If this was useful

Everything Run 3 adds before your prompt is context you chose: an MCP server's tool list, an AGENTS.md, a memory file. Deciding what belongs in that prefix, and what can wait until the agent asks for it, is a big part of the Prompt Engineering Pocket Guide. If you run several agents a day, Hermes IDE puts them side by side, each in its own worktree.