Semaphore's sem-ai puts CI/CD in the agent loop: 4 design choices worth copying

Semaphore announced sem-ai this morning (September 24, 10:14 GMT): an open-source, agent-first CLI for their CI/CD platform, with an embedded MCP server and a plugin that ships agent skills for Claude Code and Codex. "Coding agents are changing where software development happens, but they still need reliable ways to test and verify the code they produce," said Marko Gaćeša, Head of Product at Semaphore.

The repo (semaphoreio/sem-ai) is Apache-2.0, written in Go, and has been public since May 7. At announcement time it sits at 12 stars, so the press release is ahead of the community traction. Full disclosure: I have no Go toolchain on this machine, so I read the docs and the source instead of building it. This is a design review, not a bench test.

The unit of work is a diagnosis, not a log fetch

The interesting command is diagnose. It chains workflow → pipeline → failed jobs → logs → parsed test results into one call. Compare that to the raw API path an agent would otherwise walk: find the project, list workflows, list pipelines, list jobs, fetch logs, parse test output. Six round trips and a pile of JSON in your context window, versus one structured answer.

The same thinking shows up across the command set:

status          pipeline state + block results for a branch
diagnose        full failure diagnosis in one call
health          pass rates, trends, verdict
critical-path   longest dependency chain
blast-radius    root failures vs cascading cancellations
rerun-failed    partial rebuild of failed blocks only

blast-radius is the one I want everywhere. When CI fails, most cancelled jobs are victims, not causes, and an agent that cannot tell the difference will burn tokens fixing jobs that were never broken. I measured 41k tokens of tool schemas loading before the first prompt with nine MCP servers attached. Compound commands are the cheapest way to give an agent reach without giving it a bloated tool surface.

MCP mode excludes the blocking commands

sem-ai mcp exposes the commands as native tools for Claude Code, Cursor, VS Code, and any MCP client. But two commands are deliberately left out: watch and promote-and-wait. They would block the single in-memory command tree that serves all tool calls. The documented alternative is status --exit-code in a poll loop.

That is the right trade. A tool call that hangs for twenty minutes eats the agent loop alive. If you expose your own system over MCP, keep every tool bounded, and push waiting into the caller. Long operations get a status endpoint, not a blocking tool.

Registration is the standard local shape:

{
  "mcpServers": {
    "semaphore": { "command": "sem-ai", "args": ["mcp"] }
  }
}

It runs on your machine and reuses ~/.sem.yaml, the same config file as the legacy sem CLI. Same tokens, same contexts, so an existing Semaphore setup works immediately.

The skills bundle teaches when, not just how

The plugin installs 17 skills (debug-pipeline, fix-flaky, testbox, gha-to-semaphore, and friends) plus namespaced slash commands. Yesterday I wrote about skills taking over from raw MCP for teaching procedures. sem-ai does both, and the split is the instructive part: the MCP surface carries the verbs, the skills carry the playbooks for when to reach for which verb.

/plugin marketplace add semaphoreio/sem-ai
/plugin install sem-ai@semaphoreio

Then /sem-ai:init bootstraps CI for a repo: it detects whether GitHub Actions workflows exist, applies Semaphore-side defaults (f1-standard-2 machine, ubuntu2404 image, checkout in the prologue, sem-version for language pinning, cache keyed on lockfile checksum, test-results publish in the epilogue), validates the YAML, wires required secrets, and opens a PR. That is procedure, not API access. A tool list cannot teach timing, and init sequencing is timing.

Mutations default to dry-run

Deploy commands run dry by default and require an explicit --confirm to act. The hosted MCP server at mcp.semaphoreci.com goes further: write tools like workflow_run are opt-in per organization, enabled through support rather than a checkbox.

Two tiers of friction, both sane: reads are cheap and self-serve, mutations are deliberate. For anything an agent can trigger, the safe default does more work than any permission prompt, because prompts get rubber-stamped at 2am.

What I'd check before adopting

Honest limits. The repo went public May 7 and had 12 stars at announcement, so expect early-stage rough edges. The CLI is Semaphore-only; if your CI lives in GitHub Actions, the transferable part is the pattern, not the tool (though /sem-ai:gha-to-semaphore translates workflows in that direction). And I have not executed any of this, so behavior claims come from the docs and source, not a run.

What to steal for your own setup

Whether or not you use Semaphore, the pattern generalizes:

1. Make one call do one diagnosis. Compound commands beat tool sprawl for both latency and context cost.
2. Keep MCP tools bounded. Exclude or split long-running operations, and offer a pollable status command instead.
3. Ship playbooks as skills next to the verbs. The tool list says what exists, the skill says when to use it.
4. Default every mutation to dry-run. Make the agent say --confirm.

If you maintain rules files and skills for a team of agents, versioning and validating those configs is the same discipline Semaphore applied here, and it is exactly what our AgentConfig Studio kit ($29) automates: pinned, validated config kits instead of drifted instructions.