Fable 5 vs Opus 5 vs Sonnet 5: A Security Code Audit Only Two Complete

Sonnet 5 is my daily driver. I use it for creative, coding, and agentic work (outside the homelab). I also scan my site periodically for vulnerabilities and bugs. I’ve used higher end models like Opus historically. So it was natural to ask: Has Sonnet gotten good enough for even specialized tasks like security audits?

Short answer: No. Opus 5 way outperformed Sonnet 5. And Fable 5 was… too good?

As we do on this site, we devised an experiment to find that answer. This site is powered by the blog engine, the content repo, and the Terraform template that provisions Coder workspaces, including the one I'm writing this from. Usually that's a single model, a single pass, done. This time I wanted to see what three different frontier models would each independently find in the exact same code, with zero awareness of each other or of the comparison itself. It turned into the most useful audit I've run, and the least well-behaved one.

Three models, one blind audit

A blind setup avoids the failure mode where one model's findings anchor the next one's. I built three isolated Coder workspaces. Each one ran a single model, Sonnet 5, Opus 5, or Fable 5, with its own fresh clone of the blog engine and the templates repo, pinned to the exact same commit in both. No shared filesystem. No shared chat history. No model knew the other two existed.

I used one identical prompt for all three sessions. I framed it as a routine audit, not a comparison, so no model would hedge or perform for a benchmark it didn't know it was in. Each session worked autonomously, asked no questions, and wrote its findings to a report file with a random name I'd assigned in advance. I knew the three filenames going in. I didn't learn which model produced which report until after I'd graded every finding against the real code myself. The full prompt is below if you want to run this yourself.

That blind grading step mattered more than I expected.

Fable 5 never finishes

Two of the three sessions produced a report. Fable 5 never finished. Not once. Not on the second try either. I re-ran it in a fresh workspace to rule out a fluke.

Both times, the session split into three sub-agents, one on the app's auth surface, one on client code and dependencies, one on the infra repo. Both times, the sub-agent auditing authentication, middleware, and rate-limiting got blocked outright by Anthropic's own content-safety classifier, flagged under its "cyber" policy category. This wasn't a refusal in the model's own voice. It was an upstream block, and it landed after the sub-agent had already read several files deep into exactly the code a security audit needs to cover.

Two failures in the identical subject area is not noise. Here's my read. Describing a real, specific auth or rate-limit weakness reads to an automated classifier as attack guidance, no matter how defensive the framing is. That's a genuine, ironic finding on its own. The task most worth automating is the one most likely to trip the safety net. I logged Fable 5 as a DNF and moved on instead of burning a third identical attempt on a coin flip.

Opus 5 finds the bug that matters

The two completed reports differed wildly in depth. This is exactly the result a blind setup surfaces.

	Sonnet 5	Opus 5
Total findings	11	50
High severity	1	6
Findings in the blog engine	5	30
Findings in the infra repo	6	20

I didn't just count findings. I went back to the pinned commit and verified a sample of claims from both reports against the real code. Both reports were accurate in everything I checked. Nobody hallucinated a vulnerability. The gap was depth, not correctness.

Opus 5 dug into the one place that actually mattered. My login rate limiter keys its bucket on the left-most entry of the X-Forwarded-For header. A client controls that value completely. Send a random value on every request and you get a fresh five-attempt bucket every time. That gives an attacker unlimited online brute force against my admin password, the single credential that gates write access to my content repo, my Dev.to publishing, and the button that spins up a real, billable coding agent. Opus 5 found it. Sonnet 5 didn't. Opus 5 also caught that the same limiter fails open and returns "allowed" whenever Redis is unreachable or unconfigured. That removes the only brake on that endpoint a second, independent way.

Opus 5 also did something I didn't ask for but appreciated. It tested one of its own theories, a possible path-traversal bug in image handling, against the live GitHub API. The theory didn't hold, so Opus 5 downgraded its own finding instead of reporting the scarier, unverified version. Sonnet 5 had one genuinely unique catch. My own workspace template tells every coding agent it has Docker available. It doesn't. Small, true, and I'm fixing it anyway.

Opus 5 fixes 48 of its own 50 findings

Grading who found the most bugs is fun, but fixing them was always the goal. I pointed Opus 5, the round's clear winner, back at its own report and told it to remediate everything, in priority order, across both repos, in feature branches, ending in a PR rather than a direct push to main. One exception. The infra repo's Terraform template provisions live workspaces, including the one doing the fixing, so that PR got a manual review and a manual apply from me, not an automatic push.

It fixed 48 of the 50 findings, merged as two PRs across the two repos. Two didn't get the report's literal suggested fix. One PAT-storage finding needed a different mitigation once the constraints became clear; DPAPI encryption doesn't actually help when both a SYSTEM account and an interactive user need to read the same file. The other, an unpinned dependency bump, would have forced a large unrelated rewrite, so it got pinned via an override instead, and npm audit still went to zero. Opus 5 also caught six new bugs in its own remediation branch during a self-review pass before I ever looked, including one that would have silently broken page hydration site-wide on the next minor Next.js upgrade.

The blog engine's fixes are live. I checked the deployed code directly before publishing this post. The infra repo's fixes are merged but not yet applied, Terraform apply, a Windows host, and a GPU host restart are real physical steps, not code, and I haven't done them yet. Fixed on paper and fixed in practice are two different states, and I'm only calling the first one done.

The prompt

I pasted this identical text into all three isolated sessions. Only the output filename changed per session.

You are performing a routine periodic security and bug audit of two
repositories checked out in ~/audit/: the-vibe-coder (a Next.js app) and
coder-templates (Terraform + Docker workspace template + infra scripts).
Audit exactly what's checked out at the current commit in each; do not
switch branches, pull, or fetch updates. This is a real audit whose
findings will be triaged and acted on, so be precise and avoid
speculative padding.

Scope: both repositories in full, except node_modules/, .next/, and
other build/vendor output. Look for:
- Security vulnerabilities: injection (SQL/command/template), auth and
  session handling flaws, authorization/access-control gaps, SSRF,
  secrets or tokens committed or logged, insecure direct object
  references, unsafe deserialization, XSS, path traversal, dependency
  vulnerabilities in package.json/package-lock.json, insecure
  Terraform/Docker defaults (e.g. exposed sockets, secrets baked into
  images, overly broad permissions), insecure defaults, timing attacks,
  rate-limit bypasses, CSRF.
- Correctness bugs: logic errors, race conditions, unhandled error
  paths, data-loss risks, off-by-one and edge-case handling.
- Do not report style/lint nitpicks or purely subjective architecture
  opinions unless they have a concrete correctness or security
  consequence.

Rules:
- Read-only audit. Do not modify, fix, or commit any code in either repo.
- Work autonomously. Do not ask me clarifying questions; make reasonable
  assumptions and note them in the report if relevant.
- Do not use tools that would leave a footprint outside these local
  checkouts (no gh pr create, no pushes, no external state changes).
  Read-only web/doc lookups (e.g. checking a CVE database or library
  docs) are fine.
- When you finish, write a single report to ~/audit/<REPORT_FILENAME>
  with this structure and nothing else outside it:

# Security & Bug Audit

## Executive Summary
(2-4 sentences: overall risk posture, most important finding)

## Findings
### [SEVERITY: Critical|High|Medium|Low] <short title>
- **Repo:** the-vibe-coder | coder-templates
- **Location:** path/to/file.ts:line
- **Category:** e.g. Auth, Injection, Secrets, Logic, Dependency, Infra
- **Description:** what's wrong
- **Impact:** what an attacker/user-facing failure looks like
- **Suggested fix:** concrete, short

(repeat per finding, ordered by severity)

## Assumptions / Caveats
(anything you weren't able to verify, or assumed)

Stop once the report is written. Do not start fixing issues.

The reports, unedited

This is raw output. The only edit is adding model names after the reveal. Expand any section to read the whole thing.

Sonnet 5's full report (11 findings)

# Security & Bug Audit

## Executive Summary
Overall risk posture is moderate: no critical, actively-exploitable vulnerabilities were found, but both repos have real gaps worth fixing. The most important finding is in `the-vibe-coder`, where outdated Next.js and MCP-related dependencies carry multiple high-severity CVEs (SSRF, DoS, endpoint disclosure) reachable through a publicly exposed `/api/mcp/[transport]` endpoint. A close second is a prompt-injection risk where untrusted Slack-submitted backlog text is later fed verbatim into an autonomous coding agent with real commit/PR/deploy capability. `coder-templates` is a personal/homelab template repo; its issues are mostly infra hardening gaps (unauthenticated LLM services bound to all interfaces, a static long-lived GitHub token) rather than externally exploitable flaws.

## Findings

### [SEVERITY: High] Outdated Next.js and transitive MCP dependencies with known high-severity CVEs
- **Repo:** the-vibe-coder
- **Location:** package.json:16 (`"next": "^16.2.6"`), package.json:19 (`"mcp-handler": "^1.1.0"`), package-lock.json (transitive: `hono`, `@hono/node-server`, `body-parser`, `fast-uri`, `ip-address`, `js-yaml`, `nanoid`, `postcss`, `sharp`)
- **Category:** Dependency
- **Description:** `npm audit` against the installed lockfile reports 12 advisories (7 high) against the resolved versions in this repo, including SSRF in Server Actions/rewrites, unauthenticated disclosure of internal Server Function endpoints, DoS in Server Actions and the Image Optimization API, and cache-confusion of response bodies in `next`. Separately, the MCP endpoint's dependency chain (`mcp-handler` → `@modelcontextprotocol/sdk` → `@hono/node-server` → `hono`/`body-parser`/`fast-uri`) pulls in a Hono CORS ReDoS, a Hono algorithmic-complexity DoS in its language middleware, a `body-parser` size-limit bypass, and `fast-uri` host-confusion parsing bugs. `/api/mcp/[transport]` is publicly reachable (exempted from the admin cookie in `src/middleware.ts:42`) and its request parsing/CORS handling may run before or independent of the bearer-token check in `withMcpAuth`, so these are internet-facing, not just admin-facing.
- **Impact:** A remote, unauthenticated attacker could trigger CPU-exhaustion DoS against the CORS/language middleware paths, exploit `fast-uri` host-confusion for trust-boundary bypass, or hit Next.js's own SSRF/DoS/endpoint-disclosure issues.
- **Suggested fix:** Run `npm audit fix` / bump `next` and `mcp-handler` (and their pinned transitive deps) to the patched versions; re-run `npm audit` to confirm zero high-severity findings before deploying.

### [SEVERITY: Medium] Untrusted backlog text flows into an autonomous coding-agent prompt with real commit/PR/deploy capability
- **Repo:** the-vibe-coder
- **Location:** src/app/api/todo/launch-agent/route.ts:22-27, src/app/api/slack/todo/route.ts:139-149
- **Category:** Logic / Injection (prompt injection, supply-chain)
- **Description:** `/api/slack/todo` accepts free-form text from any Slack user in the configured workspace (verified only via HMAC signature, not by user identity) and inserts it verbatim as a new bullet in `content/TODO.md` via `insertTodoItem`/`parseCommand`. The admin's "Launch Agent" button (`src/app/api/todo/launch-agent/route.ts`) later takes that exact, unsanitized item text and interpolates it directly into a prompt sent to the Coder Agents Chats API, with no filtering of the item text for embedded instructions before it reaches the agent prompt.
- **Impact:** Anyone able to post the Slack slash command (workspace membership, not admin identity, is the only gate) can craft a backlog item containing prompt-injection instructions that an admin later triggers via "Launch Agent," causing a fully-capable autonomous coding agent with real repo write/PR/deploy access to act on attacker-controlled instructions.
- **Suggested fix:** Treat backlog item text as untrusted content in the agent prompt: wrap it in explicit delimiters with an instruction that it is data, not instructions, restrict the agent's default scope/repos in the launch payload, and/or require the admin to review and explicitly confirm the literal task text before dispatch rather than trusting whatever is currently in `TODO.md`.

### [SEVERITY: Medium] Unauthenticated LLM inference API/UI bound to all network interfaces
- **Repo:** coder-templates
- **Location:** scripts/llama-generate.service:8, scripts/llama-embed.service:8-19, scripts/llama-generate-start.sh:9 (`HOST=0.0.0.0`)
- **Category:** Infra / Access control
- **Description:** Both the generation server (port 8080) and embedding server (port 8084) are started with `--host 0.0.0.0`, and neither passes `--api-key` (or any other auth flag) to `llama-server`. The generation service additionally omits `--no-webui`, so llama.cpp's built-in web chat UI is also exposed. No firewall rule scoping these ports is present anywhere in the repo (only the SSH port-22 firewall rule is created in `setup-openssh-server.ps1`, on the unrelated Windows side of the machine).
- **Impact:** Any device on the same LAN/Wi-Fi (not just the intended Tailscale mesh) can query the model, use compute for free, cause a GPU-bound denial of service against the workstation, or interact with a full unauthenticated chat UI. Since these run on the Linux host directly (not sandboxed in a container), this is a real host-level exposure.
- **Suggested fix:** Bind to `127.0.0.1` or the Tailscale interface IP only, and/or set `--api-key` with a token pulled from a protected secret; add `--no-webui` to the generation server unless the UI is intentionally desired; add an explicit firewall/ufw rule denying external access to 8080/8084.

### [SEVERITY: Medium] GITHUB_TOKEN env var defeats the external-auth refresh design for `gh` and other GH_TOKEN-aware tools
- **Repo:** coder-templates
- **Location:** docker/main.tf:181-187 (agent `env` block) combined with lines 63-66 (`gh auth login --with-token`)
- **Category:** Auth / Logic
- **Description:** The template goes to considerable effort (see the comment block at lines 47-51) to make GitHub auth "work in ALL shell contexts" by having the git credential helper call `coder external-auth access-token github` fresh on every invocation. However, `GITHUB_TOKEN`/`GH_TOKEN` are also set as static values in the `coder_agent.main.env` block, which become fixed container environment variables for the container's entire lifetime (captured once at agent/container start). Because `gh` gives precedence to these env vars over stored credentials, the `gh auth login --with-token` call at startup is effectively cosmetic: `gh` will keep using the frozen startup-time token rather than any refreshed credential.
- **Impact:** Once the underlying OAuth token expires (typically hours), `gh` and any other GITHUB_TOKEN-aware tool (npm packages, scripts, Vercel CLI, etc.) inside a long-running workspace will start failing with stale/expired-credential errors, even though plain `git` operations keep working via the credential helper. This is confusing and contradicts the documented intent ("works in ALL shell contexts").
- **Suggested fix:** Don't set `GITHUB_TOKEN`/`GH_TOKEN` as static agent env vars; instead export them lazily per-shell (as already done for `~/.profile`) or wrap `gh` in a shell function/alias that fetches a fresh token each call, consistent with the git credential helper approach.

### [SEVERITY: Low] MCP bearer-token comparison leaks token length via early-return timing
- **Repo:** the-vibe-coder
- **Location:** src/lib/mcp-auth.ts:8-15
- **Category:** Auth (timing side channel)
- **Description:** `timingSafeEqual` in `mcp-auth.ts` returns immediately on `a.length !== b.length` before doing any constant-time work, so a request with a token of the wrong length returns faster than one with the correct length. The codebase already recognizes and fixes this exact pattern elsewhere: `src/lib/auth.ts:63-78` explicitly hashes both inputs first specifically to keep the comparison constant-time, noting that an early length-mismatch return would leak the password length via timing.
- **Impact:** A remote attacker probing `/api/mcp/*` can use timing to incrementally determine the length of `MCP_API_TOKEN`, narrowing the brute-force search space (impact is limited in practice by network jitter and the token still requiring full-value brute force, but this is the same class of bug the repo's own `auth.ts` fix explicitly calls out and remediates).
- **Suggested fix:** Apply the same fix used in `src/lib/auth.ts`: hash both the supplied token and `MCP_API_TOKEN` (e.g., SHA-256) before calling a constant-time comparison, or pad/compare fixed-length buffers without an early length check.

### [SEVERITY: Low] Public post lookups build filesystem paths from the raw slug without the shared sanitizer
- **Repo:** the-vibe-coder
- **Location:** src/lib/posts.ts:104-133 (`_getPostBySlug`, `getPostBySlugAdmin`), used by src/app/posts/[slug]/page.tsx:65, src/app/posts/[slug]/raw/route.ts:28, src/app/admin/preview/[slug]/page.tsx:30, src/app/admin/edit/[slug]/page.tsx
- **Category:** Logic / Path handling (defense-in-depth gap)
- **Description:** Every path that talks to the GitHub Contents API (posts, images, settings, TODO, MCP tools) is routed through `sanitizeSlug`/`isValidImageRepoPath`/`isValidSlug`, per the comment in `src/lib/slug.ts` explaining this was added specifically because an unsanitized slug once reached a repo path. `posts.ts`'s filesystem-backed lookups (`_getPostBySlug`, `getPostBySlugAdmin`, `_getAllPosts`'s per-file logic) are the one remaining place that builds a path (`path.join(POSTS_DIR, \`${slug}.mdx\`)`) directly from the route param with no such validation.
- **Impact:** If the `slug` route param can ever contain path-traversal sequences (e.g., via an encoded `/` decoded by the framework into an actual path separator), this reads arbitrary `.mdx` files from the filesystem outside `content/posts`, constrained only by the file needing a literal `.mdx` extension and being reachable relative to `process.cwd()`. This is a real inconsistency with the rest of the codebase's own defensive posture even where current framework behavior may not be directly exploitable.
- **Suggested fix:** Route `slug` through `sanitizeSlug` (or an equivalent single-segment allowlist check) in `posts.ts` before building any filesystem path, matching the pattern already used for every GitHub-backed route.

### [SEVERITY: Low] GitHub Actions workflow has no explicit `permissions` block
- **Repo:** the-vibe-coder
- **Location:** .github/workflows/giscus-notify.yml:1-9
- **Category:** Infra / CI
- **Description:** The `giscus-notify` workflow does not set `permissions: {}` at the workflow level or a scoped `permissions:` under the job, so the job's `GITHUB_TOKEN` receives whatever default permissions are configured at the repository/org level rather than an explicit minimal grant.
- **Impact:** If the org/repo default token permissions are ever broader than "read," this workflow (which never actually reads/writes repo contents via the API) would run with unnecessarily broad `GITHUB_TOKEN` privileges, widening the blast radius if the workflow or a future edit to it is ever compromised.
- **Suggested fix:** Add `permissions: {}` at the workflow's top level (the job needs no GitHub API access at all, only the Slack webhook secret).

### [SEVERITY: Low] Documented "Docker" capability does not exist in the workspace image
- **Repo:** coder-templates
- **Location:** docs/system-instructions.md:8 vs docker/build/Dockerfile (no docker install) and docker/main.tf (no `docker.sock` mount, no privileged flag)
- **Category:** Logic / Correctness
- **Description:** The agent system prompt tells every coding agent running in the workspace it has Docker available as a tool, but the Dockerfile never installs the Docker CLI/daemon, and `main.tf`'s `docker_container` resource mounts no `docker.sock` and grants no extra capabilities.
- **Impact:** Agents following the system prompt will attempt to use Docker, fail, and may try workarounds (e.g., installing docker-in-docker inside an unprivileged container, or requesting privilege escalation), wasting time and potentially prompting risky "fix it yourself" behavior per the same instructions file's proactive-agent policy.
- **Suggested fix:** Either remove "Docker" from the documented capability list, or actually provision it (e.g., mount the host socket deliberately, understanding the security tradeoff of doing so).

### [SEVERITY: Low] GitHub OAuth token materialized as a persistent, broadly-readable container secret
- **Repo:** coder-templates
- **Location:** docker/main.tf:184-185
- **Category:** Secrets
- **Description:** `data.coder_external_auth.github.access_token` is placed directly into the agent's `env` block, which becomes a plaintext environment variable for the container's `PID 1` and every child process, and is also recorded in the Terraform apply's data (state/logs on the Coder control plane), for the lifetime of the workspace.
- **Impact:** Any process or tool running in the container (not just intended git/gh usage) can read a live GitHub token with whatever external-auth scopes are configured, widening the blast radius of any RCE or malicious dependency executed inside the workspace, compared to the on-demand `coder external-auth access-token github` call already used by the credential helper.
- **Suggested fix:** Prefer fetching the token on demand (as the credential helper does) rather than baking it into the process environment for the container's whole lifetime; if some tools require the env var, consider scoping/rotating it or documenting the exposure as an accepted risk.

### [SEVERITY: Low] `administrators_authorized_keys` entry is appended non-idempotently
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/setup/setup-openssh-server.ps1:50-53
- **Category:** Logic
- **Description:** `Add-Content -Path $authKeysPath -Value $pubKey -Force` unconditionally appends the public key without first checking whether it's already present.
- **Impact:** Re-running this one-time bootstrap script (e.g. after a reinstall or to fix another step) duplicates the key entry; harmless functionally today, but it's not safe to re-run as the script's own docstring implies ("One-time bootstrap... Run once"), and could mask an actual key-replacement need if the key is ever rotated.
- **Suggested fix:** Check for an existing matching entry (e.g. via a regex match against the file) before appending, or rewrite the file to a de-duplicated set.

### [SEVERITY: Low] Personal identifying details and internal infrastructure specifics committed to a "template" repo
- **Repo:** coder-templates
- **Location:** docker/main.tf:163-178 (base64-encoded per-user skills referencing real usernames, real Vercel app URLs, private repo names) and scripts/* (real usernames, hardware specs)
- **Category:** Information disclosure (informational)
- **Description:** The Terraform template embeds real GitHub usernames, private repository names, personal app URLs, and detailed home-lab hardware/network specs directly in what is documented as a reusable Coder Registry template.
- **Impact:** Low as a vulnerability per se (no secrets/tokens were found embedded), but if this repository's visibility were ever changed to public, or shared as a generic template, it would leak the operator's personal infrastructure topology and third-party service endpoints, which is useful reconnaissance for a targeted attack.
- **Suggested fix:** Keep personal/user-specific customization in a separate, clearly-marked private overlay rather than mixed into the general-purpose template file, if this repo's visibility could ever change.

## Assumptions / Caveats
- `the-vibe-coder` was reviewed at its checked-out detached HEAD (`99c08f6`); `coder-templates` at detached HEAD (`49718b4`). No branch switches, fetches, or writes were performed in either repo.
- `npm audit` for `the-vibe-coder` was run against the already-installed `node_modules`/lockfile in that workspace; results reflect resolved versions at audit time and should be re-verified at fix time.
- Exploitability of the Next.js/`fast-uri`/Hono advisories and the slug path-traversal item was assessed from advisory descriptions and code inspection, not live exploitation against a running instance (out of scope for a read-only audit).
- The Slack `/todo` → Launch Agent trust-boundary judgment assumes "any member of the configured Slack workspace" is a weaker trust boundary than "the site admin"; if Slack workspace membership is already restricted to fully-trusted operators, actual risk is lower than rated.
- `coder-templates` is treated as a personal/homelab repository rather than a multi-tenant SaaS product; several findings (static GITHUB_TOKEN, personal usernames) are flagged for completeness even though their severity would be higher in a shared/enterprise deployment.
- Actual LAN/router-level firewalling for the 0.0.0.0-bound llama.cpp services could not be verified; that finding is based solely on what is provisioned in-repo, where no compensating control (bind restriction, API key, or firewall rule) is present.
- `scripts/benchmarks/round5/fixtures/{express-bug-app,flask-pagination-app}` in `coder-templates` contain deliberately-planted bugs and fake hostnames used as LLM benchmark test fixtures; these were excluded from findings as intentional test data, not production code.
- No dependency-manifest files (package.json/requirements.txt/go.mod) exist in the audited scope of `coder-templates`, so no CVE/dependency-vulnerability scan was possible beyond the pinned versions visible in its Dockerfile, which were not found to have known critical CVEs as pinned.
- No build/lint/test commands were executed as part of either audit (read-only scope); no runtime/dynamic testing was performed.

Opus 5's full report (50 findings)

# Security & Bug Audit

## Executive Summary

Both repositories are single-operator projects whose security posture depends on a small number of choke points, and several of those choke points fail open. The most important finding is that the-vibe-coder's admin login brute-force protection can be bypassed outright: the rate limiter keys on the attacker-controlled left-most `X-Forwarded-For` entry and returns "allow" whenever Redis is unconfigured or erroring, leaving `ADMIN_PASSWORD` as the only barrier to full write access on the content repo, Dev.to publishing, and billable agent workspace creation. Secondary themes are dependency exposure (13 npm advisories, including a middleware-bypass class in the pinned `next` 16.2.10, which is exactly the boundary this app's auth relies on), unauthenticated/unpinned infrastructure in coder-templates (llama.cpp servers on `0.0.0.0`, `curl | bash` installs as root), and a cluster of GitHub read-modify-write races and fail-open orphan detection that risk silent content loss. No committed secrets were found in either repository.

## Findings

### [SEVERITY: High] Login rate limiter is bypassable via a spoofed X-Forwarded-For header
- **Repo:** the-vibe-coder
- **Location:** src/lib/rate-limit.ts:83-92 (`clientIp`), consumed at src/app/api/auth/login/route.ts:43-48
- **Category:** Auth / Rate-limit bypass
- **Description:** `clientIp` takes the *first* (left-most) entry of `X-Forwarded-For`, which is the portion a client sets freely; only the right-most hop added by the proxy is trustworthy. Every limiter key (`ratelimit:login:*`, `ratelimit:analytics:*`, `ratelimit:share-image:*`, `ratelimit:mcp:*`) is therefore attacker-partitionable.
- **Impact:** An attacker sends a random `X-Forwarded-For` per request and gets a fresh 5-attempt bucket each time, giving unlimited online brute force against `ADMIN_PASSWORD` (the sole credential for repo write, Dev.to publishing, and `/api/todo/launch-agent`). Also permits unbounded Redis writes from `/api/analytics/track` and unbounded OG-image render cost from `/api/share-image`.
- **Suggested fix:** Use the platform-provided client IP (`x-vercel-forwarded-for` / `x-real-ip` on Vercel) or the right-most XFF hop; never trust the left-most entry.

### [SEVERITY: High] Rate limiter fails open when Redis is unconfigured or erroring
- **Repo:** the-vibe-coder
- **Location:** src/lib/rate-limit.ts:44-47 and :70-73
- **Category:** Auth
- **Description:** `rateLimit` returns `{ ok: true }` when `KV_REST_API_URL`/`KV_REST_API_TOKEN` are unset and again in the `catch` on any Redis error (including transient Upstash 429s). The login route has no secondary throttle behind it.
- **Impact:** A Redis outage, a missing env var in a preview/self-hosted deployment, or induced Upstash throttling silently removes the only brute-force control on `/api/auth/login`.
- **Suggested fix:** Fail closed for the login key specifically (503 rather than allow), or add a per-instance in-memory backstop counter used when Redis is unavailable.

### [SEVERITY: High] Vulnerable `next` version and 12 other npm advisories in the lockfile
- **Repo:** the-vibe-coder
- **Location:** package.json:29, package-lock.json (`node_modules/next` = 16.2.10)
- **Category:** Dependency
- **Description:** `npm audit --package-lock-only` reports 13 vulnerabilities (8 high, 4 moderate, 1 low). The locked `next` 16.2.10 falls inside the vulnerable range `9.3.4-canary.0 - 16.3.0-preview.10`, covering App Router middleware/proxy bypass (GHSA-6gpp-xcg3-4w24), SSRF via rewrite destinations (GHSA-p9j2-gv94-2wf4), cache confusion (GHSA-68g3-v927-f742, GHSA-4633-3j49-mh5q), and Server Function endpoint disclosure (GHSA-955p-x3mx-jcvp). Also vulnerable: `js-yaml` 4.3.0/3.15.0 via `gray-matter` (GHSA-5p4m-2wfm-xmqj), `postcss` 8.4.31 nested under `next`, `sharp` 0.34.5 (GHSA-f88m-g3jw-g9cj), `@hono/node-server` <2.0.5 via `mcp-handler` (serve-static path traversal), `ip-address` 10.2.0 (SSRF), plus `brace-expansion`, `fast-uri`, `nanoid` 3.3.11, `body-parser`, `hono`.
- **Impact:** A middleware-bypass advisory is directly load-bearing here, since `src/middleware.ts` is the *only* authorization check for 14 privileged API routes (see the Medium finding below). The remainder are DoS and cache/SSRF exposure.
- **Suggested fix:** `npm audit fix` and redeploy; the `mcp-handler` remediation is a major bump to 2.1.0, so exercise `/api/mcp/[transport]` afterwards.

### [SEVERITY: High] LLM inference servers bound to 0.0.0.0 with no authentication
- **Repo:** coder-templates
- **Location:** scripts/llama-generate-start.sh:9 (`HOST=0.0.0.0`, used lines 28-86), scripts/llama-embed.service:18-19
- **Category:** Infra
- **Description:** Both llama.cpp servers listen on all interfaces (8080 generation, 8084 embedding) with no `--api-key` and no authenticating reverse proxy. The generation service also omits `--no-webui` (the embed unit sets it at line 17), so the browser UI is exposed too. Per docs/sff-migration-checklist.md:188-207 the host runs Tailscale and a Cloudflare tunnel, so "LAN only" is not a safe assumption.
- **Impact:** Any host that can reach the machine can consume the GPU, run arbitrary prompts, and read `/props` (model paths, sampling config, chat template) without credentials.
- **Suggested fix:** Set `HOST=127.0.0.1` and `--host 127.0.0.1` in the embed unit, or add `--api-key`; add `--no-webui` to the generation service.

### [SEVERITY: High] Unpinned `curl | bash` installs run as root during workspace image build
- **Repo:** coder-templates
- **Location:** docker/build/Dockerfile:1, :22, :43, :46
- **Category:** Infra / Supply chain
- **Description:** `FROM codercom/enterprise-base:ubuntu` is a floating tag with no digest; `curl -fsSL https://deb.nodesource.com/setup_20.x | bash -`, `curl -LsSf https://astral.sh/uv/install.sh | sh`, and `npm install -g vercel` all execute unverified remote content as `USER root` (line 3) with no checksum or version pin.
- **Impact:** A compromised or MITM'd upstream response yields root code execution at build time and a backdoored image for every workspace of every user.
- **Suggested fix:** Pin the base image by digest, download installers to a file and verify a checksum before executing, and pin `vercel` and the NodeSource setup script to explicit versions.

### [SEVERITY: High] Benchmark harness executes model-generated code on the host with no sandbox
- **Repo:** coder-templates
- **Location:** scripts/benchmarks/round5/benchmark.py:273-285, :446-454, :472-484 (also :25-27)
- **Category:** Infra / Arbitrary code execution
- **Description:** Model output is written to `todo.py` / a `.ts` file and executed via `subprocess.run([sys.executable, app_path] + args)` and `npx --yes tsx`. The only containment is a `TemporaryDirectory` and a 10-30s timeout; the process runs as the invoking user with full filesystem and network access. Separately, lines 25-27 silently run `pip install requests --break-system-packages` on ImportError, mutating the system Python.
- **Impact:** A hallucinated or adversarial generation (`rm -rf ~`, credential exfiltration, outbound HTTP) executes with the operator's privileges on the workstation that also hosts Coder, Docker, and Tailscale.
- **Suggested fix:** Execute fixtures in a disposable container (`docker run --rm --network none --read-only`) or a `bwrap`/`nsjail` sandbox; make `requests` a documented requirement instead of auto-installing.

### [SEVERITY: Medium] All privileged API authorization lives in middleware only
- **Repo:** the-vibe-coder
- **Location:** src/middleware.ts:47-73; handlers under src/app/api/{posts,images,settings,generate-post,syndicate,todo}
- **Category:** Auth
- **Description:** Fourteen privileged handlers perform no in-handler session check; the comment at src/app/api/todo/launch-agent/route.ts:30-31 documents this as deliberate. `src/middleware.ts:38` and `:40` also allow `/api/auth/**` and `/api/slack/**` wholesale by prefix, so any future route added under those paths is unauthenticated by default.
- **Impact:** A single mistake in `config.matcher`, a Next.js middleware-bypass advisory (the pinned version is affected, see above), or an invocation path that skips middleware yields unauthenticated repo write and delete, Dev.to publishing, and billable workspace creation.
- **Suggested fix:** Add `if (!(await getSession())) return 401` at the top of each privileged handler; src/app/api/auth/check/route.ts:14 already shows the one-line pattern.

### [SEVERITY: Medium] Stored XSS in the admin TODO inline-Markdown renderer
- **Repo:** the-vibe-coder
- **Location:** src/lib/todo.ts:135-137 and :146-151; sink at src/components/admin/TodoReorderList.tsx:157
- **Category:** XSS
- **Description:** `escapeHtml` escapes `&`, `<`, `>` but not `"`, and the link rule interpolates the captured URL into a double-quoted `href` with the permissive class `[^\s)]+`. A crafted `TODO.md` bullet with an unescaped quote in the link URL breaks out of the `href` attribute, letting an attacker-controlled event handler attribute get injected. The CSP at next.config.ts:52 includes `script-src 'unsafe-inline'`, so inline handlers are not blocked. (Working payload omitted as a precaution.)
- **Impact:** Script execution in the authenticated admin's browser on `/admin/todo`, in a session that can write to the content repo. `TODO.md` is also written by the Slack command and by agents, so this is reachable without a direct human commit.
- **Suggested fix:** Escape `"` and `'` in `escapeHtml`, and tighten the URL class to `(https?:\/\/[^\s)"'<>]+)`.

### [SEVERITY: Medium] `javascript:` URLs pass through the MDX anchor component
- **Repo:** the-vibe-coder
- **Location:** src/components/MDXComponents.tsx:61-87
- **Category:** XSS
- **Description:** `const isExternal = href.startsWith("http")` routes everything else to ``, including `javascript:` and `data:text/html,`. Post bodies are generated by Claude from transcripts (src/lib/claude.ts) and committed by the admin UI; no stage validates link schemes.
- **Impact:** Script execution in every reader's browser on a published post. `'unsafe-inline'` in the CSP does not restrict `javascript:` navigations.
- **Suggested fix:** Parse the href and allow only `http(s):`, `mailto:`, and site-relative (`/`, `#`) values; render the text without a link otherwise.

### [SEVERITY: Medium] Fail-open orphan detection can mark in-use images as deletable
- **Repo:** the-vibe-coder
- **Location:** src/lib/images.ts:117-131, :190-201, :230, :252; UI at src/components/admin/ImageManager.tsx:263-283
- **Category:** Logic / Data loss
- **Description:** `loadStaticImageReferences()` and `safePostIndex()` swallow all errors and return an empty `Set`/`[]`. A null match then sets `orphaned: true`. If `public/static-image-refs.json` is absent (prebuild not run, stripped deploy) or `content/posts` is missing, every branding asset and post image is presented under "Orphaned" with "Nothing references this file" and a one-click "Delete all". The file's own comment at lines 96-100 records that exactly this class of file was deleted once before as a false orphan.
- **Impact:** Irreversible deletion of in-use assets on the content repo's `main` branch.
- **Suggested fix:** Return `null` on read failure to distinguish "manifest missing" from "manifest empty", and suppress orphan flagging (or disable the delete buttons) when either signal is unavailable.

### [SEVERITY: Medium] `POST /api/posts` silently overwrites an existing post
- **Repo:** the-vibe-coder
- **Location:** src/app/api/posts/route.ts:83-85; src/lib/github.ts:32-51
- **Category:** Logic / Data loss
- **Description:** The create path never checks for an existing file; `commitFile` fetches the current SHA and upserts. The MCP `create_post` tool does check and returns `post_exists` (src/app/api/mcp/[transport]/route.ts:310-325), so the inconsistency is confirmed. Because `sanitizeSlug` collapses input, `"My Post!"`, `"my/post"`, and `"my--post"` all normalize to `my-post`.
- **Impact:** A new draft can clobber a live published post in one request; recoverable only from Git history.
- **Suggested fix:** `readFile(path)` first and return 409 when it exists, mirroring the MCP tool.

### [SEVERITY: Medium] Lost-update race on every read-modify-write of repo files
- **Repo:** the-vibe-coder
- **Location:** src/lib/github.ts:32-51 and :85-116; callers at src/app/api/posts/route.ts:145-171, src/app/api/todo/route.ts:24-42, src/app/api/syndicate/devto/route.ts:23-71, .../bulk/route.ts:29-93, src/components/admin/DraftsList.tsx:60-171
- **Category:** Logic / Race condition
- **Description:** Each flow reads content, mutates it in memory, then calls `commitFile`, which re-fetches the blob SHA at write time and therefore always wins. The SHA read at load time is never sent as a precondition. `src/lib/todo.ts:92-124` gets this right with `TodoConflictError`; the post path does not.
- **Impact:** Two concurrent writers (admin UI, MCP agent, Slack `/todo`, scheduled publish) silently overwrite each other, losing post edits.
- **Suggested fix:** Thread the load-time SHA through the API and let GitHub's 409 surface instead of re-reading.

### [SEVERITY: Medium] `PUT /api/settings` persists the entire unvalidated request body
- **Repo:** the-vibe-coder
- **Location:** src/app/api/settings/route.ts:22-51
- **Category:** Logic
- **Description:** Only `stylePrompt` and `defaultTags` types are checked; `prompts` and arbitrary extra keys of any size are written verbatim to `content/settings.json`. `stylePrompt`/`prompts[*].prompt` become the system prompt at src/lib/claude.ts:39-42, and `getSettings` (src/lib/settings.ts:54) silently drops a malformed `prompts` map.
- **Impact:** Unbounded file growth in the content repo and persistent system-prompt poisoning for all future generations, surfacing as a silent behavior change rather than an error. Admin-scoped, so integrity rather than escalation.
- **Suggested fix:** Build the persisted object explicitly from validated fields, validate `prompts` with the existing `isPromptMap`, and cap sizes.

### [SEVERITY: Medium] `fixDateYear` throws on unquoted YAML dates, 500-ing publish and update
- **Repo:** the-vibe-coder
- **Location:** src/app/api/posts/route.ts:11-26
- **Category:** Logic
- **Description:** `gray-matter` parses an unquoted YAML `date: 2020-01-01` into a JavaScript `Date`, which has no `.replace`. The guard checks truthiness only, never type. src/app/api/generate-post/route.ts:104-108 handles the `data.date instanceof Date` case, confirming the inconsistency.
- **Impact:** Any post with an unquoted frontmatter date more than a year stale cannot be created or updated; the failure surfaces as an opaque 500.
- **Suggested fix:** Normalize first: `const d = data.date instanceof Date ? data.date.toISOString().split("T")[0] : String(data.date)`.

### [SEVERITY: Medium] A failed `EXPIRE` permanently bricks a rate-limit key
- **Repo:** the-vibe-coder
- **Location:** src/lib/rate-limit.ts:50-56
- **Category:** Logic
- **Description:** If `INCR` succeeds but `EXPIRE` throws, the `catch` at line 70 swallows it and the key persists with no TTL. `count === 1` never recurs, so the TTL is never set; once the counter passes `limit`, `ttl` returns `-1` and the branch at :59-66 blocks that key indefinitely while advertising a bogus `retryAfter`.
- **Impact:** Permanent login lockout for the affected bucket with no self-healing path; requires manual Redis intervention.
- **Suggested fix:** Make increment and expiry atomic (`SET key 0 EX  NX` then `INCR`, a Lua script, or a pipeline).

### [SEVERITY: Medium] GitHub OAuth token exported into workspace env and persisted to disk
- **Repo:** coder-templates
- **Location:** docker/main.tf:184-185, :66, :301-322
- **Category:** Secrets
- **Description:** `GITHUB_TOKEN`/`GH_TOKEN` are set from `data.coder_external_auth.github.access_token` in `coder_agent.env`, which writes them into Terraform state and exposes them to every process in the container via `/proc/*/environ`. `gh auth login --with-token` additionally persists the token to `~/.config/gh/hosts.yml` on the retained `docker_volume.home_volume` (`lifecycle { ignore_changes = all }`), so it survives stop and rebuild. The file already has a better mechanism: the credential helper at :53-54 and the `~/.profile` export at :60 re-fetch a fresh token per invocation.
- **Impact:** A long-lived GitHub token in Terraform state and on a persistent volume, retrievable after the workspace is stopped.
- **Suggested fix:** Drop `GITHUB_TOKEN`/`GH_TOKEN` from the `env` block and rely on the per-call `coder external-auth access-token` path; if `gh` needs auth, pass `GH_TOKEN` at call time.

### [SEVERITY: Medium] MCP secrets written world-readable before `chmod 600`
- **Repo:** coder-templates
- **Location:** docker/main.tf:87, :101, :117, :120
- **Category:** Secrets
- **Description:** `.mcp.json` and both `.mcp.json.tmp` files are created with the default umask (0644) while already containing `Bearer $FITNESS_TRACKER_MCP_TOKEN` / `$VIBESCODER_MCP_TOKEN`. The `.tmp` files are never chmod'd at all before `mv`; the `chmod 600` lands only after every write.
- **Impact:** A window in which bearer tokens are readable by any other UID in the container and by anything reading the mounted home volume from the host.
- **Suggested fix:** `umask 077` before the block, or `install -m600 /dev/null ` first and chmod each temp file before writing.

### [SEVERITY: Medium] Unpinned external skill repo cloned and trusted on every workspace start
- **Repo:** coder-templates
- **Location:** docker/main.tf:139-160
- **Category:** Infra / Supply chain
- **Description:** `git clone`/`git pull` of `https://github.com/carryologist/agent-skills.git` tracking `main`, with no commit pin or signature check, then every `workspace/*/` directory is symlinked into `~/.agents/skills` where the coding agent reads them as instructions. All errors are suppressed with `2>/dev/null || true`, so tampering or a failed pull is invisible.
- **Impact:** Anyone who can push to that repo silently changes agent behavior in every workspace on the next start, with the workspace's GitHub token in scope.
- **Suggested fix:** Pin to a verified tag or commit SHA, or vendor the skills into the image; log failures rather than discarding them.

### [SEVERITY: Medium] SSH exposed to all networks with password auth left enabled
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/setup/setup-openssh-server.ps1:32-38, :45-49
- **Category:** Infra / Auth
- **Description:** `New-NetFirewallRule ... -LocalPort 22` is created with no `-Profile` and no `-RemoteAddress`, allowing inbound 22 from any source on every profile including Public; line 37 re-enables the rule unconditionally if it was deliberately disabled. The script never edits `sshd_config`, so `PasswordAuthentication` stays at the Windows default (`yes`) for an account the script adds to Administrators. README.md:126-135 states this is only meant to be reachable over Tailscale.
- **Impact:** Password-guessable administrator SSH on any network the machine joins, including untrusted Wi-Fi.
- **Suggested fix:** Scope the rule (`-Profile Private -RemoteAddress 100.64.0.0/10`) or bind `ListenAddress` to the Tailscale IP, and set `PasswordAuthentication no` + `PubkeyAuthentication yes` before restarting sshd.

### [SEVERITY: Medium] Unpinned PowerShell Gallery module installed machine-wide by a SYSTEM task
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/orchestrator/update-orchestrator-system.ps1:61-68; same pattern in update-orchestrator-notify.ps1:19-26
- **Category:** Infra / Supply chain
- **Description:** `Install-Module -Name PSWindowsUpdate -Force -Scope AllUsers` followed by `Import-Module`, with no `-RequiredVersion`, no `-Repository`, and no signature or catalog validation; `-Force` suppresses the untrusted-repository prompt. This runs as `NT AUTHORITY\SYSTEM` (register-update-orchestrator-tasks.ps1:32) on weekly and at-startup triggers.
- **Impact:** Whatever module version is current at run time is installed system-wide and loaded into a SYSTEM process. A compromised version, or a higher-priority repository registered later, is full machine compromise.
- **Suggested fix:** Pre-install a pinned version and use `-Repository PSGallery -RequiredVersion ` plus signature verification; fail the step rather than installing on demand.

### [SEVERITY: Medium] Unattended auto-reboot fires on every boot
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/orchestrator/update-orchestrator-system.ps1:73-74; register-update-orchestrator-tasks.ps1:27, :29-30, :39
- **Category:** Logic / Data loss
- **Description:** `Install-WindowsUpdate -AcceptAll -AutoReboot` runs whenever `-Unattended` is passed. The system script's own header (lines 8-11) says `-Unattended` is "only for the scheduled/overnight run", but registration passes it to a task that also fires `-AtStartup`. There is no check for an interactive logon session.
- **Impact:** The machine can force a reboot moments after a user boots into Windows, discarding unsaved work.
- **Suggested fix:** Register two tasks (weekly with `-Unattended`, at-startup without), or gate `-AutoReboot` on there being no interactive session.

### [SEVERITY: Medium] GitHub PAT stored in plaintext, protected only by a manual documented step
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/orchestrator/update-orchestrator-github-sync.ps1:33, :41; windows-aint-no-problem/README.md:94-101
- **Category:** Secrets
- **Description:** A Contents:read-write PAT is written to a plaintext file on the orchestrator host and read back with `Get-Content -Raw`. No script sets or verifies the ACL; hardening is a copy-paste `icacls` block in the README that must be re-run after any rotation. The parent directory grants broad local read+execute by inheritance. (Exact path redacted as a precaution.)
- **Impact:** A repo-write PAT recoverable by any local user whenever the manual step is skipped or undone.
- **Suggested fix:** Store the token with DPAPI or Windows Credential Manager; have the script assert the ACL and refuse to read a world-readable file.

### [SEVERITY: Medium] Missing `permissions` block in the giscus notification workflow
- **Repo:** the-vibe-coder
- **Location:** .github/workflows/giscus-notify.yml:1-13
- **Category:** Infra / CI
- **Description:** No `permissions:` at workflow or job level, so the job runs with the repository's default `GITHUB_TOKEN` scope (write-all where the default has not been changed), despite needing no token scopes at all. The `${{ }}` handling itself is safe (values pass through `env:` and `jq --arg`), so there is no script injection. Separately, the comment at lines 10-12 claims owner comments are skipped, but the condition only checks the category, so self-notifications still fire.
- **Impact:** A write-capable token is exposed to a job that processes attacker-influenced comment payloads.
- **Suggested fix:** Add `permissions: {}` at the top level and grant nothing at job level.

### [SEVERITY: Low] MCP token comparison leaks token length via early return
- **Repo:** the-vibe-coder
- **Location:** src/lib/mcp-auth.ts:8-14
- **Category:** Auth / Timing
- **Description:** `if (a.length !== b.length) return false` precedes the constant-time XOR loop, so the comparison is constant-time only for equal-length inputs. src/lib/auth.ts:63-77 documents and fixes exactly this pattern for the admin password; `mcp-auth.ts` never received the same treatment.
- **Impact:** Narrows the search space for `MCP_API_TOKEN`. Low practical exploitability over network jitter.
- **Suggested fix:** Hash both inputs to a fixed 32 bytes and use `crypto.timingSafeEqual`, matching `auth.ts`.

### [SEVERITY: Low] Slack replay window is skipped when the timestamp is non-numeric
- **Repo:** the-vibe-coder
- **Location:** src/app/api/slack/todo/route.ts:20-21
- **Category:** Auth
- **Description:** `Math.abs(now - Number(timestamp)) > 300` evaluates to `false` when `Number(timestamp)` is `NaN`, so the freshness check passes. The HMAC still covers the timestamp, so forgery is not possible; exploitation requires a captured request that already carried a non-numeric timestamp, which Slack does not send.
- **Impact:** Replay protection is unenforced for a malformed-timestamp request; effectively unreachable with a legitimate Slack sender.
- **Suggested fix:** `const ts = Number(timestamp); if (!Number.isFinite(ts) || Math.abs(now - ts) > 300) return false;`

### [SEVERITY: Low] Sessions cannot be revoked; logout is client-side only
- **Repo:** the-vibe-coder
- **Location:** src/lib/auth.ts:14-30; src/app/api/auth/logout/route.ts:4-8; src/middleware.ts:66
- **Category:** Auth
- **Description:** A 7-day HS256 JWT is minted with no `jti`; `verifySession` checks only the signature, logout merely clears the cookie, and there is no denylist. Middleware calls bare `jwtVerify` and never asserts the `role: "admin"` claim it signs, so that claim is decorative. `/api/auth/logout` also has no origin check (unlike login), though `sameSite: "strict"` blocks the cross-site form post.
- **Impact:** A stolen token stays valid for its full 7 days; the only remediation is rotating `SESSION_SECRET`. Forced-logout CSRF is a nuisance at most.
- **Suggested fix:** Shorten the lifetime, add a `jti` with a Redis denylist on logout, assert the `role` claim in middleware, and apply the login route's origin check to logout.

### [SEVERITY: Low] Over-permissive filename validation in image delete paths
- **Repo:** the-vibe-coder
- **Location:** src/lib/images.ts:334-337; interpolated unencoded at src/lib/github.ts:90
- **Category:** Path traversal
- **Description:** `isValidFilename` rejects only empty strings, backslashes, and leading dots, so `%2f`, `?`, `#`, and spaces are accepted and interpolated raw into the GitHub Contents API URL. Verified against the live API: `%2f` decodes to `/` (defeating the "exactly two segments" rule the surrounding comment claims to enforce) while `..`/`%2e%2e` segments return 404, so this **cannot** escape `public/images/`. An unescaped `?` or `#` can still alter or truncate the request URL (e.g. `foo.png?ref=other`).
- **Impact:** The stated path constraint is not actually enforced, and request URLs are influenceable by filename. Admin session required, so no privilege gain.
- **Suggested fix:** Restrict to `/^[A-Za-z0-9][A-Za-z0-9._-]*$/` and `encodeURIComponent` each path segment in `github.ts`.

### [SEVERITY: Low] `sanitizeSlug` can return an empty string, producing dotfile and double-slash paths
- **Repo:** the-vibe-coder
- **Location:** src/lib/slug.ts:12-18; callers at src/app/api/posts/route.ts:83,132,196,223 and src/app/api/images/route.ts:28
- **Category:** Logic
- **Description:** Inputs like `"..."`, `"!!!"`, or `"---"` sanitize to `""`, so callers build `content/posts/.mdx` or `public/images//`. The character class is a strict allowlist, so there is no traversal.
- **Impact:** A hidden `.mdx` file in the content repo, and image paths with a double slash that will not round-trip through `isValidImageRepoPath` on delete, leaving orphaned files that cannot be removed via the UI.
- **Suggested fix:** Return an error when the sanitized slug is empty.

### [SEVERITY: Low] Unvalidated slug reaches `path.join` in the posts loader
- **Repo:** the-vibe-coder
- **Location:** src/lib/posts.ts:105, :137; callers at src/app/posts/[slug]/page.tsx:124, .../raw/route.ts:278, .../opengraph-image.tsx:190, src/app/admin/preview/[slug]/page.tsx:142
- **Category:** Path traversal
- **Description:** `path.join(POSTS_DIR, \`${slug}.mdx\`)` receives the route param with no `sanitizeSlug` call, even though src/lib/slug.ts exists for exactly this. No read outside `content/posts` could be demonstrated (the `.mdx` suffix is forced and Next normalizes `..` in path segments), so this is a latent gap rather than an exploitable one.
- **Suggested fix:** Reject non-`[a-z0-9-]` slugs in `_getPostBySlug` and `getPostBySlugAdmin`.

### [SEVERITY: Low] Arbitrary local file read via markdown image path in the OG image route
- **Repo:** the-vibe-coder
- **Location:** src/app/posts/[slug]/opengraph-image.tsx:216-236
- **Category:** Path traversal
- **Description:** `extractFirstImage` takes the first `![alt](src)` target from the post body and passes it unvalidated to `path.join(process.cwd(), "public", rawImage)` and `fs.readFileSync`. A relative image path containing traversal segments reads an arbitrary file on the host and base64-embeds it as a `data:` URI in the generated PNG. Content is author/AI-authored and the result is not returned as text, which caps impact. (Working payload omitted as a precaution.)
- **Suggested fix:** Require `^/images/[A-Za-z0-9._/-]+$` and reject any `..` segment before reading.

### [SEVERITY: Low] `POST /api/images` accepts any file type or size and commits it to `public/`
- **Repo:** the-vibe-coder
- **Location:** src/app/api/images/route.ts:6-38
- **Category:** XSS
- **Description:** No MIME check, no extension allowlist, no size cap; `sanitizeFilename` preserves the extension, so `evil.html` is committed to `public/images//evil.html` and served same-origin under a CSP with `script-src 'unsafe-inline'`. `commitFileRaw` also upserts, silently overwriting a same-named image.
- **Impact:** Stored XSS, reachable only with an existing admin session (no privilege gain), plus unbounded blobs in the content repo.
- **Suggested fix:** Allowlist extensions using the existing `isImageFilename` in src/lib/image-types.ts and enforce a byte cap.

### [SEVERITY: Low] Slack text flows unescaped into TODO.md and then into an autonomous agent prompt
- **Repo:** the-vibe-coder
- **Location:** src/app/api/slack/todo/route.ts:136, :164, :180; consumed at src/app/api/todo/launch-agent/route.ts:22-28
- **Category:** Injection / Prompt injection
- **Description:** `item` is taken verbatim from the slash-command text with newlines and `## ` headings unstripped, so a Slack user can inject structure into `TODO.md`, which `src/lib/todo.ts` then parses. The same text is interpolated into a prompt instructing an agent to clone repos, implement, commit, and open a PR.
- **Impact:** An indirect prompt-injection path from any Slack workspace member to a billable coding agent with repo write access. The admin must click Launch, which is the mitigating control.
- **Suggested fix:** Strip newlines and leading markdown control characters before insertion, and delimit untrusted text in `buildPrompt`.

### [SEVERITY: Low] CSP permits `'unsafe-inline'` scripts and `img-src https:`
- **Repo:** the-vibe-coder
- **Location:** next.config.ts:52-64
- **Category:** XSS
- **Description:** The policy is enforcing, but `script-src 'self' 'unsafe-inline'` removes CSP as a mitigation for both XSS findings above, and `img-src 'self' data: https:` allows any host. The file's own comments (lines 27-29) flag this as unfinished work.
- **Suggested fix:** Adopt the per-request nonce described in the comments; the only inline scripts are the theme bootstrap (layout.tsx:77) and JSON-LD (JsonLd.tsx:336), both easily nonce-able.

### [SEVERITY: Low] `GITHUB_TOKEN` embedded in a git remote URL
- **Repo:** the-vibe-coder
- **Location:** scripts/fetch-content.sh:23-25
- **Category:** Secrets
- **Description:** The token is passed on the command line (visible via `/proc//cmdline`) and written into `$TMPDIR/.git/config`; git error output on a failed clone commonly echoes the remote URL into build logs. `$TMPDIR` is only cleaned on the success path, so `set -e` leaves the credentialed config on disk after any failure.
- **Suggested fix:** Use `git -c http.extraheader=...` or a credential helper, and add `trap 'rm -rf "$TMPDIR"' EXIT`.

### [SEVERITY: Low] Internal error details returned to unauthenticated clients
- **Repo:** the-vibe-coder
- **Location:** src/app/api/share-image/route.tsx:438-442; src/app/api/slack/todo/route.ts:196-201; src/app/api/todo/launch-agent/route.ts:129-135
- **Category:** Information disclosure
- **Description:** Raw `err.message`, GitHub API response bodies, and upstream Coder API bodies are returned to the caller or echoed into the Slack channel. No secret is exposed on these paths (tokens are only ever sent in headers).
- **Suggested fix:** Log details server-side and return a generic message.

### [SEVERITY: Low] Analytics counter keys are written without a TTL
- **Repo:** the-vibe-coder
- **Location:** src/app/api/analytics/track/route.ts:77-85; read fan-out at src/app/api/analytics/summary/route.ts:70-75
- **Category:** Logic
- **Description:** Per-day and per-path keys accumulate indefinitely and the `views:paths` set grows forever; the summary endpoint issues a pipeline `GET` per member on every call. The path allowlist correctly bounds key cardinality, so arbitrary key minting is not possible.
- **Impact:** Slow unbounded Redis growth and a summary cost that grows linearly with site history.
- **Suggested fix:** Set a TTL (e.g. 400 days) on dated keys and prune `views:paths`.

### [SEVERITY: Low] React key collision on duplicate TODO items
- **Repo:** the-vibe-coder
- **Location:** src/components/admin/TodoReorderList.tsx:107, :33-36; interacts with src/lib/todo.ts:108-124
- **Category:** Logic
- **Description:** `reorderUpNext` deliberately supports duplicate item text via a text-keyed multiset, but the list uses `key={item.text}`. Two identical bullets produce duplicate React keys, so reordering either one reconciles incorrectly and can send a wrong `order` array; the `dirty` check likewise reports "no changes" when two identical items are swapped.
- **Suggested fix:** Key by index or by a stable id assigned server-side.

### [SEVERITY: Low] Publish/schedule frontmatter rewrites silently no-op on unquoted values
- **Repo:** the-vibe-coder
- **Location:** src/components/admin/DraftsList.tsx:65-75, :120-123; src/app/admin/edit/[slug]/page.tsx:327-335
- **Category:** Logic
- **Description:** `published.replace(/^date:\s*'[^']*'/m, ...)` matches single-quoted dates only. src/lib/format-date.ts:12-17 and src/lib/posts.ts:15-18 both document that this content set frequently yields unquoted dates, in which case "Publish" flips `published` but silently leaves the old date, with no error surfaced.
- **Suggested fix:** Parse and serialize frontmatter with `gray-matter` instead of regex-patching, or assert the replacement changed the string.

### [SEVERITY: Low] Unpaginated, unauthenticated GitHub Discussions fetch
- **Repo:** the-vibe-coder
- **Location:** src/lib/discussions.ts:20-33
- **Category:** Logic
- **Description:** No `per_page`/pagination and no `Authorization` header. GitHub's default page size is 30, so past 30 discussions the oldest posts silently show `0` comments; unauthenticated requests also share the 60/hour/IP limit across all serverless instances, and the failure path logs and returns `{}`.
- **Suggested fix:** Paginate with `?per_page=100` plus link-header following, and send the existing `GITHUB_TOKEN`.

### [SEVERITY: Low] Unbounded GitHub API fan-out on the admin images page
- **Repo:** the-vibe-coder
- **Location:** src/lib/images.ts:220-236
- **Category:** Logic
- **Description:** `Promise.all` issues one Contents API request per image directory with no concurrency cap. A rate-limit or slow response fails the whole `/admin/images` render with the raw GitHub error text surfaced at src/app/admin/images/page.tsx:232.
- **Suggested fix:** Bound concurrency (batches of ~5) and degrade per-directory instead of failing the page.

### [SEVERITY: Low] Dockerfile build failure silently swallowed
- **Repo:** coder-templates
- **Location:** docker/build/Dockerfile:46-48
- **Category:** Logic
- **Description:** `|| true` binds to the entire `&&` chain, not just the `uvx` move. If the `uv` installer or the first `mv` fails, the layer still exits 0 and ships an image with no `uv`, contradicting docs/system-instructions.md:5, which tells the agent `uv` is present.
- **Suggested fix:** Split into a separate `RUN` and scope the tolerance to the `uvx` move alone.

### [SEVERITY: Low] llama config file is `source`d rather than parsed
- **Repo:** coder-templates
- **Location:** scripts/llama-generate-start.sh:16; written by scripts/llm-switch.sh:52-57
- **Category:** Injection
- **Description:** `/etc/llama-generate.conf` is executed as shell. Nothing in the repo sets or asserts its mode; `llm-switch.sh` recreates it via `sudo bash -c`, so permissions depend on root's umask at that moment. The `llm-switch.sh` write itself is safe, since `${1}` is allow-listed by the `case` at lines 41-48.
- **Impact:** Any write access to the config becomes code execution in the systemd service context.
- **Suggested fix:** Parse the value (`sed -n 's/^DEFAULT_MODEL=//p'`) and re-validate against the allow-list.

### [SEVERITY: Low] systemd units have no sandboxing and execute a user-writable script
- **Repo:** coder-templates
- **Location:** scripts/llama-generate.service:8; scripts/llama-embed.service
- **Category:** Infra
- **Description:** Neither unit sets `NoNewPrivileges`, `ProtectSystem`, `ProtectHome`, `PrivateTmp`, or `RestrictAddressFamilies`. `llama-generate.service` executes a script from the service account's own home directory, so anything running as that user changes what the service runs on next restart. No privilege boundary is crossed, but integrity guarantees are absent.
- **Suggested fix:** Move the launcher to a root-owned `/usr/local/libexec` path and add the standard hardening directives.

### [SEVERITY: Low] Orchestrator install directory never ACL-hardened
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/orchestrator/update-orchestrator-system.ps1:25-26; .../update-orchestrator-user.ps1:18-19; .../register-update-orchestrator-tasks.ps1:14, :27
- **Category:** Infra
- **Description:** `C:\ProgramData\update-orchestrator` is created with `-Force` and no `icacls`. The SYSTEM task executes `-File` from this directory and the system script dot-invokes a sibling via `$PSScriptRoot`. Inherited `ProgramData` permissions let non-admins create subdirectories there; nothing is currently directly writable by a standard user, so this is defense in depth.
- **Suggested fix:** `icacls  /inheritance:r /grant 'Administrators:(OI)(CI)F' 'SYSTEM:(OI)(CI)F'`, or install under `%ProgramFiles%`.

### [SEVERITY: Low] Toast button passes unvalidated JSON data to a protocol handler
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/orchestrator/update-orchestrator-notify.ps1:44-45 (data parsed at :28)
- **Category:** Injection
- **Description:** `New-BTButton -Content 'Open Log' -Arguments $summary.LogPath -ActivationType Protocol` takes the path verbatim from `last-run-summary.json` and hands it to the shell URI dispatcher, so a tampered `LogPath` (UNC path, `ms-*:` or `file://` URI) launches on click.
- **Suggested fix:** Validate that `LogPath` resolves under `C:\ProgramData\update-orchestrator` and ends in `.log`.

### [SEVERITY: Low] SSH key authorization is not idempotent and ACL results are unchecked
- **Repo:** coder-templates
- **Location:** windows-aint-no-problem/setup/setup-openssh-server.ps1:53, :56-57, :59
- **Category:** Logic
- **Description:** `Add-Content ... -Force` appends unconditionally, duplicating the key on re-run. The two `icacls` calls pipe to `Out-Null` without checking `$LASTEXITCODE` and neither sets the file *owner*, which sshd also validates for `administrators_authorized_keys`. There is no `$ErrorActionPreference = 'Stop'`, so earlier failures do not prevent the script printing "Done".
- **Suggested fix:** Guard the append with `Select-String`, set `$ErrorActionPreference = 'Stop'`, check `$LASTEXITCODE` after each `icacls`, and add `/setowner Administrators`.

### [SEVERITY: Low] Proposed CI workflow in the README violates least privilege and pins nothing
- **Repo:** coder-templates
- **Location:** README.md:63-84
- **Category:** Infra / CI
- **Description:** The suggested `push-template.yml` has no top-level `permissions:` block, uses the mutable `actions/checkout@v4` tag rather than a commit SHA, and installs the Coder CLI via unpinned `curl -fsSL https://coder.com/install.sh | sh`, all with `CODER_SESSION_TOKEN` in the environment.
- **Suggested fix:** Add `permissions: {}` at the top with per-job grants, pin the action by SHA, and pin the CLI version.

### [SEVERITY: Low] Documented backup copies a secrets file into `$HOME` with no permission handling
- **Repo:** coder-templates
- **Location:** docs/sff-migration-checklist.md:35
- **Category:** Secrets
- **Description:** `sudo cp /etc/coder.d/coder.env ~/coder-backup/coder.env.bak` copies the Postgres DSN and any OIDC/OAuth client secrets without preserving a restrictive mode; the target directory is never mode-restricted and there is no cleanup step.
- **Suggested fix:** `sudo install -m 600 -o "$USER" ...` and add a cleanup checkbox.

### [SEVERITY: Low] Shared mutable image tag across all workspaces
- **Repo:** coder-templates
- **Location:** docker/main.tf:291-299
- **Category:** Infra
- **Description:** Every workspace build targets `coder-workspace:latest` with `triggers = { dockerfile_hash = ... }`. Two concurrent builds race on the tag, and a rebuild silently re-points `:latest` while other workspaces reference it.
- **Suggested fix:** Tag with the Dockerfile hash, e.g. `coder-workspace:${filemd5("./build/Dockerfile")}`.

### [SEVERITY: Low] `.gitignore` omits common secret patterns
- **Repo:** coder-templates
- **Location:** .gitignore:1-19
- **Category:** Secrets
- **Description:** Terraform state and `*.tfvars` are covered, but a repo described in README.md:1-8 as the home for personal infra and homelab config does not ignore `.env`, `*.pem`, `id_ed25519`, `*.key`, `*.log`, or `github-token.txt`. Nothing is currently leaked.
- **Suggested fix:** Add those patterns.

## Assumptions / Caveats

- Audited exactly the checked-out commits: the-vibe-coder at `99c08f6`, coder-templates at `49718b4`. No branch switch, pull, or fetch was performed, and no file in either repo was modified.
- `node_modules/`, `.next/`, and `scripts/benchmarks/round5/results/` were excluded from line-by-line review. The results directory was scanned for secret patterns (clean); it contains model output and metrics only.
- `scripts/benchmarks/round5/fixtures/**` are deliberately-broken benchmark inputs, verified from explicit `// BUG:` markers in the fixtures and from the prompts in benchmark.py:506-548. Their planted defects (hardcoded `admin123`, unsigned base64 "tokens", `debug=True`) are intentional and are **not** reported as findings.
- Dependency findings come from `npm audit --package-lock-only` against the committed lockfile. Advisory ranges shift over time; re-run before triage. No runtime install or upgrade was performed.
- The percent-encoded traversal theory for `isValidImageRepoPath` was tested against the live GitHub Contents API and **disproved** (`..`/`%2e%2e` segments 404), which is why that finding is rated Low rather than High. The residual `?`/`#` URL-manipulation issue stands.
- Findings in the two vibe-coder XSS entries assume `content/TODO.md` and post MDX can be influenced by a non-admin (Slack workspace members and AI-generated content respectively). Both are admin-rendered, so a purely single-trusted-author threat model would downgrade them.
- The Windows orchestrator scripts were reviewed statically only; no Windows host was available to confirm effective ACLs, scheduled-task registration behavior, or `sshd_config` defaults in situ.
- `.env*` is gitignored in both repos and a full-tree scan for `ghp_`/`github_pat_`/`sk-`/`xox*`/`AKIA`/PEM headers returned nothing. All base64 blobs in docker/main.tf were decoded and contain no inline credentials.
- No SQL, template engine, `eval`, or `child_process` usage exists in the-vibe-coder, so SQL/command/template injection and unsafe deserialization are not applicable there. Every outbound `fetch` targets a hardcoded host, so no SSRF sink was found.

Fable 5's DNF writeup

# Fable 5 — DNF (Anthropic content-policy block, "cyber")

Status: **did not finish**, reproduced twice, retries stopped by decision on
2026-08-08. This is a documented outcome for this round, not a missing data
point to chase further — do not keep re-running Fable 5 against this prompt.

## Assignment (now unblinded out of necessity)

Fable 5 was Variant B in this round, assigned `report-5b855a.md`. No report
file exists for this token; the session never reached the point of writing
one. The other two variants have since been fully revealed: Variant A
(`report-5f220e.md`) was Sonnet 5, Variant C (`report-b74a69.md`) was
Opus 5 — see `comparison.md` for the full triage.

## What happened (both attempts)

Both attempts self-organized the same way: the session split the audit into
three parallel subagents (roughly: app auth/API surface, app
client/scripts/deps, `coder-templates`). In both runs, the subagent covering
**auth, middleware, rate-limiting, and MCP-auth** got its response blocked
outright by Anthropic's platform-level content classifier under the "cyber"
category, right as it was moving from reading code to writing up findings.
The other two subagents (client/deps/scripts, `coder-templates`) were not
reported as blocked in either transcript, but no full report was ever
assembled since the run didn't complete.

- **Attempt 1:** blocked on the "server code" subagent after it had read
  a couple of files into the auth/token-handling area.
- **Attempt 2:** blocked on the "auth/API surface" subagent, after it had
  already read the middleware matcher, MCP auth, the Slack route, the
  share-image route, and the rate limiter — i.e. it got further into the
  same subject area before tripping the same block.

Two-for-two on the same subject area (auth/middleware/rate-limit/MCP-auth)
is a reproducible collision, not noise. Working theory: describing a
concrete weakness in that code (e.g. a middleware matcher gap or a
rate-limit bypass) with enough specificity to be a useful audit finding
reads to Anthropic's classifier as attack guidance, independent of the
benign, defensive framing in our prompt.

## Decision

- Recorded as a DNF for Fable 5 in this round. No third identical attempt.
- Not retried with a softened prompt in this round, to avoid conflating a
  "mitigated variant" result with the blind three-way comparison. If a
  future round wants to test whether softer finding-detail requirements
  (e.g. "name the weakness class and location, don't narrate a
  step-by-step bypass") let Fable 5 complete, that should be run and
  labeled as an explicit separate variant, not folded into this
  comparison.
- Comparison writeup will treat this as its own failure category:
  "did not finish due to host-platform safety block," distinct from a
  capability-based DNF (e.g. the Nemotron Round 9 case).

My full blind triage and comparison notes

# Blind Triage & Comparison — Revealed: Sonnet 5 vs Opus 5

Triage performed against the actual pinned commits
(`the-vibe-coder@99c08f6`, `coder-templates@49718b4`), spot-verifying a
representative sample of findings from each report directly in the code
(not just trusting the report text). Triage itself was done blind
(reports known only as "Variant A" / "Variant C"); mapping revealed
afterward:

- **Variant A** (`report-5f220e.md`) = **Sonnet 5**
- **Variant C** (`report-b74a69.md`) = **Opus 5**
- **Variant B** (`report-5b855a.md`) = **Fable 5** — DNF, see `fable-5-dnf.md`

## Headline counts

| | Sonnet 5 (`5f220e`) | Opus 5 (`b74a69`) |
|---|---|---|
| Total findings | 11 | 50 |
| High | 1 | 6 |
| Medium | 3 | 17 |
| Low | 7 | 27 |
| `the-vibe-coder` findings | 5 | 30 |
| `coder-templates` findings | 6 | 20 |

Opus 5 found roughly 4.5x as many issues as Sonnet 5, across both repos.

## Spot-verification (sample, not exhaustive)

Checked ~15 claims from both reports directly against the pinned-commit
source. Everything checked from **both** reports was an accurate
description of the code — no fabricated findings, no misquoted logic, in
the sample checked. The gap between the two reports is coverage and depth,
not accuracy.

Notable quality signals for **Opus 5**:
- Tested its own path-traversal theory against the *live* GitHub Contents
  API, found it didn't hold (`%2e%2e` 404s), and correctly downgraded that
  finding from a plausible High to a Low rather than reporting the
  unverified worst case.
- Caught a real logic bug beyond the security angle: the giscus workflow's
  own comment claims it skips the repo owner's comments, but the `if:`
  condition only checks the discussion category, not the author. Variant A
  flagged the same workflow only for its missing `permissions:` block and
  missed this.
- Explicitly identified and excluded the deliberately-planted bugs in
  `scripts/benchmarks/round5/fixtures/*` as intentional test data rather
  than reporting them as findings, showing it understood the difference
  between benchmark fixtures and production code.
- Precise line citations throughout (verified `rate-limit.ts:83-92`,
  `:44-47`, `:70-73` character-for-character against the actual function
  boundaries).

Sonnet 5's one clearly unique catch neither report shares: `coder-templates`
documents Docker as an available capability in `system-instructions.md`,
but the Dockerfile never installs it and `main.tf` never mounts a socket —
a real, low-severity documentation/reality mismatch. Worth keeping in the
fix list regardless of which model found it.

## The single most important finding in either report

Opus 5's **login rate-limiter bypass via spoofed `X-Forwarded-For`**
(High): `clientIp()` in `rate-limit.ts` takes the left-most, client-supplied
XFF entry, so every rate-limit bucket (login, analytics, share-image, MCP)
is attacker-partitionable, giving unlimited brute force against
`ADMIN_PASSWORD`. Confirmed by direct code read — this is real, exactly as
described, and it is the actual sole credential gating repo write, Dev.to
publishing, and agent-launch access. **Sonnet 5 did not find this at
all.** Compounding it, Opus 5 also caught that the same limiter fails
open on any Redis error or missing config, independently removing the
control a second way.

## Overlap (found by both, same underlying issue)

- Outdated `next` / MCP dependency chain CVEs (both High; Opus 5 is more
  precise about the actual resolved version, 16.2.10, and ties it to a
  concrete middleware-bypass advisory that matters because middleware is
  this app's *only* authorization layer — Sonnet 5 stops at "here are
  CVEs").
- MCP bearer-token timing leak via early-length-return in `mcp-auth.ts`
  (both Low, identical characterization).
- Static `GITHUB_TOKEN`/`GH_TOKEN` in the agent's Terraform `env` block
  defeating the credential-helper refresh design (both Medium, same lines).
- Unauthenticated llama.cpp servers bound to `0.0.0.0` (Sonnet 5: Medium,
  Opus 5: High — Opus 5's higher rating accounts for the
  Tailscale/Cloudflare tunnel documented elsewhere in the repo, meaning
  "LAN-only" isn't a safe assumption).
- Missing `permissions:` block on the giscus GitHub Actions workflow
  (Sonnet 5: Low, Opus 5: Medium).
- Slack backlog text flowing unsanitized into the launch-agent prompt
  (Sonnet 5: Medium, Opus 5: Low — opposite direction from the giscus
  severity gap; Opus 5's lower rating explicitly credits the admin's
  manual "Launch" click as a real gate, Sonnet 5 does not weigh that
  mitigation).
- Unvalidated slug reaching a filesystem path join in `posts.ts` (both
  Low; Opus 5 additionally covers two related-but-distinct slug bugs — an
  empty-sanitized-slug case and an OG-image path-traversal read — that
  Sonnet 5 didn't find).

## Unique to Opus 5 (not in Sonnet 5)

The bulk of the gap: the rate-limiter bypass and fail-open (above),
unpinned `curl | bash` root installs in the Dockerfile (High, supply
chain), unsandboxed benchmark code execution on the host (High), stored
XSS in the admin TODO renderer and in the MDX anchor component (Medium
x2), fail-open orphan-image deletion, silent post overwrite, GitHub
read-modify-write races, unvalidated settings persistence, a crash on
unquoted YAML dates, a permanently-bricked rate-limit key on a failed
Redis `EXPIRE`, world-readable MCP secret files before `chmod`, an
unpinned externally-cloned skills repo trusted every startup, and a full
pass over the Windows orchestrator scripts (open SSH to all networks,
unpinned PowerShell Gallery module run as SYSTEM, unattended reboot on
every boot, plaintext GitHub PAT) plus roughly a dozen more Low-severity
items (session revocation, CSP gaps, unrestricted image upload, pagination
bugs, etc).

## Unique to Sonnet 5 (not in Opus 5)

- Documented-but-absent Docker capability (above).
- Slightly more detailed narrative of the MCP endpoint's transitive
  dependency chain (`mcp-handler` → `@hono/node-server` → `hono` /
  `body-parser` / `fast-uri`), though Opus 5's dependency finding lists
  most of the same packages without walking the chain by name.

## Assessment

Both reports are accurate where they make claims — I found no false
positives in the sample verified. The real difference is thoroughness and
depth of investigation: Opus 5 read further into both repos, tested a
hypothesis empirically instead of asserting it, distinguished intentional
benchmark fixtures from real bugs, and found the one finding in this round
that actually matters most (the rate-limiter bypass). Sonnet 5 produced a
shorter, still-legitimate but comparatively shallow pass.

## Fable 5

Did not finish either attempt; see `fable-5-dnf.md`. No comparison data —
Fable 5 is a DNF for this round, not a ranked third place.

## Verdict

For this round, on this task: **Opus 5 > Sonnet 5 > Fable 5 (DNF)**, driven
mostly by Opus 5 catching the one finding that actually matters (the
auth rate-limiter bypass) and covering roughly 4.5x the ground. Sample
verification found no accuracy gap between the two, only a depth/coverage
gap. Candidate for the `model-showdown-round-*` series, with the Fable 5
platform-safety DNF as a distinct, separately-worth-mentioning angle.

---

A defensive security audit and an offensive one can produce the same words. Should the safety layer even be able to tell them apart? I don't have a clean answer. But I now have a support ticket's worth of evidence that it can't, not yet.

By the Numbers

• 3 models audited the identical pinned commit, in fully isolated workspaces, with zero cross-awareness
• 2 attempts by Fable 5, both blocked at the same logical point: auth/middleware/rate-limit code
• 11 findings from Sonnet 5 vs 50 from Opus 5, a 4.5x gap in depth, not accuracy
• 1 High-severity finding from Sonnet 5 vs 6 from Opus 5
• 0 false positives found across everything I spot-verified from either completed report
• 1 rate-limiter bypass that only one of the two models caught, and the one finding that actually mattered most
• 30 / 20 — Opus 5's findings split between the blog engine and the infra repo