DEV Community

Cover image for Dev Log: 29 August 2026 — Holes in the Ops Surface, and Docs That Lie
Nasrul Hazim
Nasrul Hazim

Posted on

Dev Log: 29 August 2026 — Holes in the Ops Surface, and Docs That Lie

Twenty-five commits across eight repos today. The thread running through most of them: finishing things that were 90% automated and still needed a human in the middle.


The one-step-in-the-middle problem

I've been building a deployment platform with an MCP surface — an agent can provision a node, deploy an app, diagnose it, redeploy it. Today I deployed the first real tenant through it end to end and it went green, answered on its own domain with TLS, and returned 500 on every request.

Migrations had run. The seeders hadn't. And nothing in the product could run them.

Then the second hole, right next to it: the app was missing thirteen environment variables its seeders read, and no tool on the ops surface could set one. So a pipeline that could do everything still needed somebody in a web form for the one step in the middle. That's not a boundary. That's a hole.

Two tools closed it, and the interesting part of both is what they refuse to do.

Setting env vars: values go inbound only. Every read tool on that surface reports env keys and never values, and that had to stay true. So the write tool's response names each key and what happened to it — added, changed, unchanged, removed, generated — and never echoes a value back, not even one the caller just supplied. An agent that can read a password out of a deployment is a categorically worse tool than one that can set it. The audit trail records key names for the same reason: recording values would rebuild the plaintext credential store the encrypted column exists to avoid.

There's also a generate mode, so the values nobody ever needs to read never appear in a transcript or a shell history at all. APP_KEY is the case that matters — strong, secret, stable, never typed by a human. Deliberately not offered for anything that has to match something else, like a mail password, where a generated value is just silently wrong.

And it merges, never replaces. That rule is scar tissue: a wholesale restore once discarded the APP_KEY the runtime had minted, so every release got a fresh one — signing out every user and making every encrypted column unreadable. A write that replaced the array would be the same defect from the other side.

Running first-install commands: declared, not dictated. Some work can't live in the deploy path. A preStartCommand runs on every deploy, so it can only hold work that's safe to repeat. First-install work is the opposite shape — a seeder that creates a superadmin with User::create() is a duplicate-key error on its second run.

The tool decides when first-install work runs and never what it is. The commands live on the workload definition, and a small validator refuses anything that isn't php artisan … free of shell syntax:

final class SetupCommands
{
    public static function assertAllowed(string $command): void
    {
        // Allow-list of SHAPE, not a deny-list of metacharacters.
        if (! preg_match('/^php artisan [a-z0-9:_\-\. ]+$/', $command)) {
            throw new InvalidArgumentException(
                'Setup commands must be a plain `php artisan` command.'
            );
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

An allow-list of shape, not a deny-list of dangerous characters. A deny-list is a list somebody has to keep complete forever, and the first omission is remote code execution as the application's own account. An MCP tool that accepted a command string outright would be RCE on a customer's node for anyone holding an ops token.

The rest is unglamorous and load-bearing: run as the application's own account via runuser and never as root (a root-created file in the release tree breaks the app permanently), load the unit's environment so a seeder reading SUPERADMIN_EMAIL sees what the service sees, stop at the first failure because "seed, then provision" has an order, and refuse a second run unless forced — recorded in a new setup_completed_at column.

Both tools got Pest coverage for the input systemd fails silently on: an unparseable key is dropped outright, and a newline in a value makes the remainder read as further variables. Silent input mangling is exactly what you want a test for, because nothing else will tell you.

Trusting a rebuilt host's key

Third one from the same session. You rebuild a machine, the OS regenerates its SSH host key, and every subsequent connection fails the way it should — because from the client's side, "the host key changed" and "someone is on the wire" are the same event.

The thing you must not build is a tool that shrugs and accepts whatever key answers. So the shape is: a probe that fetches and fingerprints the key that's currently answering, a result object that carries it, and a distinct exception for mismatch versus probe failed. The operator sees the new fingerprint and confirms the re-trust. The system never decides on its own that a changed key is fine.

HostKeyMismatch and HostKeyProbeFailed being separate types matters more than it looks. One means "the machine you rebuilt is answering with a new identity, as expected". The other means "I could not reach it at all". Collapse those into one exception and the UI has to guess, and it will guess wrong on the day it counts.

CSP, twice, in opposite directions

The day's other theme got its own post, because one story wasn't enough: a Laravel app that enabled CSP by accident through a null default that means "auto-enable in production", and a static site whose correctly-strict script-src 'none' broke every mailto: link on it because Cloudflare's email obfuscation needs a script to undo itself.

Both failures were silent. Both happened at a seam where two systems each behaved correctly on their own. Full write-up in the companion post.

Shipping a desktop build

A Flutter app I've been building got a Windows platform target, a release workflow, and a public releases repository — the app repo stays private, the built artifacts and their download page go somewhere public.

That split is worth stealing. Your CI needs somewhere to put binaries that end users can reach, and "make the whole repo public" is not the only answer. A separate public repo holding nothing but tagged releases gives you a URL you can link from a marketing site, without exposing source or issue history. It got an ADR, because six months from now the reason will be less obvious than the arrangement.

The marketing site now renders its download buttons from that repo's latest release — fetched at build time only. Two reasons, both worth stating plainly: the site's CSP is connect-src 'none', so a client-side fetch would be blocked anyway; and GitHub's unauthenticated rate limit of 60 requests per hour per IP does not survive real traffic. Every failure path returns null and the section simply doesn't render — a 404 is the normal state before the first release exists, and a build shouldn't fail over the expected case.

Google Sign-In, behind a contract

The API gained a native Google Sign-In endpoint plus SSO buttons on the dashboard auth screens. Structure:

interface GoogleIdTokenVerifier
{
    public function verify(string $idToken): GoogleIdentity;
}
Enter fullscreen mode Exit fullscreen mode

One implementation verifies against Google's JWKS. The action depends on the contract, so tests bind a fake and never touch the network — and if the verification strategy ever changes, the action doesn't. This is the default shape for anything that talks to a third party: a contract, a DTO for the verified result, and a driver behind it.

Docs that were quietly lying

Five commits on a documentation repo, and most of them were corrections, not additions:

  • A claim that dependency-allowlist enforcement was in place. It wasn't.
  • An endpoint documented as "planned" that had shipped.
  • A behavioural rule — guest play capped at a lower engine level, an account unlocks the rest — that existed in code and nowhere in the spec.

Plus two new ADRs, one of which forced a partial SRS rewrite. That's the correct outcome, by the way: if an architecture decision doesn't ripple into the requirements doc, one of the two documents isn't being read.

Documentation drift is the same category as the silent CSP failure. Nothing errors. The build is green. The document is just wrong, and it stays wrong until somebody reads it closely enough to notice — usually while relying on it.

Also today

  • Legal pages, with every placeholder replaced by an actual commitment, plus a Bahasa Malaysia privacy notice alongside the English one. Not an optional translation — PDPA s.7(3) requires the notice in both languages, so the bilingual pair is the deliverable, with lang and alternate props wiring up the hreflang link.
  • A UI bug worth naming: a failed in-app encounter was rendering the "you've collected everything" state instead of an error. A failure path falling through to the success path is a bug I keep meeting in different costumes. Test it as "the request failed" rather than "the request returned nothing", and it stops happening.
  • A mail history dashboard and a sidebar regression from the baseline patch — collapsible menu groups had stopped collapsing, and a logo had gone missing from two sidebars. The unglamorous half of any framework upgrade.

Takeaway

Today's pattern, said once: the last 10% of an automation is where all the security decisions live. Every tool I added had a version that would have been faster to write and wrong — echo the value back, accept any command string, trust whatever host key answers. The design work wasn't making the tools do things. It was deciding, precisely, what they must refuse to do.

Top comments (0)