DEV Community: Nasrul Hazim

Dev Log: 2026-08-11 — a tenant for every account, four frontend traps that never error, and the fourth comparison site

Nasrul Hazim — Wed, 12 Aug 2026 23:36:00 +0000

Thirty commits across three repos, and the day split cleanly into two halves: standing a control plane up on a bare host, and fixing the things that were quietly wrong once it was actually reachable.

The deploy half has its own write-up — a deploy script that had been reporting success for weeks while skipping its own git pull. This is everything else.

1. An account that can sign in and see nothing

Multi-tenant app. A user belongs to organisations; every query is scoped to the current one. Registration created the user row and stopped there — no role, no organisation.

Which produces the worst possible failure mode: the account works. Login succeeds, the session is valid, the dashboard renders. It's just empty, because every scoped query returns zero rows and every permission check runs against a user holding no role at all.

That doesn't read as "your account is awaiting access." It reads as a broken product.

Self-serve registration should hand you something you own, so it now does:

public function create(array $input): User
{
    Validator::make($input, [
        // ...
        'password' => $this->passwordRules(),
    ])->validate();

    return DB::transaction(function () use ($input): User {
        $user = User::create([
            'name' => $input['name'],
            'email' => $input['email'],
            'password' => $input['password'],
        ]);

        $user->assignRole('user');

        $organization = Organization::create([
            'name' => $name = __(":name's Organisation", ['name' => $user->name]),
            'slug' => $this->uniqueSlug($name),
            'owner_id' => $user->id,
            'plan' => OrganizationPlan::Starter,
        ]);

        $user->setDefaultOrganizationId($organization->id);

        return $user;
    });
}

The transaction isn't decoration. User with a role but no organisation is exactly the state this change exists to eliminate — so a failure halfway through must not be able to create one.

Two details worth pulling out:

The slug suffixes rather than throws. organizations.slug is unique, and two people called Zul is not an error condition, it's Tuesday. Anything derived from user-supplied names needs a collision strategy chosen deliberately, and "500 on the second signup" is rarely it.

Setting the default matters as much as creating the org. Without it the organisation switcher opens on nothing, and the console is empty for a user who does have somewhere to be. Same visible symptom, entirely different cause. Anywhere you have a "current X" concept, creating an X and selecting one are two steps, and forgetting the second looks exactly like forgetting the first.

Then the same rule, from the other direction

Registration having a tenant while the admin-created path didn't means which door an account came through decides whether it works. So both paths now go through one invokable action, and a deploy operation backfills the accounts made before either did.

Pulling it into a shared action surfaced a good trap:

private function makeDefaultFor(User $user, Organization $organization): void
{
    // The tenancy helper writes the *session's* current organisation as well
    // as the column. Called while an admin creates somebody else's account, it
    // would move the admin into the new user's tenant — silently, mid-request,
    // with nothing on screen to explain it.
    if (auth()->check() && auth()->id() !== $user->id) {
        $user->forceFill(['organization_id' => $organization->id])->save();

        return;
    }

    $user->setDefaultOrganizationId($organization->id);
}

The helper is named for the common case — you, setting your own default — and it does the extra session write because in that case you want it. Reuse it for another user and the extra write becomes a bug that manifests as "the admin panel randomly teleported me into a customer's tenant."

Pinned with a test, because this is the kind of thing that reads as correct forever:

it('leaves the acting admin in their own organisation', function () {
    $admin = User::factory()->withOrganization()->create();
    $before = $admin->fresh()->organization_id;

    actingAs($admin);

    app(CreatePersonalOrganizationAction::class)
        ->execute($subject = User::factory()->create());

    expect($admin->fresh()->organization_id)->toBe($before)
        ->and($subject->fresh()->organization_id)->not->toBe($before);
});

Any helper that touches both persistent state and request state needs a hard look before it's called on behalf of somebody else. The session write is invisible at the call site, which is precisely why it survives review.

2. Four frontend traps, none of which produce an error

A UI panel — "this provider is restricted to N components" — that counted checkboxes and reported the wrong number forever. Unpicking it turned up four separate things that all fail silently. Every one went into the project's gotchas file the same afternoon.

A component library's checkbox is often not an <input>. The one I'm using renders a custom element with role="checkbox". Any JavaScript querying input[type=checkbox] matches zero elements and reports a confident, wrong count. :checked doesn't apply either, though the element does expose a .checked property. Match both shapes, read .checked — and pin the marker attribute in a test, so a library upgrade fails CI instead of silently returning the count to zero.

A double quote anywhere inside an x-data attribute ends the attribute. Blade doesn't escape what you write in there. So a comment containing role="checkbox" truncates the expression mid-way, leaves a stray checkbox" attribute on the element, and un-Alpines the entire subtree — with no console error. Just a panel that never reacts. Single quotes for JS strings inside Alpine attributes, Blade comments outside the tag, and /* */ over // in multi-line attributes.

x-cloak does nothing without a CSS rule, and I never wrote one. It's only an attribute Alpine removes on init; all the hiding lives in:

[x-cloak] { display: none !important; }

Without it, every x-show="false" element renders visible on first paint and vanishes a moment later. The 2FA challenge, a couple of tab groups, a log disclosure — all flashing their hidden state for months. Nothing errors, so nothing catches it.

And the one that ties them together: a Livewire assertion proves the server rendered a string. It proves nothing about what the browser does with it. assertSee passes just as happily for markup whose Alpine expression is broken, whose selector matches nothing, or whose x-cloak is inert. All three shipped green suites.

So: anything whose behaviour lives in Alpine or in a component library's custom element needs a real browser once, and then the markup contract it depends on pinned in a test. The browser pass finds it; the pinned test keeps it found.

Related, from the same panel: wire:model is deferred, so any badge or counter derived from it is a round trip behind what the operator can see. Compute it client-side, and seed the initial value from the server so first paint is right.

3. Saying what the thing actually is

Two small changes with the same spirit.

An enum for infrastructure provider types used to render as its own case names. Now each type carries its real noun — a server, a cluster, a hypervisor — through label(), so the form says "add a server" instead of "add a provider (type: ssh)". This is the whole argument for enums with label()/color() rather than bare strings: the vocabulary lives in one place, and the UI stops leaking your internal taxonomy at the user.

And a capabilities checklist that claimed more than it delivered — showing a full list of supported components regardless of what the selected target could actually do, and showing the same list for a target where the honest answer was "nothing yet." A checklist whose job is to tell you what's supported, quietly not doing that, is worse than no checklist: it converts an unknown into a wrong belief.

Both of these are the same instinct as the deploy post. Anything that reports state should be reporting state it verified.

4. The fourth comparison site

One fix on a different project, and it's a good cautionary tale about "we already fixed that."

A legacy backend stores some records with inconsistent casing, and compares strings case-sensitively. The application lowercases input before querying. So a record stored in uppercase is simply invisible — the lookup finds nothing and the flow stops with a generic "unable to verify" message.

That was found and fixed weeks ago. Three comparison sites were converted to compare LOWER() on both sides. There were four.

The fourth sat on a path reached only from the very last step of the flow. So the symptom didn't disappear — it moved. The earlier steps now worked, the user got further, and it failed at submit instead of at the start. Which reads like a new, unrelated bug, and got triaged as one.

Two things I'm taking from it:

When you fix a class of defect, enumerate the class. Not "search for the failing call and fix it" — grep for every comparison against that column, list them, fix them together, then write down how many there were. Three of four is worse than zero of four, because it buys you a bug report that looks new.

A symptom moving later in a flow is evidence of a partial fix, not of a different bug. That's a genuinely useful triage heuristic and I don't apply it often enough.

5. Housekeeping

Rounding out the day: a reverse-proxy generator that now answers TLS for hostnames no vhost claims — previously the catch-all served its 404 as a binary download, which is a memorable way to discover that nginx guesses content type from the file extension and .html was missing. Native Redis provisioning, one instance per deployment, with per-OS profiles behind a contract so Debian and RHEL differences stay in one place each. A small swap file added during node bootstrap, because a two-gig box will OOM during composer install and blame something unrelated. And on the public marketing site, a couple of chores: redirects retired, sign-in pointed at the deployed console.

The through-line

Reading it back, today was almost entirely about systems that report something they haven't checked: a deploy that never pulled, a checklist claiming coverage it lacks, a badge counting elements it can't see, a fix that covered three of four sites, an enum showing its internal name.

None of them errored. That's the property they share, and it's the reason they all lasted weeks. Loud failures get fixed on the day they appear. The expensive ones are the ones that look fine.

Dev Log: 2026-08-12 — a 96s suite that became 42s, a capability that wasn't a scope, and four steps to a passkey

Nasrul Hazim — Wed, 12 Aug 2026 23:35:24 +0000

Fifteen commits, three repos, and the bulk of it in one: a control plane that got an MCP surface, a much faster test suite, and a handful of things that turned out to be quietly wrong once I looked properly.

The MCP work has its own write-up — sixty tools, four servers, and a long argument with myself about what a refusal should say. This is everything else.

1. Ninety-six seconds to forty-one

The suite was six minutes on a full run and about 96s in parallel, which is exactly the range where you stop running it before you push. Three separate problems, and each one hid the next.

A seeder in beforeEach. The access-control seeder ran before every feature test: 645 queries, ~110ms a pop, roughly half the suite's wall clock. It's now protected $seeder on the base TestCase, so RefreshDatabase seeds it once per process during migrate:fresh. Every test still runs inside a transaction that rolls back, so the visible state is identical — the per-test call was buying nothing at all. Nine test files were also re-seeding it on top of the global hook, which is the sort of thing that accretes when nobody's looking at the total.

The general lesson: per-test setup that RefreshDatabase already preserves is pure tax. Worth auditing your Pest.php for anything that could move to $seeder.

Xdebug costing 3x on every run. conf.d sets xdebug.mode = coverage and nothing overrode it. Pest's own Xdebug handling only drops it when the impact-analysis run is replaying a valid graph — a plain run was never covered. Every composer test script now pins the mode via @putenv, using the array form so @php and argument forwarding still work.

And test impact analysis that had never once finished. --tia records a dependency graph, then replays it and re-runs only what your change touched. Great idea. It was dying at Composer's default 300s process timeout partway through recording, leaving worker-edge files and no graph — an unusable artifact, so the next run re-recorded from scratch and hit the same wall. Forever. Under Xdebug a cold record is about ninety minutes.

Composer\Config::disableProcessTimeout plus pcov instead of Xdebug took the cold record to ~75s and a replay to ~5s.

Two things fell out of that which cost more time than the fix:

pcov had to be built from source (no published build for this PHP version), and a static build silently produces a .so with no get_module symbol. PHP reports that as "Invalid library (maybe not a PHP library)" — indistinguishable from a version mismatch, so you go and debug the wrong thing. The build script now configures it shared, asserts the symbol exists before installing, and verifies the extension loads.
PHP_INI_SCAN_DIR, not php -d. Paratest spawns workers without the parent's -d flags, so only an environment variable reaches them.

Final: parallel run 96.2s → 41.1s, impact-analysis replay ~5s, ~15s after an edit. 1451 passing, 1 skipped — unchanged, which is the number that makes the rest of it trustworthy.

And --tia stays out of CI deliberately. The point of CI is a full suite against a clean checkout; the point of TIA is your laptop between commits.

2. A capability permission is not a scope

One page in the app had no tenant isolation at all, sitting behind the same can:viewAny,SomeModel middleware as the correctly-scoped page next to it. The middleware was doing its job — it just isn't the job people assume it is.

can:viewAny says the user may do this kind of thing. It never says they may do it to this row. In an app with no global scopes (this one has none, by design), nothing downstream catches the difference. Every list, every bulk action, every delete, and one is_default reset all ran unscoped, and the reset in particular reached beyond the caller's own tenant.

Fixes, in order of how much I'd repeat them:

Every lookup goes through one scoped finder. Ownership enforced at five call sites is ownership forgotten at the sixth. This keeps coming up and I keep re-learning it.

Don't give a child table its own tenant column when the parent already knows. The job rows here already carry a provider id, and a provider already knows its organisation. Two copies of one fact are two chances for them to disagree.

A nullable-owner + is_system pair for shared catalogue rows — same shape as elsewhere in the app, so the visibility rule reads the same everywhere. Which immediately surfaced a factory bug: the old default produced rows matching neither branch of visibleTo(), so tests were "creating" records the UI could never list. When a model gains a two-branch visibility rule, the factory default has to land inside one of the branches.

Two migration gotchas, both found the hard way:

Schema::hasIndex() returned false mid-migration for an index that plainly existed, silently skipping the dropUnique() it guarded. The migration reported DONE with the old constraint still in place. Index presence now gets read from getIndexes() instead.
down() hits errno 1553 where up() doesn't, because the new composite unique leading with organization_id becomes the only index backing that foreign key. Drop the FK first.

Round-tripped migrate → rollback → migrate on MySQL, because SQLite rebuilds the table on ALTER and proves none of it. If your migrations touch indexes, testing them on SQLite is testing a different program.

Fourteen new tests, and each was checked to fail without the scope rather than merely pass with it. A test that passes for the wrong reason is worse than no test — it's a green tick standing where a check should be.

3. Passkeys: four steps, not one

The scaffold this app came from shipped Features::passkeys() in the Fortify config, commented out with a note, because the packages weren't installed. Both are here now, and enabling it is genuinely four steps:

Publish (config + migration)
Migrate
Put both the PasskeyAuthenticatable trait and the PasskeyUser contract on User
Then uncomment the feature

Step 3 is the trap. The trait without the interface fails static analysis with class.missingImplements and nothing at runtime, so if you don't run analyse you'll find out later and further away.

Two config values are load-bearing and derived rather than set, which is the dangerous combination:

relying_party_id comes from APP_URL. A passkey is bound to the RP ID it was registered against. A wrong APP_URL in production is not a misconfiguration you can quietly correct later — every credential registered under the wrong one stops resolving.
user_handle_secret falls back to APP_KEY. The WebAuthn user handle is what binds a credential to an account, so rotating APP_KEY invalidates every passkey on file, with no error that says so. Set PASSKEYS_USER_HANDLE_SECRET explicitly before any rotation.

That's the second thing this month where APP_KEY turned out to be permanent in practice rather than in theory. Anything deriving a secret from it deserves an explicit value.

4. One seeder per layer

A seeder doing four jobs at once — platform catalogue, tenancy, a provider, and sample workloads — meant nothing could be seeded without the rest, and db:seed on a fresh install produced data a real customer would have to delete. Split by responsibility: catalogue only, owner-and-organisation, and development sample data hanging off the dev command rather than the prepare path.

Two latent bugs fell out of the move, both the sort that only surface once code runs somewhere new:

An owner_id column that's a non-nullable FK was being written as $owner?->id. So the catalogue seeder silently depended on the owner seeder having run first. The nullsafe operator is doing you no favours where the schema says the value is required — it converts "this must exist" into "let's find out later."
$user->update(['email_verified_at' => now()]) never did anything: the column is outside the model's fillable list. It appeared to work only because db:seed wraps seeding in Model::unguarded(). Call the same seeder from a test and the owner comes out unverified. Now it's markEmailAsVerified().

That second one is worth sitting with. unguarded() in the seeding path means mass-assignment bugs in seeders are invisible until someone runs the seeder outside db:seed. If you have seeders invoked from tests, that's a real gap.

Also: the owner seeder now re-asserts roles on an existing account holding the configured email, instead of returning early. Otherwise a fresh install where someone registered that address first gives you a superadmin nobody can use, and nothing on screen to explain it.

5. Honest labels and icons that resolve

A template library rendering as text-only cards, and two things behind it were wrong rather than merely plain.

Thirteen templates were labelled Custom — the enum case meaning "free-form, no enforced structure" — while being an exact edge → app → database. That put most of the library in one bucket and made the topology filter useless. They're now labelled by the shape their layers actually form. A few stay Custom on purpose: a gateway over its own store isn't microservices until there are services.

An icon column populated on all 63 rows and rendered by nothing at all. Now resolved through a small Blade component with a deliberate fallback, drawing from a brand-icon set committed to the repo — nothing fetched at runtime.

The test I'm happiest with walks every seeded icon key and fails on one that resolves to nothing, and asserts the <svg> actually reaches the rendered markup. A missing icon is invisible in the UI, never loud. And a Blade component that swallowed the SVG entirely would pass every other assertion you'd think to write.

Also eager-loaded the component relation on the index — fifteen cards each drawing a mark per component is fifteen pages of queries otherwise.

6. Docs that had drifted into lying

A full documentation rebuild, and the interesting part is how stale docs go bad. Not gradually vague — specifically wrong:

A root README still listing six documentation links, all six dead, pointing at paths that moved into numbered folders long ago.
A CI badge pointing at a workflow that doesn't exist, rendering broken for who knows how long.
A page titled "code quality" that was actually about a helper script not in the repository.
No testing documentation at all.
Stale facts stated confidently: a real-time stack that's commented out in the bootstrap file, one database engine named where five are supported, and the lint command described as the formatter when it's the syntax checker.
A documented artisan command that doesn't exist, and nine real ones undocumented.

125 markdownlint issues → 0. 8 broken internal links → 0. Every file reachable from the index.

The through-line with everything else today: a document that names a command that doesn't exist isn't slightly out of date, it's actively sending people the wrong way. Same failure mode as a checklist claiming coverage it lacks, or a deploy that reports success it didn't verify. Wrong is worse than absent, because absent makes you go and look.

7. Elsewhere

On a second product: a dependency refresh, and a documentation pass across the planning set. Nothing structural — but worth saying that keeping planning docs current is the same discipline as item 6, just applied before the drift rather than after it.

The through-line

Today divided into making the loop faster and making things say what's true. Both matter, and the second one showed up in five different costumes: a permission that guards a capability but reads like a scope, a seeder call that never ran, an icon column nothing rendered, a topology label describing a shape it wasn't, and a README pointing at six pages that had moved.

None of them errored. Which is the whole reason they survived.

Not Found is the wrong answer to a permission problem

Nasrul Hazim — Wed, 12 Aug 2026 23:35:04 +0000

I spent today putting an MCP surface on a multi-tenant Laravel control plane — the kind of app where a wrong answer doesn't just render badly, it tears down infrastructure. Sixty-odd tools across four servers, and honestly the tools were the easy part. The hard part was the base class every tool sits on, and most of what went into it came from getting it wrong first.

Here's the thing about exposing an app to an agent: every assumption your web UI quietly relies on stops holding. There's no session. There's no human reading the screen and going "hmm, that's odd." An agent takes your response literally, and then acts on it. So the design question stops being "what can this tool do" and becomes "what does this tool say when it refuses, and can the thing reading it tell the difference between refusals?"

Four guards, in order, each one a bug I shipped once

The base tool runs four checks before a tool's own code executes:

Is there an authenticated user at all?
Does their role grant this server?
Does this token carry the required ability?
Does their role grant this tool's domain permission?

Then it resolves a tenant. That's the shape:

abstract class McpTool extends Tool
{
    use ScopesToTenant;

    /** Which server this tool belongs to — `mcp.access.{server}`. */
    abstract protected function serverPermission(): string;

    /** The tool's own work. Guards have run; actor() and organization() are non-null. */
    abstract protected function run(Request $request): Response;

    protected function permission(): ?string
    {
        return null;
    }

    /** Read-only tools inherit this. Writers must say so. Destroyers must say `mcp:destroy`. */
    protected function ability(): string
    {
        return 'mcp:read';
    }

    final public function handle(Request $request): Response
    {
        try {
            $this->boot($request);

            return $this->run($request);
        } catch (McpException $e) {
            return $this->error($e->errorCode, $e->getMessage());
        } catch (ValidationException $e) {
            return $this->error(
                McpErrorCode::InvalidArgument,
                implode(' ', $e->validator->errors()->all()),
            );
        }
    }
}

handle() is final and run() is abstract on purpose. If a tool author can override the entry point, one of sixty tools eventually will, and it'll be the one that destroys things.

Note that the guards throw rather than return a Response. That was a deliberate switch partway through. Guards compose — server, ability, permission, tenant, then the record itself — and returning a refusal from each puts five if ($denied) return ... blocks at the head of every tool. That's precisely the shape where the sixth call site forgets one. Throw a typed exception, catch it once in handle(), and forgetting becomes impossible rather than merely unlikely.

The one I reverted: hiding tools you're not allowed to call

MCP has shouldRegister() — return false and the tool doesn't appear in the caller's tool list at all. Tempting. My first pass filtered on every permission a tool needed, so you only saw what you could run.

It's wrong, and it took a confused colleague to see why.

A tool that isn't registered answers "Tool not found." For someone pointed at a server their role doesn't grant, that's honest — the whole surface genuinely isn't theirs, and an empty tool list says so. But for someone who's on the right server and merely lacks one domain permission, "not found" is a lie with a cost attached. It sends them hunting for a typo in the tool name. What they needed to hear was:

This action requires the deployments.view.operations permission.

That's a sentence they can take to an administrator. "Not found" isn't.

So shouldRegister() filters on the server permission only, and per-tool permissions are enforced in boot(), where the refusal can name what's missing:

public function shouldRegister(Request $request): bool
{
    $user = $request->user();

    return $user instanceof User && $this->allows($user, $this->serverPermission());
}

protected function boot(Request $request): void
{
    // ...
    if (! $this->allows($user, $this->serverPermission())) {
        throw McpException::serverAccessDenied($this->serverPermission());
    }
    // ...
}

Notice the server permission is checked twice — once to hide, once to refuse. shouldRegister() is never the lock. A tool list can be stale, cached, or replayed by a client that connected under a different role; if the only thing keeping a tool out of reach is that you didn't list it, the list becomes the security boundary. It isn't one. It's a UI affordance.

Generalise it: visibility is a courtesy, authorisation is a check. Any time you're tempted to hide something instead of denying it, ask what happens when the caller guesses the name.

The guard mismatch that denies everything, superadmin included

This one cost me a good half hour of staring at a role that plainly had the permission.

MCP requests authenticate on sanctum. Roles and permissions are registered on web. A bare $user->can('...') resolves against the active guard — where Spatie has no such permission registered — so every check returns false. Every single one. Including for a superadmin, which is the detail that makes it so misleading: it looks exactly like a broken role assignment, so you go and debug the role.

protected function allows(User $user, string $permission): bool
{
    if ($user->hasRole('superadmin')) {
        return true;
    }

    try {
        return $user->hasPermissionTo($permission, 'web');
    } catch (PermissionDoesNotExist) {
        return false;
    }
}

Two things worth keeping:

Pin the guard explicitly. Not "it works locally because the session guard happened to be active" — name it.

Fail closed on an unknown permission. A typo'd permission string throws PermissionDoesNotExist, and letting that bubble up turns a policy question into a 500 from the authorisation boundary. Catching it and returning false means a typo denies rather than explodes. Which side you'd rather be wrong on isn't a close call.

The general rule: any surface authenticating on a non-default guard has to pin the guard it checks permissions against. Same trap bites API-token surfaces on Passport.

Permissions and abilities are not the same thing

A permission says what the human may do. A token ability says what this credential may do. They're independent, and collapsing them is how a read-only CI token ends up able to tear down production because the person who minted it happens to be an admin.

Every tool declares its tier — mcp:read by default, mcp:write if it mutates, mcp:destroy if it's irreversible — and the check is:

protected function assertAbility(Request $request): void
{
    $token = $this->accessToken();

    if ($token === null) {
        return; // no token, no ability list to narrow against
    }

    if (! $token->can($this->ability())) {
        throw McpException::abilityDenied($this->ability());
    }
}

Tokens minted before abilities existed carry Sanctum's * wildcard and pass everything. That's Sanctum's own semantics and I left it alone deliberately — retroactively narrowing a wildcard would break every already-connected client silently, at some unpredictable later moment. Migrate credentials forward on purpose; don't change what an existing one means underneath its holder.

No session means your tenant resolution is guessing

This is the one I'd flag hardest to anyone doing the same thing.

The app has a current_organization() helper. Reads the session, and when there's no session, falls back to "the first organisation you own." Perfectly reasonable for the web app, where the fallback fires roughly never.

Under a token there is never a session. So the fallback isn't a fallback anymore, it's the primary path — and a user who belongs to three organisations silently addresses whichever one sorts first. The agent has no idea. It asks for "the deployments," gets a list, and reports it as the answer.

Two rules came out of this:

Every tool takes an optional explicit organization. Wins over any default.

Every response echoes the organisation it resolved. Not as a debug field — as part of the contract:

protected function respond(array $payload): Response
{
    if ($this->organization !== null) {
        $payload['organization'] = [
            'uuid' => $this->organization->uuid,
            'name' => $this->organization->name,
        ];
        $payload['cross_tenant'] = $this->crossTenant;
    }

    return $this->json($payload);
}

An ambiguous answer that states its own scope stops being ambiguous. That's the cheapest fix available for a whole class of "it returned the wrong data" bugs, and it applies well beyond MCP — any API where scope is inferred rather than supplied should be telling you what it inferred.

The related decision: reaching another tenant is a named capability, mcp.access.cross-tenant, deliberately not inherited from the general organizations.* permissions. It's a capability flag, not a scope — per-resource access still runs through policies. And a cross-tenant read is written to the target organisation's audit trail:

// An ops user reading a customer's tenant is legitimate — and must still
// leave a trace. Written against the organisation, so it surfaces on that
// tenant's own trail rather than only in an internal log.
Audit::create([
    'user_id' => $this->actor()->id,
    'event' => 'mcp_cross_tenant_access',
    'auditable_type' => $organization->getMorphClass(),
    'auditable_id' => $organization->id,
    'new_values' => ['tool' => $this->name(), 'server' => $this->serverPermission()],
    'tags' => 'mcp',
]);

Support reading a customer's data is a normal, legitimate thing. The customer being able to see that it happened is what makes it normal.

One more, and it's structural: this app has no global scopes. Every query scopes itself or it leaks. So tools never reach a record with a bare where('uuid', ...) — they go through finders on a ScopesToTenant trait. Ownership enforced at five call sites is ownership forgotten at the sixth.

Give the agent a vocabulary, not just prose

An agent that only ever receives sentences can't distinguish "you may not do this" from "this doesn't exist" from "your token is too narrow" — and those call for completely different next moves. Retrying the first two is pointless noise. Retrying the third after minting a wider token is exactly right.

So there's an enum:

enum McpErrorCode: string implements Contract
{
    use InteractsWithEnum;

    case NotAuthenticated = 'not_authenticated';
    case ServerAccessDenied = 'server_access_denied';
    case PermissionDenied = 'permission_denied';
    case AbilityDenied = 'ability_denied';
    case CrossTenantDenied = 'cross_tenant_denied';
    case NotFound = 'not_found';
    case InvalidArgument = 'invalid_argument';
    case ConfirmationMismatch = 'confirmation_mismatch';
    case CapabilitySimulated = 'capability_simulated';
    case Conflict = 'conflict';

    /**
     * Whether the caller could succeed by supplying different arguments.
     * A retryable error is worth trying again; an authorisation failure is
     * not, and an agent that retries one just generates noise against a
     * boundary that will refuse it every time.
     */
    public function isRetryable(): bool
    {
        return $this === self::InvalidArgument
            || $this === self::ConfirmationMismatch
            || $this === self::NotFound;
    }
}

The code travels alongside the human message, never instead of it — a client with no code support still gets something readable:

protected function error(McpErrorCode $code, string $message): Response
{
    return Response::error(json_encode([
        'error' => [
            'code' => $code->value,
            'message' => $message,
            'retryable' => $code->isRetryable(),
        ],
    ]));
}

isRetryable() living on the enum rather than in each tool is the usual argument for enums-with-behaviour: the classification is a property of the error kind, not of the caller. Same reason label() and color() belong there.

Destructive tools: make the argument come from something the caller read

Three tools here can't be undone. An agent asked to "clean up the old deployments" will cheerfully call destroy on whatever it matched, and whatever it matched is doing an enormous amount of load-bearing work in that sentence.

The lock is a typed confirmation — the resource's own name, retyped exactly:

protected function confirmOrFail(Request $request, Model $subject, string $expected): void
{
    $supplied = $request->get('confirm_name');

    if (! is_string($supplied) || $supplied !== $expected) {
        $this->auditDestructive($subject, 'mcp_destructive_refused', [
            'expected' => $expected,
            'supplied' => is_string($supplied) ? $supplied : null,
        ]);

        throw McpException::confirmationMismatch($expected);
    }
}

The point isn't friction for its own sake. It's that the destructive argument now has to come from something the caller actually read — a prior get-deployment call — rather than from a pattern it inferred from the request. Same reasoning as making a human type a count instead of clicking a browser confirm.

And note the refusal is audited too. A refused destruction is evidence: it says something tried. Logging only successes gives you a record that's blind to exactly the events you'd most want to know about.

Combined with a mcp:destroy token ability, a state check on the model's lifecycle enum, and an #[IsDestructive] annotation so a well-behaved client can prompt its human, that's four independent things that have to line up. For an irreversible action against real infrastructure, four feels about right.

Say when the work is only queued

Last one, and it's my favourite because it's a bug this repo already shipped in the UI.

A lifecycle tool dispatches a job and returns. If the response says "deployment stopping," the agent reports it as done. Meanwhile the job is sitting unclaimed, because the work goes onto the database connection and Horizon's supervisors watch redis.

protected function queuedNote(string $what): array
{
    return [
        'queued' => true,
        'connection' => self::QUEUE_CONNECTION,
        'worker_required' => true,
        'message' => __(':what queued on the `database` connection — this needs a worker '
            .'(`php artisan queue:work database`). Horizon does not claim these jobs.', ['what' => $what]),
        'worker_appears_alive' => $this->workerAppearsAlive(),
    ];
}

/**
 * A cheap liveness hint, not a guarantee: jobs older than two minutes mean
 * nothing is claiming them. Null means the question could not be asked,
 * which is deliberately distinct from "yes".
 */
protected function workerAppearsAlive(): ?bool
{
    try {
        return DB::table('jobs')
            ->where('created_at', '<=', now()->subMinutes(2)->getTimestamp())
            ->count() === 0;
    } catch (Throwable) {
        return null;
    }
}

The ?bool matters. null means "couldn't check," and that is not the same as true. Any time a health signal can fail to be read, the un-readable case needs its own value — collapsing it into the optimistic one is how monitoring lies to you.

Server instructions are where the reading rules live

Each server ships #[Instructions(...)], and I ended up using it for something I didn't expect: not "here are the tools" but "here is how to read an answer without drawing the wrong conclusion." Things like:

Unscanned is not clean. An empty vulnerability list may mean nothing was found, or nothing was looked at.
An unknown severity stays unknown. Mapping it down to "low" launders an unknown into a reassurance.
No recorded activity is not "inactive." The audit trail records actions; someone who signs in and only reads leaves no trace.
A control with no evidence is unevidenced, not failing. Different states, different responses.
An overdue clock reports negative hours, not clamped to zero — overdue is the most important thing it can say.

That's institutional knowledge that normally lives in a senior engineer's head and gets passed on in code review. Writing it into the server instructions is the first time I've had somewhere obvious to put it. Worth doing even if you never connect an agent, frankly.

What I'd take to the next one

Refusals are an API. Design them as carefully as the success payloads. A wrong refusal costs more than a missing feature, because the caller acts on it.
Hide for convenience, deny for security. Check the same thing in both places.
Non-default guard? Pin it. Explicitly, everywhere.
A default that only fires without a session becomes the primary path under tokens. Go and read every fallback in your app with that in mind.
Echo the scope you inferred. Cheapest ambiguity fix there is.
Anything that can't answer a health question needs a third value. Not the optimistic one.

Next up is exercising the whole thing from a real client and seeing which refusals actually read well in practice. My guess is at least two of the messages I'm proud of today turn out to be useless in context.

The deploy script that always said it worked

Nasrul Hazim — Wed, 12 Aug 2026 23:34:47 +0000

TL;DR

I stood a Laravel control plane up on a bare Ubuntu host today. The provisioning script was the easy part. The deploy script was the one that had been lying to me.
git pull had been failing on every deploy since the first one — dubious ownership, because the script chowns the tree to www-data on the way out and runs as root on the way in. Nothing checked git's exit status, so the deploy went on to install, migrate and health-check code it had never updated. And it passed, because the old release was fine.
php artisan operations --force had never once run. --force is a migrate flag; the deploy operations command takes --isolated. It exited non-zero every single deploy and nobody looked.
The route cache was never cleared — only config and views. When / changed from rendering a view to redirecting, production started 500ing on every page, and the health check stayed green, because /up was in that same stale cache and still answered 200.
Under set -o pipefail, cmd | grep -q exits 141, not 0. grep -q leaves on the first match and the producer takes SIGPIPE. Read into a variable, then match with a here-string.

Today was the day the control plane had to run somewhere real: a bare Ubuntu host, provisioned from nothing, deployed to over SSH. I expected to spend it on the provisioning script — nginx, PHP-FPM, Redis, supervisor, the usual apt-flavoured tedium.

Instead I spent most of it on a deploy script that had been reporting success for weeks without doing its job. Three separate faults, all with the same shape: something failed, nobody read the exit status, and the next step happily proceeded on stale state that looked healthy.

That's the whole post. Everything below is the four places it showed up.

1. A pull that failed silently, for weeks

The deploy script ends by chowning the release tree to the web user. It starts by running as root. Git, quite reasonably, refuses to touch a repository owned by somebody else:

fatal: detected dubious ownership in repository at '/var/www/app'

Which would be fine — obvious, even — except the script looked like this:

php artisan down --retry=60

echo "Pulling codes"
git checkout "$BRANCH"
git pull "$REMOTE" "$BRANCH"

echo "Install dependencies"
composer install --no-dev --optimize-autoloader

No set -e on that stretch, no status check. Both git commands printed their refusal to stdout, exited non-zero, and the script moved on. Composer installed. Migrations ran. The health check hit the site and got a 200 — of course it did, the previous release was serving and it was perfectly healthy.

Every deploy after the first one deployed nothing, and every one of them said it worked.

The fix is two things, and the second matters more than the first:

# Deploys run as root against a tree owned by www-data. Declare it safe once,
# so a deploy cannot silently skip its own checkout.
git config --global --get-all safe.directory 2>/dev/null | grep -qx "$PROJECT_PATH" \
    || git config --global --add safe.directory "$PROJECT_PATH"

if ! git checkout "$BRANCH"; then
    echo "ERROR: could not check out $BRANCH -- nothing was deployed."
    php artisan up
    exit 1
fi

if ! git pull "$REMOTE" "$BRANCH"; then
    echo "ERROR: could not pull $BRANCH from $REMOTE -- nothing was deployed."
    php artisan up
    exit 1
fi

echo "Now at: $(git rev-parse --short HEAD)"

Note the php artisan up before each exit 1. If you're going to bail mid-deploy, decide deliberately whether the site comes back. Here nothing has changed yet, so it should. Later in the script, after migrations have run, the answer flips — more on that in a second.

And Now at: <sha>. One line, and the failure mode becomes visible from the deploy log alone: if the SHA doesn't move, the deploy didn't.

The general rule: any command whose failure would be survivable is a command whose failure you will not notice. Those are precisely the ones that need an explicit check, because the ones that blow up loudly take care of themselves.

2. A command that had never run, because of one wrong flag

Same script, further down:

php artisan migrate --force
php artisan operations --force

--force is a migrate flag. It means yes, in production, I'm sure. The deploy-operations command (I use dragon-code/laravel-deploy-operations) doesn't have it — its equivalent guard is --isolated, plus --no-interaction for the prompt. So every deploy, that line printed:

The "--force" option does not exist.

…exited 1, and the script carried on to restart Horizon. Post-deploy operations had never run. Not once.

if ! php artisan operations --isolated --no-interaction; then
    echo "ERROR: deploy operations failed."
    echo "The application is still in maintenance mode at $PROJECT_PATH."
    echo "Fix the operation, then re-run this script or 'php artisan up'."
    exit 1
fi

This is the flipped case from earlier: here I don't call php artisan up. Migrations have already run, so the release is half-applied, and a half-applied release must not go live. Maintenance mode stays on and the message says so explicitly, including the way out. A 503 that tells you why is a much better outcome than a live site running new schema against code that never finished deploying.

Which sets up the nastiest one.

3. An operation that succeeded and reported failure

The migration to give every legacy account its own tenant ran as a deploy operation. I put a progress line in it, the way you would in any command:

public function __invoke(): void
{
    $action = app(CreatePersonalOrganizationAction::class);

    User::query()
        ->whereDoesntHave('ownedOrganizations')
        ->cursor()
        ->each(function (User $user) use ($action): void {
            $action->execute($user);

            $this->line("  organisation created for {$user->email}"); // 💥
        });
}

An Operation is not a Command. It extends nothing console-shaped. No line(), no info(), no $this->command. So that call threw — after $action->execute($user) had already committed.

Look at the ordering, because it's the worst one available:

The work lands. The organisation is created, in the database, for real.
The operation throws on the reporting line.
The command exits non-zero.
bin/deploy does exactly what I just taught it to do and stops, leaving maintenance mode on.
Production serves 503 over a change that had, in fact, succeeded.

And it hides on the re-run. By then every affected account already has an organisation, so the query returns nothing, the closure never executes, and the throwing line is never reached. The operation "passes". You'd have to reproduce it against a fresh copy of production data to see it again.

Two things I'd take from that:

Side effects before reporting is a footgun. If the observable outcome differs depending on whether the logging worked, the failure is uninterpretable from outside. Do the work, or don't — but don't let a cosmetic line decide whether the caller thinks it happened.

Idempotent-by-query means self-concealing. Anything that loops over "rows not yet fixed" erases its own reproduction case as it goes. That's a feature for reliability and a trap for debugging, and it's worth knowing which one you're relying on at any moment.

The fix was one deleted line and a comment explaining why it must stay deleted.

4. A health check that shared the stale state it was meant to catch

This one is my favourite, because every individual piece is correct.

The deploy cleared caches:

php artisan config:clear
php artisan view:clear

Routes: not cleared. And the host had had route:cache run on it once, at some point, by some hand. So it was serving the route table exactly as it was at that moment, forever.

Then / changed from rendering a view to redirecting. Deploy, and production 500s on every request:

View [welcome] not found

— against a release whose routes/ file says nothing of the sort. That alone will cost you twenty minutes, because you're staring at code that is plainly right.

The part that made it silent, though, is the health check:

HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "${APP_URL}/up")

if [ "$HTTP_STATUS" = "200" ]; then
    echo "Health check passed"
fi

/up is a framework route. It lives in the same stale route cache. It answered 200 throughout, so the deploy reported healthy while every real page in the application was throwing.

A health check that reads from the same cache as the thing it's checking isn't a health check. It's a second copy of the bug agreeing with the first.

The repaired shape:

echo "Clear config, route and view caches"
php artisan config:clear
php artisan route:clear
php artisan view:clear

# ... migrate, operations ...

# Rebuild after migrations, before restarting processes, so the caches always
# describe the release that is about to serve.
php artisan optimize

# ...

# /up is not evidence the release works — probe a real page too, following
# redirects, since / is one.
HTTP_STATUS=$(curl -s  -o /dev/null -w "%{http_code}" "${APP_URL}/up")
PAGE_STATUS=$(curl -sL -o /dev/null -w "%{http_code}" "${APP_URL}/")

echo "Health check: /up=$HTTP_STATUS /=$PAGE_STATUS"

if [ "$HTTP_STATUS" = "200" ] && [ "$PAGE_STATUS" = "200" ]; then
    echo "Health check passed"
else
    echo "WARNING: Health check failed (/up=$HTTP_STATUS /=$PAGE_STATUS)"
    # ... roll back to $PREVIOUS_COMMIT
fi

curl -L on the second probe, because / redirects — without it you assert on a 302 and learn nothing about whether the destination renders.

The generalisation: probe at the layer you care about. /up proves PHP is alive and the framework booted. It says nothing about whether your application's pages render. Those are different claims and they fail independently, so check both — and print both numbers, so the log tells you which one broke.

Bonus: `grep -q` and `pipefail`

While hardening the provisioning script I hit a genuinely confusing one. This aborts under set -euo pipefail:

if apt-cache policy "php${PHP_VERSION}-opcache" | grep -q 'Candidate:'; then

grep -q exits the moment it finds a match. apt-cache is still writing, gets SIGPIPE, dies with 141 — and pipefail makes the pipeline's status the last non-zero one. So a successful match reports failure. Assign first, match against a here-string:

OPCACHE_POLICY="$(apt-cache policy "php${PHP_VERSION}-opcache" 2>/dev/null || true)"

if grep -q 'Candidate:' <<<"$OPCACHE_POLICY"; then
    PHP_PKGS="$PHP_PKGS php${PHP_VERSION}-opcache"
fi

Why the conditional at all: opcache stopped being a separate package in PHP 8.4. It's compiled into the interpreter now. Asking apt for php8.4-opcache doesn't warn, it aborts the whole install — so on a modern PHP you have to not request it, while on 8.3 and below you must. Hence: ask apt whether the package exists, then assert on the outcome instead of the package:

PHP_MODULES="$(php"${PHP_VERSION}" -m 2>/dev/null || true)"

if ! grep -qi 'zend opcache' <<<"$PHP_MODULES"; then
    echo "ERROR: opcache is not loaded for PHP ${PHP_VERSION}"
    exit 1
fi

A missing accelerator is a silent slowdown, never an error. So make it one.

Same category, different tool: install the PostgreSQL client from PGDG, not from the distro. Ubuntu noble ships client 16, and pg_dump is only backward compatible — point it at a newer server and it refuses with aborting because of server version mismatch. Managed clusters upgrade on their own schedule; your backup script finds out at 3am. Pin a client at least as new as the server and print pg_dump --version during provisioning so the log carries the evidence.

Testing the untestable bit

None of the above is unit-testable in any satisfying way — it's a shell script talking to a real host. What is testable is the thing the shell script deploys, and after today I test one specific claim: that the health-check target actually renders.

it('serves the root page the health check probes', function () {
    $this->get('/')
        ->assertRedirect()          // / redirects; the check follows it
        ->assertSessionHasNoErrors();
});

it('answers the framework health route', function () {
    $this->get('/up')->assertOk();
});

Trivial tests. They would have caught the stale-route incident in CI instead of in production — not because they're clever, but because they assert the same two things the deploy asserts, from a place where nothing is cached.

That's the pattern worth keeping: whatever your deploy uses as its definition of "healthy", assert the same thing in your test suite. If the two ever disagree, one of them is reading a stale copy — and it's the one running in production.

Takeaway

Every fault today was an unchecked exit status wearing a different costume. The lesson isn't "add set -e" — I had set -e where it mattered and it still slipped through the pipelines and the ifs.

It's narrower than that:

When a step fails and the next step can still succeed on yesterday's state, you have built a machine that reports success. Check the status, print the SHA, probe a real page — and make the deploy prove it changed something.

Next up: getting the provisioning script to the point where a fresh host is one command away, and having the deploy write its resulting SHA somewhere the application itself can display. If the release is going to claim it deployed, it should be able to show me what.

Dev Log: 2026-08-10 — a toast that lied, an enum with a missing case, and a design system that finally has an address

Nasrul Hazim — Tue, 11 Aug 2026 01:57:47 +0000

Four commits across two repos, and a short day by commit count that still managed to contain the most instructive bug I've hit in a while. The through-line, if there is one: things that report a state they haven't verified.

The big piece — bulk actions and how they report partial outcomes — has its own write-up here. This is the rest, plus the one detail from that work that generalises furthest.

1. The toast that was confidently wrong

Bulk lifecycle jobs went out with a named queue on the wrong connection. The queue name was registered, present in the supervisor list, entirely plausible in review. The jobs sat unclaimed in the table while the UI said "3 deployments stopping."

Two things worth carrying out of that:

A queue name registered for Horizon is a Redis queue. Force a job onto the database connection and it never reaches that supervisor — it's watching a different connection. The name being in config is not evidence that anything is listening on it. Local dev made it worse: the listener ran with no --queue flag, so it consumed default only, and a correctly-named queue with nobody on it looks exactly like a healthy one from the browser.

The message is what made it silent. "Stopping" is a claim about the world. The request can't honestly make that claim — all it did was enqueue. It now says "queued to stop — needs a queue worker on the database connection", which costs one clause and turns a mystery into a checklist item.

That went into the project's gotchas file the same afternoon. Traps like this aren't the kind you rediscover; they're the kind you re-introduce, six weeks later, in a different feature. A gotchas doc is cheap insurance against your own future confidence, and it's worth writing the entry while you're still annoyed enough to be specific.

2. An enum defined by the case it doesn't have

Providers grew a stored health status from their last connection check, and the enum has three cases:

enum ProviderHealthStatus: string
{
    case Healthy   = 'healthy';
    case Failed    = 'failed';
    case Simulated = 'simulated';
}

Two design notes, both of which are about refusing to let distinct situations collapse into one another.

There is deliberately no Unknown case. A provider nobody has ever checked has a null status, and null is what the UI renders as "never checked". Add an Unknown case and a never-checked provider can be stored as though a check ran and came back inconclusive. Those are different facts. Null already means "no value here" perfectly well; inventing an enum case for absence gives absence a way to be written down as presence.

Simulated exists because a fake driver passing is not a pass. If the driver behind a provider is a declared fake, the check exercised nothing real — no socket opened, no credential used. Folding that into Healthy would put a green badge next to a provider that has never been contacted, which is the exact failure mode this whole feature was built to catch. So it's amber, and its description says so plainly: no real driver for this provider type — nothing was contacted.

If you've read the last few of these you'll spot the pattern. Most of my week has been variations on don't let "we didn't check" render as "we checked and it's fine".

Enums with label(), description() and color() keep paying for themselves here, by the way. The moment the UI has to distinguish three shades of "we know something about this", the presentation belongs next to the case, not in a Blade @if ladder that four views each get slightly differently.

3. Bulk actions, briefly

The feature itself: select rows in any listing, act on all of them, and get back a sentence that says what happened and why the rest didn't. Skip reasons are counted individually, so the toast reads "2 not permitted, 1 already stopped" rather than "3 skipped". A run that changed nothing is a warning even when every skip was legitimate — the operator asked for something and got no change, and green means "your thing happened".

Selection is keyed on the public UUID rather than the internal id (these are client-writable properties), resolved through a tenant-scoped query so a UUID from elsewhere doesn't resolve at all, and cleared on any filter, search or page change — a selection that outlives its filter means confirming "delete 12" against a list you can no longer see.

Full write-up in the other post. Six listings now share the trait; the one still to do is the destroy confirmation, which asks you to retype the selected count when the number it should show is the count that will actually be acted on.

4. The design system got an address

Different repo, different kind of work. developers-hub-my/website — the DevHub site — got a persisted design system file and lost a stale planning doc.

The deletion is the more interesting half. DESIGN_PROPOSAL.md had been sitting in the repo describing work that was, by now, mostly already done. A planning document that outlives its plan is worse than no document: it reads as current, and the three items in it that genuinely remain are buried in twenty that don't. Those three moved to a tracked issue along with the tokens it documented, and a longer benchmark study broke out into sixteen individual issues, each with a verified current state and a target state.

The general rule I keep relearning: a document describes how things are; an issue describes work to be done. The moment a doc starts carrying intentions, it acquires an expiry date nobody is watching. Files in a repo are read as fact.

What replaced it is a MASTER.md that's the opposite — purely descriptive. Colour tokens with their CSS variable names, heading and body fonts, the spacing and shadow scales, and a pre-delivery checklist, all under one rule at the top: page-specific files override the master, otherwise the master applies. It codifies a direction the site had already committed to ("Trust & Authority" — navy and blue, WCAG AAA, full light/dark) so page work can be checked against a single source of truth instead of whichever page someone last looked at.

Nothing exotic. The value isn't in the tokens — it's that there's now exactly one place where a disagreement about them gets settled.

What's next

The destroy confirmation showing a count it hasn't verified is, on today's evidence, an offence I'm apparently determined to commit in every corner of the app. That one next.

"3 skipped" tells the operator nothing

Nasrul Hazim — Tue, 11 Aug 2026 01:57:35 +0000

TL;DR

Bulk actions over a mixed selection always skip rows. A count of skips is not a result — the operator needs to know which reason applied to how many, because "not permitted" and "already gone" call for completely different next moves.
Count skip reasons individually, not in total. One counter per reason is about ten lines of code and it's the entire difference between an actionable toast and a shrug.
A run that changed nothing is a warning, even when every single skip was legitimate. The operator asked for something and got no change; a green toast there reads as "done".
Selection is keyed on the public UUID, never the auto-increment id — it's a client-writable property.
The nastiest bug of the day wasn't in any of that. It was a toast that said "3 deployments stopping" while all three jobs sat unclaimed in the queue table, because the job was dispatched to a queue no worker was listening on.

Most of yesterday went into one feature that sounds boring on the changelog line: act on many rows at once. Select some rows in a listing, hit Stop, done.

It is boring, right up until you notice that a bulk action is the one place in a CRUD app where the operation is guaranteed to be partially refused. Single-row actions are binary — the button is either there or it isn't, and if you click it and it fails you get an error about that one thing. Bulk is different. Ten rows go in, the system has opinions about each of them individually, and something has to come back out that a human can act on.

The lie is "3 skipped"

Here's the shape I started with, which is the shape almost every bulk implementation ships with:

$succeeded = 0;
$skipped = 0;

foreach ($this->resolveSelected() as $row) {
    if (! $this->canAct($row)) {
        $skipped++;
        continue;
    }

    $this->act($row);
    $succeeded++;
}

$this->toast("{$succeeded} stopped, {$skipped} skipped.");

"7 stopped, 3 skipped." Looks like a result. It isn't.

Put yourself behind that message. Three rows didn't move. Is that because:

your role can't perform this action on them,
they were already stopped,
they belong to another team and never resolved at all,
or they're in a state where this action doesn't apply?

Each of those has a different next action. The first is "ask someone with the right role". The second is "nothing to do, carry on". The third is "you're looking at the wrong page". The fourth is "wait, then retry". A single integer collapses all four into "hmm", and "hmm" reliably becomes "click it again and see".

So the counter grew a key:

final class BulkActionResult
{
    /** @var array<string, int> reason => count */
    private array $skipped = [];

    private int $succeeded = 0;
    private int $failed = 0;

    public function skip(string $reason): self
    {
        $this->skipped[$reason] = ($this->skipped[$reason] ?? 0) + 1;

        return $this;
    }

    public function succeeded(): self
    {
        $this->succeeded++;

        return $this;
    }

    public function didNothing(): bool
    {
        return $this->succeeded === 0;
    }
}

That's it. That's the whole idea. A map instead of an integer, so the toast can say "2 not permitted, 1 already stopped" instead of "3 skipped", and the difference between refused and nothing to do stops being invisible.

Small object, and it earns its keep by being the only place that knows how to phrase the outcome:

public function toastMessage(string $summary): string
{
    $parts = [$summary];

    foreach ($this->skipped as $reason => $count) {
        $parts[] = __(':count skipped — :reason.', ['count' => $count, 'reason' => $reason]);
    }

    if ($this->failed > 0) {
        $parts[] = __(':count failed.', ['count' => $this->failed]);
    }

    return implode(' ', $parts);
}

Green is a claim

The severity of the toast turned out to need a rule of its own, and it's the bit I'd have got wrong on autopilot:

public function toastType(): string
{
    if ($this->failed > 0) {
        return $this->succeeded > 0 ? 'warning' : 'error';
    }

    return $this->didNothing() ? 'warning' : 'success';
}

Read the last line carefully. A run where every row was legitimately skipped is still a warning. No errors occurred. Nothing is broken. The system behaved exactly as designed. And it is still not a success, because success is green, green means "your thing happened", and the operator's thing did not happen.

This is the same class of bug as a stub that returns an empty array: technically correct, and it teaches the human the wrong thing. Colour is a claim about what changed. Don't make a claim you can't back.

Two rules the selection itself has to encode

Bulk selection lives in a trait shared by every listing, and two rules went in there because both are quietly easy to lose:

Selection is keyed on the public UUID, never the internal id. These are Livewire properties — client-writable, by definition. Resolving a bulk mutation from an auto-increment key that a browser handed you is guessable-id enumeration with a helpful loop around it. And even with UUIDs, the resolution query has to be tenant-scoped, so a UUID from another organisation simply doesn't resolve. The policy check is the second gate. It is not the first one.

Selection is current-page-only, and any filter, search or page change clears it. A selection that survives a filter change means the operator confirms "delete 12" while looking at a list of 4. Whatever they're about to destroy, they can no longer see it. Same reasoning for always clearing after the run — including when nothing succeeded — because the rows behind that selection were just re-evaluated, and leaving the boxes ticked invites a second click on a set nobody has re-read.

protected function reportBulkResult(BulkActionResult $result, string $summary): void
{
    $this->clearSelection();

    $this->dispatch(
        'toast',
        type: $result->toastType(),
        message: $result->toastMessage($summary),
    );
}

Check order is a UX decision, not a style one

In the loop, I check the row's status before the policy, and that ordering is deliberate.

The destroy path also refuses an already-destroyed row. So if the policy check came first, an operator with a weaker role selecting a row that's already gone would be told "not permitted" — which sends them off to ask for elevated access to do something that didn't need doing at all.

The usual argument for checking authorization first is to avoid leaking state to someone who shouldn't see it. That doesn't apply here: the row's status is rendered in the list they're looking at. Nothing leaks, and the more specific reason is the useful one.

Worth generalising: when two guards both refuse, the one whose reason is more actionable should run first — unless the other one's reason is the thing you're protecting.

The one that actually bit me

None of the above was the real bug of the day.

The bulk lifecycle jobs were dispatched with ->onConnection('database')->onQueue('deployments') — a queue name that is registered, that appears in the supervisor list, that looks entirely correct in review. The toast came back cheerfully: "3 deployments stopping."

Nothing was stopping. All three jobs were sitting unclaimed in the jobs table.

Two layers to it, and both are worth having in your head:

A queue name registered for Horizon is a Redis queue. A job forced onto the database connection never reaches it — the supervisor is watching a different connection entirely. The name being present in config is not evidence that anything is listening.
The local dev listener had no --queue flag, so it consumed default and nothing else. A named queue on the right connection with no worker on that name is indistinguishable, from the browser, from a queue that's working fine.

The fix in the code was one line — match the dispatch of the job that already got this right, which uses that connection with no named queue at all. The fix that matters more was in the message:

"3 deployments queued to stop — needs a queue worker on the database connection."

A message that claims the work is happening is what makes the failure silent. "Stopping" is a claim about the world. "Queued to stop" is a claim about what this request did, which is the only thing the request can honestly report. The extra clause costs nothing and turns a mystery into a checklist item.

And because a message is now load-bearing, the test asserts the message:

it('skips a row whose status forbids the action and says which requirement failed', function () {
    Queue::fake();

    $active  = deploymentAt(DeploymentStatus::Active);
    $stopped = deploymentAt(DeploymentStatus::Stopped);

    Livewire::test(Index::class)
        ->set('selected', [$active->uuid, $stopped->uuid])
        ->call('bulkStop')
        ->assertDispatched(
            'toast',
            type: 'success',
            message: '1 deployment queued to stop — needs a queue worker on the database connection. '
                   . '1 skipped — only an active or degraded deployment can be stopped.',
        );

    Queue::assertPushed(DeploymentLifecycleJob::class, 1);
});

Asserting on a full user-facing string feels brittle, and it is a little. I'd normally push back on it. Here it's the point: the string is the feature. If someone later "tidies" it back to "3 deployments stopping", I want a red test, not a shrug.

The connection and the null queue are pinned by that test too — because the trap isn't the kind of thing you rediscover, it's the kind of thing you re-introduce.

Takeaway

A bulk action's real output isn't the mutation. It's the sentence you hand back to the person who clicked it. Budget for that sentence like it's a feature:

one counter per skip reason, not one counter,
warning when nothing changed, even if nothing was wrong,
and never phrase a queued job as though the work already happened.

Next up is making the destroy confirmation carry the same honesty — it currently asks you to retype the count, which is good, but the count it shows is the selected count, not the count that will actually be acted on. Those two numbers are allowed to differ, and the moment they do, the confirmation is confirming the wrong thing.

Dev Log: 2026-08-09 — 72 classes that should have been 8 rows, and a compliance clock

Nasrul Hazim — Mon, 10 Aug 2026 02:02:26 +0000

Sixteen commits, all on one private deployment platform, spread across a very long day — a batch just after midnight and then a second run in the late afternoon.

The big architectural thread of the day — drivers that admit what they can't do — is written up properly in a separate post. This is everything else, which turned out to have its own through-line: most of it was about making a claim checkable.

1. Managed services as data, not as 72 classes

The catalogue seeds 24 managed service types. Four of them had provisioners. So a blueprint asking for MongoDB, RabbitMQ or MinIO got refused at validation — a clear failure, but a missing capability.

Filling that gap the obvious way meant 20 types × 3 backends = 60 more near-identical classes, each a copy of its two siblings, with one service's image tag and port living in three files that immediately start drifting apart.

Each of those classes was already about fifty lines of pure declaration — image, port, env, command, readiness probe, connection string — sitting inside a base class that held all the actual behaviour. That's not a class. That's a row.

final readonly class ComponentDefinition
{
    public function __construct(
        public string $slug,
        public string $image,
        public int $port,
        public string $connection,
        public array $env = [],
        public array $command = [],
        public ?array $k8sCommand = null,
        public ?string $readiness = null,
        // ...
    ) {}
}

Twelve hand-written provisioners became eight definitions and three thin adapters that answer their base class's seams from the data. Net: 679 lines added, 742 removed, and adding a service is now an entry, not a directory.

Three things the format had to admit rather than paper over, and they're the interesting part:

k8sCommand exists because the shells genuinely differ. Docker needs sh -c to expand $VAR; kubelet expands $(VAR) itself and a shell there is an extra process for nothing. Redis is the case. Pretending one command fits both would have shipped a broken Redis on one backend. When two things are 95% the same, the escape hatch for the 5% is not a design failure — pretending the 5% doesn't exist is.
No readiness probe resolves to false, never to ready. Nothing to check must not mean nothing to wait for.
A slug that isn't in the registry stays refused. An entry that merely guessed an image and a port would provision something that starts, accepts connections, and stores nothing where anyone expects. Kafka and friends are honestly absent, and a test asserts they are.

The refactor caught two things I'd have shipped otherwise, both because I kept the old tests running against the new implementation: a readiness probe I had quietly tightened, and one backend's --entrypoint argument that the definition initially applied only on the other. Both restored byte-identical.

A refactor that changes behaviour is not a refactor. The old tests are the only thing that can tell you which one you did.

There's also a test that walks every definition against the seeded catalogue, so a definition for a slug the picker never offers can't masquerade as coverage.

2. Scanning the images that are actually running

Vulnerability scanning went from a fake scanner to a real one wired to a CLI scanner, with severity as a proper enum and a scheduled command.

The framing in the commit title is the bit worth keeping: the images that are actually running. Not the images in the manifest, not the images in the registry, not the base images someone listed in a spec once. A CVE report about an image you aren't running is noise that trains people to ignore the report.

It also grew a scanner-availability check, because a security scan that silently doesn't run is — you'll recognise the pattern by now — worse than one that isn't installed.

3. Finishing the alert lifecycle

Alert rules and incidents already existed. What didn't exist was the part where anything happens: evaluation, notification, escalation.

So: an evaluator, a notifier, a mailable, an escalated_at timestamp on the response record, and a scheduled command to drive it. An alert rule with no evaluator is a monitoring dashboard's version of the same lie — it looks like coverage.

Paired with a real Prometheus / Grafana / Loki adapter to replace the simulated one, and a blueprint change that provisions the observability and identity layers as part of a deployment rather than as a thing you remember to do afterwards.

4. A compliance clock, and what a clock has to be

PDPA work: data residency validation, data subject export, and breach notification.

Two design notes generalise well beyond one jurisdiction's privacy law:

Residency is validated at blueprint time, not at audit time. The check went into the blueprint validation step, so a deployment that would place data in the wrong region fails before it exists. Compliance you can only discover after the fact isn't a control, it's a report.

A breach clock needs a start it didn't choose. The notification window runs from when the breach was detected, and detection has to be recorded by whatever noticed it — not set by the person who later has to explain the delay. A field that the interested party fills in is a field you can't rely on.

Subject access got a console command as well as a UI path, which is the pragmatic call: these requests arrive rarely, urgently, and often to someone who isn't going to click through an admin panel under time pressure.

5. Three checks before the deploy button

The onboarding checklist happily sends a new user to the deploy button, where three well-documented failure modes are waiting — all of which look like the application being broken:

No queue worker on the right connection: the UI says "provisioning started, this page updates live", and then nothing happens. Forever.
No scheduler: certificates lapse and rotation windows never close, silently.
A provider that resolves to fakes: the deployment reports Active with no infrastructure behind it.

A preflight check answers all three before the last step, and each answer carries what to do about it — a failing check that doesn't say how to fix it is just a red badge.

The thing that took the actual work was making each check answerable rather than plausible:

Laravel records nothing when a scheduled task fires, so the scheduler stamps a heartbeat every minute. Without it the check could only ever return "unknown", which is a guard that never fires wearing a checkmark.
The queue check asks whether an unclaimed job has been sitting for two minutes — not whether a worker is configured. A configured worker that isn't running looks identical from the config.
The provider check had to move from per-type to per-provider, because an adopted VM is real with credentials and fake without them. The type-level answer would have told an operator their half-configured provider could deploy.

That last one is a good general rule: check the instance, not the class. Configuration is a claim; state is the answer.

6. Helm chart, and bootstrap templates as data

Two smaller ones. A Helm chart so the platform itself installs onto Kubernetes the way anything else does — dogfooding, and also the fastest way to find out which of your config values were secretly required.

And bootstrap templates moved into seeds with the form able to set the rules they carry, which is the same move as §1: a template is data, and data belongs in a table where you can see all of it at once.

What's next

The definition registry covers eight services out of twenty-four. The remaining sixteen are now a data-entry problem with a test that refuses to let me fake it, which is roughly the best shape an unfinished feature can be in.

A driver that quietly does nothing is worse than one that isn't there

Nasrul Hazim — Mon, 10 Aug 2026 02:02:02 +0000

TL;DR

A method that can't do its job has three options: do it, throw, or return "I can't" as a value. What it must never do is return normally having done nothing.
I gave a firewall abstraction two extra methods — enforces() and unavailableReason() — so "not implemented here" became something a caller can act on instead of something it has to infer.
The security fix that actually landed wasn't a firewall at all. It was binding published ports to loopback, because a port bound to 127.0.0.1 is closed no matter what any firewall says or fails to say.
Fail closed on unrecognised input. An unknown exposure value binds loopback — a typo must never open a port.
If two lists in your codebase have to agree, derive both from one predicate and write the test that asserts they partition the set.

Yesterday I wrote about stubs that answer politely — the fakes and empty columns that make a system look complete while it does nothing. Today's work was the next layer of the same problem, and it's the more dangerous one.

A stub that returns an empty array is at least inert. What I spent the day on was code that reported success for work it had never done.

The record of intent that no packet ever meets

There was a network_policies table. Rows were written on every provision, from the first day the provisioning pipeline existed. Ingress rules, egress rules, allowed CIDRs, ports. All of it stored, validated, displayed in the UI.

Nothing ever read it.

Not "read it and got it wrong" — nothing read it at all. There was no code path anywhere that turned a row in that table into a rule on any machine. The deployment pipeline had a step called apply network policies, that step wrote the rows, the step returned success, and the pipeline reported the deployment as fully provisioned with policy applied.

That's a record of intent that no packet ever encounters. And it's a nastier bug than an unimplemented feature, because it produces evidence. An operator looking at that screen has every reason to believe their ingress is restricted.

So the fix was a contract — but the interesting part of the contract isn't apply():

interface FirewallContract
{
    /** @param list<FirewallRuleSpec> $rules */
    public function apply(array $rules): void;

    /** Read back from the enforcement point, not from the database. */
    public function current(): array;

    public function teardown(): void;

    /**
     * Whether this driver can actually filter packets.
     *
     * False means callers must not report policy as applied. It exists so
     * "not implemented here" is a value a step can act on, rather than
     * something inferred from a no-op.
     */
    public function enforces(): bool;

    /** Why enforcement is unavailable, when enforces() is false. */
    public function unavailableReason(): ?string;
}

Those last two methods are the whole point of the change. enforces() makes capability an explicit return value rather than something you deduce by reading the implementation and noticing it has an empty body.

And the only implementation that ships right now is the unenforced one, whose apply() throws. Every provider family resolves to it, each with its own reason naming the actual mechanism it would need — the DOCKER-USER chain for one runtime, the routing mesh for another, the CNI for the cluster case.

The step that used to report success now records Skipped, with the reason attached. The pipeline's test expectation changed from "20 steps succeeded" to "18 succeeded, 2 skipped." That diff — a green number getting smaller — is the actual deliverable.

Two steps stopped claiming work they had not done. That is a system getting more correct while its dashboard gets worse-looking, which is the trade you want and the one that's hardest to sell.

What I deliberately didn't ship

No real iptables driver. Writing DOCKER-USER rules against a host you can't verify against is how somebody locks themselves out of their own machine over SSH, and I'd rather ship a driver that says "I cannot" than one that half-applies.

A firewall that half-applies is worse than one that says it can't. Same principle as the no-op, one step further along.

Fix the exposure where the exposure is

Here's the part that reframed the whole issue for me.

The actual security problem wasn't the missing firewall driver. It's a well-documented Docker behaviour: published ports install DNAT and FORWARD rules that iptables evaluates before ufw's INPUT chain. So a node with a perfectly correct deny-inbound baseline still answers on every published port. ufw status says closed. nmap says otherwise. Both are telling the truth about different chains.

The instinct is to reach for a wider firewall baseline. Wrong instinct — the baseline is correct for what it covers. The fix is that a published port should bind an address:

final readonly class PortMapping
{
    public const LOCAL_BIND = '127.0.0.1';
    public const PUBLIC_BIND = '0.0.0.0';

    public function bindAddress(): string
    {
        return $this->public ? self::PUBLIC_BIND : self::LOCAL_BIND;
    }

    /** The value for `docker run -p`. Never a bare `host:container`. */
    public function publishArg(): string
    {
        return $this->bindAddress().':'.$this->publish.':'.$this->container;
    }
}

A port bound to loopback is invisible from outside the host whatever any firewall does or fails to do. Containers still reach each other by network alias. Anything that must be public goes through the reverse proxy, which is the one deliberate exception and writes 0.0.0.0 literally rather than inheriting it — it's the front door, and port 80 has to keep answering or ACME renewals fail.

Three design decisions in that small class worth stealing:

Fail closed on garbage. An unrecognised exposure value binds loopback. Not an exception, not a default-to-public — loopback. A typo in a config file must not open a port to the internet.

Write the limitation into the code, not the wiki. One orchestrator genuinely can't do this: its routing mesh publishes on every node and offers no bind address. mode=host narrows it to nodes running a task and no further. So a published port there is world-facing and the cluster firewall is the only control — and that sentence lives in a comment next to the code that can't fix it, rather than in a doc someone will find after the incident.

Verify the guard fires. The regression test fails on any -p argument that goes back to a bare host:container. I verified it by putting the old code back and watching it fail, because a regression guard nobody has seen fire is indistinguishable from one that cannot:

it('never publishes a port without a bind address', function () {
    $args = app(ContainerRuntime::class)->runArgumentsFor($workload);

    foreach (portArgs($args) as $arg) {
        expect($arg)->toMatch('/^(127\.0\.0\.1|0\.0\.0\.0):\d+:\d+$/');
    }
});

"I can't" as an exception type

Separate thread, same idea. I added a native runtime — deploying a workload straight onto a VM with systemd, no containers. Air-gapped and hardened estates need it, and containers aren't always on the table.

That runtime has to satisfy the same contract as the container runtimes, and some of that contract it genuinely cannot honour. scale() on a single VM has no honest implementation.

final class UnsupportedByRuntime extends RuntimeException
{
    public static function capability(string $runtime, string $capability, string $reason): self
    {
        return new self("{$runtime} cannot {$capability}: {$reason}");
    }
}

The alternative — returning quietly — would let an operator believe a workload had been scaled. The exception isn't a failure mode, it's the honest answer to a question that has no other answer. A named exception type also makes it greppable: "what can't this runtime do?" becomes a search rather than an archaeology exercise.

Note it's a different tool from enforces(). Use a query method when the caller can reasonably route around the gap (the pipeline can skip a step and record why). Use an exception when there is no sensible alternative path and continuing would be a lie. Getting that split right is most of the design.

Capability is per-provider, not per-type

The bug that made me split the axes properly:

Only one provider type was marked "ready", even though every adopted-VM type — cloud droplet, on-prem box, whatever — resolves to the same real SSH driver, the same agent, the same runtime once it has credentials. So an operator with a cloud VM had to register it as "Bare Metal". A lie the form tells once and a support ticket repeats forever.

Two things came out of fixing it.

One predicate, two lists. The provider form had a hand-written literal list of ready types, and a "coming soon" list computed from isReady(). So the moment a type became ready it vanished from both — excluded from one, filtered out of the other. Another type had been sitting in that hole for a while: a real driver that had never once appeared on the form.

Both lists now derive from the same predicate, and there's a test asserting they partition the enum. If two collections in your code have to agree, the fix is never "remember to update both."

it('partitions every provider type into ready or coming soon', function () {
    $ready = InfraProviderType::readyTypes();
    $soon  = InfraProviderType::comingSoonTypes();

    expect($ready->merge($soon)->sort()->values())
        ->toEqual(collect(InfraProviderType::cases())->sort()->values())
        ->and($ready->intersect($soon))->toBeEmpty();
});

Two axes, not one. Where the machine came from and how workloads run on it are independent. A bare-metal box can host containers; a hardened one cannot. So the container/native choice moved onto the per-provider capability set, not the type enum:

/**
 * Whether this provider deploys workloads as containers.
 *
 * Per provider rather than per type on purpose: a bare-metal box can host
 * containers and a hardened one cannot, so the type is the wrong axis.
 */
public bool $containers = true,

Defaulting to true keeps every provider that predates the key behaving exactly as it did. New capability flags should default to the old behaviour — otherwise adding a flag is a silent migration.

And "ready" got a definition worth writing down: deployable, not complete. A type is ready if a real driver, agent and runtime back it — while its load balancer and some provisioners are still simulated and still declared as such. The UI renders that as a per-capability list rather than a single badge, because collapsing it to one boolean is exactly what overclaims.

One more: the checklist where you tick which components a provider supports grew a warning when the provider has no provisioners at all. Ticking a box there restricts what the engine may schedule; it doesn't install anything. A restriction reading as a promise is the same class of bug as everything else on this page.

The takeaway

Every one of these is the same rule wearing a different hat:

Doing nothing must be distinguishable from doing the thing.

At the method level, that's an exception type. At the contract level, it's a capability query. At the pipeline level, it's a Skipped status carrying a reason. At the UI level, it's a per-capability list instead of a green badge.

None of it is clever. All of it is the difference between a system you can reason about and one that is confidently wrong in a direction you'll discover from a customer.

What's next

The rules table is now the desired state a future enforcer reads, which means the enforcer is a well-defined piece of work rather than an open question — with a contract that already tells it what it must never do. When I do write the real driver, the first test is that it can't be mistaken for the unenforced one.

Dev Log: 2026-08-08 — stubs that answer politely, and a CSS property that doesn't exist

Nasrul Hazim — Sun, 09 Aug 2026 02:18:52 +0000

Heavy day, mostly on one private deployment platform, plus a small but instructive fix in a PDF book engine and a pass over a product marketing site.

The through-line, and I didn't see it until I read the day back: almost everything was a thing that answered without being able to answer. A stub that says yes. A column with no producer. A CSS rule the renderer doesn't implement.

1. The platform: making the fakes declare themselves

Fourteen commits, and they're variations on one theme.

An approval gate that approved everything. A rollback engine with a contract and a fake whose step classes didn't exist — so a previous piece of work read the whole seam as scaffolding and routed around it. A complete access-review model with zero callers, meaning the compliance control it was written for couldn't be evidenced at all. A public-IP column with a migration, a UI row, and no driver anywhere that ever set it.

The fix that generalises: make "which parts of this are real?" an enumerable question. One enum case per resolvable capability — driver, agent, reverse proxy, load balancer, runtime — so a deliberate fake and an accidental fake stop looking identical in the code.

Two rules I'll be reusing:

Write compliance evidence outside the transaction it describes. Inside, it rolls back together with the thing it was recording — you lose the record exactly when you need it.
Null is unknown, not zero. An unreachable provider is not a cluster that shrank to nothing. Keep the last known value and say so.

Also landed: a bootstrap pipeline that handles both Debian and RHEL families from one path. The trick there was giving up on package lists — dnsutils is bind-utils, gnupg is gnupg2, iproute2 is iproute. A template naming packages is a template that works on exactly one family. So the baseline is stated as capabilities ("DNS tools", "time sync", "firewall") and an OS profile turns each one into the right names for the machine in front of it. Steps stop branching on the OS entirely, which is what makes the difference between the two families assertable in a test instead of only on a real box.

Written up properly in a separate post.

2. The book engine: I explained a bug with a property that doesn't exist

My favourite kind of commit — the one that deletes yesterday's fix.

I had a pagination problem in a print stylesheet: headings getting stranded at the bottom of a page with their content pushed to the next. The fix I'd reasoned out relied on break-after: avoid being transitive — heading welded to intro, intro welded to whatever follows.

It isn't transitive. It doesn't exist. The rendering engine honours break-inside only and ignores break-before/break-after: avoid along with the legacy page-break-* aliases entirely.

So every weld I'd added was inert, and the chain-cutter I'd introduced to stop the welds running away cut nothing. The bug was real; the mechanism I'd explained it with was fiction.

Two things came out of it:

The dead rules stay in the sheet — they're correct CSS and cost nothing if the print path ever moves to an engine that implements them — but they're now documented as dead, in a comment at the head of their section. Specifically so the next pagination bug isn't diagnosed in their terms again.
The actual fix uses the mechanism that does work: wrap the pair that must move together in one box with break-inside: avoid, and tag oversized code panels so they're allowed to break rather than being forced onto a fresh page and stranding the heading above them.

The general lesson, and it's not really about CSS: before you explain a bug in terms of a mechanism, check that the mechanism is implemented. A plausible explanation that produces a plausible fix and no observable change is far more expensive than no explanation, because it stops you looking.

3. Marketing site: every claim has to be checkable in the codebase

Smallest diff, most discipline. A pass over a product site to align what it says with what actually ships — a provider list corrected to what's implemented, deployment modes folded down to the ones that exist, spec tables replacing prose.

Not a code lesson. Still a real one: the marketing page is documentation with a shorter feedback loop and a worse failure mode. If a claim on it can't be traced to something in the repo, it's a bug — just one that gets reported by a customer instead of CI.

What's next

The capability enum only covers the resolver's own methods right now. The obvious next step is rendering it as a per-provider readiness table in the UI, so "this provider is 60% real" is something an operator sees before they deploy onto it, not after.

Which parts of this are real? Should be a question with an answer

Nasrul Hazim — Sun, 09 Aug 2026 01:37:12 +0000

TL;DR

The most expensive bug I hit yesterday wasn't a crash. It was a stub that answered politely.
Four shapes of the same problem: a column nobody writes to, a model nobody calls, an engine backed by a fake, and a resolver that falls back to a null implementation silently.
The fix isn't "delete the fakes". Fakes are how you ship in slices. The fix is making which parts are real an enumerable question instead of a code-reading exercise.
Corollary that keeps paying off: null means unknown, not zero. An unreachable provider is not a cluster that shrank to nothing.

Yesterday was a long day on a private deployment platform, and by the end of it I noticed every commit was the same commit wearing a different hat.

Nothing had thrown an exception. Nothing had failed a test. The system had simply been answering questions it had no business answering, confidently, for weeks.

Shape 1: the column with no producer

There was a ip_public column on a node record. It had a migration, it had a row in the UI panel, and something downstream preferred it over the internal IP when present.

It was always null. Not sometimes — always. No driver ever set it, and the status object the drivers return didn't even have a field for it, so the three places that re-read a node couldn't have refreshed it if a driver had.

The downstream branch that preferred it had never once executed. It looked like a feature. It was a comment.

Here's the thing about a column with no producer: it doesn't fail loudly, it fails as absence. Every reader takes the null branch, every reader is correct to, and the whole path stays green.

The interesting part came when I went to fix it, because it turned out different providers can't honestly answer the same question:

One scheduler backend can report the cluster node's external address — or its internal one, since a single-node dev cluster has no external address and still serves traffic there.
A hypervisor driver has to sort the guest agent's addresses by scope. Taking "the first IPv4" is how you end up publishing a Docker bridge address.
The container drivers report null on purpose. A container's published port answers on the daemon host, and that's provider credentials, not node state. Copying it onto every node row goes stale the moment someone edits that credential.

So the fix wasn't "make every driver fill in the field". It was deciding, per driver, what the field means — and then pinning the deliberate nulls in tests so a future contributor doesn't read them as a gap and helpfully fill them in.

it('reports no public ip for container-backed nodes', function () {
    $status = app(ContainerDriver::class)->getNodeStatus('node-ref');

    // Deliberate: the published port answers on the daemon host, which is
    // provider-level config, not node state. Do not "fix" this to non-null.
    expect($status->publicIp)->toBeNull();
});

A test whose whole job is to say this null is on purpose is one of the highest-value tests you can write. It converts an absence into a decision.

Shape 2: the model nobody calls

There was a complete access-review model. Enums, migration, markInProgress(), complete(). Zero references anywhere in the application.

The compliance control it was written for — periodic access review — couldn't be evidenced at all, because there was no way to actually run one.

The load-bearing part turned out to be completion, and it's worth stating plainly:

A review that records "revoke" and leaves the access in place is a permanently green control. That's worse than an absent one, because it produces evidence certifying nothing.

So completion does the revocation and the record in one transaction — deactivate the membership, clear the roles — and writes the evidence outside it. That second half is the part people get wrong, and I've now been bitten by it twice:

DB::transaction(function () use ($session) {
    $session->applyDecisions();   // actually revokes
    $session->markCompleted();
});

// Outside. Evidence written inside the transaction it describes gets rolled
// back together with it — you lose the record of the failure precisely when
// you need it.
$this->evidence->record('access-review.completed', $session);

Two more rules fell out of it, both the "obvious in hindsight" kind:

Completion refuses while any member is undecided. A review signed off with blanks certifies people nobody looked at.
Opening is idempotent per scope. Two open sessions mean two reviewers working from different membership lists, and whoever finishes last silently overwrites the other.

And the membership list is snapshotted when the review opens, not recomputed on render. A review of "who has access right now" is not a review, it's a dashboard.

Shape 3: the engine backed by a fake

The rollback mechanism had a contract, a plan/step schema, and a fake implementation whose step-class strings named handlers that didn't exist.

This one has a nasty second-order effect. A previous piece of work read the whole mechanism as scaffolding and routed around it — wrote its own path. So now you have two rollback stories, one of which is decorative, and the decorative one is the one with the nice contract.

That's the real cost of a convincing fake: it doesn't just fail to work, it teaches the next engineer that this seam isn't load-bearing.

Making it real meant four idempotent step classes, resumable from the first step that isn't already successful, and one seam — a single releaser class — where a previously-built image goes back onto a workload. Both callers now go through it, which means there's exactly one file to change when this needs to become zero-downtime.

Two things the tests caught that I'd have shipped:

Repeated rollbacks walked forward through the release history instead of back. Rolling back twice is not "go to the previous release" twice unless you're tracking where you already are.
A workload-scoped rollback whose health check fails should escalate to a full rollback — unless the runtime refused the image outright. That's a broken artifact, not evidence the infrastructure is at fault. Escalating there just widens the blast radius of a bad build.

Same category as the promotion engine, which was also a fake: it approved everything. An approval gate that cannot say no is not a gate, it's a log line. Replacing it meant a real exception type carrying which checks failed and why, so the UI can say something an operator can act on instead of "promotion failed":

class PromotionBlockedException extends RuntimeException
{
    /** @param  array<int, PromotionCheck>  $failed */
    public function __construct(public readonly array $failed)
    {
        parent::__construct(__('Promotion blocked by :gates.', [
            'gates' => implode(', ', array_map(
                fn (PromotionCheck $c) => $c->type->label(),
                $failed,
            )),
        ]));
    }
}

A distinct exception type rather than a bare RuntimeException, because the caller has something useful to say about it. If your error only carries a message, every handler upstream is reduced to string-matching or shrugging.

Shape 4: the silent fallback

This is the worst one, and it's the one that generalises furthest beyond infrastructure.

A resolver picked a driver per provider. When it couldn't find what it needed, it fell back to a no-op implementation. Quietly. So a provider configured with credentials that could never work reported "connection fine" — because the thing answering was the fake, and the fake always says yes.

The same fallback meant one deployment reported a load balancer it had never created.

The fix that actually stuck wasn't better error messages. It was making the question enumerable:

enum DriverCapability: string
{
    case Driver = 'driver';
    case Agent = 'agent';
    case ReverseProxy = 'reverse_proxy';
    case LoadBalancer = 'load_balancer';
    case Runtime = 'runtime';

    public function description(): string
    {
        return match ($this) {
            self::Driver => __('Creates, destroys, starts and stops nodes.'),
            self::Agent => __('Runs commands and reports metrics on a node.'),
            // …
        };
    }
}

One case per resolver method. Now "which parts of this provider are real?" has an answer you can render in a table, assert in a test, and show an operator — instead of an answer you get by reading every match arm by hand.

A deliberate fake and an accidental fake look identical in the code. They only become distinguishable when something declares which is which.

And for the probes themselves: stop asking the abstraction whether it works, and ask the underlying thing. A "test connection" that goes through the resolver tests the resolver. A probe that opens a real SSH session and reads /etc/os-release proves the login and answers a question you needed anyway.

The unit problem, briefly

One more from the same day, different flavour. Every driver implements provisionNode(). On one it creates a VM. On another, a container. On another, a single-replica scheduled workload.

The interface is honest — all of them provision a node. The word isn't. Counting node rows uniformly bills fifty containers on one host exactly like fifty machines across fifty hosts.

So the difference got named:

enum NodeBillingModel: string
{
    /** We create the machine — each node row is a host. */
    case PerProvisionedNode = 'per_provisioned_node';

    /** We schedule onto machines the customer already runs. */
    case PerClusterHost = 'per_cluster_host';
}

The discriminator I landed on: does the thing have its own OS kernel? It sorts every provider type cleanly, and — the part that matters for anything commercial — it's a line a procurement officer can verify without reading the code.

Then one class owns the count. Licence checks, quotas, invoice lines, the admin UI all come through it. A second implementation anywhere is how the dashboard and the invoice come to disagree in front of a customer.

And the rule I keep coming back to:

// An outage is not a cluster that shrank to zero. Keep the last known value.
if ($count === null) {
    return;
}

Unknown and zero are different answers. Conflating them is how a monitoring system reports perfect health five minutes after the region went down.

What I'd take from this

If you're carrying a system with staged, partially-built seams — and you are, everyone is — three things:

Every stub declares itself. Not in a comment. In something you can enumerate, list, and assert on.
Deliberate nulls get a test that says so. Otherwise your next contributor "fixes" them.
Probe the real thing, not your own abstraction over it. Your abstraction was written by someone who assumed it worked.

None of this is exotic. It's just that "does this work?" and "does this answer?" are different questions, and only one of them has been tested.

Dev Log: 2026-08-07 — scrubbing secrets, reconciling drift, and extracting a book engine

Nasrul Hazim — Sat, 08 Aug 2026 02:46:10 +0000

A heavy day across six repos. Rather than list commits, here's the through-line: almost everything was about making an implicit thing explicit — an implicit redaction rule, an implicit assumption that the database matches reality, an implicit colour palette baked into a rendering engine.

1. Secret scrubbing, as a package

cleaniquecoders/pii-protection picked up three pieces: SecretScrubber (pattern-based — PEM blocks, JWTs, URL credentials, AWS key IDs, KEY=value assignments), LiteralScrubber (mask exact values the caller already holds), and RedactedStrategy (fixed placeholder, no length preserved).

The design decision I keep coming back to: secrets and PII are separate passes on purpose. Point a PII scrubber at an infra log and it deletes the IP addresses you need to debug, while leaving the connection string untouched.

Also: don't mask credentials with a length-preserving strategy. ******* tells an attacker the value was seven characters.

Written up properly in a separate post.

Repo: cleaniquecoders/pii-protection

2. Drift reconciliation and a scaling engine

On a private deployment platform: a scheduled drift reconciler and a real scaling engine. Both landed as "observe → compare to intent → decide → act", behind contracts, driven by scheduled commands.

The rule that made it worth building: actual state is read from the provider, not from our own tables. A row saying "running" beside a VM that no longer exists is precisely the drift you're looking for — counting rows reports a healthy deployment right until someone tries to use it.

Second lesson, learned the annoying way: no baseline means unknown, not drifted. The first version flagged every layer of every pre-existing deployment.

Also separate post.

3. A hand-rolled redactor, deleted

Small commit, disproportionate satisfaction. The same platform had a bespoke LogRedactor — three regexes, no tests, written inside a feature ticket because nobody budgets a week for a redactor. It now delegates to the package above.

That's the whole case for extracting things: the standalone version gets the tests and the edge cases, because it's the only thing in the repo.

4. Email automations get an identity

On a private CRM: sequences gained a sender identity, a preview, a test send, goals, and analytics.

The interesting bit is goals. A sequence without a goal can only report "did it send". With a goal — replied, booked, purchased — it can report whether it worked, and stop emailing someone who already converted. That's one enum and one table, and it changes what the whole feature is for.

Worth noting for anyone building sequences: "preview" and "test send" are different features. Preview renders the merge fields against a sample contact; test send proves your sender identity, SPF and DKIM actually work. Shipping only the first is how you discover deliverability problems in production.

5. Extracting a book engine from the workspace it grew in

A private tooling project got the treatment every internal tool eventually needs: pull the engine out of the workspace it grew inside, and make everything it silently assumed into configuration.

The specific change I liked: the colour palette moved out of the rendering code and into the book's own config, alongside a token table applied across four output surfaces. Before, a "theme" was a patch. After, it's a file — and the proof it's a real seam is that two themes exist and look genuinely different.

Also a sanitisation pass to strip workspace-specific paths out of the engine. If your tool only works from one folder on one machine, it isn't a tool yet.

6. Public sites

The company site: dependency upgrade clearing all outstanding advisories, plus a correction to the business address and the map coordinates. Boring, necessary — a wrong map pin is a support ticket generator.
A new public site for a community summit went up. First commit, plain and static.

The through-line

Four of six items were the same move: something that was working fine as an assumption became a named, tested, configurable thing. A redaction rule became a package. "The DB is right" became a provider probe. A palette became a config file. A sequence's success became an enum.

None of them added a feature users would name. All of them made the next feature cheaper.

Drift detection and autoscaling are the same shape: ask the provider, then decide

Nasrul Hazim — Sat, 08 Aug 2026 02:45:20 +0000

TL;DR

Drift detection and autoscaling look like different features. They're the same loop: observe → compare to intent → decide → act.
The single most important rule: read actual state from the provider, not from your own tables. Your table saying "running" next to a VM that no longer exists is the drift.
Both belong behind a contract, driven by a scheduled command, so the decision logic is testable without touching real infrastructure.
A fake context object beats mocking the world.

Yesterday I built two things on a deployment platform: a drift reconciler and a scaling engine. I expected two unrelated features. Halfway through the second one I realised I was writing the first one again with different nouns.

Sharing the shape, because it generalises well beyond infrastructure — anything with "what I intended" and "what is actually true" has this problem.

The loop

observe actual state  →  compare to desired state  →  produce a decision  →  act (or just record)

For drift: desired = the recorded blueprint snapshot, actual = what the provider reports, decision = "this layer has 2 nodes and should have 3".

For scaling: desired = the policy thresholds, actual = live utilisation from the nodes, decision = "scale up by 1".

Same skeleton. Which means the same contracts, the same scheduling, and — the part people skip — the same testing strategy.

Rule 1: actual state comes from the provider

This is the one that decides whether the feature is worth building at all.

The lazy implementation counts your own rows:

// Wrong. This can never detect the failure you care about.
$actual = $deployment->nodes()->where('status', 'running')->count();

A row saying running beside a VM that somebody deleted by hand at 2am is exactly the drift you're hunting. Counting rows reports a perfectly healthy deployment right up until someone tries to use it.

So the reconciler asks the provider:

interface DriftReconcilerContract
{
    public function recordDesiredState(Deployment $deployment): void;

    public function detectDrift(Deployment $deployment): DriftReport;

    public function reconcile(Deployment $deployment): DriftReport;
}

and detectDrift() goes out to the provider driver for each node's real status. Slower, chattier, occasionally times out. Also the only version that's true.

Generalise the lesson: if your "verification" reads the same table your "write" wrote, you haven't verified anything. You've asserted your own database is consistent with itself.

Rule 2: no baseline means no drift

The first bug I wrote: every layer of every pre-existing deployment reported as drifted, because nothing had ever recorded a desired state for them.

$desired = $this->latestDesiredState($deployment, $layer);

// Nothing was ever recorded for this layer, so there is no baseline to
// diff against. Reporting drift here would flag every layer of every
// deployment provisioned before this ran.
if ($desired === null) {
    continue;
}

A comparison with a missing side isn't a difference — it's an unknown. Alerting on unknowns is how you train a team to ignore alerts in week two.

Related: write the baseline to a table you actually read. Mine was being snapshotted into a metadata JSON column that nothing queried, so the reconciler had to record it properly itself. Worth checking whether your "we already store that" is stored somewhere anyone reads.

Rule 3: pick a unit and make everything agree

The scaling engine works in utilisation percentages, not absolute cores and gigabytes. Not because percentages are nicer — because the node agent already reports fractions, and converting to absolutes needs per-node capacity that nothing reliably records yet.

Same for replicas. A layer's replica count is its node count — the same definition the provisioning pipeline uses when it honours min_replicas. If scaling and bootstrap disagreed about what a replica is, they'd fight each other forever, quietly, on a schedule.

Pick the definition your cheapest reliable source can produce, and make every component use that one. A more "correct" unit that only two of five components can compute is worse than a rough unit everybody shares.

Absolute consumption is still needed — for cost and quotas — so it lives in a separate resource tracker. Two concerns, two units, no fudging one into the other.

Rule 4: the schedule is the trigger, not the logic

Both features ship as an invokable service behind a contract, plus a thin console command:

// routes/console.php
Schedule::command('drift:detect')->everyFifteenMinutes()->withoutOverlapping();
Schedule::command('scale:evaluate')->everyFiveMinutes()->withoutOverlapping();

The command does argument parsing, iteration and output. The engine does the deciding. That split is what lets you run the whole decision path in a test in milliseconds, and it's why --dry-run is three lines rather than a parallel code path.

withoutOverlapping() is not optional here. A scaling evaluation that starts before the previous one's provisioning finishes will see the old node count and scale up again. Twice.

Rule 5: fake the context, don't mock the world

The genuinely useful trick. Both engines take a context object — the thing that knows how to reach this deployment's provider, node agent and DNS. In tests, hand them a fake:

$context = new FakeScalingContext(
    metrics: ['web' => ['cpu' => 0.91]],
    nodes: ['web' => 2],
);

$decisions = app(ScalingEngineContract::class)->evaluate($deployment);

expect($decisions)->toHaveCount(1)
    ->and($decisions[0]->action)->toBe(ScalingAction::Up)
    ->and($decisions[0]->delta)->toBe(1);

Not Http::fake(). Not five chained mock expectations. One object that answers the questions the engine asks, constructed with the scenario you're testing.

The difference matters when you're covering the cases that actually bite:

utilisation sitting exactly on the threshold
already at max_replicas and still hot
provider throws mid-evaluation
a node the provider has never heard of
two layers, one scaling up, one scaling down, same pass

Every one of those is a constructor argument away with a fake context. With mocks, each is a small research project, so in practice they don't get written.

it('does not scale beyond max_replicas', function () {
    $policy = ScalingPolicy::factory()->create(['max_replicas' => 2, 'scale_up_threshold' => 70]);

    $decision = evaluateWith(cpu: 0.95, nodes: 2);

    expect($decision->action)->toBe(ScalingAction::None)
        ->and($decision->reason)->toBe('at_max_replicas');
});

Note the decision carries a reason. "Did nothing" and "did nothing because it's capped" are very different answers at 3am, and a nullable reason string costs you nothing.

What I'd watch out for

Reconciling automatically is a big step. Detect-only first, log the reports, look at a week of them. Auto-correcting drift you don't understand yet means your platform fights your ops team.
Cooldowns before cleverness. A crude "don't act again within N minutes" prevents more incidents than any sophisticated prediction.
Provider calls fail. A failed status probe is unknown, not missing. Treating a timeout as "node is gone" is how a scheduled job deletes production.
Every decision should be recorded, including the no-ops. The scaling event log is what makes the thresholds tunable later.

The general pattern, stripped of infrastructure: record what you intended, observe reality from the authoritative source, diff on a schedule, and record every decision — including the decision to do nothing. That's an accounting reconciliation, a config sync, a cache invalidator, an inventory audit. Same shape every time.

DEV Community: Nasrul Hazim

Dev Log: 2026-08-11 — a tenant for every account, four frontend traps that never error, and the fourth comparison site

1. An account that can sign in and see nothing

Then the same rule, from the other direction

2. Four frontend traps, none of which produce an error

3. Saying what the thing actually is

4. The fourth comparison site

5. Housekeeping

The through-line

Dev Log: 2026-08-12 — a 96s suite that became 42s, a capability that wasn't a scope, and four steps to a passkey

1. Ninety-six seconds to forty-one

2. A capability permission is not a scope

3. Passkeys: four steps, not one

4. One seeder per layer

5. Honest labels and icons that resolve

6. Docs that had drifted into lying

7. Elsewhere

The through-line

Not Found is the wrong answer to a permission problem

Four guards, in order, each one a bug I shipped once

The one I reverted: hiding tools you're not allowed to call

The guard mismatch that denies everything, superadmin included

Permissions and abilities are not the same thing

No session means your tenant resolution is guessing

Give the agent a vocabulary, not just prose

Destructive tools: make the argument come from something the caller read

Say when the work is only queued

Server instructions are where the reading rules live

What I'd take to the next one

The deploy script that always said it worked

TL;DR

1. A pull that failed silently, for weeks

2. A command that had never run, because of one wrong flag

3. An operation that succeeded and reported failure

4. A health check that shared the stale state it was meant to catch

Bonus: grep -q and pipefail

Testing the untestable bit

Takeaway

Dev Log: 2026-08-10 — a toast that lied, an enum with a missing case, and a design system that finally has an address

1. The toast that was confidently wrong

2. An enum defined by the case it doesn't have

3. Bulk actions, briefly

4. The design system got an address

What's next

"3 skipped" tells the operator nothing

TL;DR

The lie is "3 skipped"

Green is a claim

Two rules the selection itself has to encode

Check order is a UX decision, not a style one

The one that actually bit me

Takeaway

Dev Log: 2026-08-09 — 72 classes that should have been 8 rows, and a compliance clock

1. Managed services as data, not as 72 classes

2. Scanning the images that are actually running

3. Finishing the alert lifecycle

4. A compliance clock, and what a clock has to be

5. Three checks before the deploy button

6. Helm chart, and bootstrap templates as data

What's next

A driver that quietly does nothing is worse than one that isn't there

TL;DR

The record of intent that no packet ever meets

What I deliberately didn't ship

Fix the exposure where the exposure is

"I can't" as an exception type

Capability is per-provider, not per-type

The takeaway

What's next

Dev Log: 2026-08-08 — stubs that answer politely, and a CSS property that doesn't exist

1. The platform: making the fakes declare themselves

2. The book engine: I explained a bug with a property that doesn't exist

3. Marketing site: every claim has to be checkable in the codebase

What's next

Which parts of this are real? Should be a question with an answer

TL;DR

Shape 1: the column with no producer

Shape 2: the model nobody calls

Shape 3: the engine backed by a fake

Shape 4: the silent fallback

Bonus: `grep -q` and `pipefail`