DEV Community: Eric Mollenthiel

Postgres RLS in Symfony: with one tenant in the fixture, your isolation suite passes with no policy at all

Eric Mollenthiel — Mon, 31 Aug 2026 08:33:49 +0000

Ten days ago I wrote that the Postgres role which runs your migrations bypasses every row-level security policy you wrote. Mads Hansen replied with the obvious next step, and it was better than the article: run the same cross-tenant query suite twice, once as the serving role and once as the owner role, and compare.

I did. It works, and I published the assertion. Then Marco took it apart with one question: what happens if you remove the violating state from the fixture? I measured that too, and it caught my own advice out. Equality is green in four databases out of five, and three of them are broken.

So this is both halves. If you only take the first one home, you get a test that is green on a database with row-level security switched off.

The invariant is not "the owner run must fail"

My first instinct was to assert that the owner run leaks: it sees rows from other tenants, so the assertion is assertNotSame. That assertion is wrong, and it goes red on the day someone applies the correct fix.

ENABLE ROW LEVEL SECURITY leaves the table owner exempt. FORCE ROW LEVEL SECURITY does not. After FORCE, the owner sees exactly what the serving role sees. Measured on PostgreSQL 18.3, same fixture, same query:

### 1. ENABLE only: the owner ignores the policy
tenant 1 : serving=[1] owner=[1,2] DIVERGENT  <-- leak
tenant 2 : serving=[2] owner=[1,2] DIVERGENT  <-- leak

### 2. After FORCE ROW LEVEL SECURITY: both runs return the same set
tenant 1 : serving=[1] owner=[1] IDENTICAL
tenant 2 : serving=[2] owner=[2] IDENTICAL

So the invariant that survives both worlds is: both runs return the same set. Not "the owner run fails". Write it as equality and it stays green when the schema is fixed, red when someone drops FORCE in a migration.

That is the assertion I gave in the thread, and it is where I would have stopped.

Equality alone is green in three databases that are broken

Marco's reply is the sentence I would keep from the whole exchange: the fixture is part of the security claim, not just test data. And its consequence, which is the part that made me go back to psql: if removing the violating state doesn't make the test fail, then the test was never actually proving the invariant.

So I built the same table five different ways and ran three assertions side by side. Tenant 1 is the one being queried. Equality is the double run above. Scoped is the ordinary one everybody writes: the serving role returns only its own rows. Disjoint is Marco's, made executable: set the context to the second tenant, assert that set is non-empty and does not intersect the first.

                            equality  scoped  disjoint
2 tenants, no RLS at all      GREEN     RED     RED
2 tenants, ENABLE only        RED       GREEN   GREEN
2 tenants, FORCE              GREEN     GREEN   GREEN
1 tenant,  ENABLE only        GREEN     GREEN   RED
1 tenant,  no RLS at all      GREEN     GREEN   RED

Only one row of five is a sound database. Read the columns rather than the rows.

Equality is green in four rows and three of them are broken. Two runs that both return everything are still equal, so it never notices the world where there is no policy at all. It is red in exactly one row: the one my article was about.

Scoped does not rescue it. Look at the bottom two rows: with a single organisation in the fixture, the serving role returns its one row whether the policy applies, or is inert because the app account owns the table, or does not exist at all. Both assertions are green, and one of those databases has the owner bypass sitting in it waiting for a second customer to sign up.

That is the headline, and it is worth saying without the table: if your fixture has one tenant, your tenant-isolation test passes against a database with no row-level security on it. The failure direction here is always too many rows, never zero, so a fixture that contains nothing to refuse cannot tell a working boundary from an absent one.

Only the disjointness column notices that the fixture stopped being a violation. It is red in the three broken rows equality sleeps through, green in the sound one, and between them the pair covers all four broken worlds.

Disjointness has to stay inside the boundary it is checking

The part I did not expect was how to write it without reaching for privilege.

The obvious way to assert "there is still a second tenant in that table" is to go look with a connection that can see everything. Under FORCE there is no such connection left, which is the whole point of FORCE, and grabbing a superuser to check the fixture puts the check back outside the boundary you are trying to prove. You would be proving the policy from outside the policy.

Querying as the second tenant is what stays inside it: same role, same policy, same request-scoped setting, nothing the application could not do itself. If tenant 2 comes back empty, the fixture has gone stale and the negative control is gone. If it comes back overlapping tenant 1, the boundary is not separating anything.

function visibleIds(Connection $c, int $tenantId): array
{
    $c->beginTransaction();
    try {
        // set_config(..., true) is SET LOCAL, but it takes a bound parameter.
        $c->executeStatement('SELECT set_config(?, ?, true)', ['app.tenant_id', (string) $tenantId]);
        $ids = $c->fetchFirstColumn('SELECT id FROM invoice ORDER BY id');
    } finally {
        $c->rollBack();
    }

    return array_map('intval', $ids);
}

$servingA = visibleIds($serving, 1);
$ownerA   = visibleIds($owner,   1);
$servingB = visibleIds($serving, 2);

// Nobody is above the policy.
self::assertSame($servingA, $ownerA);

// There is a policy, and the fixture still contains something for it to refuse.
self::assertNotSame([], $servingB);
self::assertSame([], array_intersect($servingA, $servingB));

That set_config call is worth a line of its own. SET LOCAL app.tenant_id = ? is not parameterisable: SET takes a literal, so you end up interpolating a tenant id into SQL, in the one place where you least want to. set_config(name, value, is_local) is the same thing as a function call, and DBAL binds it like any other query.

And the run prints the ids rather than a count on purpose. serving=[1] owner=[1,2] says something. 1 !== 2 says the same thing with much less to go on when it fails in CI at 2am.

The two ways "no tenant context" fails, and they are not alike

fresh connection, no context     : is_null=yes, rows=0            (silent)
reused connection, no context    : SQLSTATE[22P02] invalid input syntax for type integer: ""   (loud)

A connection that never had the setting returns NULL from current_setting('app.tenant_id', true), the policy matches nothing, and you get zero rows in silence. A connection that had it set with SET LOCAL inside a transaction gets the empty string back once that transaction ends, not NULL, and the cast to int raises 22P02.

The tempting fix is to make the policy tolerant:

USING (tenant_id = NULLIF(current_setting('app.tenant_id', true), '')::int)

Measured: it works, and it is a trap. It turns the loud failure into the silent one. Zero rows, no error, on a connection that just lost its tenant context. If you have to choose, choose the version that raises: a query that explodes is an incident, a query that quietly returns nothing is a customer email three weeks later.

The leak that hides under PHP-FPM

Last one, and it is the reason any of this matters outside the test suite. SET app.tenant_id = '2' without LOCAL, in a transaction that commits, survives the transaction. Not the request: the connection.

BEGIN; SET app.tenant_id='2'; SELECT count(*) FROM invoice; COMMIT;
SELECT current_setting('app.tenant_id', true), count(*) FROM invoice;
-- still_here | count
--  2         |     1

A rollback undoes it, a commit does not. Under PHP-FPM this is invisible, because the connection dies with the request and the next one starts clean. Put PgBouncer in transaction mode in front of it, or run the same code in a Messenger worker that holds a connection for hours, and the next transaction on that connection reads the previous tenant. The LOCAL keyword is the entire difference, and no test that runs one request at a time will ever tell you it is missing.

Reproduce it

Two bash scripts and two PHP files: they create a throwaway database and two roles, build the table in each of the five configurations, run the assertions under both connections, and drop everything on the way out. PostgreSQL 18.3, PHP 8.5.7, Doctrine DBAL. Nothing in them is specific to my schema, and I would rather you ran them against yours than believed my output. They came out of the multi-tenant kit I work on, ShipAnvil, but there is no product in them: it is a table called invoice with two rows.

Thanks to Mads Hansen for the double run, and to Marco for the question that showed the double run was half an answer. If you are running RLS in a Symfony app, the cheapest thing you can do this week is open your isolation test and count the tenants in its fixture.

Row-level security in Symfony: the role that ran your migrations bypasses every policy you wrote

Eric Mollenthiel — Fri, 21 Aug 2026 16:38:18 +0000

A comment on my last article was better than the article. The subject was single-database multi-tenancy, and @to21as argued that the predicate should not live in the ORM at all: put it in Postgres as a row-level security policy, and a Messenger worker, a line of native SQL and an ad-hoc psql session all get the same WHERE clause whether anyone remembered it or not. That is correct, and it is the strongest version of the case against doing it in Doctrine. It closes four of the five holes I had just finished listing.

Then came the warning: their two RLS bugs had both been invisible in tests, because the test connection was a superuser and superusers bypass RLS.

The trap is wider than superusers, and the wider version is the one that lands on a Symfony deployment. A plain role that merely owns the table bypasses that table's policies too. Not a superuser. No BYPASSRLS. Just the owner. And the role that owns your tables is, in almost every Symfony deployment I have read, the same role your application connects with.

Everything below was measured on PostgreSQL 18.3, on a throwaway database, and every command is in the article so you can disagree with the result rather than with me.

The setup, which is the one you would write

CREATE ROLE app LOGIN PASSWORD '...';
CREATE DATABASE app_db OWNER app;

That second line is not a strawman. It is what my own deploy guide says, and it is what makes doctrine:migrations:migrate work without a privilege dance, which is why it tends to be what a deploy guide says. Then a migration does the usual:

CREATE TABLE invoice (
    id              int PRIMARY KEY,
    organization_id int NOT NULL,
    total_cents     int NOT NULL
);

ALTER TABLE invoice ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON invoice
    USING (organization_id = current_setting('app.organization_id', true)::int);

RLS is on. The policy is right. The role is not a superuser and has no BYPASSRLS:

 current_user | is_superuser | has_bypassrls | table_owner | rls_enabled | rls_forced
--------------+--------------+---------------+-------------+-------------+------------
 app          | f            | f             | app         | t           | f

Two invoices in the table, one for each of two organizations. The application connects as app, sets nothing, and asks:

SELECT count(*) FROM invoice;

2. No error, no warning, no log line. Every tenant's rows, through a policy that is enabled and correct.

One line changes the answer

ALTER TABLE invoice FORCE ROW LEVEL SECURITY;

Same role, same connection, same query. 0. Set the tenant and you get exactly the one row you should:

SET app.organization_id = '2';
SELECT count(*) FROM invoice;   -- 1

The documentation is not hiding this. PostgreSQL 18, section 5.9, read on 2026-08-21:

Superusers and roles with the BYPASSRLS attribute always bypass the row security system when accessing a table. Table owners normally bypass row security as well, though a table owner can choose to be subject to row security with ALTER TABLE ... FORCE ROW LEVEL SECURITY.

"Normally bypass" is doing a lot of work in a sentence most of us read once, while looking for the syntax of CREATE POLICY.

Why a Symfony project walks into this and a test suite does not catch it

Three things have to line up, and a standard deployment lines up all three.

Doctrine creates the tables, under the credentials in DATABASE_URL. There is one connection string in a Symfony app. It runs the migrations and it serves the requests, so the serving role is the owning role. Nothing in the framework, in Doctrine or in Postgres considers that unusual, because it is not unusual. It is the default shape.

The failure direction is extra rows, not missing ones. A broken filter that returns nothing gets noticed in about four seconds. A filter that returns everything looks like a working page. Under RLS with an owner role you are not in a degraded state, you are in the state you were in before you wrote any of it.

Your fixtures make the two indistinguishable. A functional test that seeds one organization, logs a user in and asserts it sees its own row passes identically whether the policy applies or is inert. The assertion that catches this is the negative one: seed a second organization, and assert the first user cannot count it. If your suite has only the positive assertion, the day the policy stops applying is a day nothing turns red.

Which is how you ship a database with row-level security enabled on every tenant table, policies written and reviewed, and not one of them in force.

Two fixes, and I measured both

`FORCE ROW LEVEL SECURITY`

Add it to the same migration that enables RLS, next to the policy:

ALTER TABLE invoice ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoice FORCE  ROW LEVEL SECURITY;

Cheap, local, no infrastructure request. The catch is that it is per table and there is no default: the next tenant table someone adds in six months comes back with relforcerowsecurity = false and a policy that does nothing. This is a guarantee that decays.

A serving role that does not own anything

CREATE ROLE app_migrate LOGIN PASSWORD '...';   -- owns the schema, runs migrations
CREATE ROLE app_serve   LOGIN PASSWORD '...';   -- serves requests, owns nothing
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO app_serve;

Connecting as app_serve, with no FORCE anywhere and nothing set, the same two tables return 0 of 2. Set the tenant inside a transaction and you get 1. The bypass never existed, because ownership never existed.

This is the version that does not decay, and it is the version that costs you something real: two connection strings, a migration step that runs as a different user than the app, default privileges to get right for future tables, and a deploy document that now has a paragraph in it. If you are shipping a repo that other people deploy on infrastructure you will never see, that paragraph is a support cost forever. That trade is the actual decision, and it is not a database question.

The guard that survives the next table

Whichever you pick, this belongs in your suite, not in a runbook:

SELECT c.relname, c.relrowsecurity AS enabled, c.relforcerowsecurity AS forced
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
JOIN pg_attribute a ON a.attrelid = c.oid
                   AND a.attname = 'organization_id'
                   AND NOT a.attisdropped
WHERE c.relkind = 'r' AND n.nspname = 'public';

Every table with an organization_id column, and whether its policies actually apply. Assert that the list of unprotected ones is empty and the test names the new table for you the day someone adds it. On my demo database it correctly reported one table protected and one not, which is the only reason I trust the rest of this article.

And once RLS does apply, the setting has two ways to be missing

The policy reads current_setting('app.organization_id', true). There are two different empty states behind that call, they fail differently, and only one of them is loud.

SET LOCAL outside a transaction block does nothing. Postgres raises a warning, not an error, and PDO does not turn warnings into exceptions:

$pdo->exec("SET LOCAL app.organization_id = '2'");  // no exception
$pdo->query("SELECT current_setting('app.organization_id', true)")->fetchColumn();
// string(0) ""

A Symfony request has no open transaction until something flushes, so a kernel.request listener that issues SET LOCAL is issuing it into nothing. Scoping a request with RLS means an explicit transaction wrapped around the whole request, which is a much bigger architectural commitment than the two lines it looks like.

Then the two empty states diverge, and this is worth knowing before it happens in production:

never set: current_setting(..., true) returns NULL, NULL::int is NULL, the comparison is NULL, you get zero rows and no error. A blank dashboard.
set, then discarded: the setting exists and is the empty string, and ''::int throws SQLSTATE[22P02] invalid input syntax for type integer: "". A 500 on every query against a tenant table.

Same missing tenant, one silent and one fatal, decided by whether that variable was ever touched on the connection. Write the policy so you choose which one you get, rather than finding out.

And the reason it has to be SET LOCAL rather than SET: a plain SET outlives the transaction. Measured on one connection, two consecutive transactions:

-- request A, plain SET, tenant 2
   rows: 1
-- request B on the same connection, sets nothing
   current_setting: '2'
   rows it can see: 1

Request B never identified itself and is reading tenant 2. With persistent connections or a pooler in transaction mode, request B is a different customer.

What I would keep

RLS is the stronger mechanism. The comment that started this was right that it closes the holes an ORM-level filter leaves open, and I said so at the time. What I would not do is adopt it on the strength of ENABLE ROW LEVEL SECURITY and a policy that reviews well, because that pair is exactly the configuration I measured returning every row in the table.

Three things, if you are reaching for it this week. Check relforcerowsecurity, not relrowsecurity, because the first is the one that means anything when your app owns its tables. Decide between FORCE and a non-owning serving role on deployment cost, not on elegance, since both were airtight when measured. And test the negative case, because the positive one passes with the policy switched off.

I maintain ShipAnvil, a Symfony 7.4 LTS SaaS starter. Its tenancy layer is the Doctrine filter from the previous article, not this, and the paragraph about deployment cost is why. The article stands on its own. If your production database has RLS enabled today, the pg_class query above takes ten seconds and I would run it before finishing this page.

Single-database multi-tenancy in Symfony: a 31-line Doctrine filter, and the five places it never runs

Eric Mollenthiel — Wed, 19 Aug 2026 15:51:43 +0000

Single-database multi-tenancy is the cheapest kind: one schema, one connection, an organization_id column on every tenant-owned table. The whole design rests on one promise, and it is a promise about forgetting: no developer on the team will ever have to remember to write WHERE organization_id = ?, because forgetting it once leaks another customer's data.

Doctrine has had the tool for this for years. It is a SQLFilter, it is about thirty lines, and almost every article about it stops at the happy path. The interesting part is not the filter. It is the map of the places where it is simply not there, because that map is what you actually have to defend.

Everything below is read from Doctrine ORM 3.6.7 and from a suite that runs on every commit.

The filter

final class OrganizationFilter extends SQLFilter
{
    public const string NAME = 'organization';
    public const string PARAMETER = 'organization_id';

    public function addFilterConstraint(ClassMetadata $targetEntity, string $targetTableAlias): string
    {
        if (!$targetEntity->getReflectionClass()->implementsInterface(OrganizationOwnedInterface::class)) {
            return '';
        }

        return \sprintf('%s.organization_id = %s', $targetTableAlias, $this->getParameter(self::PARAMETER));
    }
}

OrganizationOwnedInterface is a marker with one method, getOrganization(). An entity opts into tenancy by implementing it, and that is the entire public API of the mechanism. No attribute to remember, no base class to extend, no trait whose absence is invisible in a diff.

The filter is declared in doctrine.yaml with enabled: false. That is deliberate, and it is the first design decision worth arguing about: a filter that is on by default in the container is on in your fixtures, in your migrations, in your data-repair scripts, and it will bite you at three in the morning. It gets turned on by the layer that knows who is asking.

The layer that knows who is asking

public static function getSubscribedEvents(): array
{
    // Right after the firewall (priority 8) so the user is available.
    return [KernelEvents::REQUEST => ['onKernelRequest', 7]];
}

public function onKernelRequest(RequestEvent $event): void
{
    if (!$event->isMainRequest()) {
        return;
    }

    if (str_starts_with($event->getRequest()->getPathInfo(), '/admin')) {
        return;
    }

    $organization = $this->organizationContext->getOrganization();

    if (null === $organization) {
        return;
    }

    $this->entityManager
        ->getFilters()
        ->enable(OrganizationFilter::NAME)
        ->setParameter(OrganizationFilter::PARAMETER, (string) $organization->getId());
}

Priority 7 is not a magic number. Symfony's Firewall listener subscribes to kernel.request at priority 8, so 7 is the first slot where Security::getUser() is populated. Higher and you get no user and therefore no filter at all, which is the worst possible failure mode, because the page still renders.

That is the whole mechanism. Now the useful part.

Place 1: the console, and every Messenger worker

There is no kernel.request in a CLI process. Your commands, your cron jobs and your Messenger consumers therefore run with the filter off, seeing every tenant's rows.

This is correct behaviour and you want it: a nightly billing command has to iterate over all organizations. But it means the guarantee "tenant data is invisible by default" is a guarantee about HTTP, not about your application. Every command that touches tenant-owned entities has to scope itself explicitly, and there is no compiler to remind you.

The honest framing is: the filter is a safety net under your controllers. Under your workers there is no net, and pretending otherwise is how a support script mails the wrong invoice to the wrong customer.

Place 2: your own back office, on purpose

An admin panel exists precisely to look across tenants. Filtering it would make the metrics wrong and the CRUD useless, so /admin is exempted by path.

The trade is that a path prefix now carries a security consequence, and path prefixes are easy to change. It only holds because the same prefix is locked to ROLE_ADMIN in access_control. If you copy this pattern, copy both halves, and treat the exemption list as security-critical code rather than as configuration.

Place 3: `find()` when the entity is already in the identity map

// EntityManager::find(), doctrine/orm 3.6.7
$entity = $unitOfWork->tryGetById($sortedId, $class->rootEntityName);

find() returns from the identity map before any SQL is generated. If an entity belonging to another organization was loaded earlier in the same request, by a fixture, by a cascade, by a getReference() that got initialized, then find() hands it back and no filter is consulted, because no query happens.

The same applies to getReference() itself, which builds a proxy from an id without touching the database at all. Passing a user-supplied id to getReference() and trusting the filter to reject it does nothing: the proxy is returned, and it only fails much later, when something initializes it.

Practical rule: a filter protects queries, not object identity. Authorization on an id that came from the outside is still authorization. Keep your voters.

Place 4: DBAL, and every line of native SQL

Filters are a DQL concern. $connection->executeQuery() and createNativeQuery() never see them. This is obvious once stated, and it is where the leaks actually happen, because native SQL is exactly what people reach for on the reporting and export screens, which are exactly the screens that show a lot of rows at once.

Place 5: joined inheritance, when the column is not on the root table

This one is silent, and it is the reason to read the ORM source rather than the documentation.

// SqlWalker::generateFilterConditionSQL(), doctrine/orm 3.6.7
case ClassMetadata::INHERITANCE_TYPE_JOINED:
    // The classes in the inheritance will be added to the query one by one,
    // but only the root node is getting filtered
    if ($targetEntity->name !== $targetEntity->rootEntityName) {
        return '';
    }

With JOINED inheritance, Doctrine only ever offers the root entity to your filter. So if your abstract root does not implement the tenancy interface and each concrete subclass does, the subclass is skipped by Doctrine and the root is skipped by your own implementsInterface() check. Two correct-looking guards, and the result is no WHERE clause at all.

The fix is a modelling rule, not a code change: in a joined hierarchy, organization_id and the marker interface belong to the root entity. Worth an architecture test if your domain uses inheritance.

And one place it does run, where most people assume it does not

The folklore says Doctrine filters only apply to SELECT. That is false in ORM 3, and it is easy to check: walkUpdateStatement() and walkDeleteStatement() both call walkWhereClause(), which is exactly where filters are injected. Here is the SQL Doctrine actually generated for me, with the filter enabled:

-- DQL: DELETE FROM SampleNote n
DELETE FROM sample_notes WHERE (sample_notes.organization_id = '...')

-- DQL: UPDATE SampleNote n SET n.title = :t
UPDATE sample_notes SET title = ? WHERE (sample_notes.organization_id = '...')

Bulk DQL statements are scoped. Note the table name in place of an alias, since useSqlTableAliases is false for those statements. Good news, but do not let it lull you: this is the ORM's DQL path only, and place 4 still stands one line away.

Proving it, rather than believing it

An isolation guarantee that is not tested is a comment. The check is short, and the part that matters is the assertion on the filter itself, the one that fails loudly the day someone "simplifies" the subscriber:

$client->loginUser($user);
$client->request('GET', '/dashboard');

self::assertTrue($entityManager->getFilters()->isEnabled(OrganizationFilter::NAME));

$notes = $entityManager->getRepository(SampleNote::class)->findAll();
self::assertCount(1, $notes);
self::assertSame('Mine', $notes[0]->getTitle());

Plus its mirror image, that an anonymous request leaves the filter off, which is what catches a firewall-priority regression before your customers do.

Two organizations, one row each, one authenticated request. It runs in under a second, and it is the only reason anyone should believe the paragraph at the top of this article.

What I would keep

Three sentences, if you are building this today.

The filter belongs off in the container and on at the edge, because the layer that knows the tenant is the only layer entitled to turn it on. The marker interface belongs on the root entity, and an architecture test should say so. And the filter is a net under HTTP only, so every command, every consumer and every line of native SQL is code you have to read with tenancy in mind.

I maintain ShipAnvil, a Symfony 7.4 LTS SaaS starter, and this is the tenancy layer it ships, isolation tests included. The article stands on its own, though. If you find a sixth hole, I would genuinely like to hear about it.

Every no-code builder generates code. I shipped an interpreter instead.

Eric Mollenthiel — Sat, 15 Aug 2026 13:00:20 +0000

Every no-code builder I have used ends the same way: you drag boxes around, you press
Save, and somewhere a template engine writes source code you are now responsible for.

I spent a few months building a plugin builder for Minecraft servers, and the obvious
version of it generates Java. Blocks in, .java out, Gradle, a jar you download. I got
close enough to that design to see where it ends, then threw it away and wrote an
interpreter instead.

This is what that decision actually costs and actually buys, with the parts that only
became obvious after the engine existed.

The domain, in four lines

A Minecraft server plugin is a jar built against the Paper API. It hooks events (a player
joins, a block breaks, someone types in chat) and reacts.

Twice a year Minecraft ships a version, the API moves, and every plugin on every server
breaks at once. Plugin authors spend their lives recompiling. Server owners spend theirs
waiting. Fifteen years of this, and it is the single loudest complaint in the ecosystem.

That release cycle is not a detail of my domain. It is the thing that decides the
architecture.

Why generating Java is the trap

Generated code looks free at the design stage. You already know the shape of the output,
templates are easy, and the result is a normal plugin that behaves like every other plugin
on the server. The build is a solved problem.

Then you write down what happens on the day the API moves.

Every jar you have ever generated is frozen. The generator can be fixed in an afternoon,
but the fix reaches nobody: the broken code is already sitting in other people's server
directories. To ship it you have to regenerate and recompile every plugin of every user,
which means you need the original inputs of all of them, a build farm, a queue, and a
migration story for the ones whose regenerated output no longer compiles. A bug in a
template becomes an outage with a per-customer fan-out.

And that is the good day. On the bad day you are compiling and executing generated code in
production, which means a JDK, Gradle, a worker pool and a cache in the serving path, and
an attack surface shaped like "arbitrary Java, from the internet, run on a game server".
Minecraft in particular had a real supply chain attack in 2023 (fractureiser), spread
through plugin jars. Becoming another distribution channel for opaque bytecode was not a
neutral choice.

What I shipped instead

The studio produces a versioned JSON spec: trigger, conditions, actions. One hand-written
Paper plugin, the runtime, reads specs and executes them. Nothing generates Java anywhere.

A complete plugin:

{
  "specVersion": 1,
  "name": "Welcome",
  "rules": [{
    "name": "Welcome message",
    "trigger": { "type": "player.join" },
    "conditions": [
      { "type": "player.has_permission", "params": { "permission": "nimblock.vip" } }
    ],
    "actions": [
      { "type": "player.send_message", "params": { "message": "&6Welcome {player}!" } }
    ]
  }]
}

Saving that writes a row. Installing it drops a file in the server volume and hot reloads.
No build step exists to be slow, so nothing has to be made fast.

The comparison that decided it:

	Generate and compile	Interpret a spec
Save a plugin	a Gradle build, tens of seconds	a database write
Install on a running server	rebuild, redeploy, restart	drop the JSON, hot reload
Production infrastructure	JDK, Gradle, cache, workers, queue	none
Attack surface	compiling and running generated code	closed grammar, no arbitrary Java
Fix an engine bug	recompile every plugin of every user	publish one jar
Minecraft ships a version	every generated jar is broken	the runtime absorbs it once

The last row is the whole product. When the API moves, I fix one plugin, everyone's rules
keep running, and a spec written today should still run in three years untouched.

The part I did not see coming: the grammar becomes data

Here is the second-order effect, and it turned out to matter more than the build times.

When the grammar is a data file instead of a code generator, it can be the single source of
truth for both sides of the system. Mine is one catalogue-1.json: 9 triggers, 8
conditions, 14 actions. The JSON Schema is derived from it. The PHP validator in the studio
reads it. The Java engine reads the same file, copied into the jar at build time.

That buys two things a code generator cannot have.

Nothing can be offered and not implemented. The worst possible bug in a builder like
this is silent: an action the editor accepts, that nothing executes. The user places it,
the panel says fine, and nothing happens in game with no message anywhere. So the runtime
has a test that compares the catalogue to the handler registry in both directions:

@Test
void everyActionHasAHandler() {
    assertEquals(new TreeSet<>(catalogue.actionIds()), new TreeSet<>(new Actions(null).ids()));
}

Nothing in the catalogue without a handler, nothing handled outside the catalogue. With
generated code that test does not exist, because there is no registry to compare against:
you would be diffing templates.

The catalogue can carry a type system. Each trigger declares what it provides, each
action declares what it requires:

{ "id": "schedule.repeat", "context": [] }
{ "id": "player.heal",     "requires": ["player"] }
{ "id": "cancel_event",    "requires": ["cancellable"] }

So "heal the player" under "every N seconds" is refused as you write it, not at runtime,
because that trigger provides no player. cancel_event only exists under a cancellable
trigger. A typo'd {palyer} is rejected instead of landing in someone's chat as literal
text. That is roughly a type checker, expressed in a data file, and it exists only because
the grammar is data rather than a template.

Third rule, less about types and more about failure modes: a spec is accepted or rejected
as a whole. A protection plugin that loaded half its rules would fail silently in game,
in the worst way, so it does not load half.

The bill

An interpreter is not free, and the price is paid by the user, not by me.

The grammar is closed. Whatever is not in those 9 triggers, 8 conditions and 14 actions is
impossible, not "on the roadmap". Conditions do not nest either: all / any plus a not
per condition, no boolean tree. That last one is a genuine v1 limitation rather than a
principled stand, and it has an ugly reason: nested booleans draw badly as blocks.

If you need a guild system with its own database and its own GUI, this is the wrong tool
and it always will be. If you need "when a player joins for the first time, give them a kit
and announce it", it is two minutes. Knowing which side of that line you are on before you
invest is worth more than a longer feature list, so the full grammar is a public page and
you do not need an account to read it.

The escape hatch is a single "run as console" action, with exactly the power of the server
console and no more.

Three things the interpreter forced me to get right

Command registration is not hot. Paper only accepts new command names during its
COMMANDS lifecycle event, so a rule that declares a new command exists only after a
restart, while everything else in the same spec is live immediately. Rather than guess, the
agent records what Paper actually registered at boot; the difference with what the
installed specs declare is the pending list the panel shows. Same field left null tells
me a server predates the studio, so a reload would only produce a baffling "unknown
command" and the panel asks for a restart instead.

Cancellation is applied before the rule's actions, not after. For an ordinary rule the
order is unobservable. But chat arrives on another thread, and the remaining actions have
to hop back to the main thread, which is far too late to cancel anything. Applying it first
is the only version where chat moderation rules work at all.

A failing action stops its rule, never the server. Unknown item name, deleted world, a
numeric comparison on text: those are spec mistakes. They are worth one line in the console
and nothing more.

The obvious objection

"Fine, but my server is not hosted by you."

Right, and this is the part where the design pays off again rather than the part where it
falls apart: an export does not need a JDK either. The runtime jar is already compiled, so
exporting is injecting the specs as resources, rewriting plugin.yml, and rezipping.
ZipArchive in PHP, on the order of a hundred milliseconds, no build server anywhere. One
jar per account containing all of its specs, never one jar per plugin, so installing two of
them on the same external server cannot collide.

To be straight with you: that is designed and costed, not shipped. It is the next batch. I
am saying it here because it is the standard objection to a hosted builder, and because
"the escape route is cheap" is a consequence of the interpreter, not a promise bolted on
afterwards.

Where it is

It is live and free. The stack is Symfony 8 and PostgreSQL 18 on a single box, Paper
servers in rootless Podman containers, one network per server, filtered egress. The machine
holds twelve 1 GB servers, so there are twelve slots, which is a hardware limit rather than
a growth-hacking number.

The full grammar, every trigger, condition and action, generated from the same file the engine reads: https://nimblock.com/en/grammar
The studio, with a hosted Paper 26.1.2 server to try it on: https://nimblock.com/en

What I want back is not signups. It is the list of things you tried to express and could
not, because that list is what decides whether the closed grammar was the right call or
just the easy one.

Your storage format is the one thing you cannot refactor

Eric Mollenthiel — Sat, 15 Aug 2026 09:52:30 +0000

Ten days ago I published a post about encrypting personal data at rest in Symfony. A reader, Mads Hansen, left a comment that did not argue with the cryptography at all. It argued with seven characters that were not there.

The type I had shipped told encrypted rows from legacy plaintext rows by looking at the first byte: a { or a [ meant "this row predates encryption, hand it back as JSON". His objection, compressed: that makes the storage format ambiguous forever, and the ambiguity is a security property, not a cosmetic one, because a row that merely looks like JSON opts out of decryption no matter who wrote it.

He was right, and fixing it cost about eighty lines including tests. But the reason I am writing a second post rather than editing the first one is that the bug was not really about encryption. It was about a rule I had followed everywhere else in that codebase and broken here, and it is the one rule of software design I would keep if I had to throw away all the others:

everything you write can be refactored later, except the format of the bytes you have already stored.

Code is soft, storage is not

We say "software" and mean it. A class name is a rename away from being better. A bad abstraction survives until someone has an afternoon. Even a database schema, which feels heavy, is genuinely malleable: a column is added, backfilled, and dropped, and the tooling for that has been boring and reliable for twenty years.

Formats are different in kind, not in degree, and it took me an embarrassing number of years to feel the difference in my hands rather than know it as a maxim. A format is not code you own. It is an agreement between the process that wrote a value and every process that will ever read it, including the ones running a version of your code that no longer exists, on rows written by a colleague who left, in a database restored from a backup taken before the decision you are about to make.

You cannot change one side of an agreement unilaterally. What you can do is read the old shape and write the new one, which is exactly what a migration is, and which is exactly what requires you to be able to tell the two apart. That last part is the whole game.

Sniffing is the tax you pay for skipping the version

When a stored value carries no marker, the only way to know what it is, is to look at it and guess. Every codebase has these, and once you learn to see them they are everywhere:

if ($v[0] === '{') then it is JSON, otherwise it is PHP serialize() output.
if unserialize() throws, fall back to json_decode().
if the first two bytes are 0x1f 0x8b, it is gzipped, otherwise it is raw.
if the string is 32 hexadecimal characters, it is the old token format.
if the value is base64-decodable, it is the encrypted one.

Each of these works. That is precisely the problem: they work well enough that nobody replaces them, and each one quietly promotes an accident of the payload into part of the contract. The last example is the one I shipped, and it is the worst of the family, because "looks like plaintext" and "is plaintext" were being treated as the same statement on a column that exists to keep buyers' names and addresses from being readable.

There is also a subtler cost, the one that bites long after. A heuristic is not just a guess, it is a constraint on all future values. Once "starts with {" means legacy, no future version of the format may ever start with {. You have spent a byte of design space without noticing, and you will notice the day you want to store something that starts with {.

The window closes when the first row is written

Here is the asymmetry that makes this worth a post.

Adding a version marker to an empty column costs seven bytes and about four minutes. Adding one to a column with a million rows costs a migration that must read values written without a marker, which means the migration needs a heuristic, which is the very thing you were trying to get rid of. You do not escape the guess, you only get to bury it in a one-off script instead of the read path, and you get to keep it in the read path anyway until the backfill is provably complete.

So the marker is free exactly once, and the free moment is the moment the column is empty. Which is also the moment when you are most certain you will never need it, because there is only one format and it is obviously the right one. Every ambiguous storage format I have ever met was created by a competent person on the day it was cheapest to prevent.

My table had five rows. That is the only reason this post describes a fix rather than a permanent scar. If it had held a million, the honest advice would have been the opposite of what follows: keep the heuristic, document it as load-bearing, and write the version prefix only into new values, carrying both paths for years.

What the fix looks like

The value stored is now an envelope, and the envelope is three fields that each earn their place:

enc:v1:k1:<base64 nonce + ciphertext>

enc:v1: is the format version. k1 is the key identifier. The payload is the payload. Written out as PHP, the read path stops asking the value what it is:

public function convertToPHPValue($value, AbstractPlatform $platform): ?array
{
    if (!\is_string($value) || '' === $value) {
        return null;
    }

    if (!CipherEnvelope::isEnveloped($value)) {
        // Fail closed: a readable row is a bug or an intrusion, never a fallback.
        throw new \RuntimeException('This value is not in the "'.self::PREFIX.'" envelope.');
    }

    [$keyId, $ciphertext] = CipherEnvelope::split($value);

    return $this->decode(self::keyring()->get($keyId)->decrypt($ciphertext));
}

Three consequences follow, and only the first one is obvious.

Reading fails closed

There is no longer a branch that returns plaintext. A row that does not carry the envelope stops the application instead of quietly serving personal data in clear.

That is a real cost, paid at the worst moment: restore a pre-migration backup and the app breaks rather than degrades. I want that trade on this column, because the failure mode it replaces is silent. As a general rule for anything with a security dimension: when a read path has two branches and one of them hands back the unprotected version, the branch is the vulnerability, no matter how carefully the condition is written.

The key identifier makes rotation real

The previous post admitted, in as many words, that key rotation was not implemented and probably never would be, because rotating means holding two keys at once and the ciphertext carried no way to say which one produced it.

Once the identifier is in the envelope, rotation stops being a flag day and becomes three ordinary steps: promote a new active key while the old one stays in the keyring as read-only, backfill in the background, then drop the old key. Nothing has to happen atomically, and the day a key is retired too early, the failure is a message naming the missing key identifier instead of an authentication error you have to reverse-engineer.

/** @throws \RuntimeException when a stored value names a key we no longer hold */
public function get(string $id): TokenCipher
{
    return $this->ciphers[$id]
        ?? throw new \RuntimeException(\sprintf('This value was encrypted with key "%s", which is not in the keyring.', $id));
}

Note what happened there: a format decision bought back a capability. Seven bytes plus an identifier turned "we cannot rotate keys" into "we can rotate keys during business hours". That is the usual shape of this trade, and it is why the version prefix is not speculative generality. It does not anticipate a feature, it preserves the right to have one.

One detail worth stealing: the identifier that comes out of the database is validated against /^[A-Za-z0-9_-]{1,32}$/ before it is used or logged. It is attacker-influenced data on the day the threat model says the database is the thing you lost, and it ends up in exception messages.

The backfill does not guess either

The conversion command reads pre-envelope values by attempting decryption first, with every key in the keyring, and only treating the value as plain JSON if all of them fail:

foreach (EncryptedJsonType::keyring()->all() as $cipher) {
    try {
        return $this->decode($cipher->decrypt($stored));
    } catch (\Throwable) {
        // Not this key, or not a ciphertext at all: try the next one.
    }
}

// Nothing decrypted it: it can only be a value from before encryption.
try {
    return $this->decode($stored);
} catch (\Throwable) {
    return null;
}

This ordering is not stylistic. Base64-decoding {"name":"Ada"} fails outright, so plaintext can never be mistaken for ciphertext, while the reverse test (does it look like JSON?) has no such guarantee. When you must disambiguate, prefer the test that fails for structural reasons over the test that inspects appearances.

And because the type now refuses anything unenveloped, the same command doubles as the answer to "is the database fully converted?":

bin/console app:encrypt-buyers --dry-run

exits non-zero as long as one row is outside the envelope. That is a cron job and an alert, rather than a paragraph in a runbook that nobody reads after the deploy.

Where else the seven bytes apply

Encryption made this vivid, but the column was not special. Anything that outlives the process that wrote it is a format, and deserves a version:

Serialized columns. A json column holding a value object is a format the moment the object gains or loses a field. Version it, or you will be reading array_key_exists archaeology in two years.
Cache payloads. On disk, in Redis, anywhere. The classic failure is a deploy that changes a cached structure and reads yesterday's shape for the ten minutes before the cache turns over. A version in the key is the cheapest variant: old shape, old key, no reader ever sees it.
Queue messages. During a rolling deploy, two versions of the consumer run at once, by design. The message on the wire is a format shared between two codebases that happen to be the same repository at different commits.
Public identifiers. Anything you print on an invoice, put in a URL, or let a customer paste into a support ticket has escaped your control permanently. A one-character prefix (inv_, cus_, the whole Stripe convention) is the same seven bytes wearing a different hat, and it also makes a leaked identifier greppable.
Exported files. A CSV whose column order is the contract, because someone built a spreadsheet on top of it.

The test I now apply, and it takes one second: will this value be read by code I have not written yet? If yes, it needs to say what it is. Not because I expect to change it, but because "I will never need to change this" is a prediction about the future, and the version prefix is the cheapest insurance ever written against being wrong about one.

Is this not YAGNI?

The objection deserves a straight answer, because "you are not going to need it" is usually right and I use it constantly.

YAGNI is a rule about code, and it works because code is deletable. Write the abstraction you do not need, discover it, delete it: the cost of being wrong is bounded and it is paid by one person in one afternoon.

Formats break the assumption the rule stands on. You cannot delete a format that is already in the database, because you do not own all of its copies: some are in backups, some in a partner's system, some in a CSV a customer downloaded last March. The cost of being wrong is unbounded and it is paid by whoever is on call. When the cost of being wrong is asymmetric by three orders of magnitude, the cheap option is not the smaller diff.

What it actually cost

Eighty lines, tests included, and two hours. A prefix constant, a small class that wraps and splits the envelope, a keyring with one active key and any number of retired ones, a read path that throws instead of falling back, and a backfill command that tries decryption before it tries trust.

The part I keep thinking about is that none of it came out of a threat model, a checklist, or a review. It came from someone reading a public post carefully enough to notice that a condition was doing more work than it looked like. If you write technical posts and treat the comments as the part that comes after the work, that is backwards: the objection is the work, and it is the only free code review you will ever be offered.

The code above lives in InvoiceAnvil, a Shopify app that issues compliant invoices, built on ShipAnvil, the Symfony kit that ships the auth, billing, admin and deploy pipeline these apps stand on. The post that started this one is encrypting personal data at rest in Symfony, and the foundations are in deploying Symfony to a production VPS.

https://shipanvil.com/blog/storage-format-cannot-refactor

Six decisions behind a shared-expense app, and the one where you must refuse to round

Eric Mollenthiel — Thu, 13 Aug 2026 13:00:39 +0000

Splitting one bill is a division. Settling a group of five people over three weeks of shared expenses is a different problem, and it is the one that actually has edge cases.

I have been building Kotisso, a shared expense tracker. The money side of it turned out to be far less about UI than about a handful of decisions that are cheap on day one and expensive to retrofit. This is the list I wish I had started from.

(One decision is missing here because it already has its own post: who gets the indivisible cent when you split 10 EUR three ways. That is the largest remainder method and the tiebreak everyone forgets. Everything below assumes you got that part right.)

The invariant that makes everything testable

Start from one property and refuse to break it:

In any group, the sum of all balances is always exactly zero.

A balance is what a person is owed. It is paid - owed + sent - received: what they advanced, minus what is theirs to bear, plus reimbursements they sent, minus reimbursements they received. Every unit of money entering a group is advanced by exactly one person and borne by one or more people, so the positives and the negatives cancel.

That single line gives you a free assertion after every write:

public function isConsistent(array $balances): bool
{
    $sum = 0;
    foreach ($balances as $balance) {
        $sum += $balance->getBalanceCents();
    }

    return 0 === $sum;
}

Every accounting bug I hit in the first month showed up as a non-zero sum before it ever showed up on a screen. A share attached to the wrong member, a deleted expense whose shares survived, a settlement counted on one side only: all of them break the invariant, none of them break the layout. That is why you find them with an assertion and not with your eyes.

Store hundredths even for currencies that have none

Integer cents, never floats. That part is well known. The part that is not:

1500 JPY is stored as 150000. The yen has no decimal places, so this looks like pure waste. It is what lets a single integer travel through the splitter, the balance calculator, the settlement planner, the CSV exporter and the PDF renderer without any of them needing to know which currency it carries. Exactly one place in the codebase knows that JPY has zero decimals: the formatter at the very end, and it learns it from ICU rather than from a table I maintain.

The alternative, letting the scale vary with the currency, means every arithmetic site becomes currency-aware. You will miss one.

Turning balances into transfers

You have balances that sum to zero. You want a small number of bank transfers that zeroes everybody out.

The exact minimum is NP-hard: it reduces to partitioning debtors into subsets that exactly match creditors. It is also irrelevant at these sizes. What people want is a plan that is short and obviously correct, and greedy gives that with a bound you can explain in one sentence.

// Both lists hold positive cents, sorted descending,
// with the member id as tiebreak so the plan is reproducible.
$transfers = [];
$i = 0;
$j = 0;

while ($i < count($debtors) && $j < count($creditors)) {
    $amount = min($debtors[$i]['cents'], $creditors[$j]['cents']);

    if ($amount > 0) {
        $transfers[] = new Transfer($debtors[$i]['member'], $creditors[$j]['member'], $amount);
        $debtors[$i]['cents'] -= $amount;
        $creditors[$j]['cents'] -= $amount;
    }

    if (0 === $debtors[$i]['cents'])   { $i++; }
    if (0 === $creditors[$j]['cents']) { $j++; }
}

Each iteration settles at least one person completely, so n members need at most n - 1 transfers. A flatshare of four ends up with three payments instead of the six that "everybody pays everybody back" produces.

Two details worth keeping. The two ifs at the end are independent, not an if/else: when a debtor and a creditor match exactly, both are finished, and advancing only one index leaves a zero-cent entry the loop then has to step over. And the descending sort needs a deterministic tiebreak, or two calls on the same data propose two different plans and the user watches the suggestion move around under them.

The mode where you must refuse rather than fix

Four split modes: equally, by shares, by percentage, and by exact amounts. The first three are proportional. The fourth is a trap.

In exact-amounts mode the user types each person's share. If those shares do not add up to the expense total, the tempting move is to absorb the difference into the largest one. My code did that for a while. Then I did the arithmetic on a realistic case: a 240 EUR restaurant bill entered as exact amounts that fall 15 EUR short, and one person silently pays 15 EUR more than they agreed to, with nothing on screen saying so.

Now it returns 422 and names the gap. Whoever does not want to count to the cent has three other modes.

A rounding rule is acceptable when nobody chose the numbers. It is not acceptable when somebody did.

Freeze the exchange rate at entry

You are on holiday, you pay 1 200 THB, the group counts in euros. The obvious implementation calls a rate API.

Do not. A live rate means every balance in the group drifts every morning, so a settled trip un-settles itself and a debt someone already paid comes back at a different number. Store what was entered, the currency, and the rate that was applied, then convert once and never again. The card's real rate is never the ECB's anyway, so the field has to be editable regardless.

The downstream consequence is the good kind: amountCents holds the amount in the group's currency and nothing below the entry point knows a conversion ever happened. Balances, the invariant, the settlement plan, the exports, none of them were touched when multi-currency shipped.

A pending expense counts nowhere

Groups can require expenses to be approved by the people they concern. An expense that is awaiting approval, or disputed, is excluded entirely: from the balances, from the group total, from the per-category breakdown.

The tempting alternative is to count it as provisional and mark it visually. That keeps the invariant intact and makes every number on the page mean something slightly different from what it says. Excluding it in one place, at the top of the balance loop, is one continue and no ambiguity:

foreach ($group->getExpenses() as $expense) {
    if (!$expense->getStatus()->countsInBalances()) {
        continue;
    }
    // ...
}

And one thing that is not about money at all

A free plan shows fewer rows, never different numbers. Expenses outside the visible window still count in the balances, in the group total and in the category breakdown, and the export says in the file itself what it summarised rather than dropping it.

A truncated statement is a false statement, and an accounting tool that produces one is worth nothing. That rule cost more design time than the algorithms did.

The order that worked

Invariant first, integer cents second, splitter third, screens last. Each of those is a few hours at the start and a data migration later.

The long version of the balance calculation, written for people rather than developers, is here.

A week is seven nights, not 168 hours: dates in a shared-custody calendar

Eric Mollenthiel — Wed, 12 Aug 2026 16:51:44 +0000

Add seven days to a timestamp and you get the same day next week. That holds right up until the last Sunday in October, when it quietly stops holding and nothing tells you.

I have been building Nestido, a planner for separated parents: who has the children, when, and where. The family declares its custody rhythm once (alternating weeks, 2-2-3, 5-2-2-5), the app predicts the calendar from it, and the parents only record the deviations. Symfony 8, PostgreSQL, one server.

From the outside it is a calendar with colours. From the inside it is date arithmetic with a domain that punishes you twice a year, in a way no user will ever report. Nobody files a bug that says "your daylight saving handling is off". They see a number that looks wrong, they say nothing, and they stop trusting the app.

Here are the five rules I ended up with. Every one of them exists because the obvious version was wrong.

1. The unit is the night, and a night is not a duration

The first instinct is to store intervals and divide by 24. It is wrong in both directions, and you can prove it without leaving your own timezone.

In Paris, 1 June 23:30 to 2 June 22:30 is twenty-three hours and it is one night. 1 June 00:30 to 1 June 23:30 is also twenty-three hours and it is zero nights: nobody slept anywhere new. Identical durations, different answers.

So the count is not a duration at all, it is a number of local midnights crossed:

public function nights(\DateTimeZone $zone): int
{
    $from = $this->startsAt->setTimezone($zone)->setTime(0, 0);
    $to   = $this->endsAt->setTimezone($zone)->setTime(0, 0);

    return (int) $from->diff($to)->days;
}

Truncate both ends to local midnight, then diff. days already accounts for the transitions, so the two cases that break the hour-based version come out right:

28 March 20:00 to 30 March 08:00 in Paris is 35 hours (the clocks jump forward). Two nights.
24 October 20:00 to 26 October 08:00 is 37 hours (the clocks fall back). Two nights.

Divide by 24 and you get 1 and 1. In spring you have just deleted a night from every family in the country, and the statistics screen, which is the whole reason the app exists, is quietly off by one for the rest of the year.

2. Do the arithmetic in the family's timezone, store the result in UTC

Everything in my database is UTC. That is the easy half, and it lulls you into doing the arithmetic in UTC too. That part is wrong.

A handover happens at 18:00 local. It happens at 18:00 in July and at 18:00 in November. If you add seven days to a UTC instant, you get the same UTC instant one week later, which is a different wall clock time on the other side of a transition:

$start = new DateTimeImmutable('2026-03-25 18:00', new DateTimeZone('Europe/Paris'));

$start->setTimezone(new DateTimeZone('UTC'))
      ->modify('+7 days');          // 1 April, 19:00 local. One hour late, forever.

The rule that survived contact with the domain: cycle boundaries are computed on local dates at the local switch time, and converted to UTC only at the very end.

private function boundary(\DateTimeImmutable $day): \DateTimeImmutable
{
    return $day
        ->setTime(intdiv($this->switchMinutes, 60), $this->switchMinutes % 60)
        ->setTimezone(new \DateTimeZone('UTC'));
}

The test that guards it is the clearest statement of the whole idea. Two consecutive weekly boundaries around the October change, stored as UTC instants:

self::assertSame('2026-10-23 16:00', $segments[0]->start->format('Y-m-d H:i'));
self::assertSame('2026-10-30 17:00', $segments[1]->start->format('Y-m-d H:i'));

Different UTC instants, one hour apart. Both are 18:00 in Paris. If your two boundaries have the same UTC time on either side of a transition, your handover has drifted, and it will drift again in March.

3. Never walk a calendar from midnight. Walk from noon.

To render a month you loop over days. The obvious loop starts at local midnight and adds one day at a time. It works everywhere in Europe and North America, which is exactly why it is dangerous: in some zones midnight does not exist.

Chile moves its clocks at midnight. So do a handful of others, and Brazil did until 2019. On the transition night, 00:00 is not a valid local time and PHP moves you to 01:00. From then on, the loop keeps that hour:

$c = new DateTimeImmutable('2026-09-04 00:00', new DateTimeZone('America/Santiago'));
for ($i = 0; $i < 5; $i++) {
    echo $c->format('Y-m-d H:i P'), "\n";
    $c = $c->modify('+1 day');
}

2026-09-04 00:00 -04:00
2026-09-05 00:00 -04:00
2026-09-06 01:00 -03:00   <- midnight does not exist here
2026-09-07 01:00 -03:00   <- and it never comes back
2026-09-08 01:00 -03:00

Every day after the transition now starts an hour late. A night recorded at 00:30 falls outside the day it belongs to, and the calendar cell for that day is empty while the previous one holds two.

The fix is one line and it is the least obvious line in the codebase:

$noon = $firstDay->setTimezone($zone)->setTime(12, 0);

for ($i = 0; $i < $length; $i++) {
    $days[] = $noon->modify('+' . $i . ' days')->setTime(0, 0);
}

Walk from noon, offset from a fixed origin rather than accumulating, then drop to midnight at the end. Noon is never skipped by a transition, offsetting from the origin means an error cannot compound, and the final setTime(0, 0) renormalises to a midnight that actually exists. The same trick guards the cycle anchor, for the same reason.

4. A date is not an instant, and the type system will not tell you

Three fields in the custody rule are days, not moments: the anchor date, the start date, the end date. They are Y-m-d, they have no time, and giving them one is how you lose a day.

They are stored at UTC midnight and read back by their Y-m-d only. In Symfony forms, the date field carries both model_timezone and view_timezone set to UTC:

->add('anchorDate', DateType::class, [
    'model_timezone' => 'UTC',
    'view_timezone'  => 'UTC',
])

Set only one and a plain round trip through the form, with no edit at all, walks the date back by a day for anyone west of Greenwich. Save the form twice, lose two days, and the whole predicted calendar shifts under a family that changed nothing.

The handover time gets the same treatment from the other direction: it is a smallint of minutes since midnight, not a time column. 18:00 is 1080. It is not an instant, it has no date, and there is no timezone to attach to it. A time column invites exactly the conversion that must never happen.

5. The one line that silently poisons all of the above

None of this holds if PHP disagrees with you about what timezone the database is speaking.

Doctrine's datetime_immutable type reads a timestamp column with PHP's default timezone. My server runs Europe/Paris. So a value written as UTC comes back as a Paris wall clock reading of the same digits, which is a different instant:

date_default_timezone_set('Europe/Paris');
$v = DateTimeImmutable::createFromFormat('Y-m-d H:i:s', '2026-08-12 23:30:00');

$v->format('Y-m-d H:i P');                          // 2026-08-12 23:30 +02:00
$v->setTimezone(new DateTimeZone('UTC'))->format(); // 2026-08-12 21:30 +00:00

Two hours early, on every single row, in summer only. A handover recorded at 00:30 UTC is read as 22:30 UTC the day before: the night lands on the wrong day and is credited to the wrong parent. Nothing throws. The tests pass, because the test environment happened to agree with itself.

So the kernel says it out loud, before anything else boots:

public function __construct(string $environment, bool $debug)
{
    date_default_timezone_set('UTC');

    parent::__construct($environment, $debug);
}

Display is the only place that leaves UTC, and it leaves for the family's timezone, not the server's.

How I know any of this holds

The three services that do the counting (the cycle, the timeline, the statistics) know nothing about Doctrine, entities or repositories. They take a list of durations, an anchor date, a switch time, a timezone, and they return values. No fixtures, no database, no HTTP.

That is not architectural taste. It is what makes the two properties worth asserting cheap enough to assert everywhere:

A week across a transition is still seven nights, and the handover is still 18:00 on both sides. One test, both directions, spring and autumn.
Regenerating the plan twice produces exactly the same plan. Cycle segments are always generated whole, never clipped to "now", so nothing shrinks by a sliver on each pass. The day this stops being true, the predicted calendar starts moving on its own.

An off-by-one in a date library shows up as a stack trace. An off-by-one in a custody calendar shows up as a parent driving to a school where nobody is waiting. Same bug class, very different failure mode, and it is the reason all of the above got written down instead of remembered.

The app is at nestido.com, and the reasoning behind the night as the unit of account is written up for parents rather than for developers in counting overnights without an argument.

Cloudflare was 403-ing ChatGPT, Perplexity and Claude on my site, and my logs never knew

Eric Mollenthiel — Sun, 09 Aug 2026 05:50:43 +0000

For three weeks I wrote content aimed squarely at answer engines. An llms.txt,
FAQPage JSON-LD on two pages, a comparison page built to be quotable, a "how it
works" page with HowTo markup. The site is a small dating app I run on my own
server, loviam.com, Symfony and PostgreSQL on a single
box behind Cloudflare.

Referrals stayed at zero. The only external referrer my analytics table had ever
recorded, over its whole history, was chatgpt.com: three visits, one day in
July, never seen again.

I assumed the content was not good enough. The robots had never read it.

The five minute probe that should have been the first thing I did

Nothing in my usual instruments could see the problem, so I stopped looking at
them and asked the site directly, once per user agent, at its real public
address:

for ua in \
  "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  "Mozilla/5.0 (compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)" \
  "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" \
  "Mozilla/5.0 (compatible; Claude-SearchBot/1.0; +claudebot@anthropic.com)" ; do
  code=$(curl -s -o /dev/null -w '%{http_code}' -A "$ua" https://www.loviam.com/)
  echo "$code  $ua"
done

200  Googlebot
403  OAI-SearchBot
403  PerplexityBot
403  Claude-SearchBot

Same result for ChatGPT-User, Claude-User and MistralAI-User. Meanwhile
bingbot, DuckDuckBot, Applebot, YandexBot and an ordinary browser all
got a 200.

The 403 body was eight bytes long, blocked., with server: cloudflare in the
headers, and there was no matching line in the Apache access log. The refusal
happened at the edge. My origin never heard about it.

Why every instrument I had was blind to this

This is the part worth stealing, because the failure mode generalises.

Application logs cannot record a request that never arrives. Obvious once
said, easy to forget when you are grepping the access log for PerplexityBot
and concluding "it never came".

First party analytics is worse than blind, it is reassuring. My beacon fires
from the page. A robot that gets a 403 never gets the page, so it never appears,
so the dashboard looks exactly like "nobody is interested".

My deployment smoke tests talked to the wrong server. They run with
curl --resolve www.loviam.com:443:127.0.0.1, which is the right call for a
deployment gate: you want to test the code you just shipped, not the CDN cache.
But it means the whole test suite speaks to Apache directly and Cloudflare is
invisible to it by construction. Thirty-eight green checks, every deploy, on a
path no visitor uses.

And the managed robots.txt actively pointed the wrong way. Cloudflare
injects its own block above yours. Mine named GPTBot, ClaudeBot, CCBot,
Bytespider, Amazonbot, meta-externalagent, Google-Extended and
Applebot-Extended, all of them training crawlers. Blocking those is a
defensible editorial decision and I stand by it.

But the HTTP filter was blocking something else: the search and on-demand
reading robots, which were not named in that file at all. So a perfectly
obedient crawler read "you are allowed", requested the page, and got a 403. That
mismatch between the two settings is the tell. It is not a policy, it is a
misconfiguration, and reading either file alone will never show it to you.

The fix is one toggle, and it is not the one named after crawlers

Cloudflare dashboard, zone level: Security → Settings → Block AI bots.

Not "AI Crawl Control", which is where I looked first and where the interesting
per-crawler table lives. The switch that returns the 403 is the plain one in
Security Settings, and it treats "AI bot" as a single category: the crawler that
builds ChatGPT's search index and the crawler that scrapes you for training data
are the same thing to it, even though for a publisher they are close to
opposites. One sends you traffic. The other does not.

Also worth checking afterwards: the managed robots.txt block, so it does not
still name a robot you have just decided to let through.

The guard I put in, and why it is a timer and not a test

What broke here is not my code. It is an edge configuration that can change
without a deploy, from a dashboard, possibly by a provider default I never
chose. A test in CI would only prove it was fine at build time.

So it is a daily command instead, app:seo:check-crawlers, which:

probes the real public URL, through Cloudflare, never --resolve;
walks sixteen user agents in three families: search, answer engines, training;
checks two independent things per robot, the actual HTTP status and whether the served robots.txt names it in a Disallow, because those two disagreeing is the exact signature I missed;
keeps an ordinary browser user agent as a control, so that "the robots are blocked" is never confused with "the site is down";
fails for search and answer engines, and never for training crawlers, which stay blocked on purpose.

A systemd timer runs it every morning and mails me only on state change.
An alert that fires every day is an alert nobody reads.

The second thing Cloudflare was doing, which was worse

While I was in there, I checked what else the proxy changed about requests, and
found the site had been taking itself down for weeks.

Every request reached Apache carrying the IP of a Cloudflare relay.
mod_remoteip was loaded but never configured, so nothing restored the real
client address:

172.71.135.63 - - [25/Jul/2026:12:21:03 +0000] "GET /fr/ HTTP/2.0" 200 10399

fail2ban reads those logs. So fail2ban was banning Cloudflare's own relays.
Cloudflare could then no longer reach my origin, and served HTTP 521 to every
visitor routed through that relay. The correlation was exact, to the second:

18/07 14:50:39  fail2ban  [apache-auth] Ban 141.101.98.192   (a Cloudflare range)
18/07 14:50:39  uptime probe: DOWN (HTTP 521)
18/07 15:49     unban cascade
18/07 15:54:02  uptime probe: RECOVERED

Five episodes in one week, the longest 42 minutes. My apache-auth jail triggers
at maxretry = 2, and my staging host sits behind htpasswd and therefore
manufactures 401s: two fat-fingered logins behind a shared relay were enough to
take production down for everyone else behind it.

Two quieter casualties of the same root cause: every rate limiter keyed on
$request->getClientIp() was bucketing the entire planet into a handful of
relays, and the visitor hash behind my "unique visitors" number was hashing the
relay, not the visitor. The weekly figure I had been reading to judge growth was
not counting what I thought it counted.

The fix is to trust CF-Connecting-IP, but only when the request comes from a
published Cloudflare range. Without that restriction anyone could forge the
header and walk straight past your rate limiting and your bans. Second layer, as
a net: put those same ranges in fail2ban's ignoreip, so that if the first layer
ever stops working you ban nobody rather than banning your own CDN. Both files
are generated by a monthly script, because the ranges do move, and a frozen list
would quietly reopen the trap.

What I would tell myself in June

If something sits between your users and your server, test through it, with
the user agent of the thing you care about. Not from your laptop, not with
--resolve, not from the logs. The whole class of bug here is that the failure
happens in a place none of your instruments can observe, and every instrument
you own will report a calm, plausible, completely wrong "nobody came".

Splitting €10 three ways: the largest remainder method, and the tiebreak everyone forgets

Eric Mollenthiel — Fri, 07 Aug 2026 23:44:05 +0000

Split €10 three ways and you get €3.33, €3.33, €3.33. That is €9.99. One cent has gone missing, and you now have to decide, in code, who pays it.

It sounds like a rounding detail. It is not: it is the difference between an app whose numbers close and an app whose numbers almost close. I hit it while building an expense-sharing app, and the fix turned out to be a voting-theory algorithm from the 1790s.

The three wrong answers

Round each share and hope. round(1000 / 3) = 333 per person, 999 total. You are one cent short of the expense. Every balance downstream inherits that error, and it compounds: fifty three-way expenses and the group's books are off by fifty cents with no line item to point at.

Give the remainder to the largest share. This is the common fix, and it is fine at €10. It is not fine when the split is uneven. If someone entered exact amounts that do not add up to the total, "absorb the difference on the biggest share" silently decides that one person pays €15 more than they typed. Three cents of rounding go unnoticed. Fifteen euros go unnoticed too, right up until someone checks.

Use floats. 0.1 + 0.2 != 0.3. You know this. The whole domain is integer cents or nothing.

The right answer is a 1792 apportionment method

The problem (divide a whole number of indivisible units proportionally to weights) is the same problem as allocating seats in a parliament to parties by vote share. Alexander Hamilton proposed a solution for the US House of Representatives in 1792. It is called the largest remainder method, and it is three steps:

Compute each participant's exact (fractional) share.
Give everyone the floor of it.
Hand the leftover units, one each, to whoever has the largest fractional remainder.

For €10 among three people: exact share is 333.33 cents each, floor is 333, allocated is 999, one cent left over. All three remainders are 0.33, so one of them gets the extra cent: 334 / 333 / 333.

The sum is exactly 1000. Always. Not approximately.

Here is the core of it, in PHP, working entirely in integer cents:

/**
 * @param array<int, float> $weights  participant id => weight
 * @return array<int, int>            participant id => cents
 */
private function prorate(int $totalCents, array $weights): array
{
    $sum = array_sum($weights);
    if ($sum <= 0) {
        throw new \InvalidArgumentException('No shares to split.');
    }

    $amounts = [];
    $remainders = [];
    $allocated = 0;

    foreach ($weights as $id => $weight) {
        $exact = $totalCents * $weight / $sum;
        $floor = (int) floor($exact);

        $amounts[$id] = $floor;
        $remainders[$id] = $exact - $floor;
        $allocated += $floor;
    }

    $left = $totalCents - $allocated;

    if ($left > 0) {
        $order = array_keys($remainders);
        usort($order, static function ($a, $b) use ($remainders, $weights) {
            $cmp = $remainders[$b] <=> $remainders[$a];
            if ($cmp !== 0) {
                return $cmp;
            }
            $cmp = $weights[$b] <=> $weights[$a];

            return $cmp !== 0 ? $cmp : ($a <=> $b);
        });

        foreach (array_slice($order, 0, $left) as $id) {
            ++$amounts[$id];
        }
    }

    return $amounts;
}

$left is bounded by the number of participants minus one, so this is never more than a handful of increments.

The tiebreak is the part people skip

Look at the usort comparator. It does not stop at comparing remainders. When two remainders are equal, which is exactly what happens in the €10-among-three case and is the case you will hit most often, it falls through to the weight, and then to the participant id.

Without that fallback you have a non-deterministic split. PHP's usort is not stable across all inputs, and even a stable sort leaves you at the mercy of insertion order. The same expense, recalculated after an edit, can hand the cent to someone else. Balances shift by a cent for no visible reason. Someone notices, does not trust the app any more, and they are right not to.

So the rule is: the tiebreak chain must terminate in something total and immutable. The id works. "Whoever was added to the group first" works. "Whatever order the hash table gave me" does not.

What this buys you: an invariant you can assert

Once every split sums exactly to its expense, a much stronger property falls out of the model for free. Each participant's balance is:

balance = what they paid - what they owe + what they sent - what they received

Sum that across every member of a group and every term cancels: every euro paid is owed by someone, every transfer sent is received. The balances of a group always sum to exactly zero.

That is not a nice-to-have, it is a test oracle. It turns "did I get the money maths right" into a single assertion you can run after every operation:

public function testBalancesAlwaysSumToZero(): void
{
    $balances = $this->calculator->forGroup($group);

    self::assertSame(0, array_sum(array_map(
        static fn (Balance $b) => $b->cents(),
        $balances,
    )));
}

Any bug that loses or invents a cent anywhere trips this: a bad split, a mishandled refund, a currency conversion, a deleted participant. It is the cheapest high-value test in the codebase, and it only exists because the splitter is exact.

Three edge cases worth stealing

Negative totals. Refunds and corrections are negative expenses. floor(-333.33) is -334, not -333, so the remainder logic inverts and you over-allocate. Take the absolute value, split that, negate at the end. Two lines, and it stops a whole category of sign bugs.

Exact amounts should refuse, not repair. If a mode lets people type each share by hand and the total does not match, do not silently fix it. Reject the input and name the gap: "you entered €85.00, the expense is €100.00, €15.00 missing". Someone who does not want to do the arithmetic has the other modes. Someone who does want to do it deserves to be told they got it wrong rather than have it quietly rewritten.

Zero-decimal currencies. Store everything in hundredths regardless. ¥1,500 is 150000. Then one integer travels the entire calculation without ever needing to know what currency it is, and formatting stays a presentation concern where it belongs.

Where this came from

I ran into all of this building Kotisso, a shared-expense tracker for flatshares, group holidays and separated parents. The zero-sum invariant is the whole design: everything else in the app is arranged so that it cannot be violated.

The largest remainder method is old, well-studied, and takes about thirty lines. If you are dividing indivisible units by proportion anywhere (money, seats, inventory, rate limits) it is probably the algorithm you want, and the tiebreak is probably the part you are about to forget.

Encrypting personal data at rest in Symfony: a 60-line Doctrine type, and the four things it breaks

Eric Mollenthiel — Wed, 05 Aug 2026 11:27:59 +0000

A review form asked me a yes/no question I could not answer with yes: do you encrypt personal data at rest and in transit?

In transit, sure, TLS everywhere, that part is free in 2026. At rest, no. The app runs on a plain VPS with no full-disk encryption, and the database held customer names, addresses and email addresses in a json column, in clear, exactly as PostgreSQL received them.

There are two ways out of that question. One is to click yes and move on, because everybody knows the reviewer will not ssh into your box. The other is to make the answer true. This post is the second one, taken from InvoicePilot, a Shopify app that issues invoices and therefore stores, by definition, the buyer's identity.

The encryption itself is the easy part, and it is about sixty lines. What follows it is the interesting part: the moment a column becomes an opaque blob, four things you took for granted stop working, and two of them will not fail loudly.

Why application-level encryption at all

The usual objection is fair: if an attacker gets your database, they probably have your application server too, and the key lives there. So what is the point?

The point is that "the attacker has your whole box" is one threat among several, and the cheaper ones are more common:

a database dump copied to a laptop, a CI job, or a support ticket;
a backup file on object storage with the wrong ACL;
a SELECT run by a future you at 2am, against production, with the output scrolling into a terminal buffer that gets pasted somewhere;
a restored snapshot handed to a contractor for debugging.

Application-level encryption does nothing against a full host compromise and everything against those four. That is a real gain, and it is the gain the compliance question is actually asking about. Nothing here protects you from yourself if you keep the key in the same dump as the data, which is the mistake at the end of this post.

Full-disk encryption solves a different problem entirely: a stolen physical drive. On a rented VM, where the disk is a network volume you do not control and the machine boots unattended, it protects roughly nothing that matters. So on a VPS, column-level encryption is not the cheap approximation of disk encryption. It is the one that fits the threat.

The cipher: libsodium, no bundle

PHP has had libsodium in core since 7.2. A secretbox is authenticated encryption (XSalsa20 for confidentiality, Poly1305 for integrity), it takes a 32-byte key and a nonce, and there is no way to hold it wrong except reusing a nonce, which random_bytes() makes a non-issue.

final readonly class TokenCipher
{
    private string $key;

    public function __construct(string $base64Key)
    {
        $key = base64_decode($base64Key, true);
        if (false === $key || \SODIUM_CRYPTO_SECRETBOX_KEYBYTES !== \strlen($key)) {
            throw new \InvalidArgumentException(
                'The encryption key must be 32 random bytes, base64-encoded.'
            );
        }
        $this->key = $key;
    }

    public function encrypt(string $plaintext): string
    {
        $nonce = random_bytes(\SODIUM_CRYPTO_SECRETBOX_NONCEBYTES);

        return base64_encode($nonce.sodium_crypto_secretbox($plaintext, $nonce, $this->key));
    }

    public function decrypt(string $encoded): string
    {
        $raw = base64_decode($encoded, true);
        if (false === $raw || \strlen($raw) <= \SODIUM_CRYPTO_SECRETBOX_NONCEBYTES) {
            throw new \RuntimeException('Invalid ciphertext.');
        }

        $nonce = substr($raw, 0, \SODIUM_CRYPTO_SECRETBOX_NONCEBYTES);
        $plaintext = sodium_crypto_secretbox_open(
            substr($raw, \SODIUM_CRYPTO_SECRETBOX_NONCEBYTES),
            $nonce,
            $this->key,
        );

        if (false === $plaintext) {
            throw new \RuntimeException('Decryption failed (wrong key or corrupted data).');
        }

        return $plaintext;
    }
}

Two details worth stating out loud, because they are what people get wrong when they roll this by hand:

The nonce travels with the ciphertext. It is not a secret, it must just never repeat under the same key. Prepending 24 random bytes and slicing them off on the way back means you never have to store or manage a nonce column.

sodium_crypto_secretbox_open() returning false is not a decoding error, it is a failed authentication. The value was truncated, corrupted or tampered with. Throwing there is the whole point of using an AEAD construction instead of raw openssl_encrypt with a mode you picked from a Stack Overflow answer.

I generate the key with one line and put it in the environment, next to the database URL:

php -r "echo base64_encode(random_bytes(32)), PHP_EOL;"

The Doctrine type

Now the part that makes it disappear from the rest of the codebase. Doctrine's custom types sit exactly at the boundary you want: one hook on the way to the database, one on the way back.

final class EncryptedJsonType extends Type
{
    public const string NAME = 'encrypted_json';

    private static ?TokenCipher $cipher = null;

    /** Test seam: lets a test drive the type without touching the environment. */
    public static function setCipher(?TokenCipher $cipher): void
    {
        self::$cipher = $cipher;
    }

    public function getName(): string
    {
        return self::NAME;
    }

    public function getSQLDeclaration(array $column, AbstractPlatform $platform): string
    {
        return $platform->getClobTypeDeclarationSQL($column);
    }

    public function convertToDatabaseValue($value, AbstractPlatform $platform): ?string
    {
        if (null === $value) {
            return null;
        }

        return $this->cipher()->encrypt(json_encode($value, \JSON_THROW_ON_ERROR));
    }

    public function convertToPHPValue($value, AbstractPlatform $platform): ?array
    {
        if (!\is_string($value) || '' === $value) {
            return null;
        }

        // Rows written before the switch are still plain JSON.
        $json = ('{' === $value[0] || '[' === $value[0])
            ? $value
            : $this->cipher()->decrypt($value);

        $decoded = json_decode($json, true, 512, \JSON_THROW_ON_ERROR);
        if (!\is_array($decoded)) {
            throw new \RuntimeException('An encrypted JSON column must hold an object.');
        }

        return $decoded;
    }

    private function cipher(): TokenCipher
    {
        // Doctrine builds its types statically, outside the container: read the
        // key the way Dotenv exposes it to every process (bin/console, FPM,
        // the messenger worker and migrations alike).
        $key = $_SERVER['ENCRYPTION_KEY'] ?? $_ENV['ENCRYPTION_KEY'] ?? '';

        return self::$cipher ??= new TokenCipher(\is_string($key) ? $key : '');
    }
}

Registered once:

doctrine:
    dbal:
        types:
            encrypted_json: App\Doctrine\EncryptedJsonType

And used like any other type, which is the whole payoff. Everything above the entity, the repositories, the controllers, the PDF renderer, the tests, keeps handling a plain PHP array:

/** @var array<string, mixed> */
#[ORM\Column(type: EncryptedJsonType::NAME)]
private array $buyer;

Three decisions in there deserve a sentence each.

The type reads $_SERVER directly, and that is not laziness. Doctrine instantiates types through a static registry, before and outside the service container. There is no constructor injection available, and there is no container to ask. Reading the same superglobal that Symfony's Dotenv component populates is what makes the type behave identically under FPM, bin/console, a Messenger worker and a migration, which is exactly the set of processes that will touch this column.

The static setter is a test seam, and it earns its keep. A unit test sets a fixed key, exercises the round trip, and asserts on the stored value:

public function testTheStoredValueLeaksNothing(): void
{
    $stored = (string) $this->type->convertToDatabaseValue(self::BUYER, $this->platform);

    self::assertStringNotContainsString('Claire', $stored);
    self::assertStringNotContainsString('example.com', $stored);
}

That second assertion is the one I would keep if I could only keep one. It is the test that fails the day someone "simplifies" the type back to a plain JSON column, and it is worth more than any amount of documentation saying please do not.

Reading tolerates plaintext, writing never produces it. A JSON document always starts with { or [; a base64 ciphertext, in practice, does not. That single-byte check buys a deploy where old rows and new rows coexist, and it is the difference between a migration you can run calmly and one that has to be atomic with the code deploy.

Update, 9 August 2026. A reader pushed back on that paragraph, and the objection holds: detecting plaintext by its first byte leaves the storage format ambiguous forever. A row written through another path, or edited by anyone holding a database write, opts out of decryption merely by looking like JSON. That is a permission check spelled as a guess. The fix is to record the format instead of inferring it: every value now reads enc:v1:<key id>:<base64>, anything else is refused, and the key id turns the "prefix a key id now" advice further down this post into an actual rotation path. The section The envelope, and why the first byte was a mistake, near the end, has the code and what it costs.

Migrating the rows you already have

Changing the column type is one line. Rewriting the existing rows is the migration, and Doctrine Migrations has the right hook for it: postUp() runs after the schema change, with the new type available.

public function up(Schema $schema): void
{
    $this->addSql('ALTER TABLE invoice ALTER buyer TYPE TEXT');
}

public function postUp(Schema $schema): void
{
    $type = Type::getType(EncryptedJsonType::NAME);
    $platform = $this->connection->getDatabasePlatform();

    foreach ($this->connection->iterateAssociative('SELECT id, buyer FROM invoice') as $row) {
        $buyer = $row['buyer'];
        if (!\is_string($buyer) || '' === $buyer || ('{' !== $buyer[0] && '[' !== $buyer[0])) {
            continue; // already encrypted
        }

        $this->connection->update(
            'invoice',
            ['buyer' => $type->convertToDatabaseValue(json_decode($buyer, true), $platform)],
            ['id' => $row['id']],
        );
    }
}

Note iterateAssociative() rather than fetchAllAssociative(): the migration streams, so the memory it uses does not depend on how many rows you have. And the same loop lives in a console command (app:encrypt-buyers), because the migration is not the only way a plaintext row can appear. A rollback to a previous release, or a dump restored from before the switch, both produce rows the migration will never see again. Having a command means the answer to "prove the database holds no readable personal data" is something you run, not something you believe.

Write the down() too, and make it decrypt. A migration you cannot reverse is a deploy you cannot abort.

What breaks, part 1: you cannot query it any more

This is the obvious one, and it is still the one that bites.

An encrypted column has no equality, no LIKE, no index, no ORDER BY, no GROUP BY. Two rows holding the same email produce two different ciphertexts, because the nonce differs. Every predicate you had on that data is gone, and the compiler will not tell you.

In my case exactly one query touched it, and it came from an obligation I could not drop: the GDPR data-request webhook, which must answer "what do you hold about this customer?" given an email address. It used to be a JSON predicate in SQL. It became a stream in PHP:

/**
 * The buyer block is encrypted at rest, so no SQL predicate can match on
 * it: the rows are streamed and compared in PHP (case-insensitively, as
 * an email is). This runs on the GDPR customer webhooks only, never on a
 * merchant-facing page.
 *
 * @return list<string>
 */
public function findNumbersByShopAndBuyerEmail(Shop $shop, string $email): array
{
    $connection = $this->getEntityManager()->getConnection();
    $type = Type::getType(EncryptedJsonType::NAME);
    $platform = $connection->getDatabasePlatform();
    $needle = mb_strtolower(trim($email));

    $numbers = [];
    $rows = $connection->iterateAssociative(
        'SELECT number, buyer FROM invoice WHERE shop_id = :shop ORDER BY number',
        ['shop' => $shop->getId()],
    );

    foreach ($rows as $row) {
        $buyer = $type->convertToPHPValue($row['buyer'], $platform);
        $buyerEmail = \is_array($buyer) ? ($buyer['email'] ?? null) : null;
        $number = $row['number'];

        if (\is_string($buyerEmail) && \is_string($number)
            && mb_strtolower(trim($buyerEmail)) === $needle) {
            $numbers[] = $number;
        }
    }

    return $numbers;
}

Yes, that is a full scan per tenant, decrypting as it goes. I am fine with it, and the docblock says why: it runs on a webhook that fires a handful of times a year, never on a page a merchant waits for. The scope is narrowed by shop_id, which is indexed, so the scan is over one tenant's rows, not the table.

The decision this really forces is upstream. Before you encrypt a column, list every query that reads it, and sort them into "rare and allowed to be slow" and "on a hot path". If anything lands in the second pile, you have a genuine design choice to make, and none of the options are free:

Deterministic encryption for that one field (same input, same ciphertext, no nonce): equality queries and unique indexes come back, and you leak which rows share a value, plus you become vulnerable to frequency analysis on low-cardinality fields.
A blind index: store hash(hmac_key, normalize(value)) in a second, indexed column, query on that, keep the real value encrypted. Equality only, but it is exact and fast. This is what I would build the day the lookup moves onto a hot path.
Do not encrypt that field. Encrypting the fields that carry identity and leaving a non-identifying one queryable is a defensible line, as long as you draw it deliberately and write down why.

What is not defensible is discovering the constraint after the migration, in production, because a page that used to filter now returns nothing.

What breaks, part 2: your logs become the plaintext copy

This one is quiet, and it is the one I would go looking for first in someone else's codebase.

The same compliance form that asks about encryption also asks for access logs: prove you know who read personal data and when. The obvious implementation logs the record. Do that and you have carefully encrypted a column while writing its contents, in clear, into a file that is rotated, shipped to a log aggregator, and backed up by a completely different policy.

So the audit trail names the actor and the object, never the data:

/**
 * The trail names the shop, the document and the channel, never the
 * personal data itself: a log file must not become a second,
 * unencrypted copy of what the database encrypts.
 */
public function documentRead(Invoice $invoice, string $channel, string $format): void
{
    $this->logger->info('Document read.', [
        'shop' => $invoice->getShop()->getShopDomain(),
        'document' => $invoice->getNumber(),
        'channel' => $channel, // 'admin' (the merchant) or 'buyer-link'
        'format' => $format,
    ]);
}

While you are there, check the three other places that leak the same way: exception context (an entity dumped into a stack trace), your error tracker's breadcrumbs, and the Symfony profiler in any environment where it is enabled against real data.

What breaks, part 3: the backup that carries the key

The failure I want to spell out, because it turns the whole exercise into theatre.

Encrypting a column protects a dump only if the dump does not also contain the key. Mine did not, the key lives in an environment file, not in PostgreSQL. But the backup itself was a plain pg_dump sitting on the same machine, and half of the app's personal data lives in columns that are not encrypted anyway, because they are the merchant's own data, not their customers'.

So the dumps are now encrypted too, streaming, with the key readable only by root and the postgres user:

pg_dump -Fc "$DB" \
  | zstd -T0 -q \
  | openssl enc -aes-256-ctr -pbkdf2 -pass file:/etc/pg-backup.key \
  > "$DEST/$DB.dump.zst.enc"

Two things to get right, and both are about the day you need it rather than the day you set it up. Test the restore path before you need it, including the decryption, because an encrypted backup you cannot open is worse than no backup: it looks like a backup in the monitoring. And write down where the key is and who can read it, somewhere that is not the encrypted backup, which sounds obvious right up to the moment you watch someone reason in a circle about it.

What breaks, part 4: key rotation, which you will not do

Be honest about this one from day one. With the design above, rotating the key means decrypting every row with the old key and re-encrypting with the new one, which means both keys have to exist at once, which means the cipher needs a key-id prefix in the ciphertext, which nothing above has.

I did not build that, and I decided so explicitly: one product, one key, rotation is a scripted maintenance window and I would rather do that once in five years than carry a key hierarchy from day one. If you are in a context where rotation is a scheduled control rather than an incident response, prefix a key id to the ciphertext now, when the column is empty. It is one byte of format and it costs nothing today.

The compensating control is smaller than key rotation and worth more than it looks: the key is in the environment, so a leaked dump never contains it, and a leaked repository never contains it either. What would force a rotation is a compromised host, and on a compromised host, rotating the key is the last item on a long list.

The envelope, and why the first byte was a mistake

Added 9 August 2026, after a comment on this post.

The version above recognised a legacy plaintext row by its first byte. The objection, in one sentence: that makes the storage format ambiguous forever, and the ambiguity is exploitable, because a row that merely looks like JSON bypasses decryption whoever wrote it.

The failure is not the encryption, it is that convertToPHPValue() was asking the payload what it was. A payload is not a trustworthy narrator. The format and the key version belong next to the data, as data:

public const string PREFIX = 'enc:v1:';

public function convertToDatabaseValue($value, AbstractPlatform $platform): ?string
{
    if (null === $value) {
        return null;
    }

    $keyring = self::keyring();

    return self::PREFIX.$keyring->activeId().':'
        .$keyring->active()->encrypt(json_encode($value, \JSON_THROW_ON_ERROR));
}

public function convertToPHPValue($value, AbstractPlatform $platform): ?array
{
    if (!\is_string($value) || '' === $value) {
        return null;
    }

    if (!str_starts_with($value, self::PREFIX)) {
        // Fail closed: a readable row is a bug or an intrusion, never a fallback.
        throw new \RuntimeException('This value is not in the "'.self::PREFIX.'" envelope.');
    }

    [$keyId, $ciphertext] = self::split($value);

    return $this->decode(self::keyring()->get($keyId)->decrypt($ciphertext));
}

Three things change, and only the first is obvious.

Reading fails closed. There is no longer a branch that returns plaintext. The cost is that a dump restored from before the switch stops the application instead of quietly serving personal data in clear, which is the trade I want on a column that exists because a compliance form asked about it.

The key id makes rotation real. get($keyId) reads from a keyring: one active key that writes, any number of retired keys that only read. Rotating is now three steps that never need a flag day (promote the new key, backfill, drop the old one from the keyring), and the day a key leaves, the failure is a message naming the missing key id rather than a MAC error you have to guess at.

The backfill stops guessing too. The conversion command tries decryption first, with every key in the keyring, and only then treats the value as plain JSON. Base64 decoding of {"name":...} fails outright, so plaintext never decrypts and a ciphertext is never mistaken for plaintext. The heuristic is gone from the read path and from the migration path.

And because the type now refuses anything unenveloped, the command doubles as the check that says whether the database is fully converted:

bin/console app:encrypt-buyers --dry-run

exits non-zero as long as one row is outside the envelope, which is a cron-shaped alert rather than a paragraph in a runbook.

The whole change is about eighty lines including tests, and it was possible because the table held five rows. That is the real lesson, and it is the one this post got wrong by one step: storage formats are not refactorable. Everything else in this codebase can be changed later. A format that is already in a million rows cannot, and the version prefix that would make it changeable costs seven bytes.

The part that is not code

Two hours of work and about a hundred lines. The reason to write it up is not that any of it is clever. It is that the whole thing was triggered by a yes/no question in a form, and the interesting decision was upstream of all the code: treating the form as an audit of what was true rather than a box to tick.

That reframing found three false answers, not one. Encryption at rest was the first. Encrypted backups was the second, and it was a server-wide fix that now protects every other project on the same box. Access logging was the third, and it did not exist at all.

If you sell software to businesses, you will meet that form. Shopify, Stripe, any enterprise procurement questionnaire, the ISO 27001 checklist your first serious customer sends you. The answers are cheaper to make true while the codebase is small, and the questions are a surprisingly good roadmap for a solo developer who cannot afford a security review.

The app this comes from is InvoicePilot, compliant invoices for Shopify merchants, built on ShipAnvil, which ships the auth, billing, admin and deploy pipeline these apps stand on. If you are wiring the Shopify side of things in PHP, the previous post covers the whole embedded-app authentication path in Symfony, and the foundations are in deploying Symfony to a production VPS.

Building an embedded Shopify app in Symfony: session tokens, token exchange, and the parts nobody writes in PHP

Eric Mollenthiel — Sun, 02 Aug 2026 23:01:29 +0000

Shopify's app documentation has exactly one first-class path: Node, the
official CLI, and a Remix template that wires authentication for you.
Pick any other language and you leave the paved road at the first turn,
because everything interesting happens before your framework's router
sees the request: the app runs in an iframe inside the Shopify admin, the
browser will not give you a cookie there, and the token you need to call
the Admin API has to be traded for on the fly.

None of that is hard. It is just undocumented outside JavaScript. This
post is the complete authentication and security path for an embedded
Shopify app written in Symfony 7.4 on PHP 8.5, taken from
StockAnvil, an app that passed
Shopify's App Store review and is live today. Every snippet below is
production code, not a sketch.

The one architectural decision everything else follows from

An embedded app is a page inside admin.shopify.com, in a cross-origin
iframe. Third-party cookies are dead in that context, so a PHP session
is not available to you and never will be. Shopify's answer is a
short-lived JWT called a session token: App Bridge (their JS shim,
served from Shopify's CDN) mints one per request, valid for one minute,
and attaches it to every same-origin fetch() your page makes.

Which forces a split most PHP developers do not make by default:

The HTML shell is public and carries no shop data. It is a layout, a nav, and empty containers. Serving it requires no authentication, sets no cookie, and reveals nothing.
Every byte of merchant data is behind /api, authenticated by the session token on each call.

That split is worth stating out loud because it inverts the usual
Symfony instinct (secure the controller, render the data server-side).
Here, rendering data into the shell would mean authenticating a page
load that has no credentials to authenticate with. The payoff is that
the shell is trivially cacheable and the security surface is one
firewall over one path prefix.

# config/packages/security.yaml
firewalls:
    # Webhooks authenticate via HMAC signature in the controller, not a firewall.
    webhooks:
        pattern: ^/webhooks
        security: false
    # Embedded admin API: App Bridge session token (JWT) on every request.
    api:
        pattern: ^/api
        stateless: true
        provider: users_in_memory
        custom_authenticators:
            - App\Security\SessionTokenAuthenticator
    main:
        lazy: true
        provider: users_in_memory

access_control:
    - { path: ^/api, roles: ROLE_SHOP }

stateless: true is not decoration. It tells Symfony not to try to
store the token in a session it does not have.

Verifying a session token: five checks, not one

The session token is a JWT signed HS256 with your app's client secret.
Verifying the signature is the part everyone does. The four claim checks
after it are the part that gets skipped, and each one closes a real hole:
without aud you accept tokens minted for a different app that happens
to share nothing but the algorithm; without the iss/dest match you
accept a token that claims one shop in one place and another shop
elsewhere.

// src/Shopify/SessionTokenVerifier.php
$expected = hash_hmac('sha256', $encodedHeader.'.'.$encodedPayload, $this->apiSecret, true);
if (null === $signature || !hash_equals($expected, $signature)) {
    throw new InvalidSessionTokenException('Invalid JWT signature.');
}

$now = $this->clock->now()->getTimestamp();

// exp / nbf, with a small leeway for clock skew
if (!\is_int($exp) || $now - self::LEEWAY_SECONDS >= $exp) {
    throw new InvalidSessionTokenException('Session token has expired.');
}

// aud must be *our* client id
$audiences = \is_array($aud) ? $aud : [$aud];
if (!\in_array($this->apiKey, $audiences, true)) {
    throw new InvalidSessionTokenException('Session token audience mismatch.');
}

// dest is the shop; iss must live on the same host
$shopDomain = parse_url($dest, \PHP_URL_HOST);
if (!\is_string($shopDomain) || !ShopDomain::isValid($shopDomain)) {
    throw new InvalidSessionTokenException('dest claim is not a valid shop domain.');
}
if (parse_url($iss, \PHP_URL_HOST) !== $shopDomain) {
    throw new InvalidSessionTokenException('iss and dest claims do not match.');
}

Two details that cost time if you get them wrong. Use hash_equals, not
===, on the signature: this is the textbook timing-attack surface and
it costs one function name. And take the clock from
Symfony\Component\Clock\ClockInterface rather than calling time(),
so your test suite can produce an expired token without sleeping.

A five-second leeway on exp and nbf is deliberate. The token lives
sixty seconds; a server clock a couple of seconds behind Shopify's
would otherwise reject perfectly good tokens at a rate that looks like a
random, unreproducible bug.

Token exchange, and the `expiring=1` that returns 403 without it

A session token proves who is asking. It does not let you call the
Admin API. For that you trade it for an access token, using OAuth 2.0
token exchange (RFC 8693) rather than the old authorization-code dance
with redirects. With managed installation, this is the entire install
flow: no /auth route, no redirect, no callback. The first authenticated
request from a new shop simply performs the exchange.

// src/Shopify/TokenExchanger.php
private const GRANT_TYPE = 'urn:ietf:params:oauth:grant-type:token-exchange';
private const SUBJECT_TOKEN_TYPE = 'urn:ietf:params:oauth:token-type:id_token';
private const OFFLINE_TOKEN_TYPE = 'urn:shopify:params:oauth:token-type:offline-access-token';

public function exchangeForOfflineToken(string $shopDomain, string $sessionToken): AccessToken
{
    return $this->requestToken($shopDomain, [
        'grant_type' => self::GRANT_TYPE,
        'subject_token' => $sessionToken,
        'subject_token_type' => self::SUBJECT_TOKEN_TYPE,
        'requested_token_type' => self::OFFLINE_TOKEN_TYPE,
        'expiring' => 1,
    ]);
}

That last line is the one to copy. Ask for a non-expiring offline
token and the Admin API answers 403 (observed live on 2 July 2026),
with an error that does not say so. An expiring token lives about an
hour and is renewed with a refresh token, which Shopify rotates on
every call: whatever persists your tokens has to write the new refresh
token back, or your background jobs work for an hour and then stop.

Offline (not online) is the right request for anything that runs without
a merchant present: webhook handlers, nightly digests, sync jobs.

The authenticator, and the header that saves your error rate

Symfony's custom authenticator maps onto this cleanly. The passport is
self-validating because the JWT signature is the credential check.

// src/Security/SessionTokenAuthenticator.php
public function supports(Request $request): bool
{
    return str_starts_with($request->headers->get('Authorization', ''), 'Bearer ');
}

public function authenticate(Request $request): Passport
{
    $jwt = substr($request->headers->get('Authorization', ''), \strlen('Bearer '));

    try {
        $sessionToken = $this->sessionTokenVerifier->verify($jwt);
        $shop = $this->shopInstaller->ensureInstalled($sessionToken);
    } catch (InvalidSessionTokenException|TokenExchangeFailedException $e) {
        throw new CustomUserMessageAuthenticationException($e->getMessage(), [], 0, $e);
    }

    return new SelfValidatingPassport(new UserBadge(
        $sessionToken->shopDomain,
        static fn (): ShopUser => new ShopUser($shop, $sessionToken->subject),
    ));
}

private function unauthorized(): JsonResponse
{
    return new JsonResponse(
        ['error' => 'Unauthorized.'],
        Response::HTTP_UNAUTHORIZED,
        ['X-Shopify-Retry-Invalid-Session-Request' => '1'],
    );
}

X-Shopify-Retry-Invalid-Session-Request is worth a paragraph of its
own. Tokens last sixty seconds, so a page left open on a merchant's
second monitor will eventually fire a request with a token that expired
while they were in another tab. Return a bare 401 and the merchant sees
an error. Return 401 with that header and App Bridge silently fetches
a fresh token and retries once. One header turns a class of user-visible
failures into nothing at all.

Note also where installation happens: ensureInstalled() inside
authenticate(). A shop that appears with a valid token and no row in
your database is a new install, and the token exchange happens right
there. There is no separate install endpoint to secure.

Webhooks: raw body, constant time, and the query HMAC that is not `http_build_query`

Webhooks carry no session token. They are signed: HMAC-SHA256 over the
raw request body, base64-encoded in X-Shopify-Hmac-Sha256. Raw
means raw, before any JSON decoding, before any middleware touches it.
Shopify's automated App Store review sends a deliberately mis-signed
webhook and requires a 401.

public function verifyWebhook(string $rawBody, string $hmacHeader): bool
{
    if ('' === $hmacHeader) {
        return false;
    }

    $expected = base64_encode(hash_hmac('sha256', $rawBody, $this->apiSecret, true));

    return hash_equals($expected, $hmacHeader);
}

The second signature type is the sharp one. Links from the Shopify admin
carry an hmac query parameter computed over the sorted query string,
and the message Shopify signs is not a URL-encoded query string. Only
&, % and = are escaped, and only in the places shown below.
Reaching for http_build_query() here produces a signature that is
wrong for any value containing a space or a slash, which is exactly the
kind of bug that passes every test you thought to write:

$pairs = [];
foreach ($query as $key => $value) {
    $pairs[] = strtr($key, ['&' => '%26', '%' => '%25', '=' => '%3D'])
        .'='
        .strtr($value, ['&' => '%26', '%' => '%25']);
}

$expected = hash_hmac('sha256', implode('&', $pairs), $this->apiSecret);

Two things the App Store checks that are pure infrastructure

Access tokens encrypted at rest. A shop's access token is a
credential for someone else's business. Storing it in plaintext means a
read-only SQL injection anywhere in your app hands over every merchant's
store. Libsodium makes this eight lines, and PHP ships it:

// src/Shopify/TokenCipher.php — XSalsa20-Poly1305 secretbox
public function encrypt(string $plaintext): string
{
    $nonce = random_bytes(\SODIUM_CRYPTO_SECRETBOX_NONCEBYTES);

    return base64_encode($nonce.sodium_crypto_secretbox($plaintext, $nonce, $this->key));
}

The key is 32 random bytes, base64-encoded, in an environment variable,
and the constructor rejects anything else at boot rather than at the
first decrypt.

A frame-ancestors CSP scoped to the current shop. Your app must be
frameable by the merchant's admin and by nobody else, which means the
header is computed per request from the shop parameter, not set once
in the vhost:

// src/EventListener/EmbeddedAppHeadersListener.php
$frameAncestors = ShopDomain::isValid($shop)
    ? \sprintf('frame-ancestors https://%s https://admin.shopify.com;', $shop)
    : "frame-ancestors 'none';";

$event->getResponse()->headers->set('Content-Security-Policy', $frameAncestors);

Deny-all on the fallback is the correct default: a request without a
valid shop is not an admin iframe, and it should not be framed at all.
That request is also your public landing page: the app URL registered in
the Partner Dashboard is the address a crawler or a curious merchant
reaches from a link outside the admin, and serving them the App Bridge
shell shows an inert blank page. Branch on the shop parameter and
render marketing HTML instead.

One more App Store requirement that is easy to miss in a Twig layout:
App Bridge must be the first script in <head>, loaded synchronously
from Shopify's CDN, never bundled, never deferred, on every embedded
page.

The Symfony bug that this app found, which has nothing to do with Shopify

Worth knowing whoever you build for. This email subject line lost its
first two words in production, silently:

digest.attachment: 'Attached: your restock list (%count% items).'

It rendered as "your restock list (12 items)." because
%count% is numeric, which routes the string through Symfony's
pluralization logic. There, in
symfony/translation-contracts/TranslatorTrait.php, each part is tested
against '/^\w+\:\s*(.*?)$/' — an explicit-interval syntax for keyed
plural rules — and a message that innocently begins with a word followed
by a colon matches. The prefix is consumed as if it were a rule name. No
exception, no log line, just a shorter sentence.

Two lessons, one specific and one general. Specific: with %count% in a
message, never start it with Word:. General, and the more expensive
one, is why the test suite missed it. The assertion checked
assertStringContainsString('restock list', $subject) — it started
matching in the middle of the sentence, so it could only ever have
verified the part that never breaks. Assert a string from its first
character. The fix now also walks both translation catalogues and fails
CI on any %count% message matching that pattern.

What this adds up to

An embedded Shopify app in Symfony is roughly 400 lines of security code
you cannot copy from the docs: a JWT verifier, a token exchanger with
refresh-token rotation, a stateless authenticator, an HMAC verifier with
two algorithms, an encryption wrapper, and a response listener. After
that, it is a Symfony app like any other, with Doctrine, Messenger for
the background sync, and the same testing tools you already use. The
platform-specific surface is small and it stays where you put it.

The app that produced this code is StockAnvil,
low-stock alerts for Shopify merchants, live in the App
Store, built on
ShipAnvil with its auth, billing, admin and deploy pipeline
already in place. If you are weighing an inventory tool because of the
Stocky shutdown on 31 August 2026,
that page is an honest map of which replacement covers which part,
including the parts StockAnvil does not do.

For the Symfony foundations under all of this, start with the
production VPS deploy and
Symfony 7.4 LTS support math.

https://shipanvil.com/blog/shopify-embedded-app-symfony

A league season is not a long tournament: four product bugs my test suite could never catch

Eric Mollenthiel — Fri, 31 Jul 2026 00:56:22 +0000

I built a football prediction game for the 2026 World Cup. A month long, 104 matches,
one winner at the end. It worked, people played, nothing caught fire.

Then I pointed the same app at a domestic league: Ligue 1, 306 matches, 34 matchdays,
nine months. Same data model, same scoring, same templates. Nothing crashed. Nothing
threw. Every test stayed green.

And almost every product decision inside the app was suddenly wrong.

Here are the four that mattered, because none of them were technical, and none of them
would have shown up in a test suite.

1. A cumulative leaderboard is decided by October

In a tournament, the overall standings are the whole game. You play for four weeks and
the table at the end is the story.

Over 34 matchdays, that table stops being a game around week eight. The player who
started well is 60 points ahead, the player who joined in November is mathematically out,
and everyone else is reading a scoreboard they cannot change. The product still worked.
It just had no stakes left.

The fix was not a better algorithm, it was a second unit of time: a per-matchday
leaderboard, so each weekend has its own winner, plus a season honours table counting
how many matchdays each player has won. Same points, same scoring, sliced differently.
A player who is 14th overall can still win this weekend, and that is the thing that
makes them come back on Friday.

Worth noting what I did not do: no reset, no handicap, no catch-up bonus. Anything
retroactive on a scoring system that people are currently playing destroys trust in the
standings, and the standings are the entire asset.

2. A sliding 24 hour reminder becomes 100 emails

The reminder job was built for a tournament: "if a player has unpredicted matches
kicking off in the next 24 hours, email them." Matches trickle in daily, so that reads
as one email a day, and it is fine.

A Ligue 1 matchday runs from Friday 20:45 to Sunday 20:45. The same job, unchanged,
would have sent three emails per weekend to the same person: one on Friday for one
match, one on Saturday for three, one on Sunday for five. Around a hundred emails per
season, per player. That is not a reminder, that is a spam complaint with extra steps.

It would also have arrived on the morning of the first match, when the natural gesture in
a league is the opposite: you fill all nine games in one sitting, once, whenever you
think of it.

So the job now has two disjoint code paths in the same command. Tournaments keep the
sliding window. Leagues get one email per matchday, fired when the first kickoff of
that matchday is 24 to 48 hours out, listing every still open match of the matchday.

The part I like: there is no reminder_sent table. Idempotency comes from the window
itself. "The first kickoff is between 24 and 48 hours away" is true on exactly one
calendar day, and the cron runs once a day. The one case that legitimately produces a
second email is a postponement that drags the first kickoff back into the window, and a
calendar that moved is exactly when you want to remind people again.

The tradeoff is written in the crontab in plain words: doubling the cron frequency would
double the emails. A comment is cheaper than a table, as long as the comment is where the
mistake would be made.

3. "Upcoming matches" is not "all future matches"

The dashboard listed every future match. In a tournament that is at most a few dozen
cards, and the progress badge reads "12/18 predicted", which feels achievable.

In a league it is 306 cards and "12/306", which feels like homework.

Now the list is bounded to the next two matchdays. The subtle part is how you pick
them. My first version took MIN(round_number) over unplayed matches. That is wrong in
any real league, because postponements are routine: one match of matchday 3 replayed in
November would have pinned the dashboard to matchdays 3 and 4 and hidden the actual
weekend. The window follows nearest kickoff times, not round numbers, and the
postponed match reappears by itself when its new slot comes around.

4. In a tournament there is always something to do

Four weeks of a World Cup is four weeks of permanent attention. Nine months is not. Most
of a league season, an engagement product is competing with the user forgetting it
exists.

The two moments I built for are both moments the app already knew about and was throwing
away: the player who just won a matchday (one per week, peak pride) and the player
who just finished predicting the coming matchday (everyone, 34 times a season, peak
engagement and then several days of nothing). Both now offer a share, native share sheet
on mobile with a WhatsApp fallback, and both link to the public competition page rather
than the private league.

That last detail took a minute of thought and is worth the minute: sharing a league
invite code from a mobile share sheet means it can land in a public post, and a private
standings table with strangers in it is not a feature.

The actual lesson

The data model was right. Event, Game, Prediction, a round_number column that was
already there. Not one migration was needed for any of this.

What was wrong was every assumption about cadence: how often the user shows up, how
long a unit of competition lasts, how far ahead they can see, how long they wait between
two moments of interest. Those assumptions are almost never in your schema. They are
spread across cron expressions, query limits, email conditions, and empty state copy,
which is exactly where nobody looks when they say "we just need to support a new
competition format".

If you are about to reuse a working product on a longer or shorter timescale, grep your
codebase for time: every 24 hours, every setMaxResults, every "next" and "current"
and "upcoming". That is your real diff.

The app is a free prediction game for friends and coworkers, no betting and no money
involved, built with Symfony, Turbo Streams over Mercure for live standings, and 16
locales. It is at pronoarena.com if you want to see the result,
and the Ligue 1 season starts on August 21.

DEV Community: Eric Mollenthiel

Postgres RLS in Symfony: with one tenant in the fixture, your isolation suite passes with no policy at all

The invariant is not "the owner run must fail"

Equality alone is green in three databases that are broken

Disjointness has to stay inside the boundary it is checking

The two ways "no tenant context" fails, and they are not alike

The leak that hides under PHP-FPM

Reproduce it

Row-level security in Symfony: the role that ran your migrations bypasses every policy you wrote

The setup, which is the one you would write

One line changes the answer

Why a Symfony project walks into this and a test suite does not catch it

Two fixes, and I measured both

FORCE ROW LEVEL SECURITY

A serving role that does not own anything

The guard that survives the next table

And once RLS does apply, the setting has two ways to be missing

What I would keep

Single-database multi-tenancy in Symfony: a 31-line Doctrine filter, and the five places it never runs

The filter

The layer that knows who is asking

Place 1: the console, and every Messenger worker

Place 2: your own back office, on purpose

Place 3: find() when the entity is already in the identity map

Place 4: DBAL, and every line of native SQL

Place 5: joined inheritance, when the column is not on the root table

And one place it does run, where most people assume it does not

Proving it, rather than believing it

What I would keep

Every no-code builder generates code. I shipped an interpreter instead.

The domain, in four lines

Why generating Java is the trap

What I shipped instead

The part I did not see coming: the grammar becomes data

The bill

Three things the interpreter forced me to get right

The obvious objection

Where it is

Your storage format is the one thing you cannot refactor

Code is soft, storage is not

Sniffing is the tax you pay for skipping the version

The window closes when the first row is written

What the fix looks like

Reading fails closed

The key identifier makes rotation real

The backfill does not guess either

Where else the seven bytes apply

Is this not YAGNI?

What it actually cost

Six decisions behind a shared-expense app, and the one where you must refuse to round

The invariant that makes everything testable

Store hundredths even for currencies that have none

Turning balances into transfers

The mode where you must refuse rather than fix

Freeze the exchange rate at entry

A pending expense counts nowhere

And one thing that is not about money at all

The order that worked

A week is seven nights, not 168 hours: dates in a shared-custody calendar

1. The unit is the night, and a night is not a duration

2. Do the arithmetic in the family's timezone, store the result in UTC

3. Never walk a calendar from midnight. Walk from noon.

4. A date is not an instant, and the type system will not tell you

5. The one line that silently poisons all of the above

How I know any of this holds

Cloudflare was 403-ing ChatGPT, Perplexity and Claude on my site, and my logs never knew

The five minute probe that should have been the first thing I did

Why every instrument I had was blind to this

The fix is one toggle, and it is not the one named after crawlers

The guard I put in, and why it is a timer and not a test

The second thing Cloudflare was doing, which was worse

What I would tell myself in June

Splitting €10 three ways: the largest remainder method, and the tiebreak everyone forgets

The three wrong answers

The right answer is a 1792 apportionment method

The tiebreak is the part people skip

What this buys you: an invariant you can assert

Three edge cases worth stealing

Where this came from

Encrypting personal data at rest in Symfony: a 60-line Doctrine type, and the four things it breaks

`FORCE ROW LEVEL SECURITY`

Place 3: `find()` when the entity is already in the identity map

Token exchange, and the `expiring=1` that returns 403 without it

Webhooks: raw body, constant time, and the query HMAC that is not `http_build_query`