Daemons, resources, migrations and schedules are declared once. Everything else - down to the name of every environment variable - follows from that one declaration.
👋 I'm Anton - a software engineer working mostly in PHP/Symfony and Go, currently carving a live PHP monolith into Go services. This series is about what a service gets in the environment without anyone writing it a second time. Notes: github.com/brilliant-almazov.
The thought I want to put on the table is small and slightly uncomfortable: in our setup a thing exists in the environment only because one YAML file says it exists. Not because the code is there. Not because the tests are green. Maybe you already work this way and do it better; maybe you look at it completely differently. Either way I'd rather hear it than guess.
This part is the declaration itself, and the incident that made me take it seriously.
The daemon that wasn't there
A piece of background work needed a home. I gave it its own binary - a new cmd/*, its own entry point, its own loop. Everything about that decision looked finished:
- the code was in the repository, reviewed and merged;
- the binary built locally, first try;
- the tests were green, including the ones covering the work itself;
- the deploy pipeline ran without a single red step.
And after the rollout, nothing was processed.
Not "processed slowly". Not "processed with errors". Nothing. No log line to grep, no failed job to look at, no error to classify. The symptom read as the work is quiet, which is a debugging problem, when the truth was the work does not exist, which isn't.
Here is what I had missed. The image is built by a job per declared binary, from one Dockerfile with a build argument selecting which binary to compile. The list of binaries it iterates over is not "what's in cmd/". It's what the service manifest declares. The manifest declared two daemons. My third binary was not one of them - so the build had nothing to build it from, the deploy had nothing to deploy, and every check on the way stayed green, because none of them was ever asked about a binary nobody declared.
service manifest one build job per declared binary
┌──────────────────────┐
│ daemons: │
│ - name: server │──────▶ build server ──▶ image ──▶ deploy
│ - name: worker │──────▶ build worker ──▶ image ──▶ deploy
└──────────────────────┘
┌──────────────────────┐
│ a third binary │ builds locally
│ in cmd/, undeclared │ tests green, pipeline green
└──────────────────────┘ nothing to build in the environment
No check was ever asked about a binary nobody declared.
That's the part worth sitting with. There was no failure anywhere. Local build: fine. Tests: fine. Merge: fine. Pipeline: fine. The only thing that was wrong was an absence, and an absence has no failure mode - it just quietly isn't.
The rule the incident leaves behind
Not declared - does not exist.
Not "exists but is misconfigured". Not "exists but isn't wired". Simply not there, with no signal that it should have been.
Once you accept that, a second rule follows immediately, and it's the one I now enforce hardest:
The set of binaries is the service owner's decision, and nobody else's. A new cmd/* is not something an executor - human or otherwise - gets to introduce while solving a task. Any new background work goes into a daemon that already exists. If it genuinely doesn't fit any of them, that's a one-line question to the owner, not a new entry point invented on the way past.
That is written as an explicit prohibition in every control prompt I hand out. Not as a style preference - as a hard rule, in the same place as "don't push to the main branch". The failure it prevents is exactly the one above: work that is finished, tested, merged, deployed, and absent.
| What people expect to be enough | What actually decides |
|---|---|
the code is in cmd/
|
the manifest lists the daemon |
| the binary builds locally | the build job iterates declared binaries |
| the tests are green | the tests never ask what exists in the environment |
| the pipeline is green | the pipeline deploys only what was declared |
The whole manifest
Here is the declaration in full - not an excerpt, not a simplified version. This is the entire file that decides what a service is, with the name replaced by a neutral placeholder:
service:
name: <service>
version: 0.1.0
daemons:
- name: server
handlers: [grpc]
- name: worker
handlers: [scheduler]
infra:
postgres:
- name: main
migrations: false
tx:
- name: main
pool: main
grpc:
- name: api
daemons: [server]
scheduler:
- name: main
daemons: [worker]
resources:
- type: messaging
name: main
Two blocks, and it's worth reading them as two different kinds of statement.
service declares who runs. Two daemons: server, which carries the gRPC handler, and worker, which carries the scheduler. Both come out of a single image - the build takes the binary as an argument. In our case worker is where the outbox relay lives, along with background calculations and audit-partition retention. That's three jobs in one daemon, not three daemons, and that's the point of the rule above: the daemon list is short and stays short.
infra declares what they need. A Postgres pool named main with migrations turned off on it, a transaction manager bound to that pool, a gRPC listener named api attached to the server daemon, a scheduler named main attached to the worker daemon, and one generic resource of type messaging named main.
Note what the daemons: key does in the grpc and scheduler entries. It's not documentation - it's the wiring. The listener exists in server because that line says so; the scheduler runs in worker because that line says so. There is no second place where a daemon picks up its resources.
service — who runs infra — what they need
┌────────────────────────┐ ┌──────────────────────────┐
│ daemons: │ │ postgres: main │
│ - name: server │ │ migrations: false │
│ handlers: [grpc] │ │ tx: main │
│ - name: worker │ │ pool: main │
│ handlers: │ │ grpc: api │
│ [scheduler] │ │ daemons: [server] │
│ │ │ scheduler: main │
│ │ │ daemons: [worker] │
│ │ │ resources: main │
│ │ │ type: messaging │
└────────────────────────┘ └──────────────────────────┘
the wiring, and there is no second place that carries it:
grpc: api ──▶ runs in the daemon server
scheduler: main ──▶ runs in the daemon worker
No ports, no DSNs, no URLs, no credentials, no host names —
only objects and relations.
And note what isn't in the file: no ports, no DSNs, no URLs, no credentials, no host names. The manifest declares objects and their relationships. Values arrive from the environment - and the manifest decides the names those values arrive under.
Every variable name is derived, not chosen
This is the part that surprised people the most when we adopted it, so let me state it plainly: nobody picks environment variable names. They fall out of the declaration mechanically, by one rule:
<TYPE>_<NAME>_<FIELD>
Type of the resource, name of the resource, field being supplied. Take the three lines from the manifest above and turn the crank:
| Declared in the manifest | Variable the platform reads |
|---|---|
postgres: main |
POSTGRES_MAIN_DSN |
grpc: api |
GRPC_API_PORT |
messaging: main |
MESSAGING_MAIN_RABBITMQ_URL |
The difference between this and a naming convention matters. A convention is something you follow, remember, and occasionally get wrong. This is a derivation: the platform builds the name at runtime by concatenating what the manifest says, and reads that. There is no list of variable names in the code to keep in sync, because there is no list. Rename the resource in the manifest and the variable name changes with it - by construction, not by grep.
The practical consequence: the answer to "what environment variables does this service need?" is not found by searching the code. It's read off the manifest, one resource at a time, with the crank turned.
The name that looks like a typo
Every time someone new reads our environment, they stop on this line:
OUTBOX_DOMAIN_EVENT_OUTBOX_BATCH_SIZE
OUTBOX twice. It reads as a copy-paste accident, and the reflex is to "fix" it.
It isn't one. Run it through the formula:
| Part | Value |
|---|---|
<TYPE> |
OUTBOX |
<NAME> |
domain_event_outbox |
<FIELD> |
BATCH_SIZE |
OUTBOX_DOMAIN_EVENT_OUTBOX_BATCH_SIZE not a typo
└────┘ └─────────────────┘ └────────┘
type resource name field
Type and name are independent axes.
A derivation you are willing to override is not a derivation.
The type is OUTBOX. The resource's name happens to contain the word outbox too, because that's an honest description of what it is - a domain-event outbox. Type and name are independent axes; nothing stops a word from appearing on both. The repetition is a fact about the name someone chose for the resource, not a defect in the derivation.
I like this example precisely because it's ugly. It's the cheapest possible test of whether you actually believe in a mechanical rule: the first time the rule produces something you find aesthetically annoying, do you keep the rule or do you special-case it? Special-casing here would cost the whole property - that names are derivable, and that reading the manifest is enough to know them. That's far more valuable than a variable being pretty.
The general form of the lesson: a derivation you're willing to override isn't a derivation. It's a default with exceptions, and then you're back to keeping a list.
Two details that catch people
Two consequences of "declare the object, get the variables" that are not obvious and have cost me explanations more than once.
The schedule lives on the resource
The scheduler is a declared resource, and the cron expression belongs to it. The service itself reads no CRON_* variables of its own. If you go looking for the schedule in application config, you won't find it - not because it's hidden, but because it was never the application's to hold. The declaration owns it.
This trips people who expect "there's a scheduled job, so there must be a schedule setting somewhere in the service". The mental correction is: the job doesn't have a schedule; the declared scheduler resource does, and the daemon it's attached to runs it.
The migration resource has no DSN of its own
Migrations are a resource too. The obvious expectation is a ..._DSN variable for it, sitting alongside the pool's. There isn't one. The migration resource resolves its DSN from the pool's variable - it points at the declared pool and takes the connection string from there.
Which is the right shape when you think about it for a second: two DSN variables for the same database is two chances to point them at different databases. One declaration, one value, one place to be wrong.
Notice that in the manifest above, migrations: false sits on the postgres: main entry. Migrations aren't a separate thing you configure elsewhere; they're a property of the declared pool.
What the same declaration buys further down
I'll keep this short because it's the subject of a later part, but it's the reason the manifest is worth this much attention.
Those two daemon names - server and worker - are not just runtime facts. They're the input to everything downstream:
-
The build takes the binary as an argument and produces an image per declared binary from a single
Dockerfile. The list it iterates is the daemon list. -
The deploy is a manual run with two parameters: the version, and which services -
server,worker, orall. That third value isn't a special case; it's "everything the manifest declares".
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐
│ manifest │──▶│ build │──▶│ image │──▶│ deploy │
└──────────┘ └──────────┘ └──────────┘ └──────────────┘
server one job per one artefact version, and
worker declared per declared server · worker
binary binary · all
work that never entered the chain
┌────────────────────────────────────────────────────────────┐
│ falls out at step one — and every step after it stays green│
└────────────────────────────────────────────────────────────┘
So the incident at the top wasn't bad luck about one pipeline. It was the system behaving exactly as designed: a declaration drives the build, the build drives the artefacts, the artefacts drive the deploy, and something outside the declaration falls out of that chain at step one, silently, with everything downstream still green.
The details of that chain - and what the human still decides in it - are the next part but one. Here it's enough to see that they all read the same file.
What it costs
I don't think this is free, and the costs are specific.
The manifest is a bottleneck, on purpose. You cannot add background work "quickly, just this once" by dropping in an entry point. Every new runnable thing goes through the file, and the file goes through the owner. When you're mid-task and you know exactly what you want, that feels like friction - because it is friction. It's the same friction that would have caught my missing daemon before it shipped.
The names get long, and sometimes graceless. MESSAGING_MAIN_RABBITMQ_URL is fine. OUTBOX_DOMAIN_EVENT_OUTBOX_BATCH_SIZE is not something you'd choose by hand. Mechanical derivation means accepting whatever the mechanism produces, including the awkward output - and the moment you start hand-tuning names, you've traded a property for an aesthetic.
You have to read the manifest to know what a service is. This is the cost I underrate most often. The code no longer tells you. You can read every package in the repository and still not know how many daemons run, which resources are attached to which one, or what the environment is expected to supply. That information moved into one file, and someone arriving at the service has to be told that the file is where it lives. It's a smaller cost than the alternative - the knowledge being spread across a Dockerfile, a pipeline, and someone's memory - but it isn't zero, and pretending otherwise would be dishonest.
Where I might be wrong
This is one team's shape, on one platform, arrived at after one incident I'd rather not repeat. It buys a property I care about a lot: what runs and what it needs are one artefact, and everything else is derived from it. It pays for that property with rigidity and with some genuinely ugly variable names.
So - how is this solved where you work? Some of you do this better than we do and have a cleaner way to keep declaration and reality in step. Some of you have been through exactly this and know the failure modes I haven't hit yet. And some of you look at it differently altogether and think a single declaration file is the wrong centre of gravity for a service.
I'd like to hear all three, and especially the third. What broke on your side, and what did you change because of it?
Operations out of the box - Part 1.
Next: the variable catalogue as a repository artefact - a generated snapshot of every variable the service reads, with the file and line it's declared in, and a CI check that fails the build when the snapshot and the code drift apart.
Top comments (0)