59 variables, generated not written - and drift fails the build.
👋 I'm Anton - a software engineer working mostly in PHP/Symfony and Go, currently carving a live
PHP monolith into Go services. Part 1 of this series was about the service manifest: one
declaration that the runtime reads instead of a pile of wiring code. This part is about the file
that falls out of that declaration - the catalog of environment variables the service actually
reads. Notes: github.com/brilliant-almazov.
Here's the thought I want to put down, and you may well look at it differently: an env catalog is
not documentation. It's an artifact - a generated file that lives in the repository and fails the
build when it stops matching the code.
As always: this is what I'm doing on one codebase right now, with the price attached. Not a
recommendation for yours.
Start with the count
One service. Not the whole platform, not the fleet - one service, one repository:
| Where the variable is declared | Count |
|---|---|
| The platform library | 45 |
| The service's own configuration | 14 |
| Total the service reads | 59 |
Fifty-nine is the number that decides the whole question. At five variables, a README section is
fine. At fifty-nine, hand-written documentation survives exactly until the next pull request -
someone adds a field to a config struct, nobody touches the README, and from that moment the
README is confidently wrong. Not missing. Wrong, which is worse, because it still reads like an
answer.
And the split matters as much as the total. Three quarters of those variables were never written
by anyone on this service - they came in with the platform library. Nobody on the service side can
document a list they didn't author and don't control.
The shape of the snapshot
So the file is generated. It's YAML, it sits in the repository, and it opens by telling you not to
touch it:
# AUTO-GENERATED by <snapshot tool>. DO NOT EDIT.
schema_version: 1
service: <service>
generated_at: 2026-08-16T01:24:46Z
vars:
- name: APP_ENV
source: platform
catalog: <platform package>
required: false
secret: false
help: prod/staging/dev label exposed in platform_info
- name: AUDIT_RECORDS_RETENTION_MONTHS
source: service_config
defined_in: internal/daemon/worker/retention/config.go:16
required: false
secret: false
Two records, and they're deliberately the two different kinds. APP_ENV comes from the platform,
so it carries the catalog that owns it. AUDIT_RECORDS_RETENTION_MONTHS is this service's own, so
it carries the file and the line where it's declared.
schema_version is there because the file is read by machines as well as people. When the record
shape changes, consumers have something to branch on instead of guessing from the keys present.
What one record carries
Six fields, and no more than six:
| Field | What it answers |
|---|---|
name |
The variable as the process sees it |
source |
platform or service_config - who declared it |
catalog |
For platform variables: which catalog owns the declaration |
defined_in |
For service variables: file and line of the declaration |
required |
Does the process refuse to start without it |
secret |
Must the value never be logged or printed |
help |
One line of meaning, where the declaration carries one |
defined_in is the field that changes what the file is for. A list of names answers what
exists. A file and a line answer where it came from:
internal/daemon/worker/retention/config.go:16
That's a click, not an investigation. The difference shows up in the questions the catalog can
close. "Do we still need this one?" - open the line, see who reads it. "Why is this set in the
deploy config?" - open the line, see the struct. "Who owns this?" - the path names the daemon.
Without the line, every one of those turns into a grep across the repository, and the grep is run
by whoever is unlucky, at the least convenient time.
The secret flag earns its place for the same reason - it's read by things that dump
configuration. A snapshot that says which variables must never appear in a log is more useful than
a rule that says the same thing in prose.
┌──────────────────────────────┐ ┌────────────────────────────────┐
│ name: APP_ENV │ │ name: │
│ source: platform │ │ AUDIT_RECORDS_ │
│ catalog: <platform package> │ │ RETENTION_MONTHS │
│ required: false │ │ source: service_config │
│ secret: false │ │ defined_in: │
│ help: prod/staging/dev │ │ internal/daemon/worker/ │
│ │ │ retention/config.go:16 │
│ │ │ required: false │
│ │ │ secret: false │
└──────────────────────────────┘ └────────────────────────────────┘
carries the catalog carries the file and the line
that owns the declaration where it is declared
Drift fails the build
A generated file that nobody regenerates is just a stale file with better formatting. So the CI
job runs the same generator, in --check mode: regenerate in memory, compare with what's
committed, exit non-zero on any difference.
It runs on four triggers:
| Trigger | Why it can move the catalog |
|---|---|
Any .go file |
A config struct field is a variable declaration |
| The manifest | Declared resources change the runtime's variable set |
| The modules file | A platform bump can add or rename platform variables |
| The snapshot itself | Someone edited the file the header told them not to edit |
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ any .go │ │ manifest │ │ modules file│ │ snapshot │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │ │
▼ ▼ ▼ ▼
┌──────────────────────────────────────────────────────────────┐
│ the same generator, run in --check mode │
│ regenerate in memory · compare with what is committed │
└──────────────────────────────┬───────────────────────────────┘
┌───────────────┴───────────────┐
▼ ▼
no diff — pass any diff — build fails
The outcome is deliberately blunt. A mismatch is a red pull request, not a review comment.
Nobody has to notice it, nobody has to remember to look, and nobody has to be the person who
brings it up. The build says no, and the fix is one command.
The detail that makes the check honest
This is the part I'd have got wrong if I hadn't hit it: the generator is installed at the same
platform version the service has in its modules file.
Concretely, the check does this before it runs anything:
- Resolve the platform version from the service's modules file.
- Fetch the platform at that ref.
- Build the generator binary from it.
- Run that binary in
--checkmode.
Not "install the latest generator". Not "use whatever binary the runner has cached". The generator
is a build artifact of a specific platform version, because the catalog of 45 platform variables
is a property of that version.
Skip this and the check quietly becomes a liar. It would compare a snapshot generated against the
version the service pins with a catalog from some other version - so a service that changed nothing
goes red the moment someone else's change lands upstream. A check that fails on other people's
work isn't a check. It's a queue of people asking why their pull request is red, and after the
third such time the team learns to re-run the job until it goes green, which is the end of the
check as a signal.
Pinning the generator to the pinned platform makes the failure mean one thing only: this
repository's snapshot no longer matches this repository's code at this platform version.
1 resolve the platform version from the service's modules file
│
▼
2 fetch the platform at that ref
│
▼
3 build the generator binary from it
│
▼
4 run that binary in --check mode
┌──────────────────────────────┐ ┌──────────────────────────────┐
│ latest generator │ │ pinned generator │
│ │ │ │
│ goes red on other people's │ │ the failure means one thing │
│ work — so the job gets │ │ only: this snapshot no │
│ re-run until it goes green │ │ longer matches this code at │
│ │ │ this platform version │
└──────────────────────────────┘ └──────────────────────────────┘
Regenerating locally
The other half is a local script that does exactly what CI does - same resolution, same version,
same binary - so that fixing a red build never requires reading the CI config to find out what it
actually ran.
It has one extra behaviour worth stealing:
If the only change is
generated_at, the script reverts the file.
Every run stamps a new timestamp. Without the revert, every developer who touches the service
carries a one-line diff that says nothing, half of them commit it, and the file's history fills
with changes that don't record a change. Then git log on the catalog stops being useful, and
"when did this variable appear" becomes unanswerable from the history of the file whose entire job
is to answer it.
One conditional revert keeps the file's history equal to the history of the variables.
What the snapshot does not cover
The honest part. Resource variables are not in the snapshot.
The manifest declares resources, and the platform builds their variable names by concatenation at
runtime: <TYPE>_<NAME>_<FIELD>. The generator reads configuration declarations - structs, in
code, with fields. A name that only exists as a concatenation performed while the process boots is
not a declaration it can see.
So those names are documented as a table instead, next to the manifest:
| Declared in the manifest | The variable it produces |
|---|---|
postgres: main |
POSTGRES_MAIN_DSN |
grpc: api |
GRPC_API_PORT |
messaging: main |
MESSAGING_MAIN_RABBITMQ_URL |
The convention is mechanical enough to be read backwards, which is the point: given a variable,
you can name the resource that produced it. OUTBOX_DOMAIN_EVENT_OUTBOX_BATCH_SIZE looks like a
typo and isn't - it's the type OUTBOX plus a resource named domain_event_outbox. The scheduler
resource carries its cron expression in the manifest, so the service reads no CRON_* variables at
all. The migrations resource resolves its DSN from the pool's variable and has none of its own.
┌──────────────────────────────┐ ┌──────────────────────────────┐
│ in the snapshot │ │ not in the snapshot │
│ generated │ │ hand-maintained table │
│ │ │ │
│ configuration declarations │ │ resource names built at │
│ — structs, in code, with │ │ runtime by concatenation │
│ fields │ │ as the process boots │
│ │ │ postgres: main │
│ │ │ ──▶ POSTGRES_MAIN_DSN │
│ │ │ grpc: api │
│ │ │ ──▶ GRPC_API_PORT │
│ │ │ messaging: main │
│ │ │ ──▶ MESSAGING_MAIN_ │
│ │ │ RABBITMQ_URL │
└──────────────────────────────┘ └──────────────────────────────┘
<TYPE>_<NAME>_<FIELD> reads backwards: given a variable, you can
name the resource that produced it.
A hand-maintained table has exactly the failure mode this whole article is about, so it doesn't
stand alone either. A test over the snapshot asserts two things:
- presence - the platform variable names that must be there, are there;
- absence - names that must never appear, don't.
The absence half is the one that surprises people. It catches the case where a dependency arrives
that nobody asked for: a broker the service doesn't run showing up in the catalog means something
pulled it in, and the test says so on the pull request that pulled it in - not six months later,
when someone is trying to work out why the deploy config has a URL for it.
Reminder, rule, check
The general shape behind all of this, which I keep re-deriving on different problems:
reminder → rule → check
- A reminder in a conversation lives for one session. The next session has never heard it.
- A rule in a written instruction works while people read it, and works less every month.
- A check - a linter, a failing test, a blocking build step - works whether or not anyone remembers, agrees, or is paying attention.
The catalog is a clean case for the third rung, because the discipline it replaces is both boring
and constant. "Update the env docs when you add a variable" is a rule everyone agrees with and
nobody performs on the day they're chasing a bug. The check costs one CI step and never forgets.
┌──────────────────────────────────────────────────────┐
│ reminder said in a conversation │
│ lives for one session │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ rule written in an instruction │
│ works while people read it │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ check a linter, a failing test, a build step │
│ works whether or not anyone remembers │
└──────────────────────────────────────────────────────┘
Not everything deserves this. The rung is worth climbing when the drift is silent, frequent and
cheap to detect mechanically - which is the exact profile of a generated inventory of anything.
What it costs
Nothing here is free, and three of the costs are real:
Another CI step, and a binary to build. The check isn't "run a script" - it resolves a version,
fetches the platform at that ref, and builds a binary before it can compare anything. That's build
time on every pull request that touches a .go file, which is most of them.
Red builds over a file that "is only documentation". The first time someone's pull request goes
red because a config struct gained a field and the YAML didn't, the reaction is that the check is
pedantic. It is pedantic. That's the feature, and it costs some goodwill until the habit of running
the local script settles in.
Resource variables stay on a hand-maintained table. The generator can't see them, so the most
mechanical-looking names in the whole set - the ones a convention produces - are the ones still
kept by hand. The test covers presence and absence of names; it does not make the table generated.
That's a gap I know about and haven't closed.
The one conclusion
If a list is derivable from the code, generate it and let the build enforce it. The value isn't the
list - it's that the list can't quietly stop being true.
That's my experience on one service and my price for it. Three ways you might be reading this and I
want all three: you do this better - tell me what your generator covers that mine doesn't;
you've been through it - tell me what broke, especially around version pinning; you see it
differently - tell me why a check here is over-engineering.
How is this solved where you work, and what broke when it wasn't?
Operations out of the box - Part 2.
Next: the metrics a service exposes without a single line written for them - what's in the default
set, and the second machine-generated snapshot that keeps it from drifting.
Top comments (0)