Add a service and you get code, observability and deploy. If the second half is done by hand after the first, the description of the system and the system itself drift apart — and you always find out later than it happened.
👋 I'm Anton — a software engineer working mostly in PHP/Symfony and Go, currently carving a live PHP monolith into Go services. This is the last part of a block about what a service gets for free the moment it exists. Notes and code: github.com/brilliant-almazov.
Here's the thought I want to leave the block on. It may be useful to you, or you may look at it from a completely different angle — both are interesting to me. As always: this is one codebase, my constraints, my price. Not a recommendation for yours.
The thesis
Generation and operations are not two neighbouring processes. They are one.
Add a service and you get the code, the observability and the deploy — in one motion, from one declaration. The alternative is not "generate the code first, wire up operations after". The alternative is that the environment-variable catalogue, the metrics snapshot and the pipelines get written by hand, some time after the code, by someone who remembers most of it.
And then the platform's description of the system stops matching the system. Not loudly. Quietly, one file at a time, and the discovery happens much later than the divergence.
Three ways it drifts
Every drift I've hit looks like one of these:
- A variable appeared, and the catalogue doesn't have it. The code reads it. The document that is supposed to list every variable this service needs does not mention it. The service starts on your machine and doesn't start in an environment nobody re-read the document for.
- A metric was deleted, and the dashboard still draws it. The panel is there, the line is flat, and flat reads as healthy until someone asks why it has been exactly zero since spring.
- A daemon was declared, and there's nothing to deploy it. The manifest says there are two binaries. The build job knows about one. The new one exists in the repository and nowhere else.
code description of the system
───────────────────── ─────────────────────────────
env var added ✗ catalogue missing it
metric removed ✗ dashboard still draws it
daemon declared ✗ nothing to deploy it
✗ = the gap nobody is told about
They have the same shape. In all three the code is right and the description of the system is wrong — and in all three you learn about it from production, not from the repository.
That last part is what makes this worth engineering around. A wrong description that fails loudly is a bug. A wrong description that fails silently is a habit: after the second time, nobody trusts the document, and then there is no point maintaining it at all.
What holds it together, part one: a single declaration
There is exactly one place where a service says what it is — its manifest:
service:
name: <service>
version: 0.1.0
daemons:
- name: server
handlers: [grpc]
- name: worker
handlers: [scheduler]
infra:
postgres:
- name: main
migrations: false
tx:
- name: main
pool: main
grpc:
- name: api
daemons: [server]
scheduler:
- name: main
daemons: [worker]
resources:
- type: messaging
name: main
That file is short, and almost everything operational is downstream of it.
Environment variable names are derived, not chosen. The convention is <TYPE>_<NAME>_<FIELD>, applied mechanically:
| Declared in the manifest | Variable the platform reads |
|---|---|
postgres: main |
POSTGRES_MAIN_DSN |
grpc: api |
GRPC_API_PORT |
messaging: main |
MESSAGING_MAIN_RABBITMQ_URL |
Once you know the rule, a double-looking name like OUTBOX_EVENT_OUTBOX_BATCH_SIZE stops being a typo and becomes readable: type OUTBOX, resource named event_outbox, field BATCH_SIZE. The scheduler resource carries the cron expression, so the service never reads a CRON_* variable of its own. The migration resource resolves its DSN from the pool's variable and has none of its own.
Pipelines are downstream of the same file. The build job builds per declared daemon out of a single Dockerfile with --build-arg BINARY=; the deploy job takes a version and which of the declared services to move. Nobody maintains a second list of binaries anywhere.
┌──────────────────────┐
│ the service manifest │
└───────────┬───────────┘
┌──────────────────┼──────────────────┐
▼ ▼ ▼
┌────────────┐ ┌─────────────────┐ ┌────────────┐
│ env names │ │ metrics snapshot│ │ pipelines │
└─────┬──────┘ └────────┬────────┘ └─────┬──────┘
▼ ▼ ▼
┌────────────┐ ┌─────────────────┐ ┌──────────────────┐
│ --check │ │ snapshot check │ │ build by declared│
│ │ │ │ │ daemons │
└────────────┘ └─────────────────┘ └──────────────────┘
drift check drift check the build itself
One declaration, one source. The point isn't elegance — it's that there is no second place for a human to forget.
What holds it together, part two: drift checks
A single source only stays single if something enforces it. So the derived artefacts are generated, committed, and then guarded.
The environment-variable catalogue
The catalogue is a generated file in the repository — every variable this service can read, where it came from, whether it's required, whether it's secret:
# AUTO-GENERATED by <snapshot tool>. DO NOT EDIT.
schema_version: 1
service: <service>
generated_at: 2026-08-16T01:24:46Z
vars:
- name: APP_ENV
source: platform
catalog: <platform package>
required: false
secret: false
help: prod/staging/dev label exposed in platform_info
- name: AUDIT_RECORDS_RETENTION_MONTHS
source: service_config
defined_in: internal/daemon/worker/retention/config.go:16
required: false
secret: false
59 variables — 45 declared by the platform, 14 by the service's own configuration. A service variable records defined_in with the file and line number; a platform variable records the catalogue package that owns it. So the answer to "where does this come from" is in the file, not in someone's memory.
Drift is a build failure. CI runs the same generator in --check mode on a change to any .go file, the manifest, the modules file, or the snapshot itself. If the regenerated catalogue differs from the committed one, the pull request is red.
One detail in that check took a while to get right, and it's the difference between the check working and the check lying: the generator is built at the same platform version the service's modules file pins. The version is resolved from the modules file, the platform is fetched at that ref, the binary is built from it. Skip that, and CI compares your snapshot against somebody else's version of the catalogue — and reports a difference that is not yours.
Locally there's a script that does the same regeneration and reverts the file if the only change is the generation timestamp. Timestamp-only noise in history is worse than no history.
And an honest gap in this one: resource variables don't make it into the snapshot. The platform builds their names by concatenation at runtime, while the snapshot reads configuration declarations. So resource names are documented in a table instead, and the snapshot's test asserts that the platform names are present and that forbidden names are absent — for example, the names of a broker this service does not use.
The metrics snapshot
The second machine-written artefact: name, type, help, labels, histogram buckets, source (platform or service), and where it's declared. 67 entries, including a <dynamic> record standing in for the dynamic-metric factory.
Drift in it fails CI too, by a separate check built the same way.
That snapshot is also where "a metric was deleted and the dashboard still draws it" gets caught: the deletion shows up as a diff in a reviewable file rather than as a flat line on a panel nobody is looking at.
The generated skeleton — specified, not running
The third check is designed and not built, so let me be precise about it.
The plan is that every generated file carries the Code generated by … DO NOT EDIT. header — that header is a contract, not decoration — and that a comparison mode returns a non-zero exit code when what the generator produces differs from what's committed. Editing a generated file by hand is forbidden by the same rule.
That work exists as a 12-iteration specification and a standing instruction not to start it without an explicit go-ahead. It is not a tool anyone can run today.
| Check | Watches | On mismatch | Status |
|---|---|---|---|
Env catalogue --check
|
any .go, the manifest, the modules file, the snapshot |
red build | running |
| Metrics snapshot check | metric declarations | red build | running |
| Generated-skeleton compare | files carrying the DO NOT EDIT. header |
non-zero exit | specified |
Drift checks are the whole mechanism. They are how "declared" and "actual" stay the same thing instead of two things that agree for a while.
Why this is one piece with the generator
The generator does not only produce code. It produces the things operations are assembled from: the declarations that environment variable names are derived from, the metric declarations the snapshot reads, the daemon list the pipelines build and deploy.
Which means you can separate "generation" from "operations" only by paying for it in manual synchronisation — of three artefacts, forever, by hand, each with its own way of being quietly wrong.
There's a small example that shows how quiet that failure can be, from the build side. The linker silently ignores -X pointing at a symbol that doesn't exist. The build is green, the version stays dev, and dev travels into the audit trail and into the trace attribute for as long as nobody looks. So there's a build target that compiles both binaries with a test tag and greps the binary for that tag; empty output means the symbol is wrong. The ldflags target has to be correct in both the Makefile and the Dockerfile — the template historically carried a dead target, and a dead target is exactly the kind of defect that never announces itself.
Same class of problem as the drift checks: something declares a fact, nothing verifies it, and the mismatch surfaces months later in a place you weren't looking.
The honest boundary
What this block described is not uniformly finished, and the difference matters more than the story:
in place still a plan
────────────────────────────────── ────────────────────────────────
the manifest - one declaration skeleton generator - 12 iterations
env catalogue - 59 variables, --check coverage threshold - 0 vs 86.7%
metrics snapshot - 67 entries wait-for-health step in deploy
three pipelines - build, tests, deploy
| In place | Still a plan |
|---|---|
| The manifest as the single declaration | The skeleton generator — a 12-iteration specification, not started |
The env catalogue: 59 variables, --check in CI |
The coverage threshold: the ratchet is 0 today |
| The metrics snapshot: 67 entries, its own CI check | A "wait for health" step in the deploy pipeline |
| Three pipelines: build, tests, deploy |
To spell out the ones that are easy to overstate:
- The skeleton generator is not written. There is a specification of 12 iterations and an explicit rule not to begin it without a direct instruction. Everything above about generated headers and compare mode is design, not a tool.
- There are no dashboard or alert files in this repository, and there won't be. The dashboard is assembled by a generator alongside the rest of the fleet's dashboards; the alert rules are described in text and live on the fleet's monitoring side.
-
The deploy pipeline has no "wait for health" step. The platform serves
/healthand/ready; checking after a rollout is the environment's job, not the pipeline's. -
The coverage threshold is
0. The ratchet exists and can only go up, but the bar hasn't been set. Actual line coverage is 86.7% — which is the number that makes leaving the threshold at0slightly embarrassing rather than harmless.
What it costs
Three drift checks are three places where a pull request goes red because of a file that looks like documentation. That reads as bureaucracy right up until the first time one of them catches a variable you added and forgot. It still reads as bureaucracy to whoever didn't add the variable.
A single declaration means there is nowhere to cut a corner. You can't add a quick variable "just for now" without the catalogue noticing, can't drop a metric without the snapshot noticing, can't add a binary the pipelines don't know about. That is the entire benefit, and it is also the entire complaint.
And the coupling earns its keep in proportion to how many services you have. On one service, generating the catalogue, guarding the snapshot and deriving pipelines from a manifest is more machinery than the problem deserves — you'd know all 59 variables by heart. The value shows up when the same shape repeats across services and nobody can hold any of them in their head.
For scale, the service these numbers come from is 2733 .go files across 252 packages, averaging 39 lines per file. Nothing about that is memorable by hand. That's the point.
The one conclusion
"Declared" and "actual" should match by construction, not by agreement. An agreement holds until the first hurried Friday; construction doesn't need anyone to remember.
That's my experience and my price for it, on one codebase, with the boundary above drawn as honestly as I can draw it.
If you do this better — if your platform derives more from one declaration than mine does — I'd like to hear how. If you've been through this and came out the other side deciding the checks weren't worth it, that's just as interesting. And if you look at it from a different angle entirely and think the coupling is a mistake, say so.
The question I'd actually like answered: how is this solved on your side, and what broke before you solved it that way?
Operations out of the box — Part 5. That closes this block: what a service gets the moment it's declared, and what keeps the declaration honest.
Next block: data and scaling — starting with treating a database as the unit of tenancy, and what that costs once there's more than one of them.
Top comments (0)