One visible complaint, six different causes - five of them outside the service being blamed.
👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go.
Most of what I do right now is carving a live PHP monolith into Go microservices, and much of that job is checking whether the extracted services behave the same way in production.
This series is about one question that keeps coming back from that work: why does this entity have no result? Running notes live on my GitHub: github.com/brilliant-almazov.
This is how I answer that question today, with the price attached. Maybe some of it transfers; maybe you look at it differently, and I'd like to hear how.
The complaint
Users open a screen and see nothing where they expect markup. The rule is configured, the result is empty.
The complaint arrives already addressed: the rule-set-markup service. One sentence, one accused service, no numbers.
It is a reasonable accusation: that service is the last link before the screen, and the last link is what the user sees.
The first way I counted, and why it was wrong
My first move was a sweep: run every object in the vertical through the endpoint and count the answers.
Roughly two-thirds of the objects answered not found. A further group failed on time. Read as a report card, that says: unusable.
It was wrong, and the reason is worth naming precisely. The sweep measured the endpoint, not the data.
Nobody had asked whether those objects were supposed to have a result at all. Not found is a truthful answer about an object that never had input. Counted as a defect, it becomes noise - and the handful of real failures drown inside it.
What the recount from the data showed
The second pass asked a different question of each object: does it have input, and if so, does it have output? That turns one complaint into arithmetic.
| What | Value |
|---|---|
| Configurations in the vertical | 276 |
| With a source snapshot | 76 |
| Of those, with a result | 76 |
| Snapshot rows / result rows | 433,348 / 2,239 |
| "Raw rows exist, no snapshot" | 0 |
| "Snapshot exists, no result" | 0 |
Those two zeros are the whole verdict on the accused service. Every object that had input had output.
Separately, a reference calculation of the same markup agreed with production on 1,752 entities with 0 discrepancies - so it is not producing merely something.
The empty screens are real, and the service blamed for them lost nothing. That is the actual shape of the problem.
Six causes behind one empty screen
The rest of the vertical does not collapse into a single explanation. It splits into named cohorts, and each has a different owner:
- 184 - the configuration's schedule collects a different type of data, one that produces no position tasks at all. Owner: the scheduler.
- 14 - the configuration has no keyword queries at all. Nothing to plan. Owner: the dictionary/catalog service.
- 1 - the schedule exists and is active, but has never fired once. Owner: the scheduler.
- 1 - tasks are created daily and stand in the provider's queue, waiting for collection. Queued, not lost. Owner: the provider-request service.
- 4 - the configuration's region is not covered by the provider at all. Decided by region alone. Owner: outside our system entirely.
- and the sixth, the only one belonging to the accused service - failures on time, which turned out to be a property of the pace of my own sweep rather than of the objects.
Five of the six sit before the service that received the complaint.
The chain, and where it breaks
The cohorts are addressable because the path from a schedule to a rendered result is a chain of links, and every link has an owner:
LINK OWNER EMPTY HERE
───────────────────────────────── ────────────────────────── ──────────────
schedule of the configuration scheduler 184 · 1
set of keyword queries catalog service 14
collection task tool-task service ·
task run + its counters tool-task service ·
raw result rows provider + raw-data service 1 queued · 4 region
snapshot of entities domain-rule-map service 0
applied rules → markup rule-set-markup service 0
rebuild of the result on request rule-set-markup service timeouts
Read the right-hand column and the complaint stops being a complaint. It becomes a distribution: most of the emptiness is decided at the first link, and the two links owned by the accused service carry a zero.
The honest mistake, and what it cost
Here is the part I got wrong. Leaving it out would make this a sales pitch.
The first conclusion I published from the sweep was: "23 objects don't rebuild." That was a defect report against the service, and it was false.
The failures on time came in batches - 20 to 23 of them, back to back rather than scattered. I took the same objects and ran them one at a time with a pause: 21 succeeded immediately, 2 after a pause.
The failure was a property of the pace I was driving the endpoint at, not of the objects.
The cost was a whole extra round of measurement.
The cause is not mysterious: a conclusion drawn from one run without a control case. One run tells you what happened; it does not tell you what the run itself contributed.
What the breakdown changes in the work
While "no result" is a single complaint, it cannot be closed and it cannot be handed to anyone. It sits on whichever service is nearest the screen, and every conversation about it restarts from zero.
Split into cohorts with an owner each, every part has an addressee: the scheduler, the keyword catalog, the provider, the processing service.
Some parts turn out not to be work at all. 184 configurations that were never meant to produce this type of result are a fact about the setup, not a backlog item.
What remains is small, specific, and belongs to someone by name.
That is the difference. Not "the service is fine", but "here is where each empty screen stops, and here is who owns that link."
What it costs
The breakdown took several runs and a lot of manual work, because the system does not answer "why is this object in this state" through its own interfaces.
There is no place to ask whether a configuration even has a schedule that produces this type of task, so the answer is assembled by hand, one link at a time.
Until the system returns those answers itself, every new breakdown starts by clearing the noise from the previous one - a cost paid again on every investigation.
The part that generalises
A mass sweep is cheap now. Any executor can push a whole vertical through an endpoint before lunch.
The conclusion drawn from it is not cheap. "23 objects don't rebuild" was produced quickly and was wrong, and being wrong quickly cost more than being right slowly would have.
The discipline that matters is not speed. It is making the conclusion checkable without re-running the thing: a control case next to the suspect, one number per link, and a stated rule for what counts as a normal empty. Speed multiplies whatever discipline is already there - including its absence.
This is my experience and my price, on one vertical, in one system halfway through a migration.
So: maybe you do this better - maybe your services already answer "why is this empty" themselves. Maybe you have been through this and found where it stops being worth the effort. Maybe you see it differently and think the sweep was the right instrument all along.
How is this solved on your side, and what broke there?
Why there is no result - Part 1.
Next: the chain of links as an instrument - one cheap number per link instead of guessing which service to accuse, and the eight cohorts any case reduces to.
Top comments (0)