DEV Community

Anton Brilliantov
Anton Brilliantov

Posted on

Processing and Completeness Are Different Problems

One verdict for two aspects keeps a closed task open forever.


👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go, currently carving a live PHP monolith into Go services.

This series is about the complaint that keeps landing on that migration: a user sees no result, and one service gets blamed by default. This part is about the last step of such a breakdown - writing down what was actually solved.

Running notes live on my GitHub: github.com/brilliant-almazov. It is how I write a verdict today, not a methodology - maybe you already do it better, maybe you'd do it another way entirely.


The thesis

Mixing two aspects into one verdict turns a closed task into a permanently open one.

Everything below is a consequence of that.

The two aspects are processing and completeness. They come out of the same breakdown and get written into the same report, and they are not the same problem.

Two definitions

Processing - how the system computes and answers on the data it already has: rules applied, the result rebuilt, the response deadline, concurrency, response codes.

Completeness - whether the data reaches the system at all: the schedule, the scheduler, the provider's queue, the region coverage.

┌──────────────────────────────────┐   ┌──────────────────────────────────┐
│ PROCESSING                       │   │ COMPLETENESS                     │
│ how it computes on the data      │   │ whether the data arrives at all  │
│ it already has                   │   │                                  │
├──────────────────────────────────┤   ├──────────────────────────────────┤
│ rules applied                    │   │ schedule                         │
│ rebuild of the result            │   │ scheduler                        │
│ response deadline                │   │ provider queue                   │
│ concurrency                      │   │ region coverage                  │
│ response codes                   │   │                                  │
├──────────────────────────────────┤   ├──────────────────────────────────┤
│ owner: the processing side       │   │ owner: the collection side       │
└──────────────────────────────────┘   └──────────────────────────────────┘

no arrows: neither side moves the other column by working harder
Enter fullscreen mode Exit fullscreen mode

Two columns side by side, processing on the left with rules applied, rebuild, deadline, concurrency and response codes, completeness on the right with schedule, scheduler, provider queue and region coverage, each column under its owning role, no arrows between them

No qualifiers on either definition.

A fact belongs to one column or the other, and one question decides which: does the system have the data yet?

Why one number can't hold both

The columns have different owners and different repairs.

The team that owns processing cannot speed up somebody else's collection queue, and cannot create a schedule on a user's behalf.

The team that owns collection cannot fix a response deadline.

Neither of them moves the other column by working harder on its own.

So a single "percent solved" over both averages two scales. Work on either side barely moves it. It stops being a signal and becomes a thing people argue about in the standup.

What it looked like in one breakdown

Processing. Every entity that had source data had a result: 76 of 76.

A reference computation run beside production agreed with it on 1 752 entities, with 0 discrepancies.

The two states where the accused service would actually be losing data - raw rows in with no snapshot, snapshot in with no result - came back 0 and 0.

That is a verdict, and the verdict is solved. Where there was input, there was output, and the output matched an independent computation of it.

Completeness. 21 configurations were waiting for collection. 184 are scheduled for a different type of collection entirely. 14 have no keyword queries to collect for.

Those are facts, and they stay facts. They are not a percentage of anything, and none of them says the computation is wrong.

The shape of the verdict I write now

At the bottom of every breakdown, one line per aspect:

processing:   solved - 76/76 with data have a result; 1 752 entities, 0 discrepancies
completeness: 21 awaiting collection, 184 other schedule type, 14 without queries
              owner: the collection side
Enter fullscreen mode Exit fullscreen mode

Two verdict lines, processing marked solved with 76 of 76 entities with data having a result and 1 752 entities matched with 0 discrepancies, completeness listing 21 configurations awaiting collection with the collection side named as owner

The owner sits on the completeness line on purpose. A number with no address is a number somebody has to chase.

The reader of a report should not have to derive the verdict. They should be able to read it.

Everything above those two lines is the evidence. The two lines are the answer.

What one verdict would have done

The task would have been "partially solved" - and it would have stayed that way.

Follow it through. Under a single verdict, the task closes only once the missing data arrives.

That data arrives when a schedule fires, a queue drains, a region gets covered. None of those sit with the team that owns processing.

So the task closes on somebody else's timeline, which in practice means it does not close.

ONE VERDICT                            TWO VERDICTS
one number over two aspects            two markers, no line between them

┌──────────────────────────────┐       ┌──────────────────────────────┐
│ partially solved             │       │ processing - closed          │
└──────────────────────────────┘       └──────────────────────────────┘
               │
               ▼
┌──────────────────────────────┐       ┌──────────────────────────────┐
│ closes only when the missing │       │ completeness - waiting,      │
│ data arrives                 │       │ owner named                  │
└──────────────────────────────┘       └──────────────────────────────┘
               │
               ▼
┌──────────────────────────────┐       each closes on its own owner's
│ it arrives on somebody       │       timeline
│ else's timeline              │
└──────────────────────────────┘
               │
               ▼
┌──────────────────────────────┐
│ so it comes back at every    │
│ review                       │
└──────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

One verdict on the left as a progress bar stuck at partially solved with an arrow looping back into reopened, two verdicts on the right as two independent markers, one closed and one waiting, with no line between them

This isn't status bookkeeping. An open task has a running cost: it comes back at every review, it gets discussed again, and the breakdown gets requested a second time by someone who wasn't in the first conversation.

That is paid weekly, forever, for a computation that was already correct.

Where the split stops being honest

The split is honest only while processing genuinely does not affect completeness.

If the system loses data on its own leg - rows arrive and no snapshot is taken, a snapshot exists and no result is produced - that is processing, however much nicer it would be to hand it outward.

So the split comes with a check, and the check gets named out loud: the two loss states must both be zero.

Here they were 0 and 0, and that is the only reason the completeness column was allowed to belong to someone else. If either one is non-zero, there is no conversation about splitting aspects - there is a defect in processing, and it is mine.

What it costs

Two verdicts mean two addressees and two discussion threads, where one line used to do.

The reader has to be walked through the difference once, and for a while afterwards the two still get merged in conversation - usually by whoever quotes the number without the line above it.

I pay that, because the alternative is a task that never closes.

The part that generalises

An executor of any speed returns a verdict in the shape it was asked for.

Ask for one number over two aspects and you get one number over two aspects, delivered faster. The averaging happens either way; it just happens sooner.

The shape of the verdict is part of the assignment, not a matter of taste for whoever is reporting. It decides whether the work can ever be called done.


This is how I split a verdict right now, and the price of it is written above. I'd like to hear the other side: if you do this better, tell me how you separate what your system computes from what it receives; if you have been through this, tell me which aspect ended up owning the never-closing task; if you see it differently, say where drawing this line does more harm than the merged verdict. How is it solved on your side, and what broke there?


Why there is no result - Part 4.

Next series: what an executor has to be given before a production check is repeatable - the surface, the exact shape of every request, the domain conditions no read exposes, and the difference between "not checked" and "passed". It starts in what an agent needs to test production.

Top comments (0)