DEV Community

Cover image for The Float Is a Liveness Probe: Four Failure Modes
Den
Den

Posted on

The Float Is a Liveness Probe: Four Failure Modes

I cook pelmeni about twice a week, and for years I got inconsistent results from what is, on paper, a trivial process: put frozen parcels in boiling water, take them out when done. Two failure modes kept recurring — parcels welded to the bottom of the pot, and parcels that burst and leaked their filling into the water.

Both turned out to be the same class of mistake I make at work, which is why I'm writing this down.

Failure one: the batch was too large

Dropping a large batch of frozen parcels into boiling water stops the boil. The water is at 100°C, the parcels are at −18°C, and there's only so much thermal energy in the pot. The temperature drops below boiling, the surface starch doesn't set, and the dough touches the metal while it's still sticky. That's the welding.

I had been treating pot capacity as a volume question — will they fit — when it's actually a throughput question: how much cold can this system absorb before it stops doing the thing it's for. A pot that fits thirty parcels comfortably processes twenty of them correctly.

Capacity is not the same as the load at which quality holds. You can size a worker pool by what fits in memory and still be wrong, for exactly this reason: the system keeps accepting work long after it stopped doing it well.

Failure two: I trusted the wrong signal

Everyone knows the rule: pelmeni float when they're done. Everyone is wrong, and I was wrong for a long time.

They float when the air and steam trapped inside expand enough to make the parcel buoyant. That happens when the dough is hot. It says nothing about the filling, which is denser, colder, and at the centre of the parcel — the last part of the system to get warm. Pull them at the moment they surface and the outside is finished while the inside is raw.

Floating is a liveness signal. It tells you the process is running. It is not a readiness signal, and the difference between those two is exactly the difference between "the container started" and "the container can serve traffic". Confusing them ships raw meat.

The actual readiness signal is time after the liveness signal — a few minutes at a gentle boil once they've surfaced. Which is an unsatisfying answer, because it's a timer, and everybody wants an event.

When you can't observe the thing you care about, you time it from the last thing you can. That's not a workaround, that's the design. The mistake is pretending the observable proxy is the real state.

Failure three, which I still make

The lid. A lid gets you back to boiling faster after the cold batch goes in, which is good, and then the starch-laden foam climbs the sides and comes over the edge, which is not.

The correct move is to leave the lid slightly ajar — you sacrifice some of the recovery rate to keep the process inside the pot. This is a backpressure trade and it has the same shape everywhere: maximum throughput and stability are different set points, and choosing throughput means you will eventually clean the stove.

What the burst parcels were really telling me

The leaking-filling failure isn't overcooking in the ordinary sense. It's what happens when a parcel stays under a hard rolling boil: mechanical agitation, not heat, tears the seam. The fix isn't less time, it's less turbulence — drop the heat once the parcels are in and let it finish at a bare simmer.

I had been tuning the wrong parameter for months because the symptom appeared at the end of the process, so I assumed it belonged to the end of the process. It belonged to the environment the whole way through.

A failure that shows up last isn't caused last. Applies to dinner and to production incidents in roughly equal measure.

Failure four: I was salting for the wrong reason

I used to salt the water for flavour. It does that, but the more important effect is thermal: dissolved salt raises the boiling point substantially, so a well-salted pot runs hotter than a plain one and gets you through the cold-batch recovery noticeably faster. Two tablespoons per litre is roughly where the gain stops being worth the taste.

This is the one that changed my routine the most, because it means the salt goes in before the water boils rather than with the parcels.


So: batch smaller than capacity, don't trust the float, lid ajar, simmer not boil, salt early.

One of the four claims above is wrong — confidently stated, cheerfully believed by me for years, and demonstrably false. I know which one. I'm curious whether anyone catches it, and whether this kind of thinking about cooking is a useful frame or just professional deformation. What do you think?

Top comments (0)