Think about a hostel or PG. When you check out, you're supposed to tell the warden, and the warden marks your room as free. Simple system. Works fine until the day it doesn't.
Say you check out in a hurry. You mean to tell the warden, but you get a call, you rush off, and you forget. Now here's the problem: the warden still thinks you're in that room. Nobody told them otherwise. The room sits "occupied" on paper, even though it's actually empty.
A new student comes looking for a room. The warden says "sorry, full" even though there's a perfectly empty room sitting right there. All because one message never got delivered.
This exact thing happens in software all the time. And most people don't notice it until it causes a real problem.
The pattern: "tell them" vs. "check yourself"
In most systems, when something is done using a resource a server, a slot, a seat, whatever the usual approach is:
"When you're done, tell the system so it can free you up."
This is called a push you push a message saying "I'm done, release me."
It sounds reasonable. And it works most of the time. But it has one weak point: it only works if the message actually arrives.
What if:
- The app crashes right before it sends the message?
- The network hiccups for one second?
- The other system is busy and drops the message?
None of these are rare. They happen constantly in any system with enough traffic. And when it happens, you get exactly what happened with our hostel example: something that's actually free, still marked as occupied. Forever. Because nobody told the system otherwise, and the system has no reason to ever ask.
Why this is sneaky to catch
The annoying part is that this bug hides really well.
If you look at the "occupied" resource from the outside, it looks completely normal. It's not doing anything wrong. It's not crashing. It's not slow. It just... sits there, marked as busy, when it isn't.
Your first instinct is usually to assume the numbers are correct "the dashboard says it's full, so it must be full." But sometimes the dashboard is just repeating a lie nobody ever corrected.
The fix: stop waiting to be told, go check yourself
The real fix isn't to make the "tell them" message more reliable (you can retry it, sure, but you can never make it 100% guaranteed networks fail, that's just reality).
The better fix is to add a second habit: every so often, actively go check what's actually true, instead of only trusting what you were told.
Back to the hostel: instead of only relying on students to report checkout, imagine the warden also does a quick round every evening actually walks the hallway, checks which rooms are truly empty, and fixes the register if it's wrong. Even if a student forgot to report checkout, the warden's own round catches it within a day.
In software, this is usually a small background job that runs every few minutes:
- Look at everything the system currently thinks is "in use."
- Actually verify is this still true, right now?
- If not, correct it.
This is called reconciliation not a fancy word, just "double-check what you believe against what's real, on a regular schedule."
The one rule that makes this safe
There's one important rule when you build this kind of self-checking loop: if you're not sure, don't touch it.
If your checking system can't get a clear answer maybe the other service is down, maybe it timed out the safe move is to leave things as they are and try again next round. Acting on an unclear answer is how you accidentally free up something that's actually still in use, which is a much worse mistake than leaving a false "occupied" mark for a few extra minutes.
So the loop should really be:
- Clear "yes, still in use" → leave it alone.
- Clear "no, not in use anymore" → free it up.
- Unclear/can't tell → do nothing, check again next time.
The takeaway
Any time your system has a "please tell me when you're done" step, ask yourself: what happens if that message never arrives?
If the answer is "it just stays wrong forever," you don't have a bug yet you have a bug waiting for the day traffic is high enough, or a network blip happens at just the wrong moment. Adding a simple, regular "go check yourself" habit is usually all it takes to fix it for good, not just once.
Top comments (0)