Eliminate the Impossible

I had intended to write a post about one of the key techniques of debugging effectively, but circumstances this week prompted me to focus on something different because it is a critical "power move" to getting to the root of any problem.

It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. – Sherlock Holmes, The Adventure of the Beryl Coronet

It is human nature to be wedded to our ideas. That is just the way we are wired up, and it is very, very difficult to let go. Debugging, in no small part, is about letting go of preconceived ideas, and simply collecting facts and observations. Those facts and observations should then guide you to the truth, either directly or by pointing you to other things you can try to hack away at the problem.

This is not unlike the detective work performed by our friend, Mr. Sherlock Holmes.

And his maxim is spot on.

In a particularly difficult debugging problem this week, we kept hitting a wall: _ this _ simply could not be happening this way, given the facts that we know. So we kept trying to eliminate options, validate facts, and reconcile conflicting observations until we uncovered the single nugget that had to be true. And, from that conclusion, we uncovered the mystery: a foundational "fact" on which a lot of investigation was based was untrue.

Just the facts

Facts are a tricky thing, because unless you are really precise, they can be:

1. Obscured by language. "Oh, you mean _ that _? I really meant _ this _."
2. Obscured by bias. "I am not quite sure what that means, but it would support my case if it meant _ this _."
3. Obscured by omission. "Yes, I checked _ that _ (but didn't check this or the other , making _ that _ irrelevant.)"

I have found, by being burned many, many times, that the only fact you can trust is the fact you see with your own eyes. Even if it is from someone you trust, ask them to show you how they got that fact. Agree on the fact. Then move on. If you cannot agree on the fact, dig deeper. Ask questions. If the fact is important enough to investigate, then it is important enough to ascertain its correctness. Yes, this can be a bit tedious. And yes, some people are put off by being challenged on their description of the facts. But by being rational and non-judgemental, most people will appreciate the precision of a methodical approach.

Being wedded to ideas

Getting back to being wedded to our own ideas, how many times have you looked at your software and said, "I know _ this part_ works, so let's look somewhere else" while debugging a particularly hard problem. Does it really work? How sure are you? A complex problem needs to be broken down into its parts, doing what amounts to a binary search of the major components to isolate where the issue likely is. If you are wedded to something you wrote or an idea you have, you can be tempted to shortcut the search process in an attempt to speed up getting to the root. In my experience, this can lead to wasted time as what you skip ends up being material to the solution.

I opened this post by saying that I would defer discussing a key debugging technique until next time, but I think post really is about this key question:

If the code or system used to work, but now does not, what has changed?

The "what has changed" is about reconciling facts to match your observations. They don't match, so what is different? While the meaning of "what has changed" is generally about uncovering differences that make a working system now broken, it is useful to ask: did the system work before, the way you think it did? How do you know?

Question everything.

So, have I solved the problem from this week? Not entirely, but now there is a crystal clear path forward that was, previously, obscured by a hidden fact. We have the truth, and now simply need to execute on it. The improbable nature of that fact is not without its own repercussions, but that is another story.