Everything worked.
Which, in hindsight, was the problem.
I had just shipped a small backend change. The kind you barely think about. Tests passed. Local setup was green. I even did that extra manual check we all pretend to always do.
A few hours later, production started acting… weird.
Not broken. Not down. Just off.
Some requests failed. Others succeeded. Refresh the page and the result changed. At one point I honestly wondered if I was accidentally load testing my own sanity.
My first thought was data. Then traffic. Then timing. Then maybe I had angered the JavaScript gods.
I added logs. Lots of logs. The kind you swear you’ll remove later.
Nothing obvious showed up.
That’s when I noticed something small and extremely annoying. One configuration value was undefined in production but perfectly fine on my machine.
I stared at it longer than I’d like to admit.
I had assumed my environment variables were the same everywhere. They weren’t. Locally, I had an old config file quietly saving me. In production, that variable simply did not exist, and my code reacted to that fact with chaos.
Once I knew that, the fix was almost boring. Add validation. Set a default. Deploy again.
The real bug wasn’t the code. It was my assumption that environments behave politely and consistently.
Since then, whenever a bug feels random, I start by asking a simple question: what am I assuming is “obviously the same” when it probably isn’t?
It saves time. And a small amount of dignity.
Top comments (4)
This is a good reminder that production is a very different environment.
I like that this focuses on the thinking process instead of just the fix.
It is interesting how often the problem is not the code itself but the assumptions around it.
This is the type of post I would send to a junior dev to normalize mistakes.