The Bug That Only Happened in Production (and Only on Tuesdays)

A few months ago, I ran into one of those bugs that makes you question your sanity.

Everything worked locally.
Everything worked in staging.

But in production… some users couldn’t log in.

Not all users. Not consistently. Just sometimes.

🧩 The Symptom

We started getting vague reports:

“Login fails randomly”
“Works after refresh”
“Only happens sometimes”

No errors in logs. No obvious crash. Just silent failure.

Which is always worse.

🔍 First Clue

After digging into logs more carefully, I noticed something odd:

Failed requests had empty request bodies
But only on POST /login

That didn’t make sense. The frontend was definitely sending data.

🧪 Reproducing the Issue

Tried everything:

Different browsers ✅
Incognito mode ✅
Slow network simulation ✅

Nothing.

Then one teammate casually said:

“I saw it once on mobile.”

That changed everything.

📱 The Breakthrough

I tested on mobile + weak network.

Boom. Reproduced.

Now I could finally see what was happening:

Request was sent
But sometimes the body was missing on arrival
⚠️ The Real Problem

We were using a middleware like this:

app.use(express.json());

Pretty standard.

But here’s the catch:

👉 Our server also had a reverse proxy in front (NGINX).

And it was configured with a small client body buffer size.

When the network was slow:

Request body arrived in chunks
Proxy didn’t handle it properly
Backend received… nothing

No error. Just an empty body.

🛠️ The Fix

We updated the proxy config:

client_max_body_size 10M;
proxy_request_buffering on;

And suddenly:

No more empty requests
No more random login failures
🤯 Why It Was So Hard to Find

Because:

It only happened on slow networks
Only affected certain users
No clear errors anywhere
Everything worked perfectly in dev

Classic production-only bug.

🧠 What I Learned

1. “Works locally” means nothing

Production has:

Proxies
CDNs
Real network conditions

Your laptop has none of that.

1. Silent failures are the worst

If something can fail, log it.

We added:

if (!req.body || Object.keys(req.body).length === 0) {
console.warn("Empty request body detected");
}

That alone would’ve saved hours.

1. Always question the layers in between

Frontend → Proxy → Backend → Database

The bug wasn’t in our code.
It was in the space between systems.

🚀 Final Thought

The trickiest bugs aren’t the ones that crash your app.

They’re the ones that:

almost work
sometimes fail
and leave no trace

If you ever see something “random” in production…

It’s probably not random at all.