The Debugging Approach That Saved a Deployment

Karan Padhiyar — Wed, 06 May 2026 07:36:52 +0000

We had a rollout where requests in one client environment started timing out under load.

Not locally. Not in staging. Only in their infra.

At first, everything looked normal. No crashes. No clear errors. Just slow requests piling up until the system started struggling.

The obvious move would have been to add more logs and redeploy.

We didn’t do that.

When your system runs continuously, every redeploy is a risk. You don’t push changes blindly. You need to understand the problem before touching anything.

So instead of changing code, we changed how we looked at the system.

Start With One Request

Instead of scanning logs randomly, we focused on a single request and followed it through the system.

That changed everything.

What we saw was simple:

The API received the request instantly
An internal service call was taking several seconds
The downstream AI layer was fast

So the problem wasn’t external. It was inside our own system.

Break Down the Time

Looking at request-level logs wasn’t enough.

We needed to know where time was actually being spent.

Once we broke things down step by step, the issue became clear.

A database operation that normally takes milliseconds was taking multiple seconds under load.

No errors. Just delay.

The Real Issue

This wasn’t a slow query problem.

It was a resource problem.

The database connection pool was getting exhausted.

In this client environment, the limits were lower than what we had assumed. Under load, requests weren’t failing. They were waiting.

That’s why nothing looked broken. Everything was just slow.

Fixing It the Right Way

We could have increased the connection pool size and moved on.

That would have created new problems later.

Instead, we changed how the system handled load.

We limited how many requests could run at the same time.

We added control so requests didn’t pile up endlessly.

We made the system aware of environment limits instead of assuming defaults.

What Changed

After that:

Requests stopped piling up
Latency stayed stable under load
No infrastructure changes were needed from the client

What Actually Worked

The fix wasn’t adding more logs.

It wasn’t redeploying faster.

It was understanding the system properly.

Track one request completely
Measure time at each step
Find the real bottleneck before changing anything

When your system runs 24/7, debugging is different.

You don’t guess.

You prove where the problem is, and then you fix only that.

WebSockets + AI pipelines - why real-time AI breaks more than you expect

Karan Padhiyar — Fri, 01 May 2026 09:41:31 +0000

Real-time AI feels simple when you use it.

You type something - it responds instantly.

You ask a question - it streams the answer live.

From the outside, it looks smooth. Almost effortless.

But that experience hides a different reality.

Most systems on the internet are short-lived.

You send a request.

You get a response.

The connection ends.

Real-time AI doesn’t behave like that.

The connection stays open.

The system keeps running.

The response is not a single event - it’s a continuous flow.

That small difference changes how everything works underneath.

Now think about normal user behavior.

People close tabs randomly.

Internet drops without warning.

Apps get minimized mid-response.

Nothing unusual.

But the system doesn’t always know that the user is gone.

So it keeps going.

The AI keeps generating.

The backend keeps processing.

Resources keep getting used for something no one will ever see.

This is where things start to matter.

AI responses are not cheap.

Every response uses compute.

Every second of processing has a cost.

If even a small percentage of users leave mid-way, the system starts doing unnecessary work at scale.

You don’t notice it immediately.

There’s no sudden crash.

Instead:

performance slowly drops
costs quietly increase
behavior becomes inconsistent

That’s harder to detect and harder to fix.

Real-time AI is not just a feature.

It’s a continuous system.

You’re not just answering users anymore.

You’re maintaining a live interaction that can break at any moment.

And when it breaks, it often doesn’t tell you.

That’s the gap most people underestimate.

The difference between something that works once and something that keeps working all the time.

A demo shows the experience.

A real system deals with everything that interrupts that experience.

Real-time AI feels instant.

But what really defines it is not speed.

It’s how the system behaves when the user disappears.

DEV Community: Karan Padhiyar