DEV Community

Cover image for The NGO App's AI Worked Locally. Then Vercel Gave Me a 504.
nishchal singh
nishchal singh

Posted on

The NGO App's AI Worked Locally. Then Vercel Gave Me a 504.

#nextjs #vercel #ai #debugging

I was testing the AI assistant locally, and everything was working.

Then I deployed it.

I opened the production app and asked:

how start meditation. im a beginner

Instead of an answer, I got:

Unexpected token 'A', "An error o"... is not valid JSON
Enter fullscreen mode Exit fullscreen mode

At first, I thought I had broken something in the frontend.

I hadn't.

The error was hiding the actual problem

I checked the browser Network tab.

The request was:

POST /api/ai/chat
Enter fullscreen mode Exit fullscreen mode

and the response was:

504 Gateway Timeout
Enter fullscreen mode Exit fullscreen mode

So the JSON error wasn't the actual problem. The frontend was trying to parse an error response as JSON, while the request itself had already timed out.

That changed my debugging path.

Instead of looking at the JSON parser, I needed to find out why the server request was timing out.

But it worked locally

This was the confusing part.

The same AI request worked on my local machine.

The local terminal showed:

POST /api/ai/chat 200 in 21994ms
Enter fullscreen mode Exit fullscreen mode

The logs showed:

AI context retrieved
model: openai/gpt-5-mini
Enter fullscreen mode Exit fullscreen mode

The AI completion itself took around 17.76 seconds.

The usage was:

promptTokens: 1663
completionTokens: 1500
totalTokens: 3163
finishReason: "length"
Enter fullscreen mode Exit fullscreen mode

So the complete local request was taking roughly 22 seconds, but it still succeeded.

That was an important clue.

The model wasn't simply broken.

OpenRouter wasn't obviously unreachable.

The application could make the request and receive a response.

Something was different in production.

Vercel gave me the useful error

I checked the Vercel production logs.

This time, the error was much more useful:

FUNCTION_INVOCATION_TIMEOUT
Enter fullscreen mode Exit fullscreen mode

And there was a number that immediately stood out:

Execution Duration / Maximum
10.37s / 10s
Enter fullscreen mode Exit fullscreen mode

The logs also showed that parts of the request had already completed:

Clerk authentication succeeded
AI context retrieval succeeded
POST request to OpenRouter was made
AI completion did not return before the function was terminated
Enter fullscreen mode Exit fullscreen mode

So the function wasn't failing immediately.

It was running out of time while waiting for the AI completion.

That explained the 504.

Vercel settings fluid

Where was that 10-second limit coming from?

I checked the repository first.

There was no:

vercel.json
Enter fullscreen mode Exit fullscreen mode

There was no route-level:

maxDuration
Enter fullscreen mode Exit fullscreen mode

There was no runtime override.

And there wasn't a Next.js configuration setting a 10-second function duration.

So I hadn't configured this timeout in the application code.

I checked the Vercel project settings next.

That's where I found it.

Fluid Compute was disabled.

Screenshot: Vercel Functions settings showing Fluid Compute disabled

That was the missing piece.

The fix

I enabled Fluid Compute in the Vercel project settings and redeployed the application.

After the redeployment, the execution numbers changed from:

10.37s / 10s
Enter fullscreen mode Exit fullscreen mode

to:

26.31s / 5m
Enter fullscreen mode Exit fullscreen mode

The AI request could now finish.

The browser displayed the generated response instead of the 504.

The production logs also showed the rest of the request completing:

conversation_save_start
conversation_save_end
usage_persist_start
usage_persist_end
ai_chat_request_end
Enter fullscreen mode Exit fullscreen mode

The application recorded:

totalElapsedMs: 25384
Enter fullscreen mode Exit fullscreen mode

Vercel reported approximately:

26.31s
Enter fullscreen mode Exit fullscreen mode

And the API returned:

200
Enter fullscreen mode Exit fullscreen mode

Screenshot: successful production request / Vercel execution metrics

So the first production problem was fixed.

One important clarification

I don't want to turn this into the wrong Vercel lesson.

I'm not saying:

"Vercel cannot run AI functions longer than 10 seconds."
Enter fullscreen mode Exit fullscreen mode

That's not what I observed.

What I observed was that this deployment was running with a 10-second maximum while Fluid Compute was disabled.

After enabling Fluid Compute and redeploying, the same function was allowed to run for up to 5 minutes, and the request completed.

The fix wasn't making OpenRouter faster.

It removed the execution limit that was killing the request before the AI response could return.

But then I noticed something else

The 504 was gone.

The AI was working in production.

So I could have stopped there.

But the successful request was still taking around 25–26 seconds.

The local request was around 22 seconds.

So fixing one problem exposed another one.

I don't know the cause yet, and I don't want to guess.

Instead, I added temporary timing logs around different parts of the request:

authentication
rate limiting
quota reservation
database connection
conversation lookup/create
AI context retrieval
OpenRouter request start
OpenRouter response headers
OpenRouter full response
conversation save
usage persistence
total request duration
Enter fullscreen mode Exit fullscreen mode

Now I'm measuring each part instead of assuming where the time is going.

That's the next problem.

What I learned from this

Don't trust the first error

The browser showed a JSON parsing error.

The actual problem was a 504.

The Network tab gave me the next clue.

Local success doesn't guarantee production success

Locally, the request took around 22 seconds and worked.

Production was terminating it after 10 seconds.

The application code wasn't the only thing that mattered. The execution environment mattered too.

Check the platform before changing your code

There was no 10-second timeout configured in my repository.

The useful information was in the Vercel runtime logs and project settings.

Don't immediately blame the AI provider

The request reached OpenRouter, and the same model worked locally.

There wasn't enough evidence to blame the provider.

Don't blindly increase timeouts

First find out which layer is timing out.

In this case, the function itself was being terminated before the AI completion could return.

Measure again after every fix

Enabling Fluid Compute fixed the 504.

It also made the next problem visible:

Why is the request still taking ~25 seconds?

Measure first. Change second.

That's probably the simplest way I can describe this debugging session.

I started with a strange frontend error.

Then I checked the Network tab.

Then the production logs.

Then the repository configuration.

Then the Vercel project settings.

Each step removed another assumption.

Eventually, the 504 made sense.

The actual fix was only one setting.

Finding that setting was the debugging work.

The application is live here if you want to have a look:

https://www.pssmumbai.org/

Part 2: Why is the AI still taking ~25 seconds?

The timeout is fixed.

The request now completes successfully in production.

But waiting 25+ seconds for an AI response isn't something I want to leave alone.

So now I'm trying to find out where those seconds are actually going.

Is it the AI request?

Context retrieval?

Database work?

Something else?

I don't know yet.

That's what I'm measuring now.

Part 2 will be about finding that answer.

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev •

The instrumentation is the durable fix because it turns the next incident into a comparison rather than a new mystery. I would preserve a request correlation ID through the browser, function, provider call, and persistence stages, then define a small latency budget for each. That makes it clear when a timeout is configuration, queueing, model generation, or post-processing.