Keeping My Website Chatbot Under a 9-Second Serverless Timeout

Tired of website chatbots timing out and losing potential customers while spinning on a slow API? When I added an interactive assistant to dreamfuturestech.com, my serverless functions kept slamming right up against provider execution limits. Users hate staring at a loading spinner for ten seconds just to get a basic greeting. In my experience running my own business site and local automation server here in Pompano Beach, relying on heavy external AI endpoints without strict boundary checks is a guaranteed way to drop traffic.

Serverless platforms are cheap and scale well, but their strict timeout windows punish slow code. If an API request hangs, your function hangs with it. To fix this on my own infrastructure, I had to architect a resilient request pipeline that cuts off slow responses, switches to lighter models, and falls back gracefully when things go sideways. Here is the exact approach I took to keep my chatbot snappy and reliable.

Understanding the Serverless Timeout Bottleneck

Most serverless function providers enforce a strict execution window—often 10 seconds for standard tiers. When a visitor types a message into a chat widget, the browser sends an HTTP POST request to your backend function. That function then packages the prompt, sends it to a third-party language model API, waits for the completion, parses the JSON, and sends the reply back to the browser.

If the AI provider experiences high latency, your function crosses that 10-second threshold. The gateway returns a generic 504 Gateway Timeout error, the client-side widget breaks, and the user assumes your site is broken. In my judgment, you cannot trust any external API to respond within an acceptable window 100% of the time. You must build your code defensively.

Swapping to a Faster, Lightweight Free Model

My first optimization was ditching large, general-purpose models for routine chat interactions. For a standard business website, visitors usually ask predictable questions: pricing, service areas, booking links, and contact info. They do not need a massive model that takes seconds to reason through complex philosophy.

I evaluated a few options that are free to call via developer tiers and found that smaller, distilled models return text significantly faster. Cutting down the parameter size reduced the average time-to-first-token drastically. Even with network latency factored in, the response payload returns well inside a safe window.

Forcing a Strict HTTP Timeout in Code

Speed alone is not enough; you need code enforcement. If a model hangs, your function needs to abort immediately rather than waiting for the platform's hard kill. I use the native AbortController in Node.js to enforce a strict fetch timeout.

const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 4000);

try {
  const response = await fetch('https://api.example.com/v1/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
    signal: controller.signal
  });
  clearTimeout(timeoutId);
  // Process response
} catch (error) {
  if (error.name === 'AbortError') {
    console.log('AI request timed out, switching to fallback');
  }
}

By capping the external fetch at 4 seconds, I guarantee that my serverless function still has plenty of headroom to format an alternative response and return a clean 200 OK to the browser.

Building a Keyword-Based Fallback Handler

When the AI API fails, times out, or returns an error, the chatbot should never simply crash or stay silent. I implemented a deterministic keyword matcher as a fallback layer. If the fetch fails or aborts, the server parses the user's input string for common terms like "price", "hours", "contact", or "pompano".

function getFallbackResponse(userInput) {
  const text = userInput.toLowerCase();
  if (text.includes('price') || text.includes('cost')) {
    return 'Our pricing depends on project scope. Check out our services page for details!';
  }
  if (text.includes('contact') || text.includes('email')) {
    return 'You can reach us directly via the contact form on this site.';
  }
  return 'I am having trouble connecting to my primary knowledge base right now, but feel free to drop us a direct message!';
}

This ensures the user always gets an instant, helpful reply, turning a potential technical failure into a smooth conversational pivot.

Putting It All Together in My Serverless Function

Combining these patterns into a single endpoint transformed the chat experience on my site. The function receives the payload, fires the fast model with an aggressive abort timer, and catches any failures to immediately serve the keyword-backed fallback response.

Performance is a feature, especially for small business websites where every second of delay increases bounce rates. By respecting serverless constraints and designing for failure from the start, the chat widget on dreamfuturestech.com remains responsive, lightweight, and completely dependable.

I build sites and automations for small businesses at dreamfuturestech.com.