Building an AI Chatbot with Triple LLM Fallback (Gemini Groq DeepSeek)

Jordy | PACIFIK'AI — Sun, 29 Mar 2026 03:21:13 +0000

Why One LLM Isn't Enough

If you're building an AI chatbot for a business, reliability is everything. A chatbot that goes down at 2 AM when a customer needs help is worse than having no chatbot at all.

After building MANA — our production chatbot serving businesses in French Polynesia — we learned this the hard way. Gemini went down for 3 hours on a Friday evening. Our client's restaurant was getting reservation requests. No one was answering.

That's when we built the triple fallback chain.

The Architecture

User Message
    │
    ▼
┌─────────────┐   fail    ┌─────────────┐   fail    ┌─────────────┐
│  Gemini 2.0  │ ────────▶ │  Groq Llama  │ ────────▶ │  DeepSeek V3 │
│  Flash       │           │  3.3 70B     │           │              │
└──────┬───────┘           └──────┬───────┘           └──────┬───────┘
       │ success                  │ success                  │ success
       ▼                          ▼                          ▼
   Response                   Response                   Response

Each provider has different strengths:

Provider	Latency	Cost/1M tokens	Free tier	Best for
Gemini 2.0 Flash	~200ms	$0.075	1500 req/day	Primary (fast + cheap)
Groq Llama 3.3	~100ms	$0.06	14,400 req/day	Speed fallback
DeepSeek V3	~500ms	$0.14	Pay-as-you-go	Cost fallback

Implementation (Next.js API Route)

Here's the core pattern we use in production:

// app/api/chat/route.ts
import { NextRequest, NextResponse } from 'next/server';

interface LLMProvider {
  name: string;
  call: (messages: Message[], systemPrompt: string) => Promise<string>;
}

const providers: LLMProvider[] = [
  {
    name: 'gemini',
    call: async (messages, system) => {
      const res = await fetch(
        `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${process.env.GEMINI_API_KEY}`,
        {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({
            system_instruction: { parts: [{ text: system }] },
            contents: messages.map(m => ({
              role: m.role === 'user' ? 'user' : 'model',
              parts: [{ text: m.content }],
            })),
          }),
          signal: AbortSignal.timeout(8000), // 8s timeout
        }
      );
      if (!res.ok) throw new Error(`Gemini ${res.status}`);
      const data = await res.json();
      return data.candidates[0].content.parts[0].text;
    },
  },
  {
    name: 'groq',
    call: async (messages, system) => {
      const res = await fetch('https://api.groq.com/openai/v1/chat/completions', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          Authorization: `Bearer ${process.env.GROQ_API_KEY}`,
        },
        body: JSON.stringify({
          model: 'llama-3.3-70b-versatile',
          messages: [{ role: 'system', content: system }, ...messages],
          max_tokens: 500,
          temperature: 0.7,
        }),
        signal: AbortSignal.timeout(8000),
      });
      if (!res.ok) throw new Error(`Groq ${res.status}`);
      const data = await res.json();
      return data.choices[0].message.content;
    },
  },
  {
    name: 'deepseek',
    call: async (messages, system) => {
      const res = await fetch('https://api.deepseek.com/chat/completions', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          Authorization: `Bearer ${process.env.DEEPSEEK_API_KEY}`,
        },
        body: JSON.stringify({
          model: 'deepseek-chat',
          messages: [{ role: 'system', content: system }, ...messages],
          max_tokens: 500,
        }),
        signal: AbortSignal.timeout(10000),
      });
      if (!res.ok) throw new Error(`DeepSeek ${res.status}`);
      const data = await res.json();
      return data.choices[0].message.content;
    },
  },
];

async function callWithFallback(
  messages: Message[],
  systemPrompt: string
): Promise<{ response: string; provider: string }> {
  for (const provider of providers) {
    try {
      const response = await provider.call(messages, systemPrompt);
      return { response, provider: provider.name };
    } catch (error) {
      console.warn(`${provider.name} failed:`, error);
      continue; // Try next provider
    }
  }
  throw new Error('All LLM providers failed');
}

Key Design Decisions

1. Timeout per provider (not global)

Each provider gets its own AbortSignal.timeout(). If Gemini hangs for 8 seconds, we cut it and move to Groq — which might respond in 100ms. A global timeout would waste time waiting.

2. No Promise.race()

We considered running all three in parallel and taking the fastest response. But that wastes API credits and complicates error handling. Sequential with fast timeouts is simpler and cheaper.

3. System prompt stays consistent

The same system prompt goes to all three providers. We tested and found that Gemini, Llama, and DeepSeek all handle our French/English bilingual prompts well. The responses are slightly different in style but consistent in quality.

4. Logging the provider used

We log which provider served each request. This lets us track:

Gemini uptime (usually 99.5%+)
Groq activation frequency (our canary for Gemini issues)
DeepSeek usage (if this spikes, something is wrong)

The System Prompt

For a business chatbot, the system prompt is everything:

Tu es MANA, l'assistant IA de [Business Name].
Tu aides les clients avec: [services list].
Ton ton: amical, professionnel, concis.
Langue: français par défaut, bascule en anglais si le client écrit en anglais.
Quand on te demande les prix: dirige vers [pricing URL].
Quand on te demande un RDV: propose [booking link].
JAMAIS inventer d'informations. Si tu ne sais pas, dis-le et propose de contacter l'équipe.

Multilingual Support

In French Polynesia, we need French, English, and sometimes Reo Tahiti (Tahitian). The system prompt handles this naturally — LLMs are surprisingly good at language detection and switching.

We add a few Tahitian greetings to make it feel local:

"Ia ora na!" (Hello!)
"Mauruuru!" (Thank you!)
"Nana!" (Goodbye!)

Results in Production

After deploying MANA for several businesses:

99.9% uptime over 3 months (the triple fallback has never fully failed)
< 2 second average response time
~$5/month average cost per business (mostly on the free tiers)
Handles French + English + Tahitian seamlessly

Open Source

We've open-sourced the widget component:
👉 github.com/jordy-pacifikai/mana-chatbot-widget

Built by PACIFIK'AI — AI & digital agency in Tahiti, French Polynesia.

How We Built an AI-Powered Digital Agency in Tahiti, French Polynesia

Jordy | PACIFIK'AI — Sun, 29 Mar 2026 03:16:30 +0000

From Paradise to Tech: Building PACIFIK'AI

When people think of Tahiti, they imagine turquoise lagoons and overwater bungalows. They don't usually think of AI chatbots and automation workflows. That's exactly why we started PACIFIK'AI — the first AI-native digital agency in French Polynesia.

The Problem We're Solving

Most businesses in the Pacific Islands are stuck in manual mode:

Customer inquiries answered one by one via phone or Facebook Messenger
Invoices, quotes, and follow-ups handled on paper or Excel
Websites that look like they were built in 2010
Zero automation, zero AI integration

Meanwhile, the rest of the world is moving to AI-first operations. The gap is widening.

Our Stack

We're building with modern tools optimized for speed and reliability:

Frontend: Next.js 14 + TypeScript + Tailwind CSS
3D & Animations: React Three Fiber + GSAP + Lenis
Backend: Supabase (PostgreSQL + Auth + Edge Functions)
AI: Claude API, Gemini, Groq — with triple fallback chains
Automation: Trigger.dev v4 for background jobs and workflows
Deployment: Vercel (Edge Network for Pacific latency)

What We Build for Clients

1. AI Chatbots (24/7 Customer Service)

Our chatbot MANA handles customer inquiries on WhatsApp, Messenger, and websites — in French, English, and even Reo Tahiti (the Tahitian language). It uses a triple LLM fallback: Gemini → Groq → DeepSeek, so it never goes down.

2. Premium Websites That Convert

We don't do templates. Every site is custom-built with:

Awwwards-level design and animations
Mobile-first (70%+ of Pacific traffic is mobile)
SEO optimized for local search ("agence digitale Tahiti", "création site web Polynésie")
Integrated contact forms → CRM → automated follow-ups

3. Automation Workflows

We automate the repetitive stuff:

Lead capture → CRM entry → automated email sequence
Invoice generation → payment tracking → reminders
Social media scheduling → analytics → reporting
Document extraction (AI reads PDFs, extracts data)

4. Digital Strategy

Full-stack digital marketing: SEO, Meta Ads, Google Ads, content strategy, email campaigns — all powered by AI for faster execution and better targeting.

The Tahiti Advantage

Being based in Tahiti isn't a limitation — it's a differentiator:

Time zone (UTC-10): We can deliver overnight for European and American clients
Bilingual: French + English native, critical for the Pacific market
Cost-effective: Lower operational costs than Paris or Sydney agencies
Unique perspective: We understand island business challenges firsthand

What's Next

We're working on:

Reo Tahiti AI: Preserving the Tahitian language through AI (NLP training data, translation models)
Open-source tools: AI business templates for Pacific Island entrepreneurs (check our GitHub)
MANA SaaS: Making our chatbot available as a white-label product

Want to Connect?

If you're building something similar in a remote location, or if you're interested in AI for emerging markets, I'd love to chat.

This is the first post in a series about building tech products from the Pacific Islands. Follow for more about AI, automation, and digital transformation in French Polynesia.

DEV Community: Jordy | PACIFIK'AI