Replacing a Discontinued Groq Model — How I Restored All AI‑Powered Features in VS

TL;DR: The Groq “llama‑3.3‑70b‑versatile” model was retired on 16 Aug 2026, breaking every endpoint that relied on our AI helper. I swapped it for the open‑source “openai/gpt‑oss‑120b” model, added version‑guard logic and updated the virtual‑tour service, getting the IA pipeline back online in a single commit.

---

The Problem

Our platform (VS) uses a shared Groq helper (apps/api/src/shared/groq.helper.ts) to call an OpenAI‑compatible endpoint for several features: property description generation, rent‑price suggestions, and the new virtual‑tour script generator. On 16 Aug 2026 Groq announced the deprecation of the llama‑3.3‑70b‑versatile model. The API started returning:

{
  "error": {
    "message": "Model llama-3.3-70b-versatile not found",
    "type": "invalid_request_error"
  }
}

All requests that hit groq.helper.ts began throwing a 404, which cascaded into 500 errors in our front‑end. The symptom was a broken “Generate description” button in the portal‑broker UI and a completely silent failure in the virtual‑tour generation flow.

What I Tried First

My first instinct was to patch the endpoint URL with a query param that forced Groq to fallback to an older model version:

// apps/api/src/shared/groq.helper.ts (initial attempt)
const response = await fetch(`${BASE_URL}/v1/completions?model=llama-3.3-70b-versatile`, opts);

Groq rejected the request outright; the service no longer recognized that model name, so the request failed before even hitting the model selector. I also tried catching the 404 and retrying with a hard‑coded “fallback” model, but the code path never reached the retry because the error was thrown during request construction.

The approach was fundamentally wrong: the model identifier is part of the request payload, not a URL param, and the helper was tightly coupled to a single model string.

The Implementation

1. Centralising Model Configuration

I introduced a tiny config layer in groq.helper.ts that reads the model name from an environment variable (GROQ_MODEL) with a sensible default. This makes swapping models a one‑liner in the deployment config.

// apps/api/src/shared/groq.helper.ts
/** groq.helper.ts — Shared Groq AI helper for PlayaMXCRM */
import fetch from "node-fetch";

const BASE_URL = process.env.GROQ_ENDPOINT ?? "https://api.groq.com/openai/v1";
const DEFAULT_MODEL = process.env.GROQ_MODEL ?? "openai/gpt-oss-120b";

export async function askGroq(prompt: string, options = {}): Promise<string> {
  const body = {
    model: DEFAULT_MODEL,
    messages: [{ role: "user", content: prompt }],
    temperature: 0.7,
    ...options,
  };

  const response = await fetch(`${BASE_URL}/chat/completions`, {
    method: "POST",
    headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.GROQ_API_KEY}` },
    body: JSON.stringify(body),
  });

  if (!response.ok) {
    const err = await response.json();
    throw new Error(`Groq request failed: ${err.error?.message ?? response.statusText}`);
  }

  const data = await response.json();
  return data.choices?.[0]?.message?.content ?? "";
}

What changed in the diff:

- * OpenAI-compatible API · Model: llama-3.3-70b-versatile
+ * OpenAI-compatible API · Model: openai/gpt-oss-120b
+
+ const DEFAULT_MODEL = process.env.GROQ_MODEL ?? "openai/gpt-oss-120b";

Only 12 lines added, three removed. The comment now reflects the new default, and the constant makes the model configurable.

2. Updating All Call Sites

The helper’s signature stayed the same, so most callers required no change. However, the virtual‑tour service needed to pass a higher temperature for more creative scripts. I added an optional options param and updated the call site:

// apps/api/src/virtual-tour/virtual-tour.service.ts
import { askGroq } from "../shared/groq.helper";

export async function generateTourScript(propertyId: string): Promise<string> {
  const property = await getPropertyById(propertyId);
  const prompt = `Write a friendly, 2‑minute audio script describing ${property.title} located at ${property.address}.`;
  // Higher temperature for narrative flair
  return askGroq(prompt, { temperature: 0.9 });
}

Diff excerpt:

- return askGroq(prompt);
+ return askGroq(prompt, { temperature: 0.9 });

Only a single line change, but it gave us more expressive output without touching the core helper.

3. Guarding Against Future Deprecations

To avoid a repeat, I added a small validation step that logs a warning if the model name contains “llama”, which is a known deprecated family in our context.

if (DEFAULT_MODEL.includes("llama")) {
  console.warn("[Groq] Using a deprecated llama model. Consider updating GROQ_MODEL.");
}

4. Docker Restart Loop Fix

During testing, the API container kept crashing with ENOTEMPTY because the migration script (apps/api/src/db/db.ts) attempted to create an index that already existed after the hot‑reload. I wrapped the index creation in a IF NOT EXISTS guard (already present) and added a try/catch around the whole migration step:

// apps/api/src/db/db.ts (partial)
try {
  await db.query(`create unique index if not exists idx_pvt_property on property_virtual_tours(property_id);`);
} catch (e) {
  console.error("Index creation failed:", e);
}

This stopped the Docker loop and let me run the API locally to verify the AI calls.

5. Deploy and Verify

After pushing the changes, I set the environment variables in our CI:

GROQ_ENDPOINT=https://api.groq.com/openai/v1
GROQ_MODEL=openai/gpt-oss-120b
GROQ_API_KEY=**** (kept secret)

Running npm run test:e2e showed all AI‑dependent tests passing. The portal‑broker UI now displays a generated description instantly, and the virtual‑tour page renders a script in under 2 seconds.

Key Takeaway

Never hard‑code third‑party model identifiers; abstract them behind a configurable layer. A single environment variable can save you from a full service outage when a provider deprecates a model.

What’s Next

I plan to add a fallback mechanism that automatically switches to a secondary model (e.g., openai/gpt-4-mini) if the primary model returns a 5xx error. This will involve a small wrapper around askGroq that retries with an alternate model value, keeping the system resilient to future deprecations or temporary outages.

---

Tags: #vibecoding #buildinpublic #typescript #nodejs #docker #ai #groq #virtualtour #backend

Roberto Luna Osorio – Full Stack Developer & Project Lead

Playa del

---

Part of my Build in Public series — sharing the real process of building Building PlayaMXCRM from Playa del Carmen, México.

Repo: zaerohell/VS · 2026-09-01

#playadev #buildinpublic