Promote an AI Prototype to Production Behind a Provider Contract, Not a Rewrite

A teammate asks the AI summary feature to "just summarize this 40-page incident report." The demo worked last week. Today the request hits the free model endpoint, the response is truncated at the token limit, and the UI renders half a summary as if it were complete. The first layer that fails is not the model — it is the contract between your API and whatever model is behind it.

This is the classic prototype-to-production failure mode: the prototype was written directly against one provider's SDK, with one hard-coded model, no output validation, and no plan for what happens when you swap in a paid, higher-capacity model later. This article shows the pattern I use to avoid the rewrite: a provider seam with an explicit capability contract, built and exercised entirely on free infrastructure first, then promoted by changing configuration — not code.

The cost of skipping the seam

The vibe-coded version looks like this:

// ❌ prototype wired directly to one provider
import { SomeProvider } from "some-provider-sdk";

export async function summarize(text: string) {
  const res = await SomeProvider.chat({
    model: "some-model",
    messages: [{ role: "user", content: `Summarize: ${text}` }],
  });
  return res.choices[0].message.content;
}

Everything wrong here is invisible until promotion day:

• The caller assumes the response is complete; nothing validates length or finish reason.
• The model name leaks into business logic, so switching models means a code change and redeploy.
• There is no token accounting, so the first real document blows the context window silently.
• The endpoint URL and credentials are baked into the deploy, so "try it on the free server first" requires a branch.

The fix is not more prompt engineering. It is a contract.

Step 1: Define the capability contract

The seam is a small interface plus a metadata object that describes what the current provider can and cannot guarantee:

// src/ai/provider.ts
export interface SummaryRequest {
  documentId: string;
  text: string;
  maxOutputTokens: number;
}

export interface SummaryResult {
  summary: string;
  finishReason: "complete" | "length" | "error";
  inputTokens: number;
  outputTokens: number;
  model: string;      // echoed back so logs are honest
  provider: string;   // ditto
}

export interface ProviderCapabilities {
  maxInputTokens: number;   // measured, not marketing copy
  supportsStreaming: boolean;
}

export interface SummaryProvider {
  capabilities(): ProviderCapabilities;
  summarize(req: SummaryRequest): Promise<SummaryResult>;
}

Two deliberate choices:

1. finishReason is mandatory. The UI must know whether it got a full summary or a truncated one. This is the field that was missing in the failing demo.
2. capabilities() is measured at runtime, not hard-coded from documentation. Free-tier endpoints in particular may enforce limits that differ from the provider's marketing page.

Step 2: Implement two adapters — free first

Write the cheapest adapter first. I run prototypes against MonkeyCode, which offers free model access and a free server option, so the whole vertical slice — UI, API, storage, provider — runs before anyone approves a cloud bill. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The important thing for this pattern is not the specific provider, but that the prototype adapter and the production adapter implement the same interface, so the promotion path is a config swap:

// src/ai/openaiCompatibleAdapter.ts — works for any OpenAI-compatible endpoint
export function openAICompatibleAdapter(cfg: {
  baseUrl: string; apiKey: string; model: string; providerName: string;
}): SummaryProvider {
  return {
    capabilities: () => ({
      // discovered by probing, see step 3
      maxInputTokens: Number(process.env.MEASURED_MAX_INPUT_TOKENS ?? 4096),
      supportsStreaming: false,
    }),
    async summarize(req) {
      const res = await fetch(`${cfg.baseUrl}/chat/completions`, {
        method: "POST",
        headers: {
          Authorization: `Bearer ${cfg.apiKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          model: cfg.model,
          messages: [{ role: "user", content: `Summarize concisely:\n\n${req.text}` }],
          max_tokens: req.maxOutputTokens,
        }),
      });
      if (!res.ok) {
        return { summary: "", finishReason: "error", inputTokens: 0,
                 outputTokens: 0, model: cfg.model, provider: cfg.providerName };
      }
      const body = await res.json();
      const choice = body.choices?.[0];
      return {
        summary: choice?.message?.content ?? "",
        finishReason: choice?.finish_reason === "stop" ? "complete" : "length",
        inputTokens: body.usage?.prompt_tokens ?? -1,
        outputTokens: body.usage?.completion_tokens ?? -1,
        model: cfg.model,
        provider: cfg.providerName,
      };
    },
  };
}

The prototype adapter and the production adapter are both constructed through this factory, pointed at different baseUrl/model pairs from environment config:

// src/ai/index.ts
export function buildProvider(): SummaryProvider {
  const target = process.env.AI_TARGET ?? "free";
  if (target === "free") {
    return openAICompatibleAdapter({
      baseUrl: process.env.FREE_BASE_URL!,   // e.g. your free server endpoint
      apiKey: process.env.FREE_API_KEY!,
      model: process.env.FREE_MODEL!,
      providerName: "free-sandbox",
    });
  }
  return openAICompatibleAdapter({
    baseUrl: process.env.PROD_BASE_URL!,
    apiKey: process.env.PROD_API_KEY!,
    model: process.env.PROD_MODEL!,
    providerName: "prod",
  });
}

Promotion is AI_TARGET=prod plus secrets. No branch, no redeploy of business logic.

Step 3: Validate output at the API boundary

The route layer owns two jobs the provider cannot: rejecting oversized input before paying for it, and refusing to serve a truncated summary as if it were complete.

// src/routes/summaries.ts (Fastify, but any framework works)
app.post("/api/documents/:id/summary", async (req, reply) => {
  const doc = await getDocument(req.params.id, req.user.id); // ownership check!
  if (!doc) return reply.code(404).send({ error: "not_found" });

  const provider = buildProvider();
  const caps = provider.capabilities();
  const estimated = estimateTokens(doc.text);
  if (estimated > caps.maxInputTokens * 0.7) {
    return reply.code(413).send({
      error: "document_too_large",
      detail: `~${estimated} tokens exceeds safe window of ${Math.floor(caps.maxInputTokens * 0.7)}`,
      hint: "chunking required — not implemented in this slice",
    });
  }

  const result = await provider.summarize({
    documentId: doc.id, text: doc.text, maxOutputTokens: 800,
  });

  // Persist provenance with the summary, not just the text
  await saveSummary({
    documentId: doc.id,
    summary: result.summary,
    complete: result.finishReason === "complete",
    provider: result.provider,
    model: result.model,
    inputTokens: result.inputTokens,
  });

  return reply.code(result.finishReason === "complete" ? 200 : 206).send(result);
});

Note the 206 Partial Content for truncated output. That one status code is the difference between "the UI shows a half-summary with a warning badge" and "the UI silently lies."

Step 4: The promotion test matrix

Before flipping AI_TARGET=prod, run the same contract tests against both adapters. This is the artifact that catches the silent differences between a free sandbox model and the production one:

#	Test	Free adapter	Prod adapter
1	Short doc → finishReason: complete	must pass	must pass
2	Over-limit doc → HTTP 413, no provider call	must pass	must pass
3	Doc near token limit → 206, complete: false persisted	must pass	must pass
4	Provider 500 → finishReason: error, nothing persisted	must pass	must pass
5	Response usage fields populated (or -1 and logged)	observed	must pass
6	provider/model recorded match config, not assumptions	must pass	must pass

Test 5 is the one that bites: some free endpoints omit usage metadata. The contract tolerates it (-1), but production cost accounting must not. Discover that gap in the sandbox, not on the invoice.

Limitations and who should skip this

• This does not solve chunking. The 413 response is an honest refusal, not a feature. Long-document summarization needs a map-reduce or hierarchical pass, which is a separate slice.
• Free tiers are for proving the seam, not for load. Rate limits, latency, and model behavior on free access will differ from paid capacity. Do not benchmark one and ship the other.
• Token estimation is approximate. estimateTokens is a heuristic guardrail; enforce the real limit server-side via the provider's own errors too.
• If your feature is a one-shot internal script with a single document type, this seam is overhead. It pays off the moment you have a second provider, a second model, or a user-facing feature where truncated output would be a lie.

Reusable checklist

1. [ ] Provider behind an interface; model name and base URL come from env, not code.
2. [ ] finishReason/completeness surfaced through the API (status code or field).
3. [ ] Input size rejected before the provider call, with a useful error.
4. [ ] Provenance (provider, model, token counts) persisted alongside the artifact.
5. [ ] Contract test matrix run against both the free and production adapters.
6. [ ] Ownership/auth check on the route before any model spend.

If you want a zero-cost place to run steps 1–4 before requesting production budget, MonkeyCode's free models and free server are one workable sandbox — but the checklist above is what makes the eventual swap boring.

Which layer handoff is least stable in your stack — the provider call, the token accounting, or the persistence of what the model actually returned? If you have a concrete failure state (a status code, a truncated record, a surprise invoice line), I'd like to hear it in the comments.