Add Model Calls Behind a Provider Contract, Not Across Your Codebase

When a teammate clicks Generate follow-up and save, the browser fires one POST with a ticket id and a tone. The API is supposed to ask a model for a comment plan, preview it, and persist only after someone confirms. What actually dies first is not the model. It is the route that constructed a vendor client, awaited a completion, and wrote raw text into Postgres before the UI could render a preview.

I do not treat that crash as a model-quality problem. I treat it as a missing contract, and I would refuse to ship the feature in that shape. Cheap inference makes the wrong design feel productive, because every layer can “just call the model” in a single afternoon. Have you watched provider field names leak into React types, OpenAPI schemas, and even audit logs? That leak is the first layer that breaks when the host, the envelope, or the auth story changes.

My position is not gentle. A free model is not an architecture. If a feature can persist state, the model must sit behind a provider contract you can stub, swap, and fail closed. Anything else is a prototype with extra steps, and extra steps are how vibe-coded write paths reach production.

The request should stay a plan until apply is boring

Picture the end-to-end action again. A support lead wants a follow-up comment drafted from a ticket, then stored against that ticket with an actor and an idempotency key. The UI should post a plan request, receive a plan, and only later post an apply with the human’s approval. If your first handler both samples text and inserts a row, you have already given the model write authority it did not earn.

I keep using a ticket comment because it looks harmless. It is a single user action, it touches auth, it touches storage, and it is exactly the kind of slice teams bolt onto a weekend demo. The failure mode is familiar: the model returns a slightly different JSON shape, the UI still paints a paragraph, and the apply path writes a comment nobody can later attribute. Who owns that row when the provider was inlined in the route?

Think of the model like a shipping carrier. You do not let every warehouse printer speak UPS XML. You speak your own bill of lading, and the adapter translates. When the carrier changes, the warehouse does not reprint its building. Why do we let chat SDKs become the building?

What I would put in the contract

The contract is deliberately smaller than a vendor SDK. It names a purpose, a digestable prompt, and a plan the apply path can reject. This is a proposed TypeScript seam, not a claim that I ran it in a particular production fleet.

// provider-contract.ts — proposed seam
export type Purpose = "ticket_followup";

export type PlanRequest = {
  purpose: Purpose;
  ticketId: string;
  tone: "neutral" | "firm";
  actorId: string;
  maxTokens: number;
};

export type Plan = {
  providerAlias: string; // your name, not a vendor model id
  promptDigest: string;  // hash of the exact prompt bytes
  body: string;
  citations: string[];   // ticket ids or doc ids, never vendor blobs
};

export interface ModelProvider {
  plan(req: PlanRequest): Promise<Plan>;
}

Notice what is missing. There is no streaming callback, no vendor message array, no “system prompt” string leaking into the database. The apply handler should not know whether the bytes came from a stub, a paid API, or a rehearsal host. If you need those vendor fields in the UI, the contract is already broken. Why would a React component care which company sampled the tokens?

A stub provider makes the rest of the app testable on a laptop.

// stub-provider.ts
import { createHash } from "node:crypto";
import type { ModelProvider, Plan, PlanRequest } from "./provider-contract";

export class StubProvider implements ModelProvider {
  async plan(req: PlanRequest): Promise<Plan> {
    const body = `Follow-up for ${req.ticketId} in a ${req.tone} tone.`;
    const promptDigest = createHash("sha256")
      .update(`${req.purpose}:${req.ticketId}:${req.tone}`)
      .digest("hex")
      .slice(0, 16);
    return {
      providerAlias: "stub",
      promptDigest,
      body,
      citations: [req.ticketId],
    };
  }
}

Wire a live provider the same way. Map your PlanRequest onto whatever HTTP envelope you are rehearsing, then map the response back onto Plan. If mapping needs a comment, the envelope does not belong in the route. I would rather maintain one awkward adapter than three clever call sites.

Plan and apply as two permissions

The API should expose two verbs with two permission checks. POST /comments/plan may spend tokens. POST /comments/apply may spend a database row. Mixing them is how a retry creates two comments and one confused customer.

// routes.ts — proposed handlers
app.post("/comments/plan", requireSession, async (req, res) => {
  const body = PlanRequestSchema.parse(req.body);
  if (body.actorId !== req.session.userId) {
    return res.status(403).json({ error: "actor_mismatch" });
  }
  const plan = await req.provider.plan(body);
  // persist the plan as a row with status=proposed, not as a comment
  const id = await plans.insert({ ...plan, ticketId: body.ticketId, actorId: body.actorId });
  return res.status(201).json({ planId: id, plan });
});

app.post("/comments/apply", requireSession, async (req, res) => {
  const { planId, idempotencyKey } = ApplySchema.parse(req.body);
  const plan = await plans.get(planId);
  if (!plan || plan.actorId !== req.session.userId) {
    return res.status(409).json({ error: "plan_not_applicable" });
  }
  const comment = await comments.insertOnce({
    ticketId: plan.ticketId,
    body: plan.body,
    promptDigest: plan.promptDigest,
    providerAlias: plan.providerAlias,
    idempotencyKey,
  });
  return res.status(201).json({ commentId: comment.id });
});

409 plan_not_applicable is a feature, not an eyesore. I want the UI to render that code, not a generic toast. If apply is a second click, you can put a human in the gap without pretending the model was “safe.” Would you let a payment provider both quote and capture in the same unauthenticated function? Then why is a comment different?

A rehearsal on a disposable host is useful here. If you need free model access and a free server option to exercise the live adapter without baking a vendor into staging, MonkeyCode is an open-source project that offers both. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use that kind of host to prove the mapping, then point the same ModelProvider interface at whatever you actually pay for. Do not let the rehearsal hostname appear in your apply table.

Fail the handoff on purpose

I would not trust the seam until I can break it from the outside. The following is a proposed test plan, not a benchmark I am claiming to have published.

# 1) Stub plan should never write a comment.
curl -sS -X POST "$API/comments/plan" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"purpose":"ticket_followup","ticketId":"T-1042","tone":"firm","actorId":"u1","maxTokens":256}'
# expect 201, body.plan.providerAlias == "stub", no comments row

# 2) Apply with a foreign actor should die before INSERT.
curl -sS -X POST "$API/comments/apply" \
  -H "Authorization: Bearer $OTHER" \
  -d '{"planId":"'$PLAN'","idempotencyKey":"k1"}'
# expect 409 plan_not_applicable

# 3) Swap the provider alias in process env and rerun plan.
# expect the apply schema to be unchanged; only providerAlias differs

# 4) Feed apply a body that includes a vendor-only field.
# expect 400 from your parser, not a new column in comments

The interesting failure is step four. If your parser is “helpful” and strips unknown keys, the UI will keep sending them and you will not notice until a migration. Fail closed. A comment table that grows a raw_choices JSONB column is a shrine to the last SDK you copied.

Cost and maintainability follow the same line. Tokens can be free on a rehearsal host and still be expensive in engineering time when every screen knows the provider. I would rather spend a quiet hour on promptDigest and insertOnce than spend a noisy week grepping for a renamed content field. Is the digest overkill for a comment? Only if you never have to explain who wrote it.

Who should not take this advice

This slice is for features that persist, authorize, and retry. If you are sketching a throwaway notebook, inlining a client is fine, and a contract will only slow you down. If you are building a streaming chat surface where the model is the product, a plan/apply split may be the wrong grain; you still want an adapter, but the user-visible verb is the stream, not a row.

Do not use a free rehearsal server as the identity of production apply. Do not store provider hostnames as foreign keys. Do not claim “full stack” if the plan route has no actor check or the apply route has no idempotency key. And do not treat this article as a promise about any vendor’s quota, hardware, or uptime. Those numbers go stale, and stale numbers are how people copy architecture from a blog post instead of from their own failure codes.

A short path you can reuse

Stand up the contract and the stub first, with no network. Add POST /comments/plan and assert that storage still has zero comments. Add POST /comments/apply with session checks and insertOnce. Point the same interface at a live adapter on a host you can delete. Replay the four curls above until the only thing that changes across providers is providerAlias and promptDigest. Then, and only then, let the UI grow a confirm button.

I care less about which model sampled the paragraph than about which layer was allowed to write. If your apply path cannot reject a plan with a boring status code, you do not have an AI feature. You have a completion call with side effects.

Which handoff is least stable in your app right now—the UI to plan, the plan to apply, or apply to storage? Reply with a status code or a response body, not a feeling.