Add Completions Behind a Provider Contract, Not Across Your Codebase

I keep reconstructing the same staging incident, and it always starts with one hopeful click. A receiving clerk taps Explain this short shipment and waits for an exception note that never lands. The model already answered in the console, yet the API never wrote a row. The handler had mixed authentication, JSON parsing, and a vendor SDK into a single breath.

People keep arguing whether AI already codes better than most developers, and I think that question is a distraction. A completion that cannot survive a provider swap, a 401, and an idempotent apply is not a product feature. It is a demo glued to a vendor, and demos rot the moment the network retries or the schema drifts. So I am taking a blunt position: freeze a provider contract before you let the model near production credentials.

Why does the click fail at persistence instead of at the prompt? Because the first layer that must tell the truth is not the model. It is the handoff between a structured plan and a permissioned apply. If that seam is missing, every retry becomes a second author, and every schema tweak becomes a silent 500. I would rather ship a read-only plan that looks boring than a write path that looks magical.

Think of the model as a contractor who drafts a work order, not as a night-shift DBA with root. The contractor can be swapped. The work order cannot be vague. Your application owns identity, idempotency, and the actual insert, which means a free model and a disposable server are useful only when they exercise that same contract. They are not a shortcut around auth, storage, or deployment.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am using MonkeyCode here as a rehearsal bench with free model access and a free server option, not as a substitute for production ownership. The walkthrough below is a proposed vertical slice, not a claim about live traffic, quotas, or hardware. If a number is not in a test assertion, treat it as unlabeled.

Here is the end-to-end action I want frozen. A signed-in receiver posts a short-shipment event. The API asks the provider for a plan that matches a schema. Only after validation does a separate apply path insert one exception note keyed by a client idempotency token. If the provider coughs, times out, or returns poetry, the row is not created. That is the whole product, and the model is the least interesting part of it.

I start with the contract, not the SDK, because SDKs love to leak into route handlers. The request carries tenant identity, the receiving event identifier, and a frozen JSON schema version. The result is either a valid plan or a typed failure the UI can render without guessing. Notice there is no vendor client in this module. That absence is the architecture.

# proposed contract — not a vendor tutorial
from typing import Literal, Protocol
from pydantic import BaseModel, Field, conint, constr

class ShortShipmentEvent(BaseModel):
    tenant_id: constr(min_length=8, max_length=64)
    receiving_event_id: constr(min_length=8, max_length=64)
    sku: constr(min_length=1, max_length=64)
    expected_qty: conint(ge=1)
    received_qty: conint(ge=0)
    schema_version: Literal["exception_note.v1"] = "exception_note.v1"

class ExceptionNotePlan(BaseModel):
    summary: constr(min_length=12, max_length=240)
    likely_cause: Literal["carrier_short", "pick_error", "count_mismatch", "unknown"]
    follow_up: Literal["hold_dock", "message_vendor", "recount", "no_action"]
    confidence: conint(ge=0, le=100)

class ProviderFailure(BaseModel):
    code: Literal["timeout", "invalid_json", "schema_mismatch", "refused"]
    retryable: bool

class CompletionRequest(BaseModel):
    event: ShortShipmentEvent
    idempotency_key: constr(min_length=16, max_length=128)

class CompletionResult(BaseModel):
    status: Literal["planned", "failed"]
    plan: ExceptionNotePlan | None = None
    failure: ProviderFailure | None = None

class ModelProvider(Protocol):
    def complete(self, request: CompletionRequest) -> CompletionResult: ...

The apply path is deliberately dull, which is the point. Dull code is what you want touching Postgres after a model has been creative. I keep the provider call outside the transaction so a hung completion cannot hold a row lock. I also refuse to apply when the plan is missing, even if the HTTP client is impatient. Have you noticed how many vibe-coded handlers treat a 200 from a model as permission to write?

# proposed apply service
class ExceptionNote(BaseModel):
    tenant_id: str
    receiving_event_id: str
    idempotency_key: str
    plan: ExceptionNotePlan
    provider_name: str
    schema_version: str

class NoteStore(Protocol):
    def get_by_idempotency(self, tenant_id: str, key: str) -> ExceptionNote | None: ...
    def insert_if_absent(self, note: ExceptionNote) -> ExceptionNote: ...

class AuthContext(BaseModel):
    tenant_id: str
    actor_id: str
    can_write_exceptions: bool

def apply_exception_note(
    auth: AuthContext,
    request: CompletionRequest,
    result: CompletionResult,
    store: NoteStore,
    provider_name: str,
) -> tuple[int, dict]:
    if not auth.can_write_exceptions or auth.tenant_id != request.event.tenant_id:
        return 403, {"error": "forbidden"}
    existing = store.get_by_idempotency(auth.tenant_id, request.idempotency_key)
    if existing:
        return 200, existing.model_dump()
    if result.status != "planned" or result.plan is None:
        code = result.failure.code if result.failure else "invalid_json"
        http = 503 if result.failure and result.failure.retryable else 422
        return http, {"error": code}
    note = ExceptionNote(
        tenant_id=auth.tenant_id,
        receiving_event_id=request.event.receiving_event_id,
        idempotency_key=request.idempotency_key,
        plan=result.plan,
        provider_name=provider_name,
        schema_version=request.event.schema_version,
    )
    stored = store.insert_if_absent(note)
    return 201, stored.model_dump()

I rehearse that path on a disposable server before any production secret exists, because production is a terrible scratchpad. Point the same contract at a fake provider in unit tests, then at whatever free model access you have on a free server, and refuse to change the handler. If the free model returns looser prose than your paid one, that is a gift. Schema validation should fail loudly instead of the UI inventing a follow-up action.

# proposed tests — run these before any write credential exists
class FakeProvider:
    def __init__(self, result: CompletionResult):
        self.result = result
        self.calls = 0
    def complete(self, request: CompletionRequest) -> CompletionResult:
        self.calls += 1
        return self.result

class MemoryStore:
    def __init__(self):
        self.rows = {}
    def get_by_idempotency(self, tenant_id, key):
        return self.rows.get((tenant_id, key))
    def insert_if_absent(self, note):
        slot = (note.tenant_id, note.idempotency_key)
        if slot in self.rows:
            return self.rows[slot]
        self.rows[slot] = note
        return note

def test_retry_does_not_double_write():
    plan = ExceptionNotePlan(
        summary="Two cartons missing against the ASN for SKU A-19.",
        likely_cause="carrier_short",
        follow_up="message_vendor",
        confidence=74,
    )
    provider = FakeProvider(CompletionResult(status="planned", plan=plan))
    store = MemoryStore()
    auth = AuthContext(tenant_id="tenant_live", actor_id="recv_9", can_write_exceptions=True)
    req = CompletionRequest(
        event=ShortShipmentEvent(
            tenant_id="tenant_live",
            receiving_event_id="recv_evt_01",
            sku="A-19",
            expected_qty=12,
            received_qty=10,
        ),
        idempotency_key="idem_short_ship_01xx",
    )
    first = apply_exception_note(auth, req, provider.complete(req), store, "rehearsal")
    second = apply_exception_note(auth, req, provider.complete(req), store, "rehearsal")
    assert first[0] == 201
    assert second[0] == 200
    assert len(store.rows) == 1

def test_timeout_does_not_insert():
    failure = ProviderFailure(code="timeout", retryable=True)
    result = CompletionResult(status="failed", failure=failure)
    store = MemoryStore()
    auth = AuthContext(tenant_id="tenant_live", actor_id="recv_9", can_write_exceptions=True)
    req = CompletionRequest(
        event=ShortShipmentEvent(
            tenant_id="tenant_live",
            receiving_event_id="recv_evt_02",
            sku="B-04",
            expected_qty=8,
            received_qty=3,
        ),
        idempotency_key="idem_short_ship_02yy",
    )
    status, body = apply_exception_note(auth, req, result, store, "rehearsal")
    assert status == 503
    assert body["error"] == "timeout"
    assert store.rows == {}

On the rehearsal box I want HTTP proof, not a screenshot of a chat transcript. The commands below are the working path from setup to a visible result, assuming the API is listening locally after you deploy the same contract. Swap the host when the free server is up. Do not swap the headers, the schema version, or the idempotency key behavior.

# proposed rehearsal — same contract, disposable host
export HOST=http://127.0.0.1:8080
export TOKEN=staging-receiver-token

curl -sS -D - -o /tmp/plan.json \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: idem_short_ship_01xx" \
  -d '{
    "tenant_id": "tenant_live",
    "receiving_event_id": "recv_evt_01",
    "sku": "A-19",
    "expected_qty": 12,
    "received_qty": 10,
    "schema_version": "exception_note.v1"
  }' \
  "$HOST/v1/exception-notes:plan"

curl -sS -D - \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: idem_short_ship_01xx" \
  -d @/tmp/plan.json \
  "$HOST/v1/exception-notes:apply"

# cross-layer failures I actually care about
curl -sS -D - "$HOST/v1/exception-notes:apply"   # expect 401
curl -sS -D - -H "Authorization: Bearer $TOKEN" \
  -H "Idempotency-Key: idem_short_ship_01xx" \
  -d @/tmp/plan.json "$HOST/v1/exception-notes:apply"  # expect 200, not 201

The production caveats are not decorative. A free model will drift, a free server will vanish, and neither fact excuses a missing unique index on (tenant_id, idempotency_key). Provider names belong in the stored row so you can audit which seam produced a plan, but they must not leak into authorization. If you cannot replay a failing request against a fake provider, you cannot debug a customer incident without spending money and hoping the model rhymes again.

Who should not use this approach? Anyone who needs the model to execute SQL, move inventory, or send vendor email inside the completion itself. Anyone whose rehearsal data is real personal information, because a disposable server is still a server. Anyone who cannot freeze exception_note.v1 long enough to write the tests above. If your team is still shopping for a model that "just understands the warehouse," you are bargaining with prose when you needed a schema.

I also would not use a rehearsal box as a hidden production. No cron that bills customers. No shared admin token. No "we will add permissions later" comment sitting next to insert_if_absent. Later is how vibe-coded features become incident reports. The contract is the product, and the model is a plugin that has to fail closed.

Reusable checklist, kept short on purpose: freeze the schema before the prompt; keep the SDK behind ModelProvider; plan outside the transaction; apply only with auth and idempotency; map timeout to 503 without a row; replay the same key until you see 200. If any item feels optional, the feature is still a demo. Shipping the demo anyway is how teams confuse fluency with delivery.

If you want a cheap place to prove that seam, MonkeyCode's free model access and free server option are enough to run the rehearsal I sketched. They will not invent your unique index or your 403. That work stays yours, which is the whole argument.

Which layer handoff is least stable in your stack right now, and what status code does it emit when the provider times out after the UI has already shown success?