Productionizing an LLM Feature: The Full-Stack Contract

The model call is not the feature. The feature is the contract connecting a user action to an authorized request, a cancellable operation, a validated result, and a recoverable UI state.

This blueprint uses a document-summary feature to show where those responsibilities belong in a full-stack application.

Define one API resource

Treat generation as an operation with its own ID instead of a long request whose only state is “the socket is open.”

POST /api/documents/{documentId}/summaries
Idempotency-Key: 7df1...

{
  "format": "bullets",
  "max_items": 5
}

The server responds after validating the request and creating the operation:

{
  "summary_id": "sum_01J...",
  "status": "queued",
  "events_url": "/api/summaries/sum_01J.../events"
}

An operation resource gives the UI something stable to reconnect to, cancel, or inspect. The idempotency key prevents an accidental double-click or network retry from creating duplicate work.

Authorize before assembling context

The route should derive the user and tenant from the authenticated session. It must verify access to documentId before loading document text.

const document = await documents.findForTenant({
  id: params.documentId,
  tenantId: session.tenantId,
});

if (!document) return new Response("Not found", { status: 404 });

Do not ask the model whether the user may see a document. Do not accept a tenant ID generated by the model or supplied by the browser. Retrieval must preserve the same application permissions as the normal document view.

Persist lifecycle, not hidden reasoning

A compact table can support recovery and operations:

create table summaries (
  id text primary key,
  tenant_id text not null,
  document_id text not null,
  requested_by text not null,
  status text not null check (
    status in ('queued', 'running', 'succeeded', 'failed', 'cancelled')
  ),
  prompt_version text not null,
  model_alias text not null,
  result_json jsonb,
  error_code text,
  created_at timestamptz not null,
  completed_at timestamptz
);

Store the minimum content required by the product and its retention policy. Full prompts and chain-of-thought-style internal reasoning are not necessary operation records. IDs, versions, timing, status, usage, and a redacted error category are usually more useful.

Keep provider code behind an adapter

Application code should depend on a narrow interface:

type SummaryInput = {
  source: string;
  format: "bullets" | "paragraph";
  maxItems: number;
};

type SummaryOutput = {
  headline: string;
  items: string[];
};

interface SummaryGenerator {
  generate(input: SummaryInput, signal: AbortSignal): Promise<unknown>;
}

The adapter handles provider authentication, request IDs, timeout mapping, and response extraction. The service layer validates the returned unknown value against the application schema. This separation makes tests independent of a live provider and keeps provider-specific fields out of the UI.

Stream progress without treating fragments as final data

Server-Sent Events work well for a one-way stream from server to browser:

event: status
data: {"state":"running"}

event: delta
data: {"text":"The release..."}

event: complete
data: {"summary_id":"sum_01J..."}

Text deltas improve perceived responsiveness, but they are incomplete and unvalidated. Render them in a preview region. On complete, fetch or use the validated stored result, then switch the UI to its final state.

SSE streams use the text/event-stream format; MDN’s Using server-sent events documents the event format and browser API.

Make cancellation end-to-end

The browser owns an AbortController for the active connection. It should also call a cancellation endpoint so work can stop after the stream disconnects:

POST /api/summaries/{summaryId}/cancel

The worker checks cancellation between expensive stages and passes an abort signal to the provider client when supported. Mark the operation cancelled only when the backend has accepted the cancellation; closing a browser tab is not proof that server work stopped.

Validate generated output

Suppose the model returns JSON. Enforce application rules after parsing:

• headline is a non-empty string within the UI limit;
• items contains between 1 and max_items strings;
• no item exceeds the storage limit;
• URLs, identifiers, or enum values meet explicit allowlists;
• the result does not contain fields the client never requested.

On validation failure, record a typed error. One bounded repair attempt may be reasonable for low-risk output, but an endless automatic retry loop can multiply cost and hide a bad contract.

Design the UI state machine

The component should make legal transitions explicit:

idle
  → creating
  → queued
  → streaming
  → succeeded
  → failed
  → cancelling → cancelled

Preserve the user’s input on failure. Show a retry only for retryable categories. Disable conflicting actions while cancellation is pending. If a reconnect finds an already completed operation, load the stored result instead of generating again.

Test at four boundaries

1. Route tests: unauthenticated, wrong tenant, invalid limits, duplicate idempotency key.
2. Service tests: provider timeout, malformed JSON, cancellation, allowed retry.
3. Stream tests: ordered events, disconnect, reconnect, terminal event exactly once.
4. Browser tests: double submit, cancel, refresh during work, error recovery, keyboard and screen-reader status.

Use a fake generator that can emit deltas, block until aborted, or return malformed data. Reserve live-model checks for a separate evaluation suite; ordinary application tests should be deterministic.

The architecture may look larger than a one-line model SDK call because it describes the actual product. Once identity, lifecycle, validation, and recovery have explicit owners, model changes become manageable implementation changes instead of rewrites of the whole feature.