Trust the Agent Only After You Can Kill Its Server

Last Friday a teammate walked me through an invoice summarizer that looked finished inside a long-lived notebook kernel. He typed one customer identifier, watched a tidy paragraph appear, and then declared the whole feature done. I asked him to restart the kernel and run the same cell without pasting any secrets again. Could that request still succeed, and if it could, whose invoice text was still sitting in memory?

That is the only user action I trust as a starting point for an AI feature now. A signed-in user posts an invoice identifier and expects a summary without leaking the last tenant. The first layer that fails is almost never the model, which is a hard thing to admit. It is the sticky process that still holds last week's key and last customer's raw text.

I am going to take a position that will annoy people who live inside chat windows all day. An AI feature is not a feature until you can kill the process and the HTTP contract still holds. Free model access does not buy you production, no matter how generous the trial feels this week. A restartable server that you are willing to crash on purpose is the cheaper and more honest teacher.

When I need that crashable box without standing up a billable cluster, I reach for MonkeyCode's free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach, and I am using that context as a rehearsal environment, not a bake-off. The project is open, and it currently offers free model access plus that free server path for this kind of drill. I do not need a named model or a benchmark chart to learn whether my feature remembers tenants after a restart.

Here is the architectural choice that notebooks keep hiding from you until a pager goes off. The model must sit behind a tiny client contract, and the invoice text must sit behind authorization. If either of those lives as a module global, you do not have a feature, you have a souvenir. Why do we keep pretending a kernel is an application just because the JSON looks pretty in stdout?

The proposed slice is deliberately boring, because boring is what survives a restart. One process serves HTTP, one store owns invoice bytes, and one client interface owns model calls. Auth is a bearer token that maps to a tenant, not a comment in a notebook that says remember to switch customers. Persistence is a row with tenant_id, not a Python dict that outlives the engineer who created it.

# proposed seam, not a vendor SDK tour
from typing import Protocol

class ModelClient(Protocol):
    def complete(self, *, system: str, user: str) -> str: ...

class HttpModelClient:
    def __init__(self, base_url: str, api_key: str, timeout_s: float = 20.0):
        self.base_url = base_url.rstrip("/")
        self.api_key = api_key
        self.timeout_s = timeout_s

    def complete(self, *, system: str, user: str) -> str:
        # Wire this to whatever free-model endpoint you actually have.
        # The rest of the app should never import a provider SDK.
        raise NotImplementedError("bound at process boot, not at call sites")

Boot the client once from environment variables, then refuse to read those variables again inside request handlers. That sounds fussy until a hot reload leaves a stale key in memory and your "quick retry" bills the wrong account. I want the process to be disposable, which means secrets live in the supervisor, not in a cell that nobody will re-run. If the server cannot start without MODEL_BASE_URL and DATABASE_URL, good: that is a feature, not ceremony.

# proposed FastAPI slice
from fastapi import FastAPI, Depends, Header, HTTPException
from pydantic import BaseModel

app = FastAPI()
# model_client and db are built in create_app(), never at import time

class SummaryOut(BaseModel):
    invoice_id: str
    summary: str
    model_ok: bool

def tenant_from(authorization: str | None = Header(default=None)) -> str:
    if not authorization or not authorization.startswith("Bearer "):
        raise HTTPException(status_code=401, detail="missing_bearer")
    token = authorization.removeprefix("Bearer ").strip()
    tenant = lookup_tenant(token)  # hits the store, not a global dict
    if tenant is None:
        raise HTTPException(status_code=401, detail="unknown_token")
    return tenant

@app.post("/invoices/{invoice_id}/summarize", response_model=SummaryOut)
def summarize(invoice_id: str, tenant: str = Depends(tenant_from)):
    row = db.fetch_invoice(tenant_id=tenant, invoice_id=invoice_id)
    if row is None:
        raise HTTPException(status_code=404, detail="invoice_not_found")
    text = model_client.complete(
        system="Summarize the invoice for the authenticated tenant only.",
        user=row["body"],
    )
    db.insert_summary(tenant_id=tenant, invoice_id=invoice_id, summary=text)
    return SummaryOut(invoice_id=invoice_id, summary=text, model_ok=True)

Notice what this endpoint refuses to do. It does not accept raw invoice text from the client, because that is how yesterday's paste becomes today's leak. It does not let the model choose a tenant, because models are optimistic narrators, not access-control layers. It writes the summary back to the same tenant-scoped row, so a restart cannot resurrect a paragraph that never landed in storage. If the model call fails, the user should see 503 with model_unavailable, not a half sentence that looks authoritative.

The interesting test is not "did the prose sound smart." Kill the server, start it again, and replay the same request with the same bearer token. I want 200 and the same tenant's invoice, or I want a clean 401/404, not a ghost of the previous kernel. Cross-tenant replay should stay 404, even if the invoice identifier is easy to guess. If you cannot name the status code you expect after SIGTERM, you are still demoing.

# proposed restart drill; run this against localhost or a free server you control
uvicorn app:app --port 8080 &
PID=$!
curl -sS -o /tmp/a.json -w "%{http_code}\n" \
  -H "Authorization: Bearer tenant-a-token" \
  -X POST http://127.0.0.1:8080/invoices/inv_100/summarize
kill $PID
wait $PID 2>/dev/null || true
uvicorn app:app --port 8080 &
PID=$!
curl -sS -o /tmp/b.json -w "%{http_code}\n" \
  -H "Authorization: Bearer tenant-a-token" \
  -X POST http://127.0.0.1:8080/invoices/inv_100/summarize
curl -sS -o /tmp/c.json -w "%{http_code}\n" \
  -H "Authorization: Bearer tenant-b-token" \
  -X POST http://127.0.0.1:8080/invoices/inv_100/summarize
kill $PID

What failed for me, in this kind of slice, was never the fancy sampling settings. A module-level LAST_SUMMARY cache made the second tenant look served when the model was actually down. A connection pool created at import time kept an old role after I rotated database credentials. One helper logged the first two hundred characters of invoice body, which is how a "debug print" becomes a compliance incident. The model sounded confident through all of that, which is exactly why I stopped treating fluency as evidence.

Cheap generated code makes this worse, not better, because the cost of typing dropped faster than the cost of owning state. If an assistant can scaffold an endpoint in twenty seconds, it can also hide a global in twenty seconds. The antidote is not a longer prompt. The antidote is a process you can murder, a store that still has the row, and a client you can swap without touching the route. That is a provider seam plus a restart, not a vibe.

Use the free model access as a noisy dependency, not as a personality. Slow responses, empty strings, and 429 from the upstream should already have mapped status codes in your handler. The free server option matters because it gives you a machine that is allowed to die, which laptops running notebooks are emotionally unwilling to do. I would rather rehearse a crash on a complimentary box than discover implicit memory on the first paying tenant.

This approach is wrong for some people, and I will say that plainly. If you are training weights, ranking models, or chasing latency charts, a killable HTTP slice will feel like a stall. If your app never stores user data, the tenant checks are theater. If you cannot put a bearer token in front of the route, do not put a model behind it either, because you just built a public summarizer for whoever guesses an identifier.

Production caveats stay unromantic. Cap the invoice body you send upstream, because free model access is not an excuse to ship entire PDF dumps as prompt stuffing. Keep timeouts shorter than your load balancer, or users will retry and double-write summaries. Make summarization idempotent on (tenant_id, invoice_id) so a restart plus a client retry does not create two competing paragraphs. And please stop holding API keys on the object that renders the UI; the browser is not a vault, even when the demo is internal.

If you want a crashable box without a purchase order, that free server path is enough to run this restart drill once. After that, the work is local: env at boot, tenant on every read, model behind a protocol, store as the only memory. I do not care how pretty the first summary looked in a notebook. I care whether the second process still knows who paid the invoice.

Which layer handoff is least stable in your app when the process dies: auth, persistence, or the model client? Send the status code you actually get after a restart, not the one the happy-path screenshot promised.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.