Don't Run Floor-Plan Generation on the Request Thread

I'm Tomás Herrera. Day job is backend and infrastructure at ArchyBase — mostly the unglamorous path from an uploaded image to a finished artifact: preprocess, enqueue, watch workers, surface honest failures. This post is about a sibling product our team also builds, FloorAI, and specifically why floor-plan generation should not sit on the HTTP request thread. The case study is the floor plan optimizer that keeps hard constraints: upload a plan, declare what must not move, wait for a revised layout.

I'm not claiming a magic latency number or a user count. I'm claiming an architecture habit that keeps timeouts, retries, and bad days from corrupting your data model.

The request thread is a terrible place for generation

A typical naive path looks like this:

1. Client POSTs an image (or a structured plan) plus constraint payload.
2. API handler runs preprocess → model / solver → postprocess.
3. Handler returns the finished floor plan in the same response.

That works in a demo with a warm cache and a friendly network. It fails in production for boring reasons:

• Gateway and load-balancer idle timeouts sit in the 30–120s range more often than product people expect. Generation that sometimes takes longer simply never reaches the client as a 200.
• Client retries on a hung connection are not the same as server-side idempotency. Without a job id, a double submit becomes two generations and two writes.
• Worker memory and GPU / CPU pools do not scale like stateless web replicas. Pinning that work to the request fleet couples traffic spikes to the most expensive path.

So the first design rule I push for: accept work quickly, run generation elsewhere, expose status honestly.

What “queued” means in this product shape

For an optimizer-style flow, the durable unit of work is not “the HTTP call.” It is something closer to:

Job {
  job_id          // client-visible, unique
  input_ref       // uploaded plan / image object key
  constraints     // hard constraints: walls, openings, adjacency
  state           // queued | running | succeeded | failed | cancelled
  attempt         // monotonic
  result_ref?     // output object key when succeeded
  error_class?    // timeout | invalid_input | solver_infeasible | internal
}

The upload+constraints request should:

1. Validate enough to reject garbage early (empty file, unsupported type, constraints that contradict each other in an obvious way).
2. Persist the input and create the job in queued.
3. Return 202 (or an equivalent “accepted” payload) with job_id and a status URL.
4. Let a worker claim the job, run generation, and transition state.

The optimizer page’s product promise — keep hard constraints while revising circulation or room use — maps cleanly onto that model. Constraints are part of the job payload, not ambient session state. If the tab closes, the job still exists. If the user refreshes, they reattach by job_id, not by hoping the old TCP connection is alive.

Timeouts belong to stages, not to “the whole thing”

One timeout for “upload to finished plan” hides where time went. I prefer stage budgets:

Stage	Typical failure mode	Client-visible consequence
Upload / ingest	oversized file, virus scan stall	reject before queue
Preprocess	decode failure, absurd aspect ratio	failed + invalid_input
Generation / solve	soft time budget exceeded	retryable timeout or solver_infeasible
Persist / notify	object store blip	retry without re-solving if result already written

The important part is which failures are safe to retry. A pure timeout with no side effects is usually retryable. A solver that already wrote a partial artifact needs an idempotency key so a retry does not append a second “final” plan.

Retries without doubles

Idempotency for generation jobs is not optional once you leave the request thread. Minimum bar:

• Client may send an Idempotency-Key (or you mint job_id server-side and require the client to reuse it on reconnect).
• Worker claim uses compare-and-set: only one worker moves queued → running for a given attempt.
• Result write is keyed by job_id. Succeeded stays succeeded; a late worker must not overwrite with a second blob unless you explicitly version outputs.
• User-facing “Try again” creates a new attempt (or a new job) rather than mutating history in place — so support can see what ran.

Silent auto-retries on invalid_input waste money and confuse people. Auto-retries on transient infra errors are fine if the attempt counter and backoff are visible in logs and, ideally, in the status payload.

Honest failure states beat a spinner that lies

UI that spins until the gateway kills the connection teaches users the wrong lesson. Prefer a small, boring state machine:

• queued — accepted; position / ETA optional and best-effort only (don’t invent precision).
• running — a worker holds the lease.
• succeeded — result_ref is readable.
• failed — with an error_class the client can branch on (invalid_input vs retryable).
• cancelled — user or system stopped it; not the same as failed.

For constraint-heavy optimization, solver_infeasible deserves its own class. “We could not satisfy these hard constraints together” is a different product conversation from “the GPU node disappeared.” Collapsing both into a red toast sends people into pointless retries.

Cost vs latency, without fake metrics

Queued generation lets you choose where to spend:

• Latency-sensitive path: smaller input, tighter preprocess, higher-priority queue for interactive edits.
• Cost-sensitive path: batch queue, lower concurrency, accept longer queued time.

I won’t quote unverified speedups. The architectural win is control: you can shed load by refusing new jobs with a clear queued-capacity response instead of letting request threads pile up until everything 504s. You can also keep hard constraints attached to the job so a cheap retry does not drop the user’s “these walls stay” rules — which is the whole point of an optimizer that claims to respect them.

A minimal checklist before you ship “AI floor plan” on HTTP

If you’re wiring something like FloorAI’s optimizer flow (or any sibling generation API), I use this as a gate:

invariant generation_off_request_thread:
  accept() returns job_id quickly
  generation runs only in workers with lease + attempt
  every terminal state is one of {succeeded, failed, cancelled}
  failed always carries error_class
  retries never create a second succeeded result for the same job_id
  hard constraints travel with the job, not with the browser tab

Wrap-up

Floor-plan generation — especially optimization under hard constraints — is asynchronous work wearing a synchronous costume if you leave it on the request thread. Move it to a job: durable job_id, staged timeouts, idempotent retries, and failure classes humans can act on. ArchyBase’s day job taught me that the boring state machine is what keeps the flashy model from lying to the user. FloorAI’s upload-and-constrain optimizer is a good reminder of the same rule: keep the walls where the user said they stay, and keep the HTTP thread free so you can tell them the truth when the solve does not.