How we check every AI-generated room render before a user sees it

AI Flip Room — Sat, 03 Oct 2026 02:05:35 +0000

Image models are very good at restyling a room and quietly bad at leaving the room alone. Ask for a Japandi living room and you get beautiful furniture, and sometimes also:

a window that moved, shrank, or appeared over a kitchen sink where the photo shows a solid wall;
a closet door that came back as a curtain, a sliding screen, or plain wall;
a sofa parked across the only archway;
a new doorway at the frame edge, or a niche carved into a flat wall;
a camera that slid sideways;
or the opposite: a picture that is almost exactly the photo you uploaded.

Our users put these images in property listings, where a staged photo may change the furniture but not the windows, walls or doors (why that line exists). So every render is checked automatically before anyone sees it. Here is how, and what building it taught us.

The pipeline at a glance

For an indoor room, each render goes through three gates in parallel:

A vision-model judge compares the original photo with the render and answers a fixed list of yes/no questions.
A mechanical camera check measures whether the viewpoint moved, with no language model involved.
A crop-level presence check looks at each door, window and staircase found in the original, at full resolution.

If all three pass, the render is served. If not, we retry. If nothing passes, the user gets the best attempt, clearly marked as a draft.

Gate 1: a judge that only answers booleans

The judge gets the original and the render, both downscaled, and must return a fixed JSON object. We use structured output with a schema, so it cannot reply in prose. An earlier version asked for "only JSON" and scraped it out with a regex; when the output got long or cut off, the parse failed and the gate silently did nothing.

Indoors there are 16 checks, outdoors 9. Indoors they cover:

Openings: every door is still a door you could open; every passage has clear floor to walk through.
Windows: same count, position, size and shape; none invented; not hidden behind a tall wardrobe; the same view outside.
The room shell: ceiling, fireplace, no carved niches, features on the same walls, solid frame edges, no new rooms or corridors.
Finish consistency: no half-repainted rooms.
Content rules: no people, and no photo or realistic portrait of a person in the wall art.
Camera: same viewpoint, same room depth.

Outdoors: buildings, fence line and gates, terrain (no new pool), mature trees, background, and a usable driveway.

A simplified, illustrative verdict:

{
  "doors_preserved": true,
  "openings_clear": false,
  "windows_ok": true,
  "window_unblocked": true,
  "walls_uniform": true,
  "frame_edges_solid": true,
  "no_faces_in_art": true,
  "fail_reasons": ["sofa end stands on the archway floor"]
}

A render passes only if every counted boolean is true; the reasons feed logs and the retry prompt. Most of the work went into the wording:

Say what must pass. Restyling is the product, so the instructions list what is expected (new floors, paint, wallpaper, fixtures) and say "unsure means pass". The exception is a short list where unsure means fail, led by a window that may not have been in the original. An invented window is the worst defect a listing photo can have.

One question per field. While the frame-edge rule was a sub-clause of a broader "no phantom architecture" check, new doorways at the frame edge got through. Its own boolean made the model actually look.

Naming things primes the judge. An early niche check listed which shelves were allowed, and the judge started flagging every shelf. The current wording is purely geometric: is this flat wall still one plane?

Allow "nothing to compare". The "same view through the window" check kept firing on originals with blinds drawn or glass blown out to white. That case is now an explicit pass.

Wording is measurable. The old wall-art wording wrongly failed 10 of 16 good frames (stylised line drawings counted as faces). The rewrite failed 0 of 16 and still caught all 5 real portraits.

Gate 2: measuring the camera instead of asking about it

Early on, the vision model was close to blind to viewpoint changes: renders shot from a visibly different spot passed every neural check. Later, a stronger judge had the opposite problem: on 43 renders where the camera had not moved, it said it had 13 times. So indoors the judge's camera answers no longer decide pass or fail. A measurement does, with two channels:

Ceiling line. We trace the wall/ceiling boundary as a curve (dynamic programming over the vertical brightness gradient) and compare the original's curve with the render's.
Depth profile. A monocular depth model estimates both images. Per column we take how far the far wall reads, then compare the profiles with a robust fit, a rank correlation and a search for a sideways shift.

Each channel is fooled by something different. Paint fools the ceiling line (dark beams, art on a mantel). Glass fools depth (windows and mirrors are estimated differently each time). Paint is flat in depth, and glass does not move the ceiling line, so we flag a shift only when both agree: the minimum of the two normalised scores.

On 24 hand-labelled pairs from two rooms, this caught 5 of 5 real shifts with 0 false flags. The margin was thin, so a grey zone around the threshold is logged, not failed. Outdoors there is no ceiling to trace, so the mechanical check is off and the judge's camera question stays in charge.

Gate 3: crops, because a door is 80 pixels wide

At the judge's resolution a front door is a thin strip. So doors, windows and stairs are detected at upload, and after each render a model is asked about each crop at full resolution: is it still there? A mechanical channel also measures whether a window drifted or widened. On 87 labelled pairs, the crop judge caught 7 of 10 broken renders with no false alarms on the 75 clean ones; the mechanical channel caught the other 3.

When a render fails

Retries depend on the kind of defect.

Placement defects (a blocked passage, a face in the art, half-painted walls) are retried on the same image model with the judge's reasons added to the prompt.
Geometry defects go to a second, pricier image model. We measured a same-model retry fixing broken geometry about one time in five. In an earlier bake-off the two models were clean on 18 and 17 of 20 renders, and 19 of 20 together: their failures barely overlap.

The geometry retry gets a fixed, count-based instruction ("same set of openings, not one more, not one less"), never the judge's description of the defect. When a retry prompt named an arched opening at the left edge, the retry model painted exactly that.

If nothing passes, the user still gets the best attempt, under a "rough draft" banner. On paid plans the first four rejected drafts in a rolling week don't use a render; the free plan gets one. The cap is deliberate: a draft is a real, downloadable image, so unlimited forgiveness would be a free image farm.

The checks fail open. If the judge itself errors or times out, the user is not blocked: the image is served flagged, and an unverified render is not charged as a draft.

The near-copy trap

At first, "best attempt" meant the one with the fewest failed checks. The attempt that breaks the least is often the one where the model barely did anything. In one real case the first attempt was a convincing industrial restyle that failed on geometry, and the user was served the second: their old room, old wallpaper and all, plus a pair of lamps. The ranking was paying the model for doing nothing.

The fix is a change meter: shrink both images to a tiny thumbnail and count how much of the frame visibly changed. No model call, and we calibrated it on 110 real renders and 10 synthetic near-copies (recompressed, blurred, brightened, shifted, a couple of lamp-sized patches). The ranking now:

// simplified
function draftRank(a: Attempt): number {
  if (a.checkerErrored) return LAST;
  let rank = a.failedChecks.length;
  if (a.hasPeopleOrRealFacesInArt) rank += HARD_BAN;
  if (a.isNearCopy && a.intensity !== "light") rank += NEAR_COPY;
  return rank; // lowest is served
}

A light restage is exempt, because there the user asked us to keep the room.

Picking the judge model with a calibration set

The judge runs on every attempt, so a cheaper model is tempting. We keep a calibration set of 31 hand-labelled frames and run every candidate on it twice. A cheaper model at under half the price per call missed 6 defects per run against our current judge's 3. On 46 live renders with 8 defects marked by eye, it caught 0 and 1 across two runs; the current judge caught 6. A newer model wasn't clearly better either: fewer misses (2 vs 3), twice the false alarms (4 vs 2). We kept the current judge.

Takeaways

Ask the judge narrow yes/no questions, not one "is this good?".
Where geometry can be measured, measure it, with channels whose failure modes don't overlap.
Calibrate every wording change and model swap on a labelled set, twice.
Design the failure path as carefully as the check: ranking, draft label, who pays.
"Least broken" can mean "least changed". Measure change directly.

We build this at AI Flip Room.

DEV Community: AI Flip Room