Meltflex Sales

Posted on Feb 21

Tool That Converts Floor Plans to Furnitized Apartment — Here's How It Works Under the Hood

#webdev #programming #ai

We spent the last year building MeltFlex — a free AI interior design tool that takes any 2D floor plan and converts it
into an interactive 3D model where you can place real furniture, customize materials, and generate photorealistic renders.

In this post, I'll break down the technical architecture: how the AI processes floor plans, how we render interactive 3D scenes in the browser, and
what we learned shipping a WebGL-heavy product.

## The Problem

Interior design is broken for regular people. You move into a new apartment, stare at an empty room, and have no idea if that sofa you're eyeing will
actually fit. Professional interior designers charge thousands. Most room planner tools require you to manually draw walls, measure everything, and
place generic 3D blocks.

We wanted something simpler — upload a floor plan photo, get a 3D model, place real furniture, see photorealistic renders.

## Architecture Overview

The stack:

Frontend — Next.js + Three.js (WebGL)
AI Pipeline — Python (OpenCV, Potrace, custom ML models)
3D Rendering — Three.js with custom materials, lighting, and camera systems
AI Renders — Photorealistic image generation from 3D scene data
Backend — Supabase (auth, storage, database)
Deployment — Vercel

Here's how each piece works.

## Step 1: Floor Plan to Vector Geometry

When a user uploads a floor plan — an architect drawing, a hand sketch, or a photo from a real estate listing — our AI pipeline kicks in:

Wall Detection — A vision model identifies walls, doors, windows, and room boundaries from the 2D image
Vectorization — Detected geometry is converted to clean vector paths using Potrace
Room Segmentation — The AI identifies individual rooms and classifies them (living room, bedroom, kitchen)
3D Model Generation — Vector geometry is extruded into a GLB model with proper wall heights, door/window cutouts, and room metadata

The trickiest part was coordinate systems. Potrace outputs SVGs with an inverted Y-axis and path extraction libraries don't apply group transforms. We
had to manually convert between pixel space, SVG space, and 3D world space.

Off-by-one errors in 2D become off-by-a-whole-room errors in 3D. Document your coordinate transforms religiously.

## Step 2: Interactive 3D Scene with Three.js

The generated GLB model loads into a Three.js scene in the browser. This is where it gets interesting.

### Performance Optimization

The scene can have hundreds of meshes with high-res textures. We needed it smooth on both desktop and mobile.

Texture pipeline was the biggest win:

Original 4K textures — 2-4 MB — never sent to client
1024px render — 100-300 KB — used for 3D scene
200px thumbnail — 10-30 KB — used for UI picker grid

This cut initial load by around 80% with no visible quality loss.

Other key optimizations:

Imperative updates only — React state changes don't trigger re-renders of the 3D scene. All texture and material changes happen through refs, not props
RAF-throttled resize — Every resize listener uses requestAnimationFrame to prevent layout thrashing on mobile
CSS animations over JS — Spinners and loading states use pure CSS keyframes to avoid rAF leaks

### Material Application

When a user selects a floor texture or door material, we traverse the scene graph and apply materials directly to matching meshes.

Door-to-room assignment is spatial. When a user changes door texture for a room, we find which doors belong to that room using point-in-polygon and
distance-to-edge algorithms with a 1.5 unit tolerance.

## Step 3: Real Furniture Placement

This is what makes MeltFlex different from other room planners — every piece of furniture is a real product from
real brands with real prices.

We load furniture as GLB models from our catalog. Each model has accurate real-world dimensions, PBR materials, and product metadata including price,
brand, and purchase link.

Users drag furniture into rooms, rotate and position it, and see exactly how it fits. The shopping cart updates in real-time — what you see is what
you can buy.

## Step 4: Photorealistic AI Renders

Once the user has furnished their rooms, they can generate photorealistic renders. The AI takes the 3D scene data — camera position, furniture
placement, materials, lighting — and produces images that look like professional interior photography.

Every piece of furniture in the render is the actual product the user placed. No fake mockups.

## React 19 + Three.js: Lessons Learned

### The ref problem

React 19's memo(forwardRef()) pattern can lose ref.current after re-renders. Our 3D scene component uses this pattern, and we kept getting null refs
when trying to call imperative methods.

Solution — Event-based communication using CustomEvent that bypasses ref forwarding entirely. The Three.js component listens for events
internally. No refs needed.

### The mobile resize loop

Our first mobile version had a catastrophic bug — dispatching a resize event from inside a resize listener created an infinite loop. Combined with rAF
leaks from inline ref callbacks, it drained battery in minutes.

Fix — RAF-throttled listeners plus CSS animations for all loading states.

### Texture size kills mobile

Serving 4K textures to mobile browsers is a dealbreaker. Our three-tier optimization pipeline solved it. The Supabase image transform API handles
resizing server-side, so the client only downloads what it needs.

## What's Next

We're working on improved room type detection (currently around 50% accuracy), a larger furniture catalog with more brands, better mobile 3D
interaction, and more texture options for walls, floors, and doors.

## Try It

MeltFlex is free. Upload a floor plan, get a 3D model, furnish it with real products, and generate photorealistic
renders.

If you're building anything with Three.js, WebGL, or AI + 3D — I'd love to hear about your approach. What rendering challenges have you run into?

More on AI interior design: