DEV Community: Mister Blocks

Making three.js bricks look like real LEGO: studio light, hollow undersides and a real wooden table

Oleksandr Gamanyuk — Thu, 12 Nov 2026 16:00:00 +0000

The first 3D viewer on Mister Blocks drew bricks as coloured boxes with little cylinders on top. It was technically correct and looked like a spreadsheet. Kids building from instructions need to recognise the piece in their hand, so I spent a surprising amount of time making three.js bricks look like real plastic.

Here's what made the difference, roughly from biggest to smallest.

Tesla Cybertruck: 61 pieces, 9 steps. More LEGO cars.

1. Light it like a product photo

Flat lighting is what makes 3D bricks look like boxes. Real ABS plastic has a soft sheen and bright edges, because in every product photo it's sitting in a studio.

So the viewer builds one: an environment map of a soft grey room with a big softbox overhead and strip lights at the sides. Every brick reflects it a little, which gives the tops a gentle sheen and catches a highlight along every edge. That one change did more than everything else in this post.

Two smaller fixes made it read correctly:

Colour space. Colours are picked in sRGB (that's what LEGO colour charts give you), converted to linear for the renderer to mix light, and the picture goes back to sRGB at the end. Skip that and the lighting mixes the wrong numbers, so colours come out muddy.
Seam lines that work on dark bricks. Joins between bricks get a thin line: dark on light bricks, light on dark bricks. Without the second half, a black wall looked like one solid slab.

2. Put it on a real table

A model floating in a blue void reads as a render. The same model on a wooden table reads as something you could build.

The table is a top-down photo of our own wooden table, scaled so 1,125 pixels = 34 studs (27 cm), mirrored so it tiles without seams, and fading into a dim room behind the model like a close-up photo. A soft dark patch right under the model grounds it, and the sun's shadow box is fitted tightly around the model, so its shadow is sharp where it touches the table and soft further out.

Shelby Cobra 427: 69 pieces, 8 steps. One of the easy builds.

Every build's picture on the site (the gallery cards, Open Graph images, the image sitemap) is the real viewer, drawn by a headless browser at 1600×1200 on that table and stored in R2. Same renderer, same look, everywhere.

3. Bricks are hollow underneath

When you open a piece from the parts list, it slowly tips over to show its underside, because that's how you tell a brick from a stack of plates, and a 2×4 from a 2×4 tile.

The underside is real LEGO geometry (1 unit = 8 mm): a 1 mm top and 1.2 mm walls around an open cavity. Pieces 2 wide or more get a hollow tube (6.5 mm outside, 4.8 mm hole) between every four studs; 1-wide pieces get a thin pin. The cavity ceiling and tube holes are drawn dark so you see them as holes, not as a flat bottom.

Only the single-piece viewer draws this. A 1,000-piece model stays solid, because nobody sees the bottom of a brick that's sitting on another brick.

4. Draw less, so 1,300 pieces stay fast

Some of the sculptures have more than 1,300 pieces, and they have to animate on a phone.

Geometry is shared by every piece of the same size, and outlines are computed once per size.
All the studs of one piece are one mesh.
Covered studs aren't drawn. For each stud, the viewer knows the step at which something will sit on it, and hides it from that step on.

That last one had a bug that only showed in the exploded view. With the layers pulled apart, the covered studs were still hidden, so every layer looked smooth on top with nothing to connect to. Now, exploded, every piece shows all its studs.

RoboCop: 91 pieces, 9 steps. In robots and movie builds.

5. Small things that make it feel right

Scrubbing never bounces. Pressing Next drops the new pieces in with a little fall. Dragging the step slider is instant, and a hidden piece cancels any half-finished drop. Before that, scrubbing back and forth made the whole model jitter.
The slider doesn't resize under your finger. The step label used to sit next to the slider; its changing length resized the slider while you dragged it. It moved to its own line.
Tall figures keep their heads. The camera frames the whole model as seen from its viewing angle, not from straight ahead, so tall figures fit.
Degrees vs radians. Some stored builds had their camera angles saved in degrees (az 30, el 35) and the viewer read them as radians, so a few models opened looking up from under the table. Now any angle that's clearly degrees is read as degrees.
Night mode turns the lights down and lets neon pieces glow with a soft bloom, for the city and sci-fi builds.
Hover to explain. Point at a brick and a card says what it is; point at a step in the list and its pieces light up while everything else fades.

Sans (Undertale): 116 pieces, 10 steps. More video game builds.

Try it

Every build on the site uses this viewer, and every guide prints as a booklet or a PDF. A few to turn around:

Rick's Space Cruiser from TV shows and space
Xbox Series X with a controller
Bluey's Queenslander house from buildings

Or snap a photo of your own loose bricks at misterblocks.com: it works like a LEGO scanner app and gives you custom instructions for builds made from exactly what you have.

Unofficial fan builds. LEGO® is a trademark of the LEGO Group, which does not sponsor or endorse this site. Characters and brands belong to their owners.

Voxels aren't LEGO: teaching a brick packer to build like a person

Oleksandr Gamanyuk — Wed, 11 Nov 2026 22:29:18 +0000

When you ask an AI for "SpongeBob's house in LEGO", the hard part isn't the shape. The hard part is that a shape made of cubes isn't a LEGO model. Real builders don't stack 1×1 bricks into a block. They cross the layers so they lock, use one big plate for a base, leave the inside hollow, and use as few kinds of parts as possible, so you can actually collect them.

This post covers the packer I wrote for Mister Blocks, and the rules it learned, mostly from me looking at a build and saying "nobody would build it like that".

Bluey's Queenslander house: 128 pieces, 15 steps. One of the LEGO buildings the packer made.

Step 1: draw it like a top view, layer by layer

A build is described as a stack of layers. Each one is a top-down drawing where every character is one stud of a colour:

const g = brickGrid({ W: 16, D: 12, colors: { Y: ['#f2cd37', 'Yellow'], W: ['#ffffff', 'White'], G: ['#4b9f4a', 'Green'], R: ['#c91a09', 'Red'] } });
g.base(0, 'G');                                // one base, from the biggest real plates
g.layer(1, 'brick', ['..YYYY..', '..YWWY..']);  // rows of studs, '.' = empty
g.slope(4, 2, 0, 2, 4, '+z', 'R');             // hand-placed parts take their cells first
g.tall(2, 0, 0, 1, 2, 'W', 3);                 // one tall brick instead of three layers

Then pack() turns each layer into real bricks. That's where all the rules live.

Rule 1: cross the layers, stagger the seams

The first version filled each layer greedily with the biggest brick that fit. It looked right in 3D and would have fallen apart in your hands: every layer had its seams in the same place, so the model was really a set of loose columns.

So odd layers prefer pieces along one axis and even layers along the other. And when several pieces fit, the packer scores them by area plus half again if the piece bridges two or more pieces of the layer below. That's what builders do without thinking: put the 2×4 across the join.

Rule 2: one base, from real plates

A base is one layer of the biggest plates LEGO actually makes (16×16, 8×16, 6×16…), never two thin layers glued together. And a plate never sits exactly on an identical plate, because that pair only holds itself.

Gundam RX-78-2: 91 pieces. More in LEGO mechs and anime builds.

Rule 3: nothing rests on air

This one took the longest. A drawing happily puts a brick where there is nothing underneath it: an arm sticking out, the brim of a roof. In 3D it looks fine. In plastic it falls off.

The fix has two parts:

Overhangs are packed first, in a piece that also covers a cell that does sit on a stud. The overhanging piece is held from below, the way you'd build it.
If a lone overhang still can't be bridged, its colour is carried inward along the shortest path of drawn cells to the nearest supported cell, and the layer is packed again. Those inside cells are almost always hidden, so the look doesn't change, but now one piece spans from the support to the overhang.

A cell that one overhang has already claimed for its colour is never taken back. Without that rule, two overhangs side by side kept trading colours forever.

The report at the end fails the build if any piece has nothing under it.

Rule 4: hollow inside

A solid 2,000-piece sculpture is a 2,000-piece sculpture nobody will build. Real display models are hollow.

After packing, the packer flood-fills "outside air" from beyond the grid and above the top layer. Every drawn piece that touches no outside air (from the sides, above or below) can't be seen, so it's removed, one at a time, as long as the model still holds together. It only ever removes pieces, so a build never gets bigger.

Mega Greninja as a 1,183-piece sculpture, with clear supports added until the whole model is one connected piece. See all LEGO sculptures and Pokémon builds.

Rule 5: a short parts list

My first review of a small build was blunt: it was hard to collect and hard to build when such a simple model needed 25 different kinds of parts.

So after packing, the packer keeps trying to drop the rarest part type (one shape in one colour) and repack, as long as the list gets shorter and the model still holds together. One fewer kind of part is worth up to 4 extra pieces. 1×1s always stay as the fallback.

There's one guard: if a ban only works because the repair recoloured cells (an eye turned into face colour), it's rejected. A shorter list isn't worth erasing the design.

Rule 6: tall bricks, like a person would use

Then came the second round of review: why three stacked 1×2 bricks when there's a 3-high block? Why three plates when a brick is the same height?

restack walks every straight vertical run of pieces with the same footprint and colour, and re-cuts it into the fewest real pieces: tall bricks first (1×2×5, 1×2×3, 2×2×3…), then bricks, then plates. Only parts that really exist are allowed; asking for a tall brick LEGO doesn't make throws an error that lists the ones it does.

It's always safe, which is why it runs on every build: inside a run, every face is covered by the run itself, so nothing else was touching those faces. The top and bottom of the run (what it clips onto, and what clips onto it) stay exactly the same. A chimney becomes a 2×2×3; a wall becomes one 1×6×5.

Totoro: 156 pieces, 13 steps. Also in movie builds.

Rule 7: every step must be buildable

Finally the steps. Layers are grouped into instruction steps of at most 18 pieces, and arms or tails can be named sub-assemblies with their own step. Then a build-order pass checks each step: if new pieces would only be held by a piece that arrives later, that piece is brought forward. Moving a piece earlier can't break an earlier step, and a later step only gains support, so it repeats until nothing changes.

What it adds up to

Each rule came from looking at a finished build and asking whether a person would really build it that way. None of them are clever on their own. Together they're the difference between "a 3D shape in LEGO colours" and something my son can actually put together on the table.

If you want to see the results, every build has free step-by-step instructions you can turn in 3D and print. Browse by category: buildings, anime, mechs, sculptures, TV shows or easy builds. Or snap your own pile of bricks at misterblocks.com and get build ideas from exactly what you have.

Unofficial fan builds. LEGO® is a trademark of the LEGO Group, which does not sponsor or endorse this site. Characters and brands belong to their owners.

I built a LEGO builder for my son with Claude. The hard part was making sure no brick floats

Oleksandr Gamanyuk — Wed, 11 Nov 2026 06:53:19 +0000

I built Mister Blocks for my son. We have the classic giant box of mixed LEGO: sets long broken up, instructions long gone. Every time he asked "what can I build?", I didn't have a good answer.

Now he takes a photo of the pile, and a few minutes later there are a few builds made only from the bricks in that photo, with 3D step-by-step instructions he can turn, zoom and print.

The first working version came out of the very first prompt to Claude. The next few days went into the part nobody warns you about: making sure the build actually holds together in real plastic. This post is about that part.

The pipeline

There is no framework. It's a handful of Vercel serverless functions in plain Node, MongoDB for sessions, Cloudflare R2 for images and PDFs, and three.js for the 3D view:

api/sessions/[id]/recognize   photo  -> inventory (every piece: kind, size in studs, color)
api/sessions/[id]/design      inventory -> one model (parts with x/y/z positions + steps)
api/sessions/[id]/chat        "make it a tow truck" -> a changed model, same checks

Recognize. Claude (Opus 5.5) looks at the photo and returns structured JSON. Each piece gets an id (B3 is a brick, P7 a plate, W1 a wheel), its size in studs, its height in plates and its color. If it miscounts, you can correct it in the chat ("there are 48 of the 2×4s") and the inventory is reconciled without breaking the ids the models already use.

Design. Claude gets the inventory and returns a model: every part placed on a grid (x and z in studs, y in plates, where a brick is 3 plates tall), plus which step it goes in.

And that's where it gets interesting, because a language model's first answer is almost right very often. Almost right in LEGO means a brick floating in mid-air.

The stud checker

Every model goes through a checker before anyone sees it. It's one plain JS file (core.js), and the server and the browser load the exact same file, so what the page shows and what the server accepts can't disagree.

It checks the things that make a build fail in your hands:

Every piece is held by studs. A piece's bottom cells have to land on studs of something below it, or the piece has to sit on the table. Slopes only have studs on their top row, so the checker knows that too.
No two pieces occupy the same space.
Only pieces you actually have. Every part has to exist in the inventory, be of the same kind, and be used only once.
Wheels can turn. Nothing may come lower than the deck inside a tire's zone, except the mudguard over its own tire.

The checker returns problems in plain English, and they go straight back to Claude:

// lib/designer.js (simplified)
while (rounds < MAX_ROUNDS) {
  rounds++;
  const { data, message } = await askJSON({ system, messages, schema });
  model = toModel(data, inventory);
  issues = problems(model, inventory);   // "B4 is held by nothing", "P2 overlaps P5", ...
  if (!issues.length) break;
  messages.push({ role: 'assistant', content: message.content });
  messages.push({ role: 'user', content:
    `The checker found these problems:\n- ${issues.join('\n- ')}\n\nFix every one and answer again in full. Same rules.` });
}

Up to three rounds. In practice most models pass in one or two. This loop does more for quality than any amount of prompt tuning did: the model is good at fixing a specific, named problem, and much worse at avoiding every possible problem up front.

Steps you can actually build

The next surprise was in the instructions, not the model. A model can be perfectly valid at the end and still impossible to build in the given order, for example when step 3 places a piece that is only held by a plate that arrives in step 5. You'd be holding that piece in the air.

So there's a second pass (fixOrder) that walks the steps. For every group of new pieces in a step, it checks that they clip onto something already built or stand on the table. If they don't, it pulls the holding piece forward into that step, the way a real builder would put the plate on over the pieces it holds. Moving a piece earlier never breaks an earlier step, and a later step can only gain support, so the pass is safe to repeat until nothing changes.

Rules learned from a kid's truck

The prompt has a section of rules that came from real builds, mostly from my son's own truck. For example, the chassis that works: two wheel plates in a row, joined by a 2-wide strip of plates, a mudguard over each wheel set one stud outward, and the deck resting on the mudguards plus a center block. Before that rule existed, a lot of generated cars had decks that sat on the tires.

The rules are written as plain sentences, the same way you'd explain them to a person.

The viewer

The 3D viewer is a small three.js file shared by the build page, the guides and the print view. Every brick gets a gentle sheen and highlighted edges, so it reads as plastic rather than as flat boxes. You drag to turn, scroll to zoom, and step through the build one step at a time; pointing at a step in the list lights up its pieces and fades everything else back. Every guide can be printed as a booklet or saved as a PDF, and the guides you've opened work offline.

Prompt mode and bigger builds

Not everyone has a pile handy, so you can also describe a build in words. A few of my favourites, each a free guide you can turn in 3D, step through and print:

Boston Dynamics Spot, with its arm

The yellow-and-black robot dog on four backward-bending legs, reaching for a valve on a factory floor. 150 pieces, 13 steps. Build it →

SpongeBob's Pineapple House

The pineapple under the sea: round, wider in the middle, with the blue door and a crown of leaves. 237 pieces, 18 steps. Build it →

Porsche Mission S

Porsche's vision car in white, with the black canopy and the full-width red light bar. The big one: 965 pieces in 73 steps. Build it → (Want something quicker? There's also a 69-piece version.)

Mega Dragonite

A voxel sculpture of the dragon Pokémon: 1,315 pieces in 121 steps. Build it →

Cost

Each build takes a few minutes and costs a couple of dollars in API calls. Every Claude call's tokens and cost are logged to MongoDB, and there's a per-visitor daily limit plus a site-wide daily budget, so a busy day can't turn into a surprise bill.

Try it

It's free: misterblocks.com. Snap your pile and see what it comes up with. I'd love to hear what you build, and especially what it got wrong.

Unofficial fan project. LEGO® is a trademark of the LEGO Group, which does not sponsor or endorse this site. Spot, SpongeBob SquarePants, Porsche and Pokémon belong to their owners; these are fan builds.