Snapping a tap to a real corner: building a LiDAR tape measure for iPhone

Max — Tue, 10 Nov 2026 22:44:48 +0000

When you tap the screen to start a measurement, you are not tapping the corner of the table. You are tapping a pixel, and that pixel sits on a ray that leaves the camera and hits something. Often the something is the floor 40 inches behind the table, because your thumb landed a hair past the edge.

This post is how the tape measure in our iPhone app deals with that: two kinds of evidence (depth and image), a little geometry, and a filter that refuses to let the point jitter.

Depth alone is too blurry

On a Pro iPhone, ARKit hands you a depth map with every frame (sceneDepth, plus smoothedSceneDepth when the device supports it) and a confidence map next to it. It is wonderful and it is coarse. The reconstructed mesh is a few centimetres per triangle, so it rounds off exactly the thing you want to measure to: the edge.

It is also blind to edges that are flat on a surface. A picture frame on a wall, door trim, the printed edge of a box: depth sees one plane. The camera image sees a sharp line.

So the snapper looks at both, every frame, in a small window around the reticle.

Step 1: find the surfaces in a window of depth

We back-project the high-confidence depth samples around the reticle into world space and look for up to three planes with RANSAC. A few details that mattered more than we expected:

// Inliers within 3 mm or 0.4% of the distance: LiDAR noise, not a different surface.
let threshold = max(0.003, 0.004 * depth)
let minInliers = max(12, points.count / 10)

The tolerance grows with distance. A fixed 3 mm (about 1/8 in) works up close and splits a wall into slabs at 10 feet.
Merge near-parallel slabs. LiDAR noise can turn one flat surface into two planes a few millimetres apart. If you keep both, the nearer one wins every ray and your edge ends land on different "surfaces". Planes within about 5° and 1 cm of each other get merged and re-fitted.
The median distance by selection, not sorting. Quickselect with a median-of-three pivot gives the same value in a fraction of the time, and this runs every frame.
A seeded random generator. RANSAC is random; we use a tiny SplitMix generator seeded from the point count, so the same depth window always gives the same planes. That made the whole thing unit-testable.

The plane fit itself is plain least squares: the normal is the eigenvector of the covariance with the smallest eigenvalue, found with a few Jacobi sweeps on a 3 × 3 matrix.

Step 2: turn surfaces into corners and edges

Once you have planes, corners and edges fall out of the geometry:

Corner: the point where three planes meet. We only accept it when they meet clearly (the determinant of the three normals above 0.2), the way a box corner or a room corner does. Nearly parallel planes give a "corner" miles away.
Edge (a fold): the line where two planes meet, but only if they meet at more than about 25°. A slightly bowed wall is not an edge.
Nearest surface wins. When the reticle's ray passes a table edge, the edge belongs to the table, not the floor behind it. We take the nearest surface that actually has measured points near the hit, with a little slack because LiDAR blurs the last centimetre at an edge.

Step 3: ask the image for the edges depth can't see

In parallel, we copy a small grayscale patch around the reticle (a few thousand pixels, at half resolution) and look for the strongest edge or corner nearby, to a fraction of a pixel. Corners use Shi–Tomasi on a 5 × 5 structure tensor and are then refined to the point every nearby edge line passes through.

The number that took the longest to get right was the contrast floor:

// Edge contrast below this (8% of full brightness) is texture or noise:
// wood grain, carpet, wallpaper, a shadow's soft rim.
// At 5% the reticle got pulled about on textured surfaces.

When depth and image agree on a point, the snapper needs fewer confirming frames before it locks.

Step 4: hold the point still

Finding the corner again in every frame makes the dot shimmer, even when every individual find is "right". So a locked corner is held as one fixed point in the room, a Kalman filter on a static point, and each new find refines it:

Each find is weighted by how sure it is: tight across the camera ray, loose along it (depth error is mostly along the ray). The longer you aim, the steadier it gets.
A find too far from the estimate is gated out as noise (χ² for three degrees of freedom at 99%, about 11.3).
Three rejected finds in a row means you really moved to a different corner, and tracking restarts there.
A small random-walk allowance lets ARKit's own map corrections come through instead of fighting them.

Edges get the same treatment as a line in the room: the snapped point is where that line passes closest to the reticle's ray, so the dot slides smoothly along a table edge instead of hopping between separate finds.

Without LiDAR

Most iPhones don't have LiDAR. There, the tape uses ARKit's estimated planes and the image snapping, and the scale comes from the camera alone, which is where most of the error lives. Our fix is low-tech: lay any bank or ID card on a table. ID-1 cards are 85.60 × 53.98 mm (ISO/IEC 7810) everywhere in the world, so once the app finds the card it knows the true size of something in view and can correct the scale for that phone.

Room scans split the same way. With LiDAR, Apple's RoomPlan finds the walls, doors and windows while you walk the room. Without it, you tap each corner where the wall meets the floor, type the ceiling height, and get the same floor plan.

Then the part people actually want: the answer

A length is rarely the question. "Will the sofa fit through the door?" is. That check turned out to be a nice little piece of geometry:

Sort the item's three dimensions so a ≤ b ≤ c.
It goes through lengthways (along c) if a fits the door's width and b fits its height.
Which of width, depth and height ended up as a and b tells you how to carry it: on its back, upright, or on its end. Ties prefer on its back.
If that fails and it's a sofa, try it upright and tilted back, where the diagonal depth (from the bottom back corner to the midpoint of a straightedge laid from the top of the back to the front of the arm) crosses the door's width.

static func verdict(spare: Double) -> FitVerdict {
    spare >= comfortableMargin - 1e-9 ? .yes : (spare >= -1e-9 ? .tight : .no)
}
// comfortableMargin = 0.02 (2 cm, about 3/4 in): under that it says "tight".

The 1e-9 slack is there because a door measured as exactly the sofa's width should say "tight", not "no", after floating-point arithmetic has had its say.

We also wrote the classic hallway-corner check: the longest item of thickness t that can be carried flat around a right-angle corner between corridors of widths A and B is the minimum over θ of A/cos θ + B/sin θ − t(tan θ + cot θ). It's in the code and has tests, but it isn't in the app yet: today the app checks the doors on the route, not the turns between them. The open question for us is the UX, since it means asking people to measure two corridors and the item's thickness. If you've solved that nicely, I'd love to hear how.

What we'd tell anyone building this

Use both signals. Depth finds the surfaces; the image finds the edges on them. Either one alone snaps to the wrong thing in common rooms.
Make every random step seeded. A geometry bug you can't reproduce is a geometry bug you can't fix.
Filter the point, not the pixel. Once a corner is a point in the world, stability comes almost for free.
Be honest about limits. Plain white walls and glass are hard for every phone, and for a critical cut you should still check with a real tape.

We're building this into Tapepal, an iPhone tape measure that gives you the answer, not just the number: point at a door, a wall or a room and it tells you if the sofa fits, how much paint to buy or how many boxes of flooring you need. It's coming to the US App Store first. Happy to answer questions about any of the above in the comments.

(We make Tapepal.)

DEV Community: Max