A new system gives vision AIs accurate spatial reasoning by having them call specialized tools for 3D geometry and object location rather than guessing from a single image. The open, freely-available version reportedly matches closed commercial models on spatial tasks, and it keeps a persistent memory across multiple views — stitching different camera angles into one consistent picture instead of treating each frame as a fresh, amnesiac snapshot.
Key facts
- What: Vision AIs are surprisingly bad at precise 'where is this in 3D space' questions. This one stops guessing and calls dedicated spatial tools, while keeping a memory across views.
- When: 2026-06-19
- Primary source: read the source (arXiv 2606.20515)
Today's vision models describe pictures well but struggle with precise spatial questions — how far the mug is from the laptop, whether it sits left or right from your vantage point, whether it would fit on the shelf above. They tend to eyeball it and guess. The new system takes a different approach: instead of asking one model to intuit 3D geometry in its head, it lets the model reach for the right instrument. The setup treats the AI less like an all-knowing oracle and more like a smart project manager. When a spatial question comes up, it calls a specialized tool for the job — one that precisely locates objects in the flat image, another that reasons about actual 3D geometry and distance, another that knows general facts about how space and objects work — and then combines what those tools report. Each tool does one narrow thing well, and the model's job is to pick the right one and assemble the pieces, rather than to be secretly good at everything at once.
The cross-view memory is exactly the ingredient a separate result this week found missing in AI world models, which forget whatever drifts off-screen. Two different teams converging on "you need a lasting record of where things are, not just a pretty picture of the current frame" is a good sign that the field has found a real, shared gap.
The open model holding its own against closed commercial models on spatial tasks — which usually win on raw scale — is a recurring theme: when a problem has clear sub-structure, "let the model orchestrate the right specialized tools" often beats "make one giant model bigger and hope spatial sense emerges." A person answers a hard distance question not by staring harder but by grabbing a tape measure. We don't expect a brilliant novelist to also be a surveyor; we expect them to know when to call one.
The stakes are real. A pair of AR glasses telling you "the exit is twelve feet to your right, behind the pillar" has to be right about that, not vibes-right. A home robot reaching for a dropped pill bottle has to know exactly where it is in three dimensions, and remember it's still there after someone walks past and blocks the view. A confident spatial guess in these situations isn't just wrong — it's useless or dangerous. It's the same precise-spatial demand that makes a task like a robot seating a graphics card into a motherboard so hard — millimetres matter, and "roughly there" fails.
The deeper tension is about how AI gets good at the physical world at all: make one enormous model and hope competence emerges from sheer scale, or build a capable orchestrator that knows which specialized tools to call and how to combine them. This paper is a strong data point for the second camp, at least for spatial reasoning — a domain about as structured and rule-governed as the real world gets, exactly where dedicated tools should shine.
The honest limits: this is days-old research, measured on a specific battery of spatial tasks, and "matches the closed models" is a claim made against particular benchmarks rather than the messy real world. Wiring up specialized tools adds complexity and new ways to fail compared to one self-contained model — every tool is another thing that can break or be called at the wrong moment. But the direction is compelling, because it lines up with where the field keeps landing: for problems that have real structure — and 3D space is about as structured as it gets — teaching an AI to use the right tool tends to beat asking it to wing the whole thing in its head.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)