Every mixed reality demo you have seen on a conference stage is one person in a headset, moving a hologram around an empty room. It looks like the hard part is done. It is not. It is the easy version, and it quietly skips the only problem that matters once real users show up.
Put a second person in that room, looking at the same object, and the engineering problem changes shape completely. Now two headsets have to agree, to the centimeter, on where that object lives and what it is doing, and they have to stay agreed while both people walk around it and grab it. That agreement is the whole discipline of collaborative MR. The 3D content is the part users see. The agreement is the part that decides whether the thing works.
I have built shared mixed reality environments, and this is the mental model I wish someone had handed me before the first one.
Three problems wearing one trench coat
A collaborative MR app is not one system. It is three, and each one can sink the experience on its own.
Shared spatial anchoring. Both headsets need a common coordinate system mapped onto the real room. That is the foundation. If headset A thinks the model sits on the table and headset B thinks it is fifty centimeters to the left, the shared illusion dies instantly, because each person is pointing at empty air as far as the other one is concerned. Establishing that shared frame and holding it as people move is the core problem the category exists to solve.
State synchronization. Once the room is shared, the contents have to stay shared. Somebody rotates a model, moves a part, marks something up, and every other headset has to reflect that fast enough that it reads as one object rather than a laggy copy. This is exactly the problem that makes multiplayer games hard, and it carries the same tax: authority, conflict handling when two people grab the same thing, and graceful behavior when someone's network hiccups.
Presence. A shared room only feels shared if you can sense the other people in it. Where they are standing, what they are looking at, what they are pointing to. Representing position, gaze and gestures is its own layer of work, and it is usually what separates a session that feels collaborative from one that feels like ghosts editing a scene.
If you scope only the first one, you get an object that everybody sees in the right place and nobody can meaningfully touch together. If you scope only the second, you get a beautifully synced object that lands in a different spot for each person. They are not optional relative to each other.
The version I shipped
The clearest example from my own work is MR Camera, a mixed reality environment where several users place and interact with 3D models in the same shared space at the same time. Multiple people, each in their own headset, drop objects into one room and work with them together, seeing each other's changes live.
That build only works because all three systems hold at once. The anchoring has to give every headset the same map of the room, so a model one person places lands in the same real spot for everyone else. The synchronization has to carry every placement, move and edit across all participants without drift. Presence has to make it obvious who is doing what, or people talk over each other and reach for the same object.
The failure modes were never in the scene. They were in the seams. That matches what I found feeding live data into location based AR: the rendering is the well trodden part, and the integration underneath is where the real engineering time goes.
What actually moves the cost
When someone describes a collaborative MR idea to me, I do not ask about the art. I ask five questions, because these are the multipliers that decide the build.
How many people share the space at once? Two is a completely different system from twenty. Every extra participant adds synchronization load and new edge cases. Headcount is one of the biggest levers on the whole project, and people underestimate it constantly because in their head everyone is politely taking turns.
Co-located, remote, or both? People in the same physical room can share spatial anchors directly. Remote participants need a networked shared coordinate system and real infrastructure behind it. Supporting both is not one feature with a flag. It is two solutions.
How precisely must objects align? "Roughly on the table" and "this bolt lines up with that real hole" are different builds. Tight alignment held across multiple headsets is a guarantee you pay for.
How much can people edit together? Viewing a shared model is light. Several people simultaneously moving, editing and building brings conflict handling and much heavier sync. This is the requirement that most often turns a two month project into a six month one.
Which headsets? The target device sets your anchoring capabilities, your comfort ceiling and your cost per seat, and it ripples through every decision downstream. Decide it early or decide it twice.
How I would sequence it
The most expensive mistake in this space is commissioning a broad many user remote and local platform before proving that two headsets can agree on where one object is. I have watched that order of operations burn real budget.
Prove two headset agreement first. Get two devices to hold one model in the same real spot, then stand in the room with a colleague and confirm it with your own eyes. Not in a screenshot, not in the editor. In the room. Everything downstream is comparatively predictable once shared anchoring feels solid.
Start co-located before remote. Two people in one room is the simplest honest version of the problem. Nail that before you take on the networking and infrastructure that remote sessions demand.
Treat synchronization as a first class workstream. If people edit together, the state sync, conflict handling and failure behavior deserve their own budget line, the same way they would in any multiplayer build. Sneaking it in as "we will just network the transforms" is how projects discover authority bugs three weeks before a demo.
Add participants deliberately. Every extra person is a multiplier, not a free seat. Prove the interaction with a small group, then scale.
The test that saves you the most money
Before any of this, one question decides whether you should be here at all: does a second person need to see and touch the same thing in the same place?
If no, single user MR does the job for a fraction of the cost, and you should build that instead. I tell clients this regularly and it is not false modesty. Collaborative MR is a genuinely harder class of system, and paying for it when nobody needs it is the worst outcome in the category.
If yes, then you are not building a 3D viewer. You are building a distributed system that happens to render in a room, and the thing to get right is the agreement, not the geometry.
The 3D was never the hard part. Making a room full of headsets believe in the same object at the same time is.
If you are scoping something like this and want the longer version, including the honest cost bands, I wrote it up at Null Studio.
Top comments (0)