When Latency Becomes A Training Data Defect

As digital twins take over remote robot control, what matters most may be how faithfully the operator’s view matched the world the robot was actually standing in.

Somewhere right now, a person is driving a machine they cannot see with their own eyes. They are watching a screen that shows a reconstructed version of a warehouse, a surgical field or a construction site, and they are moving a controller while a robot somewhere else repeats what they do. The whole arrangement rests on one quiet assumption: that the picture on the screen is a fair description of the world at the moment the operator acts on it.

Networks do not guarantee that. Packets arrive late, out of order, or not at all, and the reconstructed scene drifts away from the real one in ways the operator cannot feel. The robot is then being steered by a slightly stale version of reality.

The Clock Is Part Of The Signal

A recent paper called K-AoI takes up exactly this question. The authors report a single spatio-temporal measure that scores how well a digital twin stays synchronized with the physical scene as network conditions change, combining how old the information is with how well the mirrored scene lines up in space, rather than relying on delay alone; the paper reports its result as the metric itself rather than as a headline score. For anyone building or backing remote robot work, that means the quality of a session becomes something you can read off the record instead of something you infer from how the operator says it felt.

I think this is one of those shifts that looks narrow and is not. For years the industry has talked about latency as a comfort problem, something that makes remote work feel sluggish. Treating it as a measurable property of the data changes what you can do with the recording afterwards. To me, the useful question is not whether a teleoperation session felt responsive. It is whether, six months later, anyone can say which frames in that session were trustworthy and which ones described a world that had already moved on.

That matters because teleoperation is not only a way to get work done today. It is the main way the field currently manufactures robot demonstrations. Every remote session produces a record of what was observed and what was commanded, and that record is the raw material behind the models meant to eventually do the job without a human in the loop.

Why Geography Stops Being Neutral

Once you accept that framing, two unglamorous things start setting the economics. The first is physics: light and switching gear impose a floor on how fast a command crosses a distance, so where the operator sits relative to the robot is a hard constraint rather than a preference. The second is labor: a timestamped stream of video and control inputs is not training data until trained people go through it and mark what happened, when it happened, and whether the operator’s view was describing the scene correctly at that instant.

That second part is the one that gets underestimated. Someone has to sit with the footage and annotate it frame by frame, and that capacity has to exist, be managed and be held to a consistent standard before anything downstream is possible. We have that capability through our acquisition of Flipside AI, which provides video and temporal annotation and data collection services for AI systems.[1] Following that acquisition, our consolidated operations consist principally of the operations of Flipside AI.[1]

Southeast Asia sits in an unusual position on both constraints at once. It is close enough to the industrial robot deployments of Asia for network round trips to stay short, and it has deep pools of people who can be trained to annotation standards that hold up across millions of frames. Those two facts are rarely discussed together, and they probably should be.

What The Record Has To Prove

A synchronization score is, underneath, a quality label on a piece of data. It says: at this moment, the operator’s picture was this fresh and this well aligned, so the action taken in response to it means what you think it means. Without something like that, a training corpus built from remote sessions carries invisible defects, and the model learns from moments where the human was reacting to a world that no longer existed.

What I would watch over the next year is whether metrics of this kind migrate from control papers into data specifications. The moment a buyer of robot demonstrations starts asking for synchronization figures alongside episode counts, the conversation about what good data costs changes completely, because freshness and alignment are expensive to guarantee and cheap to fake.

The bottleneck in embodied intelligence has never really been the models, and teleoperation makes that unusually clear. The hardware works, the networks mostly work, and the policies improve steadily. What remains hard is producing an honest, well-described record of physical experience, with the clock and the geometry attached. I think the teams that treat that record as the product, rather than as exhaust from a control session, are the ones whose robots actually learn something.

Sources

1. NOTE 1, ORGANIZATION AND DESCRIPTION OF BUSINESS, 10-Q, filed 2026-08-14, https://www.sec.gov/Archives/edgar/data/1976663/000147793226005094/nexscient_10q.htm

For disclosures, see this article on our website: https://nexscient.ai/when-latency-becomes-a-training-data-defect/#disclosure