DaC
Posted on
The difficult part is not necessarily writing the first prototype but delivering on Xbox
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (8)
This is the part most writeups skip. What I liked in the XCP chapter is that the physical Xbox run was treated as the test, not the demo: Open RTS launched fine and still went back for another pass because the interface didn't read well on the real screen. Same lesson I keep relearning with field devices: the prototype proves it can work, the hardware proves it does.
The line about what the results do not establish (no full source equivalence, explicit substitutions) is rare, and it makes the rest more credible.
Curious: of the 15 playability checks, which ones were hardest to make repeatable from build to build?
Thanks,that's exactly the distinction I was trying to preserve. The hardest checks to make repeatable were the human-facing visibility/readability ones, especially in Open RTS. The underlying selection → movement → combat → victory sequence was comparatively easy to bound with expected events and state transitions.
What was harder was proving that those transitions were actually readable and usable on the physical display from build to build.
Open RTS was the clearest example: the first physical run executed, but the normal XCP diagnostic overlay interfered with the declared game UI. So that run did not pass. I changed the generic platform behavior rather than adding an Open RTS-specific workaround, rebuilt it, and ran the physical test again. Its six checks covered scene/minimap/UI readability, absence of the routine diagnostic overlay, readable selection/movement/combat feedback, and usability of the complete bounded loop.
Minedot's analog movement/camera checks were probably the next most sensitive because they also depend on what a human can actually perceive from controller input, rather than only on a deterministic state transition.
That's also why I kept the 15/15 result separate from source-equivalence claims: the tests establish a bounded playable result on that hardware, not that the reconstruction is identical to the original project.
That detail is the best part: fixing the generic platform behavior instead of adding an Open RTS workaround. Per-case workarounds are how a platform quietly turns into a pile of exceptions; I have seen the same with protocol parsers that grow one special case per device model.
And the split you describe matches what I see in telemetry: state transitions are easy to assert, but "can a person actually see it" is where systems pass every test and still fail in the field. Do you keep reference captures per build to compare against, or is each physical run judged fresh?
It's a mix of both. I keep captures from previous runs, with hashes and execution evidence, but I don't treat them as golden images.For Open RTS, the failed run's four captures were preserved alongside the new ones. After fixing the UI, I rebuilt and repeated the physical test with the same checks. It went from 3/6 to 6/6.
I also use automatic checks for state changes and, in some cases, frame changes. But readability still needs a human looking at the actual screen. So previous runs help explain what changed, but each new build has to prove itself on the hardware.
"Each new build has to prove itself on the hardware" deserves to be on a wall. Keeping the failed run's captures next to the passing ones is a nice touch too: the 3/6 is as much evidence as the 6/6. Thanks for taking the time to explain it, I'm looking forward to the next XCP chapter.
Thanks, Israel! I've really enjoyed this exchange. And yes, that 3/6 probably taught me more than a clean pass would have. Really appreciate you taking the time to read so closely. And yes, i have one more chapter, It will be about SSD.I thought I knew everything about XCP, but there's a final chapter that I think is really important, and I forgot about that. It was yet another journey into the dark that required a lot of testing. I had to iterate and do a lot of thinking there too to figure out what the limit was. So yes, there will be a separate chapter
An SSD chapter sounds great. Storage is exactly where the real device stops agreeing with the spec. I'll be watching for it.
dev.to/danielecangi/i-tried-to-bui...