An identifier mismatch inside a vendor app rarely raises anything. The app mints a second object out of thin air, keeps serving your real one, and every log line downstream reads like a success. That's where this hunt started, one capital letter deep in a python generator, on a stream deck I rebuilt as a control surface for my agent runtime.
The build was concrete: a Ulanzi D200H, fifteen cells on a 3x5 grid with fourteen usable and one ghost key at the corner, three pages of live tiles, thirty actions across them, all of it generated by a python manifest script so nobody clicks anything in the vendor editor. The vendor software spawns a node bundle per plugin, talks to it over a websocket on 127.0.0.1:3906, and paints tiles from absolute png paths. Profiles are plain directories scanned when the app starts, there is no index file you can correct after the fact.
What the phantom looked like
Every restart booted onto a default page nobody authored. Two junk profile ids appeared in sequence, 933d1361 then 9d5e89a6, each carrying exactly one key. My three hand-written pages were still on disk, still valid json, still rendering, but the app couldn't resolve its current-profile name, so it built a fallback page and kept reporting success. Nothing in the console changed. The deck worked half the time and I couldn't tell which half until I diffed the app's own state files.
Where the fork lived
The app indexes devices by serial, case included. My generator wrote the serial with an uppercase letter where the device key the app minted was all lowercase. One character, two device keys. All three of my profile dirs bound to that second key, so the app believed I owned a phantom D200H sitting next to the actual deck, and every lookup for "which page is active" ran against a device with no pages.
Finding it took one read of the sort plist under the app support dir, because ProfileSort keys itself by device id. Two entries, one capitalized differently from the other, named the fork on sight: a real deck and a ghost. There was no stack trace and no error string anywhere, that plist diff was the only hard evidence. A data structure that had been right for two years went wrong because of how I typed a constant.
The fix, four guards deep
Lowercasing the serial in the generator was the one-line change, with a comment warning never to touch that casing again, since the value looks arbitrary and someone will clean it up later. Three more guards made the fix survive a reinstall:
- Kill the host app and the plugin before touching any profile directory, because the app rewrites its state on exit and resurrects whatever garbage it last wrote.
- Wipe all profile dirs before install, because recursive copy nests into existing directories and leaves stale tiles behind that still pass a json check.
- Pin the current page explicitly in the settings file so the app can't infer it from what it happens to find.
- Delete junk pages inside the build step itself, before the install finishes.
Verified after: exactly three profile dirs on disk, ProfileSort down to one device key with three page names under it, nothing minted at restart, one plugin process connected with events flowing. The phantom key disappearing from the plist is the proof that matters here, pressing buttons would only prove I have fingers.
Two neighbors in the same layer
The reconnect logic carried the identical disease one event earlier in the stack. My connect helper resolved its promise the moment the socket opened, so a reconnect loop stacked a fresh websocket every 500 ms and logged 183 connected lines inside a couple of minutes while the deck sat there working. Resolving on close fixed it, one live connection, no orphan sockets. Killing the host app by name orphaned the node child the same way: the app died, the child stayed alive, and when the app respawned both instances dispatched every button press. Now the plugin writes its pid into its cache dir and SIGTERMs the stale instance at startup before taking the lock.
A third miss hid inside an action id. A flip key kept rendering "?" placeholder tiles because I typed a UUID namespace off an old version of the software, com.ulanzi.page.switch, while the actual action lives under the system pages plugin with an extra token in the path. The key dispatched correctly while the tile art refused to resolve, half-working, and that split behavior is what dragged the bug into the camera round later that evening.
What generalizes past this device
String equality in a vendor layer has two outcomes, your bytes match or the app builds another object and keeps reporting success. Nothing you write can assume case folding, trimming, or normalization survives from your config file to the comparison happening three layers away, so canonicalize at the writer and treat ids as opaque bytes, with no readability pass allowed between those two points. Then verify against the index structure the app maintains (here a sort plist keyed by device id), never against the directory names you authored, since in this case the directories were right and the binding around them was wrong. When something changes at boot with no error line anywhere, check object identity before logic: count keys, dirs, processes, sockets, and compare each count against what the build claims.
The handshake lesson travels further than the deck. A promise that resolves on the event that starts an object leaks every retry; resolve it on the event that ends the object, and let one process own the lock at a time. Both bugs here cost minutes to find once I read the state files, and weeks of guessing if I'd trusted symptoms alone, which is why asserting counts after every install step beats reading logs.
Two years of working config went wrong because of how I typed a constant, and the app's answer was to build hardware I don't own. What's the quietest identifier fork you've shipped, the one where both objects kept working until someone finally noticed the extra?
Top comments (0)