The bug that froze the window was type-correct. It compiled, cargo check passed, the tests passed, and the operating system reported the application as not responding.
It was a Save As dialog called through the Tauri plugin's blocking API from inside a command, which blocks the event loop the WebKit window is running on. Nothing in the type system objects to that. Nothing in a headless harness notices. The only way to find it is to be a person, at a real desktop, clicking the button.
That defect is the whole finding of this project, and the project took four phases to earn the right to state it.
The setup
Wikicarta rebuilds Microsoft Encarta on Wikipedia — the visual browser, the atlas, the timeline, the research organiser, MindMaze — as a Tauri v2 desktop app in Rust and React, with an instant toggle between Live mode against the Wikimedia API and Offline mode against a local ZIM archive.
It is the third of four experiments in what current AI can do with legacy systems, and it occupies the worst position of the four. HAL/S had a formal spec and an independent interpreter to check against. Nautilus had a novel and built its own physics oracle. Wikicarta has neither. The source of truth is what the product felt like, and the only oracle is human judgement. There is no test that tells you whether the category wheel feels like Encarta.
An AI agent wrote essentially all of it. What follows is where that went well, where it went badly, and the fact that those two places were not the ones I expected.
The modern substrate is never the shape its docs claim
The offline story was supposed to be a pure-Rust ZIM reader. The zim crate looked right: random access, deferred loading, Send and Sync. It panicked with an integer overflow in parse_article_list before finishing the open call, against a real modern archive.
That pattern recurred with no relationship between the cases. TextExtracts HTML turned out unusable as a reader format, so article HTML comes from action=parse instead. Live image URLs came back protocol-relative — //upload..., not the https://... everything downstream assumed.
Three unrelated features, one rule: for a revival, the modern content, format or API is always slightly different from its specification and from the model's memory of it. Probe a real sample before building on it. The AI will confidently build against the documented shape, because the documented shape is what it was trained on.
The decision that paid for everything
Before it was provably needed, the reader was decoupled from its content source behind a single trait:
trait ArticleSource {
fn get_article(title: &str) -> Article;
fn search(query: &str) -> Vec<SearchResult>;
fn get_image(key: &str) -> Vec<u8>;
}
Adding a whole second content source later — live Wikipedia alongside offline ZIM — was two commands and a mode flag. No reader rewrite. Swapping the storage layer from JSON files to SQLite was invisible above the command boundary.
For software that assumed one sealed data source — a CD, a bundled database, a mainframe — inserting that seam first is what converts every later modernisation from a rewrite into an adapter. It is also the single decision an AI agent is least likely to make unprompted, because at the moment you make it there is exactly one source and the abstraction looks like overhead.
Two features in the final phase were pre-paid by decisions like it. The attribution aggregator became a GROUP BY rather than a re-fetch, because provenance had been stamped at save time back when the notes feature was built. Bundling the kiwix-serve binary into the package was a two-line change, because binary discovery had long been a prioritised candidate list rather than a hard-coded path.
Packaging work disproportionately cashes in — or punishes — architectural decisions made when the feature that needed them didn't exist yet.
Testability turned into architecture, which turned out to be good
The only layer testable without a desktop window is pure functions. So the code got steadily shaped so that the hard parts are pure functions: URL rewriting, upstream reconstruction, cache naming, database round-tripping and ordering, bookmark deduplication, legacy JSON import. The Rust suite went from nothing to 23 offline tests, and each feature's desktop-only remainder shrank to a short explicit list.
"What can I verify headlessly?" is a useful question about how to structure code, not just about how to test it. Under an AI agent it becomes close to essential, because the agent's verification loop is only as good as the surface it can reach.
Where the defects actually were
The pure logic — parsing, backoff, the media-cache URL rewrite, database ordering, live-title normalisation — compiled correct and passed on the first or second attempt. Consistently.
Every real bug was at the runtime seam:
- The blocking
Save Asthat froze the WebKit window. - DOM nodes inside an
<iframe>coming from a different JavaScript realm, so aninstanceof Elementcheck in the parent React shell fails on a clicked target that is unambiguously an element. Realm-safe DOM handling is mandatory when the app owns iframe content but drives it from the parent.
Both are invisible to the type system and to any headless harness. Both required a person at the real window.
The implication for anyone running an agent on desktop software: the logic is rarely where the risk is. Human validation time should be spent almost entirely at the host, runtime and OS boundary, and hardly at all reviewing the algorithm the model just wrote.
There is a compounding effect worth noting. Once the blocking-dialog lesson was recorded, the archive picker was built on the non-blocking callback pattern from the start — the backend opens the dialog with a callback, validates and persists inside it, and reports back over an event the React shell listens for. A hard-won runtime lesson, written down, becomes a convention rather than a bug re-lived.
Checking a model with code beats checking it with a model
MindMaze needs quiz questions, so a standalone crate generates them from article text through an LLM. Model output is treated as untrusted input: strict JSON parse, structural validation, then a verbatim grounding gate — the answer must appear literally in the source article — before anything is stored. Questions carry the revision id they were generated from, so they can be flagged stale when the article moves.
The roadmap had proposed a second LLM as a fact-checker. A deterministic string check does the job for a fraction of the cost, and the rule generalises: reach for a model to check a model only when the property genuinely isn't codeable.
The first real run against Mount Everest then demonstrated the limit of that rule. Two of the three questions were excellent, including one on the 1953 first ascent with genuinely plausible distractors. The third asked which of the options is a section heading. The answer was "Name."
That question is perfectly grounded. "Name" is a section heading and it appears verbatim in the source, so the gate accepted it. It also tests document structure rather than knowledge. Is the answer in the source? and is this a good question? are different properties, and only the first one is codeable. The fix belonged at the prompt layer — instruct the model to ask about substantive facts, never structure or formatting — which eliminated the meta-questions on the next run.
Quality control for LLM output is layered: a cheap deterministic gate for the codeable invariant, prompt engineering for the one that isn't. Confusing the two gets you a system that passes its own checks and produces garbage.
Modernisation re-opens every surface the original had sealed
A CD-era encyclopedia needed no TLS trust roots, no rate-limit backoff, no per-source provenance, no SSRF guard, no cross-platform custom-scheme origins, no data migration path. Each modern dependency reintroduces one.
The media cache restricts upstream hosts to *.wikimedia.org so it isn't an open proxy. User-selected ZIM files are validated for existence, extension and non-emptiness. The timeline parses presentation HTML defensively. None of this is polish. It is the actual cost of "keep the experience, replace the substrate," and it should be budgeted as first-class work rather than discovered late.
The last mile is where the sandbox stops being able to prove anything
Every earlier phase ended with a headlessly provable artifact. Packaging produced the first deliverable the agent's environment structurally could not generate — there is no cargo-tauri and no dpkg-deb in it, so the .deb and the AppImage can only be built by a human on a real host.
What kept that honest was decomposing the deliverable into layers each verifiable in isolation: config validity, the fetch script (run end to end against the real 20 MB tarball), runtime path resolution with its own tests. Only the final bundler invocation stayed unproven — and rather than paper over that with a CI workflow nobody had run, the repo says so.
That work also surfaced a boundary the configuration layer silently drops. Tauri copies bundled resources into the payload but does not guarantee the executable bit survives, which varies by target and archive format. A bundled binary can land non-executable and fail at spawn with a permission error that points nowhere useful. The fix lives in runtime code, not packaging config — the resolver chmods the binary before returning it.
A related lesson came from the memory pass. The roadmap prescribed virtual scrolling for an assumed giant rendered DOM. It was architecturally inapplicable: articles render inside <iframe srcDoc>, a separate browsing context whose internal DOM React cannot window, and only one is ever live. The actual unbounded growth was one layer up — a reader history that pinned every visited article's full HTML in the heap forever. When the roadmap and the architecture disagree, re-read the architecture. An inherited roadmap encodes assumptions that later decisions have already invalidated.
And since WebKitGTK has no performance.memory, the app now measures and displays its own retained HTML instead. The instrument ships with the product and works on the real target, rather than existing only in a Chromium devtools session that the shipping app never runs in.
The constraint was never throughput
Across every substantial feature, the agent's implement-and-verify cycle was fast and mostly correct. Each feature still ended with a short list of things only a person at the real window could confirm, and those checks are serialised.
That is the bottleneck in AI-assisted desktop revival. Not writing the code. The human-in-the-loop runtime check that no headless harness currently replaces — which is also, precisely, where all the bugs were.
Next: the same question with the tests made as strong as possible and the answer key sealed in an envelope, to find out whether passing every test means the model understood anything.
Code: https://github.com/singular-state/wikicarta. Not affiliated with Microsoft; Wikipedia content is CC-BY-SA and the app ships an attribution exporter for the sources you actually used.
Top comments (0)