DEV Community: tiezhu

Grounded iOS-to-Web Harness: Evidence-Driven App Migration with Behavioral and Visual Verification

tiezhu — Thu, 27 Aug 2026 12:38:43 +0000

Turning an iOS app into a React web app is no longer the hard part. Modern coding agents can generate a convincing first version quickly.

The hard part is answering three less glamorous questions:

Did we discover every important screen and state?
Did the generated app preserve the source behavior and data?
Is the result actually close to the native UI, or does it merely look plausible?

I built Grounded iOS-to-Web Harness to make those questions auditable.

🔗 GitHub: https://github.com/tiezhu0415/grounded-ios-to-web-harness

What it is

Grounded iOS-to-Web Harness is an experimental, lightweight grounding + verification layer for migrating iOS apps into complete, interactive, mobile-sized WebApps.

Claude Code remains the primary implementer. The Harness does not prescribe the React component tree, choose a state-management library, or replace the coding agent. Instead, it establishes source facts and verifies the result against evidence from the original app.

iOS source + Assets + code graph + runtime states
                         ↓
                  locked source facts
                         ↓
             per-screen implementation context
                         ↓
              agent builds the React WebApp
                         ↓
       coverage + truth + behavior + critical VRT
                         ↓
                 bounded repair, then human review

Why prompt-only migration is not enough

A prompt such as “convert this iOS app to React” can produce a good demo. But on longer tasks, an agent may miss screens, implement only one state, invent data or assets, choose navigation that differs from iOS, forget earlier facts, or optimize a screenshot while breaking real interaction.

This project treats the iOS source and Assets as the truth for content and behavior, while runtime screenshots provide evidence for what the result should look like.

The pipeline

1. Discover and reconcile source facts

Static source inspection, codebase-memory, and necessary iOS runtime exploration are combined into machine-readable facts for screens, UI states, actions, navigation outcomes, cross-screen flows, real data and asset origins, and source confidence.

Facts are locked before implementation. Important actions record preconditions, data effects, navigation effects, and visible feedback.

2. Give the agent small, relevant context

Instead of injecting the entire repository and every tool description into every prompt, the Harness creates a compact context for each screen. This reduces context drift without restricting how the agent writes React, CSS, routing, or state management.

3. Verify truth and coverage

The WebApp is checked for missing states, blank routes, broken images, fabricated data, screenshot-background cheating, desktop layouts escaping the mobile shell, and implementation evidence that does not match the locked facts.

Coverage is measured as Screen + State, not merely as a list of routes.

4. Verify behavior with Playwright

Playwright executes real clicks, input, state changes, and cross-page journeys. Expected behavior must refer back to locked action facts, preventing code and tests from agreeing on the same invented behavior.

5. Run critical-state visual regression

The Harness compares iOS and Web screenshots for the same state on the same canvas using:

Pixelmatch for changed-pixel ratio;
SSIM for structural similarity;
regional diff to expose severe local defects;
saved comparison and diff images for human review.

Visual metrics are used for triage, not marketed as a “fidelity percentage.” Thresholds remain experimental.

Reference evaluation

The repository contains a sanitized reference migration of an e-commerce app. The source iOS project is intentionally not included.

The checked-in evaluation records:

19 screens
28 states
19 navigation contracts
34 actions
3 cross-page Playwright journeys
31 screen/state behavior checks
10 critical visual states
41/41 Web checks passing
40/40 Harness regression tests passing

The final result was accepted after human visual review. Six experimental visual states were still routed to “review recommended,” and the repository reports that honestly rather than turning SSIM into a false product-quality claim.

Research inspiration, not paper reproduction

The design was informed by three recent research directions:

Coherence Debt: keeping dependent facts available and consistent during long agentic tasks;
WidgetGen: giving models selective text, color, layout, and asset evidence before UI generation;
WebDesignIter: preserving design knowledge and validating local edits across repository-scale iterations.

The repository does not claim to reproduce these research systems. It borrows their highest-ROI ideas and implements a thin orchestration layer around codebase-memory, Playwright, Pixelmatch, SSIM, Tesseract, and iOS runtime evidence.

What this project is not

not an iOS-to-React compiler;
not a promise of pixel-perfect equivalence;
not a general-purpose multi-agent workflow platform;
not a replacement for human acceptance;
not a benchmark result across many production apps—yet.

The current release is experimental. The next meaningful step is testing it against additional iOS projects and calibrating which facts, journeys, and visual states deliver the highest verification value.

If you work on app modernization, UI migration, coding-agent evaluation, or visual regression, I would love feedback.

⭐ Repository: https://github.com/tiezhu0415/grounded-ios-to-web-harness

Issues, experiments, and pull requests are welcome.

Single-page Claude writes beautifully. At 5 pages it drifts. Here's the harness I built.

tiezhu — Thu, 18 Jun 2026 05:17:55 +0000

TL;DR

I gave Claude / Codex a Figma file + a PRD and asked for 5-10 React pages of a working app. Single-page output is great. Multi-page output drifts in 4 specific ways. I spent ~3 months building a harness with 14 gates × auto-retry × handoff JSON to stop the drift. 10 demos, 54 screens, 4 unrelated business domains, build-green rate 100%.

Code: https://github.com/JiuwenDragon/harness-mini

The honest opening

Every "Figma to code with AI" demo on Twitter shows one screen. That's a real result — Claude vision is genuinely good at single-page UI. I verified this many times during my research: giving Claude a screenshot + a paragraph of PRD produces a 70-80 point page in 30 seconds.

The promise breaks at 5+ screens. Here are the 4 drift modes I measured.

Drift mode 1: Inconsistent copy

Screen 1	Screen 2	Screen 3
Username: Zhang San	Username: Li Si	Username: Test User

LLM doesn't carry a "world state" across page generations. Without explicit injection, it re-invents.

Drift mode 2: Dead-link routing

// Screen "transfer" generated:
<button onClick={() => router.push("/banking/home")}>  // ← /banking
// Screen "home" actually at:
app/bank/home/page.tsx                                  // ← /bank

Single-page review never catches this. Click-through breaks.

Drift mode 3: Shared state drift

A zustand store with 5 keys (user, balance, lastTx, recent[], selected). LLM forgets 2-3 keys on screen 4, makes new ones up. Same business concept, three different variable names.

Drift mode 4: "Claimed done" hallucination

> Codex: All 10 pages generated, ready to preview.
> me: npm run build
> 3 pages: red. 2 pages: empty <div /> stubs. 1 page: import path wrong.

This one is the most painful. Without an external check, "claimed done" ≠ done.

What the harness does (architecture)

Figma + PRD
    ↓ intake (fixture split)
    ↓ contract (frozen spec)
    ↓ generate (codex / claude / gemini)
    ↓ 14 gates (semantic / PRD / spec / UI hygiene / build / cross-canvas)
    ↓ visual review (human)
    ↓ web-preview (clickable)

Each gate is scoped to one constraint. Why? See Constraint Decay paper (arXiv 2605.06445): stuffing 10+ constraints into one prompt drops LLM performance by 30 percentage points.

The retry loop: when a gate fails, the gate's structured error report (not a vague "try again") is fed back to the LLM. Reflexion-style.

The handoff: each stage emits *_status.json so a new operator (or a new LLM session) can pick up without reading the conversation.

Why 14 gates and not 1 big one

Constraint Decay (arXiv 2605.06445) measured the drop directly.
Lost in the Middle (arXiv 2307.03172) shows the LLM ignores constraints buried in long prompts.
So I push one check per gate, max ~3 constraints per LLM round.

Generalization evidence (4-domain ablation)

Domain	Color	Screens	Build pass
Banking	Deep red	10	10/10
Fitness	Orange	3	3/3
Travel	Blue	3	3/3
Shoes	Black	3	3/3

Same 14 gates. Same Codex/Claude/Gemini providers swapped via contract. No per-domain prompt tuning.

Why not just use Builder.io / Locofy / v0 / Figma Make

Tool	Strength	Why it's not what I needed
Builder.io Visual Copilot	2M+ training data, Mitosis IR	SaaS, no PRD dim, no audit trail
Locofy LDM	Large Design Model	SaaS, design system requires strict Auto Layout
Figma Make	Highest fidelity (EPAM benchmark)	No public API, browser-only, $16/mo seat
v0 (Vercel)	Tight shadcn/Next.js	Figma link silently downgrades to screenshot (loses metadata)

These are all great for "single dev makes a pretty page." None give me multi-page consistency + PRD enforcement + audit log + on-prem + provider swap, which is the actual enterprise need.

What I'd do differently if starting over

Don't fight LLMs on single-page output. Claude with vision is already 80% there. Build the harness around what they're bad at: cross-page consistency and "claimed done."
Build a deterministic IR earlier. I attempted this (Builder.io's Mitosis-style) and abandoned at the first rendering bug. That was the wrong call — the IR is what Builder.io's whole architecture pivots on.
Get visual diff automated. I still rely on human visual review. Design2Code (arXiv 2403.03163) shows CLIP-score / CW-SSIM / TreeBLEU as auto metrics — should have wired one in.

Stuff that's open source

https://github.com/JiuwenDragon/harness-mini

14 gates as discrete Python scripts under scripts/
10 demo fixtures with full codex/claude/gemini traces
HE evolution log: every iteration with root cause + fix + prediction (87 entries)
Docs: design rationale + maturity map + workflow

MIT license (I should add the file — open to PR).

Papers cited

Constraint Decay (arXiv 2605.06445)
Lost in the Middle (arXiv 2307.03172)
Design2Code (arXiv 2403.03163)
Reflexion (arXiv 2303.11366)
Handoff Debt (arXiv 2606.02875)

Happy to answer questions in comments. The most useful feedback would be: "what other drift modes have you seen at >5 pages."