James Coombs is a design engineer who built a migration engine for a 962-file Material UI to custom design system migration. 7,065 lines of TypeScript, more than half of it tests, 5-factor confidence scoring, and a state machine with gated transitions.
Once the CSS coexistence layer was in place (I covered that in "The CSS Problem Nobody Thinks Is Solvable"), both frameworks could run side-by-side. The next question: how do you actually migrate 962 files across 12 packages without drowning in manual work or shipping regressions?
The standard answer is codemods. Write a jscodeshift transform, run it across the repo, review the output. Codemods work for mechanical substitutions: rename an import, swap a prop name, update a function signature. They don't work when the mapping isn't 1:1. A Material UI <TextField variant="outlined"> doesn't map to a single design system component. It maps to different components depending on context: <Input> for plain text, <Textarea> for multiline, <Select> for dropdowns with select prop. The prop API differs. The composition pattern differs. The styling approach differs.
962 files. Some are mechanical (swap the import, done). Some require architectural judgment. Treating them identically wastes time on the easy ones and ships regressions on the hard ones.
Confidence scoring
Weighted scoring for migration prioritization is an established practice (AWS uses it for cloud migration, Cortex uses it for service compliance). What's less common is applying it at the per-file level for frontend component migration, where the factors that predict success are different from application-level migration.
Every file in the migration gets a confidence score before anyone touches it. Five factors, weighted:
| Factor | Weight | What it measures |
|---|---|---|
| Size | 20% | Lines of code, number of components. Smaller files are easier to verify. |
| Semantic complexity | 25% | How many MUI components, how deeply nested, how much conditional rendering. |
| Mapping coverage | 25% | What percentage of MUI components in this file have complete prop-level transforms in the mapping data. |
| Test coverage | 15% | Does the file have tests? How comprehensive? Files with tests catch regressions; files without them hide them. |
| Navigation complexity | 15% | Is this file a leaf component or a route-level page? Pages compose dozens of components; a regression cascades. |
Three tiers: high (>= 0.75), medium (>= 0.45), low (< 0.45). A high-confidence file (small, well-tested leaf component with complete mappings) can be auto-migrated with automated visual regression checking. A low-confidence file (large route-level page, partial mappings, no tests) routes to human review after the agent's attempt.
"Success" means the file reaches screenshot_approved without hitting failed: TypeScript compiles, tests pass, and the visual diff is below threshold. That's the outcome the score predicts.
The weights came from the first 20 migrations. Size mattered less than expected (small files with complex conditional rendering were harder than large files with repetitive layouts). Semantic complexity and mapping coverage were the strongest predictors of first-attempt success.
The scoring is deterministic: the LLM agent writes raw factor values to the manifest; a Node.js module computes the final score. The agent never sets its own confidence. This separation matters because LLMs are optimistic about their own output. The scoring module doesn't care what the agent thinks; it scores the file's structural properties.
The state machine
A boolean (migrated/not-migrated) isn't enough. Migration is a multi-step process, and each step has a quality gate.
The state machine tracks each file through 15 statuses with gated transitions. State is stored in a JSON manifest file. Node.js is the single writer; the LLM agent proposes transitions but never mutates state directly.
not_started → checked → plan_created → plan_validated → migrating → migrated
→ ts_verified → test_verified → screenshot_captured → screenshot_approved
→ pr_created → merged
With two terminal escape states: failed (reachable from any active state) and wont_migrate (reachable from not_started for files that should never be migrated).
Each transition has a named gate:
-
not_started to checked:
ds-migrate-check-passes(migration readiness assessment) -
checked to plan_created:
confidence-score-computed(all five factors assessed) -
plan_created to plan_validated:
validate-plan-exits-0(plan passes structural validation) -
migrated to ts_verified:
tsc-exits-0(TypeScript compiles clean) -
ts_verified to test_verified:
tests-pass-or-no-tests -
test_verified to screenshot_captured:
screenshots-captured-and-diffed(visual regression screenshots taken) -
screenshot_captured to screenshot_approved:
diff-below-threshold-and-high-confidence(automated approval for high-confidence files) or routes toawaiting_human_reviewfor manual approval
Files can move backward. Any active state can transition to failed with notes on what broke. The checked state can return to not_started via a file-changed-staleness gate (the source file was modified since it was last checked, invalidating the analysis). Stale states have configurable timeouts: 1 hour for migrating, 2 weeks for awaiting_human_review, 30 days for pr_created.
Why this matters: without gated transitions, teams mark files as "done" when the PR merges. But "PR merged" and "verified in production with no regressions" are different things. 15 states and 15 named gates make that gap explicit and enforceable, 12 of them on the forward path and 3 on the routes back (a file changed since it was checked, a failed file whose changes were reverted, an explicit override of a wont_migrate decision). The remaining 11 transitions are ungated, and they are all the ones that need no evidence to take: anything to failed, and the hand-off to human review.
Component mappings
12 mapping files. 100+ MUI-to-design-system transforms with prop-level translation.
A mapping isn't just "replace <Button> with <Button>." It's:
- Import path change (
@mui/material/Buttonto@company/design-system) - Prop renames (
variant="contained"tovariant="default") - Prop removals (MUI's
disableElevationhas no equivalent) - Prop additions (design system requires
sizewhere MUI inferred it) - Composition changes (MUI's
startIconprop becomes a child<Icon>component) - Conditional mappings (MUI's
TextFieldmaps to 3 different components based on props)
Each mapping includes the confidence impact. A component with a complete mapping (all props translated, all variants covered) contributes 100% to the mapping coverage factor. A component with a partial mapping (3 of 7 props translated) contributes proportionally. This feeds directly into the confidence score.
The mappings are JSON, not code. Adding a new component mapping is a data change, not a code change. The migration bot reads the mappings; the bot code doesn't change when new components are added.
The ratchet
The ratchet pattern (monotonically decreasing violation counts, enforced through CI) is well-established. Dusty Burwell described it in 2019. What's less common is applying it at the write layer for design system imports.
PreToolUse hooks (65 lines, 17 tests) fire on every file write. If an edit introduces an import from the legacy framework in a file that's been marked "complete," the edit is rejected before it lands. Not at CI. Not at PR review. At the moment the engineer types the import. The engineer sees an error message explaining why and pointing to the design system equivalent.
This is not documentation ("please use the new components"). This is enforcement at the earliest possible point. The count of legacy imports only goes down, never up.
The ratchet catches a specific failure mode: an engineer working on an unrelated feature in a migrated file reaches for the familiar MUI component because it's what they know. Without the ratchet, the migration regresses silently. With it, the regression is blocked at write time, before the PR is even created.
Result: zero legacy imports in any file marked "complete." Zero review comments needed for that class of violation.
The learning log
10 entries over four months. Each entry: date, source, what happened, what rule it produced.
Examples:
- A static hex color (
#6464f0) wasn't converted to a design token during migration. Added a check to the migration step that flags unconverted hex values. - A migration check was too broad: it added design system config to bundles that didn't use any migrated components. Refined the check to verify component usage before applying config.
- A portal-rendered component's test asserted a class on a node that could be null, so the assertion passed vacuously. Added a not-null guard as a rule and updated the test suite.
- Separately, nesting a Radix trigger through a component that did not forward refs silently dropped the ref. Added a rule; the failure is silent, which is why it needed one.
The log is append-only with mandatory fields. Rules flow from failures, not from predictions. Starting the log on day 1 would have been better; the first three months of failures had to be reconstructed from git history.
What didn't work
Hardcoded confidence weights, never recalibrated. The five weights were set by judgment before the first file was scored, and they are still the numbers in the config: 20, 25, 25, 15, 15. Nothing in the system closes the loop. The bot records the score it predicted and the state each file actually reached, so the data to test the weights is produced as a side effect of running it, and no one has ever gone back and fitted one against the other. A scoring system that cannot be shown to predict anything is a prioritization heuristic wearing the vocabulary of a model. It still beats migrating files in directory order, which is the honest claim, and it is a smaller claim than the machinery implies. Build the recalibration step into the first batch, or the weights stay guesses with a decimal point.
Generic checks without scoping. A check that asks "does this package depend on the design system?" catches packages that imported one design system utility for an unrelated reason. Three bundles got unnecessary config added before the check was refined to verify actual component usage. Automation checks need nested conditions, not just existence tests.
The numbers
| Metric | Value |
|---|---|
| Migration bot code | 7,065 lines of TypeScript, of which 3,802 are tests |
| Component mappings | 12 files, 69 components, 131 prop-level transforms |
| Structured data files | 49 (9,471 lines of JSON) |
| Scoring factors | 5 (size, semantic, mapping, test, navigation) |
| State machine | 15 statuses, 26 transitions, 15 named gates, JSON-stored |
| Enforcement hooks | 65 lines, 17 tests |
| Learning log entries | 10 entries, 11 codified rules |
If you're building a migration engine
Score before you migrate. Any scoring system (even a rough one) is better than migrating files in directory order. The scoring reorders 962 files so the first 50 are almost guaranteed to succeed, building confidence and catching mapping gaps early.
Gate your transitions. "PR merged" is not "migration complete." Define what "complete" means, build the states between "started" and "complete," and enforce the gates.
Build the ratchet in week 1. Every day without enforcement is a day legacy imports can re-enter the codebase through unrelated work.
Start the learning log immediately. The failures from the first 10 migrations will produce the rules that prevent failures in the next 100.
Top comments (0)