DEV Community

Cover image for The Hard Way: Lessons Learned from Real-World Data Migration Projects
Kushang Tailor
Kushang Tailor

Posted on

The Hard Way: Lessons Learned from Real-World Data Migration Projects

#wordpress #webdev #tutorial #database

The Hard Way: Lessons Learned from Real-World Data Migration Projects

How three years of ignoring data migration nearly derailed my dev career — and what I spent the next years mastering.

What Is Data Migration?

Data migration is the process of moving data from one system, format, location, or environment to another. It sounds straightforward — until it isn't.

At its core, data migration is about transfer + transformation + validation. You're not just copying files; you're ensuring that data arrives at its destination intact, usable, and consistent with the target system's structure and rules.

Types of Data Migration

1. Storage Migration
Moving data from one physical or cloud storage system to another — e.g., from on-premise servers to AWS S3 or Google Cloud Storage.

2. Database Migration
Transferring data between database engines (MySQL → PostgreSQL), versions, or schemas. This often involves restructuring tables, rewriting queries, and handling incompatible data types.

3. Application Migration
Moving data as part of migrating from one application to another — e.g., from a legacy CRM to Salesforce, or from a static HTML site to a CMS like WordPress.

4. Cloud Migration
Migrating workloads and data from on-premise infrastructure to cloud platforms, or between cloud providers.

5. Business Process Migration
Triggered by mergers, acquisitions, or system upgrades — involves consolidating or restructuring data from multiple sources into a unified system.

6. WordPress-Specific Migration (more on this shortly)
A category unto itself — involving database exports, media files, plugins, theme settings, serialized data, and URL structures.

Data Migration in WordPress

WordPress stores almost everything in a MySQL database — posts, pages, users, settings, metadata, plugin configurations — and media files separately on the server. This dual structure makes WordPress migrations uniquely nuanced.

A WordPress migration typically involves:

  • Database (wp_* tables) — posts, terms, options, users, meta
  • wp-content/uploads/ — all media files
  • wp-config.php — environment-specific configuration
  • Theme and plugin files — code that must be compatible with the target environment
  • Serialized data — PHP-serialized strings stored in the database that break if you do a naive find-and-replace on URLs

WordPress migrations are common in scenarios like:

  • Changing domains or hosting providers
  • Moving from single site to Multisite
  • Migrating to managed or enterprise-grade hosting (WP VIP, Kinsta, Pagely)
  • Rebuilding a legacy site in WordPress

The Problem: What Goes Wrong (And Why)

Data migration failures are more common than the industry likes to admit. Here's what I've seen cause the most damage:

1. Broken URLs and Serialized Data

WordPress stores serialized PHP arrays in the database. A simple SQL FIND & REPLACE on the old domain will corrupt those strings because they contain byte-length metadata. The result: broken theme options, widget configurations, and plugin settings that silently fail.

2. Missing or Orphaned Media Files

The database records can transfer perfectly while media files are left behind on the old server — broken images everywhere, no obvious error.

3. Character Encoding Issues

Moving between servers with different MySQL collations (e.g., utf8 vs utf8mb4) can corrupt special characters, emojis, and multilingual content.

4. Plugin and Theme Incompatibilities

A plugin that worked on PHP 7.4 + MySQL 5.7 may fail silently or fatally on PHP 8.1 + MySQL 8.0 on the new host.

5. Skipping the Staging Phase

Going straight to production without a staging test is the single most common and most costly mistake. There's no undo button on a live site.

6. No Pre/Post Validation Checklist

Without documented baselines — post count, user count, page structure, form behavior — you won't know what broke until a user reports it.

7. SEO and Permalink Damage

URL structure changes without proper redirects tank search rankings. .htaccess rules and WordPress permalink settings must be verified post-migration.

8. Caching and CDN Serving Stale Data

After a migration, cached content from the old server can make the new site appear broken even when it isn't.

My Story: The Career Cost of Skipping This Skill

Let me be honest with you.

I spent my first three years in web development building things — WordPress themes, custom plugins, client websites. I was comfortable with HTML, CSS, PHP, and JavaScript. I thought I knew WordPress well.

What I didn't realize was that I had never touched data migration seriously. Not once.

When projects came up that required moving a site to a new server, changing domains, or restructuring content into a Multisite setup, I either avoided them or quietly handed them off. I told myself it wasn't "real development work."

That belief cost me. I walked away from projects I should have taken. I didn't get past certain interview stages at companies where migration experience was table stakes. Good opportunities at agencies working with enterprise WordPress clients — gone, because I couldn't answer questions about wp search-replace, serialized data, or staging workflows.

It wasn't a sudden failure. It was a slow leak — projects I compromised on, roles I wasn't confident enough to pursue, skills gaps that compounded over time.

Eventually, I decided to stop avoiding it.

The Safety Steps: How to Prevent Migration Disasters

Before touching any migration, build these habits:

✅ 1. Full Backup — Verified

Back up the database AND all files. Test that the backup actually restores. A backup you haven't verified is not a backup.

✅ 2. Document the Current State

Record baseline metrics: number of posts, pages, users, media files, active plugins, PHP version, MySQL version, WordPress version. Screenshot key pages. Run a broken link audit. You need a reference point.

✅ 3. Use a Staging Environment — Always

Never migrate directly to production. Set up a staging server (or use a local environment like LocalWP) that mirrors the production setup. Test everything there first.

✅ 4. Use WP-CLI for Reliable Search-Replace

Use wp search-replace with the --precise flag rather than direct SQL queries. This handles serialized data safely.

✅ 5. Verify DNS Propagation Separately

Don't conflate "migration complete" with "DNS updated." Test via hosts file override or a staging URL before pointing the domain.

✅ 6. Test Forms, Checkout Flows, and Dynamic Features

Static content is easy to verify. Test everything interactive: contact forms, WooCommerce checkout, membership logins, API integrations.

✅ 7. Have a Rollback Plan

Know exactly how you'll revert if something goes wrong. Document the steps. Set a rollback deadline during the migration window.

The Learning Curve: How Data Migration Actually Works

After committing to learning this properly, I built a workflow I still use today. Here's the foundation:

The Staging-First Philosophy

Every migration lives and dies by the staging environment. The workflow is always:

Development / Local → Staging → Production
Enter fullscreen mode Exit fullscreen mode

Never skip staging. Never.

Before Migration: Pre-Flight Checklist

  1. Audit the source site — document all plugins, theme, PHP/MySQL versions, custom tables, cron jobs
  2. Backup everything — full database dump + all files, stored off-server
  3. Verify the target environment — PHP version, MySQL version, disk space, server software (Apache/Nginx), SSL
  4. Set up staging — mirror the production environment as closely as possible
  5. Notify stakeholders — plan the migration window, communicate downtime if needed
  6. Freeze content on the source site (put it in read-only or maintenance mode) before the final migration pass

The Migration Process (Step by Step)

Step 1 — Export the database

mysqldump -u DB_USER -p DB_NAME > backup.sql
# Or with WP-CLI:
wp db export backup.sql
Enter fullscreen mode Exit fullscreen mode

Step 2 — Transfer files

rsync -avz --progress /path/to/wp-content/ user@newserver:/path/to/wp-content/
Enter fullscreen mode Exit fullscreen mode

Step 3 — Import database on target

wp db import backup.sql
Enter fullscreen mode Exit fullscreen mode

Step 4 — Update wp-config.php with new DB credentials, table prefix, and environment constants.

Step 5 — Run search-replace for the new domain

wp search-replace 'https://oldsite.com' 'https://newsite.com' --precise --all-tables
Enter fullscreen mode Exit fullscreen mode

Step 6 — Flush rewrites and caches

wp rewrite flush
wp cache flush
Enter fullscreen mode Exit fullscreen mode

Step 7 — Validate on staging — check all pages, forms, media, admin panel, user logins.

Step 8 — Go live — point DNS, verify SSL, run final flush.

After Migration: Post-Flight Checklist

  • [ ] All pages and posts load correctly
  • [ ] Images and media files display properly
  • [ ] Forms submit and send notifications
  • [ ] User accounts and roles intact
  • [ ] Admin panel fully functional
  • [ ] Plugins activated and configured correctly
  • [ ] SSL certificate active and no mixed content warnings
  • [ ] Redirects in place for any changed URLs
  • [ ] SEO plugin settings preserved (sitemaps, meta, robots.txt)
  • [ ] Google Analytics / Tag Manager firing correctly
  • [ ] Page speed acceptable (run Lighthouse)
  • [ ] Broken link scan complete

Monitor After Migration — For Weeks

Don't close the loop after go-live. Monitor actively:

  • Week 1: Check server logs daily. Watch for 404s, PHP errors, failed cron jobs.
  • Week 2–3: Monitor Google Search Console for crawl errors or ranking drops.
  • Week 4: Review uptime reports, form submissions, and any user-reported issues.
  • Set up uptime monitoring (UptimeRobot, Better Uptime) before you go live — not after.

What I Learned: Real Migration Scenarios

1. Static HTML Site → WordPress

This is a foundational migration type. The source has no database — everything lives in HTML files.

The process:

  • Inventory all HTML pages and map them to WordPress post types (pages, posts, custom types)
  • Extract content and import using WP-CLI or a custom script / WXR importer
  • Rewrite internal links to match WordPress permalink structure
  • Migrate images to wp-content/uploads/ and update references in content
  • Set up 301 redirects from old HTML URLs (e.g., /about.html) to new WordPress URLs (/about/)
  • Validate all redirects, check for orphaned pages

Key tool: WordPress Importer plugin, WP All Import for structured data, custom PHP scripts for bulk content.

2. WordPress Single Site → WordPress Multisite (Subdirectory)

This is one of the more technically involved migrations. You're converting a standalone WordPress install into a network node.

Steps:

  • Enable Multisite in wp-config.php and wp-admin/network/setup (choose subdirectory)
  • Run the network setup, update wp-config.php and .htaccess with network rules
  • Create the subsite at the target subdirectory path
  • Export content from the original site (WXR file or direct DB migration)
  • Import into the new subsite — users, posts, media, settings
  • Reassign user roles within the network context (Network Admin vs site-level roles)
  • Verify media uploads path (/wp-content/uploads/sites/[ID]/) is correctly mapped
  • Run wp search-replace scoped to the subsite's tables

Watch out for: plugins that aren't Multisite-compatible, user role conflicts, and network-activated vs site-activated plugin behavior differences.

3. WordPress to WordPress — New Domain and Hosting (WP-CLI Workflow)

This is the most common migration scenario, and WP-CLI makes it reliable.

# On the source server — export DB
wp db export old-site-backup.sql --add-drop-table

# Transfer files
rsync -avz public_html/ user@newhost:/home/user/public_html/

# On the new server — import DB
wp db import old-site-backup.sql

# Update site URL and home
wp option update siteurl 'https://newdomain.com'
wp option update home 'https://newdomain.com'

# Search and replace all instances of old domain
wp search-replace 'https://olddomain.com' 'https://newdomain.com' \
  --precise \
  --all-tables \
  --report-changed-only

# Flush permalinks and cache
wp rewrite flush --hard
wp cache flush

# Verify user count and post count match source
wp user list --format=count
wp post list --post_status=publish --format=count
Enter fullscreen mode Exit fullscreen mode

Post-migration, set up 301 redirects on the old domain pointing to the new one. Keep the old domain active (don't let it expire) for at least 6–12 months to preserve link equity.

4. WordPress → WordPress VIP

WordPress VIP is an enterprise-grade managed platform with a strict code review process and a read-only production filesystem. Migrating to VIP is as much a code audit as it is a data migration.

Key differences on VIP:

  • No direct filesystem writes in production — uploaded files go to a distributed filesystem (VIP Files)
  • All code must pass VIP's automated and manual code review before deployment
  • No direct database access — use VIP's tooling
  • Plugin approval required — not all plugins are VIP-compatible
  • Local development uses vip dev-env (Docker-based local environment)

Migration steps:

  • Audit all custom plugins and themes against VIP coding standards
  • Replace any code that writes to the filesystem directly
  • Use VIP's media migration tools to transfer uploads
  • Import content via VIP's data migration pipeline (coordinate with VIP support)
  • Run full QA on VIP's staging environment before production launch
  • Monitor VIP's log dashboard post-launch

This migration typically requires coordination with the VIP team — it's not a solo operation.

5. Enterprise WordPress Site → WordPress Multisite (Subdomain)

This scenario often emerges from brand consolidation — multiple standalone WordPress sites being brought under a single Multisite network, each on its own subdomain (brand.example.com).

Challenges at enterprise scale:

  • High post/page volume — imports must be batched to avoid timeouts
  • Multiple user bases with different roles that need consolidation
  • Multiple third-party integrations (CRMs, DAMs, marketing automation) tied to the old site URLs
  • SEO preservation across all migrated properties
  • Editorial workflow tools (editorial calendars, approval flows) that must be reconfigured for the network

Process:

  • Set up Multisite with subdomain structure
  • Map brand.example.com subdomains using WordPress's domain mapping or a plugin like Mercator
  • Migrate each site individually and sequentially — never in parallel
  • Consolidate users carefully, reconciling duplicate accounts across sites
  • Update all third-party integrations one by one, confirming webhooks, API endpoints, and OAuth tokens
  • Perform site-by-site QA with dedicated checklists per subdomain
  • Coordinate DNS changes in batches with rollback windows per site

At this scale, automation matters — write scripts, use wp eval-file, lean on WP-CLI's --url flag to target specific subsites in the network.

Closing Thoughts

Data migration is not glamorous. It doesn't have a flashy demo. Nobody tweets about a successful database import.

But it is one of the most consequential skills a WordPress developer can have. Sites break in migration. Data gets lost. SEO disappears overnight. Businesses lose revenue.

Learning it properly — staging first, validating obsessively, monitoring after go-live — is what separates developers who can be trusted with production systems from those who can't.

I learned it the hard way. You don't have to.

If you've been through a migration disaster (or a migration victory), share it in the comments. The real lessons are always in the edge cases.

Top comments (0)