Most products can move to the cloud without downtime. To do it, you need to settle four things before the cutover weekend: how the database moves, when DNS changes, how long the old and new environments run together, and what exact signal makes you roll back. When a migration causes an outage, the cause is usually one of those four decisions being skipped. The cloud provider is rarely the problem.
If you are a CTO or technical founder, the new environment is probably already running. Staging works and the load test passed. Still, you are worried about Saturday night. What happens to the orders, sign-ups and payments that arrive during the switch?
You are right to worry.
Why does downtime happen when everything was tested?
The new servers are rarely what fails during a migration. The trouble starts when two systems each believe they are the source of truth. Testing proves the new environment can run. It does not prove the handover works.
Three gaps cause most of the damage. First, data written to the old database after your last copy never reaches the new one. Second, clients and resolvers keep sending traffic to the old IP address because they cached the DNS record hours ago. Third, background jobs, cron tasks and webhooks keep running in both places, so customers get two emails and two invoices.
All three are timing problems. More servers won't fix a timing problem. A plan will.
What does a messy cutover actually cost you?
The cost you see is a few hours of error pages. The bigger cost is writes that landed in the wrong database. Someone then has to reconcile them by hand, sometimes days later, after a customer reports a missing order.
The team pays too. A migration that ends in a 3am scramble makes everyone nervous about touching infrastructure, so the next necessary change gets pushed back by months. If you sell to businesses, a data-loss incident can also come up in your next security questionnaire.
Getting the cutover right always costs less than cleaning up after a bad one.
Do you need a zero-downtime plan at all?
Not always. If your product is an internal tool used in office hours, or your users can accept a scheduled maintenance window, a planned outage is simpler and safer. Announce a two-hour window, stop writes, take a final dump, restore it, switch DNS and test.
Zero-downtime migration adds moving parts. It is worth it if you have users across time zones, payment flows that cannot pause, or uptime promises in your contracts. If none of those apply to you, take the maintenance window and spend the time you save on testing.
Should you replicate the database or take a final dump?
A final dump is enough if you can export and restore the database inside the pause your users will accept. Use replication if the restore takes longer than that, or if you cannot pause writes at all.
A dump is easy to reason about: one snapshot, one restore, nothing drifts. The risk is how long it takes. Time the full export, transfer and restore on a production-sized copy, then add margin for rebuilding indexes.
With replication, the new database follows the old one continuously. On PostgreSQL that usually means logical replication, and on MySQL it means binlog replication. Managed services like AWS Database Migration Service and Google's Database Migration Service do the same job between providers. At cutover you stop writes briefly, wait for replication lag to reach zero, then point the app at the new primary.
Before you rely on replication, check these three things:
- Sequence values. PostgreSQL logical replication does not copy sequence positions. If you don't reset them, the first insert on the new side collides with an existing ID.
- Tables without primary keys. Many replication setups cannot carry updates and deletes for these tables. Find them weeks before cutover, not on the night.
- Schema changes. Freeze migrations while replication runs. A column added on the old side can quietly break replication.
When should you lower DNS TTL?
Lower it ahead of time, by at least the length of the current TTL, and ideally two or three days before cutover. If your record has a TTL of 86400 seconds, resolvers can keep the old answer for a full day. Lowering it on Saturday morning won't help on Saturday night.
Set it to 60 or 300 seconds. Run dig against a few public resolvers to confirm they are serving the new value. Raise it again once things settle.
Some clients will ignore the TTL anyway. Mobile apps with hardcoded hosts, corporate proxies and long-lived connections will keep reaching the old address. That is why you cannot switch off the old environment at cutover.
How long should old and new environments run side by side?
Keep the old environment running in a reduced role for at least several days after cutover. It catches late traffic and it is what you roll back to.
Turn the old app into a forwarder instead of a writer. Point its load balancer or reverse proxy at the new environment, so late requests still end up in the right database. Stop its cron jobs, queue workers and scheduled tasks at the moment you cut over, and write down every one of them beforehand. Update the webhook endpoints in Stripe, Twilio and similar services too, or they will keep calling the old host.
If you can, put a load balancer or CDN in front of both environments before the migration. Then the switch is a routing change you can reverse in seconds, and DNS barely matters.
What is your rollback trigger?
Write it down before you start. Use real numbers and name one person who makes the call. For example: "If the error rate stays above 2% for 10 minutes, or the checkout smoke test fails twice, we roll back. The CTO decides, with no debate."
You need this because at 2am, with the new system almost working, everyone will want to fix forward. Sometimes that is the right call. Often it turns a short incident into a long one.
Also decide your point of no return. Once real writes land in the new database, rolling back means moving those writes back to the old one. If you keep reverse replication running from new to old, rollback stays cheap. If you don't, write down how you would recover those records, and agree on a time after which you only fix forward.
The cutover order that holds up
- Days before: lower the TTL, freeze schema changes, start replication, and list every scheduled job and webhook.
- Day before: do a full rehearsal on a copy and time every step. Teams most often want to skip this step, and it is the one that catches surprises.
- Cutover: put the app in read-only mode or queue writes, wait for zero lag, reset sequences, point the app at the new database, then switch routing.
- First hour: run scripted smoke tests on login, your main transaction and one background job. Compare the error rate with your rollback trigger.
- Following days: keep the old environment as a forwarder and fallback. Take a final backup, then shut it down.
If you would like someone to check your cutover plan before the weekend, Geminate Solutions offers a free review of your migration plan and replies within 24 hours. You can read more about how we handle cloud and DevOps work.
Top comments (0)