DEV Community: Noah Grinstein

AWS Outage One Year Later: Lessons From Every Cloud Failure

Noah Grinstein — Tue, 13 Oct 2026 21:06:30 +0000

TL;DR

Every major cloud provider has had a serious outage since mid-2025, each from a different cause, so any single provider is a single point of failure.

On October 20, 2025, a DNS race condition in DynamoDB's automation took the DynamoDB endpoint in AWS us-east-1 offline. The event ran about 15 hours and spread to EC2 instance launches and Network Load Balancers.
Google Cloud (June 2025, global), Azure Front Door (October 2025, global), Oracle Cloud US East (Ashburn) (March 2026), and an automated Google Cloud action that suspended a customer's production account (May 2026) each failed in a different layer.
Failures hit three blast radii: a zone, a region, or a whole provider. Multi-region on one provider covers the first two. It doesn't cover a provider's global control plane, edge network, or account layer.
Put a second, independent, already-running path under every layer you can't lose, keep failover off the failing provider, and drill the switch.
Control Plane combines any cloud, neocloud, and bare metal into one virtual cloud and runs workloads active-active across providers, with a 99.999% availability SLA for workloads running at least two replicas in at least two locations. During the October 20 outage, no Control Plane customer experienced downtime, according to Control Plane internal incident telemetry.

What happened in the AWS us-east-1 outage on October 20, 2025?

A latent race condition in DynamoDB's DNS management system left the DynamoDB regional endpoint in us-east-1 with an empty DNS record, and everything that resolves that name failed. According to the AWS post-event summary, the event ran from 11:48 PM PDT on October 19 to 2:20 PM PDT on October 20 (06:48 to 21:20 UTC on October 20), about 15 hours. The disruption affected dozens of AWS services in us-east-1, and AWS's Health Dashboard event history lists the full set.

AWS's summary splits the impact into three periods:

DynamoDB became unreachable (11:48 PM to 2:40 AM PDT). Two automation components, a DNS Planner and DNS Enactors, manage DynamoDB's DNS. One unusually delayed Enactor applied an outdated plan, and another Enactor's cleanup step then deleted that plan, leaving an empty record for dynamodb.us-east-1.amazonaws.com. Customer traffic and internal AWS services that depend on DynamoDB couldn't connect.
New EC2 instances failed to launch (2:25 AM to 10:36 AM PDT). After DNS was restored, internal subsystems that depend on DynamoDB stayed impaired, and AWS throttled some operations while it recovered. Some newly launched instances had connectivity problems until 1:50 PM PDT.
Network Load Balancers saw connection errors (5:30 AM to 2:09 PM PDT). Health checks began failing because new instances came into service before their network state had propagated, and automatic health check failover then removed capacity from service. AWS engineers disabled automatic NLB health check failover at 9:36 AM PDT.

AWS said it disabled the DynamoDB DNS automation worldwide until it fixed the race condition and added protections. Recovery that needed new instances was blocked for hours: AWS says Amazon Redshift's host-replacement workflows stalled while EC2 launches were impaired, leaving some clusters unavailable. AWS also notes that customers with DynamoDB global tables could still reach replicas in other regions, though replication to and from the us-east-1 replica lagged. Regional redundancy helped where it was real and tested.

What other major cloud outages happened in 2025 and 2026?

Four more major incidents hit a different provider or a different failure layer. The table lists major incidents from June 2025 to May 2026; it is a selection, and smaller incidents in the same period are not included.

Date	Provider	What failed	Blast radius	Duration	Source
June 12, 2025	Google Cloud	An invalid quota policy with blank fields replicated globally and crashed Service Control, the API management layer	Global, all regions	About 3 hours (most regions began recovering within 40 minutes; us-central1 took up to 2 hours 40 minutes)	Google Cloud incident report
October 19 to 20, 2025	AWS	A DynamoDB DNS race condition in us-east-1, cascading to EC2 launches and Network Load Balancers	One region, many dependent services	About 15 hours	AWS post-event summary
October 29 to 30, 2025	Microsoft Azure	Incompatible customer configuration metadata crashed the Azure Front Door data plane at edge sites	Global edge network	About 8.5 hours (15:41 to 00:05 UTC)	Azure Front Door post-incident review
March 3 to 4, 2026	Oracle Cloud Infrastructure	A network infrastructure issue affecting multiple OCI services in US East (Ashburn); Oracle has not published a root cause	One region	About 20 hours (13:24 UTC March 3 to 09:18 UTC March 4)	Oracle incident record
May 19 to 20, 2026	Google Cloud (account level)	An automated Google Cloud action that Railway reports incorrectly suspended its production account	One customer's whole production account, cascading to its workloads on AWS and its own hardware	About 8 hours	Railway incident report

Google Cloud (June 2025). Google's report says the policy change replicated globally within seconds, and the failing code path was never exercised during its region-by-region rollout. Google's Cloud Service Health status page was down too, delaying its first incident post by about an hour, and some customers' monitoring running on Google Cloud also failed.
Azure Front Door (October 2025). Microsoft's review says this incident was not directly related to an earlier Front Door incident on October 9, although both were broadly related to configuration propagation risk.
Oracle US East (March 2026). Oracle describes a network infrastructure issue that caused connection timeouts, errors, and increased latency across multiple OCI services. The Register reported that it disrupted parts of TikTok's U.S. service, which TikTok's U.S. joint venture acknowledged (The Register). Oracle logged a second Ashburn networking incident that began 52 minutes after the first was resolved and ran until 04:40 UTC on March 5.
Account suspension (May 2026). Railway's report says the automated action extended to many Google Cloud accounts. Its edge proxies cached routing tables from a control plane hosted in Google Cloud, so when the cache expired, workloads on AWS and on its own hardware became unreachable even though they were still running. Railway takes responsibility for that architecture. Redundant workloads can still share one dependency that decides whether anyone can reach them.

Large systems have shared layers, and shared layers fail, whichever provider runs them.

What is a blast radius in a cloud outage?

A blast radius is the set of workloads that fail together when one thing breaks. Cloud failures fall into three scales, and each needs a different kind of redundancy.

Blast radius	Typical cause	What it takes out	Redundancy that covers it	Recent example
Zone	A facility, power, cooling, or network fault in one availability zone	Workloads placed only in that zone	Replicas spread across zones in a region	None among the five above; multi-zone designs usually absorb zone faults
Region	A regional service, a regional dependency, or regional networking	Everything in that region, plus anything that depends on its endpoints	Active capacity in more than one region, with data replicated	AWS us-east-1 (October 2025), Oracle US East (March 2026)
Provider	A global control plane, global edge network, global configuration push, or account-level action	Every region of that provider, or your whole account	A second, independent provider, including the control path	Google Cloud Service Control (June 2025), Azure Front Door (October 2025), the account suspension (May 2026)

Why does multi-region fail during cloud outages?

Multi-region covers regional failures. It doesn't cover failures in layers every region shares. Three of the five incidents above weren't regional. Google's Service Control policy data replicated to every region within seconds. Azure Front Door's crash spread to all edge sites worldwide. The account suspension took down an entire account, which spans every region, and then spread to other clouds through a routing dependency hosted in Google Cloud.

Multi-region plans also fail for quieter reasons:

The failover step needs a control plane call (launch an instance, change a route, update a record) in the region that is failing. On October 20, new EC2 launches in us-east-1 failed from 2:25 AM to 10:36 AM PDT.
The failover target shares a dependency with the primary, such as a global service, an identity system, or a DNS zone.
The monitoring that tells you to fail over runs on the provider that is down. Google's June 2025 report says this happened to some customers.
The plan was written but never exercised under load.

AI workloads add their own dependencies: model APIs, GPU capacity from neoclouds, and agents that fail a customer if they die mid-task. Each is another layer that needs a second path. For a regional failure like October 20, tested multi-region capacity is a real defense, but it covers only the regional layer.

How do you keep services up during a cloud outage?

Post-incident reports don't say who stayed up, but they show which preparations would have mattered. Build them before the incident:

Keep capacity already running. When instance launches fail, a plan to scale up on demand depends on a control plane that may be struggling. Capacity already serving in another location doesn't.
Keep the failover path off the failing layer. Traffic shifting, health checks, and routing decisions should run somewhere independent of the provider or region in trouble.
Monitor from outside the provider. Google's report shows what happens when the monitoring stack and the workload share a failure domain.
Map hidden dependencies. The account-suspension report shows workloads in three environments and a routing control plane in only one of them.
Drill the switch. Teams that rehearse find the stale credential, the missing quota, or the unreplicated dataset on a Tuesday afternoon instead of during an outage.
Match redundancy to the cost of downtime. Rank services by business impact and put the strongest design under the ones that cost the most when they stop.

Control Plane builds the first two into the platform: capacity running in every location, and routing that sits above any single provider. For a deeper look at targets and trade-offs, see our guides to high availability, RTO and RPO targets, and what single-region, multi-region, and multi-cloud designs can deliver.

How does Control Plane keep you up when a provider fails?

Control Plane combines any cloud, neocloud, and bare metal into one virtual cloud, so you stay up when others are down. You choose the locations for a Global Virtual Cloud (GVC): AWS, Google Cloud, and Azure regions, Kubernetes clusters you already run, and your own servers on premises. Every workload runs active-active in all of them, with replicas already serving in each location. A built-in global endpoint sends each request to the nearest healthy location. When a zone, region, or provider fails, traffic moves to locations that are already running, and autoscaling adds replicas there as load arrives. Nothing has to boot, nobody runs a runbook, and failover doesn't depend on the failed provider's control plane.

The rest of the platform serves the same outcome. Universal Cloud Identity gives workloads short-lived, least-privilege access to 600+ AWS, Google Cloud, and Azure services, with no long-lived cloud keys in your code, so nothing needs re-keying when traffic moves. Each location keeps serving the last healthy version until a new release passes its health checks, so a broken update can't take the workload down (docs). Capacity AI right-sizes CPU and memory to actual usage in every location. None of it needs a platform team to build or operate.

On October 20, 2025, 20% of Control Plane customers had us-east-1 in their Global Virtual Clouds. Their workloads were active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane internal incident telemetry (Beyond Backups). For the Google Cloud, Azure, Oracle, and account-level failures above, a GVC that spans more than one provider is designed to route around a failure confined to one.

Control Plane offers a 99.999% availability SLA for workloads running at least two replicas in at least two locations, and those locations can be on different providers. No migration is required: existing Kubernetes clusters and your own servers join as locations, so you add failure domains from what you already run.

Next step: watch us recreate the us-east-1 outage live on October 20 at 11 AM PT. Register for the webinar, or talk to the Control Plane team.

Cloud outage resilience checklist

Use this to find the layers you can't recover without. Answer each row for every service that matters to revenue.

Check	Question to answer
Single points of failure	Have we listed every provider, region, account, DNS zone, identity system, and control plane each service depends on?
Zone spread	Are replicas spread across at least two zones?
Region spread	Can the service run in a second region without a human launching new capacity?
Provider spread	Can the service run on a second provider if a global layer or account fails?
Capacity already running	Is the failover location already running enough replicas to carry real load?
Data	Is data replicated to the failover location, with an RPO we have actually measured?
Failover path	Does traffic shifting work if the primary provider's console, API, and DNS are unavailable?
Monitoring	Does alerting run outside the provider we are monitoring?
Account risk	Do we have a documented path if the provider suspends or restricts our account?
Credentials	Can the failover location authenticate to everything it needs without a manual step?
Drills	Have we run a failover in the last quarter and written down what broke?
Communication	Does the team know who decides to fail over, and how customers are told?
SLA reality	Do we know that provider SLA credits are service credits, not compensation for lost revenue?

Frequently asked questions

What caused the AWS us-east-1 outage in October 2025?

A latent race condition in the DNS management system for Amazon DynamoDB left the regional endpoint in us-east-1 with an empty DNS record, so services that resolve that endpoint couldn't connect. Recovery then continued through EC2 instance launch failures and Network Load Balancer connection errors.

How long did the October 2025 AWS outage last?

AWS's summary gives the event window as 11:48 PM PDT on October 19 to 2:20 PM PDT on October 20, about 15 hours, with three distinct periods of impact.

Which major cloud outages happened in 2025 and 2026?

This post covers Google Cloud's global Service Control outage (June 12, 2025), the AWS us-east-1 outage (October 19 to 20, 2025), the Azure Front Door global outage (October 29 to 30, 2025), the Oracle US East outage (March 3 to 4, 2026), and an automated Google Cloud suspension of a customer's production account (May 19 to 20, 2026). Other incidents happened in the period, so treat it as a selection.

How do you survive a cloud provider outage?

Identify what each service depends on, then give each dependency a second independent path: spread across zones, run active capacity in more than one region, and for critical services run on a second provider. Keep failover and monitoring off the failing provider, keep capacity already running, and drill the switch.

Is multi-region enough to survive an outage?

Multi-region protects against regional failures, such as the AWS us-east-1 and Oracle US East incidents, if failover works without the failing region. It doesn't protect against a failure in a provider's global control plane, edge network, or your account, as in the Google Cloud, Azure Front Door, and account-suspension incidents.

What is the difference between multi-cloud and multi-region?

Multi-region means running in more than one region of the same provider. Multi-cloud means running across more than one provider. Multi-region covers regional failures, and multi-cloud also covers provider-level failures, at the price of more work in networking, identity, and data replication unless a platform such as Control Plane handles them for you.

Is multi-cloud worth it?

It depends on what an hour of downtime costs you. Provider SLA credits are service credits, not compensation for lost revenue. For revenue-critical services, a second provider removes a failure class that regions can't. Control Plane makes a second provider another location in the same GVC, with capacity scaled to demand in each location. Rank services by business impact and give the most critical ones cross-provider redundancy first.

What happened to Control Plane customers during the AWS outage?

During the October 2025 AWS outage, 20% of Control Plane customers had us-east-1 in their Global Virtual Clouds. Their workloads were active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane internal incident telemetry. Control Plane offers a 99.999% availability SLA for workloads running at least two replicas in at least two locations.

Sources

Originally published on the Control Plane blog.

How to Guarantee a Website or Service Never Goes Down (And What You Can Actually Promise)

Noah Grinstein — Mon, 12 Oct 2026 18:11:13 +0000

TL;DR

No one can guarantee that a website or service never goes down. What you can promise is a measured availability target, and with a multi-location, active-active design you can reach 99.999% (five nines), about 5 minutes 15 seconds of downtime a year. That takes redundancy at every layer, automatic health-based failover across regions and ideally providers, safe deployments, failure testing, and outside-in monitoring. Control Plane is built for that tier: it runs each workload active-active across regions and clouds, routes every request to the nearest healthy location, and offers a 99.999% availability SLA for workloads running at least two replicas in at least two locations. When AWS us-east-1 failed on October 20, 2025, no Control Plane customer experienced downtime, according to Control Plane internal incident telemetry.

What “never goes down” really means: the nines

Availability is the percentage of time a service works correctly over a period. Each additional nine cuts allowed downtime by a factor of ten.

Availability	Downtime per month	Downtime per year	Common design
99%	about 7 h 18 min	about 3.65 days	Single server, manual recovery
99.9% (three nines)	about 43 min 48 s	about 8 h 46 min	Single zone, basic redundancy
99.95%	about 21 min 54 s	about 4 h 23 min	Multiple instances, managed services
99.99% (four nines)	about 4 min 23 s	about 52 min 34 s	Multiple zones with automatic failover
99.999% (five nines)	about 26 s	about 5 min 15 s	Multi-region active-active, automated failover

Downtime figures use a 365-day year and a 730-hour month. The design column reflects the conditions cloud providers attach to their own SLAs: a single instance is covered at 99.5% to 99.9%, and 99.99% requires instances across multiple zones. Higher targets take more work and cost, as Microsoft’s reliability guidance notes, and availability is measured for the whole service as users experience it, not for one server. Match the target to the business cost of downtime. In Uptime Institute’s 2025 annual survey, reported in its 2026 Annual Outage Analysis, 57% of respondents said their most recent major outage cost more than $100,000, and one in five reported more than $1 million (Uptime Institute). A back-office tool and a checkout flow need different numbers. For a deeper comparison of single-region, multi-region, and multi-cloud designs, see Five Nines Uptime Architecture.

Why a cloud provider SLA is not a guarantee

A cloud SLA is a credit policy, not an uptime promise. If the provider misses its target, you can claim a partial credit against future bills for that service. AWS, Microsoft, and Google each state that these credits are your sole remedy, and Microsoft says they will not compensate for lost revenue, operational costs, or other losses (Microsoft SLA). The details matter:

Credits are small. Amazon EC2 credits 10% of that month’s EC2 bill for the affected region when region uptime falls below 99.99%, and 100% only below 95%, about 36 hours of downtime in a month (EC2 SLA).
Each service has its own target. AWS and Google Cloud publish a separate SLA per service. Microsoft bundles Azure’s into one document, but each service still has its own target and credit table. Your compute, database, load balancer, and DNS each carry different commitments.
Higher tiers require redundant architecture. A single EC2 instance is covered at 99.5%, about 3 hours 39 minutes of downtime a month, and the 99.99% commitment applies only to instances across two or more Availability Zones. A single Azure VM gets 99.9% only with a Premium SSD OS disk and Premium SSD, Premium SSD v2, or Ultra data disks, and one Standard HDD disk drops it to 95%. Google Compute Engine offers 99.99% for instances in multiple zones in most regions and 99.9% for a single instance of most machine families.
Exclusions vary by provider. Typical ones are problems caused by your own configuration, preview features, quota limits on Google Cloud and Azure, scheduled maintenance on Azure, and factors outside the provider’s control.
Claims need your own evidence. AWS must receive an EC2 claim by the end of the second billing cycle after the incident, with your request logs. Azure and Google Cloud require claims within 60 days, and on Google Cloud you forfeit the credit without log files.

A few services, such as Amazon Route 53 and Azure DNS, carry a 100% SLA, but the remedy is still a partial credit. Read the SLA for every service on your critical path, then design for the number you need.

How to calculate composite availability

When services sit in series in the request path, their availabilities multiply:

Effective availability = A(compute) x A(database) x A(storage) x ...

Three services at 99.99% each give 0.9999 x 0.9999 x 0.9999 = 99.97%, about 2 hours 38 minutes of allowed downtime a year instead of 52 minutes. Add a CDN, an auth provider, a queue, and a payment gateway, and the ceiling drops further.

Redundancy works in the other direction. For components in parallel, you fail only if every copy fails, so two independent 99.9% instances give 1 – (0.001 x 0.001) = 99.9999%. The catch is the word independent. If both copies share a region, a deployment pipeline, a configuration file, or a DNS provider, one fault takes out both. Real redundancy means separate failure domains.

How to keep a website from going down: a 10-step playbook

1. Set availability targets by workload

Not every service needs five nines. Rank flows such as browse, checkout, login, and reporting by business impact, and set a target for each. Over-engineering a low-priority flow spends budget your critical flow needs.

2. Map your risks

List the failures that can hit you, from most to least likely: transient network errors, instance restarts, hardware failure, datacenter outage, and region outage. Add the non-infrastructure risks: bad deployments, software bugs, expired certificates, traffic spikes, denial-of-service attacks, data corruption, and human error. Common risks belong in your high availability design and rare, catastrophic ones in your disaster recovery plan. A region outage is a disaster for a single-region app and an ordinary event for an active-active multi-region app.

3. Remove every single point of failure

Run multiple instances behind a load balancer, spread them across zones, and for critical workloads across regions. Replicate data. Check the less obvious single points too: one DNS provider, one CDN, one certificate authority, one CI/CD system, and one person who knows how the system works.

On Control Plane: a workload runs in every location of its Global Virtual Cloud at once. A new workload starts with one replica per location, so raise minScale to 2 for production. That removes the per-location single point of failure, and with two or more locations it meets the SLA condition (docs).

4. Automate failover with real health checks

Manual recovery takes minutes to hours, which is too slow for four or five nines. Use health-based routing so traffic leaves an unhealthy instance, zone, or region without paging anyone. Health checks should test real behavior, such as whether the app can read from its database, not only whether a port responds.

On Control Plane: every workload gets a global HTTPS endpoint that sends each request to the nearest healthy location. Readiness probes decide when a replica takes traffic, and a location that fails them stops receiving requests. When you want a dedicated standby, you can group locations into priority tiers for primary and failover routing (docs).

5. Plan for provider-level failure

Zones and regions protect you from datacenter and regional problems. They do not protect you from a failure in a provider’s control plane, an account or billing lockout, or a global configuration error. If the cost of downtime justifies it, run across more than one cloud provider with automated failover between them.

On Control Plane: one Global Virtual Cloud can include locations on AWS, GCP, and Azure, plus Kubernetes clusters you already run and your own servers through Managed Kubernetes. The same workload runs across all of them with one identity model and mutual TLS between services, so a provider-wide failure becomes a location failure.

6. Deploy safely

Changes cause most outages. Google’s SRE team reports that roughly 70% of outages are due to changes in a live system (Site Reliability Engineering). Use rolling, canary, or blue-green releases. Send a small share of traffic to the new version, watch error rates and latency, and roll back quickly when they cross a threshold. Treat configuration changes, certificate rotations, and key rotations with the same care as code, and require review for ad-hoc production access.

On Control Plane: each location keeps serving the last healthy version until the new one passes its readiness checks, then traffic switches and the old version drains, so a release that fails the readiness probes you define never takes traffic. Domain routes split traffic by weight for canary releases (docs), and a blue-green cutover or rollback is one route change (docs).

7. Design for degraded operation

Decide in advance what your service does when a dependency fails. A product page can load without recommendations, and checkout can queue an order when the email service is down. Use timeouts, retries with backoff, circuit breakers, and cached fallbacks so one failing dependency does not cascade.

8. Handle load

Traffic spikes look like outages to users. Use autoscaling, rate limiting, and a CDN, and keep headroom so that when one replica fails, the others can carry its load.

9. Test failure on purpose

Systems that have never been tested under failure usually fail the first time it happens. Run game days and chaos experiments: kill instances, block a zone, break a dependency, and expire a certificate in staging. Rehearse your disaster recovery runbook, not only your backups, and restore from backup on a schedule to prove it works.

On Control Plane: suspend a workload in one location to simulate a regional failure and confirm the global endpoint keeps serving from the others.

10. Monitor from the outside and be ready to respond

Dashboards that run on your own infrastructure can go dark with it. Add independent, outside-in checks from multiple regions that test what users actually do. Cover DNS resolution, TLS certificate expiry, and domain registration expiry, because an expired certificate or domain takes a service down as surely as a failed server. Alert on error rate, latency, and saturation, write runbooks, define on-call ownership, and publish a status page so customers hear from you first.

On Control Plane: TLS certificates for workload endpoints and custom domains are issued and renewed automatically, which removes certificate expiry from your list (docs).

High availability vs disaster recovery

High availability	Disaster recovery
Handles	Everyday, frequent failures
Goal	Keep serving with no user impact
Key metric	Availability percentage
Method	Redundancy, automatic failover

Recovery Time Objective (RTO): the longest downtime you will accept after a disaster.
Recovery Point Objective (RPO): the most data you can afford to lose, measured in time.

Targets near zero for both require continuous replication and automated failover. Active-active designs get closest, because every location is already serving and the data is already replicated. Set RTO and RPO per workload with the business, not only with engineering.

On Control Plane: compute runs active-active in every location, and the Template Catalog installs databases built for multi-location operation, including pgEdge multi-master PostgreSQL and CockroachDB, which survives the loss of a region when deployed to three or more locations (Template Catalog). Both templates can schedule backups to S3 or GCS.

What it takes to reach each availability target

Target	What you need
99.9%	Multiple instances in one zone, a load balancer, automated backups, basic monitoring
99.99%	Multiple zones, automated failover, zero-downtime deployments, outside-in monitoring, tested runbooks
99.999%	Multi-region, often multi-cloud, active-active compute and replicated data, health-based global routing, safe rollouts, regular failure testing

Common mistakes

Treating the provider SLA as the reliability plan.
Running redundant copies that share a region, pipeline, or DNS provider.
Relying on manual failover steps.
Never restoring from backup until the day you need it.
Monitoring only from inside your own infrastructure.
Skipping change control for configuration and certificates.
Ignoring third-party dependencies in the request path.

How Control Plane delivers five nines

Building the 99.999% tier yourself means running health checks, global routing, cross-cloud networking, deployment tooling, and observability across several environments, then keeping all of it working through every change. Control Plane handles that platform layer as Day 2 operations, so your team spends its time on the application.

Active-active compute and data. A workload runs in every location of its Global Virtual Cloud at once, across AWS, GCP, and Azure regions, your own clusters, or on premises. With latency-based routing every location in the same routing tier serves live traffic, so you are not paying for idle standby capacity, and templates such as pgEdge and CockroachDB keep data replicated across locations.
Automatic failover. The global endpoint sends each request to the nearest healthy location, and when a location fails, traffic moves to the others without a runbook.
Safe rollouts. A new version takes traffic only after it passes the readiness probes you define in each location, with canary and blue-green routing built in.
One operating model. About 30 regions across AWS, GCP, and Azure, plus your own clusters and servers, run with one identity model, mutual TLS between services, a deny-by-default firewall, per-location autoscaling, one view of logs and metrics with Grafana alerting, and TLS certificates issued and renewed automatically.
A five-nines target. A 99.999% availability SLA for workloads running at least two replicas in at least two locations (docs).

The proof is how it behaves on a bad day. No Control Plane customer experienced downtime during the October 20, 2025 AWS outage in us-east-1, which lasted about 15 hours and affected more than 100 AWS services, according to Control Plane internal incident telemetry. 20% of customers had us-east-1 in their Global Virtual Clouds, and their affected workloads were active and running in healthy locations within 10 seconds (Beyond Backups). Baerskin Tactical, a Switzerland-based direct-to-consumer e-commerce company, moved to Control Plane after vendor downtime disrupted its revenue. CTO Gus Fune: “Control Plane positively impacted DIV Brands’ daily operations by enabling us to host services in multiple regions with 99.999% availability and ultra-low latency…” (case study)

Control Plane takes compute placement, global routing, failover, rollouts, and certificates off your list, so your team can apply the rest of this guide to what remains: application code, third-party dependencies, and your data.

Frequently asked questions

Can any website guarantee 100% uptime?

No. Every system has failure modes, and even the rare 100% SLAs, such as those for Amazon Route 53 and Azure DNS, pay only partial service credits. The practical goal is a measured target, such as 99.999%, reached by making failures invisible to users through redundancy and automatic failover.

What is the highest uptime you can realistically achieve?

Five nines (99.999%), about 5 minutes 15 seconds of downtime a year. For comparison, the compute SLAs for Amazon EC2, Azure Virtual Machines, and Google Compute Engine top out at 99.99% across multiple zones. Reaching five nines takes multi-region active-active redundancy and automated failover, and Control Plane offers a 99.999% availability SLA for workloads running at least two replicas in at least two locations.

What is the difference between uptime and availability?

Uptime usually means a system is running. Availability measures whether users can use it successfully. A server can be up while the application returns errors, so availability is the more honest measure.

How do I calculate allowed downtime for an SLA?

Multiply the unavailable fraction by the period. For 99.9% over a 30-day month, 0.001 x 43,200 minutes = 43.2 minutes. Over an average month of 730 hours, it is about 43 minutes 48 seconds.

Is multi-region always necessary?

It depends on your target. For 99.9% or 99.99%, multiple zones in one region are often enough. Multi-region becomes worthwhile when a regional outage would cost more than you can accept. On Control Plane, adding a region means adding a location to the Global Virtual Cloud, and routing and failover come with it.

Does multi-cloud improve uptime?

Yes, because it protects against provider-wide failures such as a control-plane outage or an account lockout. On Control Plane, one Global Virtual Cloud can span AWS, GCP, and Azure, and the platform handles cross-cloud routing, identity, and mutual TLS, so a provider-wide failure becomes a location failure.

What causes most outages?

Changes. Google’s SRE team reports that roughly 70% of outages are due to changes in a live system, which is why safe deployment practices matter as much as redundant hardware. Microsoft’s reliability guidance also lists software bugs, human error, and unexpected traffic alongside infrastructure failures.

How often should you test failover?

Many teams test critical workloads quarterly and after any major architecture change. Microsoft recommends testing disaster recovery plans routinely and reviewing them, ideally, every six months. Rehearsing the process matters as much as testing the technology.

Checklist: is your service built to stay up?

Availability targets set per workload and flow
No single point of failure in compute, data, network, DNS, or CDN
At least two replicas per location, across zones, and across regions for critical flows
Automatic, health-based failover in place
Canary or blue-green deployments with fast rollback
Graceful degradation designed for each dependency
Autoscaling and rate limiting configured
Backups restored and failover rehearsed on a schedule
Outside-in monitoring covering DNS, TLS certificates, and domain expiry
Runbooks, on-call ownership, and a status page
RTO and RPO documented and agreed with the business

Next step: Sign up free, deploy a workload to two locations with two replicas each, and suspend one location to watch traffic fail over, or talk to the Control Plane team about the availability target your business needs.

Sources

Originally published on the Control Plane blog.

Best Cloud Disaster Recovery Solutions for 2026

Noah Grinstein — Fri, 09 Oct 2026 19:59:48 +0000

TL;DR

Control Plane is the best overall cloud disaster recovery solution. Workloads fail over automatically across regions and clouds under a 99.999% SLA. During the October 2025 AWS outage, affected workloads were running in healthy locations within 10 seconds, according to Control Plane internal incident telemetry.
Native hyperscaler DR tools (AWS Elastic Disaster Recovery, Azure Site Recovery) suit single-cloud teams, but only recover into their own cloud and failover is operator-initiated.
Replication-based DRaaS (Zerto, Veeam) fits VM-heavy, on-premises estates with active-passive standby.
Developer PaaS platforms don’t document automatic cross-provider failover.

Most disaster recovery plans are designed to survive infrastructure failures, such as a zone going down or a region becoming unavailable, but assume the cloud provider itself stays up. That assumption fails more often than engineering teams expect. When it does, the difference between a team that keeps running and a team writing incident reports is whether disaster recovery was an architectural default or a runbook nobody had tested.

This piece is the vendor-comparison companion to our technical guide. For the full breakdown of RTO/RPO tiers, failover patterns, and implementation, see Cloud disaster recovery strategies, RTO/RPO targets, and best practices. Here the focus is buyer-facing: which cloud disaster recovery platforms and approaches are available today, evaluated against the criteria a CTO or platform team uses during vendor selection: how fast recovery happens, how automatically it happens, and whether it survives losing an entire cloud provider as well as a single availability zone.

How we evaluated these

Each option was scored on four questions:

Does failover survive a full provider outage, or only a zone or region failure inside one cloud?
Is recovery automated (health-aware routing that reacts in seconds) or operator-initiated (a recovery plan a person starts under pressure)?
What RTO/RPO tier does it realistically support (hours, minutes, or near zero)?
What does it cost to run, in infrastructure spend and in team effort to build and maintain it?

The four DR topology patterns and their cost

Cloud DR strategies fall into four postures, Backup and Restore, Pilot Light, Warm Standby, and Multi-Site Active-Active, with different RTO/RPO outcomes and cost profiles. The table below is a quick orientation; for how each pattern works and how to choose between them, see the technical guide linked above.

Pattern	Typical RTO	Typical RPO	Always-on cost
Backup and Restore	Hours	Hours	Minimal (storage only)
Pilot Light	Tens of minutes	Minutes	Low (minimal standing infrastructure)
Warm Standby	Minutes	Minutes	Moderate (partial idle infrastructure)
Multi-Site Active-Active	Seconds	Near zero (synchronous) to seconds (asynchronous)	No dedicated standby; every site serves traffic, plus failover headroom

The ranking

1. Control Plane: best overall, with automatic cross-cloud failover built in

Control Plane’s disaster recovery model is Unbreakable Compute: if an entire region or cloud provider fails, workloads fail over automatically to surviving regions or other clouds with no human intervention. It is backed by a 99.999% availability SLA, which allows a maximum of about 5 minutes 15 seconds of downtime a year.

During the October 20, 2025 AWS outage, which began in us-east-1, ran roughly 15 hours, and disrupted more than 100 AWS services, 20 percent of Control Plane customers had an AWS us-east-1 location in their Global Virtual Cloud (GVC). Failover was automatic: affected workloads were active and running in healthy locations within 10 seconds, and not one Control Plane customer went down (Beyond Backups).

How it works: a GVC spans regions across AWS, GCP, and Azure, GPU neoclouds, and infrastructure you own as one deployment target, connected by a single mutual-TLS network. Universal Cloud Identity gives workloads credential-free access to 600+ services across AWS, GCP, and Azure, so failing over between those providers does not require rebuilding your identity and access model. For GVC configuration and health-based routing details, see the technical guide.

Compute and data fail over together: Control Plane runs stateful workloads alongside stateless ones in the same GVC, and ships multi-location Postgres and Redis templates that automatically promote a replica in a surviving location. Compute failover takes seconds; RPO follows the replication mode you choose for each database.

Why teams choose this over building it themselves: most engineering teams build active-active and cross-provider failover incrementally, under pressure, after something has already broken. Control Plane ships active-active, cross-provider failover, a single mutual-TLS network, and cross-cloud identity as platform defaults, so none of it becomes a second infrastructure project.

Customer proof point: SAFE Health, a digital health and connected-diagnostics platform operated in partnership with the Mayo Clinic, moved off AWS-only infrastructure: “We migrated 8 environments from AWS to Control Plane and no one noticed! Everything just worked.” Their engineering leader added that while they run on multiple AWS regions today, “with Control Plane adding GCP regions is trivial. It’s a switch.” (case study)

Compliance: SOC 2 Type II, PCI DSS Level 1, HIPAA, and GDPR, with one control set across every location, which matters when disaster recovery is evaluated alongside an audit requirement.

Best for: teams that want active-active compute and data across regions, on one cloud or several. Teams concentrated on a single hyperscaler get provider-level failover as headroom.

Other options worth knowing

These are the realistic alternatives buyers evaluate, with a side-by-side comparison at the end of this section.

Developer platforms and PaaS (such as Render, and Railway) cover build, deploy, and managed databases on one platform. Most offer database backups, and some offer high-availability database failover. They are built around deploying to regions you choose, and their documentation does not describe a single workload that fails over automatically from one cloud provider to another. For teams whose requirement is surviving the loss of an entire provider, that is the gap that matters.

Native hyperscaler DR tools (AWS Elastic Disaster Recovery, Azure Site Recovery) recover servers into their own provider’s regions well, and can replicate from on premises or other clouds. They only recover into their own cloud, so AWS Elastic Disaster Recovery cannot fail you over to GCP or Azure, and failover is operator-initiated rather than automatic (Microsoft). The October 20, 2025 incident began in one region and still broke console sign-in and IAM-dependent services in other AWS regions (AWS), which shows how far one provider’s shared dependencies reach. These tools suit teams firmly on one cloud and stop being sufficient once losing the whole provider becomes a real question.

Traditional replication-based DRaaS (Zerto, Veeam) continuously replicates VMs to a standby site or a service provider’s cloud and runs orchestrated recovery plans, reaching RTO in minutes and RPO in seconds for replicated VMs (Zerto). It fits VM-centric warm-standby postures in enterprises with significant on-premises infrastructure. It was not designed for containerized, multi-region active-active architectures: failover is an active-passive event someone initiates, and you pay for standby capacity.

Comparison at a glance

Criteria	Control Plane	Developer PaaS	Native hyperscaler DR (AWS EDR / Azure Site Recovery)	Traditional DRaaS
Survives a full provider outage	Yes, automatic cross-cloud failover	No automatic cross-provider failover documented	No, recovers only into its own cloud	Yes, to a standby replica site (active-passive)
Failover trigger	Automatic, health-aware geo-routing (seconds)	Varies; high-availability database failover on some platforms	Operator-initiated (one click or API)	Orchestrated recovery plan, typically operator-initiated
Best RTO tier supported	Seconds (active-active)	Varies; generally not published	Minutes	Minutes (replicated VMs)
Best RPO tier supported	Near zero, set by replication mode	Varies by database and backup settings	Seconds to 5 minutes	Seconds (replicated VMs)
Identity across clouds	Universal Cloud Identity: credential-free access to 600+ services across AWS, GCP, and Azure	No cross-cloud identity layer	Native to one provider	Not applicable
Data portability / vendor lock-in	Low: standard containers portable across supported clouds	Moderate to high: tied to the platform’s regions and services	High: tooling and recovery target are provider-specific	Moderate: depends on replication target
Cloud-native / containerized workloads	Yes, built for containerized workloads across clouds	Yes, container-first	Partial: best for VM-based servers	Limited: designed for VM workloads
Compliance certifications	SOC 2 Type II, PCI DSS Level 1, HIPAA, GDPR	Varies by platform	Broad certifications, scoped to one provider	Varies by vendor
Pricing model	Usage-based, metered by millicore and megabyte; no idle standby in active-active	Per-service or per-resource pricing	DR charges per protected server plus recovery infrastructure	Per-VM or per-GB replication fees plus standby infrastructure
Best fit buyer	Teams needing automatic cross-region or cross-cloud DR with no manual failover	Teams wanting one platform for build, deploy, and databases in chosen regions	Teams firmly on one cloud needing zone or region resilience	Enterprises with significant on-premises or VM-based workloads

FAQ

What’s the difference between backup and disaster recovery?

Backup protects data. Disaster recovery restores service. A backup with no automated way to bring the application back online on other infrastructure leaves your team executing a manual recovery under pressure, and that recovery is what determines your RTO, or how long you are down.

Is cloud-based disaster recovery better than on-premises?

Cloud DR usually wins on elasticity, since you are not paying for permanently idle standby hardware, and on how quickly you can add failover regions. On-premises DR offers direct control over data locality. Control Plane removes the either-or choice: you can keep specific workloads on premises or in a specific jurisdiction while the rest fail over automatically across AWS, GCP, and Azure.

Can disaster recovery span multiple cloud providers automatically?

Yes, on platforms built for it. Native hyperscaler DR tools recover only into their own cloud. Cross-cloud DR needs a layer such as Control Plane’s Global Virtual Cloud, which treats regions on multiple providers as one deployment target with a single network and identity model, so failover becomes a traffic-routing change the platform handles automatically. For configuration details, see the technical guide.

What RTO and RPO should I target?

Set RTO and RPO per workload tier. Customer-facing production systems usually justify warm standby or active-active, with RTO and RPO from minutes down to near zero. Internal tools and batch jobs are often fine with pilot light or backup and restore. For a full workload-tiering framework, see the technical guide.

Does multi-region deployment alone protect against a provider-wide outage?

Partly. Multiple regions inside one cloud protect against a regional failure, and during the October 20, 2025 AWS incident most workloads outside us-east-1 kept serving. The same incident still broke console sign-in and IAM-dependent services in other AWS regions, because those depend on us-east-1. Spreading locations across providers removes that shared dependency on one provider’s global services.

See automatic multi-cloud failover on your own workload: Book a demo

Originally published on the Control Plane blog.