DEV Community

Cover image for Moving Compute from GCP to AWS: VMs, Kubernetes and GPUs
Emmanuel R for CobuildX AI

Posted on Originally published at cobuildx.ai

Moving Compute from GCP to AWS: VMs, Kubernetes and GPUs

A plain-English guide to moving compute from Google Cloud to AWS: VMs, instance groups, Spot, GKE, GPUs, TPUs and batch. What replaces what, what behaves differently, what changes on the bill, and a step-by-step plan.

"Compute" is the part of the cloud that runs your code: virtual machines, Kubernetes nodes, GPU machines and batch jobs. When a team moves from Google Cloud (GCP) to AWS, compute is usually the first thing they plan.

Teams move for simple reasons. The rest of the company is already on AWS. A company they bought came with an AWS account. Their credits ran out on one cloud and started on the other. Or the data they need already lives in AWS.

Most teams start with a table: Compute Engine goes to EC2, GKE goes to EKS, GPUs go to GPUs. That table is a good start, but it hides the real work. Services that do the same job often behave a little differently. Machine sizes, maintenance, Spot, pod networking and discounts all change, and those changes show up in production, not on the diagram.

In this post we go through every GCP compute option one by one. For each, we say what replaces it on AWS, what is different, and what changes on the bill. At the end we show a step-by-step plan and how we run these moves with MigrateX.

This guide covers compute. For Cloud Run, Cloud Run functions and Lambda, see moving serverless from GCP to AWS. For Persistent Disk and Filestore, see moving storage from GCP to AWS. For Dataproc and Dataflow, see moving big data from GCP to AWS.

The Quick Map

Here is every GCP compute option and its closest match on AWS. The rest of the post explains each row.

Compute service map from Google Cloud to AWS: Compute Engine to EC2, managed instance groups to Auto Scaling groups, Spot VMs to EC2 Spot, GKE to EKS or EKS Auto Mode, GPUs and TPUs to EC2 P and G instances or Trainium, and Batch to AWS Batch, with what to watch for on each

GCP Closest AWS match Watch out for
Compute Engine VMs EC2 No custom machine sizes; pick the next size up
Instance templates EC2 launch templates Different fields; rebuild them, don't convert
Managed instance groups (MIGs) EC2 Auto Scaling groups Health checks and zones are set up differently
Spot VMs / preemptible VMs EC2 Spot Instances Different warning time and pricing
Sole-tenant nodes EC2 Dedicated Hosts Licence rules differ per host type
Machine images, custom images Amazon Machine Images (AMIs) Rebuild or replicate; you can't copy them across
Persistent Disk on a VM Amazon EBS Covered in the Storage post
GKE Standard EKS with managed node groups or Karpenter Pods use real VPC IPs; plan subnets
GKE Autopilot EKS Auto Mode Nodes are replaced at least every 21 days
GPU machines (A2, A3, A4, G2) EC2 P and G instances Getting capacity is the hard part
Cloud TPU Trainium / Inferentia, or GPUs There is no TPU on AWS
Batch AWS Batch Job settings and retries map loosely
App Engine Elastic Beanstalk or App Runner Often a good time to move to containers
Cloud Run / Cloud Run functions App Runner, ECS on Fargate, Lambda See the Serverless post
Bare Metal Solution EC2 bare metal (.metal) instances Check hardware and licence needs

Virtual Machines: Compute Engine to EC2

This is the easiest part of the move. Both clouds group their machines by job: general purpose, compute-heavy, memory-heavy and GPU. So matching them is mostly a lookup:

GCP series Used for AWS family Note
E2 Cheap, light general use t3 / t4g AWS uses CPU credits; busy machines can slow down
N2, N2D, N4 General purpose m7i / m8i, m7a Intel and AMD options on both sides
Tau T2A, Axion (C4A) Arm general purpose m8g, c8g (Graviton) Often the cheapest per unit of work
C3, C4, C2 Compute-heavy c7i / c8i, c7a Match by vCPU and clock speed, not by name
M1, M2, M3 Large memory r8i, x2iedn, u-series Check memory per vCPU
A2, A3, G2 GPU P4d, P5, G6 See the GPU section below

Two things don't carry over:

  • Custom machine types. On GCP you can ask for any shape, say 6 vCPUs and 20 GB of memory. EC2 only sells fixed sizes, so you round up and pay for the gap. Before you round up, look at what the VM really uses. Many custom shapes were guesses made years ago.
  • Live migration. When Google does maintenance, it moves your running VM to new hardware without a reboot. EC2 instead schedules maintenance events, which can mean a stop and start. If a single VM can't be restarted, put it behind an Auto Scaling group or run a second copy.

Rule of thumb: size EC2 from measured CPU and memory, not from the GCP machine type. Rounding up every custom shape is the quickest way to make AWS look more expensive than it is.

What changes inside the VM

On GCP On AWS
Google guest agent SSM Agent and cloud-init
Startup script in instance metadata EC2 user data
Metadata server at metadata.google.internal Instance metadata (IMDSv2) at 169.254.169.254
VM service account IAM role attached through an instance profile
OS Login / SSH keys in metadata SSM Session Manager (no open SSH port needed)
Google-provided OS images Amazon Linux, Ubuntu, RHEL or Windows AMIs

Instance Groups: MIGs to Auto Scaling Groups

A managed instance group (MIG) is a group of identical VMs that grows or shrinks on its own. It's built from an instance template. On AWS the same thing is an Auto Scaling group built from a launch template.

Scaling on CPU, rolling updates and replacing broken VMs all have direct matches. Two details are different:

  • Zones. A regional MIG spreads across zones by itself. An Auto Scaling group only spreads across the subnets you give it, so give it one subnet in each Availability Zone.
  • Health checks. A MIG uses its own health check to replace broken VMs. An Auto Scaling group only checks that the EC2 machine is up, unless you switch it to use the load balancer's health check (health_check_type = "ELB").

Load balancers change too. Google's global HTTP(S) load balancer usually becomes an Application Load Balancer (ALB), and TCP/UDP load balancers become Network Load Balancers (NLB). AWS load balancers are regional, so for global traffic add Route 53 or CloudFront in front.

Cheaper Capacity: Spot VMs

Both clouds sell spare machines at a big discount and take them back when they need them. The rules are different:

GCP Spot VMs EC2 Spot Instances
Warning before it's taken back 0 seconds by default, or 120 seconds if you set it Two minutes, best effort
Price Fixed discount, changes at most once a month Changes slowly by instance type and zone
How you find out Shutdown signal, metadata server Instance metadata and an EventBridge event
Best practice Spread across zones and machine types List many instance types and use capacity-optimized allocation

Shutdown scripts written for GCP usually still work. You just need to change them to read the AWS notice. The bigger change is to use many instance types. A group that lists one instance type gets interrupted far more often than one that lists ten.

Dedicated Hardware and Licences

Sole-tenant nodes become EC2 Dedicated Hosts. Most teams use them for software licensed per physical core, like Windows Server or SQL Server under bring-your-own-licence terms. Read your licence terms for AWS before you move. Often it's cheaper to use licence-included EC2 instances and drop dedicated hardware completely.

If you use Bare Metal Solution (for example, for Oracle), look at EC2 bare metal instances, or Amazon RDS if the database can be managed for you.

Moving the VMs Themselves

There are two ways to get a VM onto EC2: copy the running server, or rebuild it from code.

Two ways to move a Compute Engine VM: rehost with AWS Transform MGN, which installs an agent, replicates disks continuously, launches a test instance and then cuts over in minutes; or rebuild with Packer or EC2 Image Builder, producing an AMI that an Auto Scaling group launches

  • Copy it (rehost). AWS Transform MGN (formerly AWS Application Migration Service) copies servers to AWS with very little downtime. You install an agent on the GCP VM. It keeps copying the disks to AWS. You launch a test machine, check it, then switch over in minutes. Use this for old servers nobody can rebuild, licensed software, or hand-tuned databases.
  • Rebuild it. If the VM is built from a startup script, Ansible or Packer, use the same tools to build an AMI (Packer's amazon-ebs builder, or EC2 Image Builder) and launch it from an Auto Scaling group. It's more work at first, but you get something you can rebuild any time.

Either way, plan the network around the move. IP addresses change, so use DNS names, not hard-coded IPs. Lower DNS TTLs a few days before you switch. And set up a private link between the clouds (VPN or Interconnect to Direct Connect) so old and new servers can talk while both are running.

Kubernetes: GKE to EKS

Your Kubernetes manifests and Helm charts mostly move as they are. The cluster around them doesn't. Four things catch almost every team:

  • Networking. On EKS, every pod gets a real IP address from your VPC subnet. A busy cluster in a small subnet runs out of addresses fast. Use bigger subnets, or turn on prefix delegation, before you move workloads.
  • Identity. GKE Workload Identity becomes EKS Pod Identity (or the older IRSA). Each Kubernetes service account gets an IAM role. It's the same idea, but every link has to be made again. This is where most "access denied" errors come from in the first week.
  • Ingress. GKE creates Google load balancers for you. On EKS, install the AWS Load Balancer Controller, which creates ALBs and NLBs. The annotations are different, so check every Ingress.
  • Nodes. If you use GKE Autopilot, EKS Auto Mode is the closest match. If you manage node pools yourself on GKE Standard, use managed node groups or Karpenter. Karpenter picks the right instance type for each workload, which often saves money and makes Spot much safer.

One difference from Autopilot: EKS Auto Mode replaces every node after at most 21 days. Your workloads must be fine with nodes coming and going. Storage classes change too: GCE Persistent Disk volumes become EBS volumes through the EBS CSI driver.

Topic GKE EKS
Pod identity Workload Identity Federation EKS Pod Identity (or IRSA)
Pod IPs Separate IP ranges from nodes Pods take IPs from the node's subnet
Ingress GKE Ingress / Gateway creates Google LBs AWS Load Balancer Controller creates ALBs / NLBs
Node autoscaling Cluster autoscaler, node auto-provisioning Karpenter, or built in with Auto Mode
Fully managed nodes Autopilot EKS Auto Mode (21-day max node life)
Disks for pods GCE PD CSI driver EBS CSI driver
Control plane cost Cluster fee, with a monthly free credit $0.10 per cluster per hour (standard support)

Plan the subnets first. Running out of pod IPs is the most common EKS surprise for teams coming from GKE, and it's much easier to fix before the cluster exists.

GPUs Move, TPUs Don't

GPU workloads move fairly easily. Both clouds use the same NVIDIA chips, so CUDA code doesn't change:

GCP machine GPU AWS instance
A2 NVIDIA A100 P4d / P4de
A3 NVIDIA H100 / H200 P5 / P5e / P5en
A4 NVIDIA B200 P6-B200
G2 NVIDIA L4 G6
Cloud TPU TPU No match: GPUs, or Trainium / Inferentia

The hard part is getting the machines. New AWS accounts start with low GPU limits, so ask for more weeks ahead. If you need many GPUs for a fixed period, look at Capacity Blocks for ML.

TPUs are different. AWS has no TPU, so TPU code has to be changed, not just moved. JAX and PyTorch/XLA models can run on GPUs with some work. AWS's own chips (Trainium for training, Inferentia for serving) can cost less, but they need the Neuron SDK and testing for each model. Treat this as its own engineering project.

Batch Jobs

GCP Batch becomes AWS Batch. Both take a job, put it in a queue, and run it on machines they create for you. AWS Batch can run on EC2 (normal or Spot), on Fargate, or on EKS. For long jobs that can be stopped, run them on Spot with several instance types and save progress along the way (checkpoints), so a lost machine costs minutes, not hours. Retry settings are not the same on both sides, so set them on purpose.

Networking You'll Trip Over

Networking isn't compute, but it's where compute moves go wrong:

  • VPCs are regional on AWS. One global VPC on GCP becomes one VPC per region, joined with Transit Gateway or peering.
  • Subnets live in one zone. On GCP a subnet covers the whole region. On AWS each subnet is in one Availability Zone, so you need one per zone.
  • Firewall rules become security groups. GCP firewall rules use network tags. AWS security groups are attached to each machine and can point at other security groups, which is cleaner but means rewriting every rule.
  • Traffic between zones costs money. On AWS, data that crosses Availability Zones and NAT Gateway traffic are charged per GB. Keep chatty services in the same zone where you can.

A Simple Way to Decide

Compute decision flow: if it needs TPUs, port first; if it is on Kubernetes and staying there, EKS; if it is batch or interruptible, AWS Batch on Spot; if it needs per-core licences or dedicated hardware, Dedicated Hosts or licence-included EC2; otherwise EC2 in an Auto Scaling group

For each workload, ask these questions in order and stop at the first yes:

  1. Does it need TPUs? Plan the move to GPUs or Trainium before anything else.
  2. Is it on Kubernetes, and does the team want to keep it there? EKS (Auto Mode if you were on Autopilot).
  3. Is it batch work that can be stopped and restarted? AWS Batch on Spot.
  4. Does it need per-core licences or dedicated hardware? EC2 Dedicated Hosts, or licence-included EC2.
  5. Is it small, stateless and request-driven? It may not need a VM at all. See the Serverless post.
  6. Everything else: EC2 in an Auto Scaling group, sized from real usage.

What Changes on the Bill

Most cost surprises come from discounts, not list prices.

Topic GCP AWS
Automatic discounts Sustained use discounts on some series (N1, N2, N2D, C2, M1, M2) None; you pay the list price all month
Commitments Committed use discounts Savings Plans and Reserved Instances
Billing unit Per second (1-minute minimum) Per second for Linux (1-minute minimum)
Spot pricing Fixed discount, changes at most monthly Changes by instance type and zone
Managed Kubernetes Cluster fee, with a monthly free credit $0.10 per cluster per hour
Network Premium global network by default Cross-zone traffic and NAT Gateway charged per GB

The discount that quietly disappears is the sustained use discount. On GCP, a VM that runs most of the month gets up to 30% off automatically. AWS has no such thing. You only get that money back by buying a Savings Plan.

Don't buy one in the first weeks. Run on normal pricing until usage is steady, then commit to the amount you're sure you'll use. A Compute Savings Plan also covers EKS nodes, Fargate and Lambda, so you don't have to commit to one instance family.

What It Looks Like in Terraform

A regional managed instance group becomes a launch template plus an Auto Scaling group:

# GCP: a group of 3 identical VMs across zones
resource "google_compute_region_instance_group_manager" "web" {
  name               = "web"
  base_instance_name = "web"
  region             = "us-central1"
  target_size        = 3
  version { instance_template = google_compute_instance_template.web.id }
}

# AWS: the same group as an Auto Scaling group
resource "aws_autoscaling_group" "web" {
  name                = "web"
  desired_capacity    = 3
  min_size            = 3
  max_size            = 6
  vpc_zone_identifier = var.private_subnet_ids # one subnet per zone
  health_check_type   = "ELB"                  # use the load balancer's check
  launch_template {
    id      = aws_launch_template.web.id
    version = "$Latest"
  }
}
Enter fullscreen mode Exit fullscreen mode

A group that uses Spot should list several instance types. Inside the aws_autoscaling_group, replace the launch_template block with:

mixed_instances_policy {
  instances_distribution {
    on_demand_base_capacity                  = 1  # always keep one normal VM
    on_demand_percentage_above_base_capacity = 0  # the rest on Spot
    spot_allocation_strategy                 = "price-capacity-optimized"
  }
  launch_template {
    launch_template_specification { launch_template_id = aws_launch_template.web.id }
    override { instance_type = "m7i.large" }
    override { instance_type = "m7a.large" }
    override { instance_type = "m6i.large" }
  }
}
Enter fullscreen mode Exit fullscreen mode

These snippets show the shape, not every setting. Check the current provider docs before you copy them.

Things That Bite Later

  • Service account keys. Every VM service account becomes an IAM role, and every key file hidden in a script needs a new home.
  • Metadata calls. Code that calls metadata.google.internal for project, zone or tokens must switch to IMDSv2 or the AWS SDK.
  • Limits. New AWS accounts start with low limits for vCPUs, GPUs and Elastic IPs. Ask for more weeks ahead.
  • AMI IDs differ per region. Look them up per region, or use SSM parameters, instead of hard-coding them.
  • Single-zone setups. Check that every Auto Scaling group and node group covers at least two Availability Zones.
  • Monitoring and logs. Cloud Monitoring and Cloud Logging agents become the CloudWatch agent. Rebuild alerts and dashboards before you switch traffic.

Doing the Move, Step by Step

  1. List everything. Every VM, instance group, cluster and batch job, with its machine type, real usage and what it talks to.
  2. Pick the target for each one using the questions above. Write down why, so you can revisit it later.
  3. Build the base: AWS accounts, VPCs, subnets big enough for EKS, IAM roles, limits and a private link to GCP.
  4. Start with a pilot. Move one small, low-risk service end to end to prove the path works.
  5. Move in waves. Stateless groups and clusters first, then rehost the harder servers with MGN.
  6. Shift traffic slowly with weighted DNS, and keep GCP running until AWS is steady.
  7. Clean up. Turn off GCP group by group, and buy Savings Plans only once AWS usage is steady.

Compute is usually the easy part of a cloud move. The service map fits on one page. The real work is in the details: machine sizes, maintenance, Spot, pod IPs, identity and discounts. Decide those on purpose and the rest is mostly typing.

Common Questions

Compute Engine vs EC2: what is actually different?

The machines are close matches. Two things don't carry over: EC2 has no custom machine types, so you round up to the next fixed size, and EC2 has no live migration, so maintenance can mean a stop and start. The Google guest agent, startup scripts and metadata server are replaced by SSM Agent, user data and IMDSv2.

How do I map GCP machine types to EC2 instance types?

Match by job first, then by vCPU, memory and clock speed, not by name. E2 maps to t3 or t4g, N2 and N2D to m7i or m7a, C3 and C4 to c7i or c8i, M-series to r8i or x2iedn, and Arm machines like Axion to Graviton (m8g, c8g). Size from what the VM really uses, not from its old shape.

How do I migrate a VM from GCP to AWS step by step?

Either copy it or rebuild it. AWS Transform MGN (formerly AWS Application Migration Service) installs an agent on the GCP VM, keeps the disks in sync with AWS, and lets you switch over in minutes. If the VM is built from code with Packer or Ansible, build an AMI instead and run it in an Auto Scaling group.

What is the GKE to EKS migration checklist?

Your manifests and Helm charts mostly move as they are. Plan four things: bigger subnets (every pod takes a VPC IP), Workload Identity to EKS Pod Identity, GKE Ingress to the AWS Load Balancer Controller, and nodes (EKS Auto Mode for Autopilot, managed node groups or Karpenter for GKE Standard). Swap the GCE PD CSI driver for the EBS CSI driver.

Is there a TPU equivalent on AWS?

No. TPU code has to be changed, not just moved. JAX and PyTorch/XLA models can run on NVIDIA GPUs with some work, or on AWS Trainium and Inferentia through the Neuron SDK. Treat it as its own project.

What replaces GCP sustained use discounts on AWS?

Nothing automatic. On AWS you get that saving back by buying a Compute Savings Plan, which also covers EKS nodes, Fargate and Lambda. Wait until usage is steady before you commit.

About MigrateX

MigrateX

MigrateX is our platform and service for moving software, data and infrastructure from one technology to another. It combines automation with experienced engineers. The automation does the repetitive work quickly, and the engineers make the tricky decisions.

This guide is how we approach GCP to AWS compute moves with MigrateX. Here's what that looks like:

  1. Assessment. We list every VM, instance group, GKE cluster, GPU machine and batch job, with real usage, dependencies, licences and the hidden ties to GCP, like metadata calls and service account keys. You see where the work is before anyone agrees on a timeline.
  2. Plan. We work with your team to pick the AWS target for each workload, right-size the machines, design the network and accounts, and decide the order of the waves.
  3. Migration, step by step. Automation does the repetitive parts, like turning Terraform and instance templates into launch templates, Auto Scaling groups and EKS setups, and replicating servers with MGN. Our engineers handle the parts that need thought: identity, networking, Spot design and TPU code.
  4. Proof that it works. Each workload runs on both clouds side by side. We compare health, speed and cost, and only shift traffic once AWS matches or beats GCP.
  5. Handover and clean-up. We turn off GCP group by group, set up Savings Plans once usage is steady, and leave your team with AWS infrastructure they understand and own.

What you get along the way:

  • No big switch-over weekend. Your services keep running while the move happens one wave at a time.
  • Less risk. Side-by-side runs and slow traffic shifts mean you can always switch back.
  • A smaller bill, not a bigger one. Right-sizing, Spot and Savings Plans are part of the move, not an afterthought.
  • More than one kind of move. We use the same approach for storage, serverless and big data, and for front ends like Angular, Ember and jQuery.

Planning a move from GCP to AWS? We'd love to hear about it. Tell us a little about your setup through Contact Us, and we'll start with a free chat about where the work is likely to be.

References

AWS

Google Cloud

CobuildX helps teams migrate legacy applications and cloud infrastructure with MigrateX.


Originally published on the CobuildX blog.

Top comments (0)