Every organisation running workloads today is facing the same pressure: move faster, spend less, scale without limits, and do not go down. The answer the industry converged on is cloud. But moving to the cloud is not a single decision, it is a sequence of architectural, financial, and operational decisions that, made correctly, transform how a business operates. Made poorly, they create a mess that is more expensive and harder to manage than what you had before.
This guide covers the full migration journey: why organisations move, how they decide what to do with each workload, which AWS services make each move possible, and at the end, what it means to think beyond a single cloud provider entirely.
Every concept is introduced with the problem first, then the strategy, then the implementation, then the limits of that approach.
Table of Contents
- The Case for Cloud Migration - What You Are Actually Solving
- The 7Rs - Migration Strategy for Every Workload
- Discovery and Planning - Before You Move Anything
- AWS Migration Hub - Centralising the Journey
- AWS Application Migration Service (MGN) - Lift and Shift at Scale
- AWS Database Migration Service (DMS) - Moving Your Data
- AWS Snow Family - When the Network Is Not Enough
- AWS DataSync - Continuous Online Data Transfer
- AWS Transfer Family - Managed File Transfer Protocols
- VMware Cloud on AWS - The Hybrid Bridge
- Migration Execution - The Three-Phase Model
- Post-Migration Optimisation - The Work That Actually Delivers Value
- Vendor-Neutral: Designing for Multi-Cloud
The Case for Cloud Migration - What You Are Actually Solving
The Problem With On-Premises Infrastructure
An organisation running its own data centre is solving a set of problems that are not its core business. Procuring servers. Managing physical security. Negotiating power and cooling contracts. Patching hardware firmware. Replacing failed disks. Planning for capacity 18 months ahead because hardware lead times are long.
None of that creates value for the business. It is overhead. And it scales linearly, more workload means more hardware, more staff, more data centre space.
Beyond the operational burden, on-premises infrastructure has three structural problems:
Capital expenditure - Servers are bought years before they are fully utilised. The capital is spent upfront. Utilisation rates in enterprise data centres average 15-25%. You are paying for 100% of the capacity to use 20% of it.
Rigidity - Scaling up takes months (procurement, delivery, racking, configuration). Scaling down means idle hardware you already paid for. You cannot respond to demand, you guess in advance and live with the consequences.
Risk concentration - Your data centre is your single point of failure. A power event, a cooling failure, a network cut. The redundancy you can afford to build is limited by what you can physically fit in the building.
What Cloud Migration Actually Delivers
Cloud migration is not about moving servers. It is about changing the economic and operational model of running technology.
Operational expenditure - You pay for what you use, when you use it. Capital is not locked up in hardware. Infrastructure cost becomes variable, not fixed.
Elasticity - Capacity scales in minutes, not months. A spike in demand is handled automatically. A drop in demand means costs drop automatically.
Global reach - AWS operates in 33 regions worldwide. Deploying your application closer to your users is a configuration change, not a construction project.
Managed services - Every service you stop running yourself databases, message queues, load balancers, monitoring is a service your team stops maintaining and patching. That time goes back to building your product.
Security posture - AWS invests billions in physical and logical security that no individual organisation can match. The shared responsibility model means AWS handles the security of the infrastructure; you handle the security of what you run on it.
Migration is the means. These outcomes are the goal.
The 7Rs - Migration Strategy for Every Workload
The Problem
You have 200 applications. Some are modern. Some are ancient. Some are business-critical. Some are barely used. You cannot treat all of them the same way. Moving a legacy mainframe billing system the same way you move a containerised web application is wrong. You need a framework for making the right decision for each workload.
That framework is the 7Rs.
The Framework
The 7Rs are seven migration strategies. Every workload fits into one of them. The right choice depends on the workload's business value, technical complexity, age, and strategic direction.
R1 - Retire
What it is: Decommission the application entirely. Turn it off.
When to use it: The application provides no current business value, has no active users, or its functionality is already covered by another system. Analysis during the discovery phase often reveals that 10-20% of an application portfolio is in this category systems that nobody noticed were no longer being used.
What happens: Nothing moves to the cloud. The cost disappears entirely. This is the highest ROI migration strategy because there is nothing to migrate.
Question to ask: When did anyone last log into this system? If the answer is "I am not sure," it may be a candidate for retirement.
R2 - Retain
What it is: Keep the application where it is. Do not migrate it now.
When to use it: The application has a compliance requirement to remain on-premises. It depends on hardware that cannot be virtualised. It is being replaced soon and migration is not worth the investment. Or the technical complexity and risk of migration are not justified by the business benefit.
What happens: The application stays in the existing environment. It is explicitly excluded from the migration scope not forgotten, but intentionally deferred.
Important distinction: Retain is a decision, not an avoidance. You acknowledge the workload and consciously choose not to move it. Workloads with no clear decision are a risk, they fall through the cracks of a migration programme.
R3 - Rehost (Lift and Shift)
What it is: Move the application to the cloud with no changes. The same operating system, the same application binaries, the same configuration, running on EC2 instead of a physical server.
When to use it: Speed is the priority. The application is stable but not worth re-architecting. You want to exit a data centre by a deadline. The organisation needs cloud experience before attempting more complex migrations.
What happens: The application runs exactly as it did on-premises. No functionality changes. No performance improvements. No cost optimisation yet.
The trade-off: You get off the physical infrastructure quickly. But you pay cloud prices for on-premises thinking. An application running on an oversized server on-premises runs on an oversized EC2 instance in the cloud. The savings come later, during optimisation, not from the migration itself.
AWS service: Application Migration Service (MGN). Covered in section 5.
R4 - Replatform (Lift, Tinker, and Shift)
What it is: Make targeted optimisations during migration without changing the core application architecture.
When to use it: The application would benefit from a managed service, but a full rewrite is not justified. You want to reduce operational overhead without redesigning the system.
Examples:
- Move from self-managed MySQL on EC2 to Amazon RDS for MySQL. Same database engine, same data, same queries. But now AWS manages backups, patching, replication, and failover.
- Move from a self-managed message queue on EC2 to Amazon SQS. Same messaging pattern, no queue infrastructure to manage.
- Containerise the application without changing the application code, then run it on ECS instead of EC2. Same application, smaller and more portable runtime.
What happens: The application moves to the cloud with some managed services replacing self-managed components. Operational overhead decreases. The application is not refactored, it works the same way from the outside.
The trade-off: More work than rehost, but not as much as refactor. The optimisation payoff is real but bounded, you are still running an application that was not designed for the cloud.
R5 - Refactor / Re-architect
What it is: Redesign the application to use cloud-native capabilities. Change how it works, not just where it runs.
When to use it: The application has scaling problems that the current architecture cannot solve. Business agility requirements demand faster deployment cycles. The cost of running the current architecture in the cloud would be high relative to a re-architected version. The application is strategically important enough to justify the investment.
Examples:
- Break a monolithic application into microservices, each deployed independently.
- Move from a batch processing system to an event-driven architecture using Lambda and SQS.
- Replace a relational database with DynamoDB for a workload that needs to scale to millions of requests per second.
- Redesign a stateful application to be stateless, enabling horizontal scaling.
What happens: The application emerges from migration as a fundamentally different system. It scales elastically. It costs less at scale. It deploys faster. But the engineering investment is significant.
The trade-off: Highest effort, highest reward. This is the strategy that delivers genuine cloud-native benefits, not just running the same thing somewhere else.
R6 - Repurchase
What it is: Replace the application with a SaaS (Software as a Service) product.
When to use it: A commercial SaaS product covers the use case better than your existing application. The cost of maintaining the existing application exceeds the SaaS subscription cost. The organisation no longer wants to be in the business of running that type of software.
Examples:
- Replace a self-hosted CRM with Salesforce.
- Replace a self-hosted HR system with Workday.
- Replace a custom email platform with Google Workspace or Microsoft 365.
- Replace a self-hosted monitoring system with Datadog.
What happens: The application is decommissioned. Users move to the SaaS product. Data is migrated to the new system. Your team stops managing the software entirely.
The trade-off: You lose customisation. You become dependent on a vendor's roadmap. But you also eliminate all operational overhead for that system — forever.
R7 - Relocate
What it is: Move infrastructure to the cloud at the hypervisor level without changing the operating system, application, or how the application is managed.
When to use it: You are running VMware on-premises and want to move to the cloud without changing your VMware tooling, processes, or skills. You want the benefits of cloud (elasticity, managed infrastructure, pay-per-use) while keeping the exact same operational model.
What happens: Your VMware workloads move to VMware Cloud on AWS. The VMs look and behave identically. Your team continues using the VMware tools they know. AWS provides the physical infrastructure underneath.
The distinction from Rehost: Rehost moves individual workloads and re-platforms them onto EC2. Relocate moves entire VMware environments wholesale, preserving the VMware management layer.
AWS service: VMware Cloud on AWS. Covered in section 10.
Applying the 7Rs in Practice
The 7Rs are applied during the discovery and planning phase, before any migration begins. For each application in the portfolio, you answer:
- Is this application still needed? (Retire if no)
- Can it be migrated without unacceptable risk or cost? (Retain if no)
- Is a SaaS replacement available and appropriate? (Repurchase if yes)
- Does it run on VMware and should it stay on VMware? (Relocate if yes)
- Would a managed service reduce operational overhead without a rewrite? (Replatform if yes)
- Is a re-architecture justified by business or technical requirements? (Refactor if yes)
- Default: move as-is. (Rehost)
The result is a migration roadmap, not "migrate everything," but "here is what we do with each of the 200 applications and why."
Discovery and Planning - Before You Move Anything
The Problem
Organisations that skip discovery and jump straight to migration face a consistent set of problems: they discover dependencies mid-migration that they did not know existed; they underestimate the size and complexity of the work; they migrate applications that nobody needed; and they arrive in the cloud with the same technical debt and operational problems they had on-premises, just in a different location.
Discovery is not a formality. It is the work that determines whether the migration succeeds.
What Discovery Produces
Application inventory - Every application, its owner, its business function, and its technical stack.
Dependency mapping - Which applications talk to which other applications. Which databases they use. Which external services they depend on. This is what determines migration order, you cannot migrate an application before the systems it depends on.
Server inventory - CPU, memory, storage, network throughput for every server. This determines the right EC2 instance type and size for each workload.
Utilisation data - Actual resource usage over time, not just peak capacity. On-premises servers are often sized for peaks that almost never occur. Right-sizing to actual utilisation is where the first round of cost savings comes from.
TCO analysis - Total cost of ownership comparison: what it costs to run on-premises today versus what it would cost in the cloud. This is the business case for migration.
AWS Application Discovery Service
AWS Application Discovery Service automates the discovery process. It collects configuration, usage, and behaviour data from your servers, either agentlessly (via VMware vCenter integration) or with an agent installed on each server.
# Discovery agent installation on Linux
curl -o ./aws-discovery-agent.tar.gz \
https://s3.us-west-2.amazonaws.com/aws-discovery-agent.us-west-2/linux/latest/aws-discovery-agent.tar.gz
tar -xzf aws-discovery-agent.tar.gz
sudo bash install -r us-east-1 \
-k ACCESS_KEY_ID \
-s SECRET_ACCESS_KEY
The agent reports to AWS Migration Hub, where you can view the collected data, group servers into applications, and visualise dependencies.
The Migration Readiness Assessment
Before migrating, AWS's Migration Readiness Assessment evaluates six dimensions of organisational readiness:
- Business case - Is there a clear, quantified justification?
- Migration planning - Is the scope defined and sequenced?
- People - Does the organisation have the skills and change management capacity?
- Process - Are operational processes ready for cloud operations?
- Platform - Is the landing zone designed and built?
- Security - Is the security model defined?
Gaps in any of these dimensions become risks during migration. Identifying them before migration starts, not during, is the point.
AWS Migration Hub - Centralising the Journey
The Problem
A large migration involves multiple tools, multiple teams, and hundreds of workloads moving in parallel. Without a central view, you have no idea what is in progress, what has completed, what has failed, and what is blocked. You are running a complex programme blind.
What It Is
AWS Migration Hub is the central tracking and coordination service for cloud migrations. It aggregates status from all migration tools, MGN, DMS, Server Migration Service, and presents a unified view of every application's migration status.
It does not do the migrating. It tracks and coordinates the tools that do.
What It Provides
Application grouping - You define logical applications by grouping the servers that make up each application. Migration Hub tracks at the application level, not just the server level.
Status tracking - For each application, Migration Hub shows which phase it is in: not started, in progress, completed, or requiring attention.
Integration with AWS services - Migration Hub integrates with MGN, DMS, AWS Application Discovery Service, and third-party migration tools.
Migration Hub Refactor Spaces - An environment for incrementally refactoring applications. It creates a managed network that routes traffic between the old monolith and new microservices as you gradually move functionality over.
Migration Hub Orchestrator - Automates migration workflows. Instead of manually triggering each migration step in sequence, Orchestrator runs pre-defined playbooks that execute the steps in the correct order, with dependencies respected.
# Query migration status via CLI
import boto3
hub = boto3.client('mgh', region_name='us-east-1')
# List migration tasks
response = hub.list_migration_tasks(
MaxResults=50
)
for task in response['MigrationTaskSummaryList']:
print(f"Task: {task['MigrationTaskName']}")
print(f"Status: {task['Status']}")
print(f"Progress: {task['ProgressPercent']}%")
print(f"Updated: {task['UpdateDateTime']}")
print("---")
AWS Application Migration Service (MGN) - Lift and Shift at Scale
The Problem
You have decided to rehost 80 servers. The naive approach: build an EC2 instance, install the operating system, install the application, configure everything, test it, cut over. For 80 servers, that is months of manual work.
Even with automation, the challenge is replicating the exact state of a running server, configuration, data, in-flight processes, without extended downtime during cutover.
What It Is
AWS Application Migration Service (MGN) is the primary AWS service for rehost migrations. It continuously replicates your source servers to AWS and allows you to test and cut over with minimal downtime, measured in minutes, not hours.
It replaced the older AWS Server Migration Service (SMS) and is the recommended tool for all rehost migrations.
How It Works - Step by Step
Step 1 - Install the replication agent
A lightweight agent is installed on each source server. The agent begins replicating the server's disks to AWS, block by block, continuously.
# Linux agent installation
wget -O ./aws-replication-installer-init.py \
https://aws-application-migration-service-us-east-1.s3.us-east-1.amazonaws.com/latest/linux/aws-replication-installer-init.py
sudo python3 aws-replication-installer-init.py \
--region us-east-1 \
--aws-access-key-id YOUR_ACCESS_KEY \
--aws-secret-access-key YOUR_SECRET_KEY
Step 2 - Initial sync
MGN performs an initial full disk replication. For large servers with hundreds of gigabytes of data, this takes time. During this phase, the source server continues operating normally, nothing is disrupted.
Step 3 - Continuous replication
After the initial sync, MGN continuously replicates changes. The target in AWS stays within seconds of the source. This is ongoing delta replication — only changed blocks are sent.
Step 4 - Configure launch settings
You define how the replicated server should launch in AWS: instance type, network settings, security groups, whether to use EBS encryption, IAM instance profile.
{
"launchDisposition": "STOPPED",
"licensing": {
"osByol": false
},
"targetInstanceTypeRightSizingMethod": "BASIC",
"ec2LaunchTemplateID": "lt-0abc1234def56789",
"bootMode": "USE_SOURCE"
}
Step 5 - Test launch
Before cutting over production, you launch a test instance from the replication. This is a non-disruptive test, the source server keeps running, and a test EC2 instance is launched in an isolated network. You validate that the application works in AWS before committing.
Step 6 - Cutover
When you are ready, you trigger the cutover. MGN performs a final sync to capture any remaining changes, then launches the production EC2 instance. Downtime is the time between the final sync and the production instance being ready, typically minutes.
After cutover, you redirect traffic to the new EC2 instance, monitor for issues, and decommission the source server when stable.
What Would Happen Without MGN
Manual migration: build an AMI, configure networking, manually copy data, deal with in-flight transactions during the copy, coordinate a maintenance window, hope nothing was missed. MGN reduces a week of careful manual work per server to a streamlined process that scales across hundreds of servers simultaneously.
Limitations
MGN handles the operating system and application layer. It does not understand your application, it does not know that your database needs special handling, that your application has external dependencies, or that a specific service needs to be started in a specific order after launch. Post-launch validation is your responsibility.
MGN also requires network connectivity between your source environment and AWS. If bandwidth is limited, the initial replication takes longer. For environments with very slow or unreliable connectivity, the Snow Family (section 7) may be a better option for the initial data load.
AWS Database Migration Service (DMS) - Moving Your Data
The Problem
Databases are the hardest part of any migration. They contain your most critical data. They cannot be simply copied, data is changing while you copy it. They often need to change database engines (Oracle to PostgreSQL, for example) as part of the migration. And their downtime window is typically measured in minutes, not hours.
MGN can move the server a database runs on. But for serious database migrations, especially heterogeneous ones involving an engine change, you need a purpose-built service.
What It Is
AWS Database Migration Service (DMS) migrates databases to AWS with minimal downtime. It supports homogeneous migrations (Oracle to Oracle, MySQL to MySQL) and heterogeneous migrations (Oracle to Aurora PostgreSQL, SQL Server to MySQL). It keeps the source database running throughout, replicating changes continuously until you are ready to cut over.
How It Works
DMS uses a replication instance - a managed EC2 instance that DMS provisions and runs in your AWS account. The replication instance connects to both the source and target databases, reads data from the source, transforms it if necessary, and writes it to the target.
Full Load - The initial migration of all existing data. DMS reads all rows from all tables and writes them to the target.
Full Load + Ongoing Replication - After the full load, DMS switches to Change Data Capture (CDC) mode. It reads the source database's transaction log (binary log for MySQL, redo log for Oracle, WAL for PostgreSQL) and applies each change to the target in near real time. The target database stays synchronised with the source until you cut over.
import boto3
dms = boto3.client('dms', region_name='us-east-1')
# Create replication instance
replication_instance = dms.create_replication_instance(
ReplicationInstanceIdentifier='prod-migration-instance',
ReplicationInstanceClass='dms.r5.xlarge',
AllocatedStorage=100,
VpcSecurityGroupIds=['sg-0abc1234'],
ReplicationSubnetGroupIdentifier='my-dms-subnet-group',
MultiAZ=False,
PubliclyAccessible=False
)
# Create source endpoint
source_endpoint = dms.create_endpoint(
EndpointIdentifier='oracle-source',
EndpointType='source',
EngineName='oracle',
Username='migration_user',
Password='migration_password',
ServerName='on-prem-oracle.internal',
Port=1521,
DatabaseName='PRODDB',
OracleSettings={
'UseLogminerReader': True,
'SecurityDbEncryption': 'none'
}
)
# Create target endpoint
target_endpoint = dms.create_endpoint(
EndpointIdentifier='aurora-postgresql-target',
EndpointType='target',
EngineName='aurora-postgresql',
Username='postgres',
Password='target_password',
ServerName='my-aurora-cluster.cluster-abc123.us-east-1.rds.amazonaws.com',
Port=5432,
DatabaseName='proddb'
)
# Create replication task
replication_task = dms.create_replication_task(
ReplicationTaskIdentifier='oracle-to-aurora-migration',
SourceEndpointArn=source_endpoint['Endpoint']['EndpointArn'],
TargetEndpointArn=target_endpoint['Endpoint']['EndpointArn'],
ReplicationInstanceArn=replication_instance['ReplicationInstance']['ReplicationInstanceArn'],
MigrationType='full-load-and-cdc',
TableMappings=json.dumps({
"rules": [
{
"rule-type": "selection",
"rule-id": "1",
"rule-name": "include-all-tables",
"object-locator": {
"schema-name": "PRODSCHEMA",
"table-name": "%"
},
"rule-action": "include"
}
]
}),
ReplicationTaskSettings=json.dumps({
"TargetMetadata": {
"TargetSchema": "public",
"SupportLobs": True,
"FullLobMode": False,
"LobChunkSize": 64
},
"FullLoadSettings": {
"TargetTablePrepMode": "DROP_AND_CREATE",
"CreatePkAfterFullLoad": True,
"StopTaskCachedChangesApplied": False
},
"Logging": {
"EnableLogging": True,
"LogComponents": [
{"Id": "TASK_MANAGER", "Severity": "LOGGER_SEVERITY_DEFAULT"},
{"Id": "SOURCE_UNLOAD", "Severity": "LOGGER_SEVERITY_DEFAULT"},
{"Id": "TARGET_LOAD", "Severity": "LOGGER_SEVERITY_DEFAULT"}
]
}
})
)
Heterogeneous Migration - Schema Conversion
When the source and target use different database engines, the schema must be converted. Table structures, data types, stored procedures, and functions need to be rewritten for the target engine.
AWS Schema Conversion Tool (SCT) automates this. It analyses the source schema and generates the equivalent target schema, flagging any constructs that cannot be automatically converted and require manual attention.
The typical workflow:
- SCT converts the schema automatically where possible.
- Review and resolve any flagged items manually.
- Apply the converted schema to the target database.
- Run DMS for the data migration.
What Would Happen Without DMS
Manual database migration: take a backup, restore the backup to the target, measure how stale the data is, deal with the replication gap during cutover. For large databases, the backup and restore cycle alone takes hours. The downtime window is the time from the last backup to the application pointing at the new database. For production systems, that is often unacceptable.
DMS reduces the cutover window to the time it takes to flush the CDC queue after you stop writes to the source, typically seconds to low minutes.
Limitations
DMS handles data. It does not handle stored procedures, triggers, views, or functions, those require SCT. DMS also requires the source database to have its transaction log enabled and accessible, which some database configurations or hosting environments may restrict.
For very large databases (terabytes), even CDC replication has latency. If the source database has extremely high write throughput, the replication instance may need to be sized up to keep pace.
AWS Snow Family - When the Network Is Not Enough
The Problem
Your data centre has 500 terabytes of data to migrate. Your internet connection to AWS is 1 Gbps. Fully saturating that connection, which you cannot do while running production workloads, would take approximately 45 days. A more realistic 10-20% utilisation means 6-9 months just for the data transfer.
The network is not fast enough. You need a different strategy.
The Physics of the Problem
At 1 Gbps:
- 1 TB takes approximately 2.2 hours at 100% utilisation
- 10 TB takes approximately 22 hours
- 100 TB takes approximately 9 days
- 500 TB takes approximately 45 days
At realistic utilisation (10-20% to avoid impacting production traffic):
- 500 TB takes 6-9 months
Shipping a physical device with the data on it is faster than the network for any dataset over approximately 10 TB with typical internet connections. This is not a workaround. It is physics.
The Snow Family
AWS provides three physical devices for offline data transfer:
AWS Snowcone
The smallest device. Portable and rugged, small enough to fit in a backpack.
- Storage: 8 TB HDD or 14 TB SSD
- Compute: 2 vCPUs, 4 GB RAM (can run EC2 instances on the device)
- Use cases: Edge data collection in remote locations, disconnected environments, small data transfers
- Network: Can transfer data offline (ship the device) or online via DataSync
- Power: Can run on a standard USB-C power bank
Snowcone is for data collection at the edge, oil rigs, military deployments, remote construction sites, not for large data centre migrations.
AWS Snowball Edge
The workhorse of the Snow Family. Multiple variants:
Snowball Edge Storage Optimised:
- 80 TB usable storage
- 40 vCPUs, 80 GB RAM
- 1 Gbps / 10 Gbps / 25 Gbps network interfaces
- Use case: Large-scale data migration, edge storage
Snowball Edge Compute Optimised:
- 28 TB usable NVMe storage
- 52 vCPUs, 208 GB RAM, optional GPU
- Use case: Edge machine learning, local compute in disconnected environments
Snowball Edge Compute Optimised with GPU:
- Same as above with a NVIDIA V100 GPU
- Use case: ML inference at the edge, video processing, scientific computing
For a 500 TB migration, you order 7 Snowball Edge Storage Optimised devices, load them in parallel, and ship them to AWS. AWS ingests the data and it appears in your S3 bucket.
AWS Snowmobile
A 45-foot shipping container on a truck. A literal data centre on wheels.
- Storage: Up to 100 PB per Snowmobile
- Use case: Exabyte-scale migrations. Moving an entire data centre.
- Process: AWS drives the Snowmobile to your data centre, you connect it to your internal network, transfer data at up to 1 Tbps, AWS drives it back.
Snowmobile is for organisations with 10+ petabytes to transfer. It is not an API call, it is a logistics project involving AWS personnel and physical security.
The Migration Process (Snowball Edge)
# Step 1: Order device via AWS Console or CLI
aws snowball create-job \
--job-type IMPORT \
--resources '{"S3Resources":[{"BucketArn":"arn:aws:s3:::my-migration-bucket"}]}' \
--description "Data centre migration batch 1" \
--address-id ADID1234567890EXAMPLE \
--kms-key-arn arn:aws:kms:us-east-1:123456789012:key/abc123 \
--role-arn arn:aws:iam::123456789012:role/SnowballRole \
--snowball-capacity-preference T80 \
--shipping-option SECOND_DAY \
--snowball-type EDGE_STORAGE_OPTIMIZED
# Step 2: Receive device, unlock it with your credentials
# The Snowball Client is used to unlock the device
# Step 3: Copy data to the device
# Using the Snowball Edge client
snowballEdge start-service --service-id s3 \
--uriEndpoints https://192.168.1.101:8080
# Use S3 commands targeting the device's local endpoint
aws s3 cp /data/large-dataset/ s3://snowball-bucket/ \
--recursive \
--endpoint https://192.168.1.101:8080
# Step 4: Ship the device back to AWS
# AWS ingests the data into your specified S3 bucket
# Step 5: Verify data in S3
aws s3 ls s3://my-migration-bucket/ --recursive --human-readable
What Would Happen Without Snow
For large datasets, you wait months for online transfer. Production bandwidth is saturated. The migration timeline extends to an organisational timescale problem. Snow compresses a multi-month online transfer into days of logistics.
Choosing the Right Device
| Data Volume | Network Speed | Recommended |
|---|---|---|
| < 10 TB | Any | Online transfer (DataSync) |
| 10 TB – 80 TB | < 1 Gbps | Snowball Edge |
| 10 TB – 80 TB | > 1 Gbps | Online transfer or Snowball |
| 80 TB – 500 TB | Any | Multiple Snowball Edge devices |
| > 500 TB – 10 PB | Any | Multiple Snowball Edge or Snowmobile |
| > 10 PB | Any | Snowmobile |
AWS DataSync - Continuous Online Data Transfer
The Problem
Your file servers contain 50 TB of data. You want to migrate it to Amazon EFS or S3, and then keep the source and destination synchronised during the transition period, so you can cut over at any point without data loss.
Writing scripts to do this reliably, handling retries, tracking what has changed, verifying data integrity, throttling to not saturate your network — is significant engineering effort.
What It Is
AWS DataSync is a managed data transfer service that automates moving data between on-premises storage and AWS storage services (S3, EFS, FSx), or between AWS storage services.
It handles the engineering complexity: multi-threaded transfers for maximum throughput, automatic retry on failure, data integrity verification, bandwidth throttling, and scheduling.
What It Transfers
- On-premises NAS/SAN to Amazon S3
- On-premises NAS/SAN to Amazon EFS
- On-premises NAS/SAN to Amazon FSx (for Windows, Lustre, NetApp ONTAP)
- Amazon S3 to Amazon EFS (or vice versa)
- Between S3 buckets in different regions
How It Works
You deploy a DataSync agent, a virtual appliance, in your on-premises environment. The agent connects to your NAS/SAN and to AWS. You create a task that defines the source, destination, and transfer settings.
import boto3
datasync = boto3.client('datasync', region_name='us-east-1')
# Create source location (NFS on-premises)
source_location = datasync.create_location_nfs(
ServerHostname='nas.internal.company.com',
Subdirectory='/data/files',
OnPremConfig={
'AgentArns': [
'arn:aws:datasync:us-east-1:123456789012:agent/agent-0abc1234'
]
},
MountOptions={
'Version': 'NFS4_1'
}
)
# Create destination location (S3)
destination_location = datasync.create_location_s3(
S3BucketArn='arn:aws:s3:::my-migration-bucket',
S3StorageClass='STANDARD',
S3Config={
'BucketAccessRoleArn': 'arn:aws:iam::123456789012:role/DataSyncS3Role'
},
Subdirectory='/migrated-files/'
)
# Create transfer task
task = datasync.create_task(
SourceLocationArn=source_location['LocationArn'],
DestinationLocationArn=destination_location['LocationArn'],
Name='NAS-to-S3-migration',
Options={
'VerifyMode': 'ONLY_FILES_TRANSFERRED',
'OverwriteMode': 'ALWAYS',
'Atime': 'BEST_EFFORT',
'Mtime': 'PRESERVE',
'Uid': 'NONE',
'Gid': 'NONE',
'PreserveDeletedFiles': 'PRESERVE',
'PreserveDevices': 'NONE',
'PosixPermissions': 'NONE',
'BytesPerSecond': 104857600, # 100 MB/s bandwidth cap
'TaskQueueing': 'ENABLED',
'LogLevel': 'TRANSFER'
},
Schedule={
'ScheduleExpression': 'cron(0 */6 * * ? *)' # Every 6 hours
},
CloudWatchLogGroupArn='arn:aws:logs:us-east-1:123456789012:log-group:/datasync/tasks'
)
# Start a task execution
execution = datasync.start_task_execution(
TaskArn=task['TaskArn']
)
DataSync vs Snow vs Direct Transfer
| Scenario | Use |
|---|---|
| Ongoing sync, data < 10 TB | DataSync |
| One-time migration, data < 10 TB, good bandwidth | DataSync |
| One-time migration, data > 10 TB, poor bandwidth | Snow Family |
| One-time migration, data > 10 TB, excellent bandwidth | DataSync or Snow |
| Initial load + ongoing sync | Snow for initial, DataSync for ongoing |
A common pattern: use Snowball Edge for the initial bulk transfer of a large dataset, then switch to DataSync for the ongoing incremental synchronisation until cutover.
AWS Transfer Family - Managed File Transfer Protocols
The Problem
Your trading partners send you files via SFTP. Your legacy systems use FTP. Your partners expect to connect to a server at a fixed hostname and authenticate with credentials they have used for years. You are migrating to S3 as your storage layer. But you cannot ask all your partners to change their file transfer workflows, some of them are large organisations with change management processes that take months.
You need to accept SFTP/FTP/FTPS connections as before, but store the files in S3 instead of a local filesystem.
What It Is
AWS Transfer Family provides managed file transfer endpoints that support SFTP, FTP, FTPS, and AS2 protocols. Files transferred through these endpoints are stored in Amazon S3 or Amazon EFS. Your trading partners' workflows do not change, they connect to the same protocol, authenticate the same way, and transfer files. The difference is invisible to them.
import boto3
transfer = boto3.client('transfer', region_name='us-east-1')
# Create an SFTP server backed by S3
server = transfer.create_server(
Protocols=['SFTP'],
IdentityProviderType='SERVICE_MANAGED',
EndpointType='PUBLIC',
LoggingRole='arn:aws:iam::123456789012:role/TransferLoggingRole',
SecurityPolicyName='TransferSecurityPolicy-2022-03',
Tags=[
{'Key': 'Environment', 'Value': 'production'},
{'Key': 'Project', 'Value': 'file-migration'}
]
)
SERVER_ID = server['ServerId']
print(f"SFTP endpoint: {SERVER_ID}.server.transfer.us-east-1.amazonaws.com")
# Create a user
user = transfer.create_user(
ServerId=SERVER_ID,
UserName='trading-partner-acme',
Role='arn:aws:iam::123456789012:role/TransferUserRole',
HomeDirectory=f'/my-transfer-bucket/acme-files',
HomeDirectoryType='PATH',
SshPublicKeyBody='ssh-rsa AAAA...your-partner-public-key'
)
# The partner connects to the SFTP endpoint and sees their home directory
# Files they upload go directly to S3
Transfer Family eliminates the need to run and maintain your own SFTP server, handle key management, manage storage, and monitor availability. AWS manages all of that. You manage the users and the S3 bucket policies.
VMware Cloud on AWS - The Hybrid Bridge
The Problem
You have 2,000 VMware VMs. Your operations team knows VMware, vSphere, vCenter, NSX, vSAN. A full migration to EC2 would require rebuilding every workload, retraining your team, and changing every operational process. The risk and timescale are too large.
But you still want cloud economics, pay for what you use, elastic capacity, reduce physical data centre footprint.
What It Is
VMware Cloud on AWS (VMC on AWS) is a jointly developed service that runs VMware's SDDC (Software-Defined Data Centre) stack on bare-metal AWS infrastructure. You get VMware vSphere, vSAN, NSX, and HCX running on AWS hardware in AWS data centres.
From your team's perspective, nothing changes. They use the same vCenter console, the same tools, the same operational processes. The VMs behave exactly as they do on-premises.
From an infrastructure perspective, you have cloud economics. You pay for the bare-metal hosts you use. You can add hosts in hours instead of months. You do not manage the physical infrastructure.
The Migration Path
Phase 1 - Extend: Connect your on-premises VMware environment to VMC on AWS via a stretched network (HCX). VMs can move between on-premises and cloud without changing IP addresses, a concept called "layer 2 extension."
Phase 2 - Migrate: Use VMware HCX to live-migrate VMs from on-premises to VMC on AWS. Zero downtime. The VM keeps running throughout the migration.
Phase 3 - Optimise: Over time, refactor workloads from VMC on AWS to native AWS services (EC2, RDS, ECS) where the economics or capability justify it. VMC becomes a stepping stone, not a destination.
When VMware Cloud Makes Sense
- Organisations with a large VMware investment and no appetite for a full re-architecture
- Data centre exit deadlines where there is no time for workload refactoring
- Workloads that have genuine dependency on VMware capabilities (vSphere HA, DRS, vSAN)
- Regulated industries where change risk must be minimised
It is not the cheapest cloud strategy, VMC on AWS costs more per workload than running on native EC2. The value is in the migration speed and risk reduction, not the long-term economics.
Migration Execution - The Three-Phase Model
The Framework
Regardless of the scale or complexity of the migration, execution follows three phases. Getting these phases right is what separates a migration that completes on time and on budget from one that runs over on both.
Phase 1 - Foundation (The Landing Zone)
Before migrating a single workload, the AWS environment must be ready to receive it. The landing zone is the foundational AWS architecture, the networking, identity, security, and governance structure, that all migrated workloads will run inside.
What the landing zone includes:
Account structure - A multi-account AWS Organisation with accounts for different environments (development, staging, production) and different business units. AWS Control Tower automates the setup and governance of a multi-account structure.
Networking - A hub-and-spoke VPC architecture. The hub (Transit Gateway) connects to the on-premises environment via Direct Connect or VPN. Spoke VPCs for each application or environment connect to the hub.
Identity - IAM Identity Centre (formerly SSO) configured to federate with your corporate identity provider. No IAM users — only federated access.
Security - AWS Config rules enforcing baseline security controls. CloudTrail logging enabled in all accounts. GuardDuty active in all accounts. Security Hub aggregating findings.
Connectivity - AWS Direct Connect (dedicated private network connection to AWS) or Site-to-Site VPN (encrypted connection over the internet). During migration, the on-premises environment and AWS must be connected.
# Control Tower landing zone - simplified account vending via Service Catalog
# Account Factory configuration
accountEmail: "${AccountName}@company.com"
accountName: "${AccountName}"
managedOrganizationalUnit: "Workloads/Production"
ssoUserEmail: "${AccountOwnerEmail}"
ssoUserFirstName: "${FirstName}"
ssoUserLastName: "${LastName}"
Phase 2 - Migration Waves
With the landing zone in place, migration proceeds in waves. A wave is a group of applications migrated together during a defined period.
Wave sequencing principles:
- Start with low-risk workloads. Migrate simple, low-criticality applications first. The team learns the process. Issues surface in a low-stakes environment.
- Respect dependencies. If application A depends on database B, database B moves before or simultaneously with application A.
- Group related applications. Applications that communicate heavily move in the same wave to avoid cross-environment latency during the transition period.
- Increase complexity over time. Early waves build skill and confidence. More complex workloads move later when the team is experienced.
A typical wave structure for a 200-application migration:
| Wave | Applications | Strategy | Duration |
|---|---|---|---|
| Wave 0 | 5-10 low-risk, simple apps | Rehost | 2-4 weeks |
| Wave 1 | 20-30 medium complexity | Rehost / Replatform | 4-6 weeks |
| Wave 2 | 30-40 applications | Rehost / Replatform | 6-8 weeks |
| Wave 3+ | Remaining applications | All strategies | Ongoing |
| Final | Complex / critical apps | Refactor | 3-6 months |
Phase 3 - Optimise
After migration, the work is not finished. It is beginning. The first cloud bill often surprises organisations, cloud costs are visible in a way that buried data centre costs were not.
Optimisation is where the real financial benefit materialises:
Right-sizing - Match instance sizes to actual utilisation. Compute Optimiser analyses CloudWatch metrics and recommends appropriately sized instances. An application that was running on an on-premises server with 32 cores and 256 GB RAM may run perfectly well on an EC2 instance with 4 vCPUs and 16 GB RAM.
Savings Plans and Reserved Instances - Commit to a baseline of compute usage for 1 or 3 years in exchange for 30-72% discounts. Buy Savings Plans for baseline usage, run on-demand for variable peaks.
Spot Instances - For fault-tolerant, interruptible workloads (batch processing, CI/CD, ML training), Spot Instances cost 60-90% less than on-demand.
Storage tiering - S3 Intelligent-Tiering automatically moves objects between storage classes based on access patterns. EBS gp2 volumes migrated from on-premises should be reviewed, many can be right-sized or converted to gp3 at lower cost.
Serverless where appropriate - Identify workloads running on EC2 that could move to Lambda. If a service processes requests intermittently, a Lambda function costs nothing when idle. An EC2 instance costs money 24/7.
Post-Migration Optimisation - The Work That Actually Delivers Value
The Uncomfortable Truth
A rehosted application in the cloud is not cheaper than the same application on-premises. It is more expensive if you do nothing after the migration. EC2 on-demand pricing is designed for flexibility, not economy. The savings come from the combination of right-sizing, commitment discounts, managed service adoption, and architectural optimisation.
Organisations that migrate and stop there are paying cloud prices for on-premises architecture. Organisations that migrate and then optimise are paying cloud prices for cloud architecture, and that is a fundamentally different number.
The Optimisation Stack
Level 1 - Right-size. Match what you pay for to what you use. This is the first thing to do after every migration wave.
Level 2 - Commit. Buy Savings Plans for the baseline usage you know you will sustain. This alone saves 30-50% on compute costs.
Level 3 - Replace managed services. Every self-managed component, every EC2 instance running a database, message queue, or cache, is a candidate for replacement with a managed service. Each replacement reduces operational overhead and often costs less at scale.
Level 4 - Architect for elasticity. Auto Scaling groups. Spot Instances for batch workloads. Lambda for event-driven processing. This is where the elastic cost model of cloud, paying only for what you use, when you use it, actually materialises.
Level 5 - Re-architect strategically. The highest-value applications justify genuine re-architecture. Decompose monoliths into microservices. Adopt event-driven patterns. Move to purpose-built databases.
The difference between an organisation that moved to the cloud and an organisation that transformed its technology through cloud is almost entirely in how far they pursue this stack.
Vendor-Neutral: Designing for Multi-Cloud
The Reality of Multi-Cloud
Most organisations that say they are "multi-cloud" mean one of three things:
- Different business units independently chose different cloud providers.
- They use SaaS products that happen to run on different clouds.
- They have a deliberate strategy to run workloads across multiple clouds for resilience, cost optimisation, or avoiding vendor lock-in.
Only the third is genuinely architectural multi-cloud. This section is about that third case.
Why Organisations Choose Multi-Cloud
Resilience - A cloud provider outage is rare but real. A region-wide or global incident at a single provider can take down every workload. Running critical workloads across two providers eliminates that single point of failure.
Avoiding vendor lock-in - Dependence on a single cloud provider's proprietary services creates switching costs that compound over time. A multi-cloud strategy limits that dependence.
Best-of-breed services - Some services are genuinely better at specific things. Google BigQuery for analytics. AWS for breadth of services and global reach. Azure for Microsoft workload integration. A multi-cloud strategy can use the best service for each job.
Regulatory requirements - Some jurisdictions or regulated industries require workloads to run on multiple independent infrastructure providers.
Negotiation leverage - Credible multi-cloud capability gives an organisation genuine negotiating leverage with cloud providers on pricing and contract terms.
The Real Cost of Multi-Cloud
Multi-cloud is not free. It has costs that are easy to underestimate:
Operational complexity - Every additional cloud provider means additional tooling, additional training, additional operational processes, additional monitoring. Each cloud has its own IAM model, its own networking constructs, its own deployment tools.
Data transfer costs - Moving data between cloud providers is not free. Egress costs are real and compound at scale. An architecture that requires frequent data movement between clouds is architecturally expensive.
Lowest common denominator - If you design for portability across clouds, you avoid proprietary services on any cloud. That means avoiding some of the most valuable services on each platform.
Tooling fragmentation - Separate consoles, separate CLIs, separate SDKs, separate billing, unless you invest in abstraction tooling.
Architecture Principles for Genuine Multi-Cloud
Abstract at the application layer, not the infrastructure layer. The goal is not to run identical infrastructure on two clouds. It is to design your application so it can run on either cloud with configuration changes, not code changes.
Standardise on open standards. Use Kubernetes instead of ECS or GKE's proprietary constructs. Use PostgreSQL instead of Aurora-specific features where possible. Use OIDC for authentication instead of cloud-specific identity. Open standards reduce the cost of moving between providers.
Containerise everything. A containerised application that runs on Kubernetes is portable by design. EKS on AWS, GKE on Google Cloud, AKS on Azure — the Kubernetes API is the same. Your application deployments do not change.
Use infrastructure as code that supports multiple providers. Terraform is cloud-agnostic. A Terraform module can be written to deploy to AWS or GCP with a provider change. CloudFormation is AWS-specific — it does not contribute to multi-cloud portability.
Separate stateless and stateful tiers. Stateless compute is easy to move between clouds. Stateful data is expensive to move. Design your data tier to be the stable layer and your compute tier to be the portable layer.
Use a cloud-agnostic networking layer. Tools like Consul, Cilium, and Istio provide service discovery and networking that works across clouds. AWS-specific networking (VPC, Transit Gateway, PrivateLink) does not.
Multi-Cloud Tooling
| Layer | AWS-Native | Multi-Cloud Alternative |
|---|---|---|
| Infrastructure as Code | CloudFormation | Terraform, Pulumi |
| Container Orchestration | ECS | Kubernetes (EKS/GKE/AKS) |
| Observability | CloudWatch | Datadog, Grafana, Prometheus |
| Secrets Management | AWS Secrets Manager | HashiCorp Vault |
| Service Mesh | App Mesh | Istio, Linkerd |
| CI/CD | CodePipeline | GitHub Actions, Tekton, ArgoCD |
| Identity | IAM + Identity Center | Okta, Auth0, Keycloak |
For most organisations, a well-designed single-cloud architecture with proper resilience (multi-region, multi-AZ) is better than a poorly designed multi-cloud architecture. The operational complexity of multi-cloud is real. The benefits require discipline and investment to realise.
The right question is not "should we be multi-cloud?" The right question is "what specific risks or requirements does multi-cloud solve for us, and is the cost of solving them this way justified?"
If the answer involves genuine resilience requirements, regulatory mandates, or strategic risk management, multi-cloud is the right answer. Build it deliberately, with open standards, container portability, and cloud-agnostic tooling. If the answer is "we do not want to be locked in" without a specific failure mode in mind, invest that energy in optimising your primary cloud first.
Written by Onyedikachi Obidiegwu | Cloud Security Engineer
Top comments (0)