DEV Community

Rajeev
Rajeev

Posted on

We Ditched MongoDB Atlas for AWS DocumentDB — Here's Exactly How We Did It

#mongodb #aws #devops #database

We Ditched MongoDB Atlas for AWS DocumentDB — Here's Exactly How We Did It

One connection string change. Zero downtime. One very long night. Here's the full story.

I want to tell you something most engineering blogs won't.

Migrations are messy. They go sideways at 11pm. They reveal assumptions you didn't know you had. And the "5-minute fix" always takes 3 hours.

This is the real story of how we migrated our production MongoDB database from Atlas to AWS DocumentDB — every command, every mistake, every fix, and every lesson learned the hard way.

If you're planning a similar migration, read this before you start. It will save you hours.

Why We Even Did This

Our stack was already 100% on AWS:

  • ECS Fargate running our Node.js backend
  • EC2 for our app server
  • RDS PostgreSQL for relational data
  • S3 for file storage
  • ALB for load balancing

And then there was MongoDB Atlas — sitting outside AWS, on MongoDB's cloud, billed separately, with data crossing the public internet every single request.

The CTO asked a simple question: "Why are we paying Atlas when we're already on AWS?"

That question started this migration.

The goals were clear:

  • Consolidate to one cloud provider
  • Cut cross-cloud data transfer costs
  • Put the database inside our VPC (zero public internet exposure)
  • One bill, one security boundary, one team managing everything

What We Were Working With

Before writing a single command, we mapped the full picture:

Source — MongoDB Atlas:

  • ~32 collections
  • ~2,400 documents (and growing daily)
  • Used by 2 ECS services (prod + staging) and an EC2 backend
  • Publicly accessible via mongodb+srv://

Target — Amazon DocumentDB:

  • MongoDB 5.0 compatible
  • Lives inside a private VPC (no public endpoint — this matters, a lot)
  • Managed by AWS (auto backups, failover, CloudWatch)
  • Tight integration with IAM, VPC Security Groups, KMS

The architecture we were building toward:

┌──────────────────────────────────────────────────┐
│                  AWS VPC (private)               │
│                                                  │
│  ┌─────────────────┐      ┌──────────────────┐  │
│  │  ECS Fargate    │      │  EC2 App Server  │  │
│  │  prod + staging │      │  Public IP       │  │
│  └────────┬────────┘      └────────┬─────────┘  │
│           │                        │             │
│           └──────────┬─────────────┘             │
│                      │ private network            │
│           ┌──────────▼──────────┐               │
│           │   Amazon DocumentDB  │               │
│           │   port 27017        │               │
│           │   encrypted at rest │               │
│           └─────────────────────┘               │
└──────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Clean. Secure. Everything private. No data crossing the public internet.

Choosing the Migration Strategy

We had three options:

Strategy Complexity Downtime Best For
AWS DMS (live CDC) High Zero Large DBs, can't afford downtime
mongodump + mongorestore Low Minutes Small-medium DBs
Custom change streams script Very High Zero Custom transformations

We chose mongodump + mongorestore. Here's why:

Our database was under 1GB. Our staging environment would absorb the testing. And mongodump/mongorestore is the most battle-tested MongoDB migration tool in existence — boring is good when it comes to production data.

Phase 1: Infrastructure Setup (The Foundation)

Step 1 — DocumentDB Subnet Group

DocumentDB needs subnets across at least 2 availability zones for redundancy. We used our existing private subnets:

aws docdb create-db-subnet-group \
  --db-subnet-group-name myapp-docdb-subnet-group \
  --db-subnet-group-description "Subnet group for DocumentDB" \
  --subnet-ids subnet-xxxxxxxx subnet-yyyyyyyy \
  --region ap-south-1
Enter fullscreen mode Exit fullscreen mode

Step 2 — Security Group (Port 27017, VPC Only)

Never open 27017 to the internet. VPC CIDR only:

# Create security group
aws ec2 create-security-group \
  --group-name myapp-docdb-sg \
  --description "DocumentDB security group" \
  --vpc-id vpc-xxxxxxxxxx

# Allow port 27017 from VPC only
aws ec2 authorize-security-group-ingress \
  --group-id sg-xxxxxxxxxx \
  --protocol tcp \
  --port 27017 \
  --cidr 10.0.0.0/16

# Also allow from your ECS security group
aws ec2 authorize-security-group-ingress \
  --group-id sg-xxxxxxxxxx \
  --protocol tcp \
  --port 27017 \
  --source-group sg-your-ecs-sg
Enter fullscreen mode Exit fullscreen mode

Step 3 — Create the Cluster + Instance 

# Create cluster (encrypted at rest by default)
aws docdb create-db-cluster \
  --db-cluster-identifier myapp-docdb \
  --engine docdb \
  --master-username admin \
  --master-user-password "YourStrongPassword123!" \
  --db-subnet-group-name myapp-docdb-subnet-group \
  --vpc-security-group-ids sg-xxxxxxxxxx \
  --storage-encrypted \
  --region ap-south-1

# Add an instance to the cluster
aws docdb create-db-instance \
  --db-instance-identifier myapp-docdb-instance-1 \
  --db-cluster-identifier myapp-docdb \
  --db-instance-class db.t3.medium \
  --engine docdb
Enter fullscreen mode Exit fullscreen mode

⏱️ Wait 8-10 minutes for the cluster to become available. Grab a coffee.

Phase 2: The Migration — Where It Gets Interesting

The Problem Nobody Warns You About

Here's what I wish someone had told me before I started:

DocumentDB has no public endpoint.

It lives inside your VPC. Period. You cannot run mongorestore from your laptop to DocumentDB. You need a machine inside the VPC to act as a bridge.

The architecture looks like this:

Your Laptop
    │
    │ 1. mongodump ──► reads Atlas
    ▼
/tmp/dump/ (BSON files on your machine)
    │
    │ 2. scp ──► copies files to EC2
    ▼
EC2 Instance (inside VPC) ◄── this is your bridge
    │
    │ 3. mongorestore ──► writes to DocumentDB
    ▼
Amazon DocumentDB (private, inside VPC) ✅
Enter fullscreen mode Exit fullscreen mode

We used our existing EC2 app server as the bridge. No new infrastructure needed.

Step 4 — Dump Everything from Atlas 

# Full database dump
mongodump \
  --uri="mongodb+srv://user:password@cluster.mongodb.net/mydb" \
  --out=/tmp/myapp_backup

# Verify the dump
ls -lh /tmp/myapp_backup/mydb/
# You should see .bson + .metadata.json for every collection
Enter fullscreen mode Exit fullscreen mode

Step 5 — Transfer to EC2 

# Copy all dump files to EC2
scp -i your-keypair.pem -r \
  /tmp/myapp_backup/ \
  ubuntu@<ec2-public-ip>:/tmp/myapp_backup/

# Verify files arrived
ssh -i your-keypair.pem ubuntu@<ec2-ip> "ls /tmp/myapp_backup/mydb/ | wc -l"
Enter fullscreen mode Exit fullscreen mode

Step 6 — Download TLS Certificate on EC2

DocumentDB requires TLS. The cert needs to be on the machine running mongorestore:

ssh -i your-keypair.pem ubuntu@<ec2-ip>

# Inside EC2:
wget https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
Enter fullscreen mode Exit fullscreen mode

Step 7 — Restore to DocumentDB 

# Run this inside EC2
mongorestore \
  --uri="mongodb://admin:password@myapp-docdb.cluster-xxxxx.region.docdb.amazonaws.com:27017/mydb?tls=true&tlsCAFile=global-bundle.pem&replicaSet=rs0&retryWrites=false" \
  --db=mydb \
  --dir=/tmp/myapp_backup/mydb \
  --drop
Enter fullscreen mode Exit fullscreen mode

The --drop flag is crucial — it drops each collection before restoring. This makes the command idempotent (safe to re-run if something goes wrong).

Step 8 — Verify Every Collection Count

Never skip this step.

# Check Atlas counts
mongosh "atlas-connection-string" --quiet --eval \
  "db.getCollectionNames().forEach(c => print(db[c].countDocuments() + '\t' + c))" | sort -rn

# Check DocumentDB counts (from EC2)
mongosh "docdb-connection-string" --quiet --eval \
  "db.getCollectionNames().forEach(c => print(db[c].countDocuments() + '\t' + c))" | sort -rn
Enter fullscreen mode Exit fullscreen mode

Every single number must match. If one doesn't — fix it before moving on.

Phase 3: The Connection String — The Devil Is In The Details

This is where most migrations break silently.

Atlas string:

mongodb+srv://user:pass@cluster.mongodb.net/mydb?retryWrites=true&w=majority
Enter fullscreen mode Exit fullscreen mode

DocumentDB string:

mongodb://user:pass@myapp-docdb.cluster-xxxxx.docdb.amazonaws.com:27017/mydb?tls=true&tlsAllowInvalidCertificates=true&replicaSet=rs0&retryWrites=false&authMechanism=SCRAM-SHA-1
Enter fullscreen mode Exit fullscreen mode

The critical differences:

Parameter Atlas DocumentDB Why
Protocol mongodb+srv mongodb:// SRV not supported
retryWrites true false DocumentDB doesn't support it
replicaSet not needed rs0 Required for DocumentDB
Port auto (SRV) 27017 Must be explicit
TLS cert automatic manual global-bundle.pem needed

Miss any of these and your app either crashes silently or throws cryptic errors.

Phase 4: Application Switchover

Always Staging First. Always.

We updated the ECS task definition environment variable:

# Register new task definition revision with updated MONGODB_URI
aws ecs register-task-definition \
  --family myapp-backend \
  --container-definitions '[{
    "name": "backend",
    "environment": [
      {
        "name": "MONGODB_URI",
        "value": "mongodb://user:pass@docdb-endpoint:27017/mydb?tls=true&tlsAllowInvalidCertificates=true&replicaSet=rs0&retryWrites=false&authMechanism=SCRAM-SHA-1&maxPoolSize=10&minPoolSize=2&connectTimeoutMS=5000&socketTimeoutMS=30000&serverSelectionTimeoutMS=5000"
      }
    ]
  }]'

# Deploy to staging ONLY first
aws ecs update-service \
  --cluster myapp-cluster \
  --service myapp-staging \
  --task-definition myapp-backend:NEW_REVISION \
  --force-new-deployment
Enter fullscreen mode Exit fullscreen mode

Watch the logs until you see:

🚀 Server running on port 5000
MongoDB connected successfully ✅
Enter fullscreen mode Exit fullscreen mode

Phase 5: The 504 Nightmare

After switching staging to DocumentDB — login started taking 3-5 seconds. Users were getting 504 Gateway Timeout errors.

This one hurt.

The root cause: DocumentDB connection latency is higher than Atlas. Without connection pool configuration, every request was creating a new DB connection from scratch. Cold connections on DocumentDB take 3-5 seconds.

The fix was adding connection pool parameters to the URI:

&maxPoolSize=10
&minPoolSize=2
&connectTimeoutMS=5000
&socketTimeoutMS=30000
&serverSelectionTimeoutMS=5000
&heartbeatFrequencyMS=10000
Enter fullscreen mode Exit fullscreen mode

Full optimized string:

mongodb://user:pass@docdb-endpoint:27017/mydb?tls=true&tlsAllowInvalidCertificates=true&replicaSet=rs0&retryWrites=false&authMechanism=SCRAM-SHA-1&maxPoolSize=10&minPoolSize=2&connectTimeoutMS=5000&socketTimeoutMS=30000&serverSelectionTimeoutMS=5000&heartbeatFrequencyMS=10000
Enter fullscreen mode Exit fullscreen mode

Result: Login went from 3-5 seconds → under 500ms. ✅

minPoolSize=2 is the key — it keeps 2 warm connections alive at all times, so the first request never has to wait for a cold connection.

Phase 6: The Re-sync Problem

We migrated on Day 1. We deployed to staging on Day 3. We did a final check on Day 16.

Atlas had 157 new documents that DocumentDB didn't.

This is normal and expected. Any live database will accumulate new data between your migration and your cutover date. The fix is a re-sync run right before prod switchover:

# Step 1: Fresh dump from Atlas
mongodump --uri="atlas-uri" --out=/tmp/resync

# Step 2: Transfer to EC2
scp -r /tmp/resync/ ubuntu@ec2-ip:/tmp/

# Step 3: Restore with --drop (fully safe, idempotent)
mongorestore \
  --uri="docdb-uri" \
  --dir=/tmp/resync/mydb \
  --drop

# Step 4: Verify counts again
# Atlas count == DocumentDB count? ✅ Good to go.
Enter fullscreen mode Exit fullscreen mode

Run this re-sync as close to prod cutover as possible — ideally within hours.

What's Different in DocumentDB vs Atlas

Before you migrate — test these in your app:

Feature Atlas DocumentDB
Basic CRUD ✅ Full ✅ Full
Mongoose ODM ✅ Full ✅ Full
$lookup aggregation ✅ Full ⚠️ Limited
Transactions ✅ Full ACID ⚠️ Limited
Change streams ✅ Full ⚠️ Limited
Atlas Search ✅ Yes ❌ No
retryWrites=true ✅ Yes ❌ Must be false
Text search ✅ Yes ⚠️ Limited

If your app uses basic Mongoose CRUD — you're fine. If you rely heavily on complex aggregations, transactions, or Atlas Search — test everything on staging before touching prod.

Developer Access After Migration

This confused our dev team at first.

Atlas: Anyone with the connection string can connect from anywhere.
DocumentDB: Private VPC. Can't connect from outside without a tunnel.

Dev Laptop ──► SSH Tunnel ──► EC2 ──► DocumentDB
Enter fullscreen mode Exit fullscreen mode

SSH tunnel command:

# Keep this running in a terminal
ssh -i your-keypair.pem -N \
  -L 27017:docdb-endpoint:27017 \
  ubuntu@ec2-public-ip
Enter fullscreen mode Exit fullscreen mode

Then connect to 127.0.0.1:27017 from your app or MongoDB Compass.

Our recommendation: Keep Atlas for local development. Use DocumentDB for prod/staging only. Zero friction, cleanest separation.

The Rollback Plan (Non-Negotiable)

Before every prod migration, write your rollback plan:

If DocumentDB fails:
1. Update ECS task def → revert MONGODB_URI to Atlas string
2. aws ecs update-service --force-new-deployment
3. Back on Atlas in under 5 minutes

Rule: Never delete Atlas until DocumentDB runs in prod for 30+ days.
Enter fullscreen mode Exit fullscreen mode

We kept Atlas running as a safety net the entire time. This is not optional.

The Numbers

Metric Before (Atlas) After (DocumentDB)
Cross-cloud data transfer Charged per GB Free (same VPC)
Public internet exposure ✅ DB accessible publicly ❌ DB fully private
Security surface Atlas IP whitelist AWS Security Groups + VPC
Monitoring Atlas UI CloudWatch + Alerts
Backup Atlas managed AWS automated + S3 cron
Login response time (after pool fix) ~200ms ~450ms

The Complete Migration Checklist 

INFRASTRUCTURE
☐ DocumentDB subnet group (private subnets, 2+ AZs)
☐ Security group (port 27017, VPC only)
☐ Allow ECS + EC2 security groups in DocDB SG
☐ DocumentDB cluster (storage encrypted)
☐ DocumentDB instance (right size for workload)

MIGRATION
☐ mongodump full backup from Atlas
☐ Verify dump files exist for all collections
☐ scp files to EC2
☐ Download global-bundle.pem on EC2
☐ mongorestore with --drop flag
☐ Verify doc counts match on EVERY collection

APPLICATION
☐ New MONGODB_URI with all required params
☐ Add connection pool params to URI
☐ Deploy to staging first
☐ Watch logs for "MongoDB connected successfully"
☐ Test all critical API endpoints on staging
☐ Monitor for 24h on staging before touching prod
☐ Re-sync Atlas → DocumentDB right before prod cutover
☐ Deploy to prod
☐ Monitor prod logs and CloudWatch

POST-MIGRATION
☐ Set up daily mongodump → S3 backup cron
☐ Configure CloudWatch alarms (connections, CPU, latency)
☐ Keep Atlas alive for 30 days minimum
☐ Delete Atlas cluster only after team sign-off
Enter fullscreen mode Exit fullscreen mode

Key Takeaways

1. The bridge EC2 pattern is your best friend.
DocumentDB's private-only design is a feature. Use your existing EC2 as the migration bridge.

2. Connection pool tuning is not optional.
minPoolSize=2 alone will save you from 504 nightmares. Add it from day one.

3. Always staging first.
We caught the 504 issue on staging before it ever touched prod. This is why staging exists.

4. Plan for re-sync.
Your live database never stops growing. Always re-sync within hours of prod cutover, not days.

5. Keep Atlas longer than you think you need to.
The cost of one extra month of Atlas is nothing compared to the cost of a production incident with no rollback.

6. Your app code changes nothing.
With Mongoose, only the MONGODB_URI env var changes. Zero code changes required.

Final Thoughts

Moving from Atlas to DocumentDB is fundamentally a security and infrastructure consolidation play. Your data moves from the public internet into a private VPC. Your costs consolidate to one provider. Your team manages one less external service.

The technical work is real — but it's manageable. The hardest part isn't the commands. It's the patience to test everything on staging, verify every collection count, and resist the urge to rush to prod.

Do it right, and your app code never knows the difference.

Have questions about your own migration? Drop them in the comments — happy to help.

If this saved you time, consider sharing it with someone who's about to go through the same thing.

Tags: #AWS #MongoDB #DocumentDB #DevOps #DatabaseMigration #BackendEngineering #CloudArchitecture #NodeJS #ECS #SoftwareEngineering

Top comments (0)