DEV Community: Harshit Mehta

MongoDB from One Machine to a Multi-Region Cluster

Harshit Mehta — Thu, 24 Sep 2026 19:52:15 +0000

A beginner's guide to where data lives, how capacity grows, and what Atlas manages

MongoDB can begin as one database process on one computer. As an application gets more important, you add copies for safety. As its data or traffic outgrows one machine, you split data across several groups of machines. When users are worldwide, you place those groups in more than one region.

This article builds that picture slowly, first using MongoDB Atlas and then using a self-managed MongoDB Community deployment.

1. Four ideas to learn first

Term	Plain-English meaning	Main job
Node	One running `mongod` or `mongos` process, usually on one VM or one container/pod	Performs one MongoDB role
Replica set	A group of nodes holding copies of the same data	Survives node failures
Shard	One slice of the total data in a sharded cluster	Adds storage and write capacity
Cluster	The whole database system: shards, routers, config servers, and their networking	Serves the application

A node is not necessarily a physical machine. It is a running program. In a simple production design, one MongoDB data-bearing node normally gets its own VM or Kubernetes pod and persistent disk.

2. Stage zero: one computer, one database

A local development setup can be as small as this:

Laptop or one cloud VM
┌──────────────────────────────────┐
│ mongod                            │
│  * database engine                │
│  * data files on local disk       │
│  * indexes on local disk          │
└──────────────────────────────────┘

The application connects straight to mongod.

App  ─────────►  mongod  ─────────►  disk

At this point, every collection, document, and index lives on one machine. It is easy to run, but it has two large weaknesses:

If the machine or disk fails, the database is unavailable and data may be at risk.
One machine has a fixed ceiling for disk, memory, CPU, and write throughput.

3. First production step: a replica set

A replica set is MongoDB's high-availability unit. The members hold the same logical data.

                    writes
App ─────────────► Primary
                     │
              replication stream
              ┌──────┴──────┐
              ▼             ▼
          Secondary A   Secondary B

The primary accepts normal writes. Secondaries continuously copy the primary's operation log and apply the same changes. If the primary disappears, the remaining members elect a new primary.

A common three-member AWS layout is:

AWS Region: eu-west-1

Availability Zone A       Availability Zone B       Availability Zone C
┌──────────────────┐      ┌──────────────────┐      ┌──────────────────┐
│ VM / pod          │      │ VM / pod          │      │ VM / pod          │
│ Primary           │      │ Secondary A       │      │ Secondary B       │
│ Persistent volume │      │ Persistent volume │      │ Persistent volume │
└──────────────────┘      └──────────────────┘      └──────────────────┘

The three members are in the same AWS region but separate Availability Zones (AZs). An AZ is a distinct failure domain, roughly a separate data-center location within a region. If AZ A fails, B and C can elect a primary and continue.

What is stored on each replica-set member?

Each data-bearing member stores:

The database documents assigned to that replica set
The normal MongoDB indexes for those documents
The operation log used to replicate changes
Local runtime state and diagnostics

This means storage is deliberately duplicated. If the replica set contains 1 TB of data, three fully data-bearing members require roughly three copies of that data, plus working space and backups.

4. When one replica set is not enough: sharding

Replication gives copies. It does not split the data load. If a single collection is too large or too busy for one replica set, MongoDB can shard it.

A sharded cluster divides a collection into ranges of a chosen field, called the shard key. Each range is called a chunk, and chunks are placed on shards.

Collection: orders, sharded by customerId

customerId 0 ───────── 999       ► Shard 1
customerId 1000 ────── 1999      ► Shard 2
customerId 2000 ────── 2999      ► Shard 3

Each shard is usually itself a replica set, so each slice is safe against node failures.

                              Sharded cluster

                 ┌──────────────────────────────────────┐
App ───────────► │ mongos routers                        │
                 └───────────────┬──────────────────────┘
                                 │ consult metadata
                 ┌───────────────▼──────────────────────┐
                 │ Config server replica set (CSRS)     │
                 │ cluster map: shards, chunks, rules   │
                 └──────────────────────────────────────┘

   ┌────────────────────┐  ┌────────────────────┐  ┌────────────────────┐
   │ Shard 1 replica set│  │ Shard 2 replica set│  │ Shard 3 replica set│
   │ data range A       │  │ data range B       │  │ data range C       │
   └────────────────────┘  └────────────────────┘  └────────────────────┘

The three special roles in a sharded cluster

Shards store the actual application documents and their normal indexes.

Config servers store the durable map of the cluster: which shards exist, which databases and collections are sharded, which chunks exist, and where those chunks belong. Config servers are a three-member replica set in a typical production deployment. They are metadata servers, not the place where normal application collection data lives.

mongos routers are stateless request routers. Your application connects to them. A router uses the config-server metadata to decide which shard or shards need a request. It does not persist the collection data or the cluster map as its primary store.

A cluster can have several mongos routers for availability and connection capacity. They may run beside application servers, as dedicated services, or as Kubernetes pods.

5. How data grows inside one region

Suppose your orders collection is growing. MongoDB initially spreads chunks among existing shards. When a shard is too full or a new shard is added, the balancer can migrate chunks to spread the data and workload.

Before adding capacity

Shard 1: [A][B][C][D][E]        Shard 2: [F][G]

After the balancer moves chunks

Shard 1: [A][B][C]              Shard 2: [D][E][F][G]

Adding a new shard gives the balancer another destination:

Shard 1: [A][B][C]     Shard 2: [D][E]     Shard 3: [F][G]

This is horizontal growth: more machines share the work. It is different from vertical growth, where you give one machine a larger disk or more CPU.

Why the shard key matters

The shard key determines how documents are distributed and how efficiently requests can be targeted.

A query that includes the shard key can often go to one shard.
A query without it may need to ask every shard, called a scatter-gather query.
A poorly chosen key can send most writes to one shard, creating a hotspot.
A well-chosen key spreads writes and data predictably.

For example, an ever-increasing timestamp can concentrate inserts at the newest end of the data range. A hashed shard key can distribute inserts more evenly, while a ranged shard key can be useful for locality or range queries. The right choice depends on actual query and write patterns.

6. Map: local, regional, and multi-region placement

A “region” is a geographic cloud location. AWS examples include us-east-1 (Northern Virginia), eu-west-1 (Ireland), and ap-southeast-2 (Sydney). A region contains multiple AZs.

                         World map, conceptual

   North America                    Europe                       Asia-Pacific
┌──────────────────┐         ┌──────────────────┐         ┌──────────────────┐
│ AWS us-east-1    │         │ AWS eu-west-1    │         │ AWS ap-southeast │
│ Virginia         │         │ Ireland          │         │ Sydney           │
└──────────────────┘         └──────────────────┘         └──────────────────┘

Same-region, multi-AZ: the normal availability design

For most applications, put members of every replica set across three AZs in one region. This gives low latency and withstands the loss of one AZ.

Region: eu-west-1

AZ A                 AZ B                 AZ C
[Primary]            [Secondary]          [Secondary]

Multi-region: choose the reason before the topology

A second region is not automatically better. It adds network latency, cross-region transfer costs, operational complexity, and failure scenarios. Use it for a clear goal:

Disaster recovery: keep a copy far from the primary region.
Read locality: let users read from a nearby region when the consistency model permits it.
Data residency: keep certain users' data in a required geography.
Low-latency writes for distinct regional data sets: keep each region's data near its writers.

A common disaster-recovery layout looks like this:

Primary region: eu-west-1                 Recovery region: us-east-1

AZ A          AZ B          AZ C           AZ A
[Primary]     [Secondary]   [Secondary]    [Secondary / delayed or voting member]

The cross-region member receives replication over a wide-area network. This can protect against a regional outage, but it will have higher replication lag than local members. The exact voting and election configuration must be designed carefully so a remote, lagging node does not unexpectedly become primary during a partial network split.

Multi-region sharding: place data, not merely copies

For a globally distributed workload, you can use zones (previously called tag-aware sharding) to associate shard-key ranges with geographic regions.

Shard-key range / zone              Preferred shard location
customers in EU  ► zone EU          EU shard(s)
customers in US  ► zone US          US shard(s)
customers in APAC ► zone APAC       APAC shard(s)

                    Global application
                  /          |           \
             EU users      US users     APAC users
                |              |            |
        EU mongos/cluster   US ...       APAC ...
                |              |            |
       EU-zone data shards  US-zone     APAC-zone

Zones can help keep a particular shard-key range on selected shards. They are powerful, but they require a shard key that actually represents the desired placement, such as tenant region plus tenant ID. They do not magically route arbitrary documents geographically.

7. What this looks like on VMs versus Kubernetes pods

The logical architecture is the same. The infrastructure packaging differs.

On virtual machines

EC2 VM 1: mongod, shard-1 member, attached EBS volume
EC2 VM 2: mongod, shard-1 member, attached EBS volume
EC2 VM 3: mongod, shard-1 member, attached EBS volume

Each node runs as an operating-system service. The database files live on durable attached storage such as EBS. A VM restart should not erase that storage.

On Kubernetes

StatefulSet: shard-1

Pod shard-1-0  ──► PersistentVolumeClaim ──► PersistentVolume
Pod shard-1-1  ──► PersistentVolumeClaim ──► PersistentVolume
Pod shard-1-2  ──► PersistentVolumeClaim ──► PersistentVolume

A pod is disposable. Its database data must not be stored only in the pod's temporary filesystem. Each mongod pod needs persistent storage, stable network identity, resource limits, anti-affinity rules to avoid placing all replicas on one worker, and a backup plan.

mongos routers are different: they are stateless. They can typically run as a Deployment with multiple pods, and do not require persistent volumes for application data.

App pods ──► Service / load balancer ──► mongos pods ──► shard and config-server pods

8. Atlas: the managed version of this work

MongoDB Atlas is a managed service. You choose the desired outcomes, and Atlas provisions and operates the underlying MongoDB deployment.

In Atlas, you typically choose:

Cloud provider and region(s)
Cluster tier and storage capacity
Replica-set or sharded-cluster topology
Number of shards, when sharding is needed
Multi-region placement and regional priorities
Network access, private connectivity, encryption, backups, and alerting

Atlas then manages work that a self-managed team would otherwise own:

Task	Atlas	Self-managed Community deployment
Provision VMs/pods and storage	Atlas does it	Your team does it
Place nodes across AZs	Atlas does it from your topology choice	Your team designs and enforces it
Run/configure config servers and `mongos`	Atlas does it	Your team runs them
Monitoring and alerting	Built in	Your team builds/operates it
Backups and restore workflows	Managed options	Your team designs, tests, and operates them
Patching and upgrades	Managed workflow	Your team plans and executes them
Replica-set elections and recovery	MongoDB behavior, with Atlas operating the hosts	MongoDB behavior, with your team operating the hosts

Atlas does not remove application-level decisions. You still must choose good data models, indexes, access patterns, shard keys, retention policies, security roles, and regional requirements. Atlas makes the infrastructure and operational layer substantially easier.

9. Self-managed MongoDB Community: what you configure

MongoDB Community can run replica sets and sharded clusters. You supply the infrastructure and configure the processes.

At a high level, a self-managed sharded cluster comes up in this order:

Create machines/pods, networking, DNS, persistent disks, certificates, monitoring, and backups.
Start the three config-server replica-set members.
Start the replica-set members for each shard.
Start one or more mongos routers, pointing each at the config-server replica set.
Connect to a mongos router and add the shard replica sets to the cluster.
Enable sharding for the database and collection, choosing a shard key.
Test failure, recovery, backup restore, monitoring, and scaling before production use.

A mongos is told how to find the config servers, not every shard directly:

mongos
  │
  └── config server replica set address
          │
          └── cluster metadata identifies the shard replica sets

The cloud locations themselves are configured outside MongoDB: AWS account/region/AZ selection, VPCs, subnets, routing, security groups, DNS, and Kubernetes scheduling. MongoDB is configured with network addresses that let its members communicate.

10. A practical growth path

A sensible progression for a new team is usually:

Start with one replica set in one region, distributed across three AZs.
Monitor disk growth, working-set memory, slow queries, CPU, replication lag, and backup/restore performance.
Improve schema design and indexes before assuming sharding is needed.
Scale the replica-set hardware when that is sufficient.
Shard only when capacity or throughput truly exceeds what one replica set can reasonably serve, and only after choosing a proven shard key.
Add multi-region placement when there is a concrete recovery, latency, or residency requirement.

Sharding is an architecture choice, not merely a “large database” switch. It makes storage and writes scale horizontally, but it also introduces routers, metadata, balancing, chunk movement, and more operational complexity.

11. Key takeaways

A cluster is the whole system, not one machine.
A shard is normally a replica set that stores one portion of total data.
Replica-set members are separate processes on separate VMs or pods, each with durable storage.
Config servers store the sharded-cluster map; mongos routes requests using that map.
Same-region, multi-AZ placement is the usual first high-availability design.
Multi-region deployment should serve a clear purpose and be designed around latency, recovery, and data placement.
Atlas manages the operational machinery; self-managed Community requires your team to build and run it.

Earn While You Learn - Opportunities at Topcoder

Harshit Mehta — Tue, 12 Jan 2021 03:16:58 +0000

Hey Learners,

Now that you have landed here to learn. I am sure you understand that the tech landscape is changing more rapidly than ever. The need for upskilling and reskilling is becoming all the more important. With the growing digital economy, there is a tremendous opportunity for growth. But, let’s not forget that growth opportunities come with a need for upskilling the existing and potential workforce.

Once you have realized the importance of and need for upskilling, you’ll quickly learn that you need to practice those skills. The most beneficial form of practice comes from solving real-world problems.

Topcoder gives you just the right opportunity to do that. Not only can you practice skills, you can also earn money while you apply what you have learned in competitions. While you compete your Topcoder profile keeps improving, with verified skill badge additions and an increasing Topcoder rating.

I am sure your next question will be:

What is Topcoder?

And how do I get to do work on real-world projects? How can I earn while I learn?

Topcoder is a marketplace where we connect talent (like you) with companies that are seeking innovation from our community. We have several Fortune 500 clients like NASA, IBM, T-mobile, Adobe, etc., that give us interesting problems to solve. We then turn those problems over to the Topcoder community members in the form of challenges and gig work opportunities.

While it may be easy to think that large organizations like T-Mobile, Microsoft, and Macy’s have the inner depth and scale to tackle projects solo, we find innovative Fortune 500 companies are always looking for ways to launch faster and maximize their development outputs. For example, for T-Mobile, Topcoder delivers the scale they need to increase their in-house teams’ productivity and throughput.

Other than competing, there’s actually a lot you can do at Topcoder. Read more below in the last section.

How to get started: Still a beginner? Don’t Worry!

There are two ways to jump right into the mix of things at Topcoder.

Topcoder consists of the best developers, designers, and data scientists from all over the world; but that is not a reason to feel nervous or not good enough. Consider the Topcoder platform as an opportunity to learn and grow with the best. The community is always welcoming and helps newbies get up to speed, learn and start winning. They love tough competition and are constantly helping each other to grow that competition.

Topcoder offers you things no other company can offer:

No background, experience check or CV to start competing and learning. Some traditional companies will reject your CV just because they believe you are not experienced enough or offer you a small fixed pay for the next year until you gain the necessary experience. Topcoder encourages you to join and try your skills on real problems right away, and to win money consistent with your skills.
No discrimination. Ever. Your location, sex, or anything else makes no difference! From day one you are judged only by the quality of the output you deliver. That’s how the community gets to know you and is the only thing that matters for your success at Topcoder.
Incredibly rapid learning curve. The power of community: even when you lose a challenge, you gain priceless experience. You can ask the copilot (the technical coordinator of the challenge), other competitors and community members if you are stuck with something and you can see the winning submission after the challenge. If you take it seriously, after each challenge, you will presumably learn enough to perform well in the next challenge of the same kind. Also, the diversity of technologies and problems offered by Topcoder clients helps you gain wide experience, unlike most of the job roles at conventional companies.
A power-packed tournament and a fully paid-trip. Topcoder Open that will allow you to prove yourself as the best of the best. Regular Competing helps you earn points and win a fully-paid trip to the United States, where you compete with the best of the best

In short, at Topcoder you can easily solve the right problems and find the right guidance. But, Topcoder is not only about competing. Imagine a readily available community for you to be a part of, discussing the challenges with the best and at the end of the competition getting access to the winning submission.

Here’s a link to our practice challenges and Skill Builder Challenges designed specifically to practice/test the skills you are learning which will help you get started on Topcoder Challenges.

Also, when you have some time, check out this article about a member’s first year at Topcoder.

Will my Topcoder experience help me get jobs outside of Topcoder?

Yes, your Topcoder profile will help you!

Topcoder profiles and ratings have been used by many customers and clients at Topcoder. Companies like Google, Yahoo, Nvidia, Brooks Automation, Microsoft, Paypal, Verizon, Citigroup, J.P. Morgan, Cryptic Studios, Facebook, etc. have hired Topcoder talent in the past.

While your Topcoder background helps you get these opportunities, adding your Topcoder rating and skills to your CV gives it that all-important edge to stand out from the pack. Imagine a CV which showcases experience of working for clients like NASA, IBM, T-Mobile and many more. Who would not pick that CV up?

Why even search for a job? You can also compete and perform other roles at Topcoder for a living. Many of our members do just that.

Does Topcoder sound interesting? Gig Work and Challenges are not the only things you can do at Topcoder.

So, what else can you do at Topcoder?

There are a number of different avenues you can take as a Topcoder community member. You can start out by taking on some gig work or trying out some challenges to get accustomed to our platform. Just remember that is not all you can do. This article is a great way to learn all about the different roles you can take on at Topcoder to grow your career or earn extra money. Take a look at The Many Roles You Can Take At Topcoder.

You should definitely check out how Topcoder members around the world become friends and travel together. Join our Topcoder Nation Facebook group to be even closer to the community, establish new friendships, learn, travel and grow together.

Also, now that 2021 is here with opportunities missed out in 2020, check out some of the things we have planned for the community this year and pick the ones you can benefit from.

Weekly chats with the community team every Monday at 10:00 UTC -5. Kick-off each week with the latest at Topcoder and talk to us face to face for help, networking, or to just say "Hello".
Beta Testers group - access to the latest features of the Topcoder platform and help us with new testing experiences.
Our Crowd for Good initiative for the Yellowstone Economic Research Center (YERC).
Interesting timed SRM-like contests for Software Developers - Rapid Development Matches (RDMs) focus on making you practice and get better at web/mobile development languages, technologies, libraries and frameworks.
New Topcoder Skill Builder challenges where you can ramp up on new technologies, get the verified skill added to your profile, practice for RDMs and help you get a gig.
More Rookie SRMs for newbies to get motivated for competitive programming.
More THRIVE articles for you to digest with plans to top the over 1,000 we added in 2020.
New webinars with field experts and professional speakers looking to help you advance your career, your skills, and your personal brand.

At Topcoder, we are not just some website you can earn money from or learn on, we truly are a community of like-minded folks cheering each other on for success. We will welcome you and help you grow into what you want to be.