DEV Community

Cover image for Troubleshoot MongoDB Replica Sets Easily with Ivory
Andrei Sergeev
Andrei Sergeev

Posted on

Troubleshoot MongoDB Replica Sets Easily with Ivory

#mongodb #database #devops #ai

Primary, secondaries, heartbeats and lag of the whole replica set in one screen — plus a step-down and a config editor when you need them

MongoDB is one of the easier databases to run highly available. A replica set elects its own primary, fails over by itself, and doesn't need a separate HA tool next to it.

But "it fails over by itself" doesn't mean "you never need to look at it". You still need to know who's primary right now, whether the secondaries keep up, and why one member keeps missing heartbeats. That usually means rs.status() — and a long JSON document to scroll through.

So I taught Ivory to read it for me.

The problems

  • rs.status() is long. The answer is in there, somewhere in a deeply nested document, one member at a time.
  • Chained replication hides lag. A secondary syncing from another secondary inherits that secondary's lag, and nothing on the surface tells you why it's slow.
  • Heartbeat failures depend on who's looking. Member A can't reach member B, but B looks fine from C. You only see it from A's point of view.
  • Members in odd states. RECOVERING, ROLLBACK, STARTUP — each means something different, and each matters during an incident.
  • The wrong replica set. Point a node at the wrong address, and you're looking at a perfectly healthy member of a different replica set.

How Ivory solves them

The whole replica set from one member

replSetGetStatus from any member returns the whole replica set, so Ivory can show everything from a single node. Add your cluster by typing its addresses, or give Ivory just one node and let it detect the rest.

The overview shows every member:

  • Role — PRIMARY is the leader, SECONDARY is a replica. RECOVERING and ROLLBACK are replicas still catching up, STARTUP is starting, an arbiter is running with no data role, and a member that's DOWN is unreachable.
  • Lag — seconds a member is behind the primary. Ivory warns past 30 seconds.
  • Tags — only what's worth your attention: the member state when it's anything but primary or secondary, the sync source when a secondary replicates from another secondary, and the heartbeat ping.

That sync source tag is the one I like most. When a secondary is slow because it's chained behind another secondary, the answer is right on the row.

Warnings that say who saw what

  • Failed heartbeats — and which member saw them. "Heartbeat from mongo1 failed" is a very different story from "mongo2 is down".
  • MongoDB's own info message for a member, passed through as is, in MongoDB's own words.

And because every node reports its replica set name, a node from another set is flagged right away.

Operations

  • Failover — asks the current primary to step down (replSetStepDown), so the set elects another one. Great before maintenance on the primary's host.
  • View and edit the replica set configuration (replSetGetConfig / replSetReconfig). Be careful here: an edit replaces the whole configuration document.

A shell-style console

Each node has a console that feels like the Mongo shell: collection.find(), findOne, insertOne/insertMany, updateOne/updateMany, deleteOne/deleteMany, countDocuments, distinct, aggregate and db.runCommand().

You can browse databases and collections, list running operations and kill one, and use saved queries for server status, replica set status and config, current operations, index usage and the profiler.

The machine behind each node

Give Ivory an SSH key and each node shows its host's CPU, memory, network, processes and any log file. If MongoDB runs in Docker, the Container tab has its logs, inspect, start, stop and restart. A secondary that's slow because of a full disk or a noisy neighbour shows up right there.

Need a new replica set?

Ivory deploys one over plain SSH, from a template. Both templates run the official mongo:7 image as replica set members named after your Ivory cluster, and a post-script on the last node runs rs.initiate once all three are up. Access control is left off.

  • Mongo (Multi Host) — three nodes mongo1–mongo3, one per machine, on port 27017. Replace the example addresses in the rs.initiate member list with your hosts.
  • Mongo (Single Host) — all three on one machine on ports 27017–27019. Perfect for a test environment.

After a deploy, Ivory tells you how each node's deploy and initialize steps ended, so if rs.initiate fails, you know exactly where.

Ask your AI agent

Connect Claude, Cursor or Codex to Ivory's built-in MCP server and ask:

"Which node is the MongoDB primary, and is any secondary lagging?"

"Why is mongo3 slow? Check its sync source and its machine."

"Show me the current operations running longer than a minute."

"Step the primary down so I can patch its host — check the secondaries are caught up first."

It works through Ivory, with your permissions. See the MCP docs.

Try it 

docker run -p 80:80 --restart always veegres/ivory
Enter fullscreen mode Exit fullscreen mode

Open http://localhost, finish the setup wizard and add your replica set. MongoDB support is in beta: it works end to end, and your feedback will help make it stable.

Let the replica set elect its leaders — you just keep an eye on it. 😉

Useful links:

Top comments (0)