DEV Community: MongoDB

No messages table! The data model behind my own Claude-based chatbot

MongoDB Guests — Thu, 02 Jul 2026 15:47:31 +0000

This tutorial was written by Néstor Daza.

This is the second article in a series about building Claudius, my own Claude-based chatbot (Github). The prologue made the case for building it, and for choosing MongoDB as its foundation.

Open the conversations collection in Claudius’ database and you find the usual fields of a thread header but nothing else: a userId, a title, some timestamps, and so on, but no array of messages, no messages collection sitting beside it either! The text of every conversation lives somewhere else entirely, in the LangGraph checkpointer, which I wire up later in this series. This absence is a modeling decision, and how I came up with the database schema for my chatbot is the theme of this article.

If you come from a relational background, you're used to modeling the data first when designing a database. For a project like this, you would start by finding the entities and normalizing them, and the final schema would come out of the data's structure: a conversations table and a messages table with a foreign key between them, because that is what the data looks like.

Document modeling runs the other way. You start from how the application reads and writes, and the shape of the document follows the access patterns. Claudius never reads conversation messages without the agent's full working state wrapped around them, and that state is persisted using the LangGraph checkpointer. A separate messages table would add nothing, since the app would always have to join it back to that state on every read. The access pattern says the messages belong with the agent state, so that is where they go, and conversations are left as the lightweight header the list view actually needs.

That inversion, modeling around use rather than around the data, runs through everything below.

Schema-flexible is not schemaless

This is the misconception lots of people often carry, and it is worth killing on the way in. A document database does not mean no schema; it means a flexible, use-case-based data model. A relational engine holds the schema for you through column types and constraints; MongoDB will store documents in whatever shape your application gives it, and can validate that shape with a collection validator if you want. Claudius keeps the schema in code, as one Zod schema per collection, because that single definition gives both the runtime check and the TypeScript type.

import { z } from "zod";
import { zObjectId } from "./common";

export const ConversationSchema = z.object({
  _id: zObjectId.optional(),
  userId: zObjectId,
  title: z.string(),
  modelId: z.string(),
  createdAt: z.date(),
  updatedAt: z.date(),
  archived: z.boolean(),
  expiresAt: z.date().optional(),
});

export type Conversation = z.infer<typeof ConversationSchema>;

The reason is drift. Declare a TypeScript interface for a document and a separate validator for the data coming in, and the two will disagree eventually, while the type system will lie to you with complete confidence. Deriving the type from the validator collapses them into one artifact. One definition gives you the runtime check at the boundary and the compile-time type everywhere else. For a relational reader, this is your DDL (Data Definition Language) and your ORM (Object-Relational Mapping) models together in the same structure, which is something relational stacks rarely pull off.

Another advantage is that every external input, whether an incoming HTTP body or a model's structured output, is parsed through Zod at the boundary, so by the time data reaches the logic, it is already validated and typed.

A quick tour of the model

There are eight collections in this phase: users, conversations, memories, documents, chunks, usage_events, settings, and, arriving later, jobs. Two more, checkpoints and checkpoint_writes, belong entirely to the checkpointer and are never written to directly, so I do not model them. Most of these schemas are unremarkable once you see the access-pattern framing, but two modeling moves are worth pausing on.

The first is settings. It holds a handful of global singletons, each a different shape, each identified by a string _id: the member allowlist, the model catalog, the per-role tiers, and the guest spend circuit breaker. They share one collection, so the Zod schema is a discriminated union based on _id, and parsing a settings document narrows it to the correct shape from its _id literal.

export const SettingsSchema = z.discriminatedUnion("_id", [
  AllowlistSettingsSchema,        // _id: "allowlist"
  ModelCatalogSettingsSchema,     // _id: "modelCatalog"
  TiersSettingsSchema,            // _id: "tiers"
  GuestCircuitBreakerSettingsSchema, // _id: "guestCircuitBreaker"
]);
export type Settings = z.infer<typeof SettingsSchema>;

In relational terms, this is a single typed configuration table whose rows legitimately carry different columns, which is usually awkward to model relationally without resorting to a JSON column, and here comes out clean and readable.

The second move is to use document embedding as a data modeling technique. The users document carries its dailyMessageCount as an embedded object holding the count and its reset time together, rather than as two flat fields or a side table, because the application always reads and writes them together as a unit. That is the document modeling mantra: data used together is stored together.

Let the database keep the rule

The most satisfying part of this phase is how little policy I had to write. Three product rules that I would normally enforce in application code are carried by the storage engine instead, because I shaped the data so the engine could hold them.

Retention is a TTL index

Guest data in Claudius is ephemeral. Guest-created conversations and memories carry an expiresAt date; member and admin documents leave the field off entirely. A TTL (time-to-live) index with expireAfterSeconds set to 0 tells MongoDB to delete each document at the exact moment stored in its own expiresAt field; documents without this field are never touched.

// TTL on the guest-only expiresAt field. expireAfterSeconds:0 
// means "expire exactly at the date stored in the field"; 
// documents without the field are never touched, so member/admin
// data is permanent.

await db
  .collection(COLLECTIONS.conversations)
  .createIndex({ expiresAt: 1 }, { name: "expiresAt_ttl", expireAfterSeconds: 0 });

await db
  .collection(COLLECTIONS.memories)
  .createIndex({ expiresAt: 1 }, { name: "expiresAt_ttl", expireAfterSeconds: 0 });

So, guest ephemerality, a product rule, is enforced by the engine. There is no cron job and no cleanup endpoint to remember to run. The relational equivalent is a scheduled sweep; here, the policy lives in the index.

One detail ties the TypeScript compiler to the database behavior. With exactOptionalPropertyTypes turned on in tsconfig.json, an optional field means the key is absent, not present, and set to undefined. The compiler setting and the storage rule reinforce each other.

Telemetry is a time-series collection

Every billable model call writes one usage_events document, recording the token counts, the latency, and what the call was for. That data is append-only and gets queried along a few low-cardinality dimensions over a time window, which is the textbook shape for a MongoDB time-series collection. It has a timeField, timestamp, and a single metaField that holds the grouping dimensions (userId, modelId, purpose) nested alongside the measurements.

Two things follow from this choice. A time-series collection has to be created explicitly with a timeseries spec before the first insert (they cannot be created on first write, the way an ordinary collection does), so the index script creates it behind an existence check. Once it exists, the engine's own machinery becomes visible: list the collections and a system.buckets.usage_events sits next to usage_events, which is where the bucketed storage lives:

[
  {"name":"memories","type":"collection"},
  {"name":"usage_events","type":"timeseries"},
  {"name":"system.buckets.usage_events","type":"collection"},
  {"name":"settings","type":"collection"},
  {"name":"users","type":"collection"},
  {"name":"conversations","type":"collection"},
  {"name":"accounts","type":"collection"},
  {"name":"chunks","type":"collection"},
  {"name":"system.views","type":"collection"}
]

The relational analogy is a metrics or rollup table you would otherwise hand-roll and maintain, except the bucketing is automatic and the query you write against it stays simple.

A pre-filter field is a security control

The project's hardest rule is that a vector search pre-filters by userId and can never return another user's data. That rule is not bolted onto the search in the application code. It is a property of the index.

Both memories and chunks get an Atlas Vector Search index on the embedding field, with a filter on userId (and on chunks, a second filter on documentId).

await ensureVectorIndex(db, COLLECTIONS.memories, "memories_vector", [
  { type: "vector", path: "embedding", numDimensions: 1024, similarity: "cosine" },
  { type: "filter", path: "userId" },
], created, skipped);

await ensureVectorIndex(db, COLLECTIONS.chunks, "chunks_vector", [
  { type: "vector", path: "embedding", numDimensions: 1024, similarity: "cosine" },
  { type: "filter", path: "userId" },
  { type: "filter", path: "documentId" },
], created, skipped);

Pre-filtering means the userId condition is applied during the indexed search on the same documents being scored, not afterward to whatever the search returns. Post-filtering is not only slower, since you score documents you then throw away, but it is a correctness and security risk, since the wrong documents can slip through ranking and limits before your filter ever runs.

This is also why chunks denormalizes userId. A vector search has no notion of a join, so the filter has to sit on the very documents being searched. A chunk belongs to a document that belongs to a user, but the chunk has to carry the owner itself, or the index cannot enforce the boundary. That is the embed-versus-reference tension made concrete: a chunk references its parent document by id and duplicates the one field the security boundary depends on.

Two gotchas surfaced while creating these indexes, and both are worth saying out loud. A search index can only be built on a collection that already exists, and chunks had no classic index, so the collection did not exist yet the first time the script reached for its vector index:

MongoServerError: Error retrieving collection UUID and view info ::
caused by :: Collection 'claudius.chunks' does not exist. (NamespaceNotFound)

The fix is to add an existence check and create the collection first only if it is missing. The second gotcha is that Atlas builds search indexes asynchronously, so createSearchIndex returns when the build is queued, not when it is queryable! We also need a name check that keeps reruns safe while it builds.

This is the final function to build the indexes from code using the driver's createSearchIndex, with the checks that keep the script safe to run on every deploy:

async function ensureVectorIndex(
  db: Db,
  collectionName: string,
  indexName: string,
  fields: Array<Record<string, unknown>>,
  created: string[],
  skipped: string[],
): Promise<void> {
  // A vector search index can only be built on an existing
  // collection, so create it first if nothing else has (e.g.
  // chunks has no regular index).
  const collections = await db
    .listCollections({ name: collectionName })
    .toArray();
  if (collections.length === 0) {
    await db.createCollection(collectionName);
  }

  const collection = db.collection(collectionName);
  const present = await collection.listSearchIndexes().toArray();
  if (present.some((idx) => idx.name === indexName)) {
    skipped.push(`${collectionName}.${indexName}`);
    return;
  }

  // Atlas builds the index asynchronously; createSearchIndex
  // returns once the build is queued. The name check above 
  // keeps re-runs idempotent.
  await collection.createSearchIndex({
    name: indexName,
    type: "vectorSearch",
    definition: { fields },
  });
  created.push(`${collectionName}.${indexName} (vector)`);
}

The plumbing that keeps the code honest

Two small things hold the userId rule together in practice. First, userId is an ObjectId everywhere, matching the user’s _id, so filters and lookups line up without conversion. Second, no collection is ever queried through a raw driver call. Every access goes through a typed accessor, conversationsCol(), memoriesCol(), and so on, each returning a typed collection for the right document shape. Centralizing the collection names means a rename happens in one place, and typing each accessor keeps the userId filter visible to the compiler at every call site, which is how a rule that lives in prose stays honest in code.

The whole application and the Auth.js adapter share a single, serverless-safe MongoDB client to a single fixed database name, so there is exactly one connection pool. How that database name quietly caused two separate failures is a good story, and it belongs to the next article, where I stand the foundation up.

What this article bought

No chatbot yet, but the shape of the thing is already decided. Retention, the telemetry layout, and the tenant-isolation boundary are properties of the schema and its indexes now, not code I have to remember to run later. That is the payoff of modeling around the access patterns first: the database carries more of the load because you shaped it to. The data model is the foundational piece of the whole app.

Next, we'll build this foundation up, with identity the client never gets a vote in, and a health check that proves the app can reach Atlas and Bedrock, plus a view on the unglamorous errors that cost me real time. After that comes the streaming chat backbone, which is where the conversation finally gets a place to live, and I open a real checkpoint document to show you the messages that were never in their own collection.

Advanced API Authentication Strategies with Node.js and MongoDB (Part 3)

MongoDB Guests — Thu, 02 Jul 2026 09:00:00 +0000

This article was written by Moses Anumadu.

Secure Session Management with JWT and Refresh Tokens in Node.js and MongoDB

This is the final part of this 3-part series. In parts 1 and 2, we implemented email-based authentication. Then we went further to implement TOTP-based multi-factor authentication (MFA), QR code onboarding, and half-authentication login flow.

What we have built so far is an API that allows users to prove their identity using email and password. In this section, we will go beyond that. The application should not only be able to recognize users but also protect the user through concepts like secure session management, token expiration, and session revocation. We will implement these concepts through the following:

JWT access tokens
refresh tokens
MongoDB-backed sessions
refresh token rotation
protected routes
secure logout flows

By the end of this section, we should have an authentication system that can be deployed to production and improved upon for more advanced features.

Pre-requisites

To follow along with this tutorial, ensure you have prior experience working with the following technologies:

Node.js and Express
MongoDB and Mongoose
A MongoDB Atlas account
Postman
Ngrok
Parts one and two of this series

With that said, let's get started.

Understanding Access Tokens vs Refresh Tokens

An access token is a JWT token that grants a user access to a protected route. It often has a very short-lived life cycle. A refresh token is a token with a longer expiration time solely for requesting an access token. Without the refresh token, the user would need to log out and log back in to generate a new access token. The refresh token is stored in the database, enabling the system to recognize the user without terminating the session.

You are probably wondering why you need both, right? Let's look at an example. Imagine that a hacker steals a user's access token, and with this token, he (the hacker) can do malicious things. If this access token does not have a short-lived life cycle, the attacker would have a longer window to do more harm. To prevent this, the refresh token is required to generate a new access token from time to time. So, even if an access token is stolen, within 15 minutes, that token becomes invalid. The attacker can't do any harm.

This is an extra layer of security necessary for a secure authentication system and to improve the user experience. The flow looks like: a user logs in, access token and refresh token are granted, access token lasts for 15 minutes, refresh token lasts for 7 days, access token expires after 15 minutes, user does not need to log out, user generates a new access token using the refresh token saved in the database.

Creating Token Utilities

Now, we understand why we need both an access token and a refresh token. Let's proceed to the implementation. In part 2, when we created the MFA token, we created src/utils/tokenUtils.js file. We need to update that file to generate our access token, refresh token, and MFA token. Open src/utils/tokenUtils.js and replace the code with the code below:

const jwt = require("jsonwebtoken");

const crypto = require("crypto");

const generateAccessToken = (user) => {
  return jwt.sign(
    {
      userId: user._id,
      email: user.email,
    },
    process.env.JWT_ACCESS_SECRET,
    {
      expiresIn: process.env.ACCESS_TOKEN_EXPIRES_IN,
    }
  );
};

const generateRefreshToken = () => {
  return crypto.randomBytes(64).toString("hex");
};

const hashToken = (token) => {
  return crypto
    .createHash("sha256")
    .update(token)
    .digest("hex");
};

const generateMFAToken = (user) => {
  return jwt.sign(
    {
      userId: user._id,
      type: "mfa",
    },
    process.env.JWT_ACCESS_SECRET,
    {
      expiresIn: process.env.MFA_TOKEN_EXPIRES_IN,
    }
  );
};

module.exports = {
  generateAccessToken,
  generateRefreshToken,
  hashToken,
  generateMFAToken,
};

We added three methods generateAccessToken, generateRefreshToken, hashToken to the page. The hashToken will be used later. It will be used to encrypt our refresh token into a one-way SHA-256 hash before it is stored in the database or compared when validating. The generateAccessToken and generateRefreshToken generate an access token and a refresh token, respectively.

Why Refresh Tokens Are Not JWTs

Also, did you notice that our refresh tokens were generated using secure random bytes? This line of code here: crypto.randomBytes(64).toString("hex");. This simply gives us more control over it. The refresh token should be random, unpredictable, and most importantly, database-controlled. Refresh tokens, unlike access tokens, do not need readable claims.

So, this way, we can easily revoke it, rotate it, and control the session validation or invalidation.

Creating the Session Model

Our refresh token is database-controlled. That ultimately means we need to store and retrieve it from the database. To achieve this, we need to create a MongoDB model for storing user sessions. Create src/models/Session.js and update the page with the code below:

const mongoose = require("mongoose");

const sessionSchema = new mongoose.Schema(
  {
    userId: {
      type: mongoose.Schema.Types.ObjectId,
      ref: "User",
      required: true,
    },

    refreshTokenHash: {
      type: String,
      required: true,
    },

    userAgent: {
      type: String,
      default: null,
    },

    ipAddress: {
      type: String,
      default: null,
    },

    expiresAt: {
      type: Date,
      required: true,
    },

    revokedAt: {
      type: Date,
      default: null,
    },
  },
  {
    timestamps: true,
  }
);

module.exports = mongoose.model("Session", sessionSchema);

Notice that we are storing refreshTokenHash. Always remember that sensitive data at rest should also be encrypted. We created the fields we need to save for each user session. Notice fields like ipAddress, userAgent, expiresAt and revokedAt.

Issuing Tokens After MFA Verification

Now we can generate both a refresh token and an access token, and our MongoDB model is ready to save session values in the database. Next, we need to modify the authentication flow to include these. The flow should be: a user logs in, completes MFA, the application issues an access token and a refresh token to the user, and finally, the session values (including the refresh token) are stored in our MongoDB session collection. Let's proceed to the implementation.

!!!
Returning refresh tokens in JSON bodies is unsafe for browser clients and encourages XSS-exposed storage patterns. The safer pattern is HTTP-only cookie transport for refresh credentials. This is the approach we will follow in the next section.
!!!

Updating the Login Controller

In the next section, we will send refresh tokens using HTTP-only cookies transport as this is the safer option to prevent XSS-expose attacks.
The code in src/controllers/authController.js needs to be updated. Let's start by adding the import values and HTTP-only cookies configurations.

Open src/controllers/authController.js and add the following import statement to the top part of the page

const jwt = require("jsonwebtoken");
const Session = require("../models/Session");

const {
  generateAccessToken,
  generateRefreshToken,
  hashToken,
  generateMFAToken,
} = require("../utils/tokenUtils");

After the import statements, add the following HTTP-only cookie configurations. We will use these later in the code.

const getRefreshTokenExpiryMs = () => {
  const refreshDays = Number(
  process.env.REFRESH_TOKEN_EXPIRES_DAYS
  );

  return refreshDays * 24 * 60 * 60 * 1000;
};

const getRefreshTokenCookieOptions = () => {
  return {
    httpOnly: true,
    secure: process.env.NODE_ENV === "production",
    sameSite: "strict",
    path: "/api/auth",
    maxAge: getRefreshTokenExpiryMs(),
  };
};

const setRefreshTokenCookie = (res, refreshToken) => {
  res.cookie(
  "refreshToken",
  refreshToken,
  getRefreshTokenCookieOptions()
  );
};

const clearRefreshTokenCookie = (res) => {
  res.clearCookie("refreshToken", {
    ...getRefreshTokenCookieOptions(),
    maxAge: undefined,
  });
};

Updating the MFA Login Verification Flow

Next, replace the success section inside verifyMFALogin controller method with the code below:

Revoke this section

return res.status(200).json({
success: true,
message: "MFA login successful",
});

And replace with this:

// Issue access token
const accessToken = generateAccessToken(user);

// Generate refresh token
const refreshToken =
  generateRefreshToken();

// Hash refresh token
const refreshTokenHash =
  hashToken(refreshToken);

// Session expiry
const expiresAt = new Date();

expiresAt.setDate(
  expiresAt.getDate() +
    Number(
      process.env.REFRESH_TOKEN_EXPIRES_DAYS
    )
);

// Store session
await Session.create({
  userId: user._id,
  refreshTokenHash,
  userAgent: req.headers["user-agent"],
  ipAddress: req.ip,
  expiresAt,
});
setRefreshTokenCookie(res, refreshToken);
      return res.status(200).json({
       success: true,
       message: "Login successful",
       accessToken,
     });

From the additions we made, after password verification and MFA verification, a token is issued, and a session is created for the authenticated user.

Creating JWT Authentication Middleware

Now, we can issue an access token and a refresh token. Next, we need a way to protect our routes so that only users with the right access can access them. To do this, we need to implement middlewares.

A good example can be creating a middleware that protects routes and lets only authenticated users have access to it. Let's implement that. Create the following file src/middleware/authMiddleware.js, and update it with the code below:

const jwt = require("jsonwebtoken");

const User = require("../models/User");

const requireAuth = async (
  req,
  res,
  next
) => {
  try {
    const authHeader =
      req.headers.authorization;

    if (
      !authHeader ||
      !authHeader.startsWith("Bearer ")
    ) {
      return res.status(401).json({
        success: false,
        message: "Access token required",
      });
    }

    const token = authHeader.split(" ")[1];

    const decoded = jwt.verify(
      token,
      process.env.JWT_ACCESS_SECRET
    );

    const user = await User.findById(
      decoded.userId
    );

    if (!user) {
      return res.status(401).json({
        success: false,
        message: "Invalid user",
      });
    }

    req.user = user;

    next();
  } catch (error) {
    return res.status(401).json({
      success: false,
      message:
        "Invalid or expired access token",
    });
  }
};

module.exports = requireAuth;

Creating a Protected Route

At this point, the middleware is ready. We need to attach it to our routes to restrict access to only authenticated users. Let's create a controller to get the authenticated user getMe and a route for it /api/auth/me then protect it with the middleware we created above. To create the controller method, open src/controllers/authController.js and add the following controller method to the page.

const getMe = async (req, res) => {
  return res.status(200).json({
    success: true,
    user: {
      id: req.user._id,
      email: req.user.email,
      mfaEnabled: req.user.mfaEnabled,
    },
  });
};

Also, update the export block to contain the getMe method we just added, like so

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
  getMe,
};

Adding the Protected Route

Next, let's create the route. Open src/routes/authRoutes.js and add the following to the page.

First, import the middleware by copying and pasting the code below to the top section of the page

const { getMe } = require("../controllers/authController");
const requireAuth = require("../middleware/authMiddleware");

Then add the route below to the route on the page.

router.get("/me", requireAuth, getMe);

Testing Protected Routes

We've made progress. Let's test what we've built so far. We will test with Postman on the web and Ngrok. Ngrok will be used to tunnel our localhost address to something on the internet. Check Part One of this article for more on the setup. With that said,

Login

To generate an access token, we need to go through a series of verifications. First, we need to log in and generate an MFA token to verify MFA. Send a post request to the route below to log in.

POST /api/auth/login

Add the following to the body

{ 
  "email": "test_user@example.com",
  "password": "password123"
}

Your response should look similar to the image below if everything is done right.

MFA verification and access key generation

We are partially logged in. We need to verify MFA to be fully logged in. Send a Post request to the route below:

POST /api/auth/mfa/login/verify

Also, add the following to the body of the request:

{ "token": "TOKEN_GENERATED_BY_AUTH_APP",
  "mfaToken": "MFA_TOKEN_GENERATED_FROM_LOGIN"
}

Protected route

Next, send a GET request to the route below

GET /api/auth/me

With the following Header values

Authorization: Bearer YOUR_ACCESS_TOKEN

If the token is valid, the request should return the authenticated user information, like the image below

Also, in MongoDB Atlas, you can see that user sessions were created and stored in our session collection.

Implementing Refresh Token Rotation

Let's take our authentication a step further by implementing refresh token rotations. What we want to achieve is simple. Every refresh request should generate a new access token, generate a new refresh token, and invalidate the old refresh tokens. Let's jump right into it.

Creating the Refresh Endpoint

Add the following controller method to src/controllers/authController.js.


const refreshAccessToken = async (req, res) => {
   try {
     const refreshToken = req.cookies?.refreshToken;
      if (!refreshToken) {
       return res.status(401).json({
         success: false,
         message: "Refresh token cookie required",
       });
     }
      const refreshTokenHash = hashToken(refreshToken);
      const session = await Session.findOne({
       refreshTokenHash,
       revokedAt: null,
     });
      if (!session) {
       clearRefreshTokenCookie(res);
       return res.status(401).json({
         success: false,
         message: "Invalid session",
       });
     }
      // Expired session
     if (session.expiresAt < new Date()) {
       clearRefreshTokenCookie(res);
       return res.status(401).json({
         success: false,
         message: "Session expired",
       });
     }
      const user = await User.findById(session.userId);
      if (!user) {
       return res.status(401).json({
         success: false,
         message: "Invalid user",
       });
     }
      // ROTATE REFRESH TOKEN
     const newRefreshToken = generateRefreshToken();
      const newRefreshTokenHash = hashToken(newRefreshToken);
      session.refreshTokenHash = newRefreshTokenHash;
      await session.save();
     setRefreshTokenCookie(res, newRefreshToken);
      // Generate new access token
     const accessToken = generateAccessToken(user);
      return res.status(200).json({
       success: true,
       accessToken,
     });
   } catch (error) {
     console.error(error);
      return res.status(500).json({
       success: false,
       message: "Failed to refresh token",
     });
   }
 };

Next, let's update the exports with the freshAccesstoken method like so:

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
  getMe,
  refreshAccessToken,
};

Adding the Refresh Route and Testing

Let's proceed by adding a route to test the refresh token feature. Open src/routes/authRoutes.js and update the page with the code below

router.post(
  "/refresh",
  refreshAccessToken
);

Let's proceed to test. Send a POST request to the route below.

POST /api/auth/refresh

Request body:

{
  "refreshToken": "YOUR_REFRESH_TOKEN"
}

If everything was done right, your response should look like the screenshot above. And the previous tokens should no longer work.

Implementing Logout and Session Revocation

A complete logout system not only removes the access token from the browser in the front end. It should also invalidate the refresh token and revoke the session from the backend. That is exactly what we will do in this section.

Creating the Logout Endpoint

Add the following controller method to src/controllers/authController.js

const logout = async (req, res) => {
   try {
     const refreshToken = req.cookies?.refreshToken;
      if (!refreshToken) {
       return res.status(400).json({
         success: false,
         message: "Refresh token cookie required",
       });
     }
      const refreshTokenHash = hashToken(refreshToken);
      await Session.findOneAndUpdate(
       {
         refreshTokenHash,
         revokedAt: null,
       },
       {
         revokedAt: new Date(),
       }
     );
     clearRefreshTokenCookie(res);
      return res.status(200).json({
       success: true,
       message: "Logged out successfully",
     });
   } catch (error) {
     console.error(error);
      return res.status(500).json({
       success: false,
       message: "Logout failed",
     });
   }
 };

From the code, we did not delete the session records. We marked them as revoked instead, as we can see below.

await Session.findOneAndUpdate(
{
refreshTokenHash,
revokedAt: null,
},
{
revokedAt: new Date(),
}
);

This is a good practice as it helps preserve session history for audit and security visibility.

Implementing Logout From All Devices

Let's take it a step further by allowing users to revoke all active sessions simultaneously. This would let them log out from all devices at once. This is a nice feature to have; it helps a user protect their account in the case of a malicious login from an unknown device.

Let's proceed, modify src/controllers/authController.js file, and add the following controller method:

const logoutAll = async (req, res) => {
   try {
     await Session.updateMany(
       {
         userId: req.user._id,
         revokedAt: null,
       },
       {
         revokedAt: new Date(),
       }
     );
     clearRefreshTokenCookie(res);
      return res.status(200).json({
       success: true,
       message: "Logged out from all devices",
     });
   } catch (error) {
     console.error(error);
      return res.status(500).json({
       success: false,
       message: "Failed to logout all sessions",
     });
   }
 };

Update the export block to contain all the methods below

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
  getMe,
  refreshAccessToken,
  logout,
  logoutAll,
};

Adding Logout Routes

To add the routes for logout, update src/routes/authRoutes.js and add the code below. Also ensure that you have imported both logout and logoutAll methods from authController.js.

router.post("/logout", logout);

router.post(
  "/logout-all",
  requireAuth,
  logoutAll
);

Testing Logout

Send a post request to the route below. Also, pass the body value in the body of your request.

Request:

POST /api/auth/logout

Request body:

{
  "refreshToken": "YOUR_REFRESH_TOKEN"
}

Your response should look similar to the screenshot below

You can also test the logout all feature the same way. Send a Post request to the route below with a valid access token.

Request:

POST /api/auth/logout-all

Also, add a header with a user's access token.

Authorization: Bearer YOUR_ACCESS_TOKEN

Expected response:

{
  "success": true,
  "message":
    "Logged out from all devices"
}

Did you notice that the logout-all requires an access token passed in as a request header, not a refresh token like the logout route? This is because the logout-all does not have an active user session or know the user sending the request. It needs a way to identify the user, so it requires a user's access token.

All refresh token sessions should now become invalid. To be sure, you can also try using the same refresh token. It should fail because this session has been revoked.

Conclusion

A big thumbs up to you for sticking around till the end of this three-part series. We built a complete authentication API using Node.js, Express, MongoDB Atlas, and tested using Postman on the web and Ngrok for localhost routing.

In this final part of this series (part three), we discussed the difference between JWT access tokens and refresh tokens, why we need both, and walked through the programmatic implementation of the authentication by implementing the following:

JWT access token authentication
Session storage
refresh token rotation
Middleware and protected routes
secure logout and session revocation

The complete flow of the authentication with all three (3) parts put together looks like:

User registration, password authentication, MFA setup and verification, JWT access tokens and refresh token issuance, refresh token sessions, middlewares and protected routes, session.

Security is more of layers than single features. For further reading, check out the following topics for additional layers that can be added to an authentication.

Rate limiting
Email verification
HTTPS enforcement
Secure HTTP-only cookies
Brute-force protection
Device monitoring
Anomaly detection

Awesome. You can find the complete code on GitHub.

I'm building my own Claude-based chatbot, powered by MongoDB

MongoDB Guests — Thu, 02 Jul 2026 08:39:00 +0000

This tutorial was written by Néstor Daza.

I use Claude every day. It's a great tool for my day-to-day work, helping with everything from research and text processing to, above all, writing code (nothing surprising here). But today, that little coder voice in my head asked me: what would it take to build my own version, running on infrastructure I understand and control? What if I could have my own AI bot? So that is what I am doing, and this is the first entry in a series documenting it.

The app is called Claudius. It is a personal, Claude-style chat workspace I am building for myself, with MongoDB as its entire backbone and Claude models served through AWS Bedrock. Every article in the series will ship with public code. The app runs on MongoDB Atlas’s free tier, so nothing here is a pattern you read about and cannot reach. You can clone it, run it, and lift the parts you want into your own work.

Why build it when Claude already exists?

Yes, yes, Claude already exists, and it is very good, so why build a smaller, less-featured version of it? Because the best teacher is you, building with your own hands! Building the agent loop by writing the code, with state and retrieval, I actually have to figure out how to make it work, and the entire infrastructure needed to run it will not only show me a lot of what a finished product hides, but should also be a really fun project!

There is a personal payoff, too. I want a chat app where I own the data, one that remembers the things I tell it across conversations, sitting in a database I run. It is built for me first, and keeping it personal also keeps the scope honest.

The build is also great content. Writing is part of what I do, so a project I would build anyway turns into a series that other developers can follow. That's the reason you are reading this.

MongoDB is the whole backbone

Having MongoDB as the only datastore in the project is the decision that shapes everything else. Not a database plus a cache plus a vector store plus a queue, just MongoDB.

That is uncommon enough to need some explaining. A chat app like this has to keep users and conversations somewhere. It has to persist the agent's working state so a conversation survives a page reload. Retrieval over your documents and long-term memories needs a vector store. Background work, once it shows up, needs somewhere to queue. The usual answer is to use a separate product for each of those, and you end up operating four systems that have to be kept in sync. I am using MongoDB throughout, because the document model and Atlas Vector Search cover that whole surface, and one database is far less to run and reason about.

Embeddings come from Voyage AI, which is now part of MongoDB. That matters more than it first sounds. The data and the retrieval layer that sits on top of it come from the same place, so there is no second vendor to manage for the part of the app that handles search. Voyage's models are also strongly multilingual, which I care about because I work across AMER and LATAM in English and Spanish, and embedding quality in Spanish is the difference between retrieval that works and retrieval that only pretends to while quietly missing the point.

If you come from a relational background, I will translate and mention parallels and differences as we go. When a document-model choice replaces something you would have built with a join table or an extra service, I will say so, explain it, and show the relational equivalent to make the tradeoff clear.

Why is Claude running through Bedrock?

The one big piece that is not MongoDB is the model layer, and every Claude model the app uses is served through AWS Bedrock. This choice comes down to convenience and ease of use: being able to move from a fast, cheap model to a slower, more capable one is a single string in a request, which makes the model picker in the UI almost free to build. Keeping model access within a single cloud relationship also puts credentials and billing in one place, which matters the moment there is a usage bill to watch.

What it does, and what it isn't

So what does Claudius actually do? Four things only. I'm capping the scope of the whole project right here.

It holds a streamed conversation and lets me switch Claude models in the middle of a thread. It remembers, both within and across conversations, persistent facts instead of hoarding transcripts. It lets me upload a file and ask questions answered from its contents. And it can run a longer research task that plans an approach and works through web sources to return a cited answer.

But it is not a product! There is no commercial plan behind it, no growth target to hit, and more importantly, I'll leave the full-blown version of a Claude Agent to the good folks at Anthropic; my plan is to keep Claudius very slim. Sign-in is Google and nothing else, and a user's role is decided on the server, so no public path hands out anything on its own. It is built for me, and late in this series, I open a small public tier so readers can try the finished thing. That tier is hard-capped but open enough to show you what the end result looks like, so stay tuned for one of the final articles of this series!

This is not a clone!

One very important thing to be clear about: Claudius is a personal homage and a learning project. It is not affiliated with or endorsed by Anthropic, and it is not meant to replace Claude, which remains the real thing. This is me rebuilding the shape of a tool I use and admire, to understand how it stands up, while showcasing the power of MongoDB as the sole data foundation of a complete agentic AI application.

How it will be built

The build runs in six phases, and the article series will track them closely.

Phase 0 lays the foundations, the data model, and the auth/role system that everything else assumes. Phase 1 brings the streaming chat backbone, Claude over Bedrock wired into the frontend, the first real conversation. Phase 2 adds file upload and retrieval for what you upload. Phase 3 is the one I am most looking forward to: long-term memory that survives across conversations. Phase 4 moves the heavier work, including the research agent, onto a background worker. Phase 5 closes the loop with usage metering and cost controls, and opens the small public tier.

Because every phase ships with public code, hosted in Github. The repo carries an annotated .env.example that lists every credential the build needs and the phase it belongs to, so you can reproduce each step instead of just reading it.

When the public tier goes live near the end of the series, it will live at askclaudius.dev, and you will be able to open the finished app and use it yourself.

That's enough on the why. The next article comes with the decision that surprised me most: Claudius has no message table! I'll show you where the conversation really lives, and why the document model is the right call and not just a clever hack. If you came up on SQL, this is the one to read!

Building a Kotlin Application with Quarkus and MongoDB: A Step-by-Step Guide

Ricardo Mello — Tue, 30 Jun 2026 13:43:36 +0000

Kotlin, Quarkus, and MongoDB — do these three technologies work well together?

In this tutorial, we’ll build a simple application that connects to MongoDB using the Panache pattern and performs aggregation queries.

Along the way, we’ll set up the project, explore the structure, and run a basic aggregation pipeline to query a movies dataset.

Prerequisites

Basic knowledge of Kotlin
Maven installed
A MongoDB instance (Atlas or local)
Docker installed and running (optional, if you want to use the local lab setup)

What we will build

We’re going to build a simple application that connects to a database containing around 21,000 movies.
Each movie document has a structure similar to this:

{
 "title": "The Great Train Robbery",
 "fullplot": "Among the earliest existing films in American cinema...",
 "plot": "A group of bandits stage a brazen train hold-up, only to find a determined posse hot on their heels.",
 "genres": [
   "Short",
   "Western"
 ],
 "cast": [
   "A.C. Abadie",
   "Gilbert M. 'Broncho Billy' Anderson",
   "George Barnes",
   "Justus D. Barnes"
 ],
...
}

Our goal is to create an aggregation that selects cast members and counts how many movies each one appears in, returning results like this:

{
 "_id": "Gérard Depardieu",
 "totalMovies": 67
},
{
 "_id": "Robert De Niro",
 "totalMovies": 58
}

Setting up MongoDB (Optional)

To get started, we first need to set up a MongoDB connection.
You have a couple of options:

Create a free MongoDB Atlas account and set up a cluster by following this tutorial
Or use a local MongoDB instance To make things easier, you can use this CLI tool to spin up a local MongoDB environment with sample data already loaded:

1 - Install the cli:

npm install -g @ricardohsmello/mongodb-cli-lab

2 - Create the cluster:

mongodb-cli-lab up --topology standalone --mongodb-version 8.2 --port 28000 --sample-databases sample_mflix

This is a community-driven initiative and not an official MongoDB product.
It is intended for development, demos, and learning purposes only.
Not intended for production use.

This command starts a local MongoDB instance and automatically loads the sample_mflix dataset, which includes the movies collection we’ll use in this tutorial.

Once your MongoDB instance is up and running, make sure you have access to the movies collection, as we’ll use it to build our aggregation queries.

Creating the Quarkus project

Now that our database is ready, let’s create our Quarkus application.
We’ll use the Quarkus Maven plugin to bootstrap the project with the required extensions:

mvn io.quarkus.platform:quarkus-maven-plugin:3.34.1:create \
   -DprojectGroupId=com.example.mongodb.lab \
   -DprojectArtifactId=kotlin-quarkus-lab \
   -Dextensions='kotlin,rest-jackson, quarkus-mongodb-panache-kotlin'

Here, we are including a few important extensions:

Kotlin – to write our application using Kotlin
REST (Jackson) – to expose HTTP endpoints and handle JSON
MongoDB Panache – to simplify data access and interactions with MongoDB

Once the project is created, open it in your favorite IDE to continue.

Configuring the MongoDB Connection

With the project open in your IDE, the first step is to configure the connection to MongoDB.
To do that, open the application.properties file and add the following properties:

%dev.quarkus.mongodb.connection-string=mongodb://localhost:28000
%dev.quarkus.mongodb.database=sample_mflix

These settings configure the application to connect to our local MongoDB instance and use the sample_mflix database.
The %dev prefix means that these properties will only be applied when running the application in development mode.

Defining the Movie entity

Now, we need to create a class to represent our movies collection.
Using Panache, we can map this collection to a Kotlin class Movie.kt, like this:

import io.quarkus.mongodb.panache.common.MongoEntity

@MongoEntity(collection = "movies")
class Movie {
   lateinit var title: String
   var plot: String? = null
   var cast: List<String>? = null
}

The @MongoEntity annotation tells Quarkus which collection this class maps to.
Here, title is defined as a required field using lateinit, since we expect every document to have it.
On the other hand, plot and cast are optional fields, so we declare them as nullable (?) and initialize them with null. This allows the application to handle documents where these fields might be missing.

Creating the Repository

Next, we’ll create a repository to interact with MongoDB using Panache.

import io.quarkus.mongodb.panache.kotlin.PanacheMongoRepository
import jakarta.enterprise.context.ApplicationScoped

@ApplicationScoped
class MovieRepository : PanacheMongoRepository<Movie> {
}

By extending PanacheMongoRepository, we get access to built-in methods to interact with the database, such as queries and data operations.
This is where we’ll implement our aggregation logic.

Exposing the repository through a resource

Next, we’ll interact with this repository through a REST resource:

import jakarta.ws.rs.GET
import jakarta.ws.rs.Path
import jakarta.ws.rs.Produces

@Path("/movies")
@Produces("application/json")
class MovieResource(
   val repository: MovieRepository
) {
   @GET
   fun list() = repository.findAll()
       .list()
}

This resource exposes a /movies endpoint that returns all documents from the collection.

Running the Application and Testing the Endpoint

Now, let’s run the application and test the endpoint.
To make that easier, I’ll create an HTTP file called movie.http and add the following request:

### FIND
GET http://localhost:8080/movies
Accept: application/json

Once we run it, we’ll immediately notice the first problem: we are returning all 21,000 movies at once.

That is not a good idea. We should always be careful with how much data we return, both to avoid unnecessary load on the database and to make the API more efficient.

A simple improvement here is pagination.
So let’s go back to the MovieResource class and add two query parameters: page and size.

import jakarta.ws.rs.DefaultValue
import jakarta.ws.rs.QueryParam

@GET
fun list(
   @QueryParam("page") @DefaultValue("0") page: Int,
   @QueryParam("size") @DefaultValue("20") size: Int,
) = repository.findAll()
   .page(page, size)
   .list()

With this change, the endpoint will return only 20 results by default.
If we run the request again, we’ll get a much smaller, and more manageable response.

Working with the Aggregation Framework

The MongoDB Aggregation Framework is a powerful feature for transforming and analyzing data directly in the database. Instead of just retrieving documents, we can use aggregation stages to reshape the data, group values, sort results, and answer more interesting questions.

In this example, we want to find the actors who appear most frequently in the dataset. To do that, we’ll use the following aggregation pipeline:

[
 {
   $unwind:    
     {
       path: "$cast"
     }
 },
 {
   $group:    
     {
       _id: "$cast",
       totalMovies: {
         $sum: 1
       }
     }
 },
 {
   $sort:
     {
       totalMovies: -1
     }
 },
 {
   $limit: 3      
 }
]

Let’s break it down:

$unwind takes the cast field, which is an array, and splits it into multiple documents. In simple terms, if a movie has three actors in its cast, it will temporarily become three separate records during this stage, one for each actor.
$group then groups those records by actor name and counts how many times each one appears.
$sort orders the results in descending order, so the most frequent actors appear first.
$limit keeps only the top results.

Implementing the Aggregation in the Repository

Now let’s implement this pipeline inside the MovieRepository:

fun findTopFrequentActors(): List<Document> =
   mongoCollection().withDocumentClass(Document::class.java)
       .aggregate(
           listOf(
               Aggregates.unwind("\$cast"),
               Aggregates.group(
                   "\$cast", Accumulators.sum("totalMovies", 1)
               ),
               Aggregates.sort(Sorts.descending("totalMovies")),
               Aggregates.limit(3)
           )
       ).toList()

Here, we are using the underlying MongoDB collection and defining the aggregation stages with the MongoDB Java driver API.

Since the result of this pipeline does not map directly to our Movie class, returning a Document is a simple and practical choice.

Next, let’s expose this aggregation through a new endpoint in our MovieResource:

@GET
@Path("/top-frequent-actors")
fun findTopFrequentActors() = repository.findTopFrequentActors()

With this, our application now provides a dedicated endpoint to return the actors who appear in the highest number of movies.

### TOP FREQUENT ACTORS
GET http://localhost:8080/movies/top-frequent-actors
Accept: application/json

Once you run it, you should see a response similar to this:

[
 {
   "_id": "Gèrard Depardieu",
   "totalMovies": 67
 },
 {
   "_id": "Robert De Niro",
   "totalMovies": 58
 },
 {
   "_id": "Michael Caine",
   "totalMovies": 51
 }
]

And that’s it! We now have a working aggregation pipeline in a Kotlin application using Quarkus, Panache, and MongoDB.

Conclusion

Yes — Kotlin, Quarkus, and MongoDB work very well together in a practical and efficient way.
In this tutorial, we saw how to quickly build an application, map a collection using Panache, and leverage the MongoDB Aggregation Framework to work with data in a powerful yet simple way.

If you’d like to explore further, the full source code is available on GitHub.

Advanced API Authentication Strategies with Node.js and MongoDB (Part 2)

MongoDB Guests — Mon, 29 Jun 2026 09:00:00 +0000

Implementing MFA and TOTP with Node.js and MongoDB

This article was written by Moses Anumadu.

In part one (1) of this series, we implemented basic authentication. Users can register and also log in. In this section, we will take it a step further by adding multi-factor authentication (MFA) using a time-based one-time password (TOTP).

Basic authentication, which uses only a password and email, is arguably not secure enough for today's internet. Many users reuse their passwords, and attackers can exploit this vulnerability.

The core principle of multi-factor authentication is that it combines something the user knows with something the user owns to verify the user's identity. Something the User knows can be a password. Something the user owns can be a token sent to the user's device or email. The user needs to prove he knows the password and own the device where TOTP was sent.

If conditions are met, the user can be verified. With that said, let's get started on how to implement this in a Node.js and MongoDB application.

Pre-requisites

To follow along with this tutorial, ensure you have prior experience working with the following technologies:

Node.js and Express
MongoDB and Mongoose
A MongoDB Atlas account
Postman
Ngrok
Part one of this series

Adding MFA with TOTP

Let's go over what the flow of our MFA implementation would look like. So think of it like this:
First, the authenticated user has to enable MFA in the app. Next, the server generates a shared secret. Next, the user would need to scan a QR code with their phone camera or any of the authenticator apps of their choice (Google Authenticator, Authy, Microsoft Authenticator, etc). Next, the authenticator app would store the secret; the next app generates a 6-digit code. Finally, the server would verify the code. If correct, the user is authenticated.

The important thing to note here is that both the server and the authenticator app share the same secret. Using the current time and that secret, both sides independently generate matching one-time codes.

Creating a Temporary Authentication Middleware

One more step before we proceed. JWT authentication will be implemented in part three (3) of this tutorial. Since we have not implemented it yet, but need a way to identify the logged-in user, we will identify the user using the request header. This is temporary and will be updated to use JWT in part three. For now, create tempAuth.js in src/middleware/ directory. Update the file with the code below

const User = require("../models/User");

const tempAuth = async (req, res, next) => {
  try {
    const email = req.headers["x-user-email"];

    if (!email) {
      return res.status(401).json({
        success: false,
        message: "Missing x-user-email header",
      });
    }

    const user = await User.findOne({ email });

    if (!user) {
      return res.status(401).json({
        success: false,
        message: "User not found",
      });
    }

    req.user = user;

    next();
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Authentication middleware error",
    });
  }
};

module.exports = tempAuth;

This middleware is only temporary and will later be replaced with proper JWT authentication.

Generating MFA Secrets and QR Codes

We now have a way to temporarily identify the user. Let's proceed to the MFA secrets and QR codes implementation. If you can recall, in the first part of this series, we installed all the NPM packages we would need. Two of them were Otplib and Qrcode. It is the package of choice for this implementation. If you do not have them installed already, you can install them using the NPM command below:

npm install otplib

And

npm install --save qrcode

Once you confirm you have them installed, let's import and use them in the authController.js. Open src/controllers/authController.js. Add the code below to the top section of the page.

const QRCode = require("qrcode");

const {
  generateSecret,
  generateURI,
  verify,
} = require("otplib");

Next, add the code below after the login method on the page

const setupMFA = async (req, res) => {
  try {
    const user = req.user;

    // Generate shared secret
    const secret = generateSecret();

    // Generate OTP auth URL
    const otpAuthUrl = generateURI({
      label: user.email,
      issuer: "AdvancedAuthAPI",
      secret,
    });

    // Generate QR code image
    const qrCodeImageUrl =
      await QRCode.toDataURL(otpAuthUrl);

    // Store secret temporarily
    user.mfaSecret = secret;

    await user.save();

    return res.status(200).json({
      success: true,
      message: "MFA setup initialized",
      secret,
      qrCodeImageUrl,
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Failed to setup MFA",
    });
  }
};

module.exports = { register, login, setupMFA};

The code is well commented. Regardless, let's still go over a few lines and what they do. This line generates the shared secret: const secret = generateSecret();

The generated secret is then embedded into a special OTP URI in the block of code below

const otpAuthUrl = generateURI({
  label: user.email,
  issuer: "AdvancedAuthAPI",
  secret,
});

For the user to be able to scan, we converted the URI into a QR code using the qrcode package in the line of code below:

const qrCodeImageUrl =
await QRCode.toDataURL(otpAuthUrl);

When a user scans the QR code with their authenticator app, the authenticator app securely stores the shared secret.

Verifying MFA Codes

We have completed the MFA setup functionality. Let's proceed to the verification functionality. We need another controller method to verify the 6-digit code generated by the authenticator app. This will have a different endpoint also. Let's proceed. Still in src/controllers/authController.js add the following verification controller method below setupMFA:

const verifyMFA = async (req, res) => {
  try {
    const user = req.user;

    const { token } = req.body;

    if (!token) {
      return res.status(400).json({
        success: false,
        message: "MFA token is required",
      });
    }

    const verification = await verify({
      token,
      secret: user.mfaSecret,
    });

    const isValid = verification?.valid === true;

    if (!isValid) {
      return res.status(401).json({
        success: false,
        message: "Invalid MFA code",
      });
    }

    user.mfaEnabled = true;

    await user.save();

    return res.status(200).json({
      success: true,
      message: "MFA enabled successfully",
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Failed to verify MFA",
    });
  }
};

module.exports = { register, login, setupMFA, verifyMFA };

The token is passed along as req.body. When received in the code, it is verified using

const verification = await verify({
       token,
       secret: user.mfaSecret,
     });

If the code is valid, the user is logged in, and MFA is enabled. Did you notice that the MFA is only enabled after a successful verification? user.mfaEnabled = true;. This was done to prevent a broken MFA setup, which can accidentally lock a user out of their account. This prevents broken MFA setups and accidental account lockouts.

Adding MFA Routes

We need to update our authRoutes.js to include routes for both the setupMFA and verifyMFA controller methods. To do this, open src/routes/authRoutes.js and replace the content of the page with the code below

const express = require("express");

const {
  register,
  login,
  setupMFA,
  verifyMFA,
} = require("../controllers/authController");

const tempAuth = require("../middleware/tempAuth");

const router = express.Router();

router.post("/register", register);
router.post("/login", login);
router.post("/mfa/setup", tempAuth, setupMFA);
router.post("/mfa/verify", tempAuth, verifyMFA);

module.exports = router;

Testing MFA Setup

In part one (1) of this series, we set up Ngrok for tunneling our local host to the internet and tested our endpoints using Postman. If you do not have that setup, follow the link to set up to follow along.

If your project is not running already, start it using the command below

npm run dev

Then, send a POST request to the following endpoint:

POST /api/auth/mfa/setup

Remember our temporary middleware, set the header value to the value below

Headers:
x-user-email: test_user@example.com

The response should include a QR code image URL. Check out the images below for more details

Copy the URL from the response and paste it into a browser. You should see a QR code image, like what we have below

Next, scan the QR code using any authenticator app of your choice. After scanning, your authenticator app should begin generating 6-digit codes like the image below.

Testing MFA Verification

Let's proceed to test the MFA verification endpoint. Send a post request to the route below

POST /api/auth/mfa/verify

Also, send the following header values:

x-user-email: test_user@example.com

Then, the request body should contain the following:

{
  "token": "MFA_app_generated_code"
}

Don't forget to replace the token placeholder in the request body with the correct code from your authenticator app.

The images below should throw more light on the steps.

Implementing the Half-Authenticated Flow

Our MFA setup and verification now work. However, it is completely independent of the login and sign-up. We have created routes for the MFA but have not linked it to the login and sign-up flow. What we have now gives a user full authentication once a correct email and password are provided. We need to change that. We need a partial authentication flow: this is what I mean, when a user provides a correct email and password, we need to partially identify the user, and yet, prompt for MFA. Only when MFA is completed can the user be fully authenticated.

To achieve this, we will issue a temporary JWT token to the user after a correct password and email are provided; at this state the user is not yet authenticated but can be identified with this JWT token. Then, when the MFA process is completed, only then will the user be fully authenticated.

When we are done, the flow should be: password verified, issue a temporary JWT token to identify the user, MFA code generated from the authenticator app, the user submits the OTP code, and finally, authentication is completed. Let's proceed.

Creating MFA Tokens

Let's write the code to generate the temporary JWT token for the MFA process. Create src/utils/tokenUtils.js the following file and update it with the code below

const jwt = require("jsonwebtoken");

const generateMFAToken = (user) => {
  return jwt.sign(
    {
      userId: user._id,
      type: "mfa",
    },
    process.env.JWT_ACCESS_SECRET,
    {
      expiresIn: process.env.MFA_TOKEN_EXPIRES_IN,
    }
  );
};

module.exports = {
  generateMFAToken,
};

In the code above, we imported require("jsonwebtoken"); and used it to generate the JWT token. Token expiration and secret values come from the env file created earlier in part one of this series.

Updating the Login Flow

We need to update the login controller method to add the generated MFA JWT token. To continue, open src/controllers/authController.js and add the following.

First, add the import below to the top section of the page.

const jwt = require("jsonwebtoken");

const {
  generateMFAToken,
} = require("../utils/tokenUtils");

Next, let's update the login controller with the code below. Paste the code just above the return statement.

// MFA enabled
if (user.mfaEnabled) {
  const mfaToken = generateMFAToken(user);

  return res.status(200).json({
    success: true,
    requiresMFA: true,
    mfaToken,
  });
}

Now, users with MFA enabled no longer receive full authentication immediately after password verification. Instead, they receive a temporary MFA token.

Completing MFA Login

We need another controller method to process the MFA verification. Add the following controller below verifyMFA method:

const verifyMFALogin = async (req, res) => {
  try {
    const { token, mfaToken } = req.body;

    if (!token || !mfaToken) {
      return res.status(400).json({
        success: false,
        message:
          "Token and MFA token are required",
      });
    }

    // Verify temporary MFA token
    const decoded = jwt.verify(
      mfaToken,
      process.env.JWT_ACCESS_SECRET
    );

    if (decoded.type !== "mfa") {
      return res.status(401).json({
        success: false,
        message: "Invalid MFA session",
      });
    }

    const user = await User.findById(
      decoded.userId
    );

    if (!user || !user.mfaEnabled) {
      return res.status(401).json({
        success: false,
        message: "Invalid user",
      });
    }

    // Verify OTP
    const verification = await verify({
      token,
      secret: user.mfaSecret,
    });

    const isValid =
      verification?.valid === true;

    if (!isValid) {
      return res.status(401).json({
        success: false,
        message: "Invalid MFA code",
      });
    }

    return res.status(200).json({
      success: true,
      message: "MFA login successful",
    });
  } catch (error) {
    console.error(error);

    return res.status(401).json({
      success: false,
      message:
        "Invalid or expired MFA session",
    });
  }
};

Also, don't forget to update the export block with the code below.

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
};

Adding the MFA Login Verification Route

Let's create routes for the above changes. Open src/routes/authRoutes.js and update the page with the code below:

router.post(
  "/mfa/login/verify",
  verifyMFALogin
);

Testing the Full MFA Login Flow

The first step is to log in normally. Send a Post request to the route below

POST /api/auth/login

If MFA is enabled, the response should contain an mfaToken just like the image below:

Next, verify MFA by sending a Post request to the route below

POST /api/auth/mfa/login/verify

The request should pass the code below in the body.
Request body:

{
  "token": "123456",
  "mfaToken": "..."
}

After testing, your response should look similar to the image below.

Conclusion

In this section, we added an extra layer on top of the existing authentication system we built in part one. After entering the correct email and password, users must configure and verify MFA using TOTP. To achieve this, we implemented the following in the code:

login verification
TOTP-based MFA
QR code onboarding
half-authenticated login flows

We've gone halfway, but we're not done. In Part 3, we will take it a step further by adding

JWT access tokens
refresh tokens
MongoDB-backed sessions
refresh token rotation
protected routes
secure logout and session revocation

See you in part 3. Cheers.

Advanced API Authentication Strategies with Node.js and MongoDB (Part 1)

MongoDB Guests — Thu, 25 Jun 2026 09:00:00 +0000

This article was written by Moses Anumadu.

Building Password Authentication with Node.js and MongoDB

Most database-driven applications require users to be authenticated. This is not a surprise. The surprise might come when you hear about the number of cyberattacks carried out daily and their complexity. According to Akamai Technologies, there are over 300 million and over 500 million credential-stuffing attacks per day. Given this volume of attacks, it is reasonable for applications to add more layers of authentication (MFA) to the traditional email-and-password authentication.

This three-part tutorial will guide you through building a production-style authentication system that goes beyond the basic signup and login using Node.js, Express, and MongoDB Atlas.

In this first part (Part 1) of the three-part series, we will implement the foundation of the authentication that we will build upon throughout the rest of the series. We will implement the following:

A secure password hashing with bcrypt.
User registration with email and password.
User logs in with email and password.

In Part 2, we will continue building on the existing codebase. We will add secure session management to the code by implementing the following:

Login verification
TOTP-based MFA setup and verification
QR code onboarding for TOTP setup
Half-authenticated login flows

And in Part 3, we will make it an authentication that you can deploy or use in your real-world application by implementing the following:

JWT access tokens
refresh tokens
MongoDB-backed sessions
refresh token rotation
protected routes
secure logout and session revocation

Pre-requisites

To follow along with this tutorial, ensure you have your development environment set up for the stack above and:

An understanding of Node.js and Express
An Understanding of MongoDB
A MongoDB Atlas account
Postman for testing API endpoints

Alright, let's get started.

Project Setup

To get started, we need to create a fresh Node.js project and install all the necessary dependencies. In your directory of choice, create the project using the command below:

mkdir devrel-node-api-authentication
cd devrel-node-api-authentication
npm init -y

An advanced authentication system like the one we are building relies on several dependencies in Node.js. We’ll install a few packages to handle those responsibilities. From your terminal, navigate to the project directory and then install dependencies using the command below

npm install express mongoose dotenv bcryptjs jsonwebtoken otplib qrcode && npm install --save-dev nodemon

Let's take a moment to go over all the packages we just installed. Express provides the web server and routing system for building our API, while Mongoose letting us interact with MongoDB through models and schemas. dotenv helps us load environment variables such as database credentials and secret keys. bcryptjs securely hashes passwords before storing them in the database and jsonwebtoken enables us to create and verify JWT tokens for authentication. For MFA functionality, otplib generates and validates time-based one-time passwords (TOTP), while qrcode converts our MFA setup information into a scannable QR code for authenticator apps. Finally, nodemon improves the development experience by automatically restarting the server whenever code changes are detected.

After the installation, we need to update the script section in our package.json file with the code below:

"scripts": {
  "dev": "nodemon src/server.js"
}

This would enable us to serve the project using npm run dev

Creating the Project Structure

We are making progress. Let's proceed by creating a structure for the project. In your project, create the following files and folders. The project structure should look like the sample below.

src/
├── config/
├── controllers/
├── middleware/
├── models/
├── routes/
├── utils/
├── app.js
└── server.js

Configuring MongoDB Atlas

Next, let's configure our application to interact with our MongoDB Atlas cluster. Ensure you have your Atlas connection string. If you do not have an Atlas account, follow the link to sign up and create an Atlas cluster, then obtain your connection string. With that said, create a


 file in the project root directory and update the content with the code below:


```bash
MONGO_URI_BASE=mongodb+srv://USERNAME:PASSWORD@cluster0.xxxxx.mongodb.net

JWT_ACCESS_SECRET=supersecretaccesskey
JWT_REFRESH_SECRET=supersecretrefreshkey

ACCESS_TOKEN_EXPIRES_IN=15m
REFRESH_TOKEN_EXPIRES_DAYS=7
MFA_TOKEN_EXPIRES_IN=5m

PORT=5000

Now that we have our Atlas connection string, let's proceed to connect our application to MongoDB using Mongoose. Create src/config/db.js and update the file with the code below.

const mongoose = require("mongoose");

const databaseName = "devrel-node-auth-db";
const appName = "devrel-tutorial-nodejs-authentication-geeksforgeeks";

const connectDB = async () => {
try {
  const mongoUri = `${process.env.MONGO_URI_BASE}/${databaseName}?retryWrites=true&w=majority&appName=${appName}`;

  await mongoose.connect(mongoUri);

  console.log("MongoDB connected");
  console.log(`Database: ${databaseName}`);
  console.log(`App Name: ${appName}`);
} catch (error) {
  console.error("MongoDB connection failed:");
  console.error(error.message);

  process.exit(1);
}
};

module.exports = connectDB;

Notice that the database name and appName are defined directly inside the codebase rather than inside the environment file. This keeps infrastructure-level configuration consistent across environments.

Creating the Express App

Now that we have the database configuration done, let's proceed to create an Express application. To keep things organized, we will create the app instance and register global middlewares in src/app.js. Update the file with the code below.

const express = require("express");
const authRoutes = require("./routes/authRoutes");
const app = express();

app.use(express.json());

app.use("/api/auth", authRoutes);
module.exports = app;

In src/app.js, we imported and used the authRoutes.js file. Let's go ahead and create this file before we proceed.
In src/routes/ directory, create the authRoutes.js file and update it with the code below:

const express = require("express");

const router = express.Router();

router.get("/health", (req, res) => {
 res.status(200).json({ message: "Auth routes are working" });
});

module.exports = router;

We created a simple /health route to verify that the application setup was correct.

Creating the Server Entry Point

Next, in copy and paste the code below into the src/server.js file.

const dotenv = require("dotenv");
const app = require("./app");
const connectDB = require("./config/db");

dotenv.config();

const PORT = process.env.PORT || 5000;

const startServer = async () => {
 await connectDB();
 app.listen(PORT, () => {
   console.log(`Server running on port ${PORT}`);
 });
};

startServer();

Now, we can test the configuration so far. In your project directory, start the development server using the command below:

npm run dev

If everything is configured correctly, you should see the following from your terminal:

MongoDB connected
Server running on port 5000

Creating the User Model

With all the configurations done, let's jump to the exciting part. So, the first step in authentication is creating a user. We need to implement code to create a user and securely store the password using bcrypt for Node.js in our MongoDB collection.

Create src/models/User.js file and update the file with the code below:

const mongoose = require("mongoose");

const userSchema = new mongoose.Schema(
 {
   email: {
     type: String,
     required: true,
     unique: true,
     lowercase: true,
     trim: true,
   },

   passwordHash: {
     type: String,
     required: true,
   },

   mfaEnabled: {
     type: Boolean,
     default: false,
   },

   mfaSecret: {
     type: String,
     default: null,
   },
 },
 {
   timestamps: true,
 }
);

module.exports = mongoose.model("User", userSchema);

From the code above, we created a User model. Mongoose was used to create our User schema and User model. One more thing to note is that, according to the schema, we are storing passwordHash instead of password. PasswordHash ensures that the password is not stored in plain text but is hashed using bcrypt.

Building User Registration

Let's proceed to the actual logic to create a new user. In src/controllers/authController.js, copy and paste the code below.

const bcrypt = require("bcryptjs");
const User = require("../models/User");

//Now add the registration controller:

const register = async (req, res) => {
  try {
    const { email, password } = req.body;

    // Basic validation
    if (!email || !password) {
      return res.status(400).json({
        success: false,
        message: "Email and password are required",
      });
    }

    // Password length validation
    if (password.length < 6) {
      return res.status(400).json({
        success: false,
        message: "Password must be at least 6 characters",
      });
    }

    // Prevent duplicate accounts
    const existingUser = await User.findOne({ email });

    if (existingUser) {
      return res.status(409).json({
        success: false,
        message: "User already exists",
      });
    }

    // Hash password
    const passwordHash = await bcrypt.hash(password, 12);

    // Create user
    const user = await User.create({
      email,
      passwordHash,
    });

    return res.status(201).json({
      success: true,
      message: "User registered successfully",
      user: {
        id: user._id,
        email: user.email,
      },
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Server error",
    });
  }
};
module.exports = { register };

From the code above, you can see that we validated for empty email and password values, password length, and prevented duplicate accounts by checking whether the email already exists. The next important thing to note is that the password was hashed using bcrypt
const passwordHash = await bcrypt.hash(password, 12);.

Bcrypt automatically handles salting, secure hashing, and computational cost. The second parameter (12) represents the salt rounds value. The higher this number, the more difficult it is for an attacker to crack this password.

Updating Authentication Routes

Let's create a route link for the Register logic. We already created the src/routes/authRoutes.js file. Now open the file and update the content with the code below.

const express = require("express");

const {
  register,
} = require("../controllers/authController");

const router = express.Router();

router.post("/register", register);

module.exports = router;

Testing Registration

It's time to test what we have built so far. For this tutorial, I will be testing with Postman on the web and tunneling my local environment using Ngrok. Feel free to test with what works for you or follow along with me.

If your server is not already running, navigate to your project directory and start the server with the command below;

npm run dev

In a different terminal window, start Ngrok using the command below:

Ngrok http 5000

You should expect an output similar to the image below:

Ngrok tunnels our localhost:5000 URL to https://b76b-197-157-218-195.ngrok-free.app or something similar. The Ngrok URL will be used to test in Postman, not our localhost URL.

To test, send a POST request to the register route: /api/auth/register. Also, add the following to the request body:

{
  "email": "test_user@example.com",
  "password": "password123"
}

Your output should be similar to the image below:

Try creating a duplicate user with the same record, and notice you should get a response similar to the image below

If you inspect your MongoDB Atlas database, you should now see a users collection containing the new user document with an encrypted password.

Building Password Authentication

Now, users can register. Let's proceed to Login. Registered users should be able to securely login to the system using their email and password. Still in src/controllers/authController.js, just below the register function, copy and paste the code below.

const login = async (req, res) => {
  try {
    const { email, password } = req.body;

    // Basic validation
    if (!email || !password) {
      return res.status(400).json({
        success: false,
        message: "Email and password are required",
      });
    }

    // Find user
    const user = await User.findOne({ email });

    // Prevent email enumeration attacks
    if (!user) {
      return res.status(401).json({
        success: false,
        message: "Invalid credentials",
      });
    }

    // Compare password with stored hash
    const isPasswordValid = await bcrypt.compare(
      password,
      user.passwordHash
    );

    if (!isPasswordValid) {
      return res.status(401).json({
        success: false,
        message: "Invalid credentials",
      });
    }

    return res.status(200).json({
      success: true,
      message: "Login successful",
      user: {
        id: user._id,
        email: user.email,
        mfaEnabled: user.mfaEnabled,
      },
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Server error",
    });
  }
};

Also, update the export, add the register function we just created, like so:

module.exports = {
  register,
  login,
};

In the code, we did a number of validations. The first one checked for empty strings, then we checked if this user exists in the database. Did you notice that both invalid email and invalid password return the same response: "Invalid credentials"? This prevents attackers from discovering which email addresses exist in the system. After that, we compared the input password with the password in the database. Also note that the password was not decrypted, it was instead compared against the encrypted version already stored in our database.

Adding the Login Route

Next, let's add a route for the login. Replace the code in src/routes/authRoutes.js with the code below to create a route for both /register and /login.

const express = require("express");

const {
  register,
  login,
} = require("../controllers/authController");

const router = express.Router();

router.post("/register", register);
router.post("/login", login);

module.exports = router;

Testing Login

To test, send a POST request to the login route: /api/auth/login. Also, add the following to the request body:

{
  "email": "test_user@example.com",
  "password": "password123"
}

We are making progress. We now have a very basic authentication system with Login and Register features. This is obviously nowhere near battle-ready for today's internet. So, we will proceed and strengthen this system with Multi-Factor Authentication (MFA).

Conclusion

In this part, we kept it simple and implemented user registration and login. Users can register with an email and a password; a JWT token is not issued yet at this stage. The system now only knows that the user's email and password are valid. To achieve this, we implemented the following:

A secure password hashing with bcrypt.
User registration with email and password.
User logs in with email and password. In the next part ( 2), we will build upon the foundation we create here by adding secure session management using:
login verification
TOTP-based MFA setup and verification
QR code onboarding
half-authenticated login flows See you in part 2.

.NET Server Side Events with MongoDB Change Streams

MongoDB Guests — Fri, 19 Jun 2026 09:00:00 +0000

This tutorial was written by Kevin Smith.

Real-time updates with almost no plumbing

If you want real-time browser updates in ASP.NET Core, you usually think of WebSockets first.

But there is a very nice middle-ground that is often enough for dashboard-style or feed-style UIs:

MongoDB Change Streams to detect database changes
Server-Sent Events (SSE) to push those changes to the browser

That gives you one-way, server-to-client streaming with a very small amount of code.

This post walks through the exact demo app and then covers where SSE shines, where it falls short, and when to move up to SignalR.

Demo in action

Here is the browser demo in action:

What are Server-Sent Events?

Server-Sent Events are a browser API (EventSource) built for receiving a stream of text events over a single HTTP connection.

Think of SSE as:

a long-lived HTTP response
UTF-8 text messages pushed by the server
automatic client reconnect support

SSE is one-way only (server to browser), which is often exactly what you need for:

live timelines
notifications
audit/activity feeds
admin dashboards

In ASP.NET Core Minimal APIs, this is now very straightforward using TypedResults.ServerSentEvents(...).

What are MongoDB Change Streams?

MongoDB Change Streams let you subscribe to real-time changes in collections, databases, or entire deployments.

Instead of polling MongoDB every few seconds, you can react to actual insert/update/delete operations as they happen.

At a high level:

your app opens a change stream cursor
MongoDB emits change events
your app maps each change into something your client can consume

For real-time systems, this removes a lot of waste from polling loops.

Wiring it together in ASP.NET Core

Let us build this from scratch.

1) Create the app

dotnet new web -n MongoDbServerSideEvents

cd MongoDbServerSideEvents

dotnet add package MongoDB.Driver

dotnet new web gives us a minimal ASP.NET Core app, which is perfect for a small real-time demo.

2) Replace Program.cs

Use this as your starting point:

using System.Runtime.CompilerServices;

using MongoDB.Bson;

using MongoDB.Bson.Serialization;

using MongoDB.Bson.Serialization.Attributes;

using MongoDB.Driver;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSingleton<IMongoClient>(_ =>

    new MongoClient("mongodb://host.docker.internal:27017"));

builder.Services.AddSingleton<IMongoDatabase>(sp =>

    sp.GetRequiredService<IMongoClient>().GetDatabase("test"));

builder.Services.AddSingleton<IMongoCollection<Order>>(sp =>

    sp.GetRequiredService<IMongoDatabase>().GetCollection<Order>("orders"));

var app = builder.Build();

app.MapPost("/orders", async (

    IMongoCollection<Order> orders,

    Order order,

    CancellationToken cancellationToken) =>

{

    await orders.InsertOneAsync(order, cancellationToken: cancellationToken);

    return TypedResults.Ok();

});

app.MapGet("/orders", (IMongoCollection<Order> orders) =>

{

    async IAsyncEnumerable<SseItem<Order>> StreamOrders(

        [EnumeratorCancellation] CancellationToken cancellationToken)

    {

        var cursor = await orders.WatchAsync(

            new EmptyPipelineDefinition<ChangeStreamDocument<Order>>()

                .Match(x =>

                    x.OperationType == ChangeStreamOperationType.Insert),

            new ChangeStreamOptions

            {

                FullDocument = ChangeStreamFullDocumentOption.UpdateLookup

            },

            cancellationToken);

        while (await cursor.MoveNextAsync(cancellationToken))

        {

            foreach (var change in cursor.Current)

            {

                if (change.FullDocument is not null)

                {

                    yield return new SseItem<Order>(change.FullDocument, "order")

                    {

                        EventId = EncodeResumeToken(change.ResumeToken)

                    };

                }

            }

        }

    }

    return TypedResults.ServerSentEvents(StreamOrders());

});

app.UseStaticFiles();

app.Run();

static string EncodeResumeToken(BsonDocument resumeToken)

{

    return Convert.ToBase64String(resumeToken.ToBson());

}

static BsonDocument DecodeResumeToken(string eventId)

{

    var bytes = Convert.FromBase64String(eventId);

    return BsonSerializer.Deserialize<BsonDocument>(bytes);

}

record Order(

    [property: BsonRepresentation(BsonType.ObjectId)] string? Id,

    string Name,

    DateTime Date);

There are three important ideas in this code:

We write new orders using a normal HTTP POST endpoint.
We open a MongoDB change stream and yield each incoming change.
We expose that stream to browsers as SSE on the same route (GET /orders).

Notice that both write and streaming code paths use cancellation tokens so work can stop quickly if the request is aborted or the app is shutting down.

2.1) Continue a stream with ResumeAfter and SSE EventId

One really useful feature of MongoDB Change Streams is that you can continue from a known point in the stream instead of always starting "from now".

In .NET, SseItem<T> gives us an EventId field. We can put the MongoDB resume token in there and set the event type to order.

When the browser reconnects, EventSource automatically sends the last event id in the Last-Event-ID header. We can decode that value and pass it to ChangeStreamOptions.ResumeAfter.

In practice, this means:

Capture the change.ResumeToken for each event you process.
Encode it and set SseItem.EventId.
Read Last-Event-ID on reconnect and map it to ResumeAfter.

Here is a minimal version:

app.MapGet("/orders", (

    IMongoCollection<Order> orders,

    HttpRequest request,

    CancellationToken cancellationToken) =>

{

    async IAsyncEnumerable<SseItem<Order>> StreamOrders(

        [EnumeratorCancellation] CancellationToken streamCancellationToken)

    {

        var lastEventId = request.Headers["Last-Event-ID"].ToString();

        var options = new ChangeStreamOptions

        {

            FullDocument = ChangeStreamFullDocumentOption.UpdateLookup,

            ResumeAfter = string.IsNullOrWhiteSpace(lastEventId)

                ? null

                : DecodeResumeToken(lastEventId)

        };

        var cursor = await orders.WatchAsync(

            new EmptyPipelineDefinition<ChangeStreamDocument<Order>>()

                .Match(x =>

                    x.OperationType == ChangeStreamOperationType.Insert),

            options,

            streamCancellationToken);

        while (await cursor.MoveNextAsync(streamCancellationToken))

        {

            foreach (var change in cursor.Current)

            {

                if (change.FullDocument is not null)

                {

                    yield return new SseItem<Order>(change.FullDocument)

                    {

                        EventId = EncodeResumeToken(change.ResumeToken),

                        EventType = "order"

                    };

                }

            }

        }

    }

    return TypedResults.ServerSentEvents(StreamOrders(cancellationToken));

});

The key point is that ResumeAfter is not an arbitrary ID; it is the opaque token generated by MongoDB for each change event.

If you are using browsers, Last-Event-ID gives you a clean reconnect story. For non-browser consumers, you can still persist the token yourself and pass it back the same way.

3) Add a simple browser client

Createwwwroot/index.html and add:

<!DOCTYPE html>

<html lang="en">

<head>

    <meta charset="UTF-8">

    <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <title>Orders</title>

</head>

<body>

<h1>Orders</h1>

<div id="status">Connecting...</div>

<form id="orderForm">

    <input type="text" id="name" placeholder="Order name" required />

    <input type="datetime-local" id="date" required />

    <button type="submit">Create Order</button>

</form>

<ul id="orders"></ul>

<script>

    const ordersList = document.getElementById("orders");

    const status = document.getElementById("status");

    function addOrder(order) {

        const li = document.createElement("li");

        li.innerHTML = `<strong>${order.name}</strong><br>${new Date(order.date).toLocaleString()}`;

        ordersList.prepend(li);

    }

    const eventSource = new EventSource("/orders");

    eventSource.onopen = () => {

        status.textContent = "Connected";

    };

    eventSource.onerror = () => {

        status.textContent = "Disconnected";

    };

    eventSource.addEventListener("order", event => {

        addOrder(JSON.parse(event.data));

    });

    document.getElementById("orderForm").addEventListener("submit", async e => {

        e.preventDefault();

        const name = document.getElementById("name").value;

        const date = document.getElementById("date").value;

        const response = await fetch("/orders", {

            method: "POST",

            headers: { "Content-Type": "application/json" },

            body: JSON.stringify({ name, date })

        });

        if (!response.ok) {

            alert("Failed to create order");

        }

    });

</script>

</body>

</html>

4) Run it

dotnet run

Open the app, create a few orders, and you should see each change streamed into the list in real time.

SSE vs SignalR: where each fits

SSE is excellent when you want a lightweight one-way push channel from server to browser. It is easy to reason about, browser-native, and usually simpler to operate than a full duplex setup.

The trade-off is that SSE is intentionally limited. It is text-based, one-way, and less ergonomic when your app grows into richer real-time interactions like rooms, server calls from clients over persistent connections, or advanced client coordination.

This is where SignalR starts to earn its place. SignalR gives you a higher-level real-time programming model, automatic transport negotiation (WebSockets, SSE, Long Polling), groups, hubs, and a mature ecosystem for scaling out.

Topic	SSE	SignalR	Practical guidance
Direction	Server -> client	Server <-> client	If your UI mostly listens, SSE is often enough
Client support	Native EventSource	SignalR client library	If you want richer interaction patterns, SignalR is easier
Payload style	Text/event-stream	Structured hub messages	If you need binary or richer protocols, lean SignalR
Infrastructure	Simple HTTP streaming	More moving parts, more capability	Choose based on complexity, not fashion
Scale-out patterns	Manual design choices	Well-known backplane patterns	SignalR is usually easier at larger scale

A practical way to choose is:

Start with SSE when you need fast, simple, server-to-browser updates.
Move to SignalR when you need bi-directional messaging or richer collaboration semantics.

Also keep in mind one MongoDB-specific caveat regardless of transport choice: Change Streams require a replica set or sharded cluster. A standalone MongoDB server will not work.

Production hardening ideas

If you take this pattern to production, consider:

project events into lightweight DTOs rather than streaming full documents
include cancellation support and proper disposal around streaming cursors
add structured logging for stream open/close/reconnect behavior
filter change streams as early as possible to reduce noise
protect endpoints with auth and authorize at data scope
keep an eye on proxy settings for idle connection timeouts

Moving towards SignalR

SSE is a really strong starting point. But you should move to SignalR when you need more than one-way feeds.

SignalR helps when you need:

true two-way communication
protocol negotiation (WebSockets with fallbacks)
groups and richer hub-based interaction patterns
scale-out patterns that are already well trodden in ASP.NET Core ecosystems

A natural progression can look like this:

Start with SSE + MongoDB Change Streams for straightforward live updates.
Grow into SignalR when clients need to send real-time messages back over the same channel.
Add a backplane when you scale out across multiple app instances.

If you want to continue using MongoDB in that architecture, you can use Kevsoft.AspNetCore.SignalR.MongoDB as a SignalR backplane.

That gives you the flexibility of SignalR transports (WebSockets, SSE, Long Polling) while still leveraging MongoDB as part of your real-time pipeline.

Final thoughts

For many apps, the pairing of MongoDB Change Streams and SSE is a sweet spot:

minimal moving parts
small amount of code
fast path to real-time UX

Then when requirements evolve, SignalR is a natural next step rather than a rewrite from scratch.

Start simple, get value quickly, and step up transport complexity only when the product actually needs it.

Bringing MongoDB Atlas and Voyage AI to Dify: Build RAG Workflows and Data Agents Without Heavy Glue Code

Pash10g — Sun, 31 May 2026 06:24:29 +0000

AI applications are moving quickly from simple chatbots to systems that can search, reason, recommend, summarize, and act on live business data. For developers, that usually means wiring together databases, embedding models, vector search, rerankers, orchestration logic, and application code. For no-code AI builders, it often means waiting for those integrations to exist before an idea can become a working prototype.

The MongoDB extensions for Dify help close that gap.

With the new MongoDB Atlas and Voyage AI extensions, Dify builders can visually compose AI workflows and agents that connect directly to MongoDB data, perform semantic retrieval with Atlas Vector Search, improve result quality with Voyage AI embeddings and reranking, and optionally interact with operational documents through controlled database tools.

The result is a practical path from idea to working AI application: less custom orchestration code, more reusable building blocks, and a smoother experience for both developers and no-code builders.

Why Dify and MongoDB Belong Together

Dify provides a visual environment for building AI apps, workflows, and agents. It makes it easy to connect user input, model calls, tools, prompts, and outputs into a working application. MongoDB Atlas provides the data foundation: flexible documents, operational queries, aggregation, full-text search, and vector search in one platform.

Together, they create a powerful pattern:

Dify orchestrates the AI experience — workflows, agents, prompts, tools, and user interactions.
MongoDB Atlas stores and retrieves the data — documents, application records, knowledge sources, and vector embeddings.
Voyage AI improves retrieval quality — embeddings for semantic search and reranking for precision.

For a no-code builder, this means you can assemble a retrieval-augmented generation workflow visually. For a developer, it means the integration points are packaged as reusable Dify tools rather than one-off glue code.

Meet the Extensions

The extension set includes two complementary pieces.

MongoDB Atlas Tool Extension

The MongoDB Atlas tool extension exposes MongoDB operations as Dify tools. These tools let workflows and agents interact with MongoDB collections directly from the Dify canvas.

Available capabilities include:

Finding documents
Running aggregation pipelines
Performing Atlas Vector Search
Performing full-text search
Inserting documents
Updating documents
Deleting documents

This is useful for more than just retrieval. It enables agents that can inspect data, summarize records, recommend actions, and — when safely configured — update operational collections.

For example, a project management agent can search a database of team members, skills, previous projects, and availability, then recommend the best team for a new initiative. With carefully scoped permissions, that same agent could also update a draft team assignment or write a recommendation record back to MongoDB.

Voyage AI Extension

The Voyage AI extension adds embedding and reranking tools to Dify.

Embeddings convert text into vectors so MongoDB Atlas Vector Search can find semantically similar documents. Reranking takes an initial set of retrieved documents and reorders them by relevance to the user’s query.

That two-step retrieval pattern matters. Vector search is excellent for finding likely candidates quickly, while reranking helps surface the best candidates before the final answer is generated or returned.

The MongoDB-RAG Template

The included MongoDB RAG template demonstrates how these extensions work together in a Dify workflow.

At a high level, the pipeline does the following:

Accepts user input
Embeds the query with Voyage AI
Searches MongoDB Atlas using Atlas Vector Search
Reranks the retrieved documents with Voyage AI
Formats the results into a prompt-ready output

This is the core pattern behind many production-grade RAG systems.

Instead of sending a user question directly to an LLM and hoping the model already knows the answer, the workflow first retrieves relevant information from MongoDB. The retrieved context can then be used by a downstream answer node, chat model, or agent to produce a more grounded response.

How the Workflow Works

The MongoDB RAG workflow is intentionally simple and reusable. It separates each retrieval step into a dedicated node so builders can understand, tune, and replace parts of the pipeline as needed.

1. User Input

The workflow starts with a text input. This could be a question, a search phrase, a support request, a project description, or any natural-language query.

Example:

What would be a good team to build scalable Rust applications?

2. Embed the Query

The input is sent to the Voyage AI embedding tool. The embedding model converts the text into a vector representation that captures semantic meaning.

For search use cases, the embedding input type should be optimized for queries. This helps improve retrieval quality because the model understands that the text represents a search intent rather than a document to be indexed.

3. Search MongoDB Atlas

The generated query vector is passed to the MongoDB Atlas Vector Search tool. Atlas compares the query vector against document embeddings stored in a MongoDB collection and returns the nearest semantic matches.

The template uses two important retrieval settings:

numCandidates: how many approximate nearest-neighbor candidates Atlas considers before returning final results.
limit: how many results are passed forward to the next step.

Increasing candidates can improve recall, while lowering them can reduce latency. This gives builders and developers a clear tuning knob depending on the application’s needs.

4. Rerank the Results

The top vector search results are then sent to the Voyage AI reranking tool. Reranking compares the original user query against each candidate document and sorts the documents by relevance.

This step is especially valuable when the first-stage vector search returns many plausible matches. Reranking helps the workflow prioritize the documents most likely to answer the user’s actual question.

5. Format the Output

Finally, the template node formats the reranked documents into a structured output. That output can be returned directly, or it can become context for a downstream LLM answer node.

This makes the template flexible. It can be used as a standalone search pipeline, or as the retrieval layer inside a larger Dify chatbot, workflow, or agent.

What No-Code AI Builders Can Create

For no-code builders, the biggest advantage is composability. Instead of implementing a RAG backend from scratch, you can drag tools into a Dify workflow and connect them visually.

With these extensions, builders can create:

Knowledge-base assistants that answer questions from MongoDB documents
Support copilots that search prior cases and recommend resolutions
Project management agents that recommend teams based on skills and history
Document search apps that combine semantic and full-text retrieval
CRM or account assistants that retrieve relevant customer information
Operations agents that read from MongoDB and create structured recommendations

The same building blocks can support simple workflows or more autonomous agents. A workflow might only retrieve and format context. An agent might decide when to search, when to aggregate, and when to update a document — depending on the tools you enable.

What Developers Get

Developers still benefit from the visual experience, but the value goes deeper.

These extensions reduce the amount of custom integration code required to connect Dify with MongoDB Atlas and Voyage AI. Instead of hand-building every request, response parser, embedding call, and database operation, developers can rely on packaged tools with clear inputs and outputs.

The architecture also follows a clean separation of concerns:

Embedding is handled by the Voyage AI embed tool.
Retrieval is handled by MongoDB Atlas Vector Search.
Precision tuning is handled by the Voyage AI rerank tool.
Formatting is handled by the Dify template node.
Application behavior is handled by Dify workflows or agents.

That separation makes the system easier to debug and extend. Developers can tune vector search without changing reranking. They can swap embedding models without rewriting MongoDB logic. They can add an LLM answer node without changing the retrieval pipeline.

Example: A Project Management Agent

One example use case is a project management agent that recommends a team for a new project.

A user might ask:

What would be a good team to build scalable Rust applications?

The agent can use semantic search to find relevant candidates, previous projects, skills, and experience stored in MongoDB. It can then assemble a recommendation that explains why each person fits the project.

In a Dify agent setup, MongoDB tools can be made available alongside the RAG workflow. The agent can search documents, inspect structured records, run aggregations, and produce a recommendation grounded in database results.

This pattern is useful because business data is rarely just static documentation. It often includes operational records: people, cases, accounts, tickets, projects, tasks, products, and events. MongoDB allows that data to remain flexible and queryable, while Dify makes it accessible to AI workflows and agents.

Best Practices for Building with These Extensions

To get the best results, keep a few practical guidelines in mind.

Use the Right Embedding Mode

When embedding user questions for retrieval, use query-optimized embeddings. When embedding documents for storage, use document-optimized embeddings if the model supports it. This improves the alignment between search queries and indexed content.

Tune Vector Search for Recall and Latency

Atlas Vector Search settings such as numCandidates and limit affect both result quality and performance. A larger candidate pool can improve recall, but may increase latency. Start with sensible defaults, then tune based on your dataset and user experience goals.

Rerank Before Generating

Reranking helps improve the quality of the context that reaches the final model. This can reduce irrelevant context, improve answer accuracy, and make the final output easier to trust.

Scope Write Tools Carefully

MongoDB insert, update, and delete tools are powerful. When exposing them to agents, use careful scoping, clear instructions, and appropriate permissions. Many applications should start with read-only tools, then add mutation capabilities only when the workflow and safety boundaries are well understood.

Keep Indexes Aligned with Your Data

For vector search, the Atlas index should match the embedding field and embedding dimensions used by your model. For full-text search, index the fields users are likely to search. Good indexing turns a promising prototype into a responsive application.

Why This Matters

The value of these extensions is not just that Dify can call MongoDB or Voyage AI. The value is that builders can now compose a complete AI retrieval and data-interaction pattern inside Dify:

Search semantically across MongoDB documents
Rerank results for precision
Feed grounded context into an LLM
Let agents inspect and operate on database records
Reuse the same tools across multiple apps and workflows

For no-code builders, this means faster experimentation and fewer blockers. For developers, it means a cleaner integration surface and less repetitive orchestration work.

Conclusion

The MongoDB Atlas and Voyage AI extensions make Dify a stronger platform for building data-aware AI applications. They bring together visual AI orchestration, operational MongoDB data, Atlas Vector Search, full-text search, embeddings, reranking, and agent tools in a way that is approachable for no-code builders and credible for developers.

The template shows the foundation: embed a query, retrieve relevant documents from MongoDB Atlas, rerank them, and format the result. From there, teams can build knowledge assistants, recommendation agents, support copilots, document search experiences, and operational AI workflows.

In short: Dify becomes the place where AI behavior is designed, and MongoDB Atlas becomes the data layer that keeps those AI experiences grounded in real, useful information.

Building a RAG Knowledge Base on MongoDB Atlas

MongoDB Guests — Thu, 28 May 2026 17:28:09 +0000

This tutorial was written by Samuel Molling.

Composition, retrieval fidelity, and the failure modes between $vectorSearch and a cited answer.

The hard part of a production RAG system is not running a single $vectorSearch. It is composing keyword retrieval, vector retrieval, ranking fusion, reranking, and a generative model into a pipeline that returns the right passage, with the right confidence, every time, under concurrency, across tenants, and with predictable latency.

In MongoDB Atlas, this composition can be done end-to-end inside a single cluster. Operational data, full-text indexing, vector indexing, multi-tenant filtering, and the retrieval that feeds an LLM are not separate stores. They are stages of the same aggregation framework.

This text walks through the architectural decisions behind a complete RAG knowledge base built on Atlas Search and Atlas Vector Search, examining where each stage contributes, where each stage fails silently, and what it takes to keep the pipeline honest end-to-end.

Search interface with a streaming AI answer, inline citations, and the source passages that grounded the response.

Composition, not capability

A knowledge base in production exposes a single search box. Behind that box, there are five distinct stages of execution. Treating them as a single primitive (“the LLM answers questions about our docs”) is the most common reason these systems return confidently wrong answers.

The actual pipeline looks like this:

The first four stages are MongoDB-native. The fifth is the only component outside Atlas, and it consumes context retrieved entirely from MongoDB. Each transition between stages has a cost and a contract. Understanding those contracts is what separates a demo from a production system.

The remainder of this article examines those contracts: data layout, index design, ranking, the role of reranking as a truth signal, and the failure mode that vector search shares with every nearest-neighbor algorithm: the one where it never tells you it found nothing.

Data layout: chunks as their own collection

The first non-trivial decision in any RAG system on MongoDB is whether to store chunks as an embedded array inside the parent article, or as documents in their own collection.

Embedded arrays look attractive. They preserve locality, the document hierarchy mirrors the application model, and a single findOne returns everything. They are also the wrong choice for non-trivial corpora.

Three reasons argue for a separate chunks collection.

First, retrieval granularity. $vectorSearch returns the documents it scored, not their parents. With chunks embedded inside articles, a vector hit identifies the article but not the passage. The user receives a long document containing the answer somewhere; the system has lost the ability to point at the exact section. With chunks as documents, each hit carries its own _id, its position in the article (chunkIndex), and the precise text that scored.

Second, re-indexing cost. Updating one article when chunks are embedded means rewriting the entire parent document, including every untouched chunk and its embedding. With chunks in their own collection, re-indexing is a deleteMany({ articleId }) followed by an insertMany of the new chunks. The articles collection remains compact and serves the operational queries of the application (list, filter, render) without dragging embedding payloads through every read.

Third, and most consequentially, pre-filter at index time. The vector search index can pre-filter candidates inside $vectorSearch itself, before any document is hydrated, but only on top-level fields of the indexed collection. When chunks live as documents, denormalizing workspaceId, visibility, tags, and category onto each chunk turns multi-tenancy from a downstream $match into an index-level prune. The HNSW traversal walks only the candidate set that the tenant is allowed to see. This is the difference between a system that scales linearly with the corpus and one that scales with the per-tenant slice.

chunks: {
  _id:         ObjectId,
  articleId:   ObjectId,
  workspaceId: string,
  chunkIndex:  Number,
  text:        String,
  embedding:   [Number],   // 1024 floats from voyage-3
  tags:        [String],
  category:    String,
  visibility:  String,
}

One detail of the denormalization is worth calling out. workspaceId is stored on chunks as a string, not as an ObjectId, because Atlas Vector Search filter fields do not compare ObjectId values directly. The chunk contains the hex representation of the parent workspace's _id, which is what the $vectorSearch filter clause compares against. Articles that are not filtered through a vector index continue to use the native ObjectId type.

Index design: static mappings and filter fields

Two Atlas indexes handle retrieval, one per collection. Both use static mappings. Dynamic mappings index every field of every document, including fields that never appear in queries, which inflates the index and slows every query. Static mappings force every indexed field to justify its existence.

The full-text index on articles declares language analyzers per field:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "title":       { "type": "string", "analyzer": "lucene.english" },
      "content":     { "type": "string", "analyzer": "lucene.english" },
      "summary":     { "type": "string", "analyzer": "lucene.english" },
      "tags":        { "type": "string", "analyzer": "lucene.keyword" },
      "category":    { "type": "string", "analyzer": "lucene.keyword" },
      "workspaceId": { "type": "objectId" }
    }
  }
}

title, content, and summary use the English analyzer (lowercasing, stemming, stopword removal) so that natural-language queries match natural-language text. tags and category use the keyword analyzer for exact-match behavior without lexical transformation. workspaceId is indexed as an ObjectId, so tenant scoping is exact.

The vector index on chunks declares the embedding geometry and the fields eligible for pre-filtering inside $vectorSearch:

{
  "fields": [
    { "type": "vector", "path": "embedding",
      "numDimensions": 1024, "similarity": "cosine" },
    { "type": "filter", "path": "workspaceId" },
    { "type": "filter", "path": "visibility" },
    { "type": "filter", "path": "tags" },
    { "type": "filter", "path": "category" }
  ]
}

numDimensions: 1024 matches the output dimensionality of the chosen embedding model. Cosine similarity is the standard metric for normalized text embeddings. The four filter declarations are not metadata. They are the difference between a vector index that scopes to a single workspace inside HNSW traversal and one that scans the entire corpus before filtering downstream. At any non-trivial corpus size, the first behaves and the second does not.

Section-aware chunking and breadcrumb context

Chunking decides retrieval quality before any embedding model is involved. A 1024-dimensional vector is a single point representing an entire chunk of text. If that text contains three unrelated paragraphs, the resulting embedding sits in the centroid of three semantic regions and is meaningfully close to none of them.

Fixed-window chunkers (“every 800 characters with 120 character overlap”) produce exactly this problem. They cut mid-sentence, mid-section, and mid-topic, and the embedding is a faithful representation of the resulting incoherence.

Section-aware chunking solves this at the document level. ATX headings (#, ##, ###) demarcate semantic units that the author has already labeled. Splitting along those boundaries produces chunks that map to a single logical idea each, and the embedding represents that idea with no contamination from neighboring sections.

Two additional properties matter at scale.

The breadcrumb path is prepended to every chunk of text:

Designing a chunking strategy for RAG > Semantic chunking by section  
The most precise approach for structured documents is to chunk by heading: every ## or ### section becomes one chunk…

The embedding now encodes hierarchical context, not just local content. When the user asks about a specific topic, the model sees the chunk’s position in the document graph and weights it accordingly. The reranker sees the same context, and the LLM, in the final stage, receives a passage with a self-identifying header, which materially improves citation quality.

The fallback for oversized sections is unconditional. A section larger than the maximum chunk size (1500 characters in this implementation) falls back to paragraph splitting, then to sentence splitting, while the breadcrumb is preserved on every sub-chunk. The system never produces a chunk without context.

Fenced code blocks are excluded from heading detection. A # comment inside a bash block is content, not structure, and treating it as a heading would cause chunks to break unpredictably.

Hybrid retrieval and the score-scale problem

Each retrieval stage is a single aggregation pipeline. The keyword stage ranks articles by BM25, with fuzzy tolerance and per-field boosts:

db.articles.aggregate([
  {
    $search: {
      index: "articles_search",
      compound: {
        should: [
          { text:  { query, path: "title", fuzzy: { maxEdits: 1 } },
            score: { boost: { value: 3 } } },
          { text:  { query, path: "content", fuzzy: { maxEdits: 1 } } },
          { text:  { query, path: "tags" },
            score: { boost: { value: 2 } } },
        ],
      },
    },
  },
  { $match: { workspaceId: ObjectId("...") } },
  { $limit: 30 },
  { $project: { _id: 1, score: { $meta: "searchScore" } } },
])

The semantic stage runs in parallel, on the chunks collection, with the workspace filter applied inside the index:

db.chunks.aggregate([
  {
    $vectorSearch: {
      index: "chunks_vector",
      path: "embedding",
      queryVector: [/* 1024 floats from Voyage */],
      numCandidates: 200,
      limit: 50,
      filter: { workspaceId: "65f1b…" }   
    },
  },
  { $project: {
      articleId: 1, chunkIndex: 1, text: 1,
      score: { $meta: "vectorSearchScore" } } },
])

Both stages produce ranked result lists, but on incomparable scales.

Atlas Search returns BM25-style scores that depend on term frequency, inverse document frequency, and document length. Scores routinely climb into the tens and can spike even higher when a document is short, and the queried term is densely used.

Atlas Vector Search returns cosine similarities bounded between 0 and 1, with realistic values for relevant matches typically clustering in 0.6 to 0.9.

Adding these scores, or normalizing them to a common scale, is fragile. A handful of documents with anomalously high BM25 scores will dominate the fused ranking regardless of their actual relevance. Linear weighting with hand-tuned coefficients works for one corpus and fails on the next.

Reciprocal Rank Fusion sidesteps the problem by discarding scores entirely and working only on rank position:

rrf_score(doc) = Σ over rankers   1 / (k + rank_i(doc))     // k = 60

A document at rank 1 contributes 1/61 per ranker, rank 2 contributes 1/62, and so on. The scale collapses. Documents that appear near the top of multiple rankers accumulate score; documents that appear near the top of only one ranker also score, but less. The fusion is parameter-free in practice: k = 60 is the value from the original 2009 paper and remains the production default.

In this implementation, fusion is anchored on chunks. Semantic search provides the candidate list; keyword search acts as a boost when a chunk’s parent article also matches the keyword. This asymmetry is intentional: expanding every keyword-matched article into all of its chunks would pollute the candidate pool with off-topic content from articles that matched on a single tangential term.

const fused = semanticChunks
  .map((chunk, idx) => {
    let score = 1 / (k + idx + 1);                              // semantic rank
    const kwRank = keywordArticleRank.get(chunk.articleId.toString());
    if (kwRank !== undefined) score += 1 / (k + kwRank + 1);    // keyword boost
    return { ...chunk, rrfScore: score };
  })
  .sort((a, b) => b.rrfScore - a.rrfScore)
  .slice(0, 20);

MongoDB 8.1 introduced $rankFusion as a first-class aggregation stage that performs RRF natively. Production code should prefer it; the explicit implementation above exists to make the algorithm visible.

Reranking as the truth signal

After fusion, the system has a ranked list of twenty chunks, but no calibration of how good that ranking actually is. A list of twenty chunks always exists, even when none of them is relevant.

Reranking solves two problems simultaneously. First, it improves precision: the top of the fused list is reordered by a cross-encoder model that scores each (query, passage) pair directly, which is materially more accurate than approximate nearest-neighbor distance for the small set of candidates that matters. Second, and more critically, the reranker’s score becomes a calibrated relevance signal that the rest of the system can interpret.

Voyage’s rerank-2 model, exposed through the MongoDB-managed gateway at ai.mongodb.com/v1, returns scores in the [0, 1] range. Empirically, on a well-formed corpus:

A top score above 0.5 indicates strong relevance: the passage genuinely contains the answer.

A top score between 0.3 and 0.5 indicates partial relevance: adjacent topics, paraphrases, weak matches.

A top score below 0.3 indicates the corpus does not contain an answer to the question.

This calibration is what makes the downstream stages safe. The LLM receives passages with known quality. The UI can surface a low-confidence banner. The application can decide at query time whether to attempt a generated answer.

The reranker is more expensive per pair than the embedding model, which is why it runs only on the shortlist produced by fusion. The cost is linear in the number of candidates reranked, and reranking twenty candidates is small even at high concurrency.

The generative stage as a constrained consumer

The LLM does not “answer questions about the knowledge base”. It receives a prompt containing retrieved passages and is instructed, by system message, to use only those passages.

The contract is explicit, and the model honors it:

SYSTEM:
You are a helpful assistant answering questions using ONLY the passages
provided below. Cite the passages you used as bracketed numbers like [1]
or [2]. If the passages do not contain enough information to answer, say
so honestly. Reply in the same language as the user's question.
USER:
Passages:
[1] Designing a chunking strategy for RAG > Semantic chunking by section
    The most precise approach for structured docs is to chunk by heading...
[2] Designing a chunking strategy for RAG > Reasonable defaults
    A starting point is 500-1500 characters per chunk...
[3] Getting started with MongoDB Atlas Vector Search > The chunking pattern
    A common pattern is to chunk long documents, embed each chunk...
Question: How do I chunk by markdown section?

What returns is the text that the model generated by reading those passages. The citation discipline ([1], [2]) is enforced by the prompt, parsed by the frontend, and rendered as in-text links that scroll to the source card. The frontend never displays an unreferenced answer alongside hidden sources, and the model never sees passages it isn’t allowed to cite.

Streaming changes the user experience but not the architecture. The response is delivered as newline-delimited JSON, with three event types:

{"event":"passages","passages":[...]}     // emitted once, before the LLM call
{"event":"token","text":"..."}            // emitted per token
{"event":"done","model":"gpt-4o-mini"}    // emitted at the end

Passages are sent first, before the LLM call begins. The frontend renders source cards immediately and updates the answer panel as tokens arrive. From the user’s perspective, the system is already useful before the model has finished generating.

The AI answer card with inline numbered citations linking to the source passages retrieved from MongoDB.

The failure mode: $vectorSearch never says “no”

Nearest-neighbor search is by construction non-empty. Given a query vector, the HNSW graph returns the N closest vectors regardless of whether those vectors are actually relevant to the query. A query about the capital of a country, posed to a knowledge base about MongoDB, will receive five chunks ranked by cosine distance: chunks that share no semantic content with the question.

This is not a bug. It is the contract of kNN. The pipeline must compensate at the application layer.

Low-confidence banner shown when the top reranker score falls below the relevance threshold; no sources are surfaced.

Three layered defenses keep the system honest.

The system prompt instructs the LLM to refuse questions whose passages do not contain the answer. The model honors this when scores are uniformly low; it produces “the passages provided do not contain information about that” rather than hallucinating an answer from training data.

The rerank score is the truth signal. When the top reranker score falls below the strong-relevance threshold, the system knows the corpus is not coherent with the question. This is the only stage in the pipeline that produces calibrated relevance. Vector distance and BM25 scores cannot be interpreted in isolation, but rerank scores can.

The UI threshold makes the calibration visible. Below a confidence threshold (empirically 0.4 for the top rerank), the interface displays a banner indicating that the knowledge base does not cover the topic, and hides individual passages whose scores fall below a stricter threshold (empirically 0.3). The user does not see five low-quality “sources” that contradict the LLM’s refusal.

const STRONG_RELEVANCE = 0.4;
const WEAK_PASSAGE     = 0.3;
const confidence = passages[0].rerankScore >= STRONG_RELEVANCE ? "strong" : "weak";
const s

Each defense alone is insufficient. The prompt can fail when scores happen to land in the partial-relevance band. The threshold can hide useful passages in rare edge cases. The UI banner alone cannot prevent a confident hallucination. Together, they make confidently wrong answers very unlikely.

Conclusion

A production RAG system is the composition of five well-understood stages, each with its own contract and its own failure modes. The interesting engineering does not live in any single stage. It lives at the boundaries between them: in the data model that makes pre-filtering possible, in the chunking that makes embeddings coherent, in the fusion that reconciles incomparable scores, in the rerank that produces a calibrated truth signal, and in the prompt that translates that signal into a refusal when the corpus does not have the answer.

MongoDB Atlas is the substrate that lets these stages compose without crossing infrastructure boundaries. The operational data, the search indexes, and the vector indexes are not federated systems with their own consistency models. They are aggregation stages in the same query language, against the same documents, with the same access control. The cost saved is not raw latency, which is a property of every well-engineered system; it is the entire class of consistency bugs that arises when a document, a search index, and a vector store have to agree about the same change.

RAG is not a prompt problem. It is a systems engineering problem. When the boundaries are honest, the answers are honest.

Comparing Replication and Failover in PostgreSQL and MongoDB

MongoDB Guests — Thu, 21 May 2026 09:00:00 +0000

This article was written by Arek Borucki.

Every production database eventually faces the same question: what happens when the machine running it dies? An EC2 instance gets retired with three minutes’ warning. A data center loses connectivity to its peers. The database keeps the company’s data, and the company cannot wait for someone to drive to a colocation facility. So databases learned to replicate themselves. To keep more than one copy of the data, on more than one machine, ready to take over when something breaks.

PostgreSQL and MongoDB both solve this problem. They both replicate data. They both promote a new primary when the current one fails. But they took radically different paths to get there, and those paths reflect when each system was born and what its designers believed about how databases should work. PostgreSQL grew out of an academic project in the 1990s, in an era when high availability meant a tape backup and a phone call to a DBA. MongoDB was conceived in 2009, after a decade of internet companies discovering that machines fail constantly and recovery cannot involve humans. Their replication systems carry that history in their bones.

This article carefully walks through both systems. You will learn how PostgreSQL streams write-ahead logs to standby servers and how MongoDB tails its operation log on every secondary. You will see why PostgreSQL needs Patroni or repmgr to fail over automatically, while MongoDB has Raft consensus built into its replica set protocol. You will understand why a transaction in flight on a PostgreSQL primary simply dies during failover, while a prepared transaction on a MongoDB primary survives and resumes on the new primary. By the end, you will have a clear framework for choosing between them. Not based on marketing pages, but on which trade-offs actually fit your system.

Understanding the philosophical divide

The most important thing to know about PostgreSQL replication is that it does not include automatic failover. Read that sentence twice. PostgreSQL can stream every byte of its write-ahead log to a standby server in real time, keep that standby in lockstep with the primary, and let you read from it. But if the primary dies at three in the morning, PostgreSQL itself does not promote the standby. A human, or a piece of software written by humans, must notice the failure, decide which standby is the most up-to-date, and run the promotion command.

This is not an oversight. It is a design choice that goes back to PostgreSQL’s origins as the Berkeley POSTGRES project in the mid-1980s and its long incubation through the 1990s. Database research at the time treated replication as an engineering problem with many possible solutions and no single right answer. Should the primary wait for the standby to acknowledge writes, or should it commit and replicate asynchronously? Should the standby be a hot spare, ready to serve reads, or a cold backup? Should failover require human judgment, or should it be automatic? PostgreSQL refused to choose. It built the primitive, write-ahead logs, replication slots, hot standbys, and left the high-level policy to the operator.

MongoDB started from the opposite assumption. It was designed in 2009, in the wreckage of the early-2000s internet, where companies like Google, Amazon, and Facebook had spent a decade learning that machines fail constantly. A typical Google datacenter sees thousands of disk failures, hundreds of machine failures, and dozens of rack failures per year, as documented in Jeff Dean’s famous 2009 talk on cluster reliability. By 2009, the operational lesson was clear: if your database requires a human to recover from a single machine failure, your database is broken. MongoDB’s designers chose to bake high availability into the database itself. A MongoDB replica set is not a primary plus a few read replicas; it is a small cluster that elects its own leader, detects its own failures, and recovers from them without human intervention.

A useful analogy is the difference between a car that comes with a spare tire and a car that comes with run-flat tires. PostgreSQL hands you a spare tire and a jack: it tells you exactly how to replace the wheel, but you have to be the one who notices the flat and pulls over. MongoDB ships run-flat tires that detect a puncture and let the car keep driving. Both approaches keep you on the road. The first gives you more control over what happens when something goes wrong; the second works at three in the morning when nobody is paying attention.

This difference shapes everything else in this chapter. When PostgreSQL operators want automatic failover, they install Patroni, repmgr, or pg_auto_failover. External systems that wrap PostgreSQL and add the missing logic. These tools watch the primary, detect when it stops responding, run an election among the surviving standbys, and promote the winner. Some of them use etcd or Consul to coordinate, treating PostgreSQL itself as a passive replicated state machine that responds to instructions from the outside. The PostgreSQL process knows nothing about its peers; it just streams WAL to whoever asks. When a MongoDB operator wants automatic failover, they do nothing. It is already there.

Why has PostgreSQL not absorbed this functionality? The community has debated it for years, and the answer is partly philosophical and partly practical.

Philosophically, PostgreSQL favors small, composable primitives over integrated systems. Adding automatic failover to core PostgreSQL would mean adding consensus algorithms, leader election, and cluster membership management. Substantial complexity that the community has been reluctant to take on, when external tools already do the job. Practically, automatic failover requires answering questions that depend on the deployment: how many failures can the cluster tolerate, what counts as the primary being down, and how should clients discover the new primary. PostgreSQL’s answer has been to expose enough hooks that external tools can implement any policy, rather than commit to one.

The trade-off is real. With PostgreSQL, you get flexibility. You can build exactly the HA system your environment needs, or pick from several mature options. You also get complexity. Patroni alone is a substantial system, and getting it right requires understanding etcd, distributed consensus, network partitions, and split-brain scenarios. A misconfigured Patroni cluster can cheerfully promote two primaries during a network partition, and only careful fencing prevents the resulting data divergence. With MongoDB, you get simplicity. The rs.initiate() command and three nodes give you a working HA cluster, and you give up flexibility. The MongoDB replica set protocol is what it is; you cannot swap out its election algorithm or its heartbeat interval beyond the configuration knobs the database exposes.

Warning: the apparent simplicity of MongoDB replication can mislead operators into deploying minimum configurations that fail in surprising ways. A two-node replica set looks reasonable until you realize that any partition leaves both nodes unable to form a majority, so neither can serve writes. An odd number of voting members is not optional advice; it is a structural requirement of the protocol. PostgreSQL operators face the equivalent trap when they configure synchronous replication with a single standby and discover that losing the standby blocks all writes on the primary. Both systems have failure modes that follow from their consistency guarantees, and both reward operators who understand those guarantees before deploying.

Throughout this article, you will see the philosophical divide play out at every level. PostgreSQL has two replication mechanisms because operators wanted both physical and logical replication. MongoDB has one. PostgreSQL’s synchronous replication is configured through cryptic strings in synchronous_standby_names; MongoDB exposes write concerns at the query level. PostgreSQL clients connect to a hostname, and MongoDB drivers maintain a topology view of the cluster. Each system is internally consistent, but they answer different questions, and the answers reflect the era in which each was built.

Tracing how PostgreSQL replicates data

PostgreSQL replication starts with the write-ahead log. Every change to the database, every INSERT, every UPDATE, every CREATE INDEX, is written to the WAL before it touches the actual data files. The WAL exists for crash recovery: if the database process dies between writing a row and flushing it to disk, replaying the WAL on restart reconstructs the lost change. This is how every modern relational database achieves durability. What PostgreSQL noticed, sometime in the early 2000s, is that the same WAL stream that lets a single server recover from a crash can be sent to a second server to keep it in sync. Replication, in PostgreSQL, is fundamentally a side effect of crash recovery extended across the network.

PostgreSQL exposes two replication mechanisms built on top of the WAL: physical replication and logical replication. They operate concurrently, on the same WAL stream, and serve different use cases. Understanding the difference between them clarifies why PostgreSQL has accumulated the configuration knobs and external tools it has.

Physical replication: the WAL on the wire

Physical replication ships the WAL byte-for-byte from the primary to one or more standby servers. The standby is, in a literal sense, a replica of the primary at the binary level. Every page in the standby’s data files matches the corresponding page on the primary. This makes physical replication conceptually simple: the primary writes its WAL; a process called walsender streams those records to the standby; the standby’s walreceiver process receives them; and the recovery system on the standby applies them as if it were replaying a crash. The standby is always in recovery mode and only stops being a standby when it is promoted to primary.

Streaming replication is the modern way to ship WAL. The standby connects to the primary via the replication protocol, identifies itself, and asks for WAL records starting from a specific log sequence number (LSN). The primary streams records as they are generated. To prevent the primary from deleting WAL files that a standby still needs, PostgreSQL uses replication slots: small pieces of state on the primary that track how far each standby has consumed. As long as a slot exists, the primary will not delete WAL beyond the slot’s position. This is critical because a standby that falls too far behind can otherwise discover that the WAL it needs has been recycled, at which point it cannot catch up without a fresh base backup.

PostgreSQL also supports cascading replication, where a standby acts as a relay, streaming WAL it has received to other downstream standbys. This reduces network bandwidth on the primary and lets you build replication trees that span multiple data centers without each standby holding a connection back to the origin. Cascading replication carries a subtlety: the downstream standbys can never be more current than the relay standby, which means failover policies must account for the relay sometimes lagging behind the primary.

Synchronous replication adds durability guarantees on top of streaming. By default, the primary acknowledges a commit as soon as the WAL is flushed to its own disk; the standby may receive and apply the change later. Synchronous replication makes the primary wait for one or more standbys to acknowledge the write before reporting success to the client. This is configured through synchronous_standby_names, a string that lists which standbys count as synchronous and how many of them must acknowledge. PostgreSQL supports two methods. FIRST n (s1, s2, s3) requires the first n standbys in the list, in priority order, to confirm. ANY n (s1, s2, s3) requires any n of the listed standbys to confirm. A quorum-style policy that tolerates individual standby failures better. The synchronous_commit parameter further refines what acknowledgment means: remote_write requires the standby to write the WAL to the filesystem cache (surviving a PostgreSQL crash, but not necessarily an OS crash), while on requires the WAL to be flushed to disk, and remote_apply requires it to actually apply the change so subsequent reads on the standby see it.

A common mistake is to enable synchronous replication with a single standby and a strict synchronous_commit value, then panic when writes hang during a standby restart. The primary, by design, waits for the synchronous standby to confirm the commit before acknowledging success to the client. If the standby is gone, the primary blocks. Production deployments either configure ANY-style synchrony with multiple standbys or use synchronous_commit = remote_write to balance durability with availability. The right setting depends on whether you would rather lose a recent transaction during a primary crash or stop accepting writes during a standby outage.

Logical replication: the row-level alternative

Physical replication is powerful, but it has constraints that follow from its byte-level nature. The standby must run the same major version of PostgreSQL as the primary; you cannot replicate from version 14 to version 16 because the on-disk page formats might differ. The standby replicates the entire PostgreSQL cluster (that is, the entire database instance, including all databases in it); you cannot replicate just one database or table. The standby is read-only; you cannot write to it without breaking replication. These constraints are fine for high availability, where you want a hot spare identical to the primary, but they make physical replication unsuitable for use cases like cross-version upgrades, partial replication for analytics, or feeding data into systems that are not PostgreSQL.

Built-in logical replication, introduced in PostgreSQL 10 and refined heavily since, solves these problems by replicating at the row level instead of the page level. Where physical replication says, “here is the byte change at offset 8192 in file 16384,” logical replication says, “here is an INSERT into table users with these column values.” This higher-level representation lets logical replication cross PostgreSQL major versions, replicate selected tables, and lets subscribers process the changes through plugins or external systems.

The architecture is publish-subscribe. On the publisher node, you create a publication that defines which tables to replicate. On the subscriber node, you create a subscription that connects to the publisher and asks for one or more publications. The publisher records each change in its WAL as usual, and a separate logical decoding process extracts the high-level row changes from the WAL and sends them to the subscriber, which applies them to its own tables.

-- On the publisher
CREATE PUBLICATION mypub FOR TABLE users, departments;

-- On the subscriber
CREATE SUBSCRIPTION mysub
  CONNECTION 'dbname=foo host=publisher.local user=repuser'
  PUBLICATION mypub;

Logical replication uses several worker processes coordinated by the logical replication launcher (ApplyLauncherMain in the source). The Apply Worker, one per subscription, receives changes from the publisher and applies them. The Table Sync Worker handles the initial copy of each table’s existing data when the subscription is first created. The Sequence Sync Worker synchronizes sequence values. The Parallel Apply Worker can apply streaming transactions in parallel, which matters for large transactions that would otherwise serialize on the subscriber.

Behind the scenes, logical decoding does the heavy lifting. The reorder buffer assembles individual WAL records into complete transactions, spooling large transactions to disk if they exceed memory. The pgoutput plugin transforms the assembled changes into the logical replication protocol that subscribers understand. Snapshot-build (snapbuild.c) maintains historical catalog snapshots so that a row decoded from old WAL is interpreted using the schema that existed when the row was written, not the current schema. These mechanisms make logical decoding work across schema changes, but they also make it more expensive than physical replication.

Logical replication has an important limitation that confuses operators on first encounter: it does not replicate DDL. A CREATE TABLE on the publisher does not propagate to the subscriber. Schema changes must be applied separately on each node, and they must be applied carefully. If a column is added to the publisher and not yet to the subscriber, replication of new rows including that column will fail until the subscriber catches up. The same applies to TRUNCATE behavior, sequence values for newly created sequences, and other DDL-adjacent operations. Logical replication is therefore well-suited for replicating data between known schemas but poorly suited as a general-purpose schema mirror.

Replication origins, exposed through the pg_replication_origin catalog, track where each subscriber is in each publisher’s WAL stream. Each origin records both the remote LSN it has consumed and the local LSN where the change was applied. This metadata is what makes logical replication safe across crashes: if the subscriber dies mid-apply, it knows on restart exactly which transaction to resume. Replication origins also prevent infinite loops in bidirectional replication setups. A change that came in from origin A is tagged with that origin, so the subscriber knows not to send it back.

Setting up replication, in either form, requires explicit configuration. Physical replication needs wal_level = replica, max_wal_senders, and a base backup made with pg_basebackup. Logical replication needs wal_level = logical and one of the available output plugins (almost always pgoutput in modern deployments). PostgreSQL ships utilities like pg_basebackup and pg_createsubscriber to make these workflows easier, but the operator is still in charge of running them in the right order and pointing the standby or subscriber at the right primary. There is no equivalent to MongoDB’s rs.initiate() command that creates a working replicated cluster from scratch.

Tracing how MongoDB replicates data

MongoDB replication looks deceptively simple from the outside. You start three mongod processes, run rs.initiate() against one of them, add the other two with rs.add(), and wait a few seconds. The first node becomes primary, the other two become secondaries, and the cluster begins serving traffic. There is no separate replication daemon, no external coordination service, and no configuration file string that lists which nodes count as synchronous. Everything runs inside the same mongod process. Behind that simplicity sits a careful design that turns Raft consensus, an operation log, and pull-based replication into a system that genuinely needs no human intervention to recover from a primary failure.

The oplog: a log of operations, not bytes

The heart of MongoDB replication is the oplog, a capped collection named oplog.rs in the local database that exists on every replica set member. Where PostgreSQL’s WAL records byte-level changes to data pages, the oplog records logical operations: insert this document into this collection, update this document with these field changes, drop this index. Each oplog entry carries a timestamp called an OpTime that uniquely identifies the operation in the cluster’s history. Secondaries replicate by pulling entries from the primary’s oplog and applying them to their own copies of the data.

The oplog, being a capped collection, has consequences. It is a fixed-size circular buffer, sized at startup or via the replSetResizeOplog command. When it fills up, the oldest entries are overwritten. This means a secondary that falls too far behind can discover that the oplog entries it needs have been overwritten, at which point it cannot catch up incrementally and must perform a full initial sync. Sizing the oplog is therefore an operational decision: too small, and a brief secondary outage forces a multi-hour resync; too large, and you waste storage on history nobody will replay. The MongoDB documentation recommends sizing the oplog to cover at least 24 to 72 hours of write traffic in production, but the right number depends on your tolerance for resync events.

The mechanics of write replication are straightforward. When a write arrives at the primary, it goes through the same path as on a standalone mongod: the storage engine commits the change to disk, and an OpObserver inserts a corresponding document into the oplog. Secondaries pull oplog entries continuously through a long-running connection. The pull-based model differs from PostgreSQL’s push-based streaming and reflects MongoDB’s assumption that secondaries should be in charge of their own progress, rather than relying on the primary to keep them up to date. If a secondary lags or disconnects, the primary keeps writing without slowing down; the secondary catches up at its own pace when it reconnects.

Initial sync handles the case where a new node joins the replica set or an existing secondary has fallen too far behind to catch up incrementally. The new secondary connects to a sync source (usually a healthy secondary, but it can be the primary), reads every collection in every database, and tails the oplog from the time it started reading. Initial sync is expensive. It can take hours or days for a multi-terabyte cluster, but the protocol is robust to interruption: if the new secondary disconnects mid-sync, it can resume rather than start over.

The replication coordinator and the topology coordinator

Inside the mongod process, MongoDB replication runs through two components that are easier to understand once you know they were deliberately factored apart. The ReplicationCoordinatorImpl is the central orchestrator. It owns the public API, commands like replSetGetStatus, replSetReconfig, and the implicit machinery that runs when a write arrives. It manages the lifecycle of replication: starting it up at server boot, transitioning between member states (PRIMARY, SECONDARY, RECOVERING, ROLLBACK), and shutting down cleanly. It tracks states like _memberState, _myLastAppliedOpTime, and _myLastWrittenOpTime, which other components query to make decisions.

The TopologyCoordinator is the brain of the cluster. It owns the election algorithm, the heartbeat policy, the sync source selection logic, and the rules for deciding when to step down. Critically, it performs no I/O. It is a pure decision engine that takes inputs (heartbeat responses, configuration changes, election timeouts) and produces decisions (start an election, vote for this candidate, step down). The separation matters because it makes the consensus logic testable: you can drive the TopologyCoordinator with synthetic events and verify its decisions without spinning up a real network.

This split mirrors how Raft is described in academic literature, which is not an accident. MongoDB’s replica set protocol implements a variant of Raft, the consensus algorithm published by Diego Ongaro and John Ousterhout in 2014. Raft was designed specifically to be easier to understand than its predecessor, Paxos, and MongoDB adopted it after a few years of running its earlier protocol (sometimes called PV0), which revealed edge cases that Raft’s formalism made easier to reason about. The current protocol version, often referred to as PV1 in the source code, applies Raft-style rules: every member has a term, every election increments the term, and votes are tallied by the majority of the voting members.

Heartbeats: knowing who is alive

Replica set members maintain their view of the cluster through heartbeats. Each member sends periodic heartbeat messages to every other member, asking for status. The default heartbeat interval is two seconds, and members are considered unreachable if they fail to respond within ten seconds (the electionTimeoutMillis default, also configurable). Heartbeats carry more than liveliness information: they include each member’s OpTime, term, configuration version, and state. This means every node knows which member is primary, who is most up-to-date, and whether a configuration change is in flight, just by observing the heartbeats.

The TopologyCoordinator maintains a MemberData structure for each peer, recording ping times, last heartbeat response, and health status. When a heartbeat response arrives, _handleHeartbeatResponse updates this structure and, if necessary, triggers state changes. If the primary stops responding for long enough, secondaries notice through their MemberData and start the election process. The election timeout is critical here: too short and transient network blips trigger spurious failovers, too long and real failures take a long time to detect. The default of ten seconds reflects a compromise that works for most deployments but can be tuned for environments with stricter latency requirements.

Pull-based replication and sync source selection

Secondaries replicate by pulling oplog entries from a chosen sync source. The choice of sync source is not fixed: a secondary picks among available members based on factors like ping time, oplog freshness, replication chain depth, and configuration preferences (members can be tagged to influence selection). A secondary may sync from another secondary rather than from the primary, which is conceptually similar to PostgreSQL’s cascading replication and reduces load on the primary in large clusters.

The pull model has one important property: it makes the secondary responsible for its own progress. If the secondary slows down for any reason, like disk pressure or network congestion, the primary continues serving writes without blocking. Replication lag accumulates on the secondary side and is visible in metrics like replSetGetStatus output and the oplog window. PostgreSQL’s push model can be configured for similar non-blocking behavior, but its synchronous mode actively ties the primary’s commit latency to the standby’s acknowledgment, which the MongoDB write concern model handles differently (covered in the next section).

A subtlety worth noting: oplog entries are designed to be idempotent. The same operation can be applied twice without changing the final state. This matters for replication safety. If a secondary crashes mid-apply, it can replay the last few operations on restart without corrupting data. PostgreSQL’s WAL records have similar properties (they are designed to be replayed), but the bookkeeping is different: PostgreSQL tracks LSN positions, MongoDB tracks OpTime, and both protocols include enough information to resume after an unclean shutdown.

Comparing durability and synchronous semantics

A write is durable when, after the database tells the client the write succeeded, the database can recover that write through any single failure. The two systems mean roughly the same thing by durability, but they let you control it through completely different interfaces, and the differences reveal each system’s assumptions about who should make durability decisions.

PostgreSQL exposes durability through the synchronous_commit and synchronous_standby_names parameters. These are server-side settings: the database administrator chooses them once for the cluster (or per role, or per session via SET), and every transaction inherits the choice. The model is rigid by design. If your deployment policy says writes must reach two standbys, the database enforces that universally. The synchronous_commit values define the durability level: off means commits are not even written to disk before acknowledging (fast, lossy), local means flushed to the primary’s disk only, on means the synchronous standbys have written but not necessarily applied, remote_write means the standbys have received the data, and remote_apply means the standbys have actually applied the change. The synchronous_standby_names string then defines which standbys count and how many must confirm.

MongoDB takes the opposite approach. Durability is specified per write, through the writeConcern parameter. A client can issue an insert with { w: "majority", j: true } and the server will wait until a majority of voting members have written the change to disk before acknowledging. Another client, on the same database, can issue a write with { w: 1 } and get an acknowledgment as soon as the primary has applied it. This is more flexible than PostgreSQL’s server-wide setting, and it makes the durability decision a property of the operation rather than the deployment. A user-facing write that creates an account uses { w: "majority", j: true }; a background job that records click metrics might use { w: 1 } because losing a click is acceptable.

Both systems converge on the same durability primitive: a quorum of disks has flushed the WAL or oplog. PostgreSQL with synchronous_commit = on and synchronous_standby_names = 'ANY 1 (s1, s2, s3)' gives you a durability guarantee similar to MongoDB with w: 2 and j: true. The difference is who chose: the operator who set up the cluster, or the developer who wrote the query. There is no universally right answer. PostgreSQL’s model is easier to audit. You can read the configuration and know what the cluster guarantees, but harder to vary by operation. MongoDB’s model is more expressive but requires every developer to think about write concerns, and code that omits the write concern silently uses the default ({ w: "majority" }).

The read side has a similar pattern. PostgreSQL supports reads on hot standbys, with hot_standby_feedback to let the standby tell the primary not to vacuum rows the standby is still using. Reads on standbys may be stale due to some replication lag. MongoDB exposes read preferences (primary, primaryPreferred, secondary, secondaryPreferred, nearest) and read concerns (local, available, majority, snapshot, linearizable) on a per-query basis. Reading with readConcern: "majority" means the data has been committed to a majority of nodes, which is stronger than reading from a possibly lagging secondary in PostgreSQL terms. linearizable read concern is even stronger. It guarantees you see the latest write, at the cost of additional round-trips. Again, the philosophical pattern repeats: PostgreSQL gives you knobs at the deployment level, MongoDB exposes them at the query level.

Why the divergence? The answer is partly about how each system thinks about clients. PostgreSQL was designed when SQL was the universal interface, and applications rarely shipped with database-level configuration awareness; the DBA controlled durability because the application could not be expected to know. MongoDB was designed when applications were richer, drivers shipped with sophisticated configuration, and developers were comfortable specifying durability per operation. Neither approach is wrong. They reflect different ideas about where the contract between the database and application sits.

A practical warning applies to both systems. Operators sometimes configure synchronous replication or majority write concern and then deploy a cluster that cannot satisfy the requirement. Too few standbys, an arbiter that does not store data and therefore cannot acknowledge majority writes, and a synchronous standby that is offline. The result is the primary blocking on writes, which presents to applications as latency or timeouts. Test your durability configuration against the failure modes you actually expect: take down a standby in a staging cluster, see what happens, then decide whether the behavior is acceptable.

Detecting failure

Replication keeps copies of the data; failure detection decides when to use them. The contrast between the two systems is sharpest here because PostgreSQL has essentially no failure detection in core, and MongoDB has it built into every node.

In MongoDB, every member of a replica set sends periodic heartbeats to every other member. The default heartbeat interval is two seconds, configurable via the settings.heartbeatIntervalMillis configuration. Heartbeats are bidirectional: every member is both a sender and a receiver, and every member maintains its own view of the cluster’s health.

When a member fails to respond to heartbeats for longer than electionTimeoutMillis (default 10 seconds), peers consider it unreachable. If the unreachable member is the primary, eligible secondaries begin the election process to choose a new one.

The heartbeat data is rich. A heartbeat response carries the responder’s state (PRIMARY, SECONDARY, etc.), its term, its OpTime, its configuration version, and various health metrics. The TopologyCoordinator on each node uses this stream of information to maintain a consistent view of the cluster. Because every node sees the same heartbeat traffic, every node can independently decide what to do when something changes. There is no single coordinator that must itself stay alive for the cluster to make decisions. This is the essence of distributed consensus: each member runs the same algorithm on the same inputs and arrives at the same conclusion.

PostgreSQL has nothing equivalent in core. The walreceiver on a standby maintains a connection to the primary and detects when the connection breaks, but it does not act on that detection. It simply tries to reconnect. The primary does not actively monitor its standbys for liveness; it only knows about a standby that has connected and is consuming WAL. There is no protocol for a standby to vote, no protocol for primary handoff, and no protocol for cluster membership beyond the operator-managed standby connection. PostgreSQL is, in this sense, a single-node database that streams its WAL to bystanders.

External tools fill the gap. Patroni, the most popular HA solution for PostgreSQL, runs alongside each Postgres instance and uses a distributed key-value store (typically etcd or Consul) to maintain cluster state. Each Patroni agent registers its node, monitors the local Postgres process, and writes liveness updates to the key-value store. If the agent on the primary fails to update its leader key within a configured TTL, the key expires, and the Patroni agents on the standbys race to acquire it through a compare-and-swap operation. The winner promotes its local Postgres to primary; the losers reconfigure themselves to follow the new primary. The leader election logic lives entirely in Patroni and etcd. PostgreSQL itself only sees commands like pg_promote() or recovery configuration changes.

repmgr, a different tool, takes a different approach. It uses a daemon (repmgrd) on each node that monitors the primary and coordinates failover through repmgr metadata stored in the database itself. The trade-offs are subtle: Patroni has a robust external coordination layer (etcd) at the cost of running an additional service; repmgr has fewer moving parts but relies on the standby cluster reaching consensus through reads of the primary’s metadata, which is harder when the primary is the failed node. pg_auto_failover, a newer option from Microsoft, uses a separate monitor process that all nodes report to, again with its own trade-offs around the monitor itself becoming a single point of failure.

The result is that failure detection in PostgreSQL is a deployment decision, not a database decision. Two PostgreSQL deployments running the same software can have completely different failure detection semantics depending on which HA tool wraps them, what its timeout values are, and how its coordination layer is configured. A MongoDB deployment’s failure detection is essentially the same on every cluster of the same version, because it is part of the database. This makes MongoDB easier to reason about for an operator who is running it for the first time. It makes PostgreSQL more flexible for an operator who needs failure detection that integrates with broader infrastructure — a Kubernetes cluster, an existing etcd-based coordination plane, or a custom orchestrator.

A common failure mode in PostgreSQL HA setups is the network partition that splits the etcd cluster from the database cluster. Patroni cannot make decisions without a quorum in etcd, so the database cluster effectively freezes, not because the database itself failed, but because the coordination layer cannot reach a quorum. MongoDB has a similar pattern, but the coordination layer and the database layer are the same nodes, so a partition that affects coordination affects the database in identical ways. The integrated approach has fewer surprises in partition scenarios but cannot offload coordination to a separate, possibly more reliable, layer.

Electing a new leader

When the primary is gone, the cluster needs to pick a new one. The procedure has to satisfy two constraints. First, it must terminate. A cluster that endlessly debates who should be primary is unusable. Second, it must produce exactly one primary. A cluster with two primaries can accept conflicting writes that diverge silently, a condition called split-brain that is one of the worst failure modes in distributed systems. Both PostgreSQL ecosystems and MongoDB solve this through consensus algorithms, but they put the consensus in very different places.

How MongoDB elects a primary

MongoDB uses a Raft-derived election protocol, implemented in the TopologyCoordinator and exercised through the ReplicationCoordinatorImpl. When a secondary detects that the primary has been unreachable longer than the election timeout, it transitions through a series of states designed to avoid wasting votes on candidates that cannot win.

The first phase is the dry run. Before requesting actual votes, the candidate sends a vote request marked as dry_run = true to all voting members. Each member responds with whether it would have voted for this candidate based on the current state. Specifically, whether the candidate is at least as up-to-date as the responder, and whether the responder has not already voted in this term. A dry run that returns a majority of yes responses means the candidate has a reasonable chance of winning a real election. A dry run that fails (typically because some other node thinks it is more up-to-date) lets the candidate back off without burning a term number.

If the dry run succeeds, the candidate increments its term, votes for itself, and sends a real vote request, ReplSetRequestVotesArgs, in the wire protocol. Voting members evaluate three things: (1) is the candidate at least as up-to-date as I am, measured by OpTime; (2) has the candidate’s term advanced beyond my current term; and (3) have I already voted in this term. If all three conditions are met, the voter grants the vote. Once the candidate collects a majority of votes from voting members, it transitions to the PRIMARY state, and the cluster recognizes it through the next round of heartbeats.

A few details matter for understanding what can and cannot happen. Only voting members count toward the majority. A member configured with votes: 0 (often used for analytics secondaries) is not part of the quorum and cannot vote in elections. Arbiters are voting members who hold no data. They exist solely to break ties in even-numbered clusters and are themselves a source of operational confusion because they affect quorum calculations without contributing data redundancy. The terminology can mislead: a five-node replica set with three data-bearing voting members and two arbiters has the same fault tolerance as a three-node cluster, not a five-node cluster, because the arbiters cannot serve data.

The dry run phase is a refinement of the stock Raft, designed to mitigate a specific bad pattern. In plain Raft, a partitioned node that loses contact with the rest of the cluster will keep starting elections, incrementing its term each time. When the partition heals, this node’s high term number forces the surviving primary to step down even though that primary was happily serving traffic. The dry run prevents the partitioned node from incrementing its term unless it can plausibly win an election, which means the high-term churn never happens.

How PostgreSQL elects a primary

PostgreSQL itself does not elect anything. Promoting a standby to primary is a single operation: pg_promote() (since Postgres 12) or, equivalently, calling the trigger file mechanism. The promoted node stops being in recovery, accepts writes, and starts producing its own WAL. The election logic, deciding which standby to promote, ensuring no other standby is also being promoted, and redirecting clients, lives entirely outside PostgreSQL.

In a Patroni-managed cluster, the election is run by Patroni agents using the underlying distributed coordination service. The protocol typically goes like this. Each Patroni agent periodically renews a leader key in etcd with a TTL of, say, 30 seconds. As long as the primary’s Patroni agent is alive and the primary database is healthy (Patroni runs local health checks), the key is renewed, and the leader is stable. If the primary fails or its agent stops renewing, the leader key expires. The Patroni agents on the standbys race to acquire the now-empty key through a compare-and-swap operation. Whichever standby wins the race and is also healthy and sufficiently caught up, it promotes its local PostgreSQL and writes itself as the new leader. Critically, the compare-and-swap is atomic in etcd, so exactly one standby can win.

The fault tolerance of this scheme depends on the etcd cluster, not on PostgreSQL. An etcd cluster of three nodes tolerates one failure; an etcd cluster of five tolerates two. If the etcd cluster loses quorum, no leader change can happen, and the existing leader (if any) eventually loses its lock. Whether this is acceptable depends on whether you can keep etcd more reliable than PostgreSQL itself. In Kubernetes deployments, etcd is often a separate cluster used for many things; in dedicated Postgres deployments, you might run etcd on the same machines as Patroni, and a failure that takes out a database node also takes out the local etcd member.

There is a subtlety around catch-up. Patroni and similar tools must decide whether to promote a standby that has slightly stale WAL or wait for it to catch up first. Promoting a stale standby loses recent writes that did not make it across; waiting risks extending the outage. Patroni’s default behavior favors availability — it will promote a standby that is reasonably caught up rather than wait — but you can configure it to be stricter. MongoDB faces the same trade-off and resolves it by prioritizing and requiring that the elected primary be at least as up-to-date as a majority of voting members. The mechanisms differ, but the choice of availability versus consistency is the same.

Preventing two primaries

Both systems take split-brain seriously, and both rely on majority requirements to prevent it. In MongoDB, a candidate must receive votes from a majority of voting members. If the cluster is partitioned into a minority and a majority partition, only the majority partition can elect a new primary; the minority cannot reach quorum and remains read-only or unavailable. The old primary, if it survives in the minority partition, eventually steps down on its own (the protocol requires the primary to maintain heartbeat contact with a majority of voters; if that contact is lost for the heartbeat timeout, the primary steps down).

In a Patroni-managed PostgreSQL cluster, split-brain prevention has two layers. The etcd lock prevents two Patroni agents from both believing they are the leader. The optional fencing mechanism (Patroni’s rewind feature, plus tools like pg_rewind) handles the case where an old primary returns after a network partition heals. MongoDB has a similar mechanism called rollback: when a stepped-down primary rejoins the cluster and discovers its tail of oplog was not replicated to the new primary, the unreplicated operations are rolled back and saved to a separate file for human inspection. Both rollback mechanisms are last resorts and should not happen in correctly configured clusters.

Handling failover at the client layer

A failover that the database handles is only useful if applications notice and adapt. The two systems take different approaches to the client side, and the differences shape how applications must be written.

MongoDB drivers maintain a topology view of the cluster. When an application connects to a MongoDB replica set, the driver does not connect to a single hostname. It connects to a list of seed hosts and discovers the rest of the cluster through the isMaster command (renamed hello in newer protocol versions). The driver tracks which member is currently primary, and routes writes to it; reads go to whichever member matches the configured read preference. When the primary changes, the driver detects the change through subsequent isMaster responses and updates its routing. From the application’s perspective, this happens transparently. A write that arrived during the failover sees a brief error, but with retryable writes enabled, it is retried automatically against the new primary.

Retryable writes are an important safety net. The MongoDB protocol assigns each write a unique transaction number derived from the client’s session. When the driver retries a failed write, the server checks whether the operation has already been applied (using the transaction number to look up the operation in a special transactions collection). If yes, the server reconstructs the original response without re-executing the write; if no, it executes the write normally. This makes retries safe against double-application, even in edge cases where the original write succeeded but the response did not reach the client. PostgreSQL has nothing equivalent at the protocol level; retrying a failed INSERT in PostgreSQL is the application’s responsibility, and idempotency is a property of the SQL the application chooses to write.

PostgreSQL clients connect to a hostname and port. There is no protocol-level concept of a cluster. If the primary fails and a standby takes over, the client must somehow be redirected to the new primary, and PostgreSQL itself does nothing to make that happen. Production deployments solve this in one of three ways.

The classic approach is a virtual IP (VIP) that floats between the primary and standby. Clients connect to the VIP; whichever node currently holds the VIP is the active primary. VIPs can be managed through Pacemaker, keepalived, or a cloud load balancer. The pattern is simple but has subtle failure modes. The VIP must move atomically (no two nodes can hold it simultaneously), and clients connected to the VIP at the time of failover see their connections drop, which they must then reconnect.

A second approach uses a connection pooler with topology awareness. PgBouncer in transaction-pooling mode, deployed in front of the cluster, can be reconfigured to point at the new primary on failover. The application sees a stable PgBouncer endpoint while the actual primary changes behind the scenes. This decouples the application from the cluster topology and tends to work well with HA tools like Patroni, which can update the pooler’s configuration during promotion.

The third approach pushes topology awareness into the client driver. JDBC supports multi-host connection strings (jdbc:postgresql://host1,host2,host3/db?targetServerType=primary), and similar mechanisms exist in many language ecosystems. The driver tries each host until it finds one that is currently the primary, retrying as needed. This is closer in spirit to MongoDB’s driver-level topology, but it lacks the protocol support to detect role changes proactively. The driver only learns about the new topology when a query fails, not before.

The practical consequence is that writing failover-tolerant applications looks different on the two systems. On MongoDB, you connect to the cluster, enable retryable writes (on by default in modern drivers), and let the driver handle most of the rest. Your application code is mostly indistinguishable from code that talks to a single mongod. In PostgreSQL, choose your failover routing strategy (VIP, pooler, or multi-host), ensure your transactions are idempotent or wrap them in retry logic, and verify that your client library handles connection drops the way you expect. Neither approach is intrinsically better, but the work that has to happen in application code is meaningfully larger in PostgreSQL.

A worth-mentioning subtlety: read preference handling. Both systems let clients route reads to non-primary nodes, but they handle staleness differently. MongoDB drivers honor read preferences (primary, secondary, secondaryPreferred, nearest) and read concerns (local, majority, snapshot, linearizable) on a per-query basis, with the driver picking the right node. PostgreSQL clients connecting through a multi-host string can specify targetServerType=preferSecondary, but they cannot specify a read concern; staleness is whatever the standby happens to be at, with no protocol-level guarantee. PostgreSQL’s hot_standby_feedback parameter helps prevent vacuum-induced query cancellations on standbys, but the application has no way to ask for a strict freshness guarantee at query time.

Surveying the operational reality

Running these systems in production looks different. Both PostgreSQL and MongoDB demand operational competence, but the shape of the work is different enough that an operator skilled in one will need to relearn habits to be effective with the other.

A typical PostgreSQL HA stack includes more components than first-time operators expect. You have PostgreSQL itself. The primary and standbys, each with its own postgresql.conf, pg_hba.conf, and recovery configuration. You have Patroni or repmgr running alongside each Postgres instance, with its own configuration file and process supervision. You have etcd or Consul (often three or five nodes for fault tolerance), with its own deployment, security, and backup considerations. You have a connection-routing layer — PgBouncer, HAProxy, a cloud load balancer, or some combination. That needs to be told about topology changes. You have monitoring, e.g., pg_exporter for Prometheus, Patroni’s own metrics, and etcd’s metrics. Each producing its own view of cluster health. Configuration drift is a real risk: PostgreSQL configuration changes have to be coordinated with Patroni, which has its own opinions about which settings it owns versus which ones the operator can change directly.

A typical MongoDB HA deployment has fewer moving parts. You have mongod processes. Three or five of them. Each running a single configuration file. You have client drivers that handle topology discovery automatically. You have monitoring (Prometheus exporters, mongostat, the built-in $listSessions and serverStatus commands). That is, in many cases, it. There is no separate orchestration daemon, no external coordination service, no connection router. The simplicity is an asset for small operations teams and a liability for large ones, because it means you cannot mix-and-match components the way you can with PostgreSQL.

The flip side: PostgreSQL’s flexibility means each shop runs a slightly different stack, and skills are partially transferable. An operator who knows Patroni can move between teams and apply roughly the same playbook. An operator who has only used vanilla MongoDB has fewer options to deploy, but the operations are more uniform across deployments. The rs.status() output is the same on every replica set, the failover behavior is the same, the metrics names are the same. If you join a team running MongoDB and want to know how to fail over, the answer is in the documentation; if you join a team running PostgreSQL with Patroni, you need to read both the PostgreSQL and Patroni documentation, plus the team’s internal runbooks for how their etcd cluster is laid out.

Upgrades

Major version upgrades are a real pain point in PostgreSQL. Physical replication does not work across major versions, so the standard upgrade path involves either pg_upgrade (which upgrades a single instance in place, requiring downtime) or logical replication (set up a new version cluster, replicate data into it, switch over). Each approach has trade-offs. pg_upgrade is fast for small clusters and complex for large ones; logical replication is more robust for large clusters but requires schema compatibility and careful coordination. Tools like pg_easy_replicate automate parts of the logical replication path, but the operator still has to think about the cutover.

MongoDB upgrades are simpler in the common case. Replica sets support rolling upgrades: you upgrade one secondary, wait for it to rejoin, repeat for the other secondaries, then step down the primary and upgrade it last. Each version of MongoDB documents its compatibility window with the previous version, and feature compatibility flags let you upgrade the binaries before enabling new on-disk features. There are still pitfalls. Some upgrades require running specific scripts to update internal collections, and major version jumps may require intermediate stops, but the basic flow is straightforward and well-trodden.

Observability and debugging

Both systems expose detailed cluster state, but through different surfaces. PostgreSQL exposes pg_stat_replication on the primary (showing each connected standby and its lag) and pg_stat_wal_receiver on the standby (showing the receiver’s state). Patroni adds its own /cluster endpoint (or patronictl list output) that shows the topology, including which node is the leader. MongoDB exposes rs.status() and rs.printSecondaryReplicationInfo(), each producing a JSON document that captures the entire replica set’s state. The MongoDB output is denser and more uniform; the PostgreSQL output is more accurate to the actual database state, but requires you to know which catalogs to query.

Replication lag is monitored differently. In PostgreSQL, lag is measured in WAL bytes (replay_lsn vs current_lsn) or in time (now() - last_replay_timestamp). In MongoDB, lag is measured in OpTime difference, which is roughly equivalent to time since the OpTime is itself a Timestamp. Both systems can produce confusing numbers when a cluster is idle: a PostgreSQL standby that is fully caught up reports zero lag, but the timestamp-based metric drifts upward in idle periods because the primary has not generated new WAL. MongoDB’s OpTime gap behaves similarly. Operators learn to distinguish “real lag” from “cluster is idle” by checking whether traffic is flowing.

Choosing between them

After seven sections of comparison, the temptation is to declare one system the winner. The honest answer is that neither system wins on its own merits; they win for specific deployments. The right way to choose is to start with the workload and the team, then ask which system’s replication and failover model fits.

Where PostgreSQL’s model wins

PostgreSQL’s philosophy of primitives over policy plays to its strengths in environments that already have strong infrastructure conventions. If your platform team runs etcd for other purposes, integrating Patroni is straightforward and gives you HA that fits your existing operational mental model. If your application has heterogeneous replication needs, a synchronous standby in the same datacenter, an asynchronous standby for disaster recovery in another region, and a logical replica for analytics with a subset of tables. PostgreSQL’s independent replication mechanisms let you build the exact topology you need without compromise. MongoDB’s replica sets are uniform; you cannot mix synchronous and asynchronous nodes within a set the way PostgreSQL’s synchronous_standby_names allows.

Regulatory environments often favor PostgreSQL for similar reasons. Auditors like to see explicit configuration: this row in the postgresql.conf file says writes must be confirmed by two standbys, and you can read it. MongoDB’s per-query write concerns are auditable too, but the audit involves tracing through application code rather than reading server-side configuration. Both can satisfy compliance requirements, but PostgreSQL’s server-side approach is easier to centralize.

Workloads that benefit from logical replication — ETL pipelines, change data capture into Kafka or other streams, and gradual migrations between database versions — are often easier on PostgreSQL than on MongoDB. PostgreSQL’s logical replication is mature, supported by the database itself, and integrated with widely used tools (Debezium, pg_logical, pglogical). MongoDB has change streams, which solve the same problem with a different shape, but they live at a higher level and are tightly coupled to the MongoDB protocol; they are great for MongoDB-aware consumers and less useful when you need to feed data into systems that want SQL semantics.

Where MongoDB’s model wins

MongoDB’s built-in HA is a real asset for teams that do not have specialized database operations expertise. A four-person backend team can deploy a three-node MongoDB replica set in production and trust that automatic failover will work without ever reading about Raft. The same team running PostgreSQL with Patroni would need to learn etcd, distributed consensus failure modes, fencing, and a half-dozen Patroni configuration parameters before they could be confident the cluster would recover correctly from a primary failure. Both teams can succeed, but the MongoDB team starts higher up the learning curve.

Cloud-native workloads often fit MongoDB’s model well. A Kubernetes deployment that uses StatefulSets, headless services, and DNS-based discovery can run a MongoDB replica set with minimal extra infrastructure. The cluster discovers itself through DNS, drivers handle topology, and the operator does not need to maintain a separate coordination layer. PostgreSQL operators on Kubernetes typically run a Postgres operator (CloudNativePG, Stolon, Zalando’s Postgres Operator, Crunchy Data’s operator) that wraps Patroni or implements similar logic, which is not much extra work but is one more dependency to keep track of.

Workloads that want fine-grained durability per operation — some writes need majority acknowledgment, others can fire-and-forget — fit MongoDB’s write concerns naturally. PostgreSQL can simulate this with synchronous_commit overrides per session, but the model is more awkward. Conversely, an application that wants every write to have the same durability guarantee finds PostgreSQL’s server-side configuration easier to reason about.

A practical decision framework

When you cannot decide based on workload features alone, fall back to operational fit. Ask three questions. First, what does your team know? An ops team with deep PostgreSQL experience will run PostgreSQL well, even with the additional HA tooling overhead. A team with MongoDB experience will deliver more reliably on MongoDB. Skills compound; the database you are best at running is the database you should run, all else being equal.

Second, what does your platform look like? If your platform already provides Patroni-style coordination (etcd, Consul, Zookeeper), running PostgreSQL on top of it is straightforward. If your platform is opinionated about minimizing dependencies (small Kubernetes clusters, edge deployments, single-tenant workloads), MongoDB’s self-contained model is a better fit. The infrastructure you do not have is sometimes the deciding factor.

Third, what is your worst-case failure mode? PostgreSQL with Patroni has well-understood failure modes (etcd partition, fencing failure, pg_rewind divergence) that experienced operators can debug. MongoDB has its own (rollback files, network partitions that strand minorities, oplog window exhaustion) that experienced MongoDB operators can debug. The systems are roughly comparable in reliability when run by experienced teams. If your team is new to both, the simpler operational model of MongoDB usually wins, because there is less to misconfigure.

A final note: the choice is rarely permanent. Both systems are mature, both have migration tools, and both can be replaced if the original choice turns out wrong. The cost of switching is real but not catastrophic for most workloads. The cost of choosing badly and then refusing to revisit the decision is much higher. If you choose PostgreSQL and your team finds itself spending half its time tuning Patroni, that is a signal worth acting on. If you choose MongoDB and discover you really need cross-version logical replication, that is worth acting on too. The replication and failover models in this chapter are a starting point for the choice, not the end of it.

Summary

PostgreSQL exposes replication primitives (WAL streaming, replication slots, hot standbys) and leaves automatic failover to external tools such as Patroni, repmgr, pg_auto_failover, or Kubernetes-native operators like CloudNativePG.
MongoDB ships replication and automatic failover as part of the database itself, implemented through the ReplicationCoordinator, the TopologyCoordinator, and a Raft-based election protocol.
PostgreSQL supports two replication mechanisms: physical (byte-for-byte WAL streaming, used for HA) and logical (publish-subscribe row-level replication, used for cross-version upgrades and selective replication).
MongoDB has a single replication model based on the oplog, a capped collection that records logical operations and is tailed by secondaries through pull-based replication.
Synchronous durability is configured at the cluster level in PostgreSQL through synchronous_standby_names (with FIRST and ANY methods) and at the per-write level in MongoDB through write concerns.
Failure detection in MongoDB happens through a heartbeat protocol that every node runs internally; PostgreSQL has no equivalent in core and relies on external monitors for detection.
Leader election in MongoDB uses Raft consensus with a dry-run phase to prevent unnecessary term increments; PostgreSQL leader election runs in external tools (Patroni with etcd, repmgr, or pg_auto_failover).
Split-brain prevention in both systems requires a majority of voting members; without a majority, the cluster cannot elect a primary or accept new writes.
MongoDB drivers maintain a topology view of the cluster and handle failover transparently, with retryable writes providing automatic safe retry across primary changes.
PostgreSQL clients connect to a hostname and require external mechanisms (VIPs, connection poolers like PgBouncer, or multi-host JDBC strings) to route connections after a failover.
In-flight transactions on a failed PostgreSQL primary are lost and must be retried by the application; MongoDB unprepared transactions are explicitly aborted, while prepared transactions survive stepdown for sharded two-phase commit safety.
A typical PostgreSQL HA stack involves PostgreSQL plus Patroni plus etcd plus a connection-routing layer; a typical MongoDB HA deployment involves only the mongod processes and standard drivers.
Major-version upgrades are easier in MongoDB (rolling upgrades within a replica set) than in PostgreSQL (where physical replication does not span major versions and pg_upgrade or logical replication is required).
Choose PostgreSQL when you need replication flexibility, mature logical replication tooling, or fit with existing infrastructure conventions (etcd, Consul, Kubernetes operators).
Choose MongoDB when you want HA without a separate coordination layer, per-operation durability semantics, or operational simplicity for teams without deep database operations expertise.

Test Password Strength and Password History with TypeScript and MongoDB

Nic Raboy — Fri, 15 May 2026 13:55:05 +0000

Just about every application you build is going to need some form of user authentication, and the moment you have user accounts, you have passwords to manage. Storing them safely is only part of the job. You also need to make sure those passwords are strong enough to be worth protecting in the first place, and in many cases, you need to make sure the same password isn't recycled every time it’s rotated.

Password reuse is a bigger deal than it sounds. Compliance frameworks frequently require that the last several passwords cannot be used again, and even outside of compliance, it is good hygiene. If a password leaks today and a user just rotates back to it six months from now, your rotation policy did not actually protect anyone.

MongoDB happens to be a really good fit for this kind of feature. A user document can store the current password hash at the root of the document, along with a growing array of previous hashes, all in the same record. There is no separate join table to set up and no migration to run when a user changes their password for the tenth time.

In this tutorial, we'll see how to build a small TypeScript API using Express Framework, Zod, and the MongoDB Node.js driver. This application registers users with strong password rules, authenticates them, and rejects any password change that has been seen before.

The Prerequisites

Prior to starting this tutorial, you'll need a few things in place:

A MongoDB instance, either a local server or an Atlas cluster on the free tier
Node.js 22+

The expectation is that you can already connect to MongoDB with a connection string. If you need help getting an Atlas cluster running, check out the MongoDB documentation for getting started.

We're working from an empty project, so the first thing to do is create the directory and initialize it:

mkdir typescript-password-history-example
cd typescript-password-history-example
npm init -y

The above commands create the project directory and a starter package.json file.

Next, we'll install the runtime dependencies:

npm install express mongodb bcrypt zod dotenv

We're pulling in Express Framework as the HTTP layer, the official MongoDB Node.js driver, bcrypt for hashing passwords, zod for schema validation, and dotenv for loading configuration from an environment file.

For TypeScript and the type definitions, run:

npm install --save-dev typescript ts-node @types/express @types/node @types/bcrypt

The ts-node package lets us run TypeScript directly during development without a separate compile step, keeping the feedback loop tight.

Because this is a TypeScript project, we need a tsconfig.json at the root of the project:

{
    "compilerOptions": {
        "target": "ES2020",
        "module": "commonjs",
        "lib": ["ES2020"],
        "outDir": "./dist",
        "rootDir": "./",
        "strict": true,
        "esModuleInterop": true,
        "skipLibCheck": true,
        "forceConsistentCasingInFileNames": true,
        "resolveJsonModule": true
    },
    "include": ["./**/*.ts"],
    "exclude": ["node_modules", "dist"]
}

The configuration emits CommonJS to a dist/ directory when we run tsc, and as strict mode is turned on, the compiler actually enforces type safety rather than letting things slide.

It also helps to update the scripts block of package.json, so we have a development command and a production command:

"scripts": {
    "build": "tsc",
    "start": "node dist/main.js",
    "dev": "ts-node main.ts"
}

The dev script will run the TypeScript entry point directly. The build script compiles to dist/, and start runs the compiled output.

By the time we're done, the project will have the following shape:

.
├── .env
├── main.ts
├── tsconfig.json
├── package.json
├── libs/
│   ├── mongodb.ts
│   └── validate.ts
└── routes/
    └── passwords.ts

Each of those files will get built up as we work through the tutorial. Go ahead and create the libs and routes directories now so you have somewhere to drop code as we go.

The last piece of setup is the environment file. Create a .env file at the root of the project with the following:

MONGODB_URI=ADD_YOUR_ATLAS_URI_HERE
MONGODB_DB_NAME=password_history_demo
PORT=3000

Replace the MONGODB_URI with the connection string for whatever MongoDB instance you're using. The password_history_demo database specified in MONGODB_DB_NAME does not need to exist ahead of time. MongoDB will create it lazily the first time we write data. If you're using version control, make sure .env is listed in your .gitignore so the connection string does not get committed.

Establish an API Foundation with Express and TypeScript

Before we get into password logic, we need an Express application that listens for HTTP requests and a clean way to start and stop it. The entry point lives in main.ts at the root of the project.

Add the following to main.ts:

import * as dotenv from "dotenv";
dotenv.config();

import express from "express";
import { getDb, closeConnection } from "./libs/mongodb";
import passwordRouter from "./routes/passwords";

const app = express();
const PORT = process.env.PORT ?? 3000;

app.use(express.json());

app.use("/users", passwordRouter);

async function start() {
    const db = await getDb();
    await db.collection("users").createIndex({ username: 1 }, { unique: true });

    const server = app.listen(PORT, () => {
        console.log(`Server running on http://localhost:${PORT}`);
    });

    process.on("SIGTERM", async () => {
        console.log("SIGTERM received, shutting down gracefully...");
        server.close(async () => {
            await closeConnection();
            process.exit(0);
        });
    });

    process.on("SIGINT", async () => {
        console.log("SIGINT received, shutting down gracefully...");
        server.close(async () => {
            await closeConnection();
            process.exit(0);
        });
    });
}

start();

That file is doing a lot, so let's pull it apart.

At the top, we have two dotenv lines followed by the rest of the imports:

import * as dotenv from "dotenv";
dotenv.config();

import express from "express";
import { getDb, closeConnection } from "./libs/mongodb";
import passwordRouter from "./routes/passwords";

The order matters here. dotenv.config() runs before the MongoDB module is imported, which means the connection string and database name will already be present in process.env by the time libs/mongodb.ts is evaluated.

Next, we have the Express application setup:

const app = express();
const PORT = process.env.PORT ?? 3000;

app.use(express.json());

app.use("/users", passwordRouter);

The express.json() middleware parses incoming JSON bodies so we can read req.body as a plain object later. The router is mounted under /users, so every endpoint in routes/passwords.ts will be reachable beneath that prefix. We'll write that router shortly.

Then we have the start function, which is the part that actually brings the server online:

async function start() {
    const db = await getDb();
    await db.collection("users").createIndex({ username: 1 }, { unique: true });

    const server = app.listen(PORT, () => {
        console.log(`Server running on http://localhost:${PORT}`);
    });

    // ... signal handlers shown below
}

The first thing it does is grab a database reference and create a unique index on the username field of the users collection. Doing it at startup means we don't have to check for duplicate usernames before every insert. MongoDB will raise an error with code 11000 if anyone tries to register a username that already exists, and we'll catch that error in the route handler. It is a much cleaner pattern than running a findOne followed by an insertOne, which is also racy.

After the index is ready, we call app.listen and hold onto the returned server reference so we can close it later.

The final piece is the shutdown handling, which lives inside the same start function:

async function start() {
    // ... database, index, and app.listen above

    process.on("SIGTERM", async () => {
        console.log("SIGTERM received, shutting down gracefully...");
        server.close(async () => {
            await closeConnection();
            process.exit(0);
        });
    });

    process.on("SIGINT", async () => {
        console.log("SIGINT received, shutting down gracefully...");
        server.close(async () => {
            await closeConnection();
            process.exit(0);
        });
    });
}

When the process receives either SIGTERM (which most container orchestrators send) or SIGINT (which is what Ctrl+C sends), we stop accepting new connections, close the MongoDB client, and exit. Without this, you'd leave an open client connection behind every time you restart the server.

Of course, none of this will compile yet because libs/mongodb.ts does not exist. That's our next step.

Create a Singleton MongoDB Class to Manage Database Connections

You could open a new MongoClient on every incoming request, but it is almost never what you want. Connecting is expensive, it adds latency, and it makes graceful shutdown much harder than it needs to be. A long-running API like ours should connect once when the process boots and reuse the same client for every request after that.

Open libs/mongodb.ts and add the following:

import { MongoClient, Db } from "mongodb";

const uri = process.env.MONGODB_URI as string;
const dbName = process.env.MONGODB_DB_NAME as string;

if (!uri) {
    throw new Error("MONGODB_URI is not defined in the environment variables");
}

if (!dbName) {
    throw new Error("MONGODB_DB_NAME is not defined in the environment variables");
}

let client: MongoClient | null = null;

export async function getClient(): Promise<MongoClient> {
    if (!client) {
        client = new MongoClient(uri, {
            connectTimeoutMS: 5000,
            socketTimeoutMS: 30000,
            serverSelectionTimeoutMS: 5000,
            appName: "devrel-tutorial-typescript-passwordhistory"
        });
        await client.connect();
    }
    return client;
}

export async function getDb(): Promise<Db> {
    const mongoClient = await getClient();
    return mongoClient.db(dbName);
}

export async function closeConnection(): Promise<void> {
    if (client) {
        await client.close();
        client = null;
    }
}

The top of the file reads the two environment variables we set earlier and fails fast if either of them is missing. There is no point in starting the server if it cannot reach the database.

The interesting bit is getClient:

export async function getClient(): Promise<MongoClient> {
    if (!client) {
        client = new MongoClient(uri, {
            connectTimeoutMS: 5000,
            socketTimeoutMS: 30000,
            serverSelectionTimeoutMS: 5000,
            appName: "devrel-github-typescript-passwordhistory"
        });
        await client.connect();
    }
    return client;
}

The module-level client variable starts as null. The first call to getClient instantiates the MongoClient, awaits connect, and caches it. Every subsequent call returns the cached instance immediately. That is the entire singleton pattern. We did not bother setting a maxPoolSize because this is a traditional long-running OLTP server, and the default value of 100 is appropriate for that workload.

The three timeout options are worth a quick mention. connectTimeoutMS caps how long a single socket connection attempt can take, socketTimeoutMS caps how long a socket can sit idle, and serverSelectionTimeoutMS caps how long the driver will spend looking for a suitable server before giving up. The appName value is cosmetic but useful. It shows up in MongoDB Atlas logs and metrics, so you can tell which application is generating which queries.

Then we have the two helpers that the rest of the application actually uses:

export async function getDb(): Promise<Db> {
    const mongoClient = await getClient();
    return mongoClient.db(dbName);
}

export async function closeConnection(): Promise<void> {
    if (client) {
        await client.close();
        client = null;
    }
}

The getDb function returns a Db reference scoped to the database name from the environment file. Anywhere in the codebase that needs to talk to MongoDB will go through this function. The closeConnection function shuts the client down and resets the module-level variable. This is what gets called by the SIGTERM and SIGINT handlers back in main.ts.

Test Password Strength with Regular Expressions and Zod

A password is only as good as the rules you enforce around it. For this tutorial, we're going to require a minimum of eight characters, at least one uppercase letter, at least one lowercase letter, at least one digit, and at least one character that is not a letter or a number. None of these rules is particularly exotic, but together they keep users from registering with something like password or 12345678.

We're going to enforce them with Zod schemas wired into Express as middleware. That way, the route handlers themselves stay focused on database work and never have to inspect raw request bodies.

Open libs/validate.ts and add the following:

import { Request, Response, NextFunction } from "express";
import { ZodSchema } from "zod";

export function validate(schema: ZodSchema) {
    return (req: Request, res: Response, next: NextFunction): void => {
        const result = schema.safeParse(req.body);
        if (!result.success) {
            res.status(400).json({ errors: result.error.flatten().fieldErrors });
            return;
        }
        req.body = result.data;
        next();
    };
}

The validate function is a middleware factory. You hand it a Zod schema, and it hands you back an Express middleware function that runs that schema against the incoming req.body. If validation fails, the middleware sends back a 400 with a flattened map of field errors, and the route handler is never called. If validation succeeds, it replaces req.body with the parsed output before calling next().

Replacing req.body with result.data is intentional. Zod will strip unknown keys and apply any transformations defined on the schema, so by the time the route handler runs, the body is guaranteed to look exactly like the schema declared. There is no risk of a stray field sneaking through.

Now we can define the actual schemas. These live alongside the route handlers in routes/passwords.ts, but we'll write that file in pieces throughout the rest of the tutorial. For now, here's the schema that encodes the password strength rules:

import { z } from "zod";

const strongPasswordSchema = z
    .string()
    .min(8, "Password must be at least 8 characters")
    .regex(/[A-Z]/, "Password must contain at least one uppercase letter")
    .regex(/[a-z]/, "Password must contain at least one lowercase letter")
    .regex(/[0-9]/, "Password must contain at least one digit")
    .regex(/[^A-Za-z0-9]/, "Password must contain at least one special character");

Each rule is its own .regex call with a custom error message. If a user submits something that breaks two rules, Zod will report both errors at once, which makes for a much better registration experience than trickling them out one at a time.

The character class [^A-Za-z0-9] is doing the work for the special character rule. It matches anything that is not an uppercase letter, a lowercase letter, or a digit. We're not picky about which special character. Punctuation, symbols, and even whitespace will satisfy it.

With the strong password schema in place, we can compose the three request schemas the API needs:

const registerSchema = z.object({
    username: z.string().min(1, "Username is required").trim(),
    password: strongPasswordSchema,
});

const loginSchema = z.object({
    username: z.string().min(1, "Username is required").trim(),
    password: z.string().min(1, "Password is required"),
});

const changePasswordSchema = z.object({
    current_password: z.string().min(1, "Current password is required"),
    new_password: strongPasswordSchema,
});

Registration uses the strict rules on password because that's a brand new password being committed to the database. The change-password schema uses the strict rules on new_password for the same reason.

Notice that loginSchema and the current_password field in changePasswordSchema do not use the strict rules. That's deliberate. The strength rules are about what we are willing to store, not what we are willing to compare against. A user who registered before you tightened your password policy still needs to be able to log in with their old, weak password so they can rotate to a stronger one. If the strong rules applied to the login flow, you would lock those users out permanently.

Store Passwords Safely in MongoDB with Password Change History

Now we get to the part this whole tutorial is named after. The shape of a user document is going to look something like this:

{
    "_id": "ObjectId(...)",
    "username": "nraboy",
    "password": "BCRYPT_HASH",
    "password_history": [
        { "password": "BCRYPT_HASH", "created_at": "ISODate(...)" }
    ],
    "created_at": "ISODate(...)",
    "updated_at": "ISODate(...)"
}

The current password hash lives at the root of the document. That's the value we compare against during login, and it is the one we update during a password change. Alongside it, we keep a password_history array where each entry is a previous hash with the timestamp it was set. When a user first registers, the history starts with a single entry that mirrors the root password. Every password change appends another entry. To check if a candidate password has been used before, we walk the history and compare against each entry.

Let's start building routes/passwords.ts. The top of the file pulls in the dependencies, defines the data shapes, and declares a salt rounds constant:

import { Router, Request, Response } from "express";
import { z } from "zod";
import bcrypt from "bcrypt";
import { getDb } from "../libs/mongodb";
import { MongoServerError } from "mongodb";
import { validate } from "../libs/validate";

const SALT_ROUNDS = 12;

type PasswordHistoryEntry = {
    password: string;
    created_at: Date;
};

type UserDocument = {
    username: string;
    password: string;
    password_history: PasswordHistoryEntry[];
    created_at: Date;
    updated_at: Date;
};

The two types describe the user document and a single history entry. We're using them as generic parameters on db.collection<UserDocument>("users") calls below so TypeScript can give us reasonable autocomplete when we read fields from the results.

Twelve salt rounds is a sensible default for bcrypt. It is slow enough to make brute-force attacks expensive and fast enough that a legitimate login does not feel sluggish on modern hardware.

Next, define three small helper functions for working with bcrypt:

async function hashPassword(plaintext: string): Promise<string> {
    return bcrypt.hash(plaintext, SALT_ROUNDS);
}

async function verifyPassword(plaintext: string, hash: string): Promise<boolean> {
    return bcrypt.compare(plaintext, hash);
}

async function isPasswordInHistory(
    plaintext: string,
    history: PasswordHistoryEntry[]
): Promise<boolean> {
    for (const entry of history) {
        const match = await bcrypt.compare(plaintext, entry.password);
        if (match) return true;
    }
    return false;
}

The first two are thin wrappers around the bcrypt API. They exist mostly to keep the route handlers readable.

The third one, isPasswordInHistory, is where the history check actually happens. It loops through every entry in the array and calls bcrypt.compare against each one. You cannot just hash the candidate password and look for an exact string match because bcrypt mixes a fresh random salt into every hash. Two calls to bcrypt.hash("Nic12345$", 12) will produce two different hash strings. The only way to know if a plaintext matches a stored hash is to feed both into bcrypt.compare.

The loop short-circuits the moment it finds a match, so in the common case where the user picks something genuinely new, the cost is proportional to the size of the history. For typical history sizes of five to ten entries, that overhead is negligible.

With the schemas from the previous section and the helpers above, we can start defining routes. Initialize the router:

const router = Router();

And start with the demo endpoint that lists every user document. This one only exists so you can poke at the database during development:

router.get("/", async (_req: Request, res: Response): Promise<void> => {
    try {
        const db = await getDb();
        const users = await db.collection<UserDocument>("users").find({}).toArray();
        res.status(200).json(users);
    } catch (err) {
        console.error(err);
        res.status(500).json({ error: "Internal server error" });
    }
});

This endpoint returns the full user document, including the bcrypt hash and the entire history array. You would never ship something like this in production. It is genuinely useful while you're developing, though, because it lets you see exactly what the password history looks like as it grows.

The registration endpoint is where new users get created:

router.post("/", validate(registerSchema), async (req: Request, res: Response): Promise<void> => {
    const { username, password } = req.body;

    try {
        const db = await getDb();
        const hashed = await hashPassword(password);
        const now = new Date();

        await db.collection("users").insertOne({
            username,
            password: hashed,
            password_history: [{ password: hashed, created_at: now }],
            created_at: now,
            updated_at: now,
        });

        res.status(201).json({ message: "User created successfully" });
    } catch (err) {
        if (err instanceof MongoServerError && err.code === 11000) {
            res.status(409).json({ error: "Username already exists" });
            return;
        }
        console.error(err);
        res.status(500).json({ error: "Internal server error" });
    }
});

There are a couple of things worth pointing out here. First, validate(registerSchema) runs before the handler. If validation fails, the handler never executes, so by the time we destructure username and password from req.body, we already know they're well-formed.

Second, the new document seeds password_history with a single entry that contains the same hash we just stored at the root. That way, the very first password is part of the history from day one. If we did not do that, the user could "rotate" their password back to the original on their first password change, which is exactly the behavior we're trying to prevent.

Third, instead of running a findOne to see if the username is taken and then doing an insertOne, we just attempt the insert and catch MongoServerError with code 11000. That's the duplicate-key error code, and it is the cleanest way to enforce uniqueness because MongoDB itself is the only thing that needs to coordinate. There is no window between the check and the insert during which two requests could both decide the username is free.

The login endpoint is much simpler:

router.post("/login", validate(loginSchema), async (req: Request, res: Response): Promise<void> => {
    const { username, password } = req.body;

    try {
        const db = await getDb();
        const users = db.collection<UserDocument>("users");

        const user = await users.findOne({ username });
        if (!user) {
            res.status(401).json({ error: "Invalid credentials" });
            return;
        }

        const valid = await verifyPassword(password, user.password);
        if (!valid) {
            res.status(401).json({ error: "Invalid credentials" });
            return;
        }

        res.status(200).json({ message: "Login successful" });
    } catch (err) {
        console.error(err);
        res.status(500).json({ error: "Internal server error" });
    }
});

Notice that both failure cases return the same 401 with the same generic message. There is a temptation to send back "User not found" when the lookup misses and "Wrong password" when the comparison fails, but doing so leaks information. An attacker who can tell those two states apart can enumerate which usernames exist in your system, which is the first half of a credential stuffing attack. Returning a single opaque error keeps both branches indistinguishable.

This is also why a real production version would want some kind of rate limiting on this endpoint. We're not adding that here, but it is worth keeping in mind.

The change-password endpoint is the one where the history check actually pays off:

router.put("/:username/password", validate(changePasswordSchema), async (req: Request, res: Response): Promise<void> => {
    const { username } = req.params;
    const { current_password, new_password } = req.body;

    try {
        const db = await getDb();
        const users = db.collection<UserDocument>("users");

        const user = await users.findOne({ username });
        if (!user) {
            res.status(404).json({ error: "User not found" });
            return;
        }

        const currentValid = await verifyPassword(current_password, user.password);
        if (!currentValid) {
            res.status(401).json({ error: "Current password is incorrect" });
            return;
        }

        const history = user.password_history ?? [];

        const alreadyUsed = await isPasswordInHistory(new_password, history);
        if (alreadyUsed) {
            res.status(409).json({ error: "New password has been used before. Please choose a different password." });
            return;
        }

        const hashed = await hashPassword(new_password);
        const now = new Date();

        await users.updateOne(
            { username },
            {
                $set: {
                    password: hashed,
                    updated_at: now,
                },
                $push: {
                    password_history: { password: hashed, created_at: now },
                },
            }
        );

        res.status(200).json({ message: "Password updated successfully" });
    } catch (err) {
        console.error(err);
        res.status(500).json({ error: "Internal server error" });
    }
});

The handler runs through four checks in order. First, we look up the user by the username from the route parameter, returning a 404 if the user does not exist. Then we verify the supplied current_password against the stored hash. A mismatch returns a 401, which is appropriate because the caller is, in effect, failing to authenticate.

After that, we run the history check:

router.put("/:username/password", validate(changePasswordSchema), async (req: Request, res: Response): Promise<void> => {
    // ... user lookup and current password verification above

    const history = user.password_history ?? [];

    const alreadyUsed = await isPasswordInHistory(new_password, history);
    if (alreadyUsed) {
        res.status(409).json({ error: "New password has been used before. Please choose a different password." });
        return;
    }

    // ... update logic below
});

The nullish coalescing operator gives us a safe default for any users that might somehow be missing the password_history field, though in practice, the registration handler always sets it. If the candidate password matches any entry in the history, we send back a 409 and explain the situation to the caller.

If we make it past all four checks, the update itself uses two operators in one call:

router.put("/:username/password", validate(changePasswordSchema), async (req: Request, res: Response): Promise<void> => {
    // ... lookup, verification, and history check above

    const hashed = await hashPassword(new_password);
    const now = new Date();

    await users.updateOne(
        { username },
        {
            $set: {
                password: hashed,
                updated_at: now,
            },
            $push: {
                password_history: { password: hashed, created_at: now },
            },
        }
    );

    res.status(200).json({ message: "Password updated successfully" });
});

The $set operator overwrites the root password field with the new hash and bumps updated_at. At the same time, $push appends the new hash to the password_history array. Because both operators are part of the same update, the change is atomic from MongoDB's perspective. The current password and the history can never disagree about the user's most recent password.

Finally, the bottom of the file exports the router so main.ts can mount it:

export default router;

That completes the route layer. Time to try it out.

Testing the API Endpoints with cURL

We'll exercise the API from the command line with curl. The repository also includes a Bruno collection in the bruno/ directory if you'd rather click through requests in a GUI, but curl is faster to demonstrate in a tutorial.

Start the server in development mode:

npm run dev

You should see Server running on http://localhost:3000 in the terminal. Leave it running and open a second terminal for the requests.

First, register a new user:

curl -s -X POST http://localhost:3000/users \
    -H "Content-Type: application/json" \
    -d '{"username":"nraboy","password":"Nic12345$"}'

A successful registration responds with a 201 and {"message":"User created successfully"}. If you try the exact same request again, you'll get a 409 because the unique index on username rejects the duplicate.

To prove the validation middleware is doing its job, try sending a weak password:

curl -s -X POST http://localhost:3000/users \
    -H "Content-Type: application/json" \
    -d '{"username":"weakuser","password":"password"}'

The response is a 400 with a per-field map of the rules that were broken. No insert ever reaches MongoDB.

Now try logging in with the user we just created:

curl -s -X POST http://localhost:3000/users/login \
    -H "Content-Type: application/json" \
    -d '{"username":"nraboy","password":"Nic12345$"}'

A correct password returns {"message":"Login successful"}. Get one character wrong, and you'll see {"error":"Invalid credentials"} with a 401.

The interesting endpoint is the password change. Rotate to a brand new password:

curl -s -X PUT http://localhost:3000/users/nraboy/password \
    -H "Content-Type: application/json" \
    -d '{"current_password":"Nic12345$","new_password":"Nic67890$"}'

That responds with a 200 and {"message":"Password updated successfully"}. Now try to rotate back to the original password:

curl -s -X PUT http://localhost:3000/users/nraboy/password \
    -H "Content-Type: application/json" \
    -d '{"current_password":"Nic67890$","new_password":"Nic12345$"}'

This time the response is a 409 and {"error":"New password has been used before. Please choose a different password."}. The history check did exactly what we wanted.

You can verify what's actually in the database by hitting the demo list endpoint:

curl -s http://localhost:3000/users

The response includes the password_history array with two entries, one for the original password and one for the rotation, each with its own timestamp and bcrypt hash. Two different hashes for the same logical password value, which is the whole point of using bcrypt.compare rather than string equality during the history check.

Conclusion

You just saw how to combine TypeScript, Express Framework, Zod, and the MongoDB Node.js driver to enforce password strength and prevent password reuse. Zod schemas plugged into Express as middleware handle the strength rules, bcrypt handles the hashing, and a password_history array on each user document keeps a running record of every hash that user has ever used. The singleton MongoDB client keeps the connection layer simple, and a unique index on username lets MongoDB itself enforce uniqueness without us having to coordinate it in application code.

What we built is good for a demo, but a few things would need to change before this could be a production system. There is no rate limiting on the login or password change endpoints, both of which are attractive targets for brute-force attempts. There is no session or token mechanism, so right now a successful login just sends back a message. The GET /users endpoint exposes every hash in the database and would need to be removed entirely. And depending on your compliance requirements, you may want to cap the size of password_history so it does not grow without bound.

As a next step, try adding a password expiration field that forces a rotation after some number of days, or capping password_history so only the last ten entries are retained.

The full source for this project is available on GitHub.

Whistleblower for Database: Set up an internal informant who exposes performance

MongoDB Guests — Thu, 07 May 2026 10:44:20 +0000

This tutorial was written by Darshan Jayarama.

As DBAs, waking up at 2 AM because production is down or slow is a nightmare, and none of us like it. However, we all know that it is not instant karma; it is like a pressure cooker where the database has reached its breaking point. Database whistleblowers constantly give signals that we ignore before it explodes into a Production Issue.

Whistleblower 1: Performance Informant

High CPU utilization?

CPU utilization is not just an alert, it is a future prediction with regard to the database. When you get constant High CPU utilization alerts, think of it as a Whistleblower reviewing the database logs, metrics, and processes that are consuming CPU. If possible, shift those processes to a different server and let your Database breathe without pressure. Ignoring high CPU utilization can lead to high latency.

Query <5 seconds?

A query that was blazing fast is now causing a slow query alert since the last data load. It is a clear indication of a Missing or Unoptimized index. If an index is missing, create one. If unoptimized, then redesign a better index. Still facing the same? It needs a deeper and wider investigation.

Cache hit ratio <80%?

A lower cache hit ratio means higher disk reads, which tends to cause high IO Wait, and an Increased disk queue depth. It can happen if your query is unoptimized (we had addressed this earlier), or if lower memory/cache is assigned. Increase the memory if there is nothing pending in Database tuning.

Whistleblower 2: Cost Informant

Storage >80%:

Set your whistleblower when storage utilization hits >80% its your duty now to analyze storage to archive any old logs, old data, or remove unwanted processes that consume your space.

IOPS > Budget?

We usually get an alert for a high document-scanned-to-returned ratio, which indicates that queries are performing more IO than required. Optimize them. Reduce IOPS consumption by rewriting queries, redesigning the schema, and implementing a better Index strategy.

Unexpected auto scale?

MongoDB Atlas has a feature that monitors system load and automatically scales up or down as needed. Implement the solution. In case of a scale down, it does require 24-hour monitoring, but if you are sure there are no more surprises, you can scale down yourself.

Whistleblower 3: Reliability Informant

Replication lag >10 Seconds?

Investigate where it is lagging. Is it in primary? Must be loaded from writes, which lowers the read request priority. It usually triggers the built-in flow-control logic. If secondary is the issue, then is it at writing or reading? A detailed understanding of flow can help resolve this data durability. Ask developers to set writeConcern to match the application's requirements.

Oplog <24Hours?

A lower oplog size means lower retention of replication events. Set at least 24 hours, resize as per the application write load minOplogRetention.

Connection >80%?

Each connection can consume 1 MB of memory; the more connections, the more memory is consumed. Maintain connection hygiene, use a connection pool, and close connections when no longer needed.

Conclusion: Your production incident is one missed alert away; hire your whistleblower. Choose: Pressure cooker explosion → OR → Proactive DBA legend?