DEV Community: MongoDB

Build Your AI-Powered Reading List Using DigitalOcean Functions, Mistral AI, and MongoDB

MongoDB Guests — Wed, 22 Jul 2026 10:24:29 +0000

This Tutorial was written by Diego Freniche.

Since I started browsing the Internet, I've used many different services, applications, and browser plugins (even the good old bookmarks) to mark articles and posts that seemed interesting and that I wanted to read. Only not now. I'll read them later, I promise. And then you hoard a bunch of links in one place. But when you go back to actually read them, you find yourself clicking again on them just to remember what those links where about and why did you store them in the first place...

Fast forward today, and I haven't changed much in how I interact with articles I want to read later. But nowadays we can use LLMs almost for free... and they are really good at writing text summaries. So, why not create a system that, given a link, generates a summary and then stores both the link and the summary in a fast database I can use to build my own UI later?

And this is now yet another read-it-later (maybe) system was born...

Prerequisites

Before you begin this guide, you'll need the following:

a DigitalOcean account, that we'll use to write a DigitalOcean Function. This function will receive our links and run the code that will generate the LLM-powered summary and store the results in a database.
a MongoDB database, also running in DigitalOcean. We will use it to store the links and summaries of the web pages. You can learn more about deploying a managed MongoDB instance in DigitalOcean here.
access to any LLM that can write a good summary. For this example, I'll use Mistral AI, so you'll need to register a free account there and get an API token. Take a note of that API key, we'll need it later!

Step 1 - Creating an Empty DigitalOcean Function

Let's start by creating the backbone of our system. We can write a backend app using Node, Spring Boot or any other web framework. But then we need to host it in a server, write the code, correctly size the server, update the operating system and dependencies, etc. Using a serverless function we focus on the task at hand. In this case, getting a URL via a parameter, producing a summary of the page, and storing the data in the database.

So head over to Functions and click on create > Functions. First, we need to choose where our serverless function will be hosted. Although it's a serverless function, our code will need a real server to run. Only we don't have to provide it. Choose a data center close to your location (or close to where the users of your function will be). Then give it a name, choosing a label. In my case, it will be read-it-later-i-pinly-promise.

After creating the namespace, you'll be in the Overview tab. Let's add some basic code to test our function! Just click on the Create Function button.

We need to select one of the supported runtimes (in our case, Nodejs 18), and we'll give the function a name: read_later. It's important to leave the Web Function box checked, as we want to call this function directly using HTTP.

Done! Now you should see the default source code of the newly created function. This just gets an argument name from the URL and returns it. As we're not configuring a web page or a JSON response, this will be printed out as plain text by our web browser.

function main(args) {
let name = args.name || 'stranger' let greeting = 'Hello ' + name + '!' console.log(greeting)
return {"body": greeting}
}

In the main function, we receive a JSON object with all the arguments passed in the function call, we search for name, and use it. If not found, we use stranger as name. Then we compose and print out a greeting which we also return from our function.

Step 2 - testing our function

As this is accessible via HTTP calls, we have several different options to test our function.

Using cURL

If your system includes curl (Microsoft Windows 10/11 and most UNIX systems, like Linux and macOS come with cURL preinstalled), you can copy the URL of your function and add the name parameter at the end. Then call curl with:

curl https://<redacted>.doserverless.co/api/v1/web/<redacted>/default/read_later? name=Diego

In your case, instead of <redacted>, you'll see the ids of your functions. If you call the function from a Terminal, you'll see the response from our function:

curl https://<redacted>.doserverless.co/api/v1/web/<redacted>/default/read_later? name=Diego
Hello Diego!%

Try it changing the name and the code in your function and experiment with it! It's super fun!

Testing directly in DigitalOcean

We can click on Test Parameters and add a JSON with our test data, in this case:

{ "name": "Diego" }

Then, we can hit the Run button and see the results in the Output and Logs windows.

Step 3 - Adding the LLM Code

First, we will change our variable names to reflect what we're doing: producing summaries from links. Instead of name we'll use a link variable. Also, we'll return a proper HTTP response, with
statusCode 200 in case we got a link.

export async function main(args) { const link = args.link;
if (!link) { return {
statusCode: 400,
headers: { "Content-Type": "application/json" }, body: { error: "Please send a link" }
};
}
const body = { URL: link }; return {
statusCode: 200,
headers: { "Content-Type": "application/json" }, body: body
};
}

After changing the code, hit the "Save" button to avoid losing your changes. You can now re-run your function and check the results.

While we're at it, let's also change the Test Parameters (if you're testing directly from the browser) to:

(
"link": "https://en.wikipedia.org/wiki/Seville"
)

Seems like we're about to travel to Seville!

After that, we'll add the code to interact with Mistral. We need to send a POST with the prompt to their endpoint. For this, we'll need an API key from Mistral AI (see prerequisites). Save the code, but don't run the function (yet) as we need to add the secret.

const API_KEY = process.env.MISTRAL_API_KEY; // secrets are stored in the function's Settings
const API_URL = 'https://api.mistral.ai/v1/chat/completions';

async function chatWithLeChat(prompt) { try {
// call the Mistral endpoint with POST const res = await fetch(API_URL, {
method: 'POST', headers: {
'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json',
},
body: JSON.stringify({ model: 'mistral-tiny',
messages: [{ role: 'user', content: prompt }],
}),
});

const data = await res.json(); console.log(res.status, data);
return data?.choices?.[0]?.message?.content || null;
} catch (error) {
console.error('Error:', error?.message || error); return null;
}

Step 4 - Adding Secrets

As you can see, in our code, we don't embed the secrets (like API keys). Instead, we use the Functions environment to provide these:

const API_KEY = process.env.MISTRAL_API_KEY;

To add this secret MISTRAL_API_KEY we click on Settings. Then, we scroll down to Environment Variables. There, we can click on Edit

Now, click on Add and then Save. You should see your new environment variable now!

Step 5 - getting the LLM summary

It's time to call our chatWithLeChat function from main! To do that, we 1st create a system prompt that includes the link we received and instructs the LLM on what to do. Then, we'll call Mistral:

const prompt = 'I need a summary of this page: ' + link;

const summary = await chatWithLeChat(prompt);

We will extend our body to include the summary:

const body = { URL: link , summary: summary
};

This is all our code right now:

export async function main(args) { const link = args.link;
if (!link) { return {
statusCode: 400,
headers: { "Content-Type": "application/json" }, body: { error: "Please send a link" }
};
}

const prompt = 'I need a summary of this page: ' + link; const summary = await chatWithLeChat(prompt);
const body = { URL: link , summary: summary
};

return { statusCode: 200,
headers: { "Content-Type": "application/json" }, body: body
};
}

const API_KEY = process.env.MISTRAL_API_KEY; // secrets are stored in the function's Settings
const API_URL = 'https://api.mistral.ai/v1/chat/completions';

async function chatWithLeChat(prompt) { try {
// call the Mistral endpoint with POST const res = await fetch(API_URL, {
method: 'POST', headers: {
'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json',
},
body: JSON.stringify({ model: 'mistral-tiny',
messages: [{ role: 'user', content: prompt }],
}),
});

const data = await res.json(); console.log(res.status, data);
return data?.choices?.[0]?.message?.content || null;
} catch (error) {
console.error('Error:', error?.message || error); return null;
}

If you save your function and run it, most probably you'll get an error message in LOGS:

2026-02-02T12:14:17.299547912Z stderr: The function exceeded its time limits of 3000 milliseconds. Logs might be missing.

We need to give our function more time to wait for the LLM to answer!

Setting Timeouts

Head to Settings again, and click Edit on Limits. By default you have 3 secs (3000 milliseconds) to run your function before you get a timeout. We'll set that to 10 secs. Save the change, go back to Source and run the function again.

After that change, you should get a summary of the page!

Persisting data into MongoDB

We're now ready to store our body object in a MongoDB collection. One of the nicest things about MongoDB is that you work with your application objects in memory, and when ready, you send them to MongoDB using the driver (a library that will interact directly with the database). No need for ORMs or complicated frameworks in this case.

We'll start by importing the MongoDB driver:

import { MongoClient } from "mongodb";

After that, we'll write a simple function that will:

receive our object as an argument, named page.
connect to MongoDB using a MongoClient instance and the connect method.
write it into MongoDB using insertOne

async function store_in_mongodb(page) {
const mongodb_URI = process.env.MONGODB_URI; // get the MongoDB URI from an environment variable

const client = new MongoClient(mongodb_URI); try {
// connect to MongoDB await client.connect();
const database = client.db('read_it_later'); const collection = database.collection('pages');

const result = await collection.insertOne(page); return result;
} finally {
await client.close();
}
};

As we did previously with our environment variable MISTRAL_API_KEY, we'll need to add a new environment variable in Settings called MONGODB_URI with the URI of our MongoDB installation. The URI looks like this:

mongodb+srv://user:password@cluster.something-here.mongodb.net/

The last step will be to call this new function in main:

await store_in_mongodb(greeting);

Once this is done, save your function and send some links to your function and see how the collection pages gets populated!

Conclusion

In this article we've explored how to write a serverless function that can interact with an LLM and store results in MongoDB. It's incredible how far a few lines of code can take us these days. Not so long ago you needed to spend days setting up web and database servers, hosting everything, exposing it to the Internet, writing your code, etc. Now, using DigitalOcean integrated suit of services and hosted applications you focus on what matters most: your application code. Our next step could be building a new function to read from MongoDB, that will be the base for our UI, or add Semantic Search using MongoDB's Vector Search capabilities.

Here you have the complete code listing for our function:

export async function main(args) { const link = args.link;
if (!link) { return {
statusCode: 400,
headers: { "Content-Type": "application/json" }, body: { error: "Please send a link" }
};
}

const prompt = 'I need a summary of this page: ' + link; const summary = await chatWithLeChat(prompt);
const body = { URL: link , summary: summary
};

await store_in_mongodb(body);

return { statusCode: 200,
headers: { "Content-Type": "application/json" }, body: body
};
}

const API_KEY = process.env.MISTRAL_API_KEY; // secrets are stored in the function's Settings
const API_URL = 'https://api.mistral.ai/v1/chat/completions';

async function chatWithLeChat(prompt) { try {
// call the Mistral endpoint with POST const res = await fetch(API_URL, {
method: 'POST', headers: {
'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json',
},
body: JSON.stringify({ model: 'mistral-tiny',
messages: [{ role: 'user', content: prompt }],
}),
});

const data = await res.json(); console.log(res.status, data);
return data?.choices?.[0]?.message?.content || null;
} catch (error) {
console.error('Error:', error?.message || error); return null;
}
}
import { MongoClient } from "mongodb"; async function store_in_mongodb(page) {
const mongodb_URI = process.env.MONGODB_URI; // get the MongoDB URI from an environment variable

const client = new MongoClient(mongodb_URI); try {
// connect to MongoDB await client.connect();
const database = client.db('read_it_later'); const collection = database.collection('pages');

const result = await collection.insertOne(page); return result;
} finally {
await client.close();
}
};

Build a Movie/TV Show Tracker with MongoDB and TanStack Start

Didicodes — Fri, 17 Jul 2026 05:48:02 +0000

A few months ago, I found myself opening multiple different apps just to answer a simple question: "What episode was I on?"

I'd start a show, get distracted by work, switch to another series for a few weeks, then come back and have no idea where I'd stopped. Sometimes I'd scroll through episode lists trying to remember the last thing I watched. Other times, I'd accidentally rewatch an episode because I couldn't remember whether I'd already seen it. After doing this one too many times, I decided to build a small tracker for myself.

The idea was simple. Search for a show, add it to a watch list, and tap a button whenever you finish an episode. Behind that simple experience, though, there are a few interesting decisions to make. How should progress be stored? Should the show information be fetched from an external API every time, or stored in MongoDB? And how do you update both progress and status without introducing race conditions?

This tutorial explains how to build a movie/TV show tracker using MongoDB, Tanstack Start, and TMDB. You can find the finished project here in this GitHub repository.

What you will learn

How to design a data model for a TV show tracking application
How MongoDB aggregation pipelines can power both progress updates and application statistics
How TanStack Start server functions make it easy to work with MongoDB from a full-stack React application

Prerequisites

Before you start, you should have:

Node.js 18+ installed (npm is included by default)
A MongoDB Atlas account or a local MongoDB server
A free TMDB account, which gives you an API token
An understanding of React, TypeScript, and TanStack Start.

Setting up the project

Let’s set up our project for the application using TanStack Start's start-basic example, then add MongoDB. To do this, run the commands below on your terminal:

npx gitpick TanStack/router/tree/main/examples/react/start-basic tanstack-start-mongodb-tvshow-tracker
cd tanstack-start-mongodb-tvshow-tracker
npm install
npm install mongodb

gitpick copies only the start-basic folder from TanStack’s repo into a newly created directory named tanstack-start-mongodb-tvshow-tracker. npm install pulls down what the starter already uses (Zod is one of them), and npm install mongodb adds the official MongoDB nodejs driver.

At this point, the project has been created, but it still includes the files from the start-basic example. Before we start building the movie/TV show tracker, let's update the file structure to match the project. To make this easier, I've created a small setup script that removes the starter example files, updates the root route, and creates the folders and files we'll use throughout the rest of the tutorial. Run the command below from the root of your tanstack-start-mongodb-tvshow-tracker project:

curl -o setup.sh https://gist.githubusercontent.com/didicodes/e35a275795d3364ebb17e488724ca132/raw/7b403929ab93dd7f64ba2d701a0137277769f808/setup.sh && chmod +x setup.sh && ./setup.sh

After running the script, your project should look like the image below. Run npm run dev to confirm the project runs successfully. If you'd rather make the changes yourself, you can follow along using the project structure shown below:

We'll build out these files as we go, but setting up the structure now makes it easier to see where each piece of the application belongs.

Getting your credentials

The app needs a MongoDB connection string and a TMDB token. For MongoDB, create a free cluster in MongoDB Atlas, then open Connect, choose Drivers, and copy the Node.js connection string. It looks like mongodb+srv://<username>:<password>@<cluster-url>. For TMDB, sign in, open your API settings, request an API key if you don’t already have one, then copy the value labeled "API Read Access Token". Using your preferred IDE, open the tanstack-start-mongodb-tvshow-tracker project and add both credentials to the .env file as shown below

MONGODB_URI=your-mongodb-connection-string
TMDB_READ_ACCESS_TOKEN=your-tmdb-read-access-token

TanStack Start reads .env in development so that both credentials will be available as process.env values in server functions. They are read only on the server, so neither one reaches the browser.

Designing the show document

Before writing any database code, I wanted to decide what a TV show document should look like. As I started sketching out the data model, I caught myself thinking about features the app didn't even need yet. Things like, "Maybe one day I should track individual episodes," or "It would be nice to keep a viewing history," or even "What if I added personal notes?" Eventually, I realized that none of those ideas solved the problem I was trying to fix today. The only question this app really needed to answer was: How far through this show am I?

Once I focused on that question, the data model became much simpler. Before writing any MongoDB code, I wanted to capture that model in TypeScript so the rest of the application could share the same definitions. In this project, those shared types will live in src/lib/types.ts, so go there and add the code below:

// src/lib/types.ts
export interface ShowDocument {
  tmdbId: number;
  title: string;
  posterPath: string | null;
  totalSeasons: number;
  totalEpisodes: number;
  episodesWatched: number;
  status: "plan_to_watch" | "watching" | "completed";
  rating: number | null;
  createdAt: Date;
  updatedAt: Date;
}

Defining the fields was only part of the design. The next question was how to represent progress. My first instinct was to store every episode a user watches. That would make it possible to answer questions like "Which episodes have I seen?" or "What did I watch last week?". After sketching it out, though, storing individual episodes started to feel like overkill. If I'm halfway through a season, I don't really care whether the app knows I watched episode 4 or episode 5. What I care about is seeing that I've watched 12 out of 24 episodes. Once I looked at it that way, a single episodesWatched field felt like the simpler choice.

Progress wasn't the only tradeoff. I also went back and forth on how much information should come from TMDB versus how much should live in MongoDB. One option was to store only the TMDB ID and fetch everything else on page load. I decided against that approach. Every visit to the watch list would trigger another API request, even though details like a TV show's title or poster rarely change. Instead, I store a snapshot of the information I need when a TV show is first added, then read that snapshot directly from MongoDB from that point on. The tradeoff is that some values can become outdated over time. For example, if a TV show is renewed and a new season is released, the stored episode count won't update automatically. For a watch tracker, that's a tradeoff I'm happy to make in exchange for faster reads and fewer external API requests.

Connecting to MongoDB

Now that we know what a show document looks like, we need somewhere to store it. The next step is setting up a MongoDB connection that the rest of the application can reuse. Before opening a connection, I like to pull reusable values into a small config file. For this app, that's the database name, collection name, and a couple of connection settings. Go to src/config/mongodb.ts and add the code below:

// src/config/mongodb.ts
export const DB_NAME = "episode-tracker";
export const COLLECTION_NAME = "shows";

export const MONGODB_CONNECTION_CONFIG = {
  minPoolSize: 1,
  maxIdleTimeMS: 60000,
} as const;

The database and collection names are fairly straightforward. The connection settings are a little more interesting. Since TanStack Start server functions can run in short-lived environments, I don't want to create a new connection for every request. Keeping a connection around for a short period means later requests can reuse it instead of starting from scratch.

In this case, minPoolSize:1 keeps one connection ready to use, while maxIdleTimeMS:60000 allows idle connections to be cleaned up after about a minute. With the configuration in place, we can create a reusable MongoDB client. To do this, add the code below to src/lib/mongodb.ts:

// src/lib/mongodb.ts
import { MongoClient, type Collection, type Db } from "mongodb";
import {
  COLLECTION_NAME,
  DB_NAME,
  MONGODB_CONNECTION_CONFIG,
} from "../config/mongodb";
import type { ShowDocument } from "./types";

const MONGODB_URI = process.env.MONGODB_URI;

interface CachedConnection {
  client: MongoClient | null;
  db: Db | null;
  promise: Promise<{ client: MongoClient; db: Db }> | null;
}

const cached: CachedConnection = {
  client: null,
  db: null,
  promise: null,
};

export async function connectToDatabase(): Promise<{
  client: MongoClient;
  db: Db;
}> {
  if (cached.client && cached.db) {
    return { client: cached.client, db: cached.db };
  }

  if (cached.promise) {
    return cached.promise;
  }

  if (!MONGODB_URI) {
    throw new Error(
      "Missing MONGODB_URI environment variable. Add it to your .env file."
    );
  }

  cached.promise = MongoClient.connect(MONGODB_URI, {
    appName: "devrel-github-tanstack-tvshow-tracker",
    ...MONGODB_CONNECTION_CONFIG,
  })
    .then((client) => {
      const db = client.db(DB_NAME);

      cached.client = client;
      cached.db = db;
      cached.promise = null;

      return { client, db };
    })
    .catch((error) => {
      cached.promise = null;
      throw error;
    });

  return cached.promise;
}

export async function getShowsCollection(): Promise<Collection<ShowDocument>> {
  const { db } = await connectToDatabase();
  const collection = db.collection<ShowDocument>(COLLECTION_NAME);
  await collection.createIndex({ tmdbId: 1 }, { unique: true });
  return collection;
}

The goal of these helper functions is simple: avoid opening more MongoDB connections than necessary. Once a connection exists, we keep using it. If another request comes in while the connection is still being created, it waits for that same promise instead of starting a second connection attempt.

The tmdbId index is also created during this first connection step. That keeps getShowsCollection() focused on one job: returning the typed shows collection. It also means the server functions can call it without asking MongoDB to create the index each time.

The .catch() block clears the cached promise if the connection fails, so the next request can try again instead of reusing a failed connection attempt. From there, the rest of the app gets a typed collection that matches our ShowDocument shape.

Defining the application's data contract

Before moving on to the server functions, I wanted a single place to define the data shapes used throughout the application. By this point, we already know what a show document looks like in MongoDB. We also know that data will be flowing between the browser, our server functions, MongoDB, and TMDB. Rather than redefining those shapes in multiple places, we'll keep them together in src/lib/types.ts. This file will contain the application's statuses, Zod validation schemas, TypeScript interfaces, and a small helper for converting MongoDB documents into a browser-friendly format. Replace the code in the file with this:

// src/lib/types.ts
import { z } from "zod";
import type { ObjectId } from "mongodb";

export const WATCH_STATUSES = [
  "plan_to_watch",
  "watching",
  "completed",
] as const;

export type WatchStatus = (typeof WATCH_STATUSES)[number];

export const addShowSchema = z.object({
  tmdbId: z.number().int().positive(),
  status: z.enum(WATCH_STATUSES).default("plan_to_watch"),
});

export const updateShowSchema = z.object({
  id: z.string().min(1, "Show id is required"),
  status: z.enum(WATCH_STATUSES).optional(),
  rating: z.number().int().min(1).max(10).nullable().optional(),
  episodesWatched: z.number().int().min(0).optional(),
});

export const showIdSchema = z.object({
  id: z.string().min(1, "Show id is required"),
});

export const searchSchema = z.object({
  query: z.string().min(1, "Search query is required").max(120),
});

// How a show is stored in MongoDB
export interface ShowDocument {
  tmdbId: number;
  title: string;
  posterPath: string | null;
  totalSeasons: number;
  totalEpisodes: number;
  episodesWatched: number;
  status: WatchStatus;
  rating: number | null;
  createdAt: Date;
  updatedAt: Date;
}

// A stored show as it comes back from a query, including Mongo's _id
export interface ShowResponse extends ShowDocument {
  _id: ObjectId;
}

// How a show is sent to the browser. ObjectId and Date are not JSON-friendly, so this converts them to strings
export interface Show {
  id: string;
  tmdbId: number;
  title: string;
  posterPath: string | null;
  totalSeasons: number;
  totalEpisodes: number;
  episodesWatched: number;
  status: WatchStatus;
  rating: number | null;
  createdAt: string;
  updatedAt: string;
}

// Summary numbers for the stats bar
export interface LibraryStats {
  totalShows: number;
  episodesWatched: number;
  watching: number;
  completed: number;
  planToWatch: number;
}

export interface TmdbSearchResult {
  tmdbId: number;
  title: string;
  posterPath: string | null;
  year: string | null;
  overview: string;
}

export interface TmdbShowDetails {
  tmdbId: number;
  title: string;
  posterPath: string | null;
  totalSeasons: number;
  totalEpisodes: number;
}

export function documentToShow(doc: ShowResponse): Show {
  return {
    id: doc._id.toString(),
    tmdbId: doc.tmdbId,
    title: doc.title,
    posterPath: doc.posterPath,
    totalSeasons: doc.totalSeasons,
    totalEpisodes: doc.totalEpisodes,
    episodesWatched: doc.episodesWatched,
    status: doc.status,
    rating: doc.rating,
    createdAt: doc.createdAt.toISOString(),
    updatedAt: doc.updatedAt.toISOString(),
  };
}

Two separate concerns are being handled in this file:

The first is validation. The Zod schemas make sure the server only accepts the inputs we expect. For example, when a show is added, the client sends a TMDB ID and an optional status. Everything else comes directly from TMDB, so there's no reason to trust those values from the browser.
The second is keeping the different data shapes organized. A show exists in several forms as it moves through the application. Before a document is stored, there is no _id, so we use ShowDocument. Once MongoDB returns that document from a query, it now includes an _id, which is represented by ShowResponse. Finally, when the data is sent to the browser, the ObjectId and date fields need to become strings, which is represented by Show. Rather than repeating those conversions throughout the codebase, the documentToShow() helper handles them in one place.

Fetching show details from TMDB

When someone adds a show to their watch list, they aren't entering the title, poster URL, season count, or episode count manually. The only information we really need from them is the TMDB ID; everything else can come from TMDB. To keep that logic in one place, we'll create a TMDB client in src/lib/tmdb.ts that handles authentication, show search, and fetching the additional details we need before saving a document to MongoDB. Add the following code to src/lib/tmdb.ts:

// src/lib/tmdb.ts
import type { TmdbSearchResult, TmdbShowDetails } from "./types";

const TMDB_BASE_URL = "https://api.themoviedb.org/3";
const TMDB_TOKEN = process.env.TMDB_READ_ACCESS_TOKEN;

// Small wrapper around the TMDB REST API
async function tmdbFetch<T>(path: string): Promise<T> {
  if (!TMDB_TOKEN) {
    throw new Error(
      "Missing TMDB_READ_ACCESS_TOKEN environment variable. Add it to your .env file."
    );
  }

  const res = await fetch(`${TMDB_BASE_URL}${path}`, {
    headers: {
      Authorization: `Bearer ${TMDB_TOKEN}`,
      accept: "application/json",
    },
  });

  if (!res.ok) {
    throw new Error(`TMDB request failed with status ${res.status}`);
  }

  return res.json() as Promise<T>;
}

interface TmdbSearchResponse {
  results: Array<{
    id: number;
    name: string;
    poster_path: string | null;
    first_air_date?: string;
    overview?: string;
  }>;
}

interface TmdbDetailsResponse {
  id: number;
  name: string;
  poster_path: string | null;
  number_of_seasons: number;
  number_of_episodes: number;
}

// Search TV shows by title. Returns a trimmed list shaped for the UI
export async function searchShows(query: string): Promise<TmdbSearchResult[]> {
  const data = await tmdbFetch<TmdbSearchResponse>(
    `/search/tv?query=${encodeURIComponent(query)}&include_adult=false&page=1`
  );

  return (data.results ?? []).slice(0, 8).map((show) => ({
    tmdbId: show.id,
    title: show.name,
    posterPath: show.poster_path,
    year: show.first_air_date ? show.first_air_date.slice(0, 4) : null,
    overview: show.overview ?? "",
  }));
}

//Fetch the season and episode counts for a single show
export async function getShowDetails(tmdbId: number): Promise<TmdbShowDetails> {
  const data = await tmdbFetch<TmdbDetailsResponse>(`/tv/${tmdbId}`);

  return {
    tmdbId: data.id,
    title: data.name,
    posterPath: data.poster_path,
    totalSeasons: data.number_of_seasons ?? 0,
    totalEpisodes: data.number_of_episodes ?? 0,
  };
}

While building this, I ran into one small limitation of the TMDB API. The search endpoint is great for helping users find a TV show, but it doesn't include the season and episode totals our tracker needs. That's why there are two functions in this file: searchShows(), which returns the results displayed in the search UI, and getShowDetails(), which runs when a show is added to the library. That second request fetches the complete show details, including the season and episode counts we store in MongoDB. You'll also notice that the TMDB token never leaves the server, as it comes from an environment variable accessible only to the server-side code. The base URL for the poster image is different. Since it's just a constant rather than a secret, we can safely keep it in src/lib/constants.ts alongside the human-readable status labels:

//src/lib/constants.ts
import type { WatchStatus } from "./types";

// TMDB serves poster images from a few fixed sizes. w342 looks crisp on cards.
export const TMDB_POSTER_BASE = "https://image.tmdb.org/t/p/w342";

export const STATUS_LABELS: Record<WatchStatus, string> = {
  plan_to_watch: "Plan to watch",
  watching: "Watching",
  completed: "Completed",
};

At this point, we can search for TV shows and fetch all the information needed to save them. The next step is wiring these functions into our server so the application can start reading from and writing to MongoDB.

Building the server functions

Now that we can read data from MongoDB and fetch metadata from TMDB, we can start wiring the application together. TanStack Start uses createServerFn to define server-side functions. Each function validates its input with Zod before executing the handler, so by the time our code runs, we already know the data is valid and correctly typed. We'll start with the TMDB search function. Add the following code to src/server/tmdb.ts:

// src/server/tmdb.ts
import { createServerFn } from "@tanstack/react-start";
import { searchSchema, type TmdbSearchResult } from "~/lib/types";
import { searchShows } from "~/lib/tmdb";

export const searchTmdb = createServerFn({ method: "GET" })
  .validator(searchSchema)
  .handler(async ({ data }): Promise<TmdbSearchResult[]> => {
    try {
      return await searchShows(data.query);
    } catch (error) {
      console.error("TMDB search failed:", error);
      throw new Error("Failed to search TMDB");
    }
  });

Most of the application's behavior lives in shows.ts. Rather than pasting several hundred lines of code into this article, I've included the complete code in this repo at src/server/shows.ts, so add it to your project. Even though we aren't walking through every line, there are three functions worth understanding because they capture most of the application's behavior:

addShow checks whether a show with the same tmdbId already exists. If it doesn't, it fetches the show's details from TMDB, creates the MongoDB document, and returns the newly created show.
incrementProgress is where I ended up moving more logic into MongoDB. My first version updated the episode count in JavaScript after reading the document. Eventually, I realized the database already had everything it needed. The aggregation update increments the episode count, prevents it from going past the total number of episodes, and automatically updates the show's status when necessary.
getStats is where storing a dedicated status field starts paying off. Instead of loading every show into memory and counting them in JavaScript, MongoDB calculates the dashboard statistics with a single aggregation pipeline and returns only the numbers the UI needs.

With these server functions in place, the backend can search, store, update, and summarize data. The only missing piece is the user interface.

Building the user interface

Up to this point, everything we've built has lived on the server. The final step is putting a user interface on top of it. I kept the front end deliberately simple. The application lives in a single route that loads the watch list and statistics, renders them on the page, and refreshes itself whenever data changes. Instead of manually updating local state after every action, I simply invalidate the route so the loader fetches the latest data from MongoDB.

The route shown below is a shortened version of the final implementation that shows the overall pattern. The complete version is available in the GitHub repository at src/routes/index.tsx.

// src/routes/index.tsx
export const Route = createFileRoute("/")({
  loader: async () => {
    const [shows, stats] = await Promise.all([getShows(), getStats()]);
    return { shows, stats };
  },
  component: Home,
});

function Home() {
  const { shows, stats } = Route.useLoaderData();
  const router = useRouter();
  const [busyId, setBusyId] = useState<string | null>(null);

  // Re-run the loader so that the shows and stats reflect the latest database state
  const refresh = () => router.invalidate();

  async function handleIncrement(id: string) {
    setBusyId(id);
    try {
      await incrementProgress({ data: { id } });
      await refresh();
    } finally {
      setBusyId(null);
    }
  }

  // ...search, add, status, rating, and delete handlers follow the same shape
  return <main className="page">{/* stats bar, search, and the show list */}</main>;
}

I also moved the stylesheet to the repository (src/styles/app.css) to keep this article focused on the application architecture rather than on visual styling. Once those files are in place, the application is ready to run. Yay!

One lesson from building this project was that I didn't need much client-side state at all. Since the loader already fetches the latest data, I could treat the database as the source of truth and reload the route after each action. That approach kept the component small and avoided much of the bookkeeping in the browser.

Try it locally

Let’s test it out. To do this, run the npm run dev command on your terminal, then go to http://localhost:3000/, search for a show, and add it to your library. Try incrementing the episode count a few times and watch the progress update. The watch list, statistics, and progress tracking should all stay in sync as the route reloads after each change. At this point, the application is complete.

As you can see, on the app, you can now search for TV shows, add them to the library, track viewing progress, update ratings and status, and calculate dashboard statistics, all using TanStack Start server functions backed by MongoDB.

Final thoughts

This project started because I wanted a simple answer to a simple question: "How far through this show am I?"

Along the way, we built a small full-stack application using TanStack Start, MongoDB, TMDB, Zod, and server functions. The final result is intentionally simple. Search for a show, add it to your library, track your progress, and see a summary of your watching habits.

There are plenty of directions you could take this project next. User accounts, episode history, recommendations, and shared watch lists would all fit naturally on top of what we've built here. I stopped here because the app already does exactly what I wanted: it tells me where I am in a TV show without any extra complexity. If you'd like to keep building, the full source code is available in this repository.

Key Takeaways

Building a small TMDB client gave the application a single place to handle API requests, making future changes easier without affecting the rest of the codebase.
TanStack Start server functions, together with MongoDB, provide a clear separation between the user interface and the application's data and business logic.
MongoDB aggregation pipelines and atomic update operations simplify application code by moving data processing closer to where the data is stored.

Building a Vibe-Based Music Recommender with MongoDB and Voyage AI

Jordan Smith — Tue, 14 Jul 2026 14:08:37 +0000

Have you ever found yourself in the mood for a certain type of music but not any specific artists, songs, or genres in mind? Maybe you just had a long day and are looking for something with mellow and relaxing vibes. Or maybe it's a Friday afternoon, and you're ready for something to pump you up for the weekend. In times like these, traditional search experiences won't work. But today, we're going to build something that will.

In a traditional search experience, you would search for a term and get back a list of songs or artists that contain keywords from that search term somewhere in their name or description. But that's not what we want here. What we actually want is semantic search: a system that understands what you mean, not just what you typed. That's exactly what we're building. This music recommender takes a natural-language description of the music's “vibe” and finds songs that match that feel. And it's all powered by MongoDB Vector Search and Voyage AI.

In this tutorial, you’ll learn how to build a working web application where you can type something like "upbeat road trip with the windows down" and get back a ranked list of songs that fit that feeling. Let's get into it!

How It Actually Works

Before we write any code, let's go through a quick overview of how the search functionality in this app actually works.

The key to all of this is vector embeddings. A vector embedding is a way of representing text (or other data like images or audio) as an array of numbers, or vectors. Data with similar semantic meanings produces similar vectors, so you can compare all of your vectors mathematically to find data that's similar in meaning.

Here's how it actually works in this app:

Every song in our sample collection has an LLM-generated "vibe description". This is a 2-3 sentence description of what the song "feels" like to listen to. Vector embeddings were generated from these descriptions using Voyage AI and are stored with the songs.
When a user types a search query, it is converted into a vector using the same Voyage AI vector embedding model.
We use MongoDB Vector Search to find the songs in the collection whose vectors are closest to the query vector.
The results are returned to the user, ranked by how closely their vector matches the query vector.

Full Project Code

If you want to see the full project with all of the code already written, you can find it on GitHub. I'll explain all the elements throughout this tutorial, and you can clone the repository and follow along as well.

Prerequisites

Before getting started, you'll need a few things in place:

Node.js (v18 or later) installed on your machine
A MongoDB Atlas account with a free cluster
A Voyage AI API key - you can get one from the MongoDB Atlas UI

Project Setup

With that out of the way, let’s get going with the actual code.

Start by creating a new project directory and initializing it:

mkdir music-recommender
cd music-recommender
npm init -y

Then install the dependencies:

npm install express mongodb voyageai dotenv

Create a .env file in the root of your project with the following variables:

MONGODB_URI=<Your MongoDB Connection String>
MONGODB_DATABASE=song_vibe_search

VOYAGE_API_KEY=<Your Voyage AI API Key>

Next, you'll need to download the seed files. These files contain scripts that will initialize your database by using sample data. The scripts also contain code that you can use to generate your own descriptions and embeddings, but we won't get into that until the Deeper Dive: Generating Embeddings section at the end of the tutorial.

The sample data is a small sample of 500 songs from the Free Music Archive with pre-generated "vibe descriptions" and embeddings. You can download the seed files from the /seed directory in the GitHub repository for this tutorial.

After you have downloaded the seed folder, place it at the root of your project. Then run the following command to connect to your, insert the seed data, and create a vector search index:

npm run seed

This creates a new database called song_vibe_search and a new collection called songs. The song collection contains 500 documents, each representing a song with a "vibe description" and an embedding vector. It also creates a vector search index on the embedding field by running the following code:

await collection.createSearchIndex({
    name: VECTOR_INDEX_NAME,
    type: "vectorSearch",
    definition: {
    fields: [
        {
        type: "vector",
        path: "embedding",
        numDimensions: 512,
        similarity: "cosine",
        },
    ],
    },
});

This is what makes $vectorSearch possible. We need to tell MongoDB the shape of our embedding field so it can build the right index structure.

The index contains the type of index (in our case, vector), the path to the field, and the following parameters that tell MongoDB how to compare vectors:

numDimensions: 512 — This must match the output size of your embedding model. Voyage AI's voyage-3-lite\ model produces 512-dimensional vectors. If you're using a different model, you should be able to find the number of dimensions in the documentation for your model.
similarity: "cosine" — This is the distance metric used to compare vectors. Cosine similarity works well for text embeddings because it measures the angle between vectors rather than their raw distance, which makes it robust to differences in text length.

Now that we have our data and our vector search index, we are ready to build the main application.

Implementing Vector Search

In the root of your project, create a new directory called src and create a new file called server.ts in that directory. This is where we will implement the search logic that powers our recommendations.

First, we need to set our imports, our interfaces and constants, and establish our connection to MongoDB. Paste the following code into server.ts:

import "dotenv/config";
import { MongoClient } from "mongodb";
import { VoyageAIClient } from "voyageai";

const MONGODB_URI = process.env.MONGODB_URI;
const VOYAGE_API_KEY = process.env.VOYAGE_API_KEY;

if (!MONGODB_URI) {
  throw new Error(
    "Missing MONGODB_URI in environment. Ensure that you have created a .env file and added your connection string."
  );
}

if (!VOYAGE_API_KEY) {
  throw new Error(
    "Missing VOYAGE_API_KEY in environment. Ensure that you have created a .env file and added your Voyage AI key."
  );
}

export const client = new MongoClient(MONGODB_URI, {
  appName: "devrel-tutorial-javascript-music-recommender",
});

const voyage = new VoyageAIClient({ apiKey: VOYAGE_API_KEY });

const DB_NAME = process.env.MONGODB_DATABASE ?? "song_vibe_search";
const COLLECTION_NAME = "songs";
export const VECTOR_INDEX = "vibe_vector_index";
const EMBEDDING_MODEL = "voyage-3-lite";
export const NUM_RESULTS = 5;
export const NUM_CANDIDATES = 100;

export interface SearchResult {
  title: string;
  artist: string;
  genre_top: string;
  genres: string[];
  score: number;
}

export interface SearchResponse {
  results: SearchResult[];
  queryVector: number[];
}

If you're unfamiliar with vector search, some of the constants above may not make sense quite yet, but we'll explain them more as they're used.

Embedding the query

When a user submits a search, the first thing we need to do is embed that query by using the same Voyage AI model that was used to embed our song descriptions. Paste the following code into server.ts to create a search function and embed the query. Note the comment that shows where we will add the vector search query code in a later step.

export async function searchSongs(query: string): Promise<SearchResponse> {

  // Embed the user's query by using Voyage AI.
  const embedResponse = await voyage.embed({
    model: EMBEDDING_MODEL,
    input: [query],
    inputType: "query",
  });

  const queryVector = embedResponse.data![0].embedding!;

  // vector search query code here
}

This code specifies a few things. First, we specify the embedding model to use (in our case, we pass in the constant that we set earlier). We then pass the actual user query as an array to the input parameter. Finally, we specify the inputType as "query". Voyage AI distinguishes between the following two types of input:

"document" — used when embedding data that gets saved into your database (the song vibe descriptions, in our case)
"query" — used when embedding a search query at runtime

Using the right input type improves retrieval quality. It's a small but important thing to keep in mind when implementing vector search applications.

Now that we have our query embedded, it's time to use it to search for similar songs. Paste the following code to complete our search function. Ensure that this code is pasted inside the function that we defined in the previous code block.

  // ... previous embedding code here

  // vector search query code here 
  const collection = client.db(DB_NAME).collection(COLLECTION_NAME);

  const results = await collection
    .aggregate<SearchResult>([
      {
        $vectorSearch: {
          index: VECTOR_INDEX,
          path: "embedding",
          queryVector,
          numCandidates: NUM_CANDIDATES,
          limit: NUM_RESULTS,
        },
      },
      {
        $project: {
          _id: 0,
          title: 1,
          artist: 1,
          genre_top: 1,
          genres: 1,
          score: { $meta: "vectorSearchScore" },
        },
      },
    ])
    .toArray();

  return { results, queryVector };

To implement vector search, we use the $vectorSearch stage of the aggregation pipeline. The $vectorSearch stage takes the following parameters:

index — The name of the vector search index we created during seeding
path — The field in the document that holds the embedding (in this case, named "embedding").
queryVector — The vector we just generated from the user's query.
numCandidates — How many candidate documents to consider before ranking. A higher number means the search will be more thorough but slower.
limit — How many results to return.

After the search stage, a $project stage shapes our output, with { $meta: "vectorSearchScore" } pulling in the similarity score for each result — a number between 0 and 1, where higher means more similar.

And that's all we need for our search functionality! Next, we'll build out a simple server with an endpoint to call this function.

Building the Server

Create a new file called server.ts in the src directory. This is where we will create any endpoints we might need. In this tutorial, we will create a single endpoint to search for songs.

To start, paste the following code into server.ts, which specifies the imports and sets up an Express server:

import "dotenv/config";
import express from "express";
import path from "path";
import { client, searchSongs, VECTOR_INDEX, NUM_RESULTS, NUM_CANDIDATES } from "./search";

const app = express();
const PORT = process.env.PORT ?? 3000;

app.use(express.json());
app.use(express.static(path.join(__dirname, "../public")));

Now, we'll create our POST route that calls the searchSongs function we created earlier. Paste the following code into server.ts:

app.post("/api/search", async (req, res) => {
  const { query } = req.body;

  if (!query || typeof query !== "string" || query.trim() === "") {
    res.status(400).json({ error: "A search query is required." });
    return;
  }

  let results, queryVector;
  try {
    ({ results, queryVector } = await searchSongs(query.trim()));
  } catch (err) {
    console.error("Search failed:", err);
    res.status(500).json({ error: "Search failed. Please try again." });
    return;
  }

  res.json({
    results
  });
});

In the preceding code, it first checks that the query is valid. Then it sends the query to our searchSongs() function and returns the results. If there is an error, it returns an error message.

The last thing we need to do is write the code that actually starts the server and handles signals to close the client cleanly on shutdown. Paste the following code into server.ts:

async function startServer() {
  await client.connect();
  console.log("Connected to MongoDB Atlas.");

  app.listen(PORT, () => {
    console.log(`Server running at http://localhost:${PORT}`);
  });

  process.on("SIGINT", async () => {
    await client.close();
    process.exit(0);
  });

  process.on("SIGTERM", async () => {
    await client.close();
    process.exit(0);
  });
}

startServer();

Now that we have our server set up, let's create a simple front end to interact with it.

Building the Front End

Since the purpose of this tutorial is the search functionality itself, we'll keep the front end simple. Create a new directory called public in the root of your project (it's important to keep the name as public since our server is already set up to serve files from this directory). Then, create a new file called index.html in that folder.

Paste the following code into index.html:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Vibe Search</title>
    <link rel="stylesheet" href="style.css" />
  </head>
  <body>
    <div class="container">
      <header>
        <h1>Vibe Search</h1>
        <p>Describe a mood or feeling and we'll find songs that match.</p>
      </header>

      <form id="search-form">
        <input
          id="query-input"
          type="text"
          placeholder="e.g. melancholy rainy afternoon, upbeat road trip..."
          autocomplete="off"
        />
        <button type="submit">Search</button>
      </form>
      <p class="token-note">Note: Each search generates a Voyage AI embedding and consumes tokens.</p>

      <div id="results"></div>
    </div>

    <script src="app.js"></script>
  </body>
</html>

This HTML creates a simple page with a header, an input box, and a section for results.

Now we need to create the style.css and app.js files. Create a new file called style.css in the public directory and paste the following code into it:

*, *::before, *::after {
  box-sizing: border-box;
  margin: 0;
  padding: 0;
}

body {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
  background: #0f0f0f;
  color: #e8e8e8;
  min-height: 100vh;
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  padding: 2rem 1rem;
}

.container {
  width: 100%;
  max-width: 720px;
}

header {
  text-align: center;
  margin-bottom: 2.5rem;
}

header h1 {
  font-size: 2.5rem;
  font-weight: 700;
  letter-spacing: -0.5px;
  margin-bottom: 0.5rem;
}

header p {
  color: #777;
  font-size: 1rem;
}

/* ── Search form ── */

#search-form {
  display: flex;
  gap: 0.5rem;
  margin-bottom: 2.5rem;
}

#query-input {
  flex: 1;
  padding: 0.75rem 1rem;
  font-size: 1rem;
  background: #1a1a1a;
  border: 1px solid #333;
  border-radius: 8px;
  color: #e8e8e8;
  outline: none;
}

#query-input:focus {
  border-color: #555;
}

#query-input::placeholder {
  color: #555;
}

#search-form button {
  padding: 0.75rem 1.5rem;
  font-size: 1rem;
  font-weight: 600;
  background: #00ed64;
  color: #000;
  border: none;
  border-radius: 8px;
  cursor: pointer;
}

#search-form button:hover {
  background: #00c853;
}

/* ── Token note ── */

.token-note {
  font-size: 1rem;
  color: #989898;
  text-align: center;
  margin-top: 0.6rem;
  margin-bottom: 0;
}

/* ── Spinner ── */

.spinner {
  width: 28px;
  height: 28px;
  border: 3px solid #2a2a2a;
  border-top-color: #00ed64;
  border-radius: 50%;
  margin: 2rem auto;
  animation: spin 0.7s linear infinite;
}

@keyframes spin {
  to { transform: rotate(360deg); }
}

/* ── Result cards ── */

#results {
  display: flex;
  flex-direction: column;
  gap: 1rem;
}

.card {
  background: #1a1a1a;
  border: 1px solid #2a2a2a;
  border-radius: 12px;
  overflow: hidden;
}

.card-color-bar {
  height: 6px;
}

.card-body {
  padding: 1.25rem 1.5rem;
}

.card-title {
  font-size: 1.1rem;
  font-weight: 600;
  margin-bottom: 0.2rem;
}

.card-artist {
  color: #777;
  font-size: 0.9rem;
  margin-bottom: 0.85rem;
}

.tags {
  display: flex;
  flex-wrap: wrap;
  gap: 0.35rem;
  margin-bottom: 1rem;
}

.tag {
  font-size: 0.75rem;
  padding: 0.2rem 0.65rem;
  border-radius: 999px;
  background: #252525;
  color: #999;
}

.card-footer {
  display: flex;
  justify-content: space-between;
  align-items: center;
}

.score {
  font-size: 0.8rem;
  color: #555;
}

.fma-link {
  font-size: 0.8rem;
  color: #00ed64;
  text-decoration: none;
}

.fma-link:hover {
  text-decoration: underline;
}

.message {
  text-align: center;
  color: #555;
  padding: 2rem 0;
  font-size: 0.95rem;
}

.message.error {
  color: #e05252;
}

The CSS added here is outside of the scope of this tutorial, but it just makes the app easier to look at and use.

The last file we need to create is app.js, which contains the front-end JavaScript code. Create a new file called app.js in the public directory and paste the following code into it:

const DEFAULT_GRADIENT = "linear-gradient(135deg, #434343, #1a1a1a)";

function renderResults(results) {
  const container = document.getElementById("results");

  if (results.length === 0) {
    container.innerHTML = '<p class="message">No results found. Try a different vibe.</p>';
    return;
  }

  container.innerHTML = results.map((result) => {
    const score = Math.round(result.score * 100);
    const tags = result.genres.map((g) => `<span class="tag">${g}</span>`).join("");

    return `
      <div class="card">
        <div class="card-color-bar" style="background: ${DEFAULT_GRADIENT}"></div>
        <div class="card-body">
          <div class="card-title">${result.title}</div>
          <div class="card-artist">${result.artist}</div>
          <div class="tags">${tags}</div>
          <div class="card-footer">
            <span class="score">Match: ${score}%</span>
            <a class="fma-link" href="https://www.youtube.com/results?search_query=${encodeURIComponent(result.title + ' ' + result.artist)}" target="_blank" rel="noopener">Find on YouTube →</a>
          </div>
        </div>
      </div>
    `;
  }).join("");
}

document.getElementById("search-form").addEventListener("submit", async (e) => {
  e.preventDefault();

  const query = document.getElementById("query-input").value.trim();
  if (!query) return;

  const container = document.getElementById("results");
  container.innerHTML = '<div class="spinner"></div>';

  const response = await fetch("/api/search", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ query }),
  });

  if (!response.ok) {
    container.innerHTML = '<p class="message error">Something went wrong. Please try again.</p>';
    return;
  }

  const { results } = await response.json();
  renderResults(results);
});

This JavaScript code adds interactivity to the page. It sends the search query to the server and displays the search results.

Running the App

At this point, you should have everything set up. Make sure you have already run the npm run seed\ command from earlier in the tutorial.

Then, run the following command to start the server:

npm run dev

Open http://localhost:3000 and try a few searches, such as:

- "melancholy rainy afternoon"

- "high energy workout, aggressive beats"

- "peaceful background music for focusing"

You should see the results ranked by similarity score, with a link to YouTube so you can listen to the songs (if they're uploaded there). In a real application, you would want to serve the audio files or link directly to a streaming service, but for the sample data we're using here, searching YouTube is the only free option without license limitations. If you want to expand on this with different music, check out APIs for streaming services like Spotify.

Next Steps

And that's the basics! You've now built a working music recommender that uses MongoDB Vector Search and Voyage AI to find songs based on their "vibe". There are lots of ways you can expand on this as well. You could modify this to use music from different sources, expand the search functionality to include other metadata or filtering options, or even implement user profiles so that you can create personalized playlists and recommendations based on their likes, dislikes, and listening habits.

In the last section of this tutorial, we'll go through how the data for this tutorial was embedded. This can be useful if you're interested in adapting this tutorial to use your own data, or if you just want to learn more about working with vector embeddings.

Deeper Dive: Generating Embeddings {#deeper-dive:-generating-embeddings}

If you look at the GitHub repo, you'll notice that there are a few files in the seed directory that we didn't use in this tutorial. The fetchTracks.ts file contains code for downloading metadata about the songs that were downloaded from the Free Music Archive. This file will be specific to the Free Music Archive data, so we won't go into depth on it here.

The generateVibeDescriptions.ts and generateEmbeddings.ts files contain the code used to generate the "vibe descriptions" and embeddings for the sample data. The generateVibeDescriptions.ts file uses the Gemini API to generate a text description of the "vibe" of each song. The generateEmbeddings.ts file uses the Voyage AI API to generate an embedding for each description.

Generating Vibe Descriptions

Before generating embeddings from our data, we need some kind of description to embed. Raw metadata like title, artist, and genre isn't rich enough to embed meaningfully on its own, so we use Google Gemini to generate a 2-3 sentence vibe description for each track. Then, we save these descriptions in a new field in our documents called vibe_description.

To run the code to generate the descriptions, you'll need to get a Gemini API key. You can do this by going to the AI Studio and creating a new project. Then, add your API key to the .env file:

GEMINI_API_KEY=<Your Gemini API Key>

Then, create a file in the seed directory called generateVibeDescriptions.ts and paste the following code into it:

import "dotenv/config";
import { GoogleGenAI } from "@google/genai";
import type { RawTrack } from "./fetchTracks";

const GEMINI_API_KEY = process.env.GEMINI_API_KEY;
const GEMINI_MODEL = "gemini-3-flash-preview";
const BATCH_SIZE = 50;

export interface TrackWithVibe extends RawTrack {
  vibe_description: string;
}

function buildPrompt(tracks: RawTrack[]): string {
  const trackList = tracks
    .map(
      (t, i) =>
        `${i + 1}. Title: "${t.title}" | Artist: "${t.artist}" | Album: "${t.album}" | Genres: ${t.genres.join(", ")}`
    )
    .join("\n");

  return `For each track listed below, write a 2-3 sentence vibe description in third person. Focus on mood, energy, atmosphere, and ideal listening context. Avoid simply restating the genre — describe how it feels to listen to it. Return a JSON array of strings, one description per track, in the same order as the input. Return only the JSON array with no additional text. Tracks: ${trackList}`;
}

function parseResponse(text: string): string[] {
  // Extract the JSON array by finding the outermost [ ... ] bounds.
  // This handles cases where the model adds explanatory text before or after the JSON.
  const start = text.indexOf("[");
  const end = text.lastIndexOf("]");
  if (start === -1 || end === -1 || end < start) {
    throw new Error(`No JSON array found in model response:\n${text}`);
  }
  return JSON.parse(text.slice(start, end + 1));
}

export async function generateVibeDescriptions(
  tracks: RawTrack[]
): Promise<TrackWithVibe[]> {
  if (!GEMINI_API_KEY) {
    throw new Error(
      "GEMINI_API_KEY is not set. Add it to your .env file.\n" +
      "Get a free key at: https://aistudio.google.com/apikey"
    );
  }

  const ai = new GoogleGenAI({ apiKey: GEMINI_API_KEY });
  const results: TrackWithVibe[] = [];
  const totalBatches = Math.ceil(tracks.length / BATCH_SIZE);

  console.log(`\nGenerating vibe descriptions via Gemini (${totalBatches} batches)...`);

  for (let i = 0; i < tracks.length; i += BATCH_SIZE) {
    const batch = tracks.slice(i, i + BATCH_SIZE);
    const batchNum = Math.floor(i / BATCH_SIZE) + 1;

    console.log(`  Batch ${batchNum}/${totalBatches}`);

    const response = await ai.models.generateContent({
      model: GEMINI_MODEL,
      contents: buildPrompt(batch),
    });

    const descriptions = parseResponse(response.text ?? "");

    if (descriptions.length !== batch.length) {
      throw new Error(
        `Batch ${batchNum}: expected ${batch.length} descriptions, got ${descriptions.length}`
      );
    }

    for (let j = 0; j < batch.length; j++) {
      results.push({ ...batch[j], vibe_description: descriptions[j] });
    }

  }

  console.log(`  ${results.length} vibe descriptions generated.`);
  return results;
}

In this code, the first function creates the prompt that we send to Gemini. The prompt is a list of all the tracks we want to generate descriptions for, along with instructions for how we want the descriptions formatted.

Then, the generateVibeDescriptions function takes our tracks and sends them to Gemini along with that prompt. Gemini will respond with a JSON array of strings, each of which is a vibe description for a track. We parse the response (and strip any extraneous text that Gemini might have added) and return the tracks with their new descriptions.

Generating Embeddings

Now that we have our vibe descriptions, we can generate embeddings for them. To do this, we use the same Voyage AI model that we used to embed our search queries earlier in the tutorial. This process is very similar to the process that we use to embed our search queries.

Create a file in the seed directory called generateEmbeddings.ts and paste the following code into it:

import "dotenv/config";
import { VoyageAIClient } from "voyageai";
import type { TrackWithVibe } from "./generateVibeDescriptions";

const EMBEDDING_MODEL = "voyage-3-lite";

export interface TrackWithEmbedding extends TrackWithVibe {
  embedding: number[];
}

export async function generateEmbeddings(
  tracks: TrackWithVibe[]
): Promise<TrackWithEmbedding[]> {
  const VOYAGE_API_KEY = process.env.VOYAGE_API_KEY;
  if (!VOYAGE_API_KEY) {
    throw new Error("Missing VOYAGE_API_KEY in environment");
  }

  const voyage = new VoyageAIClient({ apiKey: VOYAGE_API_KEY });

  console.log(`\nGenerating embeddings for ${tracks.length} tracks...`);

  const response = await voyage.embed({
    model: EMBEDDING_MODEL,
    input: tracks.map((t) => t.vibe_description),
    inputType: "document",
  });

  const results: TrackWithEmbedding[] = [];

  for (let i = 0; i < tracks.length; i++) {
    const embeddingData = response.data?.[i];
    if (!embeddingData?.embedding) {
      console.warn(`  Warning: no embedding returned for "${tracks[i].title}" — skipping`);
      continue;
    }
    results.push({ ...tracks[i], embedding: embeddingData.embedding });
  }

  console.log(`  ${results.length} embeddings generated.`);
  return results;
}

The main difference in embedding this data is that we specify the input type as "document" rather than "query". This distinction is important when embedding data with Voyage AI, as it embeds the data slightly differently depending on the type you specify.

Putting It All Together

Now that we have the code to generate descriptions and embeddings, we need to run it when seeding our data. To do so, we just need to modify the index.ts file in the seed directory and call the functions before inserting the data into the database. It will look something like this:

const tracksWithVibes = await generateVibeDescriptions(rawTracks);

const tracksWithEmbeddings = await generateEmbeddings(tracksWithVibes);

const documents = tracksWithEmbeddings.map((track) => ({
...track,
seeded_at: new Date(),
}));

console.log(`\nInserting ${documents.length} documents into MongoDB...`);
await collection.insertMany(documents);

Note that you will likely have to modify the code you created earlier so that the flow of the seed script doesn't insert multiple times (since we originally set it up to use pre-built data).

Now, run the seed command again, and you should see the script generate descriptions and embeddings for the data before inserting it into the database. Then you can run the app again to try searching through the data.

Conclusion

You have now built a fully functional application that uses MongoDB Vector Search and Voyage AI to recommend music based on their general “vibes.” While this app is pretty simple and provides only basic functionality, you can easily use it as a baseline to build much more sophisticated applications.

Hopefully, this gives you some insight into just how powerful MongoDB Vector Search can be. If you want to learn more, check out our documentation or leave a comment letting me know how you’d like to see this expanded on!

No messages table! The data model behind my own Claude-based chatbot

MongoDB Guests — Thu, 02 Jul 2026 15:47:31 +0000

This tutorial was written by Néstor Daza.

This is the second article in a series about building Claudius, my own Claude-based chatbot (Github). The prologue made the case for building it, and for choosing MongoDB as its foundation.

Open the conversations collection in Claudius’ database and you find the usual fields of a thread header but nothing else: a userId, a title, some timestamps, and so on, but no array of messages, no messages collection sitting beside it either! The text of every conversation lives somewhere else entirely, in the LangGraph checkpointer, which I wire up later in this series. This absence is a modeling decision, and how I came up with the database schema for my chatbot is the theme of this article.

If you come from a relational background, you're used to modeling the data first when designing a database. For a project like this, you would start by finding the entities and normalizing them, and the final schema would come out of the data's structure: a conversations table and a messages table with a foreign key between them, because that is what the data looks like.

Document modeling runs the other way. You start from how the application reads and writes, and the shape of the document follows the access patterns. Claudius never reads conversation messages without the agent's full working state wrapped around them, and that state is persisted using the LangGraph checkpointer. A separate messages table would add nothing, since the app would always have to join it back to that state on every read. The access pattern says the messages belong with the agent state, so that is where they go, and conversations are left as the lightweight header the list view actually needs.

That inversion, modeling around use rather than around the data, runs through everything below.

Schema-flexible is not schemaless

This is the misconception lots of people often carry, and it is worth killing on the way in. A document database does not mean no schema; it means a flexible, use-case-based data model. A relational engine holds the schema for you through column types and constraints; MongoDB will store documents in whatever shape your application gives it, and can validate that shape with a collection validator if you want. Claudius keeps the schema in code, as one Zod schema per collection, because that single definition gives both the runtime check and the TypeScript type.

import { z } from "zod";
import { zObjectId } from "./common";

export const ConversationSchema = z.object({
  _id: zObjectId.optional(),
  userId: zObjectId,
  title: z.string(),
  modelId: z.string(),
  createdAt: z.date(),
  updatedAt: z.date(),
  archived: z.boolean(),
  expiresAt: z.date().optional(),
});

export type Conversation = z.infer<typeof ConversationSchema>;

The reason is drift. Declare a TypeScript interface for a document and a separate validator for the data coming in, and the two will disagree eventually, while the type system will lie to you with complete confidence. Deriving the type from the validator collapses them into one artifact. One definition gives you the runtime check at the boundary and the compile-time type everywhere else. For a relational reader, this is your DDL (Data Definition Language) and your ORM (Object-Relational Mapping) models together in the same structure, which is something relational stacks rarely pull off.

Another advantage is that every external input, whether an incoming HTTP body or a model's structured output, is parsed through Zod at the boundary, so by the time data reaches the logic, it is already validated and typed.

A quick tour of the model

There are eight collections in this phase: users, conversations, memories, documents, chunks, usage_events, settings, and, arriving later, jobs. Two more, checkpoints and checkpoint_writes, belong entirely to the checkpointer and are never written to directly, so I do not model them. Most of these schemas are unremarkable once you see the access-pattern framing, but two modeling moves are worth pausing on.

The first is settings. It holds a handful of global singletons, each a different shape, each identified by a string _id: the member allowlist, the model catalog, the per-role tiers, and the guest spend circuit breaker. They share one collection, so the Zod schema is a discriminated union based on _id, and parsing a settings document narrows it to the correct shape from its _id literal.

export const SettingsSchema = z.discriminatedUnion("_id", [
  AllowlistSettingsSchema,        // _id: "allowlist"
  ModelCatalogSettingsSchema,     // _id: "modelCatalog"
  TiersSettingsSchema,            // _id: "tiers"
  GuestCircuitBreakerSettingsSchema, // _id: "guestCircuitBreaker"
]);
export type Settings = z.infer<typeof SettingsSchema>;

In relational terms, this is a single typed configuration table whose rows legitimately carry different columns, which is usually awkward to model relationally without resorting to a JSON column, and here comes out clean and readable.

The second move is to use document embedding as a data modeling technique. The users document carries its dailyMessageCount as an embedded object holding the count and its reset time together, rather than as two flat fields or a side table, because the application always reads and writes them together as a unit. That is the document modeling mantra: data used together is stored together.

Let the database keep the rule

The most satisfying part of this phase is how little policy I had to write. Three product rules that I would normally enforce in application code are carried by the storage engine instead, because I shaped the data so the engine could hold them.

Retention is a TTL index

Guest data in Claudius is ephemeral. Guest-created conversations and memories carry an expiresAt date; member and admin documents leave the field off entirely. A TTL (time-to-live) index with expireAfterSeconds set to 0 tells MongoDB to delete each document at the exact moment stored in its own expiresAt field; documents without this field are never touched.

// TTL on the guest-only expiresAt field. expireAfterSeconds:0 
// means "expire exactly at the date stored in the field"; 
// documents without the field are never touched, so member/admin
// data is permanent.

await db
  .collection(COLLECTIONS.conversations)
  .createIndex({ expiresAt: 1 }, { name: "expiresAt_ttl", expireAfterSeconds: 0 });

await db
  .collection(COLLECTIONS.memories)
  .createIndex({ expiresAt: 1 }, { name: "expiresAt_ttl", expireAfterSeconds: 0 });

So, guest ephemerality, a product rule, is enforced by the engine. There is no cron job and no cleanup endpoint to remember to run. The relational equivalent is a scheduled sweep; here, the policy lives in the index.

One detail ties the TypeScript compiler to the database behavior. With exactOptionalPropertyTypes turned on in tsconfig.json, an optional field means the key is absent, not present, and set to undefined. The compiler setting and the storage rule reinforce each other.

Telemetry is a time-series collection

Every billable model call writes one usage_events document, recording the token counts, the latency, and what the call was for. That data is append-only and gets queried along a few low-cardinality dimensions over a time window, which is the textbook shape for a MongoDB time-series collection. It has a timeField, timestamp, and a single metaField that holds the grouping dimensions (userId, modelId, purpose) nested alongside the measurements.

Two things follow from this choice. A time-series collection has to be created explicitly with a timeseries spec before the first insert (they cannot be created on first write, the way an ordinary collection does), so the index script creates it behind an existence check. Once it exists, the engine's own machinery becomes visible: list the collections and a system.buckets.usage_events sits next to usage_events, which is where the bucketed storage lives:

[
  {"name":"memories","type":"collection"},
  {"name":"usage_events","type":"timeseries"},
  {"name":"system.buckets.usage_events","type":"collection"},
  {"name":"settings","type":"collection"},
  {"name":"users","type":"collection"},
  {"name":"conversations","type":"collection"},
  {"name":"accounts","type":"collection"},
  {"name":"chunks","type":"collection"},
  {"name":"system.views","type":"collection"}
]

The relational analogy is a metrics or rollup table you would otherwise hand-roll and maintain, except the bucketing is automatic and the query you write against it stays simple.

A pre-filter field is a security control

The project's hardest rule is that a vector search pre-filters by userId and can never return another user's data. That rule is not bolted onto the search in the application code. It is a property of the index.

Both memories and chunks get an Atlas Vector Search index on the embedding field, with a filter on userId (and on chunks, a second filter on documentId).

await ensureVectorIndex(db, COLLECTIONS.memories, "memories_vector", [
  { type: "vector", path: "embedding", numDimensions: 1024, similarity: "cosine" },
  { type: "filter", path: "userId" },
], created, skipped);

await ensureVectorIndex(db, COLLECTIONS.chunks, "chunks_vector", [
  { type: "vector", path: "embedding", numDimensions: 1024, similarity: "cosine" },
  { type: "filter", path: "userId" },
  { type: "filter", path: "documentId" },
], created, skipped);

Pre-filtering means the userId condition is applied during the indexed search on the same documents being scored, not afterward to whatever the search returns. Post-filtering is not only slower, since you score documents you then throw away, but it is a correctness and security risk, since the wrong documents can slip through ranking and limits before your filter ever runs.

This is also why chunks denormalizes userId. A vector search has no notion of a join, so the filter has to sit on the very documents being searched. A chunk belongs to a document that belongs to a user, but the chunk has to carry the owner itself, or the index cannot enforce the boundary. That is the embed-versus-reference tension made concrete: a chunk references its parent document by id and duplicates the one field the security boundary depends on.

Two gotchas surfaced while creating these indexes, and both are worth saying out loud. A search index can only be built on a collection that already exists, and chunks had no classic index, so the collection did not exist yet the first time the script reached for its vector index:

MongoServerError: Error retrieving collection UUID and view info ::
caused by :: Collection 'claudius.chunks' does not exist. (NamespaceNotFound)

The fix is to add an existence check and create the collection first only if it is missing. The second gotcha is that Atlas builds search indexes asynchronously, so createSearchIndex returns when the build is queued, not when it is queryable! We also need a name check that keeps reruns safe while it builds.

This is the final function to build the indexes from code using the driver's createSearchIndex, with the checks that keep the script safe to run on every deploy:

async function ensureVectorIndex(
  db: Db,
  collectionName: string,
  indexName: string,
  fields: Array<Record<string, unknown>>,
  created: string[],
  skipped: string[],
): Promise<void> {
  // A vector search index can only be built on an existing
  // collection, so create it first if nothing else has (e.g.
  // chunks has no regular index).
  const collections = await db
    .listCollections({ name: collectionName })
    .toArray();
  if (collections.length === 0) {
    await db.createCollection(collectionName);
  }

  const collection = db.collection(collectionName);
  const present = await collection.listSearchIndexes().toArray();
  if (present.some((idx) => idx.name === indexName)) {
    skipped.push(`${collectionName}.${indexName}`);
    return;
  }

  // Atlas builds the index asynchronously; createSearchIndex
  // returns once the build is queued. The name check above 
  // keeps re-runs idempotent.
  await collection.createSearchIndex({
    name: indexName,
    type: "vectorSearch",
    definition: { fields },
  });
  created.push(`${collectionName}.${indexName} (vector)`);
}

The plumbing that keeps the code honest

Two small things hold the userId rule together in practice. First, userId is an ObjectId everywhere, matching the user’s _id, so filters and lookups line up without conversion. Second, no collection is ever queried through a raw driver call. Every access goes through a typed accessor, conversationsCol(), memoriesCol(), and so on, each returning a typed collection for the right document shape. Centralizing the collection names means a rename happens in one place, and typing each accessor keeps the userId filter visible to the compiler at every call site, which is how a rule that lives in prose stays honest in code.

The whole application and the Auth.js adapter share a single, serverless-safe MongoDB client to a single fixed database name, so there is exactly one connection pool. How that database name quietly caused two separate failures is a good story, and it belongs to the next article, where I stand the foundation up.

What this article bought

No chatbot yet, but the shape of the thing is already decided. Retention, the telemetry layout, and the tenant-isolation boundary are properties of the schema and its indexes now, not code I have to remember to run later. That is the payoff of modeling around the access patterns first: the database carries more of the load because you shaped it to. The data model is the foundational piece of the whole app.

Next, we'll build this foundation up, with identity the client never gets a vote in, and a health check that proves the app can reach Atlas and Bedrock, plus a view on the unglamorous errors that cost me real time. After that comes the streaming chat backbone, which is where the conversation finally gets a place to live, and I open a real checkpoint document to show you the messages that were never in their own collection.

Advanced API Authentication Strategies with Node.js and MongoDB (Part 3)

MongoDB Guests — Thu, 02 Jul 2026 09:00:00 +0000

This article was written by Moses Anumadu.

Secure Session Management with JWT and Refresh Tokens in Node.js and MongoDB

This is the final part of this 3-part series. In parts 1 and 2, we implemented email-based authentication. Then we went further to implement TOTP-based multi-factor authentication (MFA), QR code onboarding, and half-authentication login flow.

What we have built so far is an API that allows users to prove their identity using email and password. In this section, we will go beyond that. The application should not only be able to recognize users but also protect the user through concepts like secure session management, token expiration, and session revocation. We will implement these concepts through the following:

JWT access tokens
refresh tokens
MongoDB-backed sessions
refresh token rotation
protected routes
secure logout flows

By the end of this section, we should have an authentication system that can be deployed to production and improved upon for more advanced features.

Pre-requisites

To follow along with this tutorial, ensure you have prior experience working with the following technologies:

Node.js and Express
MongoDB and Mongoose
A MongoDB Atlas account
Postman
Ngrok
Parts one and two of this series

With that said, let's get started.

Understanding Access Tokens vs Refresh Tokens

An access token is a JWT token that grants a user access to a protected route. It often has a very short-lived life cycle. A refresh token is a token with a longer expiration time solely for requesting an access token. Without the refresh token, the user would need to log out and log back in to generate a new access token. The refresh token is stored in the database, enabling the system to recognize the user without terminating the session.

You are probably wondering why you need both, right? Let's look at an example. Imagine that a hacker steals a user's access token, and with this token, he (the hacker) can do malicious things. If this access token does not have a short-lived life cycle, the attacker would have a longer window to do more harm. To prevent this, the refresh token is required to generate a new access token from time to time. So, even if an access token is stolen, within 15 minutes, that token becomes invalid. The attacker can't do any harm.

This is an extra layer of security necessary for a secure authentication system and to improve the user experience. The flow looks like: a user logs in, access token and refresh token are granted, access token lasts for 15 minutes, refresh token lasts for 7 days, access token expires after 15 minutes, user does not need to log out, user generates a new access token using the refresh token saved in the database.

Creating Token Utilities

Now, we understand why we need both an access token and a refresh token. Let's proceed to the implementation. In part 2, when we created the MFA token, we created src/utils/tokenUtils.js file. We need to update that file to generate our access token, refresh token, and MFA token. Open src/utils/tokenUtils.js and replace the code with the code below:

const jwt = require("jsonwebtoken");

const crypto = require("crypto");

const generateAccessToken = (user) => {
  return jwt.sign(
    {
      userId: user._id,
      email: user.email,
    },
    process.env.JWT_ACCESS_SECRET,
    {
      expiresIn: process.env.ACCESS_TOKEN_EXPIRES_IN,
    }
  );
};

const generateRefreshToken = () => {
  return crypto.randomBytes(64).toString("hex");
};

const hashToken = (token) => {
  return crypto
    .createHash("sha256")
    .update(token)
    .digest("hex");
};

const generateMFAToken = (user) => {
  return jwt.sign(
    {
      userId: user._id,
      type: "mfa",
    },
    process.env.JWT_ACCESS_SECRET,
    {
      expiresIn: process.env.MFA_TOKEN_EXPIRES_IN,
    }
  );
};

module.exports = {
  generateAccessToken,
  generateRefreshToken,
  hashToken,
  generateMFAToken,
};

We added three methods generateAccessToken, generateRefreshToken, hashToken to the page. The hashToken will be used later. It will be used to encrypt our refresh token into a one-way SHA-256 hash before it is stored in the database or compared when validating. The generateAccessToken and generateRefreshToken generate an access token and a refresh token, respectively.

Why Refresh Tokens Are Not JWTs

Also, did you notice that our refresh tokens were generated using secure random bytes? This line of code here: crypto.randomBytes(64).toString("hex");. This simply gives us more control over it. The refresh token should be random, unpredictable, and most importantly, database-controlled. Refresh tokens, unlike access tokens, do not need readable claims.

So, this way, we can easily revoke it, rotate it, and control the session validation or invalidation.

Creating the Session Model

Our refresh token is database-controlled. That ultimately means we need to store and retrieve it from the database. To achieve this, we need to create a MongoDB model for storing user sessions. Create src/models/Session.js and update the page with the code below:

const mongoose = require("mongoose");

const sessionSchema = new mongoose.Schema(
  {
    userId: {
      type: mongoose.Schema.Types.ObjectId,
      ref: "User",
      required: true,
    },

    refreshTokenHash: {
      type: String,
      required: true,
    },

    userAgent: {
      type: String,
      default: null,
    },

    ipAddress: {
      type: String,
      default: null,
    },

    expiresAt: {
      type: Date,
      required: true,
    },

    revokedAt: {
      type: Date,
      default: null,
    },
  },
  {
    timestamps: true,
  }
);

module.exports = mongoose.model("Session", sessionSchema);

Notice that we are storing refreshTokenHash. Always remember that sensitive data at rest should also be encrypted. We created the fields we need to save for each user session. Notice fields like ipAddress, userAgent, expiresAt and revokedAt.

Issuing Tokens After MFA Verification

Now we can generate both a refresh token and an access token, and our MongoDB model is ready to save session values in the database. Next, we need to modify the authentication flow to include these. The flow should be: a user logs in, completes MFA, the application issues an access token and a refresh token to the user, and finally, the session values (including the refresh token) are stored in our MongoDB session collection. Let's proceed to the implementation.

!!!
Returning refresh tokens in JSON bodies is unsafe for browser clients and encourages XSS-exposed storage patterns. The safer pattern is HTTP-only cookie transport for refresh credentials. This is the approach we will follow in the next section.
!!!

Updating the Login Controller

In the next section, we will send refresh tokens using HTTP-only cookies transport as this is the safer option to prevent XSS-expose attacks.
The code in src/controllers/authController.js needs to be updated. Let's start by adding the import values and HTTP-only cookies configurations.

Open src/controllers/authController.js and add the following import statement to the top part of the page

const jwt = require("jsonwebtoken");
const Session = require("../models/Session");

const {
  generateAccessToken,
  generateRefreshToken,
  hashToken,
  generateMFAToken,
} = require("../utils/tokenUtils");

After the import statements, add the following HTTP-only cookie configurations. We will use these later in the code.

const getRefreshTokenExpiryMs = () => {
  const refreshDays = Number(
  process.env.REFRESH_TOKEN_EXPIRES_DAYS
  );

  return refreshDays * 24 * 60 * 60 * 1000;
};

const getRefreshTokenCookieOptions = () => {
  return {
    httpOnly: true,
    secure: process.env.NODE_ENV === "production",
    sameSite: "strict",
    path: "/api/auth",
    maxAge: getRefreshTokenExpiryMs(),
  };
};

const setRefreshTokenCookie = (res, refreshToken) => {
  res.cookie(
  "refreshToken",
  refreshToken,
  getRefreshTokenCookieOptions()
  );
};

const clearRefreshTokenCookie = (res) => {
  res.clearCookie("refreshToken", {
    ...getRefreshTokenCookieOptions(),
    maxAge: undefined,
  });
};

Updating the MFA Login Verification Flow

Next, replace the success section inside verifyMFALogin controller method with the code below:

Revoke this section

return res.status(200).json({
success: true,
message: "MFA login successful",
});

And replace with this:

// Issue access token
const accessToken = generateAccessToken(user);

// Generate refresh token
const refreshToken =
  generateRefreshToken();

// Hash refresh token
const refreshTokenHash =
  hashToken(refreshToken);

// Session expiry
const expiresAt = new Date();

expiresAt.setDate(
  expiresAt.getDate() +
    Number(
      process.env.REFRESH_TOKEN_EXPIRES_DAYS
    )
);

// Store session
await Session.create({
  userId: user._id,
  refreshTokenHash,
  userAgent: req.headers["user-agent"],
  ipAddress: req.ip,
  expiresAt,
});
setRefreshTokenCookie(res, refreshToken);
      return res.status(200).json({
       success: true,
       message: "Login successful",
       accessToken,
     });

From the additions we made, after password verification and MFA verification, a token is issued, and a session is created for the authenticated user.

Creating JWT Authentication Middleware

Now, we can issue an access token and a refresh token. Next, we need a way to protect our routes so that only users with the right access can access them. To do this, we need to implement middlewares.

A good example can be creating a middleware that protects routes and lets only authenticated users have access to it. Let's implement that. Create the following file src/middleware/authMiddleware.js, and update it with the code below:

const jwt = require("jsonwebtoken");

const User = require("../models/User");

const requireAuth = async (
  req,
  res,
  next
) => {
  try {
    const authHeader =
      req.headers.authorization;

    if (
      !authHeader ||
      !authHeader.startsWith("Bearer ")
    ) {
      return res.status(401).json({
        success: false,
        message: "Access token required",
      });
    }

    const token = authHeader.split(" ")[1];

    const decoded = jwt.verify(
      token,
      process.env.JWT_ACCESS_SECRET
    );

    const user = await User.findById(
      decoded.userId
    );

    if (!user) {
      return res.status(401).json({
        success: false,
        message: "Invalid user",
      });
    }

    req.user = user;

    next();
  } catch (error) {
    return res.status(401).json({
      success: false,
      message:
        "Invalid or expired access token",
    });
  }
};

module.exports = requireAuth;

Creating a Protected Route

At this point, the middleware is ready. We need to attach it to our routes to restrict access to only authenticated users. Let's create a controller to get the authenticated user getMe and a route for it /api/auth/me then protect it with the middleware we created above. To create the controller method, open src/controllers/authController.js and add the following controller method to the page.

const getMe = async (req, res) => {
  return res.status(200).json({
    success: true,
    user: {
      id: req.user._id,
      email: req.user.email,
      mfaEnabled: req.user.mfaEnabled,
    },
  });
};

Also, update the export block to contain the getMe method we just added, like so

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
  getMe,
};

Adding the Protected Route

Next, let's create the route. Open src/routes/authRoutes.js and add the following to the page.

First, import the middleware by copying and pasting the code below to the top section of the page

const { getMe } = require("../controllers/authController");
const requireAuth = require("../middleware/authMiddleware");

Then add the route below to the route on the page.

router.get("/me", requireAuth, getMe);

Testing Protected Routes

We've made progress. Let's test what we've built so far. We will test with Postman on the web and Ngrok. Ngrok will be used to tunnel our localhost address to something on the internet. Check Part One of this article for more on the setup. With that said,

Login

To generate an access token, we need to go through a series of verifications. First, we need to log in and generate an MFA token to verify MFA. Send a post request to the route below to log in.

POST /api/auth/login

Add the following to the body

{ 
  "email": "test_user@example.com",
  "password": "password123"
}

Your response should look similar to the image below if everything is done right.

MFA verification and access key generation

We are partially logged in. We need to verify MFA to be fully logged in. Send a Post request to the route below:

POST /api/auth/mfa/login/verify

Also, add the following to the body of the request:

{ "token": "TOKEN_GENERATED_BY_AUTH_APP",
  "mfaToken": "MFA_TOKEN_GENERATED_FROM_LOGIN"
}

Protected route

Next, send a GET request to the route below

GET /api/auth/me

With the following Header values

Authorization: Bearer YOUR_ACCESS_TOKEN

If the token is valid, the request should return the authenticated user information, like the image below

Also, in MongoDB Atlas, you can see that user sessions were created and stored in our session collection.

Implementing Refresh Token Rotation

Let's take our authentication a step further by implementing refresh token rotations. What we want to achieve is simple. Every refresh request should generate a new access token, generate a new refresh token, and invalidate the old refresh tokens. Let's jump right into it.

Creating the Refresh Endpoint

Add the following controller method to src/controllers/authController.js.


const refreshAccessToken = async (req, res) => {
   try {
     const refreshToken = req.cookies?.refreshToken;
      if (!refreshToken) {
       return res.status(401).json({
         success: false,
         message: "Refresh token cookie required",
       });
     }
      const refreshTokenHash = hashToken(refreshToken);
      const session = await Session.findOne({
       refreshTokenHash,
       revokedAt: null,
     });
      if (!session) {
       clearRefreshTokenCookie(res);
       return res.status(401).json({
         success: false,
         message: "Invalid session",
       });
     }
      // Expired session
     if (session.expiresAt < new Date()) {
       clearRefreshTokenCookie(res);
       return res.status(401).json({
         success: false,
         message: "Session expired",
       });
     }
      const user = await User.findById(session.userId);
      if (!user) {
       return res.status(401).json({
         success: false,
         message: "Invalid user",
       });
     }
      // ROTATE REFRESH TOKEN
     const newRefreshToken = generateRefreshToken();
      const newRefreshTokenHash = hashToken(newRefreshToken);
      session.refreshTokenHash = newRefreshTokenHash;
      await session.save();
     setRefreshTokenCookie(res, newRefreshToken);
      // Generate new access token
     const accessToken = generateAccessToken(user);
      return res.status(200).json({
       success: true,
       accessToken,
     });
   } catch (error) {
     console.error(error);
      return res.status(500).json({
       success: false,
       message: "Failed to refresh token",
     });
   }
 };

Next, let's update the exports with the freshAccesstoken method like so:

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
  getMe,
  refreshAccessToken,
};

Adding the Refresh Route and Testing

Let's proceed by adding a route to test the refresh token feature. Open src/routes/authRoutes.js and update the page with the code below

router.post(
  "/refresh",
  refreshAccessToken
);

Let's proceed to test. Send a POST request to the route below.

POST /api/auth/refresh

Request body:

{
  "refreshToken": "YOUR_REFRESH_TOKEN"
}

If everything was done right, your response should look like the screenshot above. And the previous tokens should no longer work.

Implementing Logout and Session Revocation

A complete logout system not only removes the access token from the browser in the front end. It should also invalidate the refresh token and revoke the session from the backend. That is exactly what we will do in this section.

Creating the Logout Endpoint

Add the following controller method to src/controllers/authController.js

const logout = async (req, res) => {
   try {
     const refreshToken = req.cookies?.refreshToken;
      if (!refreshToken) {
       return res.status(400).json({
         success: false,
         message: "Refresh token cookie required",
       });
     }
      const refreshTokenHash = hashToken(refreshToken);
      await Session.findOneAndUpdate(
       {
         refreshTokenHash,
         revokedAt: null,
       },
       {
         revokedAt: new Date(),
       }
     );
     clearRefreshTokenCookie(res);
      return res.status(200).json({
       success: true,
       message: "Logged out successfully",
     });
   } catch (error) {
     console.error(error);
      return res.status(500).json({
       success: false,
       message: "Logout failed",
     });
   }
 };

From the code, we did not delete the session records. We marked them as revoked instead, as we can see below.

await Session.findOneAndUpdate(
{
refreshTokenHash,
revokedAt: null,
},
{
revokedAt: new Date(),
}
);

This is a good practice as it helps preserve session history for audit and security visibility.

Implementing Logout From All Devices

Let's take it a step further by allowing users to revoke all active sessions simultaneously. This would let them log out from all devices at once. This is a nice feature to have; it helps a user protect their account in the case of a malicious login from an unknown device.

Let's proceed, modify src/controllers/authController.js file, and add the following controller method:

const logoutAll = async (req, res) => {
   try {
     await Session.updateMany(
       {
         userId: req.user._id,
         revokedAt: null,
       },
       {
         revokedAt: new Date(),
       }
     );
     clearRefreshTokenCookie(res);
      return res.status(200).json({
       success: true,
       message: "Logged out from all devices",
     });
   } catch (error) {
     console.error(error);
      return res.status(500).json({
       success: false,
       message: "Failed to logout all sessions",
     });
   }
 };

Update the export block to contain all the methods below

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
  getMe,
  refreshAccessToken,
  logout,
  logoutAll,
};

Adding Logout Routes

To add the routes for logout, update src/routes/authRoutes.js and add the code below. Also ensure that you have imported both logout and logoutAll methods from authController.js.

router.post("/logout", logout);

router.post(
  "/logout-all",
  requireAuth,
  logoutAll
);

Testing Logout

Send a post request to the route below. Also, pass the body value in the body of your request.

Request:

POST /api/auth/logout

Request body:

{
  "refreshToken": "YOUR_REFRESH_TOKEN"
}

Your response should look similar to the screenshot below

You can also test the logout all feature the same way. Send a Post request to the route below with a valid access token.

Request:

POST /api/auth/logout-all

Also, add a header with a user's access token.

Authorization: Bearer YOUR_ACCESS_TOKEN

Expected response:

{
  "success": true,
  "message":
    "Logged out from all devices"
}

Did you notice that the logout-all requires an access token passed in as a request header, not a refresh token like the logout route? This is because the logout-all does not have an active user session or know the user sending the request. It needs a way to identify the user, so it requires a user's access token.

All refresh token sessions should now become invalid. To be sure, you can also try using the same refresh token. It should fail because this session has been revoked.

Conclusion

A big thumbs up to you for sticking around till the end of this three-part series. We built a complete authentication API using Node.js, Express, MongoDB Atlas, and tested using Postman on the web and Ngrok for localhost routing.

In this final part of this series (part three), we discussed the difference between JWT access tokens and refresh tokens, why we need both, and walked through the programmatic implementation of the authentication by implementing the following:

JWT access token authentication
Session storage
refresh token rotation
Middleware and protected routes
secure logout and session revocation

The complete flow of the authentication with all three (3) parts put together looks like:

User registration, password authentication, MFA setup and verification, JWT access tokens and refresh token issuance, refresh token sessions, middlewares and protected routes, session.

Security is more of layers than single features. For further reading, check out the following topics for additional layers that can be added to an authentication.

Rate limiting
Email verification
HTTPS enforcement
Secure HTTP-only cookies
Brute-force protection
Device monitoring
Anomaly detection

Awesome. You can find the complete code on GitHub.

I'm building my own Claude-based chatbot, powered by MongoDB

MongoDB Guests — Thu, 02 Jul 2026 08:39:00 +0000

This tutorial was written by Néstor Daza.

I use Claude every day. It's a great tool for my day-to-day work, helping with everything from research and text processing to, above all, writing code (nothing surprising here). But today, that little coder voice in my head asked me: what would it take to build my own version, running on infrastructure I understand and control? What if I could have my own AI bot? So that is what I am doing, and this is the first entry in a series documenting it.

The app is called Claudius. It is a personal, Claude-style chat workspace I am building for myself, with MongoDB as its entire backbone and Claude models served through AWS Bedrock. Every article in the series will ship with public code. The app runs on MongoDB Atlas’s free tier, so nothing here is a pattern you read about and cannot reach. You can clone it, run it, and lift the parts you want into your own work.

Why build it when Claude already exists?

Yes, yes, Claude already exists, and it is very good, so why build a smaller, less-featured version of it? Because the best teacher is you, building with your own hands! Building the agent loop by writing the code, with state and retrieval, I actually have to figure out how to make it work, and the entire infrastructure needed to run it will not only show me a lot of what a finished product hides, but should also be a really fun project!

There is a personal payoff, too. I want a chat app where I own the data, one that remembers the things I tell it across conversations, sitting in a database I run. It is built for me first, and keeping it personal also keeps the scope honest.

The build is also great content. Writing is part of what I do, so a project I would build anyway turns into a series that other developers can follow. That's the reason you are reading this.

MongoDB is the whole backbone

Having MongoDB as the only datastore in the project is the decision that shapes everything else. Not a database plus a cache plus a vector store plus a queue, just MongoDB.

That is uncommon enough to need some explaining. A chat app like this has to keep users and conversations somewhere. It has to persist the agent's working state so a conversation survives a page reload. Retrieval over your documents and long-term memories needs a vector store. Background work, once it shows up, needs somewhere to queue. The usual answer is to use a separate product for each of those, and you end up operating four systems that have to be kept in sync. I am using MongoDB throughout, because the document model and Atlas Vector Search cover that whole surface, and one database is far less to run and reason about.

Embeddings come from Voyage AI, which is now part of MongoDB. That matters more than it first sounds. The data and the retrieval layer that sits on top of it come from the same place, so there is no second vendor to manage for the part of the app that handles search. Voyage's models are also strongly multilingual, which I care about because I work across AMER and LATAM in English and Spanish, and embedding quality in Spanish is the difference between retrieval that works and retrieval that only pretends to while quietly missing the point.

If you come from a relational background, I will translate and mention parallels and differences as we go. When a document-model choice replaces something you would have built with a join table or an extra service, I will say so, explain it, and show the relational equivalent to make the tradeoff clear.

Why is Claude running through Bedrock?

The one big piece that is not MongoDB is the model layer, and every Claude model the app uses is served through AWS Bedrock. This choice comes down to convenience and ease of use: being able to move from a fast, cheap model to a slower, more capable one is a single string in a request, which makes the model picker in the UI almost free to build. Keeping model access within a single cloud relationship also puts credentials and billing in one place, which matters the moment there is a usage bill to watch.

What it does, and what it isn't

So what does Claudius actually do? Four things only. I'm capping the scope of the whole project right here.

It holds a streamed conversation and lets me switch Claude models in the middle of a thread. It remembers, both within and across conversations, persistent facts instead of hoarding transcripts. It lets me upload a file and ask questions answered from its contents. And it can run a longer research task that plans an approach and works through web sources to return a cited answer.

But it is not a product! There is no commercial plan behind it, no growth target to hit, and more importantly, I'll leave the full-blown version of a Claude Agent to the good folks at Anthropic; my plan is to keep Claudius very slim. Sign-in is Google and nothing else, and a user's role is decided on the server, so no public path hands out anything on its own. It is built for me, and late in this series, I open a small public tier so readers can try the finished thing. That tier is hard-capped but open enough to show you what the end result looks like, so stay tuned for one of the final articles of this series!

This is not a clone!

One very important thing to be clear about: Claudius is a personal homage and a learning project. It is not affiliated with or endorsed by Anthropic, and it is not meant to replace Claude, which remains the real thing. This is me rebuilding the shape of a tool I use and admire, to understand how it stands up, while showcasing the power of MongoDB as the sole data foundation of a complete agentic AI application.

How it will be built

The build runs in six phases, and the article series will track them closely.

Phase 0 lays the foundations, the data model, and the auth/role system that everything else assumes. Phase 1 brings the streaming chat backbone, Claude over Bedrock wired into the frontend, the first real conversation. Phase 2 adds file upload and retrieval for what you upload. Phase 3 is the one I am most looking forward to: long-term memory that survives across conversations. Phase 4 moves the heavier work, including the research agent, onto a background worker. Phase 5 closes the loop with usage metering and cost controls, and opens the small public tier.

Because every phase ships with public code, hosted in Github. The repo carries an annotated .env.example that lists every credential the build needs and the phase it belongs to, so you can reproduce each step instead of just reading it.

When the public tier goes live near the end of the series, it will live at askclaudius.dev, and you will be able to open the finished app and use it yourself.

That's enough on the why. The next article comes with the decision that surprised me most: Claudius has no message table! I'll show you where the conversation really lives, and why the document model is the right call and not just a clever hack. If you came up on SQL, this is the one to read!

Building a Kotlin Application with Quarkus and MongoDB: A Step-by-Step Guide

Ricardo Mello — Tue, 30 Jun 2026 13:43:36 +0000

Kotlin, Quarkus, and MongoDB — do these three technologies work well together?

In this tutorial, we’ll build a simple application that connects to MongoDB using the Panache pattern and performs aggregation queries.

Along the way, we’ll set up the project, explore the structure, and run a basic aggregation pipeline to query a movies dataset.

Prerequisites

Basic knowledge of Kotlin
Maven installed
A MongoDB instance (Atlas or local)
Docker installed and running (optional, if you want to use the local lab setup)

What we will build

We’re going to build a simple application that connects to a database containing around 21,000 movies.
Each movie document has a structure similar to this:

{
 "title": "The Great Train Robbery",
 "fullplot": "Among the earliest existing films in American cinema...",
 "plot": "A group of bandits stage a brazen train hold-up, only to find a determined posse hot on their heels.",
 "genres": [
   "Short",
   "Western"
 ],
 "cast": [
   "A.C. Abadie",
   "Gilbert M. 'Broncho Billy' Anderson",
   "George Barnes",
   "Justus D. Barnes"
 ],
...
}

Our goal is to create an aggregation that selects cast members and counts how many movies each one appears in, returning results like this:

{
 "_id": "Gérard Depardieu",
 "totalMovies": 67
},
{
 "_id": "Robert De Niro",
 "totalMovies": 58
}

Setting up MongoDB (Optional)

To get started, we first need to set up a MongoDB connection.
You have a couple of options:

Create a free MongoDB Atlas account and set up a cluster by following this tutorial
Or use a local MongoDB instance To make things easier, you can use this CLI tool to spin up a local MongoDB environment with sample data already loaded:

1 - Install the cli:

npm install -g @ricardohsmello/mongodb-cli-lab

2 - Create the cluster:

mongodb-cli-lab up --topology standalone --mongodb-version 8.2 --port 28000 --sample-databases sample_mflix

This is a community-driven initiative and not an official MongoDB product.
It is intended for development, demos, and learning purposes only.
Not intended for production use.

This command starts a local MongoDB instance and automatically loads the sample_mflix dataset, which includes the movies collection we’ll use in this tutorial.

Once your MongoDB instance is up and running, make sure you have access to the movies collection, as we’ll use it to build our aggregation queries.

Creating the Quarkus project

Now that our database is ready, let’s create our Quarkus application.
We’ll use the Quarkus Maven plugin to bootstrap the project with the required extensions:

mvn io.quarkus.platform:quarkus-maven-plugin:3.34.1:create \
   -DprojectGroupId=com.example.mongodb.lab \
   -DprojectArtifactId=kotlin-quarkus-lab \
   -Dextensions='kotlin,rest-jackson, quarkus-mongodb-panache-kotlin'

Here, we are including a few important extensions:

Kotlin – to write our application using Kotlin
REST (Jackson) – to expose HTTP endpoints and handle JSON
MongoDB Panache – to simplify data access and interactions with MongoDB

Once the project is created, open it in your favorite IDE to continue.

Configuring the MongoDB Connection

With the project open in your IDE, the first step is to configure the connection to MongoDB.
To do that, open the application.properties file and add the following properties:

%dev.quarkus.mongodb.connection-string=mongodb://localhost:28000
%dev.quarkus.mongodb.database=sample_mflix

These settings configure the application to connect to our local MongoDB instance and use the sample_mflix database.
The %dev prefix means that these properties will only be applied when running the application in development mode.

Defining the Movie entity

Now, we need to create a class to represent our movies collection.
Using Panache, we can map this collection to a Kotlin class Movie.kt, like this:

import io.quarkus.mongodb.panache.common.MongoEntity

@MongoEntity(collection = "movies")
class Movie {
   lateinit var title: String
   var plot: String? = null
   var cast: List<String>? = null
}

The @MongoEntity annotation tells Quarkus which collection this class maps to.
Here, title is defined as a required field using lateinit, since we expect every document to have it.
On the other hand, plot and cast are optional fields, so we declare them as nullable (?) and initialize them with null. This allows the application to handle documents where these fields might be missing.

Creating the Repository

Next, we’ll create a repository to interact with MongoDB using Panache.

import io.quarkus.mongodb.panache.kotlin.PanacheMongoRepository
import jakarta.enterprise.context.ApplicationScoped

@ApplicationScoped
class MovieRepository : PanacheMongoRepository<Movie> {
}

By extending PanacheMongoRepository, we get access to built-in methods to interact with the database, such as queries and data operations.
This is where we’ll implement our aggregation logic.

Exposing the repository through a resource

Next, we’ll interact with this repository through a REST resource:

import jakarta.ws.rs.GET
import jakarta.ws.rs.Path
import jakarta.ws.rs.Produces

@Path("/movies")
@Produces("application/json")
class MovieResource(
   val repository: MovieRepository
) {
   @GET
   fun list() = repository.findAll()
       .list()
}

This resource exposes a /movies endpoint that returns all documents from the collection.

Running the Application and Testing the Endpoint

Now, let’s run the application and test the endpoint.
To make that easier, I’ll create an HTTP file called movie.http and add the following request:

### FIND
GET http://localhost:8080/movies
Accept: application/json

Once we run it, we’ll immediately notice the first problem: we are returning all 21,000 movies at once.

That is not a good idea. We should always be careful with how much data we return, both to avoid unnecessary load on the database and to make the API more efficient.

A simple improvement here is pagination.
So let’s go back to the MovieResource class and add two query parameters: page and size.

import jakarta.ws.rs.DefaultValue
import jakarta.ws.rs.QueryParam

@GET
fun list(
   @QueryParam("page") @DefaultValue("0") page: Int,
   @QueryParam("size") @DefaultValue("20") size: Int,
) = repository.findAll()
   .page(page, size)
   .list()

With this change, the endpoint will return only 20 results by default.
If we run the request again, we’ll get a much smaller, and more manageable response.

Working with the Aggregation Framework

The MongoDB Aggregation Framework is a powerful feature for transforming and analyzing data directly in the database. Instead of just retrieving documents, we can use aggregation stages to reshape the data, group values, sort results, and answer more interesting questions.

In this example, we want to find the actors who appear most frequently in the dataset. To do that, we’ll use the following aggregation pipeline:

[
 {
   $unwind:    
     {
       path: "$cast"
     }
 },
 {
   $group:    
     {
       _id: "$cast",
       totalMovies: {
         $sum: 1
       }
     }
 },
 {
   $sort:
     {
       totalMovies: -1
     }
 },
 {
   $limit: 3      
 }
]

Let’s break it down:

$unwind takes the cast field, which is an array, and splits it into multiple documents. In simple terms, if a movie has three actors in its cast, it will temporarily become three separate records during this stage, one for each actor.
$group then groups those records by actor name and counts how many times each one appears.
$sort orders the results in descending order, so the most frequent actors appear first.
$limit keeps only the top results.

Implementing the Aggregation in the Repository

Now let’s implement this pipeline inside the MovieRepository:

fun findTopFrequentActors(): List<Document> =
   mongoCollection().withDocumentClass(Document::class.java)
       .aggregate(
           listOf(
               Aggregates.unwind("\$cast"),
               Aggregates.group(
                   "\$cast", Accumulators.sum("totalMovies", 1)
               ),
               Aggregates.sort(Sorts.descending("totalMovies")),
               Aggregates.limit(3)
           )
       ).toList()

Here, we are using the underlying MongoDB collection and defining the aggregation stages with the MongoDB Java driver API.

Since the result of this pipeline does not map directly to our Movie class, returning a Document is a simple and practical choice.

Next, let’s expose this aggregation through a new endpoint in our MovieResource:

@GET
@Path("/top-frequent-actors")
fun findTopFrequentActors() = repository.findTopFrequentActors()

With this, our application now provides a dedicated endpoint to return the actors who appear in the highest number of movies.

### TOP FREQUENT ACTORS
GET http://localhost:8080/movies/top-frequent-actors
Accept: application/json

Once you run it, you should see a response similar to this:

[
 {
   "_id": "Gèrard Depardieu",
   "totalMovies": 67
 },
 {
   "_id": "Robert De Niro",
   "totalMovies": 58
 },
 {
   "_id": "Michael Caine",
   "totalMovies": 51
 }
]

And that’s it! We now have a working aggregation pipeline in a Kotlin application using Quarkus, Panache, and MongoDB.

Conclusion

Yes — Kotlin, Quarkus, and MongoDB work very well together in a practical and efficient way.
In this tutorial, we saw how to quickly build an application, map a collection using Panache, and leverage the MongoDB Aggregation Framework to work with data in a powerful yet simple way.

If you’d like to explore further, the full source code is available on GitHub.

Advanced API Authentication Strategies with Node.js and MongoDB (Part 2)

MongoDB Guests — Mon, 29 Jun 2026 09:00:00 +0000

Implementing MFA and TOTP with Node.js and MongoDB

This article was written by Moses Anumadu.

In part one (1) of this series, we implemented basic authentication. Users can register and also log in. In this section, we will take it a step further by adding multi-factor authentication (MFA) using a time-based one-time password (TOTP).

Basic authentication, which uses only a password and email, is arguably not secure enough for today's internet. Many users reuse their passwords, and attackers can exploit this vulnerability.

The core principle of multi-factor authentication is that it combines something the user knows with something the user owns to verify the user's identity. Something the User knows can be a password. Something the user owns can be a token sent to the user's device or email. The user needs to prove he knows the password and own the device where TOTP was sent.

If conditions are met, the user can be verified. With that said, let's get started on how to implement this in a Node.js and MongoDB application.

Pre-requisites

To follow along with this tutorial, ensure you have prior experience working with the following technologies:

Node.js and Express
MongoDB and Mongoose
A MongoDB Atlas account
Postman
Ngrok
Part one of this series

Adding MFA with TOTP

Let's go over what the flow of our MFA implementation would look like. So think of it like this:
First, the authenticated user has to enable MFA in the app. Next, the server generates a shared secret. Next, the user would need to scan a QR code with their phone camera or any of the authenticator apps of their choice (Google Authenticator, Authy, Microsoft Authenticator, etc). Next, the authenticator app would store the secret; the next app generates a 6-digit code. Finally, the server would verify the code. If correct, the user is authenticated.

The important thing to note here is that both the server and the authenticator app share the same secret. Using the current time and that secret, both sides independently generate matching one-time codes.

Creating a Temporary Authentication Middleware

One more step before we proceed. JWT authentication will be implemented in part three (3) of this tutorial. Since we have not implemented it yet, but need a way to identify the logged-in user, we will identify the user using the request header. This is temporary and will be updated to use JWT in part three. For now, create tempAuth.js in src/middleware/ directory. Update the file with the code below

const User = require("../models/User");

const tempAuth = async (req, res, next) => {
  try {
    const email = req.headers["x-user-email"];

    if (!email) {
      return res.status(401).json({
        success: false,
        message: "Missing x-user-email header",
      });
    }

    const user = await User.findOne({ email });

    if (!user) {
      return res.status(401).json({
        success: false,
        message: "User not found",
      });
    }

    req.user = user;

    next();
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Authentication middleware error",
    });
  }
};

module.exports = tempAuth;

This middleware is only temporary and will later be replaced with proper JWT authentication.

Generating MFA Secrets and QR Codes

We now have a way to temporarily identify the user. Let's proceed to the MFA secrets and QR codes implementation. If you can recall, in the first part of this series, we installed all the NPM packages we would need. Two of them were Otplib and Qrcode. It is the package of choice for this implementation. If you do not have them installed already, you can install them using the NPM command below:

npm install otplib

And

npm install --save qrcode

Once you confirm you have them installed, let's import and use them in the authController.js. Open src/controllers/authController.js. Add the code below to the top section of the page.

const QRCode = require("qrcode");

const {
  generateSecret,
  generateURI,
  verify,
} = require("otplib");

Next, add the code below after the login method on the page

const setupMFA = async (req, res) => {
  try {
    const user = req.user;

    // Generate shared secret
    const secret = generateSecret();

    // Generate OTP auth URL
    const otpAuthUrl = generateURI({
      label: user.email,
      issuer: "AdvancedAuthAPI",
      secret,
    });

    // Generate QR code image
    const qrCodeImageUrl =
      await QRCode.toDataURL(otpAuthUrl);

    // Store secret temporarily
    user.mfaSecret = secret;

    await user.save();

    return res.status(200).json({
      success: true,
      message: "MFA setup initialized",
      secret,
      qrCodeImageUrl,
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Failed to setup MFA",
    });
  }
};

module.exports = { register, login, setupMFA};

The code is well commented. Regardless, let's still go over a few lines and what they do. This line generates the shared secret: const secret = generateSecret();

The generated secret is then embedded into a special OTP URI in the block of code below

const otpAuthUrl = generateURI({
  label: user.email,
  issuer: "AdvancedAuthAPI",
  secret,
});

For the user to be able to scan, we converted the URI into a QR code using the qrcode package in the line of code below:

const qrCodeImageUrl =
await QRCode.toDataURL(otpAuthUrl);

When a user scans the QR code with their authenticator app, the authenticator app securely stores the shared secret.

Verifying MFA Codes

We have completed the MFA setup functionality. Let's proceed to the verification functionality. We need another controller method to verify the 6-digit code generated by the authenticator app. This will have a different endpoint also. Let's proceed. Still in src/controllers/authController.js add the following verification controller method below setupMFA:

const verifyMFA = async (req, res) => {
  try {
    const user = req.user;

    const { token } = req.body;

    if (!token) {
      return res.status(400).json({
        success: false,
        message: "MFA token is required",
      });
    }

    const verification = await verify({
      token,
      secret: user.mfaSecret,
    });

    const isValid = verification?.valid === true;

    if (!isValid) {
      return res.status(401).json({
        success: false,
        message: "Invalid MFA code",
      });
    }

    user.mfaEnabled = true;

    await user.save();

    return res.status(200).json({
      success: true,
      message: "MFA enabled successfully",
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Failed to verify MFA",
    });
  }
};

module.exports = { register, login, setupMFA, verifyMFA };

The token is passed along as req.body. When received in the code, it is verified using

const verification = await verify({
       token,
       secret: user.mfaSecret,
     });

If the code is valid, the user is logged in, and MFA is enabled. Did you notice that the MFA is only enabled after a successful verification? user.mfaEnabled = true;. This was done to prevent a broken MFA setup, which can accidentally lock a user out of their account. This prevents broken MFA setups and accidental account lockouts.

Adding MFA Routes

We need to update our authRoutes.js to include routes for both the setupMFA and verifyMFA controller methods. To do this, open src/routes/authRoutes.js and replace the content of the page with the code below

const express = require("express");

const {
  register,
  login,
  setupMFA,
  verifyMFA,
} = require("../controllers/authController");

const tempAuth = require("../middleware/tempAuth");

const router = express.Router();

router.post("/register", register);
router.post("/login", login);
router.post("/mfa/setup", tempAuth, setupMFA);
router.post("/mfa/verify", tempAuth, verifyMFA);

module.exports = router;

Testing MFA Setup

In part one (1) of this series, we set up Ngrok for tunneling our local host to the internet and tested our endpoints using Postman. If you do not have that setup, follow the link to set up to follow along.

If your project is not running already, start it using the command below

npm run dev

Then, send a POST request to the following endpoint:

POST /api/auth/mfa/setup

Remember our temporary middleware, set the header value to the value below

Headers:
x-user-email: test_user@example.com

The response should include a QR code image URL. Check out the images below for more details

Copy the URL from the response and paste it into a browser. You should see a QR code image, like what we have below

Next, scan the QR code using any authenticator app of your choice. After scanning, your authenticator app should begin generating 6-digit codes like the image below.

Testing MFA Verification

Let's proceed to test the MFA verification endpoint. Send a post request to the route below

POST /api/auth/mfa/verify

Also, send the following header values:

x-user-email: test_user@example.com

Then, the request body should contain the following:

{
  "token": "MFA_app_generated_code"
}

Don't forget to replace the token placeholder in the request body with the correct code from your authenticator app.

The images below should throw more light on the steps.

Implementing the Half-Authenticated Flow

Our MFA setup and verification now work. However, it is completely independent of the login and sign-up. We have created routes for the MFA but have not linked it to the login and sign-up flow. What we have now gives a user full authentication once a correct email and password are provided. We need to change that. We need a partial authentication flow: this is what I mean, when a user provides a correct email and password, we need to partially identify the user, and yet, prompt for MFA. Only when MFA is completed can the user be fully authenticated.

To achieve this, we will issue a temporary JWT token to the user after a correct password and email are provided; at this state the user is not yet authenticated but can be identified with this JWT token. Then, when the MFA process is completed, only then will the user be fully authenticated.

When we are done, the flow should be: password verified, issue a temporary JWT token to identify the user, MFA code generated from the authenticator app, the user submits the OTP code, and finally, authentication is completed. Let's proceed.

Creating MFA Tokens

Let's write the code to generate the temporary JWT token for the MFA process. Create src/utils/tokenUtils.js the following file and update it with the code below

const jwt = require("jsonwebtoken");

const generateMFAToken = (user) => {
  return jwt.sign(
    {
      userId: user._id,
      type: "mfa",
    },
    process.env.JWT_ACCESS_SECRET,
    {
      expiresIn: process.env.MFA_TOKEN_EXPIRES_IN,
    }
  );
};

module.exports = {
  generateMFAToken,
};

In the code above, we imported require("jsonwebtoken"); and used it to generate the JWT token. Token expiration and secret values come from the env file created earlier in part one of this series.

Updating the Login Flow

We need to update the login controller method to add the generated MFA JWT token. To continue, open src/controllers/authController.js and add the following.

First, add the import below to the top section of the page.

const jwt = require("jsonwebtoken");

const {
  generateMFAToken,
} = require("../utils/tokenUtils");

Next, let's update the login controller with the code below. Paste the code just above the return statement.

// MFA enabled
if (user.mfaEnabled) {
  const mfaToken = generateMFAToken(user);

  return res.status(200).json({
    success: true,
    requiresMFA: true,
    mfaToken,
  });
}

Now, users with MFA enabled no longer receive full authentication immediately after password verification. Instead, they receive a temporary MFA token.

Completing MFA Login

We need another controller method to process the MFA verification. Add the following controller below verifyMFA method:

const verifyMFALogin = async (req, res) => {
  try {
    const { token, mfaToken } = req.body;

    if (!token || !mfaToken) {
      return res.status(400).json({
        success: false,
        message:
          "Token and MFA token are required",
      });
    }

    // Verify temporary MFA token
    const decoded = jwt.verify(
      mfaToken,
      process.env.JWT_ACCESS_SECRET
    );

    if (decoded.type !== "mfa") {
      return res.status(401).json({
        success: false,
        message: "Invalid MFA session",
      });
    }

    const user = await User.findById(
      decoded.userId
    );

    if (!user || !user.mfaEnabled) {
      return res.status(401).json({
        success: false,
        message: "Invalid user",
      });
    }

    // Verify OTP
    const verification = await verify({
      token,
      secret: user.mfaSecret,
    });

    const isValid =
      verification?.valid === true;

    if (!isValid) {
      return res.status(401).json({
        success: false,
        message: "Invalid MFA code",
      });
    }

    return res.status(200).json({
      success: true,
      message: "MFA login successful",
    });
  } catch (error) {
    console.error(error);

    return res.status(401).json({
      success: false,
      message:
        "Invalid or expired MFA session",
    });
  }
};

Also, don't forget to update the export block with the code below.

module.exports = {
  register,
  login,
  setupMFA,
  verifyMFA,
  verifyMFALogin,
};

Adding the MFA Login Verification Route

Let's create routes for the above changes. Open src/routes/authRoutes.js and update the page with the code below:

router.post(
  "/mfa/login/verify",
  verifyMFALogin
);

Testing the Full MFA Login Flow

The first step is to log in normally. Send a Post request to the route below

POST /api/auth/login

If MFA is enabled, the response should contain an mfaToken just like the image below:

Next, verify MFA by sending a Post request to the route below

POST /api/auth/mfa/login/verify

The request should pass the code below in the body.
Request body:

{
  "token": "123456",
  "mfaToken": "..."
}

After testing, your response should look similar to the image below.

Conclusion

In this section, we added an extra layer on top of the existing authentication system we built in part one. After entering the correct email and password, users must configure and verify MFA using TOTP. To achieve this, we implemented the following in the code:

login verification
TOTP-based MFA
QR code onboarding
half-authenticated login flows

We've gone halfway, but we're not done. In Part 3, we will take it a step further by adding

JWT access tokens
refresh tokens
MongoDB-backed sessions
refresh token rotation
protected routes
secure logout and session revocation

See you in part 3. Cheers.

Advanced API Authentication Strategies with Node.js and MongoDB (Part 1)

MongoDB Guests — Thu, 25 Jun 2026 09:00:00 +0000

This article was written by Moses Anumadu.

Building Password Authentication with Node.js and MongoDB

Most database-driven applications require users to be authenticated. This is not a surprise. The surprise might come when you hear about the number of cyberattacks carried out daily and their complexity. According to Akamai Technologies, there are over 300 million and over 500 million credential-stuffing attacks per day. Given this volume of attacks, it is reasonable for applications to add more layers of authentication (MFA) to the traditional email-and-password authentication.

This three-part tutorial will guide you through building a production-style authentication system that goes beyond the basic signup and login using Node.js, Express, and MongoDB Atlas.

In this first part (Part 1) of the three-part series, we will implement the foundation of the authentication that we will build upon throughout the rest of the series. We will implement the following:

A secure password hashing with bcrypt.
User registration with email and password.
User logs in with email and password.

In Part 2, we will continue building on the existing codebase. We will add secure session management to the code by implementing the following:

Login verification
TOTP-based MFA setup and verification
QR code onboarding for TOTP setup
Half-authenticated login flows

And in Part 3, we will make it an authentication that you can deploy or use in your real-world application by implementing the following:

JWT access tokens
refresh tokens
MongoDB-backed sessions
refresh token rotation
protected routes
secure logout and session revocation

Pre-requisites

To follow along with this tutorial, ensure you have your development environment set up for the stack above and:

An understanding of Node.js and Express
An Understanding of MongoDB
A MongoDB Atlas account
Postman for testing API endpoints

Alright, let's get started.

Project Setup

To get started, we need to create a fresh Node.js project and install all the necessary dependencies. In your directory of choice, create the project using the command below:

mkdir devrel-node-api-authentication
cd devrel-node-api-authentication
npm init -y

An advanced authentication system like the one we are building relies on several dependencies in Node.js. We’ll install a few packages to handle those responsibilities. From your terminal, navigate to the project directory and then install dependencies using the command below

npm install express mongoose dotenv bcryptjs jsonwebtoken otplib qrcode && npm install --save-dev nodemon

Let's take a moment to go over all the packages we just installed. Express provides the web server and routing system for building our API, while Mongoose letting us interact with MongoDB through models and schemas. dotenv helps us load environment variables such as database credentials and secret keys. bcryptjs securely hashes passwords before storing them in the database and jsonwebtoken enables us to create and verify JWT tokens for authentication. For MFA functionality, otplib generates and validates time-based one-time passwords (TOTP), while qrcode converts our MFA setup information into a scannable QR code for authenticator apps. Finally, nodemon improves the development experience by automatically restarting the server whenever code changes are detected.

After the installation, we need to update the script section in our package.json file with the code below:

"scripts": {
  "dev": "nodemon src/server.js"
}

This would enable us to serve the project using npm run dev

Creating the Project Structure

We are making progress. Let's proceed by creating a structure for the project. In your project, create the following files and folders. The project structure should look like the sample below.

src/
├── config/
├── controllers/
├── middleware/
├── models/
├── routes/
├── utils/
├── app.js
└── server.js

Configuring MongoDB Atlas

Next, let's configure our application to interact with our MongoDB Atlas cluster. Ensure you have your Atlas connection string. If you do not have an Atlas account, follow the link to sign up and create an Atlas cluster, then obtain your connection string. With that said, create a


 file in the project root directory and update the content with the code below:


```bash
MONGO_URI_BASE=mongodb+srv://USERNAME:PASSWORD@cluster0.xxxxx.mongodb.net

JWT_ACCESS_SECRET=supersecretaccesskey
JWT_REFRESH_SECRET=supersecretrefreshkey

ACCESS_TOKEN_EXPIRES_IN=15m
REFRESH_TOKEN_EXPIRES_DAYS=7
MFA_TOKEN_EXPIRES_IN=5m

PORT=5000

Now that we have our Atlas connection string, let's proceed to connect our application to MongoDB using Mongoose. Create src/config/db.js and update the file with the code below.

const mongoose = require("mongoose");

const databaseName = "devrel-node-auth-db";
const appName = "devrel-tutorial-nodejs-authentication-geeksforgeeks";

const connectDB = async () => {
try {
  const mongoUri = `${process.env.MONGO_URI_BASE}/${databaseName}?retryWrites=true&w=majority&appName=${appName}`;

  await mongoose.connect(mongoUri);

  console.log("MongoDB connected");
  console.log(`Database: ${databaseName}`);
  console.log(`App Name: ${appName}`);
} catch (error) {
  console.error("MongoDB connection failed:");
  console.error(error.message);

  process.exit(1);
}
};

module.exports = connectDB;

Notice that the database name and appName are defined directly inside the codebase rather than inside the environment file. This keeps infrastructure-level configuration consistent across environments.

Creating the Express App

Now that we have the database configuration done, let's proceed to create an Express application. To keep things organized, we will create the app instance and register global middlewares in src/app.js. Update the file with the code below.

const express = require("express");
const authRoutes = require("./routes/authRoutes");
const app = express();

app.use(express.json());

app.use("/api/auth", authRoutes);
module.exports = app;

In src/app.js, we imported and used the authRoutes.js file. Let's go ahead and create this file before we proceed.
In src/routes/ directory, create the authRoutes.js file and update it with the code below:

const express = require("express");

const router = express.Router();

router.get("/health", (req, res) => {
 res.status(200).json({ message: "Auth routes are working" });
});

module.exports = router;

We created a simple /health route to verify that the application setup was correct.

Creating the Server Entry Point

Next, in copy and paste the code below into the src/server.js file.

const dotenv = require("dotenv");
const app = require("./app");
const connectDB = require("./config/db");

dotenv.config();

const PORT = process.env.PORT || 5000;

const startServer = async () => {
 await connectDB();
 app.listen(PORT, () => {
   console.log(`Server running on port ${PORT}`);
 });
};

startServer();

Now, we can test the configuration so far. In your project directory, start the development server using the command below:

npm run dev

If everything is configured correctly, you should see the following from your terminal:

MongoDB connected
Server running on port 5000

Creating the User Model

With all the configurations done, let's jump to the exciting part. So, the first step in authentication is creating a user. We need to implement code to create a user and securely store the password using bcrypt for Node.js in our MongoDB collection.

Create src/models/User.js file and update the file with the code below:

const mongoose = require("mongoose");

const userSchema = new mongoose.Schema(
 {
   email: {
     type: String,
     required: true,
     unique: true,
     lowercase: true,
     trim: true,
   },

   passwordHash: {
     type: String,
     required: true,
   },

   mfaEnabled: {
     type: Boolean,
     default: false,
   },

   mfaSecret: {
     type: String,
     default: null,
   },
 },
 {
   timestamps: true,
 }
);

module.exports = mongoose.model("User", userSchema);

From the code above, we created a User model. Mongoose was used to create our User schema and User model. One more thing to note is that, according to the schema, we are storing passwordHash instead of password. PasswordHash ensures that the password is not stored in plain text but is hashed using bcrypt.

Building User Registration

Let's proceed to the actual logic to create a new user. In src/controllers/authController.js, copy and paste the code below.

const bcrypt = require("bcryptjs");
const User = require("../models/User");

//Now add the registration controller:

const register = async (req, res) => {
  try {
    const { email, password } = req.body;

    // Basic validation
    if (!email || !password) {
      return res.status(400).json({
        success: false,
        message: "Email and password are required",
      });
    }

    // Password length validation
    if (password.length < 6) {
      return res.status(400).json({
        success: false,
        message: "Password must be at least 6 characters",
      });
    }

    // Prevent duplicate accounts
    const existingUser = await User.findOne({ email });

    if (existingUser) {
      return res.status(409).json({
        success: false,
        message: "User already exists",
      });
    }

    // Hash password
    const passwordHash = await bcrypt.hash(password, 12);

    // Create user
    const user = await User.create({
      email,
      passwordHash,
    });

    return res.status(201).json({
      success: true,
      message: "User registered successfully",
      user: {
        id: user._id,
        email: user.email,
      },
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Server error",
    });
  }
};
module.exports = { register };

From the code above, you can see that we validated for empty email and password values, password length, and prevented duplicate accounts by checking whether the email already exists. The next important thing to note is that the password was hashed using bcrypt
const passwordHash = await bcrypt.hash(password, 12);.

Bcrypt automatically handles salting, secure hashing, and computational cost. The second parameter (12) represents the salt rounds value. The higher this number, the more difficult it is for an attacker to crack this password.

Updating Authentication Routes

Let's create a route link for the Register logic. We already created the src/routes/authRoutes.js file. Now open the file and update the content with the code below.

const express = require("express");

const {
  register,
} = require("../controllers/authController");

const router = express.Router();

router.post("/register", register);

module.exports = router;

Testing Registration

It's time to test what we have built so far. For this tutorial, I will be testing with Postman on the web and tunneling my local environment using Ngrok. Feel free to test with what works for you or follow along with me.

If your server is not already running, navigate to your project directory and start the server with the command below;

npm run dev

In a different terminal window, start Ngrok using the command below:

Ngrok http 5000

You should expect an output similar to the image below:

Ngrok tunnels our localhost:5000 URL to https://b76b-197-157-218-195.ngrok-free.app or something similar. The Ngrok URL will be used to test in Postman, not our localhost URL.

To test, send a POST request to the register route: /api/auth/register. Also, add the following to the request body:

{
  "email": "test_user@example.com",
  "password": "password123"
}

Your output should be similar to the image below:

Try creating a duplicate user with the same record, and notice you should get a response similar to the image below

If you inspect your MongoDB Atlas database, you should now see a users collection containing the new user document with an encrypted password.

Building Password Authentication

Now, users can register. Let's proceed to Login. Registered users should be able to securely login to the system using their email and password. Still in src/controllers/authController.js, just below the register function, copy and paste the code below.

const login = async (req, res) => {
  try {
    const { email, password } = req.body;

    // Basic validation
    if (!email || !password) {
      return res.status(400).json({
        success: false,
        message: "Email and password are required",
      });
    }

    // Find user
    const user = await User.findOne({ email });

    // Prevent email enumeration attacks
    if (!user) {
      return res.status(401).json({
        success: false,
        message: "Invalid credentials",
      });
    }

    // Compare password with stored hash
    const isPasswordValid = await bcrypt.compare(
      password,
      user.passwordHash
    );

    if (!isPasswordValid) {
      return res.status(401).json({
        success: false,
        message: "Invalid credentials",
      });
    }

    return res.status(200).json({
      success: true,
      message: "Login successful",
      user: {
        id: user._id,
        email: user.email,
        mfaEnabled: user.mfaEnabled,
      },
    });
  } catch (error) {
    console.error(error);

    return res.status(500).json({
      success: false,
      message: "Server error",
    });
  }
};

Also, update the export, add the register function we just created, like so:

module.exports = {
  register,
  login,
};

In the code, we did a number of validations. The first one checked for empty strings, then we checked if this user exists in the database. Did you notice that both invalid email and invalid password return the same response: "Invalid credentials"? This prevents attackers from discovering which email addresses exist in the system. After that, we compared the input password with the password in the database. Also note that the password was not decrypted, it was instead compared against the encrypted version already stored in our database.

Adding the Login Route

Next, let's add a route for the login. Replace the code in src/routes/authRoutes.js with the code below to create a route for both /register and /login.

const express = require("express");

const {
  register,
  login,
} = require("../controllers/authController");

const router = express.Router();

router.post("/register", register);
router.post("/login", login);

module.exports = router;

Testing Login

To test, send a POST request to the login route: /api/auth/login. Also, add the following to the request body:

{
  "email": "test_user@example.com",
  "password": "password123"
}

We are making progress. We now have a very basic authentication system with Login and Register features. This is obviously nowhere near battle-ready for today's internet. So, we will proceed and strengthen this system with Multi-Factor Authentication (MFA).

Conclusion

In this part, we kept it simple and implemented user registration and login. Users can register with an email and a password; a JWT token is not issued yet at this stage. The system now only knows that the user's email and password are valid. To achieve this, we implemented the following:

A secure password hashing with bcrypt.
User registration with email and password.
User logs in with email and password. In the next part ( 2), we will build upon the foundation we create here by adding secure session management using:
login verification
TOTP-based MFA setup and verification
QR code onboarding
half-authenticated login flows See you in part 2.

.NET Server Side Events with MongoDB Change Streams

MongoDB Guests — Fri, 19 Jun 2026 09:00:00 +0000

This tutorial was written by Kevin Smith.

Real-time updates with almost no plumbing

If you want real-time browser updates in ASP.NET Core, you usually think of WebSockets first.

But there is a very nice middle-ground that is often enough for dashboard-style or feed-style UIs:

MongoDB Change Streams to detect database changes
Server-Sent Events (SSE) to push those changes to the browser

That gives you one-way, server-to-client streaming with a very small amount of code.

This post walks through the exact demo app and then covers where SSE shines, where it falls short, and when to move up to SignalR.

Demo in action

Here is the browser demo in action:

What are Server-Sent Events?

Server-Sent Events are a browser API (EventSource) built for receiving a stream of text events over a single HTTP connection.

Think of SSE as:

a long-lived HTTP response
UTF-8 text messages pushed by the server
automatic client reconnect support

SSE is one-way only (server to browser), which is often exactly what you need for:

live timelines
notifications
audit/activity feeds
admin dashboards

In ASP.NET Core Minimal APIs, this is now very straightforward using TypedResults.ServerSentEvents(...).

What are MongoDB Change Streams?

MongoDB Change Streams let you subscribe to real-time changes in collections, databases, or entire deployments.

Instead of polling MongoDB every few seconds, you can react to actual insert/update/delete operations as they happen.

At a high level:

your app opens a change stream cursor
MongoDB emits change events
your app maps each change into something your client can consume

For real-time systems, this removes a lot of waste from polling loops.

Wiring it together in ASP.NET Core

Let us build this from scratch.

1) Create the app

dotnet new web -n MongoDbServerSideEvents

cd MongoDbServerSideEvents

dotnet add package MongoDB.Driver

dotnet new web gives us a minimal ASP.NET Core app, which is perfect for a small real-time demo.

2) Replace Program.cs

Use this as your starting point:

using System.Runtime.CompilerServices;

using MongoDB.Bson;

using MongoDB.Bson.Serialization;

using MongoDB.Bson.Serialization.Attributes;

using MongoDB.Driver;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSingleton<IMongoClient>(_ =>

    new MongoClient("mongodb://host.docker.internal:27017"));

builder.Services.AddSingleton<IMongoDatabase>(sp =>

    sp.GetRequiredService<IMongoClient>().GetDatabase("test"));

builder.Services.AddSingleton<IMongoCollection<Order>>(sp =>

    sp.GetRequiredService<IMongoDatabase>().GetCollection<Order>("orders"));

var app = builder.Build();

app.MapPost("/orders", async (

    IMongoCollection<Order> orders,

    Order order,

    CancellationToken cancellationToken) =>

{

    await orders.InsertOneAsync(order, cancellationToken: cancellationToken);

    return TypedResults.Ok();

});

app.MapGet("/orders", (IMongoCollection<Order> orders) =>

{

    async IAsyncEnumerable<SseItem<Order>> StreamOrders(

        [EnumeratorCancellation] CancellationToken cancellationToken)

    {

        var cursor = await orders.WatchAsync(

            new EmptyPipelineDefinition<ChangeStreamDocument<Order>>()

                .Match(x =>

                    x.OperationType == ChangeStreamOperationType.Insert),

            new ChangeStreamOptions

            {

                FullDocument = ChangeStreamFullDocumentOption.UpdateLookup

            },

            cancellationToken);

        while (await cursor.MoveNextAsync(cancellationToken))

        {

            foreach (var change in cursor.Current)

            {

                if (change.FullDocument is not null)

                {

                    yield return new SseItem<Order>(change.FullDocument, "order")

                    {

                        EventId = EncodeResumeToken(change.ResumeToken)

                    };

                }

            }

        }

    }

    return TypedResults.ServerSentEvents(StreamOrders());

});

app.UseStaticFiles();

app.Run();

static string EncodeResumeToken(BsonDocument resumeToken)

{

    return Convert.ToBase64String(resumeToken.ToBson());

}

static BsonDocument DecodeResumeToken(string eventId)

{

    var bytes = Convert.FromBase64String(eventId);

    return BsonSerializer.Deserialize<BsonDocument>(bytes);

}

record Order(

    [property: BsonRepresentation(BsonType.ObjectId)] string? Id,

    string Name,

    DateTime Date);

There are three important ideas in this code:

We write new orders using a normal HTTP POST endpoint.
We open a MongoDB change stream and yield each incoming change.
We expose that stream to browsers as SSE on the same route (GET /orders).

Notice that both write and streaming code paths use cancellation tokens so work can stop quickly if the request is aborted or the app is shutting down.

2.1) Continue a stream with ResumeAfter and SSE EventId

One really useful feature of MongoDB Change Streams is that you can continue from a known point in the stream instead of always starting "from now".

In .NET, SseItem<T> gives us an EventId field. We can put the MongoDB resume token in there and set the event type to order.

When the browser reconnects, EventSource automatically sends the last event id in the Last-Event-ID header. We can decode that value and pass it to ChangeStreamOptions.ResumeAfter.

In practice, this means:

Capture the change.ResumeToken for each event you process.
Encode it and set SseItem.EventId.
Read Last-Event-ID on reconnect and map it to ResumeAfter.

Here is a minimal version:

app.MapGet("/orders", (

    IMongoCollection<Order> orders,

    HttpRequest request,

    CancellationToken cancellationToken) =>

{

    async IAsyncEnumerable<SseItem<Order>> StreamOrders(

        [EnumeratorCancellation] CancellationToken streamCancellationToken)

    {

        var lastEventId = request.Headers["Last-Event-ID"].ToString();

        var options = new ChangeStreamOptions

        {

            FullDocument = ChangeStreamFullDocumentOption.UpdateLookup,

            ResumeAfter = string.IsNullOrWhiteSpace(lastEventId)

                ? null

                : DecodeResumeToken(lastEventId)

        };

        var cursor = await orders.WatchAsync(

            new EmptyPipelineDefinition<ChangeStreamDocument<Order>>()

                .Match(x =>

                    x.OperationType == ChangeStreamOperationType.Insert),

            options,

            streamCancellationToken);

        while (await cursor.MoveNextAsync(streamCancellationToken))

        {

            foreach (var change in cursor.Current)

            {

                if (change.FullDocument is not null)

                {

                    yield return new SseItem<Order>(change.FullDocument)

                    {

                        EventId = EncodeResumeToken(change.ResumeToken),

                        EventType = "order"

                    };

                }

            }

        }

    }

    return TypedResults.ServerSentEvents(StreamOrders(cancellationToken));

});

The key point is that ResumeAfter is not an arbitrary ID; it is the opaque token generated by MongoDB for each change event.

If you are using browsers, Last-Event-ID gives you a clean reconnect story. For non-browser consumers, you can still persist the token yourself and pass it back the same way.

3) Add a simple browser client

Createwwwroot/index.html and add:

<!DOCTYPE html>

<html lang="en">

<head>

    <meta charset="UTF-8">

    <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <title>Orders</title>

</head>

<body>

<h1>Orders</h1>

<div id="status">Connecting...</div>

<form id="orderForm">

    <input type="text" id="name" placeholder="Order name" required />

    <input type="datetime-local" id="date" required />

    <button type="submit">Create Order</button>

</form>

<ul id="orders"></ul>

<script>

    const ordersList = document.getElementById("orders");

    const status = document.getElementById("status");

    function addOrder(order) {

        const li = document.createElement("li");

        li.innerHTML = `<strong>${order.name}</strong><br>${new Date(order.date).toLocaleString()}`;

        ordersList.prepend(li);

    }

    const eventSource = new EventSource("/orders");

    eventSource.onopen = () => {

        status.textContent = "Connected";

    };

    eventSource.onerror = () => {

        status.textContent = "Disconnected";

    };

    eventSource.addEventListener("order", event => {

        addOrder(JSON.parse(event.data));

    });

    document.getElementById("orderForm").addEventListener("submit", async e => {

        e.preventDefault();

        const name = document.getElementById("name").value;

        const date = document.getElementById("date").value;

        const response = await fetch("/orders", {

            method: "POST",

            headers: { "Content-Type": "application/json" },

            body: JSON.stringify({ name, date })

        });

        if (!response.ok) {

            alert("Failed to create order");

        }

    });

</script>

</body>

</html>

4) Run it

dotnet run

Open the app, create a few orders, and you should see each change streamed into the list in real time.

SSE vs SignalR: where each fits

SSE is excellent when you want a lightweight one-way push channel from server to browser. It is easy to reason about, browser-native, and usually simpler to operate than a full duplex setup.

The trade-off is that SSE is intentionally limited. It is text-based, one-way, and less ergonomic when your app grows into richer real-time interactions like rooms, server calls from clients over persistent connections, or advanced client coordination.

This is where SignalR starts to earn its place. SignalR gives you a higher-level real-time programming model, automatic transport negotiation (WebSockets, SSE, Long Polling), groups, hubs, and a mature ecosystem for scaling out.

Topic	SSE	SignalR	Practical guidance
Direction	Server -> client	Server <-> client	If your UI mostly listens, SSE is often enough
Client support	Native EventSource	SignalR client library	If you want richer interaction patterns, SignalR is easier
Payload style	Text/event-stream	Structured hub messages	If you need binary or richer protocols, lean SignalR
Infrastructure	Simple HTTP streaming	More moving parts, more capability	Choose based on complexity, not fashion
Scale-out patterns	Manual design choices	Well-known backplane patterns	SignalR is usually easier at larger scale

A practical way to choose is:

Start with SSE when you need fast, simple, server-to-browser updates.
Move to SignalR when you need bi-directional messaging or richer collaboration semantics.

Also keep in mind one MongoDB-specific caveat regardless of transport choice: Change Streams require a replica set or sharded cluster. A standalone MongoDB server will not work.

Production hardening ideas

If you take this pattern to production, consider:

project events into lightweight DTOs rather than streaming full documents
include cancellation support and proper disposal around streaming cursors
add structured logging for stream open/close/reconnect behavior
filter change streams as early as possible to reduce noise
protect endpoints with auth and authorize at data scope
keep an eye on proxy settings for idle connection timeouts

Moving towards SignalR

SSE is a really strong starting point. But you should move to SignalR when you need more than one-way feeds.

SignalR helps when you need:

true two-way communication
protocol negotiation (WebSockets with fallbacks)
groups and richer hub-based interaction patterns
scale-out patterns that are already well trodden in ASP.NET Core ecosystems

A natural progression can look like this:

Start with SSE + MongoDB Change Streams for straightforward live updates.
Grow into SignalR when clients need to send real-time messages back over the same channel.
Add a backplane when you scale out across multiple app instances.

If you want to continue using MongoDB in that architecture, you can use Kevsoft.AspNetCore.SignalR.MongoDB as a SignalR backplane.

That gives you the flexibility of SignalR transports (WebSockets, SSE, Long Polling) while still leveraging MongoDB as part of your real-time pipeline.

Final thoughts

For many apps, the pairing of MongoDB Change Streams and SSE is a sweet spot:

minimal moving parts
small amount of code
fast path to real-time UX

Then when requirements evolve, SignalR is a natural next step rather than a rewrite from scratch.

Start simple, get value quickly, and step up transport complexity only when the product actually needs it.

Bringing MongoDB Atlas and Voyage AI to Dify: Build RAG Workflows and Data Agents Without Heavy Glue Code

Pash10g — Sun, 31 May 2026 06:24:29 +0000

AI applications are moving quickly from simple chatbots to systems that can search, reason, recommend, summarize, and act on live business data. For developers, that usually means wiring together databases, embedding models, vector search, rerankers, orchestration logic, and application code. For no-code AI builders, it often means waiting for those integrations to exist before an idea can become a working prototype.

The MongoDB extensions for Dify help close that gap.

With the new MongoDB Atlas and Voyage AI extensions, Dify builders can visually compose AI workflows and agents that connect directly to MongoDB data, perform semantic retrieval with Atlas Vector Search, improve result quality with Voyage AI embeddings and reranking, and optionally interact with operational documents through controlled database tools.

The result is a practical path from idea to working AI application: less custom orchestration code, more reusable building blocks, and a smoother experience for both developers and no-code builders.

Why Dify and MongoDB Belong Together

Dify provides a visual environment for building AI apps, workflows, and agents. It makes it easy to connect user input, model calls, tools, prompts, and outputs into a working application. MongoDB Atlas provides the data foundation: flexible documents, operational queries, aggregation, full-text search, and vector search in one platform.

Together, they create a powerful pattern:

Dify orchestrates the AI experience — workflows, agents, prompts, tools, and user interactions.
MongoDB Atlas stores and retrieves the data — documents, application records, knowledge sources, and vector embeddings.
Voyage AI improves retrieval quality — embeddings for semantic search and reranking for precision.

For a no-code builder, this means you can assemble a retrieval-augmented generation workflow visually. For a developer, it means the integration points are packaged as reusable Dify tools rather than one-off glue code.

Meet the Extensions

The extension set includes two complementary pieces.

MongoDB Atlas Tool Extension

The MongoDB Atlas tool extension exposes MongoDB operations as Dify tools. These tools let workflows and agents interact with MongoDB collections directly from the Dify canvas.

Available capabilities include:

Finding documents
Running aggregation pipelines
Performing Atlas Vector Search
Performing full-text search
Inserting documents
Updating documents
Deleting documents

This is useful for more than just retrieval. It enables agents that can inspect data, summarize records, recommend actions, and — when safely configured — update operational collections.

For example, a project management agent can search a database of team members, skills, previous projects, and availability, then recommend the best team for a new initiative. With carefully scoped permissions, that same agent could also update a draft team assignment or write a recommendation record back to MongoDB.

Voyage AI Extension

The Voyage AI extension adds embedding and reranking tools to Dify.

Embeddings convert text into vectors so MongoDB Atlas Vector Search can find semantically similar documents. Reranking takes an initial set of retrieved documents and reorders them by relevance to the user’s query.

That two-step retrieval pattern matters. Vector search is excellent for finding likely candidates quickly, while reranking helps surface the best candidates before the final answer is generated or returned.

The MongoDB-RAG Template

The included MongoDB RAG template demonstrates how these extensions work together in a Dify workflow.

At a high level, the pipeline does the following:

Accepts user input
Embeds the query with Voyage AI
Searches MongoDB Atlas using Atlas Vector Search
Reranks the retrieved documents with Voyage AI
Formats the results into a prompt-ready output

This is the core pattern behind many production-grade RAG systems.

Instead of sending a user question directly to an LLM and hoping the model already knows the answer, the workflow first retrieves relevant information from MongoDB. The retrieved context can then be used by a downstream answer node, chat model, or agent to produce a more grounded response.

How the Workflow Works

The MongoDB RAG workflow is intentionally simple and reusable. It separates each retrieval step into a dedicated node so builders can understand, tune, and replace parts of the pipeline as needed.

1. User Input

The workflow starts with a text input. This could be a question, a search phrase, a support request, a project description, or any natural-language query.

Example:

What would be a good team to build scalable Rust applications?

2. Embed the Query

The input is sent to the Voyage AI embedding tool. The embedding model converts the text into a vector representation that captures semantic meaning.

For search use cases, the embedding input type should be optimized for queries. This helps improve retrieval quality because the model understands that the text represents a search intent rather than a document to be indexed.

3. Search MongoDB Atlas

The generated query vector is passed to the MongoDB Atlas Vector Search tool. Atlas compares the query vector against document embeddings stored in a MongoDB collection and returns the nearest semantic matches.

The template uses two important retrieval settings:

numCandidates: how many approximate nearest-neighbor candidates Atlas considers before returning final results.
limit: how many results are passed forward to the next step.

Increasing candidates can improve recall, while lowering them can reduce latency. This gives builders and developers a clear tuning knob depending on the application’s needs.

4. Rerank the Results

The top vector search results are then sent to the Voyage AI reranking tool. Reranking compares the original user query against each candidate document and sorts the documents by relevance.

This step is especially valuable when the first-stage vector search returns many plausible matches. Reranking helps the workflow prioritize the documents most likely to answer the user’s actual question.

5. Format the Output

Finally, the template node formats the reranked documents into a structured output. That output can be returned directly, or it can become context for a downstream LLM answer node.

This makes the template flexible. It can be used as a standalone search pipeline, or as the retrieval layer inside a larger Dify chatbot, workflow, or agent.

What No-Code AI Builders Can Create

For no-code builders, the biggest advantage is composability. Instead of implementing a RAG backend from scratch, you can drag tools into a Dify workflow and connect them visually.

With these extensions, builders can create:

Knowledge-base assistants that answer questions from MongoDB documents
Support copilots that search prior cases and recommend resolutions
Project management agents that recommend teams based on skills and history
Document search apps that combine semantic and full-text retrieval
CRM or account assistants that retrieve relevant customer information
Operations agents that read from MongoDB and create structured recommendations

The same building blocks can support simple workflows or more autonomous agents. A workflow might only retrieve and format context. An agent might decide when to search, when to aggregate, and when to update a document — depending on the tools you enable.

What Developers Get

Developers still benefit from the visual experience, but the value goes deeper.

These extensions reduce the amount of custom integration code required to connect Dify with MongoDB Atlas and Voyage AI. Instead of hand-building every request, response parser, embedding call, and database operation, developers can rely on packaged tools with clear inputs and outputs.

The architecture also follows a clean separation of concerns:

Embedding is handled by the Voyage AI embed tool.
Retrieval is handled by MongoDB Atlas Vector Search.
Precision tuning is handled by the Voyage AI rerank tool.
Formatting is handled by the Dify template node.
Application behavior is handled by Dify workflows or agents.

That separation makes the system easier to debug and extend. Developers can tune vector search without changing reranking. They can swap embedding models without rewriting MongoDB logic. They can add an LLM answer node without changing the retrieval pipeline.

Example: A Project Management Agent

One example use case is a project management agent that recommends a team for a new project.

A user might ask:

What would be a good team to build scalable Rust applications?

The agent can use semantic search to find relevant candidates, previous projects, skills, and experience stored in MongoDB. It can then assemble a recommendation that explains why each person fits the project.

In a Dify agent setup, MongoDB tools can be made available alongside the RAG workflow. The agent can search documents, inspect structured records, run aggregations, and produce a recommendation grounded in database results.

This pattern is useful because business data is rarely just static documentation. It often includes operational records: people, cases, accounts, tickets, projects, tasks, products, and events. MongoDB allows that data to remain flexible and queryable, while Dify makes it accessible to AI workflows and agents.

Best Practices for Building with These Extensions

To get the best results, keep a few practical guidelines in mind.

Use the Right Embedding Mode

When embedding user questions for retrieval, use query-optimized embeddings. When embedding documents for storage, use document-optimized embeddings if the model supports it. This improves the alignment between search queries and indexed content.

Tune Vector Search for Recall and Latency

Atlas Vector Search settings such as numCandidates and limit affect both result quality and performance. A larger candidate pool can improve recall, but may increase latency. Start with sensible defaults, then tune based on your dataset and user experience goals.

Rerank Before Generating

Reranking helps improve the quality of the context that reaches the final model. This can reduce irrelevant context, improve answer accuracy, and make the final output easier to trust.

Scope Write Tools Carefully

MongoDB insert, update, and delete tools are powerful. When exposing them to agents, use careful scoping, clear instructions, and appropriate permissions. Many applications should start with read-only tools, then add mutation capabilities only when the workflow and safety boundaries are well understood.

Keep Indexes Aligned with Your Data

For vector search, the Atlas index should match the embedding field and embedding dimensions used by your model. For full-text search, index the fields users are likely to search. Good indexing turns a promising prototype into a responsive application.

Why This Matters

The value of these extensions is not just that Dify can call MongoDB or Voyage AI. The value is that builders can now compose a complete AI retrieval and data-interaction pattern inside Dify:

Search semantically across MongoDB documents
Rerank results for precision
Feed grounded context into an LLM
Let agents inspect and operate on database records
Reuse the same tools across multiple apps and workflows

For no-code builders, this means faster experimentation and fewer blockers. For developers, it means a cleaner integration surface and less repetitive orchestration work.

Conclusion

The MongoDB Atlas and Voyage AI extensions make Dify a stronger platform for building data-aware AI applications. They bring together visual AI orchestration, operational MongoDB data, Atlas Vector Search, full-text search, embeddings, reranking, and agent tools in a way that is approachable for no-code builders and credible for developers.

The template shows the foundation: embed a query, retrieve relevant documents from MongoDB Atlas, rerank them, and format the result. From there, teams can build knowledge assistants, recommendation agents, support copilots, document search experiences, and operational AI workflows.

In short: Dify becomes the place where AI behavior is designed, and MongoDB Atlas becomes the data layer that keeps those AI experiences grounded in real, useful information.

Building a RAG Knowledge Base on MongoDB Atlas

MongoDB Guests — Thu, 28 May 2026 17:28:09 +0000

This tutorial was written by Samuel Molling.

Composition, retrieval fidelity, and the failure modes between $vectorSearch and a cited answer.

The hard part of a production RAG system is not running a single $vectorSearch. It is composing keyword retrieval, vector retrieval, ranking fusion, reranking, and a generative model into a pipeline that returns the right passage, with the right confidence, every time, under concurrency, across tenants, and with predictable latency.

In MongoDB Atlas, this composition can be done end-to-end inside a single cluster. Operational data, full-text indexing, vector indexing, multi-tenant filtering, and the retrieval that feeds an LLM are not separate stores. They are stages of the same aggregation framework.

This text walks through the architectural decisions behind a complete RAG knowledge base built on Atlas Search and Atlas Vector Search, examining where each stage contributes, where each stage fails silently, and what it takes to keep the pipeline honest end-to-end.

Search interface with a streaming AI answer, inline citations, and the source passages that grounded the response.

Composition, not capability

A knowledge base in production exposes a single search box. Behind that box, there are five distinct stages of execution. Treating them as a single primitive (“the LLM answers questions about our docs”) is the most common reason these systems return confidently wrong answers.

The actual pipeline looks like this:

The first four stages are MongoDB-native. The fifth is the only component outside Atlas, and it consumes context retrieved entirely from MongoDB. Each transition between stages has a cost and a contract. Understanding those contracts is what separates a demo from a production system.

The remainder of this article examines those contracts: data layout, index design, ranking, the role of reranking as a truth signal, and the failure mode that vector search shares with every nearest-neighbor algorithm: the one where it never tells you it found nothing.

Data layout: chunks as their own collection

The first non-trivial decision in any RAG system on MongoDB is whether to store chunks as an embedded array inside the parent article, or as documents in their own collection.

Embedded arrays look attractive. They preserve locality, the document hierarchy mirrors the application model, and a single findOne returns everything. They are also the wrong choice for non-trivial corpora.

Three reasons argue for a separate chunks collection.

First, retrieval granularity. $vectorSearch returns the documents it scored, not their parents. With chunks embedded inside articles, a vector hit identifies the article but not the passage. The user receives a long document containing the answer somewhere; the system has lost the ability to point at the exact section. With chunks as documents, each hit carries its own _id, its position in the article (chunkIndex), and the precise text that scored.

Second, re-indexing cost. Updating one article when chunks are embedded means rewriting the entire parent document, including every untouched chunk and its embedding. With chunks in their own collection, re-indexing is a deleteMany({ articleId }) followed by an insertMany of the new chunks. The articles collection remains compact and serves the operational queries of the application (list, filter, render) without dragging embedding payloads through every read.

Third, and most consequentially, pre-filter at index time. The vector search index can pre-filter candidates inside $vectorSearch itself, before any document is hydrated, but only on top-level fields of the indexed collection. When chunks live as documents, denormalizing workspaceId, visibility, tags, and category onto each chunk turns multi-tenancy from a downstream $match into an index-level prune. The HNSW traversal walks only the candidate set that the tenant is allowed to see. This is the difference between a system that scales linearly with the corpus and one that scales with the per-tenant slice.

chunks: {
  _id:         ObjectId,
  articleId:   ObjectId,
  workspaceId: string,
  chunkIndex:  Number,
  text:        String,
  embedding:   [Number],   // 1024 floats from voyage-3
  tags:        [String],
  category:    String,
  visibility:  String,
}

One detail of the denormalization is worth calling out. workspaceId is stored on chunks as a string, not as an ObjectId, because Atlas Vector Search filter fields do not compare ObjectId values directly. The chunk contains the hex representation of the parent workspace's _id, which is what the $vectorSearch filter clause compares against. Articles that are not filtered through a vector index continue to use the native ObjectId type.

Index design: static mappings and filter fields

Two Atlas indexes handle retrieval, one per collection. Both use static mappings. Dynamic mappings index every field of every document, including fields that never appear in queries, which inflates the index and slows every query. Static mappings force every indexed field to justify its existence.

The full-text index on articles declares language analyzers per field:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "title":       { "type": "string", "analyzer": "lucene.english" },
      "content":     { "type": "string", "analyzer": "lucene.english" },
      "summary":     { "type": "string", "analyzer": "lucene.english" },
      "tags":        { "type": "string", "analyzer": "lucene.keyword" },
      "category":    { "type": "string", "analyzer": "lucene.keyword" },
      "workspaceId": { "type": "objectId" }
    }
  }
}

title, content, and summary use the English analyzer (lowercasing, stemming, stopword removal) so that natural-language queries match natural-language text. tags and category use the keyword analyzer for exact-match behavior without lexical transformation. workspaceId is indexed as an ObjectId, so tenant scoping is exact.

The vector index on chunks declares the embedding geometry and the fields eligible for pre-filtering inside $vectorSearch:

{
  "fields": [
    { "type": "vector", "path": "embedding",
      "numDimensions": 1024, "similarity": "cosine" },
    { "type": "filter", "path": "workspaceId" },
    { "type": "filter", "path": "visibility" },
    { "type": "filter", "path": "tags" },
    { "type": "filter", "path": "category" }
  ]
}

numDimensions: 1024 matches the output dimensionality of the chosen embedding model. Cosine similarity is the standard metric for normalized text embeddings. The four filter declarations are not metadata. They are the difference between a vector index that scopes to a single workspace inside HNSW traversal and one that scans the entire corpus before filtering downstream. At any non-trivial corpus size, the first behaves and the second does not.

Section-aware chunking and breadcrumb context

Chunking decides retrieval quality before any embedding model is involved. A 1024-dimensional vector is a single point representing an entire chunk of text. If that text contains three unrelated paragraphs, the resulting embedding sits in the centroid of three semantic regions and is meaningfully close to none of them.

Fixed-window chunkers (“every 800 characters with 120 character overlap”) produce exactly this problem. They cut mid-sentence, mid-section, and mid-topic, and the embedding is a faithful representation of the resulting incoherence.

Section-aware chunking solves this at the document level. ATX headings (#, ##, ###) demarcate semantic units that the author has already labeled. Splitting along those boundaries produces chunks that map to a single logical idea each, and the embedding represents that idea with no contamination from neighboring sections.

Two additional properties matter at scale.

The breadcrumb path is prepended to every chunk of text:

Designing a chunking strategy for RAG > Semantic chunking by section  
The most precise approach for structured documents is to chunk by heading: every ## or ### section becomes one chunk…

The embedding now encodes hierarchical context, not just local content. When the user asks about a specific topic, the model sees the chunk’s position in the document graph and weights it accordingly. The reranker sees the same context, and the LLM, in the final stage, receives a passage with a self-identifying header, which materially improves citation quality.

The fallback for oversized sections is unconditional. A section larger than the maximum chunk size (1500 characters in this implementation) falls back to paragraph splitting, then to sentence splitting, while the breadcrumb is preserved on every sub-chunk. The system never produces a chunk without context.

Fenced code blocks are excluded from heading detection. A # comment inside a bash block is content, not structure, and treating it as a heading would cause chunks to break unpredictably.

Hybrid retrieval and the score-scale problem

Each retrieval stage is a single aggregation pipeline. The keyword stage ranks articles by BM25, with fuzzy tolerance and per-field boosts:

db.articles.aggregate([
  {
    $search: {
      index: "articles_search",
      compound: {
        should: [
          { text:  { query, path: "title", fuzzy: { maxEdits: 1 } },
            score: { boost: { value: 3 } } },
          { text:  { query, path: "content", fuzzy: { maxEdits: 1 } } },
          { text:  { query, path: "tags" },
            score: { boost: { value: 2 } } },
        ],
      },
    },
  },
  { $match: { workspaceId: ObjectId("...") } },
  { $limit: 30 },
  { $project: { _id: 1, score: { $meta: "searchScore" } } },
])

The semantic stage runs in parallel, on the chunks collection, with the workspace filter applied inside the index:

db.chunks.aggregate([
  {
    $vectorSearch: {
      index: "chunks_vector",
      path: "embedding",
      queryVector: [/* 1024 floats from Voyage */],
      numCandidates: 200,
      limit: 50,
      filter: { workspaceId: "65f1b…" }   
    },
  },
  { $project: {
      articleId: 1, chunkIndex: 1, text: 1,
      score: { $meta: "vectorSearchScore" } } },
])

Both stages produce ranked result lists, but on incomparable scales.

Atlas Search returns BM25-style scores that depend on term frequency, inverse document frequency, and document length. Scores routinely climb into the tens and can spike even higher when a document is short, and the queried term is densely used.

Atlas Vector Search returns cosine similarities bounded between 0 and 1, with realistic values for relevant matches typically clustering in 0.6 to 0.9.

Adding these scores, or normalizing them to a common scale, is fragile. A handful of documents with anomalously high BM25 scores will dominate the fused ranking regardless of their actual relevance. Linear weighting with hand-tuned coefficients works for one corpus and fails on the next.

Reciprocal Rank Fusion sidesteps the problem by discarding scores entirely and working only on rank position:

rrf_score(doc) = Σ over rankers   1 / (k + rank_i(doc))     // k = 60

A document at rank 1 contributes 1/61 per ranker, rank 2 contributes 1/62, and so on. The scale collapses. Documents that appear near the top of multiple rankers accumulate score; documents that appear near the top of only one ranker also score, but less. The fusion is parameter-free in practice: k = 60 is the value from the original 2009 paper and remains the production default.

In this implementation, fusion is anchored on chunks. Semantic search provides the candidate list; keyword search acts as a boost when a chunk’s parent article also matches the keyword. This asymmetry is intentional: expanding every keyword-matched article into all of its chunks would pollute the candidate pool with off-topic content from articles that matched on a single tangential term.

const fused = semanticChunks
  .map((chunk, idx) => {
    let score = 1 / (k + idx + 1);                              // semantic rank
    const kwRank = keywordArticleRank.get(chunk.articleId.toString());
    if (kwRank !== undefined) score += 1 / (k + kwRank + 1);    // keyword boost
    return { ...chunk, rrfScore: score };
  })
  .sort((a, b) => b.rrfScore - a.rrfScore)
  .slice(0, 20);

MongoDB 8.1 introduced $rankFusion as a first-class aggregation stage that performs RRF natively. Production code should prefer it; the explicit implementation above exists to make the algorithm visible.

Reranking as the truth signal

After fusion, the system has a ranked list of twenty chunks, but no calibration of how good that ranking actually is. A list of twenty chunks always exists, even when none of them is relevant.

Reranking solves two problems simultaneously. First, it improves precision: the top of the fused list is reordered by a cross-encoder model that scores each (query, passage) pair directly, which is materially more accurate than approximate nearest-neighbor distance for the small set of candidates that matters. Second, and more critically, the reranker’s score becomes a calibrated relevance signal that the rest of the system can interpret.

Voyage’s rerank-2 model, exposed through the MongoDB-managed gateway at ai.mongodb.com/v1, returns scores in the [0, 1] range. Empirically, on a well-formed corpus:

A top score above 0.5 indicates strong relevance: the passage genuinely contains the answer.

A top score between 0.3 and 0.5 indicates partial relevance: adjacent topics, paraphrases, weak matches.

A top score below 0.3 indicates the corpus does not contain an answer to the question.

This calibration is what makes the downstream stages safe. The LLM receives passages with known quality. The UI can surface a low-confidence banner. The application can decide at query time whether to attempt a generated answer.

The reranker is more expensive per pair than the embedding model, which is why it runs only on the shortlist produced by fusion. The cost is linear in the number of candidates reranked, and reranking twenty candidates is small even at high concurrency.

The generative stage as a constrained consumer

The LLM does not “answer questions about the knowledge base”. It receives a prompt containing retrieved passages and is instructed, by system message, to use only those passages.

The contract is explicit, and the model honors it:

SYSTEM:
You are a helpful assistant answering questions using ONLY the passages
provided below. Cite the passages you used as bracketed numbers like [1]
or [2]. If the passages do not contain enough information to answer, say
so honestly. Reply in the same language as the user's question.
USER:
Passages:
[1] Designing a chunking strategy for RAG > Semantic chunking by section
    The most precise approach for structured docs is to chunk by heading...
[2] Designing a chunking strategy for RAG > Reasonable defaults
    A starting point is 500-1500 characters per chunk...
[3] Getting started with MongoDB Atlas Vector Search > The chunking pattern
    A common pattern is to chunk long documents, embed each chunk...
Question: How do I chunk by markdown section?

What returns is the text that the model generated by reading those passages. The citation discipline ([1], [2]) is enforced by the prompt, parsed by the frontend, and rendered as in-text links that scroll to the source card. The frontend never displays an unreferenced answer alongside hidden sources, and the model never sees passages it isn’t allowed to cite.

Streaming changes the user experience but not the architecture. The response is delivered as newline-delimited JSON, with three event types:

{"event":"passages","passages":[...]}     // emitted once, before the LLM call
{"event":"token","text":"..."}            // emitted per token
{"event":"done","model":"gpt-4o-mini"}    // emitted at the end

Passages are sent first, before the LLM call begins. The frontend renders source cards immediately and updates the answer panel as tokens arrive. From the user’s perspective, the system is already useful before the model has finished generating.

The AI answer card with inline numbered citations linking to the source passages retrieved from MongoDB.

The failure mode: $vectorSearch never says “no”

Nearest-neighbor search is by construction non-empty. Given a query vector, the HNSW graph returns the N closest vectors regardless of whether those vectors are actually relevant to the query. A query about the capital of a country, posed to a knowledge base about MongoDB, will receive five chunks ranked by cosine distance: chunks that share no semantic content with the question.

This is not a bug. It is the contract of kNN. The pipeline must compensate at the application layer.

Low-confidence banner shown when the top reranker score falls below the relevance threshold; no sources are surfaced.

Three layered defenses keep the system honest.

The system prompt instructs the LLM to refuse questions whose passages do not contain the answer. The model honors this when scores are uniformly low; it produces “the passages provided do not contain information about that” rather than hallucinating an answer from training data.

The rerank score is the truth signal. When the top reranker score falls below the strong-relevance threshold, the system knows the corpus is not coherent with the question. This is the only stage in the pipeline that produces calibrated relevance. Vector distance and BM25 scores cannot be interpreted in isolation, but rerank scores can.

The UI threshold makes the calibration visible. Below a confidence threshold (empirically 0.4 for the top rerank), the interface displays a banner indicating that the knowledge base does not cover the topic, and hides individual passages whose scores fall below a stricter threshold (empirically 0.3). The user does not see five low-quality “sources” that contradict the LLM’s refusal.

const STRONG_RELEVANCE = 0.4;
const WEAK_PASSAGE     = 0.3;
const confidence = passages[0].rerankScore >= STRONG_RELEVANCE ? "strong" : "weak";
const s

Each defense alone is insufficient. The prompt can fail when scores happen to land in the partial-relevance band. The threshold can hide useful passages in rare edge cases. The UI banner alone cannot prevent a confident hallucination. Together, they make confidently wrong answers very unlikely.

Conclusion

A production RAG system is the composition of five well-understood stages, each with its own contract and its own failure modes. The interesting engineering does not live in any single stage. It lives at the boundaries between them: in the data model that makes pre-filtering possible, in the chunking that makes embeddings coherent, in the fusion that reconciles incomparable scores, in the rerank that produces a calibrated truth signal, and in the prompt that translates that signal into a refusal when the corpus does not have the answer.

MongoDB Atlas is the substrate that lets these stages compose without crossing infrastructure boundaries. The operational data, the search indexes, and the vector indexes are not federated systems with their own consistency models. They are aggregation stages in the same query language, against the same documents, with the same access control. The cost saved is not raw latency, which is a property of every well-engineered system; it is the entire class of consistency bugs that arises when a document, a search index, and a vector store have to agree about the same change.

RAG is not a prompt problem. It is a systems engineering problem. When the boundaries are honest, the answers are honest.