DEV Community: Alex Bevilacqua

Start With Context: Building the Retrieval Core for Agentic Apps

Alex Bevilacqua — Wed, 29 Apr 2026 13:47:44 +0000

Before you add planners, crews, or graph-shaped orchestration, build the part that decides what the model should actually see. In this first post, we’ll start an enterprise support copilot and give it the one capability every future agent depends on: retrieval that doesn’t fall apart in production.

In a recent post I made the case that MongoDB can serve as the "brain" of a modern AI application by combining durable state, retrieval, and application data in one place. That framing still holds, but brains are only useful if they can recall the right thing at the right time. I wanted to dig into agentic application development in more detail in a series of posts, so for the first real entry in this series, I want to start one layer below "agents" and one layer above raw storage: the context layer.

That might sound slightly less glamorous than "multi-agent orchestration," which is exactly why it matters. Most enterprise AI systems do not fail because they lack a clever planner. They fail because the model sees the wrong document, too much irrelevant text, or none of the operational data that actually matters.

To make this concrete, the application thread for this series will be an enterprise support escalation copilot for a B2B SaaS team. By the end of the series, it should be able to answer questions about incidents, remember previous escalations, pull account context, and coordinate specialized agents when needed. Today, though, we’re giving it its first useful skill: finding the right context for the job.

Think about the kind of question a real support engineer asks:

"Acme’s enterprise tenant started seeing INV-4421 after upgrading to 3.8. Did we see this before, is there a known workaround, and does it affect EU clusters only?"

That is not a pure semantic search problem. It is part natural language, part exact identifier lookup, part metadata filtering, and part ranking problem. Error codes matter. Version numbers matter. Tenant boundaries matter. Timing matters. That’s why this is such a good place to start - and to solve this problem we'll dig in with MongoDB and Voyage AI.

MongoDB Vector Search is built to search vector data alongside the rest of your operational data, supports filtering on other fields in the collection, and can be combined with full-text search for hybrid retrieval. MongoDB’s hybrid search documentation explicitly describes combining semantic and full-text search results with Reciprocal Rank Fusion, which is exactly what you want when a query mixes fuzzy intent with exact strings like issue IDs, SKUs, or feature flags.

On the retrieval-model side, Voyage provides high-accuracy embedding and reranking models, including newer capabilities like contextualized chunk embeddings, multimodal embeddings, and rerankers designed to refine the top candidate set after initial retrieval. MongoDB Atlas now also exposes Voyage models through its Embedding and Reranking API, currently in preview, which means you can either call Voyage models directly or keep retrieval models, vector search, and operational data closer together under Atlas.

So what does the retrieval core for our support copilot actually do?

First, it stores source material in MongoDB: runbooks, release notes, KB articles, previous incident reviews, ticket summaries, and whatever structured account data the support flow needs. Then it chunks the long-form content, embeds it with Voyage, and stores the vectors with the source text and metadata. At query time, it narrows scope using metadata like tenant, product, region, or severity; retrieves candidates semantically and lexically; reranks the best matches; and only then hands a compact, relevant context window to the LLM. In other words: don’t ask the model to be psychic when the database can be specific.

There are a lot of AI frameworks right now, and they absolutely do not all feel the same. But this is the first important pattern in the series: the framework should shape the developer experience, not force you to redesign the data layer every six months. The retrieval architecture is the stable part. MongoDB and Voyage AI are the stable parts. LangChain, LlamaIndex, Haystack, LangGraph, CrewAI, or whatever comes next should be able to sit on top of that foundation.

A framework-agnostic mental model

Before jumping into code, here is the mental model I’d keep fixed no matter which framework you prefer:

Put source documents and operational records in MongoDB.
Generate embeddings with Voyage.
Index vector fields and filter fields in MongoDB.
Use semantic retrieval for meaning.
Use full-text retrieval for exact strings.
Rerank the candidate set before generation.
Return only the context the model actually needs.

That shape maps cleanly to both MongoDB Vector Search and Voyage’s model stack. MongoDB handles vector indexes, full-text search, filterable metadata, and live application data; Voyage handles embeddings and reranking; the framework becomes the control surface.

Approach 1: LangChain for the shortest path from data to grounded answers

If the goal is to get a retrieval-backed application running quickly, LangChain remains a very practical starting point. MongoDB’s LangChain integration supports vector search, full-text search, and a hybrid retriever that combines both with Reciprocal Rank Fusion. It also supports pre-filtering with MQL expressions, which matters immediately for tenant scoping and product boundaries.

An illustrative version for our support copilot looks like this:

import os

from langchain_voyageai import VoyageAIEmbeddings
from langchain_mongodb.vectorstores import MongoDBAtlasVectorSearch
from langchain_mongodb.retrievers.hybrid_search import MongoDBAtlasHybridSearchRetriever

embeddings = VoyageAIEmbeddings(
    api_key=os.environ["VOYAGE_API_KEY"],
    model="voyage-4",
)

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
    connection_string=os.environ["MONGODB_URI"],
    namespace="support.context",
    embedding=embeddings,
    index_name="support_vector_index",
)

retriever = MongoDBAtlasHybridSearchRetriever(
    vectorstore=vector_store,
    search_index_name="support_search_index",
    k=8,
    fulltext_penalty=60.0,
    vector_penalty=60.0,
)

docs = retriever.invoke(
    "Acme tenant seeing INV-4421 after upgrading to 3.8"
)

In a production version, I’d pair that with metadata filters on fields like tenant_id, product, region, and severity, then pass the top candidates through a Voyage reranker before generation. The point is not that LangChain is magical. The point is that the MongoDB + Voyage retrieval story already fits the way LangChain applications are commonly assembled.

Approach 2: LlamaIndex when the center of gravity is the data itself

If LangChain often feels application-first, LlamaIndex tends to feel data-first. That makes it a very natural fit when you want to spend more time shaping ingestion, chunking, metadata, and query behavior.

Using MongoDB’s LlamaIndex integration we can use VoyageEmbedding alongside MongoDBAtlasVectorSearch to make metadata filters very explicit, which is helpful for real enterprise retrieval where "give me the right answer" usually means "give me the right answer for this tenant, this region, and this product line."

The shape is roughly:

import os

from pymongo import MongoClient
from llama_index.embeddings.voyageai import VoyageEmbedding
from llama_index.vector_stores.mongodb import MongoDBAtlasVectorSearch
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.core.retrievers import VectorIndexRetriever
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

embed_model = VoyageEmbedding(
    voyage_api_key=os.environ["VOYAGE_API_KEY"],
    model_name="voyage-4",
)

mongo_client = MongoClient(os.environ["MONGODB_URI"])
vector_store = MongoDBAtlasVectorSearch(
    mongo_client,
    db_name="support",
    collection_name="context",
    vector_index_name="support_vector_index",
)

storage_context = StorageContext.from_defaults(vector_store=vector_store)

# docs is your loaded support corpus, such as runbooks, incident reviews,
# release notes, and ticket summaries.
vector_index = VectorStoreIndex.from_documents(
    docs,
    storage_context=storage_context,
    embed_model=embed_model,
)

filters = MetadataFilters(
    filters=[ExactMatchFilter(key="tenant_id", value="acme")]
)

retriever = VectorIndexRetriever(
    index=vector_index,
    filters=filters,
    similarity_top_k=10,
)

nodes = retriever.retrieve("Known workaround for INV-4421?")

What I like about this path is that it keeps the retrieval pipeline honest. You can see the data model. You can see the filter model. You can see how chunking choices affect what comes back. For article one in a series like this, that clarity is useful because it keeps us focused on context quality before we get distracted by agent loops.

Approach 3: Haystack when you want explicit, composable pipelines

Haystack is a nice fit for teams that prefer explicit components over higher-level abstractions. MongoDB’s Haystack integration uses a MongoDBAtlasDocumentStore with MongoDB retrievers, and the official tutorial pairs that with Voyage embedders. Haystack’s MongoDB integration also has separate semantic and full-text retrievers, which is useful when you want to make the retrieval strategy itself a first-class part of the pipeline.

A trimmed-down version looks like this:

from haystack import Pipeline
from haystack.utils import Secret
from haystack_integrations.components.embedders.voyage_embedders import VoyageTextEmbedder
from haystack_integrations.document_stores.mongodb_atlas import MongoDBAtlasDocumentStore
from haystack_integrations.components.retrievers.mongodb_atlas import MongoDBAtlasEmbeddingRetriever

document_store = MongoDBAtlasDocumentStore(
    mongo_connection_string=Secret.from_env_var("MONGODB_URI"),
    database_name="support",
    collection_name="context",
    vector_search_index="support_vector_index",
    full_text_search_index="support_search_index",
)

pipeline = Pipeline()
pipeline.add_component("query_embedder", VoyageTextEmbedder(model="voyage-4"))
pipeline.add_component(
    "retriever",
    MongoDBAtlasEmbeddingRetriever(document_store=document_store, top_k=10),
)
pipeline.connect("query_embedder.embedding", "retriever.query_embedding")

result = pipeline.run(
    {"query_embedder": {"text": "Known workaround for INV-4421?"}}
)

This is probably the most "pipe and fitting" version of the three, and that is a compliment. For enterprise teams, explicit systems are often easier to debug, evaluate, and explain. And once again, the interesting part is not that the framework is different. It is that the same MongoDB + Voyage retrieval core still fits.

Why MongoDB

The support copilot does not just need chunks in a vector store. It needs chunks, source documents, tenant metadata, ticket references, account records, release versions, and eventually execution state. MongoDB Vector Search lets you search semantic meaning alongside that operational data, pre-filter the search space using indexed fields, and combine vector and full-text retrieval when exact terms matter. Change streams then give you a way to react to new or updated records in real time, which is exactly what you want when incidents, tickets, or KB articles change during the workday.

And if you want an even tighter platform story, MongoDB Atlas now exposes Voyage models directly through the Embedding and Reranking API. That API is database-agnostic, but it pairs especially well with Atlas because it reduces the number of moving pieces needed to stand up a modern retrieval pipeline. Fewer services, fewer credentials, less trying to debug "why is this top result here?".

This is also where the framework story becomes easier to reason about. LangChain, LlamaIndex, and Haystack all give you different ergonomics. MongoDB stays the system where the data lives. Voyage stays the retrieval layer that improves what gets surfaced. That is a much more durable architecture than betting everything on whichever orchestration framework happens to be loudest this quarter.

What next?

Once the retrieval core is solid, adding agents becomes a lot more interesting.

In the next post, I’d take this same support copilot and add short-term execution state and long-term memory. LangGraph is a natural next step as it separates persistence into checkpoints for thread state and stores for long-term memory, and MongoDB already has first-class integrations for both the LangGraph checkpointer and the long-term store. That is where the earlier "brain" idea becomes concrete: not just retrieval, but retrieval plus memory plus durable execution.

The broader trend line is pretty clear, too. Agent frameworks are converging on durable state and memory. Retrieval models are getting richer with contextualized chunk embeddings, multimodal embeddings, and better rerankers. MongoDB Atlas is moving retrieval models and database capabilities closer together. The winning application architecture is the one that can absorb those changes without forcing you to rebuild your data layer every few months. MongoDB and Voyage AI fit that direction unusually well.

Persistent multi-agent conversations with the OpenAI Agents SDK and MongoDB

Alex Bevilacqua — Mon, 27 Apr 2026 16:49:36 +0000

Version 0.14.2 added a MongoDBSession backend; here's a working multi-agent customer-support demo that uses it, and the documents it leaves behind.

The OpenAI Agents SDK has shipped session backends for SQLite, SQLAlchemy, Redis, and Dapr for a while now. With 0.14.2 (April 2026), MongoDBSession joined that list, and 0.14.6 added the docs page. If you're already running MongoDB for application data, this is the moment to stop standing up a second store just to remember what the agent said three turns ago. The demo for this walkthrough is a small e-commerce support app with three handoff-connected agents and one MongoDB instance behind everything: customers, orders, support articles, and the conversation history.

Repo: https://github.com/alexbevi/mongodb-openai-agents-sdk-example.

What you'll build

A CLI customer-support agent that identifies the user from MongoDB, hands off between a triage agent, an order-support agent, and a knowledge-base agent, and persists every turn (user message, tool call, tool output, assistant reply, handoff) to MongoDB via MongoDBSession. You quit, restart the process, log in with the same email, and the agent picks up the thread — no re-explaining the return you started yesterday.

Why MongoDB for sessions

A session backend has three jobs: store one item per turn, return them in order on the next run, and not corrupt itself when two processes write at once. The interesting part for MongoDB is how naturally each of those maps to things the database already does.

Items in a session are heterogeneous. A turn can be a user message, a tool call, a tool result, an assistant message, or a handoff record — each with its own shape. A document store takes those payloads as-is. There's no messages table you have to migrate every time the SDK adds a new run-item type, and no JSON column to parse around.

Ordering is the hard part, and $inc is built for it. MongoDBSession stamps each message with a monotonically increasing seq counter — the SDK docs call this out explicitly: it preserves ordering across concurrent writers and processes. That's a single-document atomic increment, not a distributed lock or an optimistic-retry loop. Two FastAPI workers handling the same session_id won't interleave.

One store, one connection pool. This is the angle the demo actually showcases. The ecommerce_support database holds customers, orders, and support_articles next to agent_sessions and agent_messages. Tools query operational data, the SDK persists turns, and they share the same AsyncMongoClient. Adding session memory cost zero new infrastructure.

Walkthrough

1. Prerequisites

Python 3.10+, an OpenAI API key, and either a local mongod or a MongoDB Atlas cluster. Nothing in the demo requires Atlas-only features — a 27017 on localhost is fine.

2. Install

requirements.txt pins the new extra:

openai-agents[mongodb]>=0.14.2
python-dotenv>=1.0.0
pymongo>=4.13

pip install -r requirements.txt

The [mongodb] extra pulls in pymongo's async client; the MongoDBSession class lives at agents.extensions.memory.MongoDBSession.

3. Connect

The demo uses one shared AsyncMongoClient per process (the right pattern — sessions don't own the client, they share its pool):

from pymongo.asynchronous.mongo_client import AsyncMongoClient

MONGODB_URI = os.environ.get("MONGODB_URI", "mongodb://localhost:27017")
DB_NAME = "ecommerce_support"

mongo_client = AsyncMongoClient(MONGODB_URI)
db = mongo_client[DB_NAME]

try:
    await mongo_client.admin.command("ping")
except Exception as exc:
    print(f"\nCannot connect to MongoDB ({MONGODB_URI}):\n  {exc}")
    return

4. Seed and identify

python seed_data.py loads three demo customers, five products, five orders with embedded line items, and seven support articles indexed for $text search. Then main.py looks the customer up so the triage agent doesn't have to ask for an email it already knows.

5. Instantiate the session

This is the integration:

session_id = f"support_{email.replace('@', '_at_').replace('.', '_')}"
session = MongoDBSession(
    session_id=session_id,
    client=mongo_client,
    database=DB_NAME,
)

if not await session.ping():
    print("Warning: MongoDB session storage is unavailable.")

existing = await session.get_items()

Constructing with client= (rather than MongoDBSession.from_uri(...)) means the session shares the app's connection pool and session.close() becomes a no-op — the lifecycle stays with you. session.ping() is a real round-trip against MongoDB, useful for liveness probes.

6. Run

Pass session= to the runner. Everything else is the same SDK you already know:

with trace("Customer Support", group_id=conversation_id):
    result = await Runner.run(
        current_agent,
        input=user_input,
        context=ctx,
        session=session,   # MongoDB stores every turn automatically
    )

Have a conversation, quit, run python main.py again with the same email, and the next message gets the full prior context prepended automatically.

What MongoDB actually stored

After a few turns with alice@example.com, two collections show up in the ecommerce_support database. The interesting one is agent_messages. A representative document, abridged:

{
  _id: ObjectId("6620d1f4..."),
  session_id: "support_alice_at_example_com",   // partition key for this conversation
  seq: 7,                                        // monotonically increasing turn order
  message_data: {                                // the SDK's run-item, stored as-is
    type: "function_call_output",
    call_id: "call_8b2...",
    output: "Return initiated for order ORD-1001.\nReason: Not powerful enough...\nEstimated refund: $1,484.98 (includes 10% Gold loyalty bonus)"
  },
  created_at: ISODate("2026-04-26T19:14:08.221Z")
}

Three fields earn their keep:

session_id is the only field every read filters on. It's the partition key for "this conversation."
seq is the integer that makes ordering deterministic. The SDK reads with sort({ seq: 1 }) and writes with an atomic $inc against the matching agent_sessions document, which is what makes concurrent workers safe without a distributed lock.
message_data is the SDK's run-item — a user message, tool call, tool output, assistant message, or handoff. Different shape every time. The document model just stores it.

agent_sessions holds one document per session_id with the current high-water seq and timestamps — that's the counter $inc operates on.

The SDK creates its indexes on first use (per the sessions docs). You'll see a compound index on (session_id, seq) on agent_messages (the only access pattern the SDK has — fetch ordered history for one session) and a unique index on session_id in agent_sessions.

Production notes

For Atlas, swap the URI for mongodb+srv://... — MongoDBSession accepts it without any other change. If abandoned conversations accumulate, add a TTL index on agent_messages.created_at and old turns retire on their own.

Connection lifetime matters: keep one AsyncMongoClient per process, construct MongoDBSession(client=...) per request, and let the Runner do the rest. Don't reach for MongoDBSession.from_uri(...) in a web handler — it builds and tears down a client every call. The session needs read/write on the two configured collections (defaults agent_sessions and agent_messages, both overridable via sessions_collection= and messages_collection=). The seq counter keeps concurrent writers safe, but fanning the same session_id across processes will interleave their turns — safe, but probably not what the user meant.

Try it yourself

git clone https://github.com/alexbevi/mongodb-openai-agents-sdk-example
cd mongodb-openai-agents-sdk-example
pip install -r requirements.txt
cp env.example .env          # set OPENAI_API_KEY and MONGODB_URI
python seed_data.py
python main.py

Required env vars: OPENAI_API_KEY, MONGODB_URI (defaults to mongodb://localhost:27017). Demo accounts: alice@example.com (Gold), bob@example.com (Standard), carol@example.com (Platinum).

Where to go next

The full session API surface — get_items, add_items, pop_item, clear_session, ping — is documented in the Sessions overview, including the MongoDB-specific notes on collection naming and Atlas URIs.
Wrap your MongoDBSession in OpenAIResponsesCompactionSession once threads grow long; it summarizes old turns server-side and rewrites the underlying session.
The natural next MongoDB feature for this demo is Atlas Vector Search — store embeddings on support_articles and replace the $text query in search_knowledge_base with $vectorSearch. Same database, same client, one new index.

MongoDB as the Brain of Modern AI Applications

Alex Bevilacqua — Thu, 16 Apr 2026 12:39:11 +0000

Production agents need two persistence layers: thread-scoped state and cross-session memory. Google ADK Sessions stores events and state for a single conversation, while MemoryService handles recall across sessions. LangGraph memory makes the same split with a checkpointer for short-term memory and a store for long-term memory, and LangChain long-term memory builds on LangGraph stores that persist JSON documents by namespace and key. The memory architecture has already converged.

Durable memory is not raw chat replay. Vertex AI Memory Bank is built for identity-scoped, cross-session personalization and LLM-driven knowledge extraction, and Google’s ADK memory write-up describes Memory Bank as extracting key information from session data rather than replaying every turn. LangChain’s memory model is equally explicit: long-term memory can be semantic (facts), episodic (past actions), or procedural (rules and prompts).

Agent memory should be structured data, not opaque blobs. LangChain stores persist long-term memory as JSON documents, and ADK’s DatabaseSessionService migration moved session serialization from pickle-based storage to JSON-based storage in v1.22.0. MongoDB’s document model matches that reality directly.

MongoDB is a strong fit because retrieval lives in the same system as the memory. MongoDB Vector Search supports both approximate and exact nearest-neighbor search, and the default index type is HNSW. Vector search pre-filters let you scope recall by fields like user_id, tenant_id, or memory_type before embeddings are compared. Hybrid search combines vector and full-text retrieval with reciprocal rank fusion, which is exactly what memory needs when the data mixes natural language with exact identifiers like invoice IDs, feature flags, or product SKUs.

Memory also needs retention and update policy. TTL indexes automatically expire session artifacts, scratchpads, or short-lived summaries. Change streams give you a real-time feed of inserts and updates, which is the right trigger for summarization, entity extraction, or memory distillation jobs. When the data is relationship-heavy instead of chunk-heavy, GraphRAG on MongoDB uses entities, edges, and $graphLookup for relationship-aware, multi-hop retrieval.

This is not a narrow LangChain story. MongoDB publishes integrations for LangGraph, LangChain, LlamaIndex, Semantic Kernel, Haystack, Spring AI, CrewAI, and Vertex AI. LlamaIndex can use MongoDB for the vector store, document store, and index store. Mem0 also supports MongoDB as a memory backend. MongoDB fits the storage contract these frameworks keep converging on: structured documents plus semantic, lexical, and graph-based retrieval.

LangGraph: short-term checkpoints and long-term memory in one database

MongoDB’s LangGraph integration exposes MongoDBSaver for checkpoints and MongoDBStore for durable memory, with optional vector indexing and TTL-based expiry. That maps directly to LangGraph’s own split between thread persistence and store-backed recall.

from pymongo import MongoClient
from langgraph.checkpoint.mongodb import MongoDBSaver
from langgraph.store.mongodb import MongoDBStore, create_vector_index_config
from langchain_openai import OpenAIEmbeddings

MONGODB_URI = "<connection-string>"

# Short-term memory: thread checkpoints
client = MongoClient(MONGODB_URI)
checkpointer = MongoDBSaver(client)

# Long-term memory: semantic store with metadata filters
index_config = create_vector_index_config(
    embed=OpenAIEmbeddings(model="text-embedding-3-small"),
    dims=1536,
    fields=["content"],
    filters=["user_id", "memory_type"],
)

# Assume `builder` is an existing LangGraph StateGraph
with MongoDBStore.from_conn_string(
    conn_string=MONGODB_URI,
    db_name="agent_memory",
    collection_name="memories",
    index_config=index_config,
    ttl_config={
        "default_ttl": 60 * 60 * 24 * 30,   # 30 days
        "refresh_on_read": True,
    },
) as store:
    graph = builder.compile(checkpointer=checkpointer, store=store)

    store.put(
        namespace=("user-42", "memories"),
        key="pref:vegan:soho",
        value={
            "content": "User prefers vegan restaurants near SoHo.",
            "user_id": "user-42",
            "memory_type": "semantic",
        },
    )

    results = store.search(
        ("user-42", "memories"),
        query="Where should I book dinner tonight?",
        limit=3,
    )

    for result in results:
        print(result.value)

This is the clean MongoDB story: checkpoints for working memory, a store for long-term memory, vector retrieval for recall, metadata filters for isolation, and TTL for automatic cleanup. The same database handles all of it.

LangChain: chat history plus hybrid recall

At the LangChain layer, MongoDB covers both conversation state and retrieval. MongoDBChatMessageHistory persists per-session message history, MongoDBAtlasVectorSearch stores semantic memories, and MongoDBAtlasHybridSearchRetriever fuses lexical and semantic recall.

from langchain_core.documents import Document
from langchain_mongodb.chat_message_histories import MongoDBChatMessageHistory
from langchain_mongodb.retrievers.hybrid_search import MongoDBAtlasHybridSearchRetriever
from langchain_mongodb.vectorstores import MongoDBAtlasVectorSearch
from langchain_openai import OpenAIEmbeddings

MONGODB_URI = "<connection-string>"

history = MongoDBChatMessageHistory(
    session_id="user-42:thread-7",
    connection_string=MONGODB_URI,
    database_name="agent_memory",
    collection_name="chat_history",
    history_size=20,
)

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
    connection_string=MONGODB_URI,
    namespace="agent_memory.user_memories",
    embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
    index_name="vector_index",
)

vector_store.add_documents(
    [
        Document(
            page_content="User prefers vegan restaurants near SoHo.",
            metadata={"user_id": "user-42", "memory_type": "semantic"},
        ),
        Document(
            page_content="Invoice 8419 was disputed last month.",
            metadata={"user_id": "user-42", "memory_type": "episodic"},
        ),
    ]
)

history.add_user_message("Find dinner options for me in SoHo.")

retriever = MongoDBAtlasHybridSearchRetriever(
    vectorstore=vector_store,
    search_index_name="search_index",
    top_k=5,
    fulltext_penalty=50,
    vector_penalty=50,
)

docs = retriever.invoke("Find dinner options for a vegan user in SoHo")
for doc in docs:
    print(doc.page_content, doc.metadata)

Hybrid retrieval is not optional in a serious memory system. "Prefers vegan restaurants in SoHo" is semantic. "Invoice 8419" is lexical. MongoDB’s hybrid retriever exists because production memory contains both.

Google ADK: the Sessions-and-Memory model maps cleanly to MongoDB

Google ADK makes the architecture explicit. SessionService manages session objects, applies state_delta, and appends event history. MemoryService manages long-term semantic memory across sessions. The Sessions docs currently list InMemorySessionService, VertexAiSessionService, and DatabaseSessionService, so MongoDB is not a built-in backend today. But ADK exposes base session and memory service abstractions, which makes MongoDB a natural implementation target rather than a workaround. Memory Bank then adds the identity-scoped memory semantics on top.

A MongoDB-backed ADK deployment should separate sessions, events, and distilled memories into dedicated collections. That mirrors ADK’s documented split between mutable session state, append-only event history, and searchable long-term memory.

from datetime import datetime, timezone
from pymongo import ASCENDING, MongoClient

MONGODB_URI = "<connection-string>"

client = MongoClient(MONGODB_URI)
db = client["agent_memory"]

sessions = db["adk_sessions"]
events = db["adk_events"]
memories = db["adk_memories"]

sessions.create_index(
    [("app_name", ASCENDING), ("user_id", ASCENDING), ("session_id", ASCENDING)],
    unique=True,
)

events.create_index(
    [
        ("app_name", ASCENDING),
        ("user_id", ASCENDING),
        ("session_id", ASCENDING),
        ("timestamp", ASCENDING),
    ]
)

memories.create_index([("user_id", ASCENDING), ("memory_type", ASCENDING)])
# Create a MongoDB Vector Search index on memories.embedding
# Mark user_id and memory_type as filter fields in the index definition.

def persist_session_turn(app_name: str, user_id: str, session_id: str, event: dict, state: dict) -> None:
    now = datetime.now(timezone.utc)

    sessions.update_one(
        {"app_name": app_name, "user_id": user_id, "session_id": session_id},
        {
            "$set": {"state": state, "updated_at": now},
            "$setOnInsert": {"created_at": now},
        },
        upsert=True,
    )

    events.insert_one(
        {
            "app_name": app_name,
            "user_id": user_id,
            "session_id": session_id,
            "timestamp": event["timestamp"],
            "event": event,
        }
    )

def store_memory(user_id: str, content: str, embedding: list[float], source_session_id: str) -> None:
    memories.insert_one(
        {
            "user_id": user_id,
            "memory_type": "semantic",
            "content": content,
            "embedding": embedding,
            "source_session_ids": [source_session_id],
            "created_at": datetime.now(timezone.utc),
        }
    )

def search_memories(user_id: str, query_embedding: list[float]):
    pipeline = [
        {
            "$vectorSearch": {
                "index": "memory_vector_index",
                "path": "embedding",
                "queryVector": query_embedding,
                "numCandidates": 100,
                "limit": 5,
                "filter": {"user_id": user_id, "memory_type": "semantic"},
            }
        },
        {
            "$project": {
                "content": 1,
                "memory_type": 1,
                "score": {"$meta": "vectorSearchScore"},
            }
        },
    ]
    return list(memories.aggregate(pipeline))

This is a design sketch, not an official ADK adapter. The point is that ADK’s abstractions already describe a storage model MongoDB handles well: mutable session state, append-only events, and searchable long-term memory. Once a MemoryService exists, ADK’s built-in PreloadMemory and LoadMemory tools can use it.

Dedicated memory layers also fit: Mem0 on MongoDB

MongoDB is not only useful when memory is native to the agent framework. Mem0’s MongoDB backend supports MongoDB directly as a vector database for memory storage and retrieval. That matters because it shows MongoDB works both as the application database and as the substrate beneath a dedicated memory layer.

from mem0 import Memory

config = {
    "vector_store": {
        "provider": "mongodb",
        "config": {
            "db_name": "mem0_db",
            "collection_name": "mem0_collection",
            "mongo_uri": "<connection-string>",
        },
    }
}

memory = Memory.from_config(config)

messages = [
    {"role": "user", "content": "I'm planning to watch a movie tonight."},
    {"role": "assistant", "content": "What genres do you like?"},
    {"role": "user", "content": "I love sci-fi, not thrillers."},
]

memory.add(messages, user_id="alice", metadata={"category": "movies"})

The architectural value is the same as in LangGraph and LangChain: persistent memory objects, vector retrieval, and application data can live in one operational system instead of being spread across separate services.

Why MongoDB is the best choice here

MongoDB is the best choice when you want one system to hold agent state, long-term memory, retrieval data, and the application records the agent reasons over. The document model matches how current frameworks persist memory. Vector Search and Search cover recall. TTL indexes and change streams cover retention and event-driven memory extraction. GraphRAG covers relationship-heavy data. The result is not "a vector store with extra features." It is a memory layer that can also be the system of record. That is why MongoDB works as the brain of a modern AI application.

Cloudflare + MongoDB: How to fix 'Error: Dynamic require of "punycode/" is not supported'

Alex Bevilacqua — Wed, 31 Dec 2025 13:50:50 +0000

If you've followed my previous post to try and connect to MongoDB from Cloudflare workers, it's possible you've come across the following issue:

Error: Dynamic require of "punycode/" is not supported

The TL;DR is there is an issue with how @cloudflare/vite-plugin is processing an import with a trailing slash within the tr46 library, which is a transitive dependency of the MongoDB Node.js driver. The current solution is to patch this out until a proper fix is in place.

Reproduction

Let's begin with a new application we can use as a minimum reproduction. Chances are you've already got an application ready that's hitting this issue, but if not we can verify this behavior by simply creating a new React Router app using create-cloudflare as follows, then adding the MongoDB Node.js driver as a dependency and importing it.

# create a new react router app
npm create cloudflare@latest -- my-react-router-app --framework=react-router
cd my-react-router-app
# install mongodb
npm install mongodb --save
# prepend an import to the workers/app.ts file
printf 'import { MongoClient } from "mongodb";\n%s' "$(cat workers/app.ts)" > workers/app.ts
# update wrangler.jsonc with compatibility flags to support SSR
sed -i '' '/"compatibility_date": "2025-04-04"/a\
  "compatibility_flags": ["nodejs_compat"],' wrangler.jsonc

With a freshly bootstrapped application, let's try running it to see what happens.

npm run dev
> dev
> react-router dev

11:15:07 AM [vite] (ssr) Re-optimizing dependencies because vite config has changed
11:15:08 AM [vite] (ssr) ✨ new dependencies optimized: mongodb
11:15:08 AM [vite] (ssr) ✨ optimized dependencies changed. reloading
[vite] program reload
Error: Dynamic require of "punycode/" is not supported
    at null.<anonymous> (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:11:9)
    at node_modules/tr46/index.js (/Users/alex/Temp/my-react-router-app/node_modules/tr46/index.js:3:18)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/lib/url-state-machine.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/lib/url-state-machine.js:2:14)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/lib/URL-impl.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/lib/URL-impl.js:2:13)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/lib/URL.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/lib/URL.js:499:14)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/webidl2js-wrapper.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/webidl2js-wrapper.js:3:13) {
  [cause]: undefined
}

Vite is complaining that the dynamic require of "punycode/" is not supported. The trailing slash following "punycode" is interesting, but we should first see where it's being imported. We can do this by using npm ls to quickly narrow down usage of punycode to the tr46 library:

npm ls punycode
my-react-router-app@ /Users/alex/Temp/my-react-router-app
└─┬ mongodb@7.0.0
  └─┬ mongodb-connection-string-url@7.0.0
    └─┬ whatwg-url@14.2.0
      └─┬ tr46@5.1.1
        └── punycode@2.3.1

Inspecting the tr46 library at https://github.com/jsdom/tr46/blob/main/index.js shows the trailing slash on the import as well:

"use strict";

const punycode = require("punycode/"); // <--- this is the line in question
const regexes = require("./lib/regexes.js");
const mappingTable = require("./lib/mappingTable.json");
const { STATUS_MAPPING } = require("./lib/statusMapping.js");
// ...

I initially tried to open a PR at https://github.com/jsdom/tr46/pull/73 to sort this out, but the maintainer points out that the issue is with Vite, so we'll need to look elsewhere for a solution. This change (introduced in commit fef6e95) was likely done to address punycode deprecation warnings such as that described in https://github.com/jsdom/tr46/issues/63. For more info on those deprecations see "Solving the \"Punycode Module is Deprecated\" Issue in Node.js".

Patching

We're going to solve this issue in a roundabout fashion using patch-package to modify the punycode import directly in our node_packages and then have a postinstall script that will ensure the patch is consistently applied.

# install patch-package
npm install patch-package
# update package.json to run patch-package as well as cf-typegen (which is there by default)
npm pkg set scripts.postinstall="patch-package && npm run cf-typegen"
# update node_modules/tr46/index.js to remove the trailing slash from the import
sed -i '' 's/require("punycode\/")/require("punycode")/g' node_modules/tr46/index.js
# create a patch for the tr46 package based on the above change
npx patch-package tr46
# reinstall and apply patches
npm install

That should do it! When we run npm install it will also run the postinstall, which will apply the patch we just created.

Summary

Though patching transient dependencies to work around an issue like this is not ideal, it does offer a path forward for anyone hitting this specific error. To summarize what we did to address the issue:

Install the patch-package library (npm install patch-package)
Update your package.json's scripts.postinstall to prepend a patch-package script to any postinstall scripts that may already be present
Modify node_modules/tr46/index.js to remove the trailing / from require("punycode/")
Create the patch by running npx patch-package tr46
Ensure the patch is applied by running npm install

Hopefully we can get this sorted out more cleanly (reported as https://github.com/cloudflare/workers-sdk/issues/11751), but in the meantime feel free to use this approach if you find it suitable.

MongoDB Drivers and Network Compression

Alex Bevilacqua — Thu, 13 Nov 2025 16:19:50 +0000

MongoDB's drivers communicate with a MongoDB process using the Wire Protocol, which is a simple socket-based, request-response style protocol that primarily uses the OP_MSG opcode (though prior to MongoDB 6.0 there were a number of additional legacy opcodes). Since the contents of OP_MSG messages was uncompressed, starting with MongoDB 3.4 a new opcode was introduced that would enable the Wire Protocol to support compressed messages as well: OP_COMPRESSED.

All official MongoDB drivers allow you to enable and configure compression options via the connection string. To use any of the available compressors, they'd need to first be enabled for each mongod or mongos instance through the net.compression.compressors option, however all compressors are currently enabled by default, so you'd really only need to modify this to remove support for a given compressor.

Enabling network compression for your workload is as easy as appending compressors=xxx (where xxx is one or more compressors as comma-separated values) to your connection string. For every MongoClient created with this connection string, almost all¹ database commands will be compressed, which can result in massive reductions in the amount of data that needs to be sent back and forth to a MongoDB process.

As a simple demonstration, I've instrumented a Node.js-based workload (see alexbevi/node-tcp-metrics) to hook into net.createConnection(), net.Server and tls.connect() to track the number of bytes being sent/received:

import "./tcp-metrics.js";

import { on } from "./tcp-metrics.js";
import { MongoClient } from "mongodb";
import Chance from "chance";

const uri = process.env.MONGODB_URI;
if (!uri) {
  throw new Error("MONGODB_URI environment variable is not set");
}
const client = new MongoClient(uri);
const delay = (ms: number): Promise<void> => new Promise(res => setTimeout(res, ms));

(async () => {
  try {
    await client.connect();
    const db = client.db("testdb");
    const collection = db.collection<{ _id: string; data: string }>("testcollection");
    const chance = new Chance(42); // Fixed seed for deterministic generation

    // Generate a complex document structure that's approximately 5MB
    const itemCount = 5000; // Adjust to control size
    const payload = {
      users: Array.from({ length: itemCount }, (_, i) => ({
      id: i,
      name: chance.name(),
      email: chance.email(),
      address: {
        street: chance.address(),
        city: chance.city(),
        state: chance.state(),
        zip: chance.zip(),
        country: chance.country()
      },
      phone: chance.phone(),
      company: chance.company(),
      bio: chance.paragraph({ sentences: 5 }),
      avatar: chance.url(),
      tags: Array.from({ length: 10 }, () => chance.word()),
      metadata: {
        createdAt: chance.date().toISOString(),
        lastLogin: chance.timestamp(),
        preferences: {
        theme: chance.pickone(['dark', 'light', 'auto']),
        language: chance.locale(),
        notifications: chance.bool()
        }
      }
      }))
    };
    const doc: any = { _id: "large-doc", ...payload };

    await collection.insertOne(doc);
    const result = await collection.findOne({ _id: "large-doc" });

    const buf = Buffer.from(JSON.stringify(result));
    console.log("Document insert and read complete, doc size (bytes):", buf.length);

    await collection.deleteOne({ _id: "large-doc" });
  } catch (err) {
    console.error("MongoDB error:", err);
  } finally {
    await client.close();
 }
})();

on("socketSummary", (s) => console.log(s));

When this is run, the workload will create a complex JSON document using Chance, write it to a MongoDB Atlas Database, then read it back before deleting it.

$ MONGODB_URI="mongodb+srv://USER:PASS@abc.cdefg.mongodb.net/" npm run dev

Document insert and read complete, doc size (bytes): 4725754
{ rx: 5058704, tx: 5058304, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1093, tx: 523, label: 'xxx.yyy.zzz.101:27017' }
{ rx: 1093, tx: 523, label: 'xxx.yyy.zzz.116:27017' }
{ rx: 1117, tx: 523, label: 'xxx.yyy.zzz.108:27017' }

There are 4 open sockets in the example as the default Atlas configuration is a 3 member replica set. The driver has opened one socket to send commands to the server, and has also created dedicated monitoring connections to each host. If the workload were to remain active and not exit immediately, another 3 RTT connections would also be opened (one to each host in the replica set) for a total of 7 sockets. \
See the Server Monitoring specification, or "How Many Connections is My Application Establishing to My MongoDB Cluster?" for more detail.
{: .prompt-info }

MongoDB supports 3 possible network compressors: zlib, ZStandard (zstd) and Snappy. zlib is always supported out of the box, however some drivers may require an additional package to support additional compressors. For example, when using MongoDB's Node.js driver, the following would be required from npm to support snappy and zstd:

Our workload is using the default read preference, so with a baseline of rx: 5058704, tx: 5058304 being sent to and received from the replica set primary, let's explore the impact of network compression.

zlib

zlib is a software library used for data compression as well as a data format. zlib was written by Jean-loup Gailly and Mark Adler and implements the DEFLATE compression algorithm used in their gzip file compression program. The first public version of Zlib, 0.9, was released on 1 May 1995 and was originally intended for use with the libpng image library. It is free software, distributed under the zlib License.

To test this compressor we append compressors=zlib to our connection string and re-run our script.

$ MONGODB_URI="mongodb+srv://USER:PASS@abc.cdefg.mongodb.net/?compressors=zlib" npm run dev

Document insert and read complete, doc size (bytes): 4725754
{ rx: 2417301, tx: 2361623, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1147, tx: 515, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1123, tx: 518, label: 'xxx.yyy.zzz.101:27017' }
{ rx: 1123, tx: 518, label: 'xxx.yyy.zzz.116:27017' }

With zlib compression enabled we can see about a 52% decrease in the amount of data sent over the wire for this workload:

uncompressed:      rx: 5058704, tx: 5058304
compressed (zlib): rx: 2417301, tx: 2361623

ZStandard (zstd)

Zstandard is a lossless data compression algorithm developed by Yann Collet at Facebook. Zstd is the corresponding reference implementation in C, released as open-source software on 31 August 2016. The algorithm was published in 2018 as RFC 8478, which also defines an associated media type "application/zstd", filename extension "zst", and HTTP content encoding "zstd".

To test this compressor we append compressors=zstd to our connection string and re-run our script.

$ MONGODB_URI="mongodb+srv://USER:PASS@abc.cdefg.mongodb.net/?compressors=zstd" npm run dev

Document insert and read complete, doc size (bytes): 4725754
{ rx: 2395239, tx: 2394798, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1147, tx: 519, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1123, tx: 519, label: 'xxx.yyy.zzz.101:27017' }
{ rx: 1123, tx: 519, label: 'xxx.yyy.zzz.116:27017' }

With zstd compression enabled we can see about a 53% decrease in the amount of data sent over the wire for this workload:

uncompressed:      rx: 5058704, tx: 5058304
compressed (zstd): rx: 2395239, tx: 2394798

Snappy

Snappy (previously known as Zippy) is a fast data compression and decompression library written in C++ by Google based on ideas from LZ77 and open-sourced in 2011. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. Compression speed is 250 MB/s and decompression speed is 500 MB/s using a single core of a circa 2011 "Westmere" 2.26 GHz Core i7 processor running in 64-bit mode. The compression ratio is 20–100% lower than gzip.

To test this compressor we append compressors=snappy to our connection string and re-run our script.

$ MONGODB_URI="mongodb+srv://USER:PASS@abc.cdefg.mongodb.net/?compressors=snappy" npm run dev

Document insert and read complete, doc size (bytes): 4725754
{ rx: 1125, tx: 527, label: 'xxx.yyy.zzz.116:27017' }
{ rx: 3807095, tx: 3797837, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1149, tx: 527, label: 'xxx.yyy.zzz.108:27017' }
{ rx: 1125, tx: 527, label: 'xxx.yyy.zzz.101:27017' }

With snappy compression enabled we can see about a 25% decrease in the amount of data sent over the wire for this workload:

uncompressed:        rx: 5058704, tx: 5058304
compressed (snappy): rx: 3807095, tx: 3797837

Summary

Though there may be a need for an additional dependency, Zstandard compression is likely the best option as it will provide good compression with a low memory footprint. For Node.js specifically this requirement will likely go away once the driver's minimum runtime version rises to Node 24 as zstd support was added in 23.8.0.

If you're running your workload in AWS (or anywhere really), data transfer costs will contribute to your overall costs. You can use tools like the AWS pricing calculator to dig into cost projections, but given you can potentially slash those in half (at least for the data going to/from your cluster) with a simple connection string update it makes MongoDB a more cost-effective option for your application.

Per the Wire Compression specification, some commands should not be compressed. These include hello/ismaster, saslStart, saslContinue, getnonce, authenticate, createUser, updateUser, copydbSaslStart, copydbgetnonce and copydb. ↩

MongoDB Driver Compatibility with MongoDB Servers

Alex Bevilacqua — Tue, 05 Aug 2025 09:52:39 +0000

MongoDB server versions eventually reach EOL - as MongoDB 6.0 did on July 31, 2025. If your workload is running in MongoDB Atlas, the major version of your cluster will be automatically upgraded, but what if you haven't upgraded your application, its dependencies or the runtime environment? Will your application break? Is it still compatible? I wrote about this previously at "Will Upgrading My MongoDB Server Version Break My Application?", but there are still a lot of questions that pop up regarding driver compatibility so I wanted to go further.

Though good dependency management hygiene is important, it's a time consuming process that can require extensive testing so you typically want to do it on your own terms - not because of a service upgrade.

Let's assume your application uses v4.17.2 of the MongoDB Node.js driver, but your application has been humming along for some time without issue. You got an email indicating your cluster was going to be upgraded from MongoDB 6.0 to MongoDB 7.0, but based on the driver compatibility table that version of the driver isn't even present!

What compatibility tables actually mean

MongoDB drivers are constantly being updated to add support for new features of the MongoDB server, as well as address bugs/regressions and improve performance. The compatibility tables (such as the example below) are simply a reflection of what versions of the driver have previously had their test suite run against a version of the MongoDB server.

The MongoDB drivers are all built from a set of common specifications, which are updated periodically as new MongoDB server features necessitate changes. For example, MongoDB 7.0 introduced Atlas Search Index Management, which resulted in the index management specifications being updated to define APIs drivers can implement to support the new database commands required to perform this new function.

If the version of the driver being used doesn't contain support for MongoDB 7.0, new APIs such as Collection#createSearchIndexwouldn't be directly available - but if you don't need this MongoDB 7.0 feature, your existing application using v4.17.2 of the Node.js driver would continue to function as expected.

Since createSearchIndexesis a database command, even using a version of the driver that didn't have convenient APIs for interacting with the feature could still be used to run the database command directly.

For example:
const commandDoc = {
  createSearchIndexes: "contacts",
  indexes: [{
    name: "searchIndex01",
    definition: { mappings: { dynamic: true }
  }]
};
const result = await myDB.command(commandDoc);

What happens if I don't update my drivers

The most likely outcome is - nothing. Your application will continue to connect to your cluster, serialize and transmit database commands and receive and deserialize command responses. Even if the version of your driver is not present on the compatibility matrix, the MongoDB Wire Protocol the drivers use to communicate with your cluster hardly ever changes.

As such, operationally your workload should continue to function as expected. Since the MongoDB server version has changed the performance profile of your workload may change, but this would not likely be a result of the driver remaining unchanged.

Older drivers may not receive security updates or performance improvements - however this is true of your application's dependencies. Plan to update your driver, but rest assured that doing so should be compatible with your application or business' maintenance schedule.

Can my application actually break if I do nothing

Yes - but for VERY specific reasons, all of which would be documented thoroughly and communicated prior to a major MongoDB server release.

Wire Protocol changes

As mentioned previously, the wire protocol is extremely stable and rarely changes, however with the release of MongoDB 6.0, all legacy opcodes were removed. This meant applications using drivers that had not been updated since MongoDB 3.4 (which reached EOL in 2020) would stop working as soon as their cluster was upgraded to 6.0.

THIS IS THE ONLY WIRE PROTOCOL CHANGE OF THIS NATURE TO DATE

Command removals

On occasion, database commands may be replaced or removed. This will not happen prior to a deprecation period (at least one major release prior). This happened previously when MongoDB 5.0 removed a number of deprecated commands.

If applications happen to be using those commands directly, once the MongoDB server is upgraded to a version that removes support for them, those applications would throw errors where those commands are used - such as the following example (using mongosh, which is built using the Node.js driver):

test> db.version()
4.4.29
test> db.runCommand({ resetError: 1 })
{ ok: 1 }
test> db.version()
5.0.31
test> db.runCommand({ resetError: 1 })
MongoServerError[CommandNotFound]: no such command: 'resetError'

Each major MongoDB server release will contain both release notes and compatibility changes. Make sure to review the compatibility changes to ensure if there are any command removals, they don't represent commands your application is using.

See the following for reference:

Key Takeaways

Not having a driver considered "compatible" doesn't mean the application won't continue to work
Compatibility implies "feature compatibility" - as in "new" features of the server version listed
Not upgrading your driver shouldn't result in your application breaking
There are scenarios where not upgrading the driver will break your application, but these are few, far between and well documented

There are multiple benefits to keeping your application's dependencies up to date, but there can also be drawbacks. As such it's important to test upgrades in a lower environment to ensure as many potential issues can be caught prior to applying changes in production environments.

If you're working with Node.js, consider tooling like dependency-time-machine to help with sequential dependency update automation. Since your dependency graph is likely complex, this type of approach can help update your application incrementally in a way that may minimize interdependency compatibility issues.

Call Stack, But Make It Async!

Alex Bevilacqua — Fri, 12 Jul 2024 18:10:02 +0000

Written by @nbbeeken (Blog, GitHub)

Intro

In a recent release of the MongoDB Node.js driver (v6.5.0) the team completed the effort of getting all our asynchronous operations to report an accurate asynchronous stack trace to assist in pinpointing error origination. Here, I'll walk you through what this feature of JavaScript is and how to obtain it at the low price of zero-cost.

Calls and how to stack them 📚

First, what is a call stack? A call stack is a hidden data structure that stores information about the active subroutines of a program; active subroutines being functions that have been called but have yet to complete execution and return control to the caller. The main function of the call stack is to keep track of the point to which each active subroutine should return control when it finishes executing.

Let's go through an example, take a program that parses a string from its arguments that is an equation like "2+2" and computes the result:

main()
  -> parseString()
    -> splitString()
      -> stringLength()
    -> stringToNumber()
  -> add()
  -> printResult()
-> return;

Most of us are familiar with the above procedural paradigm (whether from JavaScript, C, Java, or Python) where each step in the program is synchronous, so our call stack is a clear ordering of dependent procedures. For example, if stringLength fails, the call stack would contain stringLength, splitString, parseString, and main as active procedures that have yet to return to their callers. The error system of our runtime uses this stack trace to generate a helpful error trace:

file://addNumbers.mjs:35
    throw new Error('cannot get string length')
          ^
Error: cannot get string length
    at stringLength (file://addNumbers.mjs:35:11)
    at splitString (file://addNumbers.mjs:17:17)
    at parseString (file://addNumbers.mjs:11:19)
    at main (file://addNumbers.mjs:4:5)

The Async wrench 🔧

Everything changes when we shift to an asynchronous programming model, as the introduction of asynchronous work means we no longer have strictly dependent procedures. Essentially, async programming is about setting up tasks and adding handling that will be invoked some time later when the task is complete.

Let's add I/O (a read from standard in) into our program to see how this changes our call stack:

main()
-> readStdin(handleUserInput)
// When the user finishes typing
handleUserInput()
-> parseString()
  -> splitString()
    -> stringLength()

Now, main's only job is to ask the runtime to read from stdin and invoke a function of our choice when it is done doing so. This means main is no longer an active procedure; it returns leaving it up to the runtime to keep the process running until it has input from stdin to hand back to our function handleUserInput.

Here's what the stack trace looks like:

file://addNumbers.mjs:42
    throw new Error('cannot get string length')
    ^
Error: cannot get string length
    at stringLength (file://addNumbers.mjs:42:11)
    at splitString (file://addNumbers.mjs:24:17)
    at parseString (file://addNumbers.mjs:18:19)
    at ReadStream.handleUserInput (file://addNumbers.mjs:11:5)
    at ReadStream.emit (node:events:511:28)
    at addChunk (node:internal/streams/readable:332:12)
    at readableAddChunk (node:internal/streams/readable:305:9)
    at Readable.push (node:internal/streams/readable:242:10)
    at TTY.onStreamRead (node:internal/stream_base_commons:190:23)

No sign of main, only handleUserInput.

This is a common hazard of asynchronous programming: you are always replacing the record of your active procedures as they are all performing task setup that completes and the callbacks they created are later invoked by the runtime.

JavaScript 💚

Asynchronous programming has always been at the heart of JS and is one of the central selling points of using Node.js.

In 2015, the first Long Term Support version of Node.js was released, and with it came a stable standard library that popularized a common pattern for handling asynchronous tasks. All asynchronous tasks would accept a callback as their last argument, with the callback taking at least two arguments: an error, and the task's result. The pattern was that if the first argument was truthy (an error object) the task failed, and if it was not then the second argument would contain the result.

Here's a simplified example of a function that reads a file:

readFile('filename.txt', (error, data) => {
  if (error) {
    console.error(error);
    return;
  }
  console.log('file contents', data);
})

The Node.js callback pattern is ubiquitous and familiar, resulting in many popular libraries such as the MongoDB Node.js driver adopting it as well.

No throw, only callback 🐕

image credit: cupcakelogic

A challenge associated with the callback pattern is the requirement that the implementer keep in mind execution expectations manually, otherwise they can end up with a confusing order of operations.

Typically this is something that should be abstracted to the runtime or language, which can be broken down as follows:

Error handling

Properly implementing the callback pattern means errors are passed as variables to a chain of handlers so they eventually reach the top-level initiator of the async operation. The syntax and keywords throw/try/catch can no longer be used for control flow.

try {
  readFile('filename', (error, data) => {
    if (error) { /* ? */ }
  })
} catch (error) {
  // So what's the truth?
}

Runtime order

Callbacks also demand the developers ensure execution order is consistent. If a file is successfully read and the contents are returned in the callback passed to readFile, that callback will always run after the code that is on the line following readFile. However, say readFile is passed an invalid argument, like a number instead of a string for the path. When it invokes the callback with an invalid argument error we would still expect that code to run in the same order as the success case:

function readFile(filename, callback) {
   if (typeof filename !== 'string') {
       callback(new Error('invalid argument'))
       return;
   }
   // open & read file ...
}

readFile(0xF113, (error, data) => {
   if (error) {
       console.log('cannot read file', error)
       return;
   }
   console.log('contents:', data)
})
console.log('starting to read file')

The code above prints:

cannot read file Error: invalid argument
starting to read file

Whereas when I change readFile to be called with a non-existent path:

starting to read file
cannot read file Error: /notAPath.txt Does Not Exist

This is unexpected! The implementer of readFile calls the callback synchronously for an invalid type so readFile does not return until that callback completes. It is fairly easy to write callback accepting functions that inconsistently order their execution in this way.

Promises 🤞

Introducing a more structured approach: Promises. A Promise is an object that handles the resolution or rejection of an async operation, mitigating the above issues and allowing for many async operations to be chained together without needing to explicitly pass a finalizer callback through to each API that would indicate when all tasks are done.

// callbacks
client.connect((error) => {
 if (error) {
   return done(error);
 }
 client
   .db()
   .collection('test')
   .findOne({}, (error, document) => {
     if (error) {
       return done(error);
     }
     console.log(document);
     return done();
   });
});

// promises
client
 .connect()
 .then(() => client.db().collection('test').findOne({}))
 .then(document => console.log(document));
 .catch(error => console.error(error));

Note how in the promise code there is one error handling case as opposed to the two in the callback case. The ability to chain promises allows us to treat many async operations as one, the catch handler would be called if either the connect or the find methods were to throw an error. This chaining is convenient, but when writing JavaScript today we do even better by using special syntax for handling promises.

Enter `async`/`await` 🔁

Mid-2017 JavaScript engines shipped support for async/await syntax allowing programmers to write asynchronous operations in a familiar procedural format. Using async/await allows the programmer to encode their logical asynchronous dependencies right into the syntax of the language.

Let's return to our user input example, as we can now "await" the input which keeps main as the active procedure that began the task to read from standard in.

"For await the suspend and resume points coincide and so we not only know where we would continue, but by coincidence, we also know where we came from."

source: Zero-cost async stack traces

When the input is available, readStdin will resolve and we can continue with our parsing.

async main()
  -> await readStdin()
  -> parseString()

file://addNumbers.mjs:43
    throw new Error('cannot get string length')
          ^
Error: cannot get string length
    at stringLength (file://addNumbers.mjs:43:11)
    at splitString (file://addNumbers.mjs:25:17)
    at parseString (file://addNumbers.mjs:19:19)
    at main (file://addNumbers.mjs:9:5)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file://addNumbers.mjs:62:1

When the JavaScript engine reaches the "await", main is suspended. The engine is free to handle other tasks while the read is waiting for our user to type. We can now encode into the syntax of the function that it will suspend until some other task completes, and when it continues it maintains the context of everything that was in scope when it started.

try {
 await client.connect();
 const document = await client.db().collection('test').findOne({});
 console.log(document);
} catch (error) {
 console.error(error);
}

"The fundamental difference between await and manually constructed promises is that await X() suspends execution of the current function, while promise.then(X) will continue execution of the current function after adding the X call to the callback chain. In the context of stack traces, this difference is pretty significant."

source: Why await beats Promise#then() · Mathias Bynens

Sample Stack Traces

Prior to completing the async/await conversion down to the internal network layer of the driver, our error stack would begin at the point of converting a server's error message into a JavaScript, such as:

MongoServerError: Failing command via 'failCommand' failpoint
    at Connection.onMessage (./mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (./mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (node:events:520:28)
    at processIncomingData (./mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (./mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)
    at Writable.write (node:internal/streams/writable:502:10)
    at Socket.ondata (node:internal/streams/readable:1007:22)
    at Socket.emit (node:events:520:28)
                    ^-- Sadness, that's not my code...

Now, post v6.5.0, the stack trace points directly back to the origination of an operation (we see you main.js!):

MongoServerError: Failing command via 'failCommand' failpoint
    at Connection.sendCommand (./mongodb/lib/cmap/connection.js:290:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Connection.command (./mongodb/lib/cmap/connection.js:313:26)
    at async Server.command (./mongodb/lib/sdam/server.js:167:29)
    at async FindOperation.execute (./mongodb/lib/operations/find.js:34:16)
    at async tryOperation (./mongodb/lib/operations/execute_operation.js:192:20)
    at async executeOperation (./mongodb/lib/operations/execute_operation.js:69:16)
    at async FindCursor._initialize (./mongodb/lib/cursor/find_cursor.js:51:26)
    at async FindCursor.cursorInit (./mongodb/lib/cursor/abstract_cursor.js:471:27)
    at async FindCursor.fetchBatch (./mongodb/lib/cursor/abstract_cursor.js:503:13)
    at async FindCursor.next (./mongodb/lib/cursor/abstract_cursor.js:228:13)
    at async Collection.findOne (./mongodb/lib/collection.js:274:21)
    at async main (./mongodb/main.js:19:3)
                   ^-- Yay, that's my code!

Additional Resources

Peeling the MongoDB Drivers Onion

Alex Bevilacqua — Tue, 21 May 2024 18:59:35 +0000

The modern MongoDB driver consists of a number of components, each of which are thoroughly documented in the Specifications repository. Though this information is readily available and extremely helpful, what it lacks is a high level overview to tie the specs together into a cohesive picture of what a MongoDB driver is.

Architecturally an implicit hierarchy exists within the drivers, so expressing drivers in terms of an onion model feels appropriate.

Layers of the Onion

The "drivers onion" is meant to represent how various concepts, components and APIs can be layered atop each other to build a MongoDB driver from the ground up, or to help understand how existing drivers have been structured. Hopefully this representation of MongoDB’s drivers helps provide some clarity, as the complexity of these libraries - like the onion above - could otherwise bring you to tears.

Serialization

At their lowest level all MongoDB drivers will need to know how to work with BSON. BSON (short for "Binary JSON") is a binary-encoded serialization of JSON-like documents, and like JSON, it supports the nesting of arrays and documents. BSON also contains extensions that allow representation of data types that are not part of the JSON spec.

Specifications: BSON, ObjectId, Decimal128, UUID, DBRef, Extended JSON

Communication

Once BSON documents can be created and manipulated, the foundation for interacting with a MongoDB host process has been laid. Drivers communicate by sending database commands as serialized BSON documents using MongoDB’s wire protocol.

From the provided connection string and options a socket connection is established to a host, which an initial handshake verifies is in fact a valid MongoDB connection by sending a simple hello. Based on the response to this first command a driver can continue to establish and authenticate connections.

Specifications: OP_MSG, Command Execution, Connection String, URI Options, OCSP, Initial Handshake, Wire Compression, SOCKS5, Initial DNS Seedlist Discovery

Connectivity

Now that a valid host has been found, the cluster’s topology can be discovered and monitoring connections can be established. Connection pools can then be created and populated with connections. The monitoring connections will subsequently be used for ensuring operations are routed to available hosts, or hosts that meet certain criteria (such as a configured read preference or acceptable latency window).

Specifications: SDAM, CMAP, Load Balancer Support

Authentication

Establishing and monitoring connections to MongoDB ensures they’re available, but MongoDB server processes typically will require the connection to be authenticated before commands will be accepted. MongoDB offers many authentication mechanisms such as SCRAM, x.509, Kerberos, LDAP, OpenID Connect and AWS IAM, which MongoDB drivers support using the Simple Authentication and Security Layer (SASL) framework.

Specifications: Authentication

Availability

All client operations will be serialized as BSON and sent to MongoDB over a connection that will first be checked out of a connection pool. Various monitoring processes exist to ensure a driver’s internal state machine contains an accurate view of the cluster’s topology so that read and write requests can always be appropriately routed according to MongoDB’s server selection algorithm.

Specifications: Server Monitoring, SRV Polling for mongos Discovery, Server Selection, Max Staleness

Resilience

At their core, database drivers are client libraries meant to facilitate interactions between an application and the database. MongoDB’s drivers are no different in that regard, as they abstract away the underlying serialization, communication, connectivity, and availability functions required to programmatically interact with your data.

To further enhance the developer experience while working with MongoDB, various resilience features can be added based on logical sessions such as retryable writes, causal consistency, and transactions.

Specifications: Retryability (Reads, Writes), CSOT, Consistency (Sessions, Causal Consistency, Snapshot Reads, Transactions, Convenient Transactions API)

Programmability

Now that we can serialize commands and send them over the wire through an authenticated connection we can begin actually manipulating data. Since all database interactions are in the form of commands, if we wanted to remove a single document we might issue a delete command such as the following:

db.runCommand(
  {
     delete: "orders",
     deletes: [ { q: { status: "D" }, limit: 0 } ]
  }
)

Though not exceedingly complex, a better developer experience can be achieved through more single-purpose APIs. This would allow the above example to be expressed as:

db.orders.deleteMany({ status: "D" })

To provide a cleaner and clearer developer experience, many specifications exist to describe how these APIs should be consistently presented across driver implementations, while still providing the flexibility to make APIs more idiomatic for each language.

Advanced security features such as client-side field level encryption are also defined at this layer.

Specifications: Resource Management (Databases, Collections, Indexes), Data Management (CRUD, Collation, Write Commands, Bulk API, Bulk Write, R/W Concern), Cursors (Change Streams, find/getMore/killCursors), GridFS, Stable API, Security (Client Side Encryption, BSON Binary Subtype 6)

Observability

With database commands being serialized and sent to MongoDB servers and responses being received and deserialized, our driver can be considered fully functional for most read and write operations. As MongoDB drivers abstract away most of the complexity involved with creating and maintaining the connections these commands will be sent over, providing mechanisms for introspection into a driver’s functionality can provide developers with added confidence that things are working as expected.

The inner workings of connection pools, connection lifecycle, server monitoring, topology changes, command execution and other driver components are exposed by means of events developers can register listeners to capture. This can be an invaluable troubleshooting tool and can help facilitate monitoring the health of an application.

const { MongoClient, BSON: { EJSON } } = require('mongodb');

function debugPrint(label, event) {
 console.log(`${label}: ${EJSON.stringify(event)}`);
}

async function main() {
 const client = new MongoClient("mongodb://localhost:27017", { monitorCommands: true });
 client.on('commandStarted', (event) => debugPrint('commandStarted', event));
 client.on('connectionCheckedOut', (event) => debugPrint('connectionCheckedOut', event));
 await client.connect();
 const coll = client.db("test").collection("foo");
 const result = await coll.findOne();
 client.close();
}
main();

Given the example above (using the Node.js driver) the specified connection events and command events would be logged as they’re emitted by the driver:

connectionCheckedOut: {"time":{"$date":"2024-05-17T15:18:18.589Z"},"address":"localhost:27018","name":"connectionCheckedOut","connectionId":1}

commandStarted: {"name":"commandStarted","address":"127.0.0.1:27018","connectionId":1,"serviceId":null,"requestId":5,"databaseName":"test","commandName":"find","command":{"find":"foo","filter":{},"limit":1,"singleBatch":true,"batchSize":1,"lsid":{"id":{"$binary":{"base64":"4B1kOPCGRUe/641MKhGT4Q==","subType":"04"}}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1715959097,"i":1}},"signature":{"hash":{"$binary":"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"00"}},"keyId":0}},"$db":"test"},"serverConnectionId":140}

The preferred method of observing internal behavior would be through standardized logging once it is available in all drivers (DRIVERS-1204), however until that time only event logging is consistently available. In the future additional observability tooling such as Open Telemetry support may also be introduced.

Specifications: Command Logging and Monitoring, SDAM Logging and Monitoring, Standardized Logging, Connection Pool Logging

Testability

Ensuring existing as well as net-new drivers can be effectively tested for correctness and performance, most specifications define a standard set of tests using YAML tests to improve driver conformance. This allows specification authors and maintainers to describe functionality once with the confidence that the tests can be executed alike by language-specific test runners across all drivers.

Though the unified test format greatly simplifies language-specific implementations, not all tests can be represented in this fashion. In those cases the specifications may describe tests to be manually implemented as prose. By limiting the number of prose tests that each driver must implement, engineers can deliver functionality with greater confidence while also minimizing the burden of upstream verification.

Specifications: Unified Test Format, Atlas Data Federation Testing, Performance Benchmarking, BSON Corpus, Replication Event Resilience, FAAS Automated Testing, Atlas Serverless Testing

Conclusion

Most (if not all) the information required to build a new driver or maintain existing drivers technically exists within the specifications, however without a mental mode of their composition and architecture it can be extremely challenging to know where to look.

Peeling the "drivers onion" should hopefully make reasoning about them a little easier, especially with the understanding that everything can be tested to validate individual implementations are "up to spec".

MongoDB and Load Balancer Support

Alex Bevilacqua — Fri, 15 Mar 2024 01:53:45 +0000

Overview

A load balancer enhances your application's scalability, availability, and performance by efficiently distributing traffic across multiple servers based on a number of algorithms and techniques - but what about your database? MongoDB is a distributed database, but can it be placed behind a load balancer?

Astute readers of MongoDB's Node.js driver's test README may have noticed at some point that there is mention of a testing methodology for load balancers, and as some in the community have found you can find public SERVER tickets that also allude to this functionality existing.

Digging further you'll find the Load Balancer Support specification for MongoDB's drivers which states "To specify to the driver to operate in load balancing mode, a connection string option of loadBalanced=true MUST be added to the connection string" ... but how do you actually make that work?

In this post we're going to explore why MongoDB nodes couldn't previously be placed behind an L4 load balancer, and what changed in MongoDB 5.3 that may actually make this possible!

Replication

Coordination of data distribution and ensuring high availability is done via replication, which requires the cluster to be aware at all times which node is the primary and which are secondaries.

As there can only be one primary, any application targeting the cluster will need to be aware of the current topology as well, as trying to write to a secondary will fail:

require 'mongo'
# connect directly to a secondary host in a local replica set
client = Mongo::Client.new('mongodb://localhost:27018/test?directConnection=true')
collection = client[:foo]
collection.insert_one bar: "baz"

# => Mongo::Error::OperationFailure: [10107:NotWritablePrimary]: not primary (on localhost:27018, legacy retry, attempt 1)

All official MongoDB drivers implement the Server Discovery and Monitoring specification to ensure applications can route requests to the appropriate servers (as outlined in the Server Selection specification). When you have a single application instance with a single connection pool (as outlined in the Connection Monitoring and Pooling specification) the number of connections to the cluster is easy to identify, but application deployment configurations can vary and scale.

Thanks to MongoDB drivers all consistently providing connection monitoring and pooling functionality, external connection pooling solutions aren't required (ex: Pgpool, PgBouncer). This allows applications built using MongoDB drivers to be resilient and scalable out of the box, but based on what we understand regarding the number of connections applications establish to MongoDB clusters it stands to reason that at a certain point as our application deployments increase, so will our connections.

Can I use a load balancer though?

Due to the need for these additional monitoring connections it has been difficult (impossible?) to place a load balancer between applications and a MongoDB replica set - though adventurous users have developed some interesting HAProxy configurations in the past to try and solve this problem. The problem you'd face is that though read requests can be routed to any available server, write requests must target the cluster primary.

For the sake of argument you may ask "what if I had a 100% read workload?". In that case you could put your hosts behind a load balancer, but you'll likely run into issues as soon as you try and iterate a cursor (see getMore). Operations such as find or aggregate return a cursor (cursorId) which only exists on the originating server the command targeted. Attempting to execute a getMore on the wrong server will result in a CursorNotFound error being returned, which can be challenging to troubleshoot.

Sharding

Fortunately, MongoDB already offers a form of "load balancing" for sharded clusters in the form of the sharded cluster query router (mongos).

Assuming the cluster is sharded and if there is more than one mongos instance in the connection seed list, the driver determines which mongos is the "closest" (i.e. the member with the lowest average network round-trip-time) and calculates the latency window by adding the average round-trip-time of this "closest" mongos instance and the localThresholdMS. The driver will load balance randomly across the mongos instances that fall within the latency window.

Can I use a load balancer though?

Sharding introduces a routing layer between the application and the cluster members, which slightly simplifies how drivers route operations as there is no longer a need to track replica set state. You may think this would make placing a pool of mongos' behind a load balancer straightforward, but as Craig Wilson describes in a 2013 blog post, similar issues will still arise when trying to iterate cursors. Note that though Craig's post references the legacy opcodes, the situation would be the same if using newer drivers that leverage OP_MSG and OP_COMPRESSED.

Note that the Server Selection specification calls out that "Cursor operations [...] do not go through the server selection process. Cursor operations must be sent to the original server that received the query [...]". As this state information would not be tracked within the load balancer, issues would arise if a cursor operation were attempted and a balancer returned a different server where the cursor didn't exist.

`operationCount`-based Server Selection

As it is a form of "load balancing" it's worth just calling out that in an effort to alleviate runaway connection creation scenarios ("connection storms") the drivers approximate an individual server's load by tracking the number of concurrent operations that node is processing (operationCount) and then routing operations to servers with less load. This should reduce the number of new operations routed towards nodes that are busier and thus increase the number routed towards nodes that are servicing operations faster or are simply less busy.

Load Balancer Support

When you see a ticket called "Enable Feature flag for Support for Deploying MongoDB behind a L4 Load Balancer" closed out as fixed for MongoDB
6.0.0-rc0 and 5.3.0-rc3 it's hard not to get excited - but what does this mean? After doing a bit of digging you'll find that mongos' now support a proxy protocol which is configured via the loadBalancerPort startup parameter.

Given that there's a driver specification, driver implementations (such as for the Node.js driver and Ruby driver) and server support it should be possible to configure a sharded cluster to utilize the proxy protocol.

Before we proceed it's worth calling out that this is not considered an officially supported configuration. Until MongoDB's server team promotes this as a valid production configuration it should be considered experimental if used with a self-managed deployment.

Configuration

For our test we'll be configuring a single-shard sharded cluster with 5 mongos' behind an HAProxy load balancer. Assuming you're already familiar with HAProxy and load balancing concepts, we'll be setting up a TCP proxy to perform roundrobin balancing.

1. Setup a local sharded cluster

First we need a local sharded cluster, which we'll provision using m - the MongoDB version manager and mtools.

m 7.0.6-ent
mlaunch init --replicaset --nodes 3 --shards 1 --csrs --mongos 5 --binarypath $(m bin 7.0.6-ent) --bind_ip_all
mlaunch stop

This configuration will yield a single shard with 3 nodes, a config server replica set and 5 mongos'. Once started, we immediately stop the cluster as some additional (manual) configuration is required.

2. Update the cluster configuration to enable proxy protocol

Since we need to modify the startup parameters for our mongos' we'll update the configuration file that mlaunch (part of mtools) uses.

sed -i '' 's/ --port 27017 / --port 27017 --setParameter loadBalancerPort=37017 /g' data/.mlaunch_startup
sed -i '' 's/ --port 27018 / --port 27018 --setParameter loadBalancerPort=37018 /g' data/.mlaunch_startup
sed -i '' 's/ --port 27019 / --port 27019 --setParameter loadBalancerPort=37019 /g' data/.mlaunch_startup
sed -i '' 's/ --port 27020 / --port 27020 --setParameter loadBalancerPort=37020 /g' data/.mlaunch_startup
sed -i '' 's/ --port 27021 / --port 27021 --setParameter loadBalancerPort=37021 /g' data/.mlaunch_startup
mlaunch start

The above commands just append a setParameter call as a command line option so we can configure the loadBalancerPort parameter of each mongos. Once completed we restart the cluster.

3. Configure HAproxy

As we're using HAproxy for our test we'll to build out our custom configuration. The example below will write to a mongodb-lb.conf file, which will then be read by haproxy to create our load balanced endpoint. I'm not going to go into detail as to what all the options below mean, but if you want to investigate further see HAproxy's configuration manual.

tee mongodb-lb.conf > /dev/null <<EOT
global
  log stdout local0 debug
  maxconn 4096

defaults
  log global
  mode tcp
  timeout connect  5000ms
  timeout client  30000ms
  timeout server  30000ms
  retries 3

default-server on-error fastinter error-limit 3 inter 3000ms fastinter 1000ms downinter 300s fall 3

frontend stats
    mode http
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats admin if LOCALHOST

listen mongos
  bind      *:37000
  option    tcplog
  balance   roundrobin
  server    mongos01 *:37017 check send-proxy-v2
  server    mongos02 *:37018 check send-proxy-v2
  server    mongos03 *:37019 check send-proxy-v2
  server    mongos04 *:37020 check send-proxy-v2
  server    mongos05 *:37021 check send-proxy-v2
EOT

haproxy -f mongodb-lb.conf > haproxy.log 2>&1 &

To make monitoring a little easier you'll notice we've enabled HAProxy's stats frontend.

4. Test application connectivity through the load balancer

Since the MongoDB Shell uses the Node.js driver internally we can use to validate if our load balancer is configured properly. We've setup HAProxy to listen on port 37000, so we should be able to connect to that directly:

mongosh --quiet "mongodb://localhost:37000/test"
MongoServerSelectionError: The server is being accessed through a load balancer, but this driver does not have load balancing enabled

Seems the driver knows we're trying to connect to a load balancer, but we're missing an option. This is where the loadBalanced=true option comes into play. Appending this to our connection string will allow us to run an arbitrary workload successfully:

mongosh --quiet "mongodb://localhost:37000/test?loadBalanced=true" --eval "while(true) { result = db.foo.insertOne({ d: new Date() }); print(result); sleep(500); }"
{
  acknowledged: true,
  insertedId: ObjectId('65eb13b122c34af3037c094d')
}
{
  acknowledged: true,
  insertedId: ObjectId('65eb13b222c34af3037c094e')
}
{
  acknowledged: true,
  insertedId: ObjectId('65eb13b222c34af3037c094f')
}
...

Success! It is worth noting though that this configuration works for us locally as we have direct control of the mongos processes startup parameters.

If you have a MongoDB Atlas sharded cluster, the mongos' cannot manually be placed behind a load balancer as startup parameter configuration access is not available!

Conclusion

Now that we can successfully connect to our load balanced endpoint it's worth doing a little chaos testing to see how workloads react. The script I shared previously just loops infinitely inserting documents into a collection - but what happens if we kill one or two mongos processes?

mlaunch stop 27017
mlaunch stop 27019
mlaunch list
Detected mongod version: 7.0.6

PROCESS          PORT     STATUS     PID

mongos           27017    down       -
mongos           27018    running    28006
mongos           27019    down       -
mongos           27020    running    28013
mongos           27021    running    28016

config server    27025    running    27979

shard01
    mongod       27022    running    27994
    mongod       27023    running    27998
    mongod       27024    running    27991

Using mlaunch I just stopped two of the query routers and waited for a while. The inserts kept on - inserting - so I guess we can consider that a successful test. Note that this is obviously not extensive and should not be taken as a guarantee of any sort, but if this is a configuration that interests you give it a shot and let me know what you find.

Don't forget that you have a web-based stats UI configured that you can refer to 😉.

Understanding client library and database preferences for JS/TS developers

Alex Bevilacqua — Tue, 12 Mar 2024 20:08:04 +0000

I'm a Product Manager focusing on Developer Interfaces and am looking to better understand what client libraries and databases JavaScript/TypeScript developers currently prefer. If you're a JavaScript/TypeScript developer and have 5 minutes to spare I'd love to hear from you!

Click here for the survey

Since everyone's time is valuable, if you complete this survey your email address will be entered into a draw to win a $50 Amazon gift card!

I'll have the survey open until March 22nd, 2024. Once the results are processed I'll share my findings here in case others find it useful.

Node.js Driver failing to connect due to "unsafe legacy renegotiation disabled"

Alex Bevilacqua — Tue, 05 Mar 2024 12:54:32 +0000

Overview

In this community forum post there was a report of the MongoDB Node.js driver failing to connect with the following error:

MongoServerSelectionError: C8320000:error:0A000152:SSL routines:final_renegotiate:unsafe legacy renegotiation disabled:c:\ws\deps\openssl\openssl\ssl\statem\extensions.c:922

This error doesn't smell like a MongoDB-specific error, so digging into "final_renegotiate:unsafe legacy renegotiation disabled" specifically lead to this openssl issue that looks to elaborate on the meaning of the error message:

TLSv1.2 (and earlier) support the concept of renegotiation. In 2009 (i.e. after the TLSv1.2 RFC was published), a flaw was discovered with how renegotiation works that could lead to an attack. After the attack was discovered a fix was deployed to all TLS libraries. In order for the fixed version of renegotiation to work both the client and the server need to support it.

The original (unfixed) version of renegotiation is known as "unsafe legacy renegotiation" in OpenSSL. The fixed version is known as "secure renegotiation". So either a peer does not have the fix, in which case it will be using "unsafe legacy renegotiation", or it does have the fix in which case it will be using "secure renegotiation".

So it seems that the error originated from OpenSSL, and "that flaw" they're alluding to was likely CVE-2009-3555. What was particularly interesting about this issue is that it only occurred when the application was run using Node.js 20, while Node.js 16 didn't exhibit any issues - so what's different between those two versions? One notable change is that Node.js 17+ use OpenSSL 3.0 by default - and starting with 3.0 secure negotiation support is required by default.

For more information on secure server-side renegotiation I'd highly recommend this discussion.

Configuring OpenSSL via the Node.js Driver

A similar issue was reported on Stack Overflow for the axios library, and the solution there was to pass secureOptions: crypto.constants.SSL_OP_LEGACY_SERVER_CONNECT during request creation. As secureOptions is an option passed to Node's tls.createSecureContext API (which MongoDB documents an example of using) it should be possible to do something similar with the Node.js driver.

import { MongoClient } from 'mongodb';
import { * as crypto } from 'crypto';

const client = new MongoClient("mongodb+srv://...", {
  secureContext: {
    secureOptions: crypto.constants.SSL_OP_LEGACY_SERVER_CONNECT
  }
});

SUCCESS! The above example allows a SecureContext object to be created with the secureOptions selected from the enumerated OpenSSL options Node.js has defined.

Though the Node.js driver allows direct configuration of the SecureContext object, as other MongoDB drivers may not, DRIVERS-2823 is being considered to ensure this type of configuration is available.

Alternative Configuration

Configuring the MongoDB Node.js driver's OpenSSL options directly is likely the preferred approach, but the Node runtime can also be configured (via --openssl-config=file). In this, when the node process is executed the path to a custom OpenSSL configuration file could be provided as follows:

node --openssl-config=/path/to/openssl.conf

Where openssl.conf is setup similar to the example below:

nodejs_conf = openssl_init

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation

querySrv errors when connecting to MongoDB Atlas

Alex Bevilacqua — Thu, 29 Feb 2024 15:16:46 +0000

If your application uses MongoDB's Node.js driver or Mongoose ODM, occasionally you may observe errors such as querySrv ECONNREFUSED _mongodb._tcp.cluster0.abcde.mongodb.net or Error: querySrv ETIMEOUT _mongodb._tcp.cluster0.abcde.mongodb.net being thrown. The MongoDB Atlas documentation outlines several methods to troubleshoot connection issues, including how to handle "Connection Refused using SRV Connection String" scenarios, but why does this happen in the first place?

About DNS seedlists

To coincide with the release of MongoDB 3.6, all drivers (at the time) implemented the initial DNS seedlist discovery specification to ensure connections could be established using the new SRV connection string format, as well as the legacy standard connection string format.

This functionality was introduced to abstract away the complexity of MongoDB's connection strings (for MongoDB Atlas users at least) by moving the component parts of a connection string to two DNS records: a service record (SRV) and a text record (TXT).

Users now only need to supply a connection string such as mongodb+srv://<username>:<password>@cluster0.abcde.mongodb.net/myFirstDatabase, and regardless as to whether the underlying cluster was a replica set or sharded, the connection string would remain the same. Furthermore, use of mongodb+srv:// enables drivers to detect additions/removals of mongos in a sharded cluster¹

Tools such as nslookup or dig can be used to view the contents of these DNS records, such as in the following example:

$ dig srv _mongodb._tcp.cluster0.abcde.mongodb.net

; <<>> DiG 9.18.18-0ubuntu0.22.04.1-Ubuntu <<>> srv _mongodb._tcp.cluster0.abcde.mongodb.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24529
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;_mongodb._tcp.cluster0.abcde.mongodb.net. IN SRV

;; ANSWER SECTION:
_mongodb._tcp.cluster0.abcde.mongodb.net. 60 IN SRV 0 0 27017 cluster0-shard-00-01.abcde.mongodb.net.
_mongodb._tcp.cluster0.abcde.mongodb.net. 60 IN SRV 0 0 27017 cluster0-shard-00-02.abcde.mongodb.net.
_mongodb._tcp.cluster0.abcde.mongodb.net. 60 IN SRV 0 0 27017 cluster0-shard-00-00.abcde.mongodb.net.

$ dig txt cluster0.abcde.mongodb.net

; <<>> DiG 9.18.18-0ubuntu0.22.04.1-Ubuntu <<>> txt cluster0.abcde.mongodb.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35223
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;cluster0.abcde.mongodb.net.    IN  TXT

;; ANSWER SECTION:
cluster0.abcde.mongodb.net. 60 IN   TXT "authSource=admin&replicaSet=atlas-abcde-shard-0"

What can go wrong?

MongoDB's drivers require the information from both DNS queries in order to successfully establish, authenticate and authorize a connection to a MongoDB Atlas cluster. If either of these queries fail, only part of the connection string details will be present, and if the driver doesn't error out right away, the subsequent connection attempt may be missing necessary information.

For example, per the Authentication specification regarding connection string options, when it comes to selecting the authentication source:

if authSource is specified, it is used.
otherwise, if database is specified (in the connection string), it is used.
otherwise, the admin database is used.

Given this order of operations, if the SRV record resolves, but the TXT record doesn't, assuming the driver doesn't error out first the database provided in the connection string will be used for authentication. Using our original example of mongodb+srv://<username>:<password>@cluster0.abcde.mongodb.net/myFirstDatabase, the myFirstDatabase database will be used to authenticate .... which will result in an authentication failure such as MongoServerError: Authentication failed.

Furthermore, though MongoDB's drivers support automatic retryability of reads and writes, these DNS query failures aren't retryable. There is currently a project proposed (DRIVERS-2757) to improve this in the future, but for now these errors bubble up to the application immediately.

Can these issues be prevented?

The best way to avoid these issues entirely is to just use the legacy standard connection string format. If you're connecting to a replica set, the server discovery and monitoring functionality of each driver will ensure topology changes are automatically discovered.

Note that this will prevent new mongos' from being discovered in a sharded cluster, however if you don't anticipate these to change frequently this will likely be a non-issue as well.

Are other drivers affected?

Failure to resolve DNS records can affect all MongoDB drivers, however it's highly unlikely you'll actually encounter this in a production setting. As there still remains a non-zero chance this issue will manifest, here are some examples of failures you may see from other drivers:

Ruby driver

Mongo::Error::NoSRVRecords: The DNS query returned no SRV records for 'cluster0.abcde.mongodb.net'

Java driver or Spring Boot MongoDB

# Example 1
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, srvResolutionException=com.mongodb.MongoConfigurationException: Unable to look up SRV record for host cluster0-shard-00-01.abcde.mongodb.net, servers=[]}


# Example 2
Caused by: com.mongodb.MongoConfigurationException: Unable to look up SRV record for host cluster0.abcde.mongodb.net
        at com.mongodb.internal.dns.DnsResolver.resolveHostFromSrvRecords(DnsResolver.java:79)
        at com.mongodb.ConnectionString.<init>(ConnectionString.java:321)
        at com.mongodb.MongoClientURI.<init>(MongoClientURI.java:234)

Python driver

# Example 1
pymongo.errors.ConfigurationError: The DNS query name does not exist: _mongodb._tcp.cluster0.abcde.mongodb.net.

# Example 2
Exception has occurred: ConfigurationError
The DNS operation timed out after 20.001205682754517 seconds
dns.exception.Timeout: The DNS operation timed out after 20.001205682754517 seconds

C#/.NET driver

# Example 1
cluster0.abcde.mongodb.net IN TXT on x.x.x.x:53 timed out or is a transient error. A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

# Example 2
DnsClient.DnsResponseException:
at DnsClient.LookupClient.ResolveQuery (DnsClient, Version=1.6.0.0, Culture=neutral, PublicKeyToken=4574bb5573c51424)
at DnsClient.LookupClient.QueryInternal (DnsClient, Version=1.6.0.0, Culture=neutral, PublicKeyToken=4574bb5573c51424)
at DnsClient.LookupClient.Query (DnsClient, Version=1.6.0.0, Culture=neutral, PublicKeyToken=4574bb5573c51424)
at MongoDB.Driver.Core.Misc.DnsClientWrapper.ResolveTxtRecords (MongoDB.Driver.Core, Version=2.15.1.0, Culture=neutral, PublicKeyToken=null)
at MongoDB.Driver.Core.Configuration.ConnectionString.Resolve (MongoDB.Driver.Core, Version=2.15.1.0, Culture=neutral, PublicKeyToken=null)
at MongoDB.Driver.MongoUrl.Resolve (MongoDB.Driver, Version=2.15.1.0, Culture=neutral, PublicKeyToken=null)
at MongoDB.Driver.MongoClientSettings.FromUrl (MongoDB.Driver, Version=2.15.1.0, Culture=neutral, PublicKeyToken=null)

# Example 3
System.TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0200000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [], DnsMonitorException : "DnsClient.DnsResponseException: Query 54148 => _mongodb._tcp.cluster0.abcde.mongodb.net IN SRV on x.x.x.x:53 timed out or is a transient error.
 ---> System.Net.Sockets.SocketException (110): Connection timed out
   at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at DnsClient.DnsUdpMessageHandler.Query(IPEndPoint server, DnsRequestMessage request, TimeSpan timeout)
   at DnsClient.LookupClient.ResolveQuery(IReadOnlyList`1 servers, DnsQuerySettings settings, DnsMessageHandler handler, DnsRequestMessage request, LookupClientAudit audit)
   --- End of inner exception stack trace ---
   at DnsClient.LookupClient.ResolveQuery(IReadOnlyList`1 servers, DnsQuerySettings settings, DnsMessageHandler handler, DnsRequestMessage request, LookupClientAudit audit)
   at DnsClient.LookupClient.QueryInternal(DnsQuestion question, DnsQuerySettings queryOptions, IReadOnlyCollection`1 servers)
   at DnsClient.LookupClient.Query(DnsQuestion question)
   at DnsClient.LookupClient.Query(String query, QueryType queryType, QueryClass queryClass)

PHP driver

Fatal error: Uncaught MongoDB\Driver\Exception\InvalidArgumentException: Failed to parse URI options: Failed to look up SRV record "_mongodb._tcp.cluster0.abcde.mongodb.net": The requested name is valid but does not have an IP address.

Go driver

error parsing command line options: error parsing uri: lookup cluster0.abcde.mongodb.net on x.x.x.x:53: cannot unmarshal DNS message

Note that (per the MongoDB Go driver documentation):

Building with Go 1.11+ and using connection strings with the mongodb+srv scheme is unfortunately incompatible with some DNS servers in the wild due to the change introduced in https://github.com/golang/go/issues/10622. You may receive an error with the message "cannot unmarshal DNS message" while running an operation when using DNS servers that non-compliantly compress SRV records. Old versions of kube-dns and the native DNS resolver (systemd-resolver) on Ubuntu 18.04 are known to be non-compliant in this manner. We suggest using a different DNS server (8.8.8.8 is the common default), and, if that's not possible, avoiding the mongodb+srv scheme.

DNS is hard ...

It sure can be, as intermittent/transient network events can also impact MongoDB drivers' ability to resolve DNS queries. The drivers (typically) rely on low-level OS APIs (such as getaddrinfo) for network address and service translation. As such you may occasionally get errors such as MongooseServerSelectionError: getaddrinfo EAI_AGAIN cluster0-shard-00-01.abcde.mongodb.net even when using the legacy (mongodb://) URI scheme.

Footnotes

Sharded clusters could detect additions/removals of mongos' if the driver(s) have implemented the polling SRV records for mongos discovery specification ↩

DEV Community: Alex Bevilacqua

Start With Context: Building the Retrieval Core for Agentic Apps

A framework-agnostic mental model

Approach 1: LangChain for the shortest path from data to grounded answers

Approach 2: LlamaIndex when the center of gravity is the data itself

Approach 3: Haystack when you want explicit, composable pipelines

Why MongoDB

What next?

Persistent multi-agent conversations with the OpenAI Agents SDK and MongoDB

What you'll build

Why MongoDB for sessions

Walkthrough

1. Prerequisites

2. Install

3. Connect

4. Seed and identify

5. Instantiate the session

6. Run

What MongoDB actually stored

Production notes

Try it yourself

Where to go next

MongoDB as the Brain of Modern AI Applications

LangGraph: short-term checkpoints and long-term memory in one database

LangChain: chat history plus hybrid recall

Google ADK: the Sessions-and-Memory model maps cleanly to MongoDB

Dedicated memory layers also fit: Mem0 on MongoDB

Why MongoDB is the best choice here

Cloudflare + MongoDB: How to fix 'Error: Dynamic require of "punycode/" is not supported'

Reproduction

Patching

Summary

MongoDB Drivers and Network Compression

zlib

ZStandard (zstd)

Snappy

Summary

MongoDB Driver Compatibility with MongoDB Servers

What compatibility tables actually mean

What happens if I don't update my drivers

Can my application actually break if I do nothing

Wire Protocol changes

Command removals

Key Takeaways

Call Stack, But Make It Async!

Intro

Calls and how to stack them 📚

The Async wrench 🔧

JavaScript 💚

No throw, only callback 🐕

Promises 🤞

Enter async/await 🔁

Sample Stack Traces

Peeling the MongoDB Drivers Onion

Layers of the Onion

Serialization

Communication

Connectivity

Authentication

Availability

Resilience

Programmability

Observability

Testability

Conclusion

MongoDB and Load Balancer Support

Overview

Replication

Sharding

operationCount-based Server Selection

Load Balancer Support

Configuration

1. Setup a local sharded cluster

2. Update the cluster configuration to enable proxy protocol

3. Configure HAproxy

4. Test application connectivity through the load balancer

Conclusion

Understanding client library and database preferences for JS/TS developers

Node.js Driver failing to connect due to "unsafe legacy renegotiation disabled"

Overview

Enter `async`/`await` 🔁

`operationCount`-based Server Selection