DEV Community

Davi Orlandi
Davi Orlandi

Posted on

Walking Hierarchies with MongoDB $graphLookup (Without Melting Memory)

#mongodb #backend #performance #database

The first time $graphLookup melted a primary for me, I was not trying to be clever. We had a category tree. We wanted descendants for an admin screen. Someone omitted maxDepth, the index story on parentId was weak, and a catalog import suddenly gave a root tens of thousands of children. The operator did exactly what we asked. We asked for too much on the wrong path. Latency spiked. Memory followed. The personal stake of this essay is that recursive convenience is not free.

Can MongoDB walk a hierarchy without a closure table? Yes, with guardrails. $graphLookup recursively follows parent and child links into an as array. Keep maxDepth, index connectToField, prune with restrictSearchWithMatch, and keep unbounded walks off customer-facing p99 paths. For hot reads on stable trees, materialized paths or closure tables still win.

How $graphLookup behaves when you respect it

For each input document, MongoDB starts from startWith, matches that value to connectToField in the from collection, then repeatedly follows connectFromField to connectToField until matches stop or maxDepth says stop. Results land in the array named by as. That array is not a guaranteed sorted path. Add depthField if you need levels.

{
  $graphLookup: {
    from: "categories",
    startWith: "$_id",
    connectFromField: "_id",
    connectToField: "parentId",
    as: "descendants",
    maxDepth: 6,
    depthField: "depth",
    restrictSearchWithMatch: { active: true }
  }
}
Enter fullscreen mode Exit fullscreen mode

A minimal hierarchy that makes the direction obvious 

db.categories.insertMany([
  { _id: "root", name: "Catalog" },
  { _id: "elec", name: "Electronics", parentId: "root" },
  { _id: "phones", name: "Phones", parentId: "elec" },
  { _id: "android", name: "Android", parentId: "phones" }
])

db.categories.aggregate([
  { $match: { _id: "android" } },
  {
    $graphLookup: {
      from: "categories",
      startWith: "$parentId",
      connectFromField: "parentId",
      connectToField: "_id",
      as: "ancestors",
      depthField: "depth"
    }
  }
])
Enter fullscreen mode Exit fullscreen mode

Flip the connect fields (and start from a parent id) to walk downward into descendants. The direction is a modeling choice. The cost is not.

Guardrails that would have saved that admin screen

Always set maxDepth when cycles or huge fan-out are possible. Index connectToField and the fields you prune on. Prune early with restrictSearchWithMatch (active, tenant, region). Respect memory: wide graphs are hungry, so prefer shallow selective walks. Sort yourself with depthField plus $sortArray or client-side ordering. Keep writes simple and reads intentional: parent pointers for writes, $graphLookup for rare deep views, cache or materialize for hot paths.

When not to use it

Need Prefer
One level of children $lookup or embedded kids
Frequent ancestor checks on a stable tree Materialized path (root/elec/phones)
Hot "all descendants" reads, rare reparents Closure table
Heavy graph algorithms at huge scale Specialized graph store 
{ _id: "android", path: "root/elec/phones/android" }
Enter fullscreen mode Exit fullscreen mode

That path string looks humble until you realize it turns a recursive read into a prefix query. Humble often wins.

A debugging recipe I trust more than optimism

Run explain("executionStats") on the aggregation. Confirm index use on connectToField. Measure documents landing in as for a worst-case seed. Lower depth or tighten the prune filter. Precompute a closure for the hottest roots if the UI must stay interactive. A useful product trick: return one level first and lazy-load children. Most users never need the entire tree in one payload, and most outages start by pretending they do.

Closing

$graphLookup is safe when bounded, indexed, and pruned. It is unsafe as an unbounded "recurse the catalog" button on a hot endpoint. Use it as a sharp tool for admin and ad hoc hierarchy walks, not as a substitute for a read model you should have materialized. The operator is powerful. Power without a depth limit is just a future incident report.

Top comments (0)