Getting FTDC out of MongoDB Atlas

Zelmar Michelini — Sun, 02 Aug 2026 22:10:03 +0000

Every MongoDB server writes FTDC, Full Time Diagnostic Data Capture, into a diagnostic.data folder next to its log. Roughly 5,700 metrics, once per second, compressed hard enough that days of history fit in a few hundred megabytes. It's what MongoDB support asks for when you open a performance ticket, and it's the difference between "the database was slow" and "the WiredTiger ticket pool hit zero at 14:32:07."

On Atlas there's no such folder you can reach. Not in the UI, and the log download only gives you mongodb.gz and the audit logs. Search for a way around it and you'll find an issue on keyhole from February 2021 asking exactly this, still unanswered.

I maintain Big Hole, an FTDC viewer, so this wall is one I hit often on other people's behalf. Here's the way through it, measured on an Atlas M10 running 8.0.29.

Have Atlas build the bundle

The Admin API has an endpoint that packages FTDC on demand. It only exists in the v1.0 API, I checked the v2 OpenAPI spec, 325 endpoints, it isn't among them, but it works.

Three things you need first:

A programmatic API key (not a database user): Atlas UI → Access Manager → Project Access → Create Application → API Key. Project Owner works. If your organisation requires an access list for API keys, add your IP or the first call returns ORG_REQUIRES_ACCESS_LIST.

The project ID, 24 hex characters, not a UUID. It's in the URL: cloud.mongodb.com/v2/<PROJECT_ID>/clusters.

The replica set name, the internal one (atlas-abc123-shard-0), not the cluster's display name:

curl -sS -u "$PUB:$PRIV" --digest \
  "https://cloud.mongodb.com/api/atlas/v1.0/groups/$GROUP_ID/processes" \
  | jq -r '.results[] | "\(.hostname):\(.port) \(.replicaSetName)"'

The three calls

BASE="https://cloud.mongodb.com/api/atlas/v1.0/groups/$GROUP_ID"
AUTH=(-u "$PUB:$PRIV" --digest -sS)

# 1. create the job
curl "${AUTH[@]}" -X POST "$BASE/logCollectionJobs" \
  -H 'Content-Type: application/json' \
  -d '{"resourceType":"REPLICASET",
       "resourceName":"atlas-abc123-shard-0",
       "redacted":true,
       "sizeRequestedPerFileBytes":100000000,
       "logTypes":["FTDC"]}'
# -> {"id":"6a6bd2dd0fc2dcfc08d68af1"}

# 2. poll until SUCCESS
curl "${AUTH[@]}" "$BASE/logCollectionJobs/6a6bd2dd0fc2dcfc08d68af1"

# 3. download
curl "${AUTH[@]}" "$BASE/logCollectionJobs/6a6bd2dd0fc2dcfc08d68af1/download" -o ftdc.tar.gz
tar xzf ftdc.tar.gz

sizeRequestedPerFileBytes caps each collected file; 100 MB is more than any single FTDC file, so you get everything. resourceType also accepts PROCESS and CLUSTER, REPLICASET is what gets every member in one job.

Mine returned SUCCESS on the first poll. The job response also carries expirationDate (the bundle is kept 30 days) and uncompressedSizeTotalBytes. What comes out:

mongodb-logfiles_atlas-abc123-shard-0_2026-07-30T2240Z/
  ac-...-shard-00-00.mongodb.net/27017/diagnostic.data/metrics.2026-07-30T22-00-06Z-00000
  ac-...-shard-00-00.mongodb.net/27017/diagnostic.data/metrics.interim
  ac-...-shard-00-01.mongodb.net/27017/diagnostic.data/...
  ac-...-shard-00-02.mongodb.net/27017/diagnostic.data/...

The genuine files mongod wrote, one diagnostic.data per member, the layout every FTDC tool already understands.

Don't skip metrics.interim. It's the chunk the server hasn't flushed to a numbered file yet, and it holds the most recent samples. In my bundle the newest numbered file stopped at 22:33 while the interim carried data to 22:38.

I found this endpoint by reading maoertel/mongodb-ftdc, a Rust CLI that automates the whole flow. Five stars. It deserves more.

How far back does it go?

The bundle's metadata document carries getCmdLineOpts, which shows how Atlas starts mongod:

diagnosticDataCollectionDirectorySizeMB: 400

Double the mongod default of 200 MB, and a hard ceiling: when the directory fills, the oldest file is deleted.

How long that lasts depends on load, because FTDC compresses by delta, a metric that never moves costs nearly nothing, one that changes every second costs real bytes. On an idle cluster I measured ~0.93 MB per 32 minutes per node, which extrapolates to a window near ten days. Under real traffic, expect two to five days.

If the incident you're chasing is older than a week, it's gone and no API brings it back. Pull it while it's fresh, that's most of the value of this post.

Why is this so hard?

Worth stating plainly, because none of the above should have taken an afternoon.

FTDC is the first artifact MongoDB support asks for when you open a performance ticket. It contains no user data, it's counters, and I checked: in 228 KB of diagnostic document there are 38 distinct strings, none of which is the name of a database or collection on the cluster. It's the single most useful thing you can hand someone debugging your server.

And on MongoDB's own managed product, the only way to obtain it is an endpoint that appears nowhere in the log download UI, isn't mentioned in the Atlas documentation, and lives on a deprecated API version. The question "how do I get diagnostic data out of Atlas" has been sitting unanswered on GitHub since 2021, so I'm evidently not the first person to lose an afternoon to it.

I doubt it's deliberate, it reads more like nobody owning the problem of making it discoverable. But the effect is real: Atlas customers run one tier of observability behind the engineers who support them, and the gap is filled by community projects with single-digit star counts. A "Download diagnostic data" button next to "Download logs" would close it tomorrow.

A note on monitoring permissions

While working out the minimum privileges a diagnostic collector needs, I found something worth passing on regardless of whether you ever touch FTDC.

clusterMonitor is the role most people grant when something needs to monitor MongoDB. From src/mongo/db/auth/builtin_roles.yml in the server source:

clusterMonitor:
  adminOnly: true
  roles:
    - role: read
      db: config
    - role: read
      db: local      # <-- includes local.oplog.rs

read on local includes local.oplog.rs, and the oplog holds the full content of every recent write. An account with clusterMonitor can page back through everything your application wrote for as long as the oplog window holds, the documents themselves, not statistics about them. It reads as a metrics role and grants a data-reading one.

Monitoring agents often legitimately need it. But if something in your environment holds clusterMonitor purely to scrape counters, it can read your data, and whoever approved that grant probably didn't intend it. For comparison, Atlas accepts custom roles at this granularity:

{
  "roleName": "diagnosticsOnly",
  "actions": [
    {"action": "SERVER_STATUS",      "resources": [{"cluster": true}]},
    {"action": "REPLSET_GET_STATUS", "resources": [{"cluster": true}]},
    {"action": "COLL_STATS",         "resources": [{"db": "local", "collection": "oplog.rs"}]}
  ]
}

I tested a user holding only that: it can't read a collection, can't read the oplog, and listDatabases returns just local, it can't even establish that your application's databases exist.

Reading what you collected

The tarball is ready to open, but the tooling here is thin, which is why I built one.

Big Hole runs entirely in your browser. No backend, no upload, no container: you drop the untarred folder in and it decodes locally. That matters more than it sounds for this particular file, you're often looking at a bundle from someone else's production cluster, and "nothing leaves your machine" is the difference between being allowed to analyse it and not. It opens the Atlas tarball as-is, puts all three members on one time axis, shows who was primary when, and has automated checks for the usual pathologies (ticket pool exhaustion, cache pressure, flow control).

keyhole has been the reference tool for years and renders FTDC through Grafana. If you want dashboards and don't mind running the stack, start there.

Everything above was measured on Atlas M10 / MongoDB 8.0.29 against the current server source. The v1.0 endpoint is on borrowed time; if it stops working, I'd like to know.