I ran Keploy on my MERN app. The scariest result was a green one.

Nine of my API tests were getting 401 Unauthorized instead of 200. Keploy said the run passed. Exit code 0.

That was the last thing I found. Here's how I got there.

Why I tried this

Honestly, the first reason is that I'm applying for the DevRel internship at Keploy. The second is that I wanted to see how API testing actually works on a real app.

TaskFlow is my MERN project management app. It has an eval harness for its AI features, but the Express API itself had no tests at all. So I pointed Keploy at it and wrote down everything that happened, including the parts that went wrong.

Setup

• App: TaskFlow backend (Express, MongoDB, JWT auth, and AI endpoints that call Groq)
• Keploy: 3.8.44, Free, running natively on my MacBook Air (Apple Silicon), no Docker
• Database: a throwaway local MongoDB, not my real Atlas database
• Traffic: 12 requests covering register, login, workspace, project, task, read, update, delete, one AI call, and one request with a bad token Recording is one command:

keploy record -c "node server.js"

Use node, not nodemon. A restarting process confuses the recorder.

My first attempt just hung for five and a half minutes with no output. It turned out my Desktop folder syncs to iCloud, and most of node_modules had been offloaded, so Node was waiting on iCloud to download files one by one. That's not Keploy's fault. But Keploy also never told me that nothing was listening.

First run: 5 out of 12

When I stopped recording, Keploy immediately replayed what it had just recorded. I didn't ask it to; that's the default. 7 of 12 tests failed.

I did not expect 7 of my endpoints to fail. Then I read the diffs, and none of them were bugs. They were values that are different every time:

• MongoDB _ids. Mongoose generates them inside the app, so they can't come from the mocks.
• JWTs and the refresh-token cookie. They're freshly signed, with new timestamps.
• Etag headers. Express hashes the response body, so if anything in the body changes, the Etag changes too.
• joinedAt. A timestamp. The fix was nine lines of globalNoise in keploy.yml:

test:
  globalNoise:
    global:
      body:
        project._id: ['^[0-9a-f]{24}$']
        task._id: ['^[0-9a-f]{24}$']
        workspace._id: ['^[0-9a-f]{24}$']
        workspace.members._id: ['^[0-9a-f]{24}$']
        user.id: ['^[0-9a-f]{24}$']
        workspace.members.joinedAt: ['^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$']
        accessToken: ['^eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$']
      header:
        Etag: ['^W/"[0-9a-f]+-[A-Za-z0-9+/]{27}"$']
        Set-Cookie: ['^refreshToken=eyJ[A-Za-z0-9_.-]+; Max-Age=604800; Path=/; Expires=[^;]+; HttpOnly; SameSite=Lax$']

I used regex patterns instead of empty arrays on purpose. An empty array ignores the field completely. A pattern still checks that the new value looks right, so a null or a malformed ID would still fail.

What Keploy showed me about my own app

This was the part I didn't expect. Keploy records every database and network call, so reading the mocks felt like reading my backend's diary.

1. A background write I'd forgotten about. When a task is created, TaskFlow computes an embedding and saves it to MongoDB, but it doesn't wait for that before responding. Keploy caught these writes landing in the next test's time window. When they failed, my app just logged embedding failed and moved on. No error, nothing visible to the user.

2. PUT /api/tasks/:id makes 33 database calls. The next-highest endpoint makes 9. I have no idea why yet, and I'm going to find out.

3. A 37 MB download hiding at runtime. The first time a task is created, TaskFlow downloads an embedding model from HuggingFace. Keploy captured all of it, and that one download made up about 98% of mocks.yaml. After I pre-downloaded the model outside Keploy, mocks.yaml went from 37.7 MB to 286 KB.

Green: 12 out of 12

With the noise config and the model cached, I re-recorded and ran:

keploy test -c "node server.js" --delay 10

12 out of 12 passed.

The part I'd actually use every day: the Groq call is mocked too. The AI endpoint test replays the recorded LLM response instead of calling Groq. So it's deterministic, it's free, and it doesn't break because the model phrased something differently today.

One honest caveat: the background embedding write from point 1 is still a race. It only lined up on this run because the cached model made everything faster. The test is green, but the race is still there.

The green run that wasn't

My access tokens expire after 15 minutes. Keploy replays the exact token it recorded. So I waited and ran the tests again, 16 minutes after recording.

Nine tests got 401 Not authorized instead of the recorded 200/201. Here's what Keploy reported:

)
Default run, 16 minutes after recording.

The other nine were marked obsolete, not failed. The report said PASSED, and the process exited with code 0. In CI, that's a green check on top of nine broken endpoints.

Keploy's summary said the mocks were probably stale and suggested re-recording, or running with --update-test-mapping. But the real cause was the 401. Updating the mappings would have quietly rewritten the tests to match the broken behavior.

Here's why it happens:

1. My auth middleware rejects the expired token before touching the database.
2. So the test uses none of the database mocks it recorded.
3. Keploy sees "different mocks than expected" and marks the test obsolete.
4. By default, obsolete doesn't count as a failure. This is documented. The --help text for --assert-dependencies says such a test is "demoted to OBSOLETE today and the run still exits 0." I just hadn't read it until I went looking.

That's what stuck with me: not all bugs are easy to see, and they can still be fatal. A test suite that can't go red isn't protecting you.

What about --freezeTime? It's meant for exactly this: it pins the app's clock to the recording time. But the docs list it as an Enterprise feature for Linux, WSL and Docker. On native macOS, Keploy tried to inject a Linux library, logged an error saying time freezing couldn't be verified, and ran the tests anyway. Same result.

The fix: one flag

Same recording, same expired tokens:

Run	Passed	Failed	Obsolete	Report	Exit code
Default	3	0	9	PASSED	0
--strict-failure	3	9	0	FAILED	1
--assert-dependencies	3	9	0	FAILED	1

)
Same tests, same expired tokens, with --strict-failure.

• --strict-failure fails a test when its response is wrong.
• --assert-dependencies fails a test when its expected database or network calls never happened. That catches cases where the response still matches but the logic behind it has vanished. With --strict-failure, Keploy's summary also named the real cause (expected=200 got=401) instead of blaming stale mocks.

If you run Keploy in CI, add one of these. I'd use both.

Watch your secrets

A short checklist, because I learned this the hard way:

• Outbound calls are recorded with their headers. My Groq API key ended up in mocks.yaml in plain text. Check your mocks before you commit or share anything.
• Report upload and mock upload are separate settings. I turned off report upload, but my mocks were still uploaded to Keploy's registry on the green run. If you don't want that, set test.disableMockUpload: true as well.
• mocks.yaml is gitignored by default. That's good for git, but a search tool that respects .gitignore will skip the one file most likely to contain your keys. I rotated my Groq key. Twice.

Would I recommend it?

Yes, to check your code's reliability before it hits production. Recording real traffic and getting tests plus mocks back, including for LLM calls, is genuinely useful, and the logs explained almost everything I ran into.

My one warning: don't trust the default green. Run with --strict-failure, and read your mocks for secrets.

My last big bug was a token cap silently truncating JSON in 26% of TaskFlow's AI requests, and the lesson here was the same: a green check isn't proof that things work. I wrote about that one here.