DEV Community: FetchSandbox

We planted a Descope privilege-escalation bug — can your coding agent catch it?

FetchSandbox — Fri, 10 Jul 2026 18:39:39 +0000

We open-sourced a repo with a bug planted on purpose: github.com/fetchsandbox/playground.

The bug is in apps/descope — a small FastAPI "Agent Gateway" where AI agents exchange a Descope access key for a scoped session. The flaw: the exchange endpoint trusts whatever scopes the client requests. A read-only agent key can ask for users:write and get it.

This is the privilege-escalation shape that ships constantly in agentic auth. The token exchange returns 200, the happy path works, and nothing ever compares the granted scope against what the key was actually granted.

The 20-minute test drive

Two tasks, both run from your IDE (Cursor, Claude Code, or any MCP-capable editor). No Descope account, no API keys — the workflows run against FetchSandbox's hosted Descope sandbox over MCP, and each app ships a .mcp.json already wired.

Task 1 — greenfield. apps/descope-onboarding is a tiny notes app with placeholder auth. Ask your agent to add Descope OTP sign-up, but with one constraint: prove the OTP + session flow in the sandbox before writing any code, then propose the diff.

./fetchsandbox I'm adding Descope OTP sign-up to this app — prove the Descope
OTP + session flow in the sandbox before writing any code, then propose the
diff. I'll decide whether to apply.

Task 2 — brownfield. Point the agent at the Agent Gateway and ask it to audit:

./fetchsandbox our agent access-key exchange might be handing out more scope
than the key was granted — audit the descope agentic auth

What a real catch looks like

Static review is not the bar. "This looks vulnerable" is a guess. The bar is:

the escalation is reproduced — read-only key in, users:write session out
the proof shows buggy vs fixed on actual Descope routes (/v1/...)
there's a receipt URL — an openable run trace, not a claim in chat

Auth bugs live in the lifecycle: replayed magic links, expired sessions, JWTs that were decoded instead of verified, scope grants nobody re-checked. Mocks return clean 200s for all of these. That's the gap the playground is designed to expose.

The other planted bugs

If Descope isn't your stack, the repo also has broken Stripe (webhook dedup keyed on the wrong header — events processed 2–3×), Resend (bounces silently dropped, users stay "active"), Clerk (session validation skipped on one endpoint), and a few more. Each app is ~50–150 lines of Python.

The part where you tell us it fell flat

Findings go back as a PR — there's a FINDINGS_TEMPLATE.md in the repo. Paste your agent's session, the receipt URLs, and your honest reaction. Merged PRs show up on your GitHub contribution graph.

We explicitly want the failure reports: MCP wouldn't connect, the agent caught nothing, the proof felt fake. An honest "it didn't work" beats a polite green checkmark.

Start here: TESTING.md.

How to test Descope auth without real keys

FetchSandbox — Wed, 08 Jul 2026 14:04:28 +0000

Title options:

How to test Descope auth without real keys
How to test Descope passwordless auth without a project

Descope is easy to start with, but the part I would test first is not the OTP screen or the magic-link email.

It is what your app does with the session JWT after Descope returns it.

FetchSandbox has a Descope sandbox for the flows most apps wire up first:

email OTP signup
email magic-link sign-in
session refresh
agent/access-key exchange

No real Descope project. No real keys. No inbox setup.

The useful part is that the sandbox models the failure cases too: wrong OTP, expired session, replayed magic link, and the big one: accepting a forged session JWT because the app decoded it instead of verifying it.

Here is the concrete flow I would test.

Send a magic link:

POST /v1/auth/magiclink/signin/email

Verify the one-time token:

POST /v1/auth/magiclink/verify

The sandbox returns the shape your app should care about:

{
  "sessionJwt": "...",
  "refreshJwt": "...",
  "user": {
    "userId": "U2..."
  }
}

Call your protected route with the sessionJwt.

This is where the bug usually lives.

Bad handlers do something like:

const claims = jwt.decode(sessionJwt);
if (claims.role === "admin") allow();

That accepts whatever claims are inside the token. A forged alg: none token can become role=admin if the handler never checks the signature.

The fixed version verifies the session JWT against Descope's JWKS, or uses Descope's SDK validation path. It should reject alg: none, reject bad signatures, cache JWKS, and refresh on kid mismatch.

The sandbox route for this bug is meant to prove that difference:

forged unsigned token -> 401
valid signed session token -> 200

That is the whole point. Do not just test "magic link works." Test that the session you trust is actually verified.

You can also test refresh behavior:

POST /v1/auth/refresh
GET /v1/auth/me

If refresh returns 401, send the user back through sign-in. Do not keep rendering authenticated UI from a decoded-but-unverified token.

Docs are here if you want the exact Descope sandbox flows: https://fetchsandbox.com/docs/descope

Our CI mocked Stripe. Production still broke on step two.

FetchSandbox — Tue, 30 Jun 2026 14:23:58 +0000

Green CI. Broken staging. Sound familiar?

Our pipeline mocked POST /v1/payment_intents and asserted 200. Merge button enabled. First real checkout in staging failed because the handler never stored the PaymentIntent ID — step two of the webhook path read undefined.

Mocks proved shape. Not lifecycle.

What unit tests miss

CI check	What it proves	What it skips
Mocked `fetch()`	Request JSON looks right	Step 2 cannot read step 1's ID
Snapshot tests	Handler code unchanged	Webhook branch never runs
Manual smoke doc	Someone remembers sometimes	Agents ship faster than the checklist

The bug class: IDs do not chain across steps. Webhooks fire on mutations your mock never triggers.

What we run now — MCP locally, same workflow in CI

Before an agent touches payment code, we prove the provider flow in the IDE via FetchSandbox MCP:

quickrun stripe / accept_payment
→ create PaymentIntent
→ confirm / capture path
→ read back state

That is the contract the integration must satisfy — not a paragraph in the PR description.

Then CI runs the same curated workflow headless:

# .github/workflows/api-integration.yml
- name: Prove Stripe integration before deploy
  run: npx fetchsandbox run stripe --all --json

Exit code 0 → workflows passed, merge allowed.

Exit code 1 → named step failed (create_payment_intent, webhook reconcile, etc.) — PR blocked.

No staging dependency. No IP whitelist. No "works on my machine."

Why MCP belongs in the loop

Coding agents edit integration code constantly. Without a runnable proof:

The agent "fixes" the handler and moves on
CI mocks still pass
Production breaks on the path the agent never exercised

MCP gives the agent (and you) a receipt: sandbox timeline, step statuses, real provider-shaped IDs from the run — before opening the PR.

CI is the gate that keeps that proof from regressing on the next agent session.

What a workflow run actually checks

Not "does POST return 200 once."

Create a resource → mutate it → GET the same ID back
Trigger the webhook branch when state changes
Fail with a named step when auth or transitions break

That is integration testing — not HTTP cosplay.

What we stopped doing

Treating mocked unit tests as "integration covered"
Running only manual smoke before deploy
Letting agents merge Stripe/Resend/Clerk changes without a workflow receipt

Try it

Full write-up: API integration testing in CI

MCP + agent workflows: Agent API workflow testing

Install MCP in Cursor/Claude → run a Stripe workflow locally → paste the same npx fetchsandbox run step into Actions.

fetchsandbox.com · MCP setup

Our partners kept breaking on staging. So we gave them production access. (Don't do this.)

FetchSandbox — Thu, 25 Jun 2026 16:43:57 +0000

We whitelisted partners on production with read-only test accounts. That was attempt four — after docs-only onboarding, shared UAT, and IP-whitelisted staging all failed in different ways.

I work on an embed platform. Partners integrate our APIs for payments, identity verification, and onboarding flows. We burned six months on one question: how do partners test their integration before go-live?

Attempt 1: "Just use our docs"

We pointed partners at endpoints, auth, request/response examples. "You're good to go."

They'd hit something we didn't document, open a support ticket, wait two days, lose momentum. A few never came back. One went with a competitor because they "couldn't get a working test setup in a week."

The docs described what should happen. Partners had no way to see what actually happens until they wrote code, deployed, and hoped.

Attempt 2: Internal UAT box

Next: give partners our internal UAT environment. IP whitelisted, shared credentials. We told them "please don't break anything" without irony.

Worked for three weeks.

QA pushed a bad config on a Tuesday. The partner's integration broke. They spent two days debugging their own code before opening a ticket. By the time we figured it out, their engineering lead was cc'ing our VP.

Another team ran load tests on UAT the same week. Partner API calls timed out. Their CTO asked if our platform was "production-ready."

UAT was built for us to break things. We invited external partners into that mess.

Attempt 3: Staging with IP whitelisting

We carved out "partner staging" — same codebase, separate deployment, IP whitelisted per partner. Felt like the grown-up solution.

It wasn't.

IP whitelists became a full-time job. Every new partner meant firewall rules. Home IPs differed from office IPs. One CTO traveling couldn't hit the sandbox from his hotel. We debugged VPN configs at 11pm on a Thursday.

Test data went stale. Partners created orders for products that existed in production but not staging. "Your API returns 404 for product_id XYZ." Correct — nobody seeded staging in three weeks.

Deploys collided with demos. Engineers shipped to staging without checking if partners were testing. A deploy during a partner's live demo call gets brought up in quarterly business reviews.

Cost. Full production mirror for partner testing — database, compute, monitoring, on-call — used maybe 10 hours a week.

Attempt 4: Production access

I wish I was kidding.

Reasoning: staging is flaky, UAT is a disaster, whitelist them on production with read-only test accounts.

The API was stable. Partners were happy. For about a month.

One partner's integration bug created 400 orphaned records in production. Data team spent a weekend cleaning up.

Compliance asked why test payloads with fake PII hit the production database.

Maintenance meant coordinating downtime with partners "just running a few test calls."

We'd built the world's most expensive sandbox: production with IP whitelisting.

What we actually needed

Partners didn't need access to our infrastructure.

They needed an API that looked and acted like ours — same endpoints, schemas, auth patterns — but was completely separate. POST creates a resource. GET retrieves it. State transitions work. Nobody else's deploy breaks it.

Not a mock server — those are stateless; step 2 doesn't know about step 1. Not a shared staging box — that's everybody's problem. A dedicated sandbox generated from the same OpenAPI spec the real API uses.

This is half why I started working on FetchSandbox. Give it an OpenAPI spec, get a stateful sandbox — CRUD, state machines, webhook events, seed data. No infrastructure to maintain.

# Partner gets a sandbox from your spec
npx fetchsandbox generate ./your-api-openapi.yaml

curl https://your-api.fetchsandbox.com/v1/orders \
  -H "api-key: sandbox_abc123"
# → 200 OK, realistic seed data

fetchsandbox run your-api create-and-fulfill-order
# → ✓ Create order — 201
# → ✓ Add line items — 200
# → ✓ Submit for fulfillment — 200
# → ✓ Webhook: order.fulfilled fired

No staging to maintain. No IP whitelists. No production risk. Partners get their own URL, data, and credentials. Your team doesn't touch it.

When a partner asks "does this workflow work?" — they prove it themselves.

The time sink nobody tracks

If you run a partner API, count hours per month on:

Provisioning and maintaining test environments
Debugging "is your sandbox down?" tickets
Managing IP whitelists and VPN access
Re-seeding stale test data
Coordinating deploys around partner testing schedules

At my company: 20–30 hours across engineering and DevOps. For a problem that shouldn't exist.

Open the Stripe sandbox and run a workflow end-to-end — no signup: fetchsandbox.com/playground

Resend webhook reconciliation: map events to the right email record

FetchSandbox — Tue, 23 Jun 2026 20:02:20 +0000

This is a different failure mode from treating email.sent as delivered. That bug is here. This post is about what happens after you store the Resend email ID.

POST /emails returned 200. You saved the Resend email ID on your notification row. A minute later the webhook handler runs.

Then support asks: "Why does the dashboard say delivered when the address bounced?"

The send call worked. The reconciliation layer did not.

The bug is usually a flat status field

Most handlers start like this:

app.post("/webhooks/resend", async (req, res) => {
  const event = req.body;
  const emailId = event.data.email_id;

  const row = await db.notifications.findByResendId(emailId);
  if (!row) return res.sendStatus(200);

  if (event.type === "email.delivered") {
    await db.notifications.update(row.id, { status: "delivered" });
  }
  if (event.type === "email.opened") {
    await db.notifications.update(row.id, { status: "opened" });
  }
  if (event.type === "email.bounced") {
    await db.notifications.update(row.id, { status: "bounced" });
  }

  res.sendStatus(200);
});

Three separate problems hide in that shape:

Lookup failure — webhook arrives before your app finishes persisting the Resend ID, or the ID is stored on the wrong table. Handler returns 200, nothing updates, bug disappears until production.
Last-write-wins — email.opened overwrites delivered. Fine if you track engagement separately. Bad if status drives UX copy or retry logic.
No terminal states — email.bounced should win over delivered when events arrive out of order or retry. A single string field cannot express that without rules.

Separate delivery state from engagement

A pattern that survives production:

const TERMINAL = new Set(["bounced", "complained", "failed"]);

function nextDeliveryState(current, incoming) {
  if (TERMINAL.has(current)) return current;
  if (incoming === "bounced" || incoming === "complained") return incoming;
  if (incoming === "delivered") return current === "sent" ? "delivered" : current;
  if (incoming === "sent") return current ?? "sent";
  return current;
}

app.post("/webhooks/resend", async (req, res) => {
  const { type, data } = req.body;
  const emailId = data.email_id;

  const row = await db.notifications.findByResendId(emailId);
  if (!row) {
    await db.webhookInbox.insert({ provider: "resend", emailId, type, payload: req.body });
    return res.sendStatus(200);
  }

  if (type === "email.opened" || type === "email.clicked") {
    await db.emailEvents.insert({ notificationId: row.id, type });
    return res.sendStatus(200);
  }

  const next = nextDeliveryState(row.deliveryStatus, type.replace("email.", ""));
  await db.notifications.update(row.id, {
    deliveryStatus: next,
    lastEventType: type,
    lastEventAt: new Date(),
  });

  res.sendStatus(200);
});

Notes that matter in real apps:

email.opened is not delivery. Log it separately unless your product truly treats opens as delivery confirmation.
Store unmatched webhooks. If the event arrives before the send transaction commits, you need a replay path — not a silent 200.
Idempotency key = Resend email ID + event type. Retries are normal.

The test most teams skip

Unit tests mock one webhook payload. That proves parsing, not reconciliation.

The integration test that catches the bug:

create internal notification row
POST /emails and persist Resend email ID on that row
deliver email.sent, then email.delivered
deliver email.opened and assert delivery status did not regress
deliver email.bounced on a second send and assert terminal state wins

Step 4 and 5 are where flat status fields break.

Receipts

Real Resend send workflow run against FetchSandbox, captured 2026-06-07:

POST /emails                                       → 200
  id: 4d9acb24-5913-4e02-8b1a-a18ed10a6fad

GET  /emails/4d9acb24-5913-4e02-8b1a-a18ed10a6fad  → 200

webhook events verified:
  email.sent
  email.delivered

Full timeline: fetchsandbox.com/runs/8e4fe8e9f9?flow=run_4062510e-268e-451f-b5f4-e8844eef42d5

That run proves the provider lifecycle through email.delivered. Your app still needs its own test for mapping those events back to the correct internal row and for terminal bounce/complaint handling.

Runnable preflight (optional)

Before wiring production webhooks, I run the send workflow from Cursor with FetchSandbox MCP:

validate resend email workflow with fetchsandbox

Workflow page: fetchsandbox.com/docs/resend

Resend sandbox page (mock API + webhook replay): fetchsandbox.com/resend

That does not replace Resend's real domain, API key, or production webhook endpoint. It just makes the lifecycle obvious before you commit your local state machine.

Test AgentMail multi-tenant webhooks before they leak across tenants

FetchSandbox — Tue, 16 Jun 2026 17:16:05 +0000

The dangerous AgentMail webhook bug starts with a response that looks completely fine.

POST /v0/webhooks -> 200 OK

The response body contains the webhook object. It includes your URL, event types, and usually enough fields to make the integration feel done.

But in a multi-tenant app, the field that matters is inbox_ids. If that array is silently dropped, your webhook is no longer scoped to the tenant inbox you meant to subscribe. It is subscribed broadly, and every tenant's message events can start flooding the same endpoint.

That is not a noisy failure. It is worse. The subscribe call worked. Your endpoint receives events. The logs look active. The leak is in the scope.

If your handler routes by webhook ID before it checks the inbox ID, the wrong customer can look like a valid event. That is the kind of bug that passes observability and fails privacy.

Why unit tests miss it

Most unit tests stub the webhook create call by echoing back exactly what the app sent.

request.inbox_ids -> response.inbox_ids

That proves your client serialized the payload. It does not prove AgentMail persisted the scope.

The test passes. CI passes. The code ships. Then production receives events for inboxes that do not belong to the tenant that configured the webhook.

The annoying part is that the local test is not obviously wrong. It has the right endpoint, the right payload shape, and the right assertion against the create response. It just trusts the wrong copy of the object.

The pattern is reconcile-after-write

The fix is small enough to name:

POST -> GET -> diff

Treat the POST response as untrusted. It may be your request echoed back. Treat the later GET as the provider state. That is what AgentMail actually stored.

The diff is the bug.

const sentInboxIds = ["inbox_tenant_123"];
const created = await agentmail.post("/v0/webhooks", {
  url: "https://app.example.com/webhooks/agentmail",
  event_types: ["message.delivered", "message.bounced"],
  inbox_ids: sentInboxIds,
});
const stored = await agentmail.get(`/v0/webhooks/${created.id}`);
const got = [...(stored.inbox_ids ?? [])].sort();
const want = [...sentInboxIds].sort();
if (JSON.stringify(got) !== JSON.stringify(want)) {
  throw new Error("AgentMail webhook scope drifted");
}

That is the whole check. It is not a bigger mock. It is a read after the write.

Why the agent caught it

The reason a coding agent caught this where the unit test did not is not that it was smarter about webhooks. It ran a better-shaped test.

AgentMail has a curated webhook_lifecycle_create_read_delete workflow in FetchSandbox. When the agent ran it through the FetchSandbox MCP server, the workflow forced the boring step people skip:

create inbox
subscribe webhook scoped to that inbox
read webhook back
compare inbox_ids
delete webhook

The workflow shape matches the bug shape. A stubbed unit test checks what your app meant to send. The workflow checks what the provider kept.

This is not only AgentMail

The same bug appears anywhere a control-plane API returns an object that looks like the request you sent:

IAM policies
ACL rules
draft resources
webhook subscriptions
tenant-scoped notification settings

If a create call returns 200, but the security or tenancy boundary matters, do not stop at the create response.

Read it back. Diff the fields that protect the boundary. Fail the test before the provider starts sending another tenant's events to your endpoint.

Run the brownfield demo

The brownfield-agentmail-demo repo shows this end to end if you want to run it against the workflow yourself.

The important part is not AgentMail-specific. The habit is: after a write that configures scope, reconcile what the provider persisted.

A webhook that passes create tests can still be the webhook that leaks across tenants.

FetchSandbox's AgentMail workflow docs include the create, read, and teardown path.

Clerk webhook replay can create duplicate users unless your sync is idempotent

FetchSandbox — Mon, 15 Jun 2026 13:57:02 +0000

The login worked. The webhook worked. Then the same event arrived again.

Clerk webhooks are easy to treat like a notification:

user.created arrived
create a local user
done

That works until the event is retried, replayed, or delivered after another part of your app already created the user.

Then your app has two local rows for the same Clerk user. Or one row has the right email and the other has the right metadata. Or your onboarding job runs twice and sends the same welcome email twice.

The bug is not "Clerk webhooks are unreliable." Retrying delivery is normal webhook behavior.

The bug is trusting delivery count instead of provider state.

The narrow fix

Your webhook handler should not mean:

on user.created -> insert user

It should mean:

on user.created -> upsert by clerk_user_id

The important field is the stable provider ID, not the local row ID and not the email address.

await db.user.upsert({
  where: { clerkUserId: event.data.id },
  update: {
    email: event.data.email_addresses?.[0]?.email_address,
    firstName: event.data.first_name,
    lastName: event.data.last_name,
    clerkUpdatedAt: new Date(event.data.updated_at),
  },
  create: {
    clerkUserId: event.data.id,
    email: event.data.email_addresses?.[0]?.email_address,
    firstName: event.data.first_name,
    lastName: event.data.last_name,
    clerkUpdatedAt: new Date(event.data.updated_at),
  },
});

That one constraint does most of the work:

UNIQUE(clerk_user_id)

Now a replay updates the same user instead of creating a second one.

The part people skip

The handler can still be wrong even after the upsert.

If your app grants access, creates a workspace, starts a trial, or sends email inside the same handler, those side effects need their own idempotency rule too.

For example:

workspace owner = clerk_user_id
welcome email key = clerk_event_id
trial grant key = clerk_user_id + plan_id

The user row is only one piece of local state.

How I would test it

I would test the same user.created event twice.

First delivery:

local user count: 0 -> 1
workspace count: 0 -> 1

Replay:

local user count: 1 -> 1
workspace count: 1 -> 1

That is the whole test.

FetchSandbox has a Clerk sandbox and runnable Clerk workflow docs for this exact kind of replay check. It is also where a stateful sandbox is more useful than a static mock response: the second delivery is the thing you need to observe.

Takeaway

Do not write Clerk webhook handlers as if events happen once.

Write them as reconciliation:

given this Clerk user ID,
what should my app state be now?

If the answer changes when the same event arrives twice, the bug is already there. Production is just waiting to replay it.

WorkOS directory sync webhooks need reconciliation after Okta group renames

FetchSandbox — Thu, 11 Jun 2026 14:25:12 +0000

The bug shows up two minutes after the customer says, "we only renamed the group."

In Okta, the admin changed Enterprise Admins to Platform Admins. WorkOS sent dsync.group.updated. Your webhook handler saw the new name. The cache updated. Everything looked normal.

Then support gets the screenshot: half the enterprise customer's users can access the gated feature. Half can't. The senior employees are the first ones broken because they sit in the group your app treats as the entitlement source.

The webhook is not the final state

The tempting handler is simple:

type WorkOSGroupUpdated = {
  event: "dsync.group.updated";
  data: {
    id: string;
    name: string;
  };
};

export async function handleGroupUpdated(event: WorkOSGroupUpdated) {
  await groupsCache.set(event.data.id, {
    name: event.data.name,
    updatedAt: new Date(),
  });
}

That works only if the webhook payload is the same thing as the directory state your feature gate should trust.

It is not.

Okta sends SCIM PATCH operations to WorkOS in batches that do not necessarily match the order an admin sees in Okta's UI. WorkOS directory sync webhooks fire as individual events. They tell you something changed. They do not give you a complete, ordered snapshot of the group and every user's current membership.

If your app updates downstream feature gating from the webhook payload, it can partially reconcile. One cache entry has the new group name. Another user's group membership still reflects the old world. The feature gate now depends on which stale value a request happened to read.

Treat dsync webhooks as invalidation

For feature gating, a dsync.group.updated or dsync.user.updated webhook should invalidate local state. It should not be the state.

The safer handler does two reads after every relevant webhook:

type DSyncEvent = {
  event: "dsync.group.updated" | "dsync.user.updated";
  data: {
    id: string;
  };
};

export async function handleDirectorySyncEvent(event: DSyncEvent) {
  if (event.event === "dsync.group.updated") {
    const group = await workos.directorySync.getDirectoryGroup(event.data.id);
    const users = await workos.directorySync.listDirectoryUsers({
      directory: group.directoryId,
    });

    await featureGateStore.reconcileDirectoryGroup({
      groupId: group.id,
      groupName: group.name,
      users: users.data,
    });
  }

  if (event.event === "dsync.user.updated") {
    const users = await workos.directorySync.listDirectoryUsers();

    await featureGateStore.reconcileDirectoryUsers(users.data);
  }
}

The important part is not the SDK shape. It is the rule: ignore the webhook payload's name and groups for downstream feature gating. Use the payload ID to know what to re-fetch. Then read from GET /directory_groups/{id} and GET /directory_users.

Those reads are the source of truth your cache reconciles against.

What to test

The test is not "did we receive dsync.group.updated?"

The test is:

Start with a group that controls a feature gate.
Rename the group in the identity provider.
Receive the group update webhook.
Re-fetch the group by ID.
Re-fetch directory users.
Rebuild the local entitlement view from the reads, not the webhook payload.

That is the path that catches the bug. A unit test around the webhook JSON does not.

Receipts

Here's a real WorkOS directory group reconciliation workflow run against the FetchSandbox sandbox, captured 2026-06-11:

GET /directories → 200
GET /directory_groups → 200
GET /directory_groups/54483a43-5333-489d-8930-2a549bce4de9 → 200
GET /directory_users → 200

Sandbox ID: a8d236f155

Flow run ID: run_227a570b-1c7d-42d8-ad8d-af1c09c67b60

Full timeline: fetchsandbox.com/runs/a8d236f155?flow=run_227a570b-1c7d-42d8-ad8d-af1c09c67b60

The group ID 54483a43-5333-489d-8930-2a549bce4de9 is the value the webhook should use to trigger reconciliation. The follow-up GET /directory_users is what keeps feature gating from depending on a partial webhook payload.

How to test this without waiting on Okta

For WorkOS directory sync, the finish line is not "webhook received." It is "the app rebuilt its local entitlement view from the directory state."

We use FetchSandbox to run the directory group reconciliation workflow before wiring the production handler. The point is not to replace Okta or WorkOS testing. The point is to make the cache rule obvious before a real enterprise customer renames the group your feature gate depends on.

Clerk JWT 401 on the server? Check these 3 env vars first

FetchSandbox — Tue, 09 Jun 2026 14:58:41 +0000

A Clerk login can look fine in the browser and still fail on your server with a 401.

The frontend has a user. getToken() returns something. The request includes:

Authorization: Bearer <token>

Then FastAPI, Express, or your API route rejects it.

Before rewriting your auth code, check this:

CLERK_PUBLISHABLE_KEY
CLERK_SECRET_KEY
CLERK_JWT_ISSUER

They need to come from the same Clerk instance.

The bug

This can happen when you have multiple Clerk projects open:

Frontend:
  CLERK_PUBLISHABLE_KEY = pk_test_from_project_A

Backend:
  CLERK_SECRET_KEY = sk_test_from_project_B
  CLERK_JWT_ISSUER = https://project-c.clerk.accounts.dev

Each value can look valid by itself.

But the token was minted by one Clerk instance, and your backend is trying to verify it against another one. That is enough for server-side verification to fail.

You might see:

JWT issuer mismatch
invalid issuer
JWKS key not found
invalid signature
401 unauthorized

Different library, same root issue.

The quick fix

Open one Clerk dashboard project and copy all auth values again from the same place.

Then restart both servers.

# frontend
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_test_...

# backend
CLERK_SECRET_KEY=sk_test_...
CLERK_JWT_ISSUER=https://same-instance.clerk.accounts.dev

Do not only refresh the browser. Dev servers often keep old env values until the process restarts.

Why agents miss this

An AI coding agent can write the shape of a Clerk integration:

add middleware
read bearer token
verify JWT
sync user
protect route

But it will not know you pasted keys from two Clerk projects unless it checks the running path.

This is why I like running a small auth workflow before wiring the app.

Receipt

I ran FetchSandbox's Clerk user_signup workflow as the receipt layer:

Workflow: clerk/user_signup
Result: passed
Steps: 4/4
Duration: 77.44 ms
Verified webhooks: user.created, user.updated
Timeline: https://fetchsandbox.com/runs/7aa06cff84?flow=run_b8399f77-352c-4fe4-ad50-24e3060f8d8d

That workflow proves the user lifecycle surface:

POST /users
GET /users/{id}
PATCH /users/{id}
verify user.created + user.updated

It does not replace your real Clerk project. It gives your agent or teammate a runnable auth flow before they touch app code.

Takeaway

If Clerk works in the browser but your server returns 401, check the boring thing first:

publishable key
secret key
issuer URL

Same instance. Same environment. Then restart.

FetchSandbox MCP setup if you want the agent to run the workflow first:

{
  "mcpServers": {
    "fetchsandbox": {
      "command": "npx",
      "args": ["-y", "fetchsandbox-mcp"]
    }
  }
}

Test Clerk auth lifecycle flows without production users

FetchSandbox — Tue, 02 Jun 2026 04:22:48 +0000

Most auth bugs do not come from the login button.

They show up after login:

a user is created in Clerk but not in your app database
a session expires while your UI still thinks the user is active
a webhook arrives late or twice
a deleted user still has local app state
JWT verification works locally, then fails once keys rotate

That is the part I want API integration agents to test before they write app code.

I have been working on FetchSandbox MCP support for Clerk-style auth workflows so an IDE agent can run the auth lifecycle in a sandbox first, then wire the app around the behavior it just proved.

The narrow workflow

For a Clerk integration, the useful test is not just:

Can I create a user?

The useful test is closer to:

Can my app survive the auth lifecycle?

That includes:

Create or receive a user
Create a session
Verify the token/session state
Receive a webhook event
Reconcile that event into app state
End or revoke the session
Confirm the app no longer treats the user as active

In real apps, each of those steps tends to live in a different place: middleware, webhook handlers, user tables, session checks, background jobs, and sometimes frontend state.

That is exactly where AI coding agents need more than docs.

Where normal AI coding breaks down

If you ask an agent to "add Clerk auth," it can usually generate plausible code:

Add middleware.
Read userId.
Create a webhook route.
Verify signatures.
Sync the user.

That is useful, but it is not proof.

The agent still has to guess:

which event shape matters
what your app should store
what should happen when the same event arrives again
what happens when the session ends
whether user deletion should cascade, archive, or soft-disable local data

Those are integration questions, not syntax questions.

What FetchSandbox MCP changes

FetchSandbox gives an IDE agent a runnable API workflow environment.

Instead of only reading docs, the agent can:

route the user's intent to a Clerk-style workflow
execute the auth lifecycle in a sandbox
inspect request and response payloads
inspect webhook event shapes
identify which app files need to handle each transition
summarize proof and next steps before editing code

The ideal loop looks like this:

developer intent
↓
FetchSandbox MCP route
↓
run auth workflow
↓
inspect payloads and state transitions
↓
wire app code
↓
summarize proof + remaining edge cases

That makes the agent less likely to invent a happy-path-only integration.

Example prompt

From Cursor, Claude Code, or another MCP-capable IDE:

Help me integrate Clerk auth into this app.
Before editing code, use FetchSandbox to prove the user/session lifecycle and webhook payload shape.

The important part is "before editing code."

For auth, that order matters.

The failure modes worth testing

For a Clerk-style integration, I would rather see an agent test these before it changes the app:

User created

Does the app create or update the local user record correctly?

Session created

Does the app treat the session as active only when the provider state says it is active?

Session ended

Does the app stop trusting stale session state?

User deleted

Does the app remove access without accidentally deleting data that should be retained?

Webhook replay

If the same webhook arrives twice, does the app stay idempotent?

Signature verification

Does the webhook handler verify the signed payload before trusting the event?

None of these are solved by a code snippet alone.

They need a workflow.

Why this matters for AI agents

AI coding agents are very good at producing integration code.

But API integrations need an execution layer.

Without that layer, the agent is forced to infer behavior from docs and examples. With FetchSandbox MCP, the agent can run a workflow first and then use the result as context.

The difference is subtle but important:

"Here is auth code that should work."

versus:

"Here is the auth lifecycle I ran, the payloads I saw, and the code paths I wired."

That is the workflow I want for API integrations.

Setup

Add FetchSandbox MCP to your IDE:

{
  "mcpServers": {
    "fetchsandbox": {
      "command": "npx",
      "args": ["-y", "fetchsandbox-mcp"]
    }
  }
}

Then ask the agent to prove the provider workflow before implementation.

Use FetchSandbox MCP to test the Clerk auth lifecycle before wiring the app.

FetchSandbox:

https://fetchsandbox.com

MCP:

https://fetchsandbox.com/mcp

Takeaway

Auth integrations do not end at login.

The lifecycle is the contract:

users
sessions
tokens
webhooks
app-state reconciliation

If an AI agent is going to wire auth into your app, it should be able to run that lifecycle first.

Connect the provider.

Prove the workflow.

Ship the integration.

Brownfield Slack alerts: a 6-minute guided MCP run on Stripe + Resend webhooks

FetchSandbox — Wed, 27 May 2026 14:45:18 +0000

I recorded a raw ~6.5 minute screen capture of my IDE running a full FetchSandbox MCP guided flow. No polish, no voiceover script — just what happens when you ask an agent to add Slack notifications for failed payments in a repo that already has Stripe checkout, Clerk JWT verification, and Resend receipt webhooks.

The video:

YOUTUBE_URL

This post is the written version of that session, with the parts that are easier to scan in prose than in a screencast.

The ask (and why it is not a greenfield task)

Prompt:

add slack notifications for failed payments — should plug into the existing stripe + resend webhook handlers

Repo: a small Next.js 15 + FastAPI demo (stripe-checkout-demo) that had seen three prior integration sessions:

Session	What landed	Where Slack attaches
Stripe seed	`POST /create-payment-intent`, `POST /webhook`	`payment_intent.payment_failed` branch
Clerk	JWT dep, `POST /clerk-webhook` (Svix)	(out of scope for this alert)
Resend	`send_receipt()`, `POST /resend-webhook`	bounce/complaint branches + receipt try/except

Slack is the fourth layer. The agent cannot treat this like a tutorial repo with one file and one webhook. It has to find the exact lines where failures today only print().

That repo reading is why a full guided run is ~6–7 minutes in the recording — not because MCP is slow, but because brownfield integration is mostly comprehension.

What “guided” means in practice

FetchSandbox MCP does not jump straight to POST /chat.completions style codegen. The flow I used (fs-router skill) looks like:

intake → introspect repo → comprehend stack + failure surfaces → guide() routing → discovery questions → prove upstream contract → (then) propose changes

Step 1 — Introspect

The introspect script scanned the repo and returned framework hints (Next.js ^15, FastAPI backend) and SDK signals for Stripe in server/main.py. The intent string mentioned Resend too; Slack is not in the FetchSandbox brain catalog, so it never appears as a routable spec.

Step 2 — Comprehend (the load-bearing step)

The agent catalogued where alerts would attach, not a new /slack-webhook route:

Event	Today	Location
`payment_intent.payment_failed`	`print()` only	webhook handler ~L95–98
`email.bounced` / `email.complained`	`print()` + TODO	Resend handler ~L139–145
Receipt send raises (Resend SDK)	swallowed with `print()`	inside Stripe success path ~L92–94

The user named Stripe + Resend handlers. The comprehend step also flagged the receipt try/except as a third visibility gap — correct webhook hygiene (do not fail Stripe retries) but operationally invisible without something like Slack.

Step 3 — `guide()` routing

guide(intent="add slack notifications for failed payments — plug into existing stripe + resend webhook handlers...")

Result:

spec: stripe
workflow: accept_payment (Tier-1 default)
confidence: 0.85
reasoning: no slack spec; Stripe is the upstream that emits the event Slack will format

That routing is the right call, but it needs to be said plainly: FetchSandbox proves the upstream event, not the Slack POST.

Step 4 — Discovery (failures tab matters)

Multi-tab confirmation:

Question	Answer	Implies
Customer geo	US only	no 3DS scenario
Capture	Auto on confirm	`accept_payment` workflow
Failure modes	Declined card (~60s)	`payment_declined` scenario

If you skip the failures tab, you get a happy-path proof that does not exercise the webhook your Slack message cares about.

Step 5 — Proof run (and the honest limit)

Delegated to fs-prove-payments: spec=stripe, workflow=accept_payment, scenario=payment_declined.

Sandbox import reused the bundled Stripe spec with hand-curated workflows. Run result on paper: 6/6 steps passed, webhooks verified: payment_intent.created, payment_intent.succeeded.

Asked for	Got	Why
`payment_declined` → `payment_intent.payment_failed`	Terminal `succeeded`, success webhooks	Curated `accept_payment` uses a deterministic pass card; scenario hook exists in the engine but this Tier-1 workflow does not exercise it

Same class of limit as other Stripe sessions: the catalog workflow is a fast preflight, not a substitute for driving a real decline.

To prove the handler your Slack code will sit in:

Method	What it proves
`stripe listen --forward-to localhost:8000/webhook` + card `4000000000000002`	Full PI flow → `payment_intent.payment_failed` at your handler
`stripe trigger payment_intent.payment_failed`	Fast iteration on Slack message template from a realistic payload

What the agent got right without me asking

Five things worth stealing for your own brownfield prompts:

No new endpoint — side effects only at existing handler lines.
Third target surfaced — receipt-send swallow, not only the two webhooks the user named.
Routing limitation named — upstream Stripe proof, Slack downstream on you.
Idempotency applied to Slack — Stripe retries on 5xx; use pi.id as dedupe key for duplicate alerts.
Resend proof deferred — one workflow per delegation; bounce/complaint shapes need a separate Resend run.

What was not done in this recording

No Slack diff applied yet (proof handed back; propose-changes step not shown in the artifact).
No Resend proof run for email.bounced / email.complained.
No Slack spec proof (none in catalog).

Expected next moves from the session:

Propose payment_intent.payment_failed → Slack at L95–98 (SLACK_WEBHOOK_URL env).
stripe trigger payment_intent.payment_failed before editing message text.
Separate Resend proof for bounce/complaint templates.
Ship payment-failed path first; Resend alerts in a follow-up PR.

MCP setup (unchanged from shorter demos)

{
  "mcpServers": {
    "fetchsandbox": {
      "command": "npx",
      "args": ["-y", "fetchsandbox-mcp"]
    }
  }
}

Cursor: ~/.cursor/mcp.json, restart IDE, enable server in Settings → MCP.

Optional skills:

curl -sL https://fetchsandbox.com/skills/install.sh | bash

The takeaway

Docs tell your agent what should happen. Runnable workflows tell it what did happen in a sandbox — and brownfield comprehend tells it where your code already handles failures.

For Slack-on-Stripe, the valuable proof is not “can we call chat.postMessage” in FetchSandbox. It is: when payment_intent.payment_failed arrives, does your handler have the fields you need for an alert, and have you tested that path with a real or triggered event?

Try FetchSandbox: https://fetchsandbox.com

Stripe workflows: https://fetchsandbox.com/docs/stripe

Watch the raw demo: https://youtu.be/AB36LXwfoTQ

If you have done brownfield API work with agents, I am curious: should the default flow always map existing handlers before any proof run, or do you prefer prove-first on greenfield repos?

Test Resend webhook reconciliation before production

FetchSandbox — Fri, 22 May 2026 03:59:54 +0000

I was trying the Resend API for a small product flow, and the first POST /emails call was not the part that worried me most.

That call gives you an email ID.

The part I care about is what happens later, when events come back:

email.delivered
email.bounced
email.opened

Resend has docs for these webhook events:

email.delivered: https://resend.com/docs/webhooks/emails/delivered
email.bounced: https://resend.com/docs/webhooks/emails/bounced
email.opened: https://resend.com/docs/webhooks/emails/opened

The docs are useful. But the integration question is not just "what fields are in the payload?"

The real question is:

can my app map this event back to the right email record?

The small bug that becomes annoying later

Most apps do something like this when sending email:

create internal notification record
send email with Resend
store resend email id
wait for webhook events
update notification status

Simple enough.

But the webhook arrives later, outside the request that sent the email.

So now your app needs to answer a few things correctly:

which internal record owns this Resend email ID?
should email.opened change the same status as email.delivered?
what happens if email.bounced arrives after you already marked the email as sent?
do you store the raw event payload for debugging?
can you replay the event without creating duplicate state changes?

None of this is hard in theory.

It is just easy to skip until production.

Testing only the send call is not enough

If you test only this:

POST /emails

you have proven that Resend accepted the request.

You have not proven your app can handle the lifecycle around that request.

For a real product, I want to test this chain:

send email
store provider email id
receive delivered event
update internal status
receive opened or bounced event
reconcile final state

That is the integration.

The endpoint is just one step inside it.

Why this is awkward to test early

To test this against the real provider path, you usually need:

Resend API key
verified sender domain
public webhook endpoint
local tunnel
event logging
test recipient
some app state to update

That setup is reasonable before production.

But when I am still exploring the API, I do not want to create all of that just to understand whether the workflow makes sense.

I want to run the shape of the workflow first.

The workflow I wanted from my IDE

At that point I wanted Cursor or Claude to run a small workflow for me:

send test email
capture email id
simulate delivered event
simulate opened or bounced event
verify app-facing state
show what changed

I tried this through FetchSandbox MCP because it gives the agent a runnable API workflow instead of only docs to read.

This is not replacing Resend's real environment. I would still need the real API key, real domain, production webhook endpoint, and provider limits before shipping.

The point is earlier than that.

Before wiring the real app, I wanted the agent to understand the lifecycle:

request -> provider id -> webhook event -> internal state

What made the validation useful

What helped was not a fake "success" message.

It was seeing a small report closer to:

PASS POST /emails returned email id
PASS internal record stored provider id
PASS email.delivered event matched provider id
PASS notification status changed to delivered
PASS email.opened event was recorded without duplicating the send

Or if it fails:

FAIL webhook event could not be matched to an internal record

That is the kind of failure I would rather see before production, not after users are asking why they never got an email.

Try the workflow

If you are exploring Resend, this is the workflow page I used:

https://fetchsandbox.com/docs/resend

From an MCP client like Cursor or Claude, the ask is simple:

validate resend email workflow with fetchsandbox

The workflow should not stop at POST /emails -> 200.

It should prove the lifecycle after the send call makes sense.

That is where most integration bugs hide.

DEV Community: FetchSandbox

We planted a Descope privilege-escalation bug — can your coding agent catch it?

The 20-minute test drive

What a real catch looks like

The other planted bugs

The part where you tell us it fell flat

How to test Descope auth without real keys

Our CI mocked Stripe. Production still broke on step two.

What unit tests miss

What we run now — MCP locally, same workflow in CI

Why MCP belongs in the loop

What a workflow run actually checks

What we stopped doing

Try it

Our partners kept breaking on staging. So we gave them production access. (Don't do this.)

Attempt 1: "Just use our docs"

Attempt 2: Internal UAT box

Attempt 3: Staging with IP whitelisting

Attempt 4: Production access

What we actually needed

The time sink nobody tracks

Resend webhook reconciliation: map events to the right email record

The bug is usually a flat status field

Separate delivery state from engagement

The test most teams skip

Receipts

Runnable preflight (optional)

What to read next

Test AgentMail multi-tenant webhooks before they leak across tenants

Why unit tests miss it

The pattern is reconcile-after-write

Why the agent caught it

This is not only AgentMail

Run the brownfield demo

Clerk webhook replay can create duplicate users unless your sync is idempotent

The narrow fix

The part people skip

How I would test it

Takeaway

WorkOS directory sync webhooks need reconciliation after Okta group renames

The webhook is not the final state

Treat dsync webhooks as invalidation

What to test

Receipts

How to test this without waiting on Okta

Clerk JWT 401 on the server? Check these 3 env vars first

The bug

The quick fix

Why agents miss this

Receipt

Takeaway

Test Clerk auth lifecycle flows without production users

The narrow workflow

Where normal AI coding breaks down

What FetchSandbox MCP changes

Example prompt

The failure modes worth testing

User created

Session created

Session ended

User deleted

Webhook replay

Signature verification

Why this matters for AI agents

Setup

Takeaway

Brownfield Slack alerts: a 6-minute guided MCP run on Stripe + Resend webhooks

The ask (and why it is not a greenfield task)

What “guided” means in practice

Step 1 — Introspect

Step 2 — Comprehend (the load-bearing step)

Step 3 — guide() routing

Step 4 — Discovery (failures tab matters)

Step 5 — Proof run (and the honest limit)

What the agent got right without me asking

What was not done in this recording

MCP setup (unchanged from shorter demos)

The takeaway

Test Resend webhook reconciliation before production

The small bug that becomes annoying later

Step 3 — `guide()` routing