DEV Community: Eric Choi | Evidence Gate Studio

Supabase RLS Is Not Proven Until Alice Cannot Read Bob

Eric Choi | Evidence Gate Studio — Mon, 20 Jul 2026 21:13:45 +0000

Supabase makes it easy to build a real app quickly.

That speed is exactly why the privacy boundary deserves a boring, explicit test before launch.

The question is not only:

Can Alice log in and see Alice's data?

That is the happy path.

The launch-risk question is:

Can Alice read Bob's data if she changes an id, route, org id, request, storage path, or realtime subscription?

If you cannot answer that with evidence, your app may still be a polished prototype rather than a shippable product.

RLS “on” is not the same as RLS proven

I see builders treat Row Level Security as a checkbox:

RLS enabled
auth added
the UI filters by user
the dashboard looks private
ship it

But a private-looking UI can hide a weak data boundary.

Common examples:

a policy allows any authenticated user, not the actual owner;
a React filter hides rows, but the database policy does not;
a server route uses a trusted key without checking ownership;
storage bucket rules are different from table rules;
organization switching leaves stale access state;
realtime channels broadcast rows the page itself would not show;
admin/support paths work in development and accidentally remain reachable.

None of these require a dramatic hacker story. They are ordinary launch mistakes.

The smallest useful test: two users, two orgs, blocked cross-access

Before showing the app to real users, create the simplest negative test you can:

Create Alice.
Create Bob.
Put Alice in org Alpha.
Put Bob in org Beta.
Create private rows for both.
Log in as Alice.
Try to read Bob's rows.
Try to update Bob's rows.
Try to delete or export Bob's rows.
Try the same boundary through storage, server routes, RPC/functions, and realtime listeners.

Passing the happy path is not enough.

The useful evidence is blocked cross-access.

A practical pre-launch checklist

Before launch, I would want evidence for these checks:

RLS is enabled on every private table, not just the main customer table.
Policies prove ownership or membership using auth.uid() and trusted org/team membership.
The client never exposes a service-role key.
Storage buckets and file URLs are scoped to the same privacy boundary.
Server routes check session, role, and ownership before reads and writes.
Alice/Bob negative cases cover read, write, update, delete, export, and subscribe.
Org/team switching cannot leave stale local access.
Realtime channels follow the same ownership rules.
Admin/support paths are explicit, logged, and unavailable to ordinary users.
One written launch test records what was proven and what was not.

This is not a replacement for a professional security review.

It is a practical “do not accidentally ship the obvious leak” check for tiny SaaS apps, AI-built apps, and paid pilots.

Why this matters more for AI-built apps

AI helps people build faster than their review process can keep up.

That creates a weird failure mode:

the interface looks credible;
the demo works;
the app has auth;
the app has a database;
the founder feels close to launch;
but the boundary between users was never attacked.

When the app touches private prompts, documents, customer records, usage logs, generated outputs, or payment-linked features, this is not a small detail.

It is one of the boundaries that decides whether you can safely invite real people.

Free checker

I made a free no-login Supabase RLS/Auth Exposure Checker:

https://bespoke-pika-937432.netlify.app/supabase-rls-auth-exposure-checker?utm_source=devto&utm_medium=article&utm_campaign=supabase_rls_two_user_test

It asks the practical pre-launch question:

Could one user read another user's data?

Use it to generate a quick exposure report before launch.

If the flow still feels risky

If you want a second pass on one critical app flow, I also have a fixed-scope one-flow preflight:

AI App One-Flow Preflight — 48-Hour Evidence Review

One app, one critical flow, 10 focused checks, repro notes, and a smallest-fix-first action plan.

https://contra.com/s/r7s2Elvl-ai-app-one-flow-preflight-48-hour-evidence-review?r=eric_choi_td1brz18&utm_source=devto&utm_medium=article_cta&utm_campaign=supabase_rls_two_user_test

No giant audit. No vague “I can help.” One risky flow, checked deliberately.

More final-state checks before launch

The Alice/Bob RLS test is one piece of a larger launch proof.

I also made a free AI app launch-proof hub:

https://bespoke-pika-937432.netlify.app/tools?utm_source=devto&utm_medium=article_cta&utm_campaign=v16_tool_hub_supabase

It includes the Supabase checker, Stripe final-state checker, AI output trust-boundary checker, Vibe-Coded App Reality Score, failure scenario gallery, and a sample one-flow preflight report.

If you want the self-serve worksheet, the $5 Final-State Checklist Pack is here:

https://payhip.com/b/TIe6v?utm_source=devto&utm_medium=article_cta&utm_campaign=v16_final_state_checklist_supabase

I also made a fictional Sample One-Flow Preflight Report format:

https://bespoke-pika-937432.netlify.app/sample-one-flow-preflight-report?utm_source=devto&utm_medium=article_cta&utm_campaign=supabase_sample_preflight_report

The useful part is the evidence table: observed state, why it matters, and smallest fix first.

Built an AI App but Nobody Paid? Use a Paid-Pilot-First Plan

Eric Choi | Evidence Gate Studio — Mon, 20 Jul 2026 04:47:12 +0000

AI has made building dramatically faster. It has not made buying automatic.

That distinction explains a common solo-builder trap: the app works, the interface looks credible, and another round of improvements is already planned—yet nobody has paid. The natural response is to add features, polish the landing page, or lower the price. Those actions feel productive because they stay inside the builder's comfort zone. They may also deepen the original mistake.

The missing evidence is not more code. It is a buyer with a specific problem, enough urgency to act, and a small outcome worth paying to test.

A paid-pilot-first plan reverses the normal sequence. You do not build a complete product and then search for demand. You identify one costly workflow, speak with the people who experience it, offer a narrow paid engagement, and build only the end-to-end slice required to deliver the promised result. The objective is not to manufacture hype in 72 hours. It is to replace assumptions with progressively stronger evidence.

1. Stop treating the app as the offer

Buyers rarely wake up wanting a new AI app. They want an existing task to become faster, safer, less expensive, or easier to verify.

“AI meeting assistant” describes a category. “Turn a customer-success call into a reviewed follow-up draft before the account manager's next meeting” describes a result. The second statement gives a buyer something concrete to evaluate.

Before writing more code, write a one-sentence offer:

For [specific buyer] who struggles with [observable workflow], I will deliver [bounded result] within [short period], using only [agreed data], for [fixed pilot price].

Every bracket matters. “Small businesses” is not a specific buyer. “Save time” is not an observable result. “AI-powered” says nothing about what will be delivered. If the sentence remains vague, the product is still being designed around a technology rather than a purchase decision.

2. Use PACT to decide whether the case is ready to test

The book's PACT screen stops a feature list from standing in for commercial evidence:

Pain: A specific failure costs time, money, risk, or credibility.
Access: You can identify and contact at least 20 people who experience it.
Clarity: The before-and-after state is observable within days.
Trust: A first version can operate without dangerous authority or hidden data use.

Suppose you built an AI tool that summarizes support conversations. “Support teams have too many tickets” is too vague. A stronger case is a customer-success manager who manually reconstructs unresolved commitments before renewal calls. The missed commitment is the pain; a reachable role and account list provide access; a reviewed renewal brief creates a visible before/after result; and a human-approved draft made only from agreed source material keeps the first version inside a defensible trust boundary.

PACT does not prove demand. It tells you whether the hypothesis is concrete and reachable enough to test. If Pain is weak, find a more costly failure. If Access is missing, change the segment or channel. If Clarity is missing, shrink the result. If Trust is weak, remove authority, restrict data, and add human review before proceeding.

3. Interview for past behavior, not compliments

“Would you use this?” invites politeness. Investigate the last real occurrence instead:

“Walk me through the last time this happened.”
“What did you do from start to finish?”
“Which part required the most checking or rework?”
“What happens if it is late or wrong?”
“Who approves a change or purchase?”
“What data could an outside tool use, and what must remain off-limits?”
“Have you paid for a workaround, contractor, or tool before?”

Listen for artifacts: spreadsheets, copied emails, screenshots, checklists, handoffs, recurring meetings, and approval steps. They reveal the real workflow and show where a generic demo will fail. Look for evidence of action—time already allocated, an existing budget, a manual workaround, a deadline, or an owner actively searching—not praise.

End by testing a next step: “I can deliver one reviewed output from one defined input during a short paid pilot. Would it be useful to examine a fixed scope and price?” A no is useful when you record why. It is cheaper than building another month around a false assumption.

4. Sell a bounded paid pilot—not an unfinished SaaS subscription

A paid pilot is a small commercial agreement designed to answer one real question: can this workflow produce a useful result for this buyer under explicit constraints?

State one buyer and use case; one input-to-output workflow; a short duration; a fixed price and payment point; the exact deliverable; acceptance criteria; buyer responsibilities; data handling and deletion rules; support and revision limits; and what happens after the pilot, including the option to stop.

Payment strengthens the evidence. A free user may tolerate an unclear workflow, skip onboarding, and disappear. A paying pilot buyer has made a real tradeoff and is more likely to evaluate the promised outcome carefully.

That does not justify an arbitrary price or scarcity theater. Match price to the narrow scope, manual work still required, and value of the result. If the buyer objects, reduce scope before automatically reducing price. A smaller credible promise is better than a large discounted promise you cannot reliably deliver.

Apply a strict revenue gate: count revenue only after cleared payment and only after the buyer received the terms. A verbal yes, sent payment link, opened proposal, and cleared payment are different states.

5. Build one vertical slice from real input to accepted result

After a pilot commitment, resist rebuilding the entire product. Build one vertical slice: one real input moving through every layer required for the buyer's result.

The book's slice crosses access control → input validation → model interaction → output validation → human review → persistence → error recovery. For its InquiryBrief example, the useful path is simple: paste an inquiry, generate a structured brief, review and edit it, then save the accepted brief. That path is narrow, but it is real. It tests far more than a screen wired to a perfect prompt.

A demo can hide behind sample data. A vertical slice must survive the buyer's permitted input and reach a deliverable they can accept or reject. Manual steps are acceptable when disclosed: you may review every output, configure an account by hand, or send the final file manually. Do not present a human-assisted pilot as fully autonomous software. Manual work helps reveal where automation is valuable; concealed manual work creates the wrong expectation.

6. Define the quality boundary and data boundary before delivery

AI output can look plausible while being wrong. “It generated something” is not an acceptance test.

The quality boundary states what must be true before delivery: required fields are present; source-linked claims trace to supplied material; unsupported claims are flagged; totals, dates, names, or identifiers pass deterministic checks; a human approves the final version; and failure produces a clear stop state instead of a silent guess.

The data boundary states what the pilot may touch: permitted sources, sensitive fields, storage location, retention period, deletion process, third-party services, and authorized access. If production data is inappropriate, use redacted or synthetic data and say what that prevents the pilot from proving.

These boundaries need not become a forty-page security document. They must make the claim honest. A synthetic-data pilot can prove workflow fit, not production readiness. A human-reviewed result can prove usefulness, not safe unattended automation. Trust grows when the evidence claim is no larger than the evidence.

7. Keep an honest ledger

Activity is easy to count: posts, messages, demos, or people who said “interesting.” An honest ledger records state changes that matter.

Field	What to record
Buyer and role	User, approver, and payer if different
Problem evidence	Their words and the last real occurrence
Trigger	Deadline or event creating urgency
Offer	Exact pilot scope and price
Current state	Contacted, interviewed, proposed, paid, delivered, accepted, declined
Objection	Price, timing, trust, scope, data, authority, or no priority
Next action	Named action and date
Cash state	Invoiced, cleared, refunded, or unpaid

Never silently promote a lead. “Interested” is not “proposed.” “Proposed” is not “paid.” “Delivered” is not “accepted.” This separation tells you what to fix. Interviews without proposals may expose weak positioning or reluctance to ask. Proposals without payment point to urgency, authority, price, or trust. Payment followed by rejected delivery points to the quality boundary or acceptance criteria.

At the end of each cycle, write five lines: what was promised, what was paid, what was delivered, what was learned, and what is owed next. That is the honest commercial record—not a motivational dashboard.

8. A practical 72-hour sequence

Seventy-two hours is a decision window, not an earnings guarantee. Use it to force evidence-producing actions.

Hours 0–8: choose one case

Complete one PACT screen. Select a buyer you can reach, one recurring workflow, and one trigger. Write the one-sentence offer. Remove every feature not required for the bounded result.

Hours 8–24: collect buyer evidence

Build a small, relevant contact list and conduct short conversations. Ask about past behavior and constraints. Update PACT with what you learn. Do not automate mass outreach or misrepresent a relationship.

Hours 24–36: make the paid-pilot offer

Send a short scope with deliverable, timing, fixed price, acceptance criteria, data boundary, and stop condition. Provide a legitimate payment or invoicing path. Record the actual state in the ledger.

Hours 36–60: build or adapt the vertical slice

Only after a real commitment, implement the smallest end-to-end workflow. Test it with permitted data, add checks required by the quality boundary, and keep human review where automation is not yet trustworthy.

Hours 60–72: deliver evidence and decide

Deliver the agreed artifact, collect acceptance or rejection, and record what happened. Then choose one honest outcome:

Continue: the buyer paid, accepted the result, and has a credible next use.
Revise: the problem is real, but scope, quality, or delivery needs a bounded correction.
Stop or reposition: urgency, authority, access, or willingness to pay is absent.

All three outcomes are useful. Only one is revenue.

The real fix is changing the order of evidence

A paid-pilot-first approach does not eliminate product risk. It prevents you from hiding that risk behind more features.

Start with a buyer's real workflow. Screen it with Pain, Access, Clarity, and Trust. Interview for past behavior. Ask for a bounded paid commitment. Build one honest vertical slice. State the quality and data boundaries. Record every commercial state without inflation.

You may still hear no. You may learn that the trigger is weak, the buyer lacks authority, the data cannot be used, or the result is not valuable enough. That is evidence acquired before another cycle of overbuilding.

The goal is not merely to ship faster. It is to make each hour of building answer a purchase-relevant question.

Want the complete 72-hour field guide?

Get Paid Before You Overbuild expands this framework into a professionally designed 97-page English guide with buyer-interview scripts, a one-page offer template, paid-pilot terms, AI quality and data-boundary checks, a complete 72-hour operating schedule, and practical revenue and delivery worksheets.

Get the guide for $19 from Evidence Gate Studio

The title describes a sequencing method, not guaranteed earnings. Results depend on buyer access, fit, trust, pricing, delivery, and other factors.

Free interactive tool: AI App Launch Failure Simulator

If you want to test the idea before buying anything, I built a free interactive simulator:

AI App Launch Failure Simulator

Find the launch failure hiding in your AI app before a user does.

It walks through your app type, stack risk, failure scenarios, and proof gates, then gives you a launch-risk score, the top hidden risks, and a 72-hour rescue plan.

Run the free simulator

Need the smaller operating kit?

If you are not ready for the full guide yet, start with the smaller $9 operating kit:

Paid Pilot Starter Kit

Buyer interview worksheet, one-page paid pilot offer, scope/data boundary checklist, and lead/revenue ledger.

https://payhip.com/b/AVeCb?utm_source=devto&utm_medium=referral_content&utm_campaign=paid_pilot_kit_launch

Already have an AI-built app?

If you already have one critical flow you are about to show users, I offer a 48-hour preflight review:

Free Stripe final-state checker

If your AI-built app uses Stripe, do not stop at a successful checkout redirect or a webhook 200.

Check the final business state:

paid → entitled
canceled → downgraded
retry → not duplicated
refund → access and ledger updated
support → can inspect one user's payment/access state

Free checker:
https://bespoke-pika-937432.netlify.app/stripe-webhook-final-state-checker?utm_source=devto&utm_medium=article_cta&utm_campaign=stripe_final_state

Free AI app launch proof hub

I also made a small free tool hub for AI-built apps:

https://bespoke-pika-937432.netlify.app/tools?utm_source=devto&utm_medium=article_cta&utm_campaign=v16_tool_hub

It includes:

a Vibe-Coded App Reality Score;
a Stripe webhook final-state checker;
a Supabase RLS/Auth exposure checker;
an AI output trust-boundary checker;
a sample one-flow preflight report;
an AI app failure scenario gallery.

If you want the self-serve worksheet, the $5 Final-State Checklist Pack is here:

https://payhip.com/b/TIe6v?utm_source=devto&utm_medium=article_cta&utm_campaign=v16_final_state_checklist

I also added a transparent Sample One-Flow Preflight Report for the paid one-flow preflight.

It is a fictional sample, not a fake client case study:

https://bespoke-pika-937432.netlify.app/sample-one-flow-preflight-report?utm_source=devto&utm_medium=article_cta&utm_campaign=sample_preflight_report

If your launch flow touches payment, private data, or AI output, the useful question is not “does the demo work?”

It is:

What final state can you prove when the real user path gets messy?

5 Proof Gates Between an AI Demo and a Shippable MVP

Eric Choi | Evidence Gate Studio — Sun, 19 Jul 2026 06:18:01 +0000

AI coding agents have dramatically shortened the distance between an idea and working software. They can inspect a project, create files, run commands, write tests, and help diagnose failures.

What they have not eliminated is judgment.

A polished screen is not proof that data survives a reload. A passing unit test is not proof that keyboard users can complete the core task. A successful deployment is not proof that the intended commit reached production.

This is why I use proof gates: observable conditions that must be satisfied before a product claim becomes stronger.

A proof gate is not a meeting, a long document, or an excuse to slow down. It is a compact question:

What evidence would let another person verify that this claim is true?

Here are five gates that separate a persuasive AI demo from a small MVP you can responsibly ship.

Gate 1: Prove One Valuable User Loop

AI makes feature generation cheap, which makes uncontrolled scope especially dangerous.

Before requesting code, define one primary user in one specific situation. Then describe:

Their observable before-state
The smallest useful action they can take
The immediate result
The reason they might return

This becomes the product’s core loop.

For example, “build a productivity platform” is too broad. A more testable loop might be:

A freelancer remembers a useful client outcome.
They record the outcome and supporting evidence.
The record appears in a searchable library.
They can retrieve it later for a proposal or review.

The gate is not passed because a form exists. It is passed when a new user can complete the entire loop and explain what changed without coaching.

Write a Not Today list alongside the required capabilities. Authentication, dashboards, collaboration, billing, and AI-generated summaries may all be reasonable later. They should not compete with proof of the first useful loop.

The goal is not the fewest possible features. It is the smallest complete behavior that tests whether the product creates value.

Gate 2: Prove the Data and Recovery Contract

Many demos treat persistence as an implementation detail. For a real user, persistence is part of the product promise.

Define the data contract before allowing generated code to spread storage assumptions across the application. For every persisted field, specify:

Name and type
Whether it is required
Length or count limits
Normalization rules
User-facing meaning

Persisted data should carry enough version information to be interpreted safely. A backup envelope might include:

product
formatVersion
schemaVersion
exportedAt
records

The application should also distinguish between states that are often incorrectly collapsed into “empty”:

No data exists
Current data is valid
Older data was migrated successfully
Stored data is malformed or unsupported
Storage is unavailable
A write failed because of quota or access restrictions

A dangerous pattern is catching a parse error, returning an empty array, and later saving that empty state over the user’s last valid data. Recovery behavior must be designed, not improvised inside an error handler.

The strongest practical test is a real round trip:

Create representative records.
Export a backup.
Delete the application data through normal controls.
Import the backup.
Reload the application.
Verify that the intended records and fields survived.

A download button proves that a file can be created. The round trip proves that recovery works.

Gate 3: Prove AI Changes With Bounded Prompts

“Build the rest of the app” gives an agent too much freedom and gives the reviewer too little evidence.

A bounded implementation prompt should contain five parts:

Goal:
Boundaries:
Acceptance criteria:
Verification:
Report:

The goal names one user-visible outcome. Boundaries state what must not change. Acceptance criteria describe observable pass-or-fail behavior. Verification lists the tests and commands that must run. The report requires exact results, changed files, remaining risks, and unverified behavior.

For bugs, separate diagnosis from editing. Provide expected behavior, actual behavior, reproduction steps, and the first relevant error. Ask the agent to trace the code path and identify the smallest experiment that would confirm the suspected cause.

Only then request the focused fix and its regression test.

This reduces opportunistic refactoring and makes failures easier to understand. It also creates a clean review question: did this change satisfy its stated contract without changing unrelated behavior?

Fast code is useful. Reviewable code is shippable.

Gate 4: Prove Behavior, Failure States, and Accessibility

A green test suite can still coexist with a broken product if the tests assert implementation details instead of user-visible behavior.

Test the domain rules, but also test the complete actions users depend on:

Required fields reject invalid input.
Valid records survive reload.
Editing preserves identity and creation metadata.
Cancel operations change nothing.
Deletion requires confirmation.
Search and combined filters return deterministic results.
Failed imports leave current data unchanged.
Selected exports contain only the intended records.

Then check the interface beyond the happy path.

Keyboard order should follow the task. Focus should remain visible and return logically after dialogs. Errors should be associated with the relevant controls. Essential actions should remain usable at 320-pixel width and 200% zoom.

Automated accessibility tools are valuable, but they cannot prove the whole interaction. A manual keyboard path and narrow-layout check often expose problems that component tests miss.

Report results in three groups: passed, failed, and unverified. “No automated violations found” is not the same claim as “the workflow is accessible.”

Gate 5: Prove the Release That Users Actually Receive

A successful local build is evidence about the build, not the deployment.

Before release, reconcile documentation with implemented behavior. Run the complete relevant test suite, type check, configured lint checks, and production build. Inspect for secrets, debug output, dead placeholders, and unexplained console errors. Record the release commit and a known rollback target.

After deployment, verify the live product in a clean session:

HTTPS loads correctly.
The host serves the intended release.
Static assets and metadata load.
The core loop works.
Persisted data survives reload.
Error and recovery states remain usable.
Keyboard and narrow-screen behavior still work.
The browser console and network panel show no unexplained failures.

If a high-impact live check fails, the honest outcome is a diagnosed release problem—not a successful launch announcement.

The “one-day MVP” frame can be useful, but it should operate as a scope clock. It is not permission to skip data safety, recovery, accessibility, or release verification.

Keep a Small Evidence Record

For each gate, record:

The claim being evaluated
Reproduction steps
Tests or commands run
Manual observations
Passed, failed, and unverified items
The rollback point

This record does not need to be elaborate. Its purpose is to stop confidence from becoming detached from evidence.

AI can help us build much faster. The corresponding professional skill is learning when the available proof justifies saying, “This is ready.”

If you want the complete workflow, From Idea to Verified MVP is a 32-page evidence-first guide with a separate 7-page copyable companion pack of briefs, contracts, prompt templates, schedules, and release checks: Get the bundle here.

How to Test Stripe Webhook Idempotency: Assert the Final Business State, Not the 200

Eric Choi | Evidence Gate Studio — Sat, 18 Jul 2026 21:17:31 +0000

Your Stripe webhook returned 200 OK.

That proves one useful but narrow fact: the endpoint acknowledged an event delivery. It does not prove that the customer received exactly one entitlement, that an order was recorded once, or that a retry cannot repeat a downstream action.

This distinction matters because Stripe recommends that webhook endpoints return a successful 2xx response quickly before complex work causes a timeout. Stripe can also retry failed deliveries, send duplicate events, and deliver events out of order. A fast 200 is part of a healthy transport layer. It is not the acceptance criterion for your business workflow.

The more useful question is:

After every permitted delivery pattern, does the system end in the one business state we intended?

Start with one explicit invariant

Do not begin with “test all Stripe webhooks.” Pick one path.

For example:

When account A completes Checkout Session cs_test_123 for product P, the system ends with one paid order and one active entitlement for account A. Replaying or concurrently processing relevant events must not create another order, another entitlement, or access for any other account.

That sentence gives you something observable. You can query your own database after each test and count the records that matter.

If your product handles refunds, cancellations, or disputes, add the expected policy rather than assuming one. A refund might revoke access immediately, at the end of a billing period, or only after a manual decision. Idempotency means reaching the same defined result repeatedly; it does not choose the correct business policy for you.

Separate three identities

Many implementations keep only event.id. That is useful, but it solves only one layer of duplication.

1. Delivery identity

The Stripe Event ID identifies a particular Event object. Stripe’s webhook guidance recommends logging processed event IDs so the same event is not processed twice.

Use a unique constraint on this value. An in-memory set is not sufficient because it disappears on restart and does not coordinate multiple workers.

2. Provider object identity

Stripe notes that separate Event objects can sometimes represent the same underlying object and event type. Its guidance suggests using the ID in data.object together with event.type when identifying that case.

For a Checkout flow, the relevant provider identity might be a Checkout Session, PaymentIntent, Invoice, or Subscription ID. Which one is authoritative depends on the path you are testing.

3. Business identity

Your application also needs its own durable identity: an order ID, account-and-product entitlement, invoice allocation, shipment, or credit.

This is the layer Stripe cannot define for you.

Two different Stripe events might legitimately point toward the same business effect. Preventing a second processing of one Event ID does not automatically prevent two different events from granting the same entitlement. A domain-level unique rule such as one order per Checkout Session, or one active entitlement per account and purchased item, provides a second boundary.

Do not confuse API idempotency keys with webhook idempotency

Stripe’s API idempotency keys protect retryable outbound API requests. For API v1, Stripe accepts idempotency keys on POST requests and returns the stored result when the same request is retried with the same key.

That is valuable when your server calls Stripe to create or update an object.

It does not automatically make your inbound webhook handler safe. Your own database writes, access grants, emails, queue jobs, and third-party calls still need durable deduplication or idempotent behavior.

Use each mechanism at its actual boundary:

Stripe API idempotency key for a retryable request sent to Stripe;
Stripe Event ID for duplicate delivery receipts;
provider object plus event type when separate Event objects describe the same object transition;
your own business key or uniqueness rule for the final side effect.

Make the receipt durable before acknowledging it

A practical handler can use a small sequence:

Verify the webhook signature against the raw request body.
Attempt to insert a receipt row with a unique Stripe Event ID.
If it already exists, return success without repeating the work.
Durably enqueue or record the business command.
Return 2xx.
Let a worker apply the command under the relevant business uniqueness rule.
Record the observed final state or processing result.

Stripe specifically warns that signature verification requires the raw request body; a framework that changes the body before verification can cause verification to fail. Use an official Stripe library where possible.

The exact database and queue design will vary. The important failure boundary is durability. If the process acknowledges the event before any durable receipt or command exists, a crash can produce a successful delivery with no recoverable work. If it performs the business action and crashes before recording completion, a retry can repeat the action unless that action is independently idempotent.

Run these six tests in a sandbox

Stripe provides sandboxes and test values so these checks do not move real money. Its testing guidance says not to test live mode with real payment details.

Test 1: Establish the normal-path baseline

Complete one real sandbox Checkout path. Record:

the account and product;
Checkout Session and relevant payment object IDs;
the starting order and entitlement counts;
the expected final values; and
the actual database rows after processing.

A green success page is supporting evidence, not the final assertion. Query the stored order, entitlement, and event receipt.

Test 2: Redeliver the exact same event

Use the Dashboard’s Resend action or the documented CLI form:

stripe events resend EVENT_ID --webhook-endpoint=ENDPOINT_ID

Stripe says manual resend does not cancel any existing automatic retry, even after a 2xx. That makes this a useful real-world duplicate test.

After the resend, assert that the receipt attempts may have increased but the business result has not:

one order;
one entitlement;
no repeated fulfillment transition;
no duplicate downstream job that matters to the customer.

Test 3: Reproduce the commit-before-ack failure window

In a controlled test environment, add a temporary failure point after the database commit but before the response is returned. Let the delivery fail or time out, then resend the same event.

This tests the uncomfortable case where the business write succeeded but Stripe did not receive the acknowledgment.

The pass condition is not “the second request returned 200.” It is that the second attempt recognized the completed work and left the business state unchanged.

Test 4: Process the same command concurrently

Stripe’s Checkout fulfillment guide explicitly says a fulfillment function might be called multiple times, possibly concurrently, for the same Checkout Session.

Run two workers against the same event or fulfillment command at the same time. Application-level “check, then insert” logic can fail here because both workers can observe absence before either inserts.

The durable pass condition usually comes from a database uniqueness rule, transaction, atomic upsert, or compare-and-set transition—not timing luck.

Test 5: Use two Event objects that target one business effect

Exercise a case where different relevant Event objects could reach the same fulfillment path. Your exact combination depends on the integration; do not invent an event mapping that your application does not use.

Assert the domain invariant again. The Event IDs can both be valid and unique while the resulting order or entitlement must still exist only once.

Test 6: Reverse the assumed order

Stripe does not guarantee event delivery order and advises integrations not to depend on it. For subscriptions, for example, related subscription, invoice, and charge events might not arrive in the sequence your happy-path log showed.

Separate signature verification and transport handling from your domain processor so recorded sandbox fixtures can be applied to that processor in controlled orders. Where appropriate, retrieve the current Stripe object instead of assuming an earlier event has already arrived.

Then assert your defined final state after each sequence. Also rerun the legitimate normal path so a deduplication fix does not block valid renewals, upgrades, or later purchases.

Record evidence that answers a business question

For every run, keep a compact, sanitized record:

Field	Example purpose
Code version and environment	Identifies exactly what was tested
Stripe Event ID and type	Tracks the delivery
Provider object ID	Connects related events
Internal business key	Identifies the order or entitlement
Starting state	Shows the controlled baseline
Injected condition	Duplicate, timeout, concurrency, or reordering
Expected final state	Defines the invariant
Observed final state	Database counts and status values
Sanitized log or test output	Supports reproduction
Limits	States what was not tested

Stripe’s CLI can forward sandbox events to a local endpoint and trigger common event fixtures. For a realistic end-to-end check, however, prefer a genuine sandbox Checkout or test subscription whose objects correlate with your application records. Stripe’s Billing testing guide notes that CLI- or Dashboard-triggered subscription events can contain fake data that does not correspond to actual subscription information.

A passing webhook test is therefore not “we saw 200 in the log.” It is:

Under the tested duplicate, failure, concurrency, and ordering conditions, the same named business invariant remained true.

That conclusion is deliberately narrow. It is not a security certification, a guarantee about every payment path, or proof about an untested production environment. It is useful evidence for one revenue-critical workflow.

If you want an independent second pass on one checkout → webhook → entitlement path, the one-path Evidence Gate uses this same evidence-limited approach.

Disclosure: This article was prepared with AI assistance and checked against current Stripe documentation. It does not claim production Stripe experience or certification.

Official references

9 checks before you launch an AI-built web app

Eric Choi | Evidence Gate Studio — Sat, 18 Jul 2026 19:49:34 +0000

AI tools can take a product from an empty repository to a convincing demo quickly. That is useful, but a smooth happy path does not answer a harder question:

What happens when users, permissions, retries, and partial failures do not follow the demo script?

This does not mean AI-generated code is automatically bad. Human-written code can fail in the same ways. The practical problem is that generation often moves faster than the team’s ability to define boundaries, write negative tests, and record why a launch-critical behavior should be trusted.

You do not have to review the whole universe to make progress. Pick one 3–6-screen journey—signup, onboarding, checkout, or create / save / export—and run these nine checks.

1. Name the exact user and outcome

Do not start with “review the backend” or “check our Supabase app.” Write one sentence that identifies:

the user or role;
the first action;
the expected final state; and
the data, money, or permission that changes.

Example:

A standard member completes checkout once and receives exactly one active entitlement tied to the correct account.

Specific wording gives you observable pass and fail conditions.

2. Record the version and environment

A test result is difficult to trust if the target keeps changing. Record:

the commit or deployed version;
local, preview, or staging environment;
test accounts and roles;
sanitized data used; and
anything that cannot be tested safely.

The honest conclusion is then “this behavior was checked under these conditions,” not “the app is safe.”

3. Map each trust decision

Write the flow in plain language:

The user takes an action.
The client sends a request.
The server identifies the user and role.
A rule decides what is allowed.
Data or an entitlement changes.
The user sees a result.

At every step, ask what identity is trusted, what values the caller can choose, and where the rule is actually enforced.

4. Try the wrong user or tenant

For an authorized test account, safely ask whether changing an object or tenant identifier can expose or modify another user’s data.

The UI hiding a button is not enough. The relevant server or database boundary should reject an unauthorized action.

Use sanitized test records only. Do not probe systems you do not own or have permission to test.

5. Repeat the same action

Retries happen because browsers refresh, networks time out, users double-click, and providers redeliver events.

Repeat the same safe request or test event and check whether it creates:

duplicate records;
duplicate entitlements;
contradictory status;
repeated notifications; or
an incorrect charge in a test-mode system.

Record the expected idempotent behavior before running the check.

6. Change the order

External events may arrive late or out of order. In a controlled test environment, check whether a delayed cancellation, completion, or retry can overwrite a newer valid state.

If order matters, the system should use durable state rules rather than assuming events always arrive in the demo sequence.

7. Stop halfway

Ask what remains if the flow fails after one write but before the next.

For example:

a payment record exists but no entitlement was granted;
a file was stored but its database record failed;
a user was invited but the authorization relationship was not created.

The desired outcome may be a transaction, a retryable state, cleanup, or a clearly visible recovery path. What matters is that the half-finished state is intentional and observable.

8. Test both denial and legitimate use

Every fix or rule needs two checks:

a negative test showing the unwanted action is rejected; and
a normal-path test showing the correct user can still finish the flow.

Without both, it is easy to “fix” unauthorized access by breaking access for everyone or to stop duplicates by blocking valid retries.

9. Write down evidence and limits

For each check, record:

starting role and state;
exact safe action;
expected result;
observed result;
relevant screenshot, log, or test output with secrets removed; and
anything blocked or still outside scope.

If the behavior cannot be reproduced, say so. If access or setup prevents the check, record the blocker. Uncertainty is a result; it should not be turned into a dramatic finding.

A small next step

If you want a second set of eyes without opening your codebase, the AI App One-Flow Preflight reviews one submitted 3–6-screen journey from a sanitized recording of up to 10 minutes or up to eight screenshots.

For USD 129, the fixed deliverable is a flow map, up to three evidence-linked hypotheses, ten manual tests with expected results, a Now / Next / Later action plan, and one asynchronous follow-up. Turnaround is within two business days after complete accepted materials.

Applying is free, and a misfit application stops before checkout:

https://payhip.com/b/1FgDW

This is an evidence-limited product review, not source-code review, production testing, a security audit, penetration test, certification, defect guarantee, or approval to launch.

Disclosure: This article was prepared with AI assistance and manually reviewed for scope, factual claims, and safety boundaries.