DEV Community: Eric Choi | Evidence Gate Studio

5 Proof Gates Between an AI Demo and a Shippable MVP

Eric Choi | Evidence Gate Studio — Sun, 19 Jul 2026 06:18:01 +0000

AI coding agents have dramatically shortened the distance between an idea and working software. They can inspect a project, create files, run commands, write tests, and help diagnose failures.

What they have not eliminated is judgment.

A polished screen is not proof that data survives a reload. A passing unit test is not proof that keyboard users can complete the core task. A successful deployment is not proof that the intended commit reached production.

This is why I use proof gates: observable conditions that must be satisfied before a product claim becomes stronger.

A proof gate is not a meeting, a long document, or an excuse to slow down. It is a compact question:

What evidence would let another person verify that this claim is true?

Here are five gates that separate a persuasive AI demo from a small MVP you can responsibly ship.

Gate 1: Prove One Valuable User Loop

AI makes feature generation cheap, which makes uncontrolled scope especially dangerous.

Before requesting code, define one primary user in one specific situation. Then describe:

Their observable before-state
The smallest useful action they can take
The immediate result
The reason they might return

This becomes the product’s core loop.

For example, “build a productivity platform” is too broad. A more testable loop might be:

A freelancer remembers a useful client outcome.
They record the outcome and supporting evidence.
The record appears in a searchable library.
They can retrieve it later for a proposal or review.

The gate is not passed because a form exists. It is passed when a new user can complete the entire loop and explain what changed without coaching.

Write a Not Today list alongside the required capabilities. Authentication, dashboards, collaboration, billing, and AI-generated summaries may all be reasonable later. They should not compete with proof of the first useful loop.

The goal is not the fewest possible features. It is the smallest complete behavior that tests whether the product creates value.

Gate 2: Prove the Data and Recovery Contract

Many demos treat persistence as an implementation detail. For a real user, persistence is part of the product promise.

Define the data contract before allowing generated code to spread storage assumptions across the application. For every persisted field, specify:

Name and type
Whether it is required
Length or count limits
Normalization rules
User-facing meaning

Persisted data should carry enough version information to be interpreted safely. A backup envelope might include:

product
formatVersion
schemaVersion
exportedAt
records

The application should also distinguish between states that are often incorrectly collapsed into “empty”:

No data exists
Current data is valid
Older data was migrated successfully
Stored data is malformed or unsupported
Storage is unavailable
A write failed because of quota or access restrictions

A dangerous pattern is catching a parse error, returning an empty array, and later saving that empty state over the user’s last valid data. Recovery behavior must be designed, not improvised inside an error handler.

The strongest practical test is a real round trip:

Create representative records.
Export a backup.
Delete the application data through normal controls.
Import the backup.
Reload the application.
Verify that the intended records and fields survived.

A download button proves that a file can be created. The round trip proves that recovery works.

Gate 3: Prove AI Changes With Bounded Prompts

“Build the rest of the app” gives an agent too much freedom and gives the reviewer too little evidence.

A bounded implementation prompt should contain five parts:

Goal:
Boundaries:
Acceptance criteria:
Verification:
Report:

The goal names one user-visible outcome. Boundaries state what must not change. Acceptance criteria describe observable pass-or-fail behavior. Verification lists the tests and commands that must run. The report requires exact results, changed files, remaining risks, and unverified behavior.

For bugs, separate diagnosis from editing. Provide expected behavior, actual behavior, reproduction steps, and the first relevant error. Ask the agent to trace the code path and identify the smallest experiment that would confirm the suspected cause.

Only then request the focused fix and its regression test.

This reduces opportunistic refactoring and makes failures easier to understand. It also creates a clean review question: did this change satisfy its stated contract without changing unrelated behavior?

Fast code is useful. Reviewable code is shippable.

Gate 4: Prove Behavior, Failure States, and Accessibility

A green test suite can still coexist with a broken product if the tests assert implementation details instead of user-visible behavior.

Test the domain rules, but also test the complete actions users depend on:

Required fields reject invalid input.
Valid records survive reload.
Editing preserves identity and creation metadata.
Cancel operations change nothing.
Deletion requires confirmation.
Search and combined filters return deterministic results.
Failed imports leave current data unchanged.
Selected exports contain only the intended records.

Then check the interface beyond the happy path.

Keyboard order should follow the task. Focus should remain visible and return logically after dialogs. Errors should be associated with the relevant controls. Essential actions should remain usable at 320-pixel width and 200% zoom.

Automated accessibility tools are valuable, but they cannot prove the whole interaction. A manual keyboard path and narrow-layout check often expose problems that component tests miss.

Report results in three groups: passed, failed, and unverified. “No automated violations found” is not the same claim as “the workflow is accessible.”

Gate 5: Prove the Release That Users Actually Receive

A successful local build is evidence about the build, not the deployment.

Before release, reconcile documentation with implemented behavior. Run the complete relevant test suite, type check, configured lint checks, and production build. Inspect for secrets, debug output, dead placeholders, and unexplained console errors. Record the release commit and a known rollback target.

After deployment, verify the live product in a clean session:

HTTPS loads correctly.
The host serves the intended release.
Static assets and metadata load.
The core loop works.
Persisted data survives reload.
Error and recovery states remain usable.
Keyboard and narrow-screen behavior still work.
The browser console and network panel show no unexplained failures.

If a high-impact live check fails, the honest outcome is a diagnosed release problem—not a successful launch announcement.

The “one-day MVP” frame can be useful, but it should operate as a scope clock. It is not permission to skip data safety, recovery, accessibility, or release verification.

Keep a Small Evidence Record

For each gate, record:

The claim being evaluated
Reproduction steps
Tests or commands run
Manual observations
Passed, failed, and unverified items
The rollback point

This record does not need to be elaborate. Its purpose is to stop confidence from becoming detached from evidence.

AI can help us build much faster. The corresponding professional skill is learning when the available proof justifies saying, “This is ready.”

If you want the complete workflow, From Idea to Verified MVP is a 32-page evidence-first guide with a separate 7-page copyable companion pack of briefs, contracts, prompt templates, schedules, and release checks: Get the bundle here.

How to Test Stripe Webhook Idempotency: Assert the Final Business State, Not the 200

Eric Choi | Evidence Gate Studio — Sat, 18 Jul 2026 21:17:31 +0000

Your Stripe webhook returned 200 OK.

That proves one useful but narrow fact: the endpoint acknowledged an event delivery. It does not prove that the customer received exactly one entitlement, that an order was recorded once, or that a retry cannot repeat a downstream action.

This distinction matters because Stripe recommends that webhook endpoints return a successful 2xx response quickly before complex work causes a timeout. Stripe can also retry failed deliveries, send duplicate events, and deliver events out of order. A fast 200 is part of a healthy transport layer. It is not the acceptance criterion for your business workflow.

The more useful question is:

After every permitted delivery pattern, does the system end in the one business state we intended?

Start with one explicit invariant

Do not begin with “test all Stripe webhooks.” Pick one path.

For example:

When account A completes Checkout Session cs_test_123 for product P, the system ends with one paid order and one active entitlement for account A. Replaying or concurrently processing relevant events must not create another order, another entitlement, or access for any other account.

That sentence gives you something observable. You can query your own database after each test and count the records that matter.

If your product handles refunds, cancellations, or disputes, add the expected policy rather than assuming one. A refund might revoke access immediately, at the end of a billing period, or only after a manual decision. Idempotency means reaching the same defined result repeatedly; it does not choose the correct business policy for you.

Separate three identities

Many implementations keep only event.id. That is useful, but it solves only one layer of duplication.

1. Delivery identity

The Stripe Event ID identifies a particular Event object. Stripe’s webhook guidance recommends logging processed event IDs so the same event is not processed twice.

Use a unique constraint on this value. An in-memory set is not sufficient because it disappears on restart and does not coordinate multiple workers.

2. Provider object identity

Stripe notes that separate Event objects can sometimes represent the same underlying object and event type. Its guidance suggests using the ID in data.object together with event.type when identifying that case.

For a Checkout flow, the relevant provider identity might be a Checkout Session, PaymentIntent, Invoice, or Subscription ID. Which one is authoritative depends on the path you are testing.

3. Business identity

Your application also needs its own durable identity: an order ID, account-and-product entitlement, invoice allocation, shipment, or credit.

This is the layer Stripe cannot define for you.

Two different Stripe events might legitimately point toward the same business effect. Preventing a second processing of one Event ID does not automatically prevent two different events from granting the same entitlement. A domain-level unique rule such as one order per Checkout Session, or one active entitlement per account and purchased item, provides a second boundary.

Do not confuse API idempotency keys with webhook idempotency

Stripe’s API idempotency keys protect retryable outbound API requests. For API v1, Stripe accepts idempotency keys on POST requests and returns the stored result when the same request is retried with the same key.

That is valuable when your server calls Stripe to create or update an object.

It does not automatically make your inbound webhook handler safe. Your own database writes, access grants, emails, queue jobs, and third-party calls still need durable deduplication or idempotent behavior.

Use each mechanism at its actual boundary:

Stripe API idempotency key for a retryable request sent to Stripe;
Stripe Event ID for duplicate delivery receipts;
provider object plus event type when separate Event objects describe the same object transition;
your own business key or uniqueness rule for the final side effect.

Make the receipt durable before acknowledging it

A practical handler can use a small sequence:

Verify the webhook signature against the raw request body.
Attempt to insert a receipt row with a unique Stripe Event ID.
If it already exists, return success without repeating the work.
Durably enqueue or record the business command.
Return 2xx.
Let a worker apply the command under the relevant business uniqueness rule.
Record the observed final state or processing result.

Stripe specifically warns that signature verification requires the raw request body; a framework that changes the body before verification can cause verification to fail. Use an official Stripe library where possible.

The exact database and queue design will vary. The important failure boundary is durability. If the process acknowledges the event before any durable receipt or command exists, a crash can produce a successful delivery with no recoverable work. If it performs the business action and crashes before recording completion, a retry can repeat the action unless that action is independently idempotent.

Run these six tests in a sandbox

Stripe provides sandboxes and test values so these checks do not move real money. Its testing guidance says not to test live mode with real payment details.

Test 1: Establish the normal-path baseline

Complete one real sandbox Checkout path. Record:

the account and product;
Checkout Session and relevant payment object IDs;
the starting order and entitlement counts;
the expected final values; and
the actual database rows after processing.

A green success page is supporting evidence, not the final assertion. Query the stored order, entitlement, and event receipt.

Test 2: Redeliver the exact same event

Use the Dashboard’s Resend action or the documented CLI form:

stripe events resend EVENT_ID --webhook-endpoint=ENDPOINT_ID

Stripe says manual resend does not cancel any existing automatic retry, even after a 2xx. That makes this a useful real-world duplicate test.

After the resend, assert that the receipt attempts may have increased but the business result has not:

one order;
one entitlement;
no repeated fulfillment transition;
no duplicate downstream job that matters to the customer.

Test 3: Reproduce the commit-before-ack failure window

In a controlled test environment, add a temporary failure point after the database commit but before the response is returned. Let the delivery fail or time out, then resend the same event.

This tests the uncomfortable case where the business write succeeded but Stripe did not receive the acknowledgment.

The pass condition is not “the second request returned 200.” It is that the second attempt recognized the completed work and left the business state unchanged.

Test 4: Process the same command concurrently

Stripe’s Checkout fulfillment guide explicitly says a fulfillment function might be called multiple times, possibly concurrently, for the same Checkout Session.

Run two workers against the same event or fulfillment command at the same time. Application-level “check, then insert” logic can fail here because both workers can observe absence before either inserts.

The durable pass condition usually comes from a database uniqueness rule, transaction, atomic upsert, or compare-and-set transition—not timing luck.

Test 5: Use two Event objects that target one business effect

Exercise a case where different relevant Event objects could reach the same fulfillment path. Your exact combination depends on the integration; do not invent an event mapping that your application does not use.

Assert the domain invariant again. The Event IDs can both be valid and unique while the resulting order or entitlement must still exist only once.

Test 6: Reverse the assumed order

Stripe does not guarantee event delivery order and advises integrations not to depend on it. For subscriptions, for example, related subscription, invoice, and charge events might not arrive in the sequence your happy-path log showed.

Separate signature verification and transport handling from your domain processor so recorded sandbox fixtures can be applied to that processor in controlled orders. Where appropriate, retrieve the current Stripe object instead of assuming an earlier event has already arrived.

Then assert your defined final state after each sequence. Also rerun the legitimate normal path so a deduplication fix does not block valid renewals, upgrades, or later purchases.

Record evidence that answers a business question

For every run, keep a compact, sanitized record:

Field	Example purpose
Code version and environment	Identifies exactly what was tested
Stripe Event ID and type	Tracks the delivery
Provider object ID	Connects related events
Internal business key	Identifies the order or entitlement
Starting state	Shows the controlled baseline
Injected condition	Duplicate, timeout, concurrency, or reordering
Expected final state	Defines the invariant
Observed final state	Database counts and status values
Sanitized log or test output	Supports reproduction
Limits	States what was not tested

Stripe’s CLI can forward sandbox events to a local endpoint and trigger common event fixtures. For a realistic end-to-end check, however, prefer a genuine sandbox Checkout or test subscription whose objects correlate with your application records. Stripe’s Billing testing guide notes that CLI- or Dashboard-triggered subscription events can contain fake data that does not correspond to actual subscription information.

A passing webhook test is therefore not “we saw 200 in the log.” It is:

Under the tested duplicate, failure, concurrency, and ordering conditions, the same named business invariant remained true.

That conclusion is deliberately narrow. It is not a security certification, a guarantee about every payment path, or proof about an untested production environment. It is useful evidence for one revenue-critical workflow.

If you want an independent second pass on one checkout → webhook → entitlement path, the one-path Evidence Gate uses this same evidence-limited approach.

Disclosure: This article was prepared with AI assistance and checked against current Stripe documentation. It does not claim production Stripe experience or certification.

Official references

9 checks before you launch an AI-built web app

Eric Choi | Evidence Gate Studio — Sat, 18 Jul 2026 19:49:34 +0000

AI tools can take a product from an empty repository to a convincing demo quickly. That is useful, but a smooth happy path does not answer a harder question:

What happens when users, permissions, retries, and partial failures do not follow the demo script?

This does not mean AI-generated code is automatically bad. Human-written code can fail in the same ways. The practical problem is that generation often moves faster than the team’s ability to define boundaries, write negative tests, and record why a launch-critical behavior should be trusted.

You do not have to review the whole universe to make progress. Pick one 3–6-screen journey—signup, onboarding, checkout, or create / save / export—and run these nine checks.

1. Name the exact user and outcome

Do not start with “review the backend” or “check our Supabase app.” Write one sentence that identifies:

the user or role;
the first action;
the expected final state; and
the data, money, or permission that changes.

Example:

A standard member completes checkout once and receives exactly one active entitlement tied to the correct account.

Specific wording gives you observable pass and fail conditions.

2. Record the version and environment

A test result is difficult to trust if the target keeps changing. Record:

the commit or deployed version;
local, preview, or staging environment;
test accounts and roles;
sanitized data used; and
anything that cannot be tested safely.

The honest conclusion is then “this behavior was checked under these conditions,” not “the app is safe.”

3. Map each trust decision

Write the flow in plain language:

The user takes an action.
The client sends a request.
The server identifies the user and role.
A rule decides what is allowed.
Data or an entitlement changes.
The user sees a result.

At every step, ask what identity is trusted, what values the caller can choose, and where the rule is actually enforced.

4. Try the wrong user or tenant

For an authorized test account, safely ask whether changing an object or tenant identifier can expose or modify another user’s data.

The UI hiding a button is not enough. The relevant server or database boundary should reject an unauthorized action.

Use sanitized test records only. Do not probe systems you do not own or have permission to test.

5. Repeat the same action

Retries happen because browsers refresh, networks time out, users double-click, and providers redeliver events.

Repeat the same safe request or test event and check whether it creates:

duplicate records;
duplicate entitlements;
contradictory status;
repeated notifications; or
an incorrect charge in a test-mode system.

Record the expected idempotent behavior before running the check.

6. Change the order

External events may arrive late or out of order. In a controlled test environment, check whether a delayed cancellation, completion, or retry can overwrite a newer valid state.

If order matters, the system should use durable state rules rather than assuming events always arrive in the demo sequence.

7. Stop halfway

Ask what remains if the flow fails after one write but before the next.

For example:

a payment record exists but no entitlement was granted;
a file was stored but its database record failed;
a user was invited but the authorization relationship was not created.

The desired outcome may be a transaction, a retryable state, cleanup, or a clearly visible recovery path. What matters is that the half-finished state is intentional and observable.

8. Test both denial and legitimate use

Every fix or rule needs two checks:

a negative test showing the unwanted action is rejected; and
a normal-path test showing the correct user can still finish the flow.

Without both, it is easy to “fix” unauthorized access by breaking access for everyone or to stop duplicates by blocking valid retries.

9. Write down evidence and limits

For each check, record:

starting role and state;
exact safe action;
expected result;
observed result;
relevant screenshot, log, or test output with secrets removed; and
anything blocked or still outside scope.

If the behavior cannot be reproduced, say so. If access or setup prevents the check, record the blocker. Uncertainty is a result; it should not be turned into a dramatic finding.

A small next step

If you want a second set of eyes without opening your codebase, the AI App One-Flow Preflight reviews one submitted 3–6-screen journey from a sanitized recording of up to 10 minutes or up to eight screenshots.

For USD 129, the fixed deliverable is a flow map, up to three evidence-linked hypotheses, ten manual tests with expected results, a Now / Next / Later action plan, and one asynchronous follow-up. Turnaround is within two business days after complete accepted materials.

Applying is free, and a misfit application stops before checkout:

https://payhip.com/b/1FgDW

This is an evidence-limited product review, not source-code review, production testing, a security audit, penetration test, certification, defect guarantee, or approval to launch.

Disclosure: This article was prepared with AI assistance and manually reviewed for scope, factual claims, and safety boundaries.