DEV Community: Matthew

Your CI is green. Your billing logic is broken.

Matthew — Thu, 25 Jun 2026 01:04:42 +0000

User paid. Stripe looked fine. The app still gated them.

They had an active subscription. The checkout page worked. Tests passed. But has_paid_access in Postgres stayed false, so the user hit the paywall anyway.

This is not a rare edge case. It is the default failure mode when you ship billing logic with AI tools and mock Stripe in tests.

How the bug actually happens

The sequence is usually boring:

Agent writes checkout and a webhook handler for customer.subscription.created.
Tests mock Stripe. CI goes green.
invoice.payment_failed or customer.subscription.deleted never updates the DB flag.
Stripe says active. Your app says free.

Nothing in a unit test suite checks both systems at once. You are testing each side in isolation. The bug lives in the gap between them.

Stack Overflow's 2025 survey put AI coding tool adoption around 84%. Veracode found roughly 45% of AI-generated code samples contained OWASP Top 10 issues. Only about 33% of developers say they trust AI accuracy. Speed went up. The intersection of billing and database state did not get safer.

What v0.10.0 changed

Before v0.10, showing someone this bug took four steps: clone the SDK, cd into it, pass config flags, point at a fixtures directory. Fine for docs. Too much friction for a Reddit comment or a tweet.

v0.10.0 bundles the revenue-leak fixture inside the CLI:

npx prodverdict@0.10.0 demo

One command. No git clone. No Stripe keys. You get a FAIL on purpose: active subscription, has_paid_access false.

Two other commands round out discovery:

npx prodverdict scan    # static repo analysis, no credentials
npx prodverdict init --mcp --cursor-rule   # auto-detects stack from package.json

Architecturally, v0.10 adds a discovery layer on top of the existing engine. runContracts() still powers everything. The new commands just remove the setup tax before someone sees value.

The fix: check the intersection in CI

ProdVerdict compares billing state (Stripe or Paddle) against who your database thinks has paid access. Rules only. No LLM in the evaluation path. Missing credentials = fail, not a silent pass.

After the demo, wire it into your repo:

npx prodverdict init --mcp --cursor-rule
export STRIPE_SECRET_KEY=rk_test_...
export DATABASE_URL=postgresql://readonly:...@host/db
npx prodverdict check access

Block merges in GitHub Actions:

- uses: prodv-dev/prodverdict-action@v0.10.0
  with:
    config: ./prodverdict.yml
    contract: access
  env:
    STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }}
    DATABASE_URL: ${{ secrets.DATABASE_URL }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Secrets stay on your runner. ProdVerdict cloud never sees subscription rows.

Other contracts (after access is wired)

Once the billing check is trusted, add more: config (env drift), migration (unsafe Postgres DDL), boundary (mass-assignment), webhook (signature + idempotency lint), restore (backup smoke tests). Run all six with npx prodverdict check all.

Start with access. It ties directly to revenue. The rest is expansion.

Try it

npx prodverdict@0.10.0 demo

Docs: https://prodverdict.com/docs/quickstart
Agents + MCP: https://prodverdict.com/agents
SDK: https://github.com/prodv-dev/prodverdict-sdk
GitHub Action: https://github.com/marketplace/actions/prodverdict

Free CLI, Action, and local MCP on private repos. I built this after catching the same bug one too many times. Curious if you have shipped this class of mismatch before.

Stripe says active. Postgres says false. Nobody alerts you.

Matthew — Thu, 18 Jun 2026 07:05:02 +0000

Your Stripe dashboard looks fine. MRR is up. Tests pass. CI is green.

Then a customer emails you: "I paid, but I'm still on the free plan."

That is usually not a Stripe bug. Stripe charged them. Your app never updated the row that gates access.

Two systems, one gap

Most subscription SaaS runs on two truths:

Stripe — who paid, which plan, subscription status
Your database — has_paid_access, plan, stripe_customer_id

The webhook handler is supposed to keep them aligned. When it drifts, neither side raises an alarm. Stripe thinks everything delivered. Your app serves the wrong access level. You only find out when someone complains.

Stripe's docs say this plainly:

Events are not guaranteed to arrive in order (docs.stripe.com/webhooks)
Failed deliveries retry for up to three days in live mode
Your endpoint must return 2xx before slow work, or you hit timeouts and retries

This shows up in production, not just in tutorials. On the Ghost forum, a founder paid through Stripe successfully, then landed back on the site as a free user. Ghost staff explained: if the webhook never lands, Ghost never learns about the subscription. Another user confirmed multiple paid subscribers stuck as free until they watched the webhook log constantly.

AI makes the happy path faster and the failure path easier to skip

Developers are using AI tools everywhere:

84% use or plan to use AI tools (Stack Overflow 2025 survey, 49k+ responses)
90% of software professionals use AI daily (Google DORA 2025)

Trust did not keep up:

29% trust AI accuracy, down 11 points from 2024 (Stack Overflow blog, Feb 2026)
46% distrust AI output accuracy (SO 2025 press release)
66% say their top frustration is output that is "almost right, but not quite" (InfoWorld on the survey)

AI is good at checkout → stub webhook → set has_paid_access = true. It is weaker on the paths that actually cause drift:

invoice.payment_failed without revoking access
Out-of-order customer.subscription.updated events
Duplicate evt_... IDs without idempotency
Writing to the wrong user row for a stripe_customer_id

An Indie Hackers thread from April 2026 describes three Cursor-built SaaS products with the same handler: return 200, log the event, never touch the database. Cancelled users keep Pro. MRR looks fine. You only see the leak when you manually compare Stripe to Postgres.

This dev.to post from sravan27 lists the same bugs in the wild: trusting the success redirect, skipping idempotency, letting Pro access drift from the payment source of truth.

Reproduce a revenue leak in 60 seconds (no API keys)

git clone --depth=1 https://github.com/prodv-dev/prodverdict-sdk.git
cd prodverdict-sdk

npx prodverdict check access \
  --config examples/nextjs-stripe/prodverdict.yml \
  --fixtures \
  --fixtures-dir examples/nextjs-stripe/scenarios/fail-revenue-leak

You should see something like:

[HIGH] user:usr_alice — Active subscription but has_paid_access is false
fix: Set has_paid_access=true in your webhook handler
VERDICT: FAIL

That is a production contract: compare live billing state to live database state. Not a lint rule. Not an LLM guess. PASS / WARN / FAIL.

What fixes this (with or without a tool)

Stripe owns subscription status. Your DB is a cache.
Store processed evt_... IDs with a unique constraint.
Refetch the subscription from Stripe on critical events. Do not assume event order.
Run a nightly reconciliation: active Stripe subs vs DB rows, alert on mismatch.
Block merges when drift exists, not just when mocked unit tests pass.

jest.mock('stripe') will never catch "Stripe says active, Postgres says false."

What I'm building

I run ProdVerdict, a deterministic contract checker for AI-assisted SaaS. The access contract compares Stripe or Paddle to your Postgres users table. Config, migration, boundary, webhook, and restore contracts ship in v0.9.

CLI + GitHub Action
MCP for Cursor agents (billing secrets stay local)
Free for public repos; fixtures above work without credentials

npx prodverdict init --stack nextjs-stripe

I'm looking for three design partners on real Stripe + Postgres repos to run nightly access checks and publish anonymized drift findings. Comment if you want a free audit with a read-only Stripe key.

Have you ever found someone paying in Stripe but locked out in your app, or cancelled but still on Pro? How did you catch it?

prodverdict.com · GitHub · Discord

ProdVerdict v0.6: verify billing before your Cursor agent opens a PR

Matthew — Thu, 11 Jun 2026 07:10:21 +0000

AI agents ship fast. Tests pass. The PR looks fine. Then Stripe says active and Postgres says has_paid_access = false.

ProdVerdict v0.6 adds an agent-first workflow: diagnose credentials, run all contracts, get stable JSON back — for Cursor MCP or CLI.

Agent loop (new in v0.6)

npx prodverdict init --stack nextjs-stripe --mcp --cursor-rule
npx prodverdict doctor --format agent
npx prodverdict check all --format agent

--format agent returns schemaVersion: "1" JSON with summary and nextSteps — no LLM in evaluation.

Cursor MCP

Add @prodverdict/mcp to .cursor/mcp.json. Tools: doctor, check_all_contracts, check_access_contract, check_config_contract, check_migration_contract, validate_config, suggest_fix.

Prompts: setup_prodverdict, verify_before_pr.

Full guide: prodverdict.com/agents

Three contracts

Access — Stripe/Paddle vs Postgres billing drift
Config — env var references vs .env.example
Migration — unsafe Postgres DDL in agent-generated SQL

Try without API keys

git clone --depth=1 https://github.com/prodv-dev/prodverdict-sdk.git
cd prodverdict-sdk
npx prodverdict check access \
  --config examples/nextjs-stripe/prodverdict.yml \
  --fixtures \
  --fixtures-dir examples/nextjs-stripe/scenarios/fail-revenue-leak

CI enforcement

- uses: prodv-dev/prodverdict-action@v0.6.0
  with:
    config: ./prodverdict.yml
    contract: access
  env:
    STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }}
    DATABASE_URL: ${{ secrets.DATABASE_URL }}

Billing secrets stay on your runner. The dashboard stores verdict metadata only.

GitHub Marketplace · npm · Agents

Your AI agent shipped a billing bug. ProdVerdict blocks it in CI.

Matthew — Tue, 09 Jun 2026 12:37:53 +0000

AI coding tools are fast. Tests pass. The PR looks fine. Then production quietly leaks revenue.

The pattern I kept seeing in vibecoded SaaS:

Billing logic lands in a PR
Webhooks get skipped or half-wired
Stripe says active while Postgres says has_paid_access = false

That's not a lint issue. That's money walking out the door.

ProdVerdict is deterministic CI for production contracts — not another AI code reviewer. Zero LLM in the evaluation path. Read-only observations (Stripe/Paddle + your DB) and fixed rules. Same input, same output every time.

Three contracts in v0.5

1. Access — billing vs database

Compares live subscription state (Stripe or Paddle) against your Postgres users table. Catches:

Revenue leak — active subscription, has_paid_access = false
Wrongful access — cancelled user still has paid features
Plan drift — price ID maps to pro, row says starter
Duplicate customer — same billing ID on multiple app users

2. Config — env var drift

Scans process.env references in your code vs .env.example and CI secrets. The classic "it worked locally" killer.

3. Migration — unsafe Postgres DDL

Static SQL rules for agent-generated migrations: blocking CREATE INDEX without CONCURRENTLY, dangerous NOT NULL on hot tables, and more.

Try it in 10 seconds — no API keys

Clone the SDK once for fixture paths, then run:

git clone --depth=1 https://github.com/prodv-dev/prodverdict-sdk.git
cd prodverdict-sdk

npx prodverdict check access \
  --config examples/paddle-stripe/prodverdict.yml \
  --fixtures \
  --fixtures-dir examples/paddle-stripe/scenarios/fail-revenue-leak

You should see a FAIL with a concrete fix hint:

[HIGH] user:usr_alice — Active subscription but has_paid_access is false
fix: Set has_paid_access=true in your webhook handler
VERDICT: FAIL

Passing scenario:

npx prodverdict check access \
  --config examples/nextjs-stripe/prodverdict.yml \
  --fixtures \
  --fixtures-dir examples/nextjs-stripe/scenarios/pass

Add to your repo

npx prodverdict init --stack nextjs-stripe
# or: npx prodverdict init --stack paddle-stripe

Commit prodverdict.yml, add secrets, and wire the GitHub Action:

- uses: prodv-dev/prodverdict-action@v0.5.0
  with:
    config: ./prodverdict.yml
    contract: access
  env:
    STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }}
    DATABASE_URL: ${{ secrets.DATABASE_URL }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Runs on every PR. Secrets never leave your runner. Optional dashboard stores verdict metadata only — not subscription rows or PII.

Install from GitHub Marketplace · npm · MCP for Cursor/Claude

How this differs from AI code review

	AI code review	ProdVerdict
Evaluation	LLM suggestions	Deterministic rules
Input	Source code diff	Live billing + DB state (or fixtures)
Output	"Consider checking…"	PASS / WARN / FAIL with fix hints
CI behavior	Optional, noisy	Merge-blocking on HIGH

Agents can call ProdVerdict via MCP before opening a PR. Enforcement still happens in CI.

Pricing

Free — public repos, CLI, GitHub Action, MCP, Access + Config contracts
Pro ($39/project/mo) — private repos, scheduled checks, dashboard history

We're on Product Hunt — use code PRODUCTHUNT at checkout for 3 months free Pro.

What's next?

Webhook contract, API boundary checks (mass-assignment / forbidden fields), and more stack templates. Open an issue on prodverdict-sdk if you want a rule prioritized.

What's the scariest prod bug an agent ever shipped for you? Drop it in the comments — I'm collecting patterns for the next contracts.

ProdVerdict is MIT-licensed (SDK + Action). prodverdict.com · Discord

Building Phonton: a local-first AI coding CLI that verifies diffs before review

Matthew — Sun, 03 May 2026 04:33:25 +0000

I have been building Phonton CLI, an open-source local-first AI coding agent.

The idea is simple: instead of treating an AI coding tool like a chat box, Phonton treats it like an engineering workflow.

It tries to follow this loop:

You give it a goal.
It creates a plan.
It works against your local repo context.
It verifies the generated diff.
You review the result before accepting it.

Most AI coding tools are optimized for speed and convenience. Phonton is more focused on making AI-generated changes easier to trust.

Why I built it

I wanted a CLI agent that was:

local-first
BYOK, so you use your own provider keys
review-oriented
built around verification
able to remember repo decisions locally
useful from the terminal

The goal is not to replace every tool like Claude Code, Codex, Cursor, or OpenCode. The goal is to explore a stricter workflow for AI-assisted coding: plan, verify, review, then ship.

Current status

Phonton is still early alpha.

It currently has:

a Rust CLI/TUI
provider adapters
planning
local memory
repo indexing
verification flow
review commands
checkpoint/rollback work

There are still rough edges, and I am not claiming it is production-ready yet.

Try it

npm install -g phonton-cli
phonton doctor
phonton plan "add input validation to config loading"

GitHub:

https://github.com/phonton-dev/phonton-cli

I would appreciate feedback from developers who already use AI coding agents and care about local-first workflows, reviewable diffs, and verification.

SIGNAL; shrinking agent skills so context stays for code

Matthew — Sat, 18 Apr 2026 16:49:12 +0000

SIGNAL is a small open-source agent skill bundle for people who hit context limits: tiered modes, symbolic shorthand, checkpoints, a disk-backed .signal_state.md, and separate skills for diff/search summaries and git (commit / push / PR) plus structured review. Each skill ships as readable *.md and a tight *.min.md so you can load less instruction surface when you want.

Install

npx skills add mattbaconz/signal

Repo: github.com/mattbaconz/signal

Benchmarks (honest caveat)
The repo includes scripted fixtures and heuristic token estimates (ceil(chars/4)). That’s useful for comparing shapes, not for quoting exact billed tokens from a provider.

What I’d like from readers
If you’ve tried skills packs or fat AGENTS.md files: does the canonical + minified split match how you’d maintain this, or is it too much friction? I’m open to blunt feedback.