Changing a number on a pricing page is the easy part of a pricing experiment. The hard part is that the number is attached to live subscriptions, invoices that have already been issued, customers who signed under the old terms, and a finance close that has to reconcile both variants in the same month.
That's why most pricing tests either never ship or ship once and never get repeated. The system underneath makes each one expensive.
What does it take to run a pricing experiment safely?
You need four things: a hypothesis with one variable, a clean way to assign customers to a variant, prices stored as versioned data so the old and new rates can coexist on the same billing run, and enough time on the test to see retention rather than only conversion. Miss the third and every experiment becomes a migration. Miss the fourth and you ship a result that reverses in month two.
What you can actually test
Price level. The control group sees $59 a month, the variant sees $49, features and onboarding identical. The only variable is the number. This is the cleanest test to run and the easiest to read.
Packaging and bundling. Combine features into a bundle at a discount to buying separately, or let customers assemble their own. What you learn is which combinations people actually want, which usually differs from how the product is organized internally.
Tier structure. Three plans at different price points with different feature sets. The useful signal is what a customer needs before they move up, which matters more than which tier converts best.
Freemium boundaries. Where the free tier ends is itself a variable. Moving a limit or a feature across that line changes both conversion and the cost of serving free users.
Discounts and promotions. A time-bound offer against a control group tells you whether a discount expands volume or just lowers revenue per sale. Run this one carefully, because the group that got the discount now has an expectation.
Price presentation. How a price is displayed changes response independently of the price itself. It's a real variable, and it's worth isolating rather than mixing into a level test.
The metrics that decide the result
Conversion rate alone will mislead you. A lower price converts better almost every time, and that tells you nothing about whether it was a good idea.
Track the full set:
- Trial to paid conversion. The headline number, and the one that moves first.
- Monthly churn. Whether the customers a lower price attracted are the ones who stay.
- ARPU. Revenue per active customer, which moves against conversion.
- LTV. The number that actually settles a price-level question.
- Payback period. How long acquisition cost takes to recover under the new price.
- Cost to serve. Especially with usage pricing, since a cheaper price can attract heavier accounts and compress margin even as revenue grows.
Revenue per user minus cost to serve is the number that determines whether a variant wins. Everything else is an input to it.
The framework
1. Write the hypothesis and pick the target
Make it specific and measurable, with a deadline. "Increase trial to paid conversion by 12% over four weeks by lowering the entry price to $49" is testable. "See if cheaper works better" is not.
The target should also connect to the company's actual goal. Optimizing ARPU when the strategy is to widen the base produces a local win and a strategic loss.
2. Segment, then assign
Decide which customers are in scope. Usage maturity, company size, geography, and acquisition channel all produce genuinely different price sensitivity, and mixing them adds noise that a small sample cannot absorb.
Assignment should be random within the segment and stable per customer. A customer who sees one price at signup and a different price on their second visit is a bug, not a variant.
Keep segmentation to business characteristics. Usage level, plan, and region are fine. Protected characteristics are not.
3. Design the test
One variable. If price and feature access both move, the result is uninterpretable and you'll run it again.
Give it enough time. Two to four weeks is a practical floor for conversion signal, and longer if you need churn, because a monthly plan produces its first churn data point after a full cycle. Low-volume products need longer still, since the sample accumulates slowly.
Decide up front what you'll do with each outcome. Pre-committing to the decision rule is what stops a marginal result from becoming a post-hoc argument.
4. Run it without forking your billing
This is where the engineering work sits. Two prices for the same product have to coexist, be attributed correctly on every invoice, and be reversible.
If prices are constants in application code, this means a branch in the billing path and a cleanup project afterwards. If prices are versioned data attached to a plan, the variant is a second price record with its own effective dates, and the invoice run resolves whichever applies to that subscription.
The practical requirements:
- Versioned prices with history. You need to reproduce any past invoice at the rate that applied then.
- Per-customer overrides. So a variant or a negotiated rate doesn't require cloning the whole plan.
- Proration on switches. A customer who moves mid-cycle should be billed correctly for both segments without manual adjustment.
Flexprice handles the first two as price overrides on a subscription and plan cloning when you want a full variant, and documents the mid-cycle behaviour in Understanding proration.
5. Monitor for side effects
Watch conversion, usage growth, churn, revenue, ARPU, and cost to serve together. The failure mode is a variant that wins on the metric you were watching and loses on the one you weren't.
Tell your own teams. Support will field questions about why two customers see different prices, and sales needs to know which rate is real before a call.
6. Analyze, then decide grandfathering
Compare against control on every metric, not just the target. Segment the comparison, because a variant can win overall while losing badly in the segment that produces most of your revenue.
Then handle existing customers deliberately. Grandfathering on the old rate indefinitely is the least disruptive option and the one that accumulates pricing debt. A migration with notice is cleaner and costs goodwill. Either is defensible. Doing it quietly is not, because customers compare notes.
7. Feed the result into the next test
One experiment should generate the next hypothesis. If high-usage customers turn out to be sensitive to per-unit increases, the next test is packaging or overage thresholds rather than another level change.
The constraint on iteration speed is almost never analysis. It's how long a pricing change takes to ship.
Common ways experiments fail
Too many variables. Price, bundle, and billing frequency all moving at once produces a number you can't attribute.
Too short a window. Early lifts revert. A test that ends before the first renewal cycle has measured acquisition, not economics.
Ignoring churn. Higher conversion with higher churn is usually a worse business.
Ignoring cost to serve. A cheap price that attracts heavy usage can grow revenue and shrink profit at the same time.
Confusing customers. Different prices are fine. Different prices discovered accidentally are a trust problem.
Teams out of sync. Pricing touches product, sales, support, and finance. One team learning about a change from a customer is how a good experiment becomes an incident.
Products worth studying
A few developer-facing products publish enough of their pricing to learn from. These are five different answers to the same problem, checked in September 2026.
- Netlify runs a plan fee plus a monthly credit allowance: 300 credits on Free, 1,000 on Personal at $9, 3,000 on Pro at $20, unlimited on Enterprise. Consumption is priced in credits, at 15 per production deploy, 10 per GB-hour of compute, 20 per GB of bandwidth, and 2 per 10k web requests, with auto-recharge packs when the balance runs down.
- Vercel keeps a flat plan fee, $0 on Hobby and $20 on Pro, then meters per resource past a published allowance. Pro includes 10M edge requests a month and charges from $2 per 1M after that, and includes 1TB of fast data transfer then charges from $0.15 per GB.
- Supabase splits the bill in two: an organization plan starting at $25 a month on Pro, plus per-project compute billed separately, with $10 of compute credits included. Overage rates are published per resource, such as $0.09 per GB of egress and $0.00325 per monthly active user past 100,000.
- RunPod is pure pay as you go on GPU compute, billed per hour or per second with no plan fee. It's the most legible model on this list and the one that puts the most tracking burden on the customer.
- ElevenLabs sells subscription tiers that each carry a monthly credit allowance, from 10k credits on Free up to 6M on Business at $990 a month, with Enterprise quoted.
Two things are worth taking from that spread. Credits show up repeatedly as the unit customers see, because they decouple the price on the page from the resource actually being metered. And in every case the allowance and the overage rate are published together, which is what lets a customer estimate the bill before committing. That predictability is what makes usage pricing survivable, and it's the thing an experiment most often breaks.
What makes experiments cheap to repeat
Flexprice is enterprise-grade, open source usage based billing infrastructure for AI and SaaS companies. It can be deployed in your own VPC, on-prem, or on Flexprice's managed cloud.
For pricing experiments specifically, plans and prices are configuration rather than code, so a variant is a record instead of a release. Usage is metered in real time across both arms, invoices generate from the same engine under either price, and the history is retained so a past invoice can be reproduced at the rate that applied then.
There is also a simulation layer worth knowing about before you touch a live price. Flexprice scenarios let you copy your current pricing into a hypothesis, change the rates or the packaging, and duplicate it to compare variants side by side, with no end-customer consequences. How to evolve pricing covers that workflow. Modelling a variant against your existing price set is the cheapest way to kill a bad hypothesis before it reaches a customer.
Top comments (0)