DEV Community

Baneet Singh
Baneet Singh

Posted on

Your Shopify price test is picking the wrong winner

#analytics #startup

Most price tests on Shopify pick the wrong winner, and the reason is the metric, not the tooling.

A lower price almost always converts better. If you judge a price test on conversion rate, the cheapest price wins every time, even when it makes you less money.

Judge price tests on profit per visitor 

profit per visitor = conversion rate × (price − COGS − shipping − fees)
Enter fullscreen mode Exit fullscreen mode

A worked example with $15 of cost per order:

Price Conversion rate Profit per order Profit per visitor
$40 3.0% $25 $0.75
$48 2.7% $33 $0.89

The $48 price converts 10% worse and earns 19% more per visitor. A conversion-rate dashboard would have told you to keep $40.

Here is the same math as a tiny function you can drop into a notebook:

const profitPerVisitor = (cr, price, cost) => cr * (price - cost);
profitPerVisitor(0.03, 40, 15);  // 0.75
profitPerVisitor(0.027, 48, 15); // 0.891
Enter fullscreen mode Exit fullscreen mode

Three ways to run the test on Shopify

  1. Duplicate variants. Each price is a hidden copy of the variant and an app shows one copy per visitor. Easy, but inventory and reports split across the copies.
  2. Shopify Functions (Plus only). A cart transform sets the price at checkout. Clean data, needs Plus and a developer or an app.
  3. Separate offer pages. Two landing pages with different prices or bundles, traffic split by link. Only works for paid traffic you can route.

Rules that keep it honest

  • One visitor sees one price for the whole test, on every page and at checkout.
  • Charge the price that was shown. Never raise it at checkout.
  • Run whole weeks. Price tests need more traffic than page tests because the profit gap is small.
  • Check local consumer law before showing different prices to different people.

Sample size

Profit differences are small, so you need more orders than you think. At a 2% conversion rate, detecting a 20% difference takes roughly 21,000 visitors per version at 95% confidence and 80% power. There is a free calculator and table here.

I build Luria, which runs split tests on the parts of a page around the price (photos, offer wording, buy button, reviews placement), judged on real orders. For changing the price itself, a dedicated price testing app is the right tool. Full guide: How to A/B test prices on Shopify.

Top comments (0)