Most teams declare a winner too early. They send 100 cold emails, see 3 replies on variant A and 1 on variant B, and ship A. That is not a test. That is a coin landing on its side.
Below a certain volume you are not measuring the market — you are measuring randomness. I have watched this eat months of budget across B2B campaigns, so here is the version I wish someone had handed me: a per-channel minimum, and one rule that follows from it.
The numbers, per channel
| Channel | Minimum volume before a verdict | What it actually tells you |
|---|---|---|
| Cold email — deliverability / wording check | 50–100 sends | That the email lands and reads plausibly. Not that the offer works. |
| Cold email — reply-rate test | ~1,500–2,000 sends per variant | Whether one variant genuinely beats another, not just a quiet week. |
| Cold email — subject line / open-rate | 100–500 sends per version | How the subject performs — opens are frequent events, so small samples still say something. |
| Landing page smoke test | 100–200 targeted visitors | Whether the promise produces interest at all. At a 15% capture rate, 200 cold visitors ≈ 30 leads. |
| Strict A/B test | ~10,000 visitors per variation, ≥300 conversions | Statistical significance — usually out of reach for a first test. |
| Paid ad | Spend gate of 1–3× target CPA, 48–72 hours | Keep / re-hook / kill. Never judge inside the learning phase. |
Sources for those ranges: Getlead, Growtoro, Woodpecker, GLIDR, AB Tasty, Clikim.
Why the popular number is off by an order of magnitude
Reply rates in cold B2B email sit around 3–5%. Detecting a lift from 4% to 6% — a 50% relative improvement, a big win in practice — needs roughly 1,500–2,000 sends per variant before the difference stops being explainable by chance. At 75 sends per variant you can "prove" almost anything: one extra reply moves the number by more than the effect you are trying to detect.
The same logic applies everywhere:
- Rare event (replies, purchases, demos booked) → the sample has to be large.
- Frequent event (opens, clicks) → a small sample can be enough.
- Money spent on ads → the platform is still learning early on; your "signal" is the algorithm settling, not the creative.
The 50–100 number is useful, but only for one job: checking that the email arrives, renders, and does not sound insane. It is a smoke test, not a verdict.
What to do when the market is too small
Narrow B2B niche, 400 perfect-fit companies in the world — you cannot send 2,000 per variant. Do not silently lower the bar and pretend. Change the unit of the decision:
- Longer window — decide over a quarter instead of a week.
- Different observable — replies per twenty hand-written conversations instead of a campaign-wide rate.
- Qualitative signal — patterns from five real buyers, marked as such.
- Say it out loud — "this sample is small, the decision is directional" is an honest sentence. Quietly declaring victory is not.
The rule that follows: what comes easy, scale it
Here is the part most people miss. Once you are above the limit, the test does more than say yes or no — it shows you which thing was easier than the rest. Easier than usual is not luck. It is a signal, and it gets the next unit of effort first: your time, then your money.
Below the limit that gap is invisible: "easy" and "ordinary" look identical when the sample is noise, so people either chase randomness or ignore the one channel that was actually working.
Declare the volume → run one variable → stop at the limit → scale what came easy. Anything else is interpretation.
A short checklist
- Write down the yes/no decision the test must produce.
- Pick the limit for the channel before you start.
- Change one variable. Two changes and the result has two explanations.
- Stop at the limit. Do not extend the test because the first 20 results looked good.
- Whatever cleared the limit more easily than the alternatives goes first.
Where this comes from
I packaged this way of deciding — the limits, the three-month horizon, competitors as the first source of truth, and the scale-what-comes-easy rule — as a free MIT-licensed skill that AI agents can load and apply while working on marketing and client acquisition:
Marketing Mindset → the full skill and the limits table · source on GitHub
Install it and your agent stops producing templates and starts giving you verdicts:
npx skills add axelfreeman/marketing-mindset
If you disagree with the numbers, I would rather hear the disagreement than the praise — that is how the next version gets sharper.
Top comments (0)