Every channel autopsy I have watched ended the same way: someone opens the dashboard, sees a bad week, and argues for killing the channel. Someone else opens the same dashboard and argues for one more month. Both are reading the same numbers and both are guessing, because no threshold was written down before the data arrived.
The rule: pre-register the decision
Before a test starts, write one sentence: "If fewer than N results after X units, the channel is out." Then the result cannot be re-interpreted later. Three floors I use:
- 50-100 sends per variant before judging a channel. A 3% vs 6% reply rate at 20 sends is a coin flip, not a difference.
- 300+ sends before judging a message. Copy differences are small effects, and small effects need volume.
- Three weeks before judging a cadence. Day 1 vs day 5 reply rates move the most, and one week is too short to see it.
Why dashboards make this worse
A dashboard optimises for looking at, not deciding. It shows you the last 7 days whether or not 7 days is a meaningful window. The pre-registered threshold is the only thing that makes a small sample say anything at all - it converts noise into a decision you already agreed to.
The two failure modes it prevents
- Killing a working channel because week one was small and the timing was bad.
- Keeping a dead channel alive for months because "the numbers are small, let's wait".
Both are the same error: deciding after seeing the data instead of before.
What to do on Monday
Pick the channel you are unsure about. Write the kill rule and the keep rule, with numbers, in a file. Set the date you will read the result. Then do not touch it until that date.
I packaged these thresholds, with the reasoning behind each, as an open-source skill for AI agents - decision rules instead of templates:
https://axelfreeman.github.io/marketing-mindset/
If your agent proposes marketing work, it should carry the same floors. Free, MIT licensed.
Top comments (0)