When conversion rates rise overall but fall in every segment

Both acquisition groups convert less often. The overall conversion rate more than triples. There is no missing event and no arithmetic error.

Here is a hypothetical funnel with two mutually exclusive groups, Email and Search. Use the same rule to assign each recorded entry to one group in both periods. These are illustrative counts, not Zenovay customer results or channel benchmarks.

Period	Group	Recorded entries	Completions	Rate
A	Email	100	20	20%
A	Search	1,900	38	2%
A	Total	2,000	58	2.9%
B	Email	1,000	180	18%
B	Search	1,000	15	1.5%
B	Total	2,000	195	9.75%

Email falls from 20 to 18 percent. Search falls from 2 to 1.5 percent. Yet the overall rate rises from 2.9 to 9.75 percent, with the same total number of entries.

The difference is the audience mix: Email goes from 5 percent of entries to 50 percent. Its rate is lower than before, but it still converts more often than Search. Moving more of the traffic into that group changes the aggregate.

The overall rate is answering a different question

This is an example of Simpson’s paradox: the direction of an aggregate comparison can reverse within every subgroup. It is a known statistical effect, not a new discovery or a defect in the calculation.

The overall rate asks what share of this period’s recorded entries completed the funnel. The group rates ask what happened within Email and within Search. Those populations changed their relative size, so the aggregate is not holding the audience mix constant.

The second period does produce more completions: 195 instead of 58. That may be a valuable business result. But it does not show that the flow became easier for either group. The observed rate fell in both.

Before crediting a redesign, ask which result the claim is about: more total outcomes, a different audience, or better conversion within a comparable audience.

Hold the audience mix fixed

One useful descriptive check is to choose a reference mix and apply it to both periods’ group rates. Here, use period A’s mix: 5 percent Email and 95 percent Search.

For period B, the calculation is 0.05 × 18% + 0.95 × 1.5% = 2.325%.

Period	Observed overall rate	Rate with period A’s mix
A	2.9%	2.9%
B	9.75%	2.325%

With the mix held fixed, the rate falls. This separates the changing group weights from the changing rates within those groups.

Choose and document the reference population before judging the result. Use the same mutually exclusive groups in both periods. If a group has no observed entries in one period, its rate is undefined; do not silently turn that missing rate into zero.

Here is the calculation as a self-contained PostgreSQL query. It uses only the four hypothetical rows above:

WITH sample(period, segment, entries, completions) AS (
  VALUES ('A', 'Email', 100, 20),
         ('A', 'Search', 1900, 38),
         ('B', 'Email', 1000, 180),
         ('B', 'Search', 1000, 15)
), rates AS (
  SELECT *, completions::numeric / entries AS rate
  FROM sample
), baseline AS (
  SELECT segment,
         entries::numeric / SUM(entries) OVER () AS weight
  FROM sample WHERE period = 'A'
)
SELECT r.period,
       ROUND(100.0 * SUM(r.completions) / SUM(r.entries), 3)
         AS observed_pct,
       ROUND(100.0 * SUM(r.rate * b.weight), 3)
         AS fixed_mix_pct
FROM rates r JOIN baseline b USING (segment)
GROUP BY r.period ORDER BY r.period;

Reweighting answers a comparison question. It does not establish what a redesign caused, or predict exactly how a future audience will behave.

Step rates have their own denominator

The same care applies within a funnel. Suppose 1,000 recorded entries produce 400 signups and 100 final completions. Overall completion is 10 percent. Signup to completion is 25 percent. Both calculations are correct because their denominators differ.

In Zenovay, the overall funnel rate compares recorded completions with recorded entries. For transitions after the first step, the step breakdown uses the preceding step for its conversion calculation. Read each rate with its counts and its scope.

People, sessions, funnel entries, and event occurrences are different counting units. Someone can return several times or repeat an action. A recorded entry is not a verified individual. Keep the unit explicit when comparing reports.

Keep the definitions and observation window fixed

A smaller denominator can also raise a rate without adding a single completion: 100 out of 1,000 is 10 percent; 100 out of 500 is 20 percent. Moving the entry point closer to checkout changes the measurement, even if the report keeps the same name.

Use the same entry rule, final event, counting unit, and group definitions. Record changes to URL rules or funnel steps and establish a new baseline when the meaning of the measure changes.

Time matters too. Yesterday’s entrants have had less opportunity to finish a long journey than entrants from two weeks ago. For a cohort comparison, choose an entry period and a fixed follow-up window, then let both groups reach it. A calendar report of events and a cohort report following entrants are different views.

That is a method for comparing cohorts, not a claim that every funnel report provides that control.

What this calculation can and cannot tell you

The table demonstrates an aggregation reversal. It does not demonstrate why either group’s rate changed. Observed rates can also move through sampling variation, especially when the counts are small.

Predefine the segments that matter to the decision. Show entries and completions alongside percentages. For a claim that a product change caused improvement, use a suitable experiment or another defensible causal design; a segmented before-and-after report alone does not supply that evidence.

A conclusion someone else can check would be: “Completions rose from 58 to 195 across 2,000 recorded entries in each period. Both group rates fell, while Email’s share of entries rose from 5 to 50 percent. Holding period A’s mix fixed gives 2.325 percent for period B, compared with 2.9 percent before.”

That is more useful than “conversion improved.” It tells the team what changed, which comparison improved, and which question still needs an answer.