I'm a software engineer, and data analysis is part of my work. Cleaning data, agreeing on metric definitions, and checking a report against its source numbers are familiar parts of the job.
I built OWL Compose to help with this workflow. This post walks through the checks behind a user conversion report I made with it. The dataset is simulated and describes a fictional team collaboration product; it contains no customer performance results.
Define the population and observation window
The example covers January–June 2026 signup cohorts, observed through August 1. Each user is assigned to their first acquisition channel. The onboarding funnel follows signup, completed import, and first useful output, in that order, within seven days of signup.
Week 4 retention counts users active on days 22–28 after signup, divided by the original signup cohort. All six cohorts have completed that window. This makes the comparison meaningful across signup months.
Calculate the total from counts
The six months contain 29,080 signups and 7,486 week 4 active users. The combined retention rate is:
7,486 / 29,080 × 100 = 25.7% (rounded)
This calculation weights each cohort by its size. An unweighted average of monthly percentages would give each month equal influence even though the signup counts differ.
For each month and channel, the report uses the same definitions:
activation = first useful output within 7 days / signups
week 4 retention = active users in days 22–28 / signups
spend per week 4 active user = direct acquisition spend / week 4 active users
The spend measure is in CNY. It includes direct acquisition expenses and excludes staffing, product delivery, and service costs. It cannot establish profitability without revenue and the missing costs.
Check the aggregate against its segments
January has 3,000 signups and 781 week 4 active users: 26.0% retention. June has 7,180 signups and 1,833 week 4 active users: 25.5%.
Within each channel, June retention is higher than January retention. Paid ads, the lowest-retention channel in this example, account for a larger share of June signups. Both the channel rates and their weights contribute to the aggregate.
This is a useful report review: a change in the total can reflect a change in the population mix. The channel breakdown helps explain the arithmetic. It does not establish why users behaved differently.
Keep the presentation tied to the calculations
In the report, the metric cards, charts, and appendix draw from the same simulated aggregate data. Derived values are calculated from those inputs. The written analysis names the cohort, denominator, and limits of each comparison.
OWL Compose helps me put those pieces into a readable report with AI. Input quality and metric definitions still need review. A polished chart does not validate either one.
Explore the complete simulated report. The public example includes the funnel, channel comparison, retention matrix, and data appendix.
If you review analytics reports, which consistency check do you run first?
Top comments (0)