A new account is an encouraging notification. It is also a very small piece of evidence.
For an analytics product, someone can create an account, add a website, copy a script, and still never see a visit from their own site. Counting the first step as success hides all the work between signing up and getting something useful.
Our July activation audit started with that gap. The working notes suggested that roughly half of the nominal free accounts had not created a website. But those notes also marked parts of the baseline as inferred and mixed internal, test, and external accounts. That is enough to start investigating. It is not enough to publish a precise conversion rate.
This teardown is about what we can establish: the stages we need to separate, the installation checks the current code performs, and the evidence needed before calling an onboarding change an improvement.
Three steps that should never share one label
Account creation tells us that someone completed registration. Website creation tells us that they configured an object in the product. A first recorded event tells us that data reached that website’s analytics pipeline.
Those are different milestones. None, on its own, proves that the person understood the report or will return next week.
For this audit, the first useful technical checkpoint is a real event recorded for the intended website. That is narrower than product activation, but much easier to verify than a vague label such as “setup complete.”
Someone with no website needs help with the setup decision. Someone with a website and no events needs an installation diagnosis. Someone with events who never returns may have a different problem: the report did not answer a question worth coming back for.
Sending all three people the same reminder assumes the diagnosis before making it.
Read the implementation, not just the score
Zenovay’s current activation scoring function evaluates four milestones: a script installation flag, a first event within 24 hours of website creation, a Stripe connection within seven days of team creation, and a return login after the first day and within fourteen days of team creation. Each achieved milestone contributes 25 points.
The function counts achieved milestones independently. It does not require someone to complete a strict sequence before later milestones can count. That detail matters when interpreting a score: two accounts can reach the same total through different behavior.
A first event arriving after 24 hours also fails the time-bounded milestone even though tracking now works. “Missed the activation window” and “installation is broken” need different responses.
We previously wrote about an installation flag that did not reflect the underlying activity. The broader lesson still applies: a score summarizes its inputs. It cannot tell you whether those inputs are trustworthy.
An empty chart has several possible causes
The current tracking route checks the request origin against the website’s allowed domains. If no explicit list exists, it falls back to the registered domain. A mismatched origin can be rejected in production when domain verification is enabled.
There is also an explicit localhost exception controlled by the website’s allow_localhost setting. That exception belongs to the domain check. It is not a promise that every local test will produce a stored event: other checks still apply.
This is why “I pasted the script” is the beginning of verification, not the end. Check the actual page hostname, the configured website, the browser’s network response, and whether an event was recorded. Use a real browser visit for the user journey you are testing.
Do not weaken a security check simply to make a setup indicator turn green. First find the rejected request and understand why it was rejected. The useful repair may be the configuration, the installation instructions, or the error message.
Count people and websites separately
Subtracting the number of websites from the number of signups does not give the number of people who abandoned onboarding.
One person can create several websites. A colleague can join an existing team without creating any. Test accounts can distort a small sample.
A useful cohort report starts with a defined set of eligible accounts or teams and follows that same set through the stages. Choose the unit, exclusions, and observation window before computing the percentage.
For example, a report of new team owners could ask whether each owner created at least one website and whether any of those websites received a qualifying first event within a fixed window. Count each owner once. Report the underlying counts as well as the percentage. Keep invitations and internal testing separate.
That is a proposed measurement contract, not a claim that the historical July estimate already followed it.
What would count as an improvement?
Before comparing two onboarding versions, give both cohorts the same amount of time to reach the checkpoint. A cohort from yesterday cannot fairly be compared with one that has had two weeks to finish setup.
Track the missing step, not just the final total. Did more eligible owners create a website? Did more of those websites receive an event? Did people return after they had data? Record which changes shipped between cohorts, and keep acquisition sources in view.
We are not claiming a measured uplift from this teardown. A source review can show how a check works and identify a plausible obstacle. It cannot establish why every person left, or prove that removing a step caused more people to stay.
The next time a signup notification arrives, the useful follow-up is simple: did that person get data from their own website, and did it help them do anything?
Until we can answer that, the account count is only the start of the story.
Top comments (0)