Last Tuesday I spent almost two hours in Mixpanel.
I opened 14 charts. Signups by source. Session length. A funnel with three people in it. A retention curve that looked like a ski jump because week-one cohort was eight users and two of them were me on a second account.
At the end I still could not answer the only question that mattered: are we getting better this week, or am I just staring at noise?
That was the week I killed the dashboard theater and put five numbers in a Google Sheet. Twenty minutes every Monday. Same columns. One decision. Done.
If you are under ~100 users, this is probably enough.
Why under-100 metrics look different
At 10k users, a 2% dip in activation is real. At 47 users, a 2% dip is one person who went on holiday.
Tiny cohorts lie. Last week's "activation rate" jumped because three friends signed up and finished onboarding because you DMed them.
This week's "retention" tanked because one workspace went quiet after a feature you shipped on Friday. Neither number means what the chart caption says.
Under 100, qualitative beats vanity. A stranger who replies to your onboarding email with "I got stuck on step 2" is worth more than a heatmap of your pricing page with 19 sessions. Heatmaps need volume. You do not have volume yet.
I still look at product analytics. I just refuse to let them decide the week until the sample is less embarrassing.
The 20-minute weekly review
I do this every Monday morning before Slack. Timer on. No rabbit holes.
Open a sheet. One row per week. Columns I actually fill:
| Column | What I pull | Where it lives |
|---|---|---|
| Week of | Monday date | typed |
| New signups | count of real accounts (filter test + spam) | Stripe / auth DB / whatever |
| Activated | signed up AND did the one job that means they "got it" | event or manual check |
| Returning in 7d | activated users who came back at least once in days 2–7 | query or sheet lookup |
| Stranger replies | emails / DMs / support notes from people I did not already know | inbox folder |
| Failed payments / cancels | count + one-line reason each | Stripe + cancel survey / email |
That is five numbers plus a short notes cell. Sometimes I add MRR if money moved. Sometimes I skip it if nothing billed.
Activated needs a hard definition. Not "logged in twice." For us it used to be "created first project and invited nobody" — wrong. Now it is "finished the core action once without me on a call." Write yours in the sheet header so you do not redefine it every week when the chart looks sad.
Returning in 7d is ugly on purpose. I do not care about D30 yet. If nobody comes back in a week, D30 will not save you.
Stranger replies is the column founders skip because it is not "data." It is the most honest signal I have under 100. One cold reply that says "I almost paid but billing failed on mobile" beats a polished funnel slide.
For failed payments and cancels I write one line each: "card expired — recovered day 2" or "cancel — too expensive for side project." Patterns show up by week four. I wrote a longer note on how I track product metrics before 100 users if you want the fuller tracking setup; this post is just the Monday ritual.
Timer hits 20. I stop. Incomplete cells stay blank. Blank is better than inventing a chart.
What to ignore until later
Things I used to open and now do not until we are past ~100 active:
- Multi-step funnels with n under ~30. You will overfit to three people.
- Heatmaps and session replays as a weekly habit. Useful for one stuck flow. Useless as a ritual.
- Multi-touch attribution. Your "Google vs Twitter vs friend DM" model is fiction when half of signups are people who already know you.
- Fancy cohort charts with five segments. You have one segment: people who tried the product.
- Vanity spikes from a launch post. Celebrate, then look at activated + returning, not impressions.
If something is broken in the first session, fix that with a short walkthrough and a couple of calls — same energy as when I stopped obsessing over the landing page and fixed the first 10 minutes. The weekly sheet will show whether activation moved. The heatmap will not tell you what to ship next Monday.
Decision rule: pick ONE change
The review is useless if it ends in "interesting." It has to end in one change for the next seven days.
My rule:
- Look at activated / signups. If that ratio is soft, the change is onboarding or empty state — not ads.
- Look at returning in 7d. If activate but ghost, the change is a re-engage email, a changelog ping, or removing a confusing second step.
- Look at stranger replies. If the same friction shows up twice, that is the change. Do not invent a roadmap item from one polite compliment.
- Look at failed payments / cancels. If money is leaking, fix billing copy or the cancel reason before you build a new feature.
Then I write one line at the bottom of the row: Next week: ___.
Examples from real weeks:
- Next week: shorten invite flow from 4 screens to 2
- Next week: email day-2 activated users who never returned
- Next week: fix mobile card update link (two failed renewals)
Only one. If I write three, I do zero.
When the sheet says "I do not know why people bounce," that is not a metrics problem. That is a talking-to-humans problem. Go get on a call with your first 15 users and come back next Monday with better notes. The sheet cannot interview people for you.
Checklist you can run this week
- [ ] Create a sheet with the five columns above
- [ ] Define "activated" in one sentence in the header
- [ ] Block 20 minutes Monday (or your quiet day)
- [ ] Fill last week once as a baseline, even if ugly
- [ ] Write exactly one "Next week" change
- [ ] Do not open Mixpanel until the sheet row is done
That is the whole system. Boring on purpose.
Under 100 users you do not need a metrics stack. You need a habit that answers "are we getting better?" without lying to you with pretty charts. Twenty minutes. Five numbers. One change. Then go ship.
Top comments (1)
The "(filter test + spam)" note in the signups column is the bit I'd make mechanical. If test accounts are marked when they are created (a flag your own signup path sets, or an address on a domain you own), the filter becomes a fixed query instead of a judgment call each Monday, and the numbers stay comparable week to week. I'd also keep spam as its own column, because a jump there means something very different from a jump in real signups.