Most MVPs launch in one of two states. Either there is no analytics at all, or someone dropped in the default snippet and it has been dutifully rec...
For further actions, you may consider blocking this person and/or reporting abuse
the identity section is the one i'd move earlier. everything else in the plan assumes one person is one row, and if that assumption breaks after launch, the launch data is the part you can't trust or repair. decisions-first is good; identity-first is the precondition.
That's a fair reordering, and I think you're right. Every other number in the plan quietly assumes one person is one row, so if identity breaks, activation and retention break with it, and you can't backfill a merge you never recorded. In the post it reads like one section among several, but it's really the precondition for all of them. If I rewrote it, I'd open with: pick the user ID, decide the exact moment anonymous becomes known, and test the merge before the first event ships. Thanks for pushing on that.
glad that landed. the "one person one row" assumption is where every dashboard i've seen goes sideways first, and everything downstream quietly inherits the merge bug. opening with "pick the user id, decide the exact moment anonymous becomes known" is a much stronger first page.
Agreed, and "everything downstream quietly inherits the merge bug" is a better one-line summary of the problem than anything I wrote. Thanks for the back and forth, it will shape how I write the next version of this.
appreciate that. for the next version, the line i'd add: the merge bug doesn't just corrupt the past, it taxes every query you write after it, because each one now has to hedge against double-counting.
That is a great way to put it. The corrupted rows are a one-time cost, but the hedging is permanent: every query after it carries a DISTINCT or a dedupe join that someone has to remember, and the one person who forgets ships a number that is quietly wrong. Which is the best argument I know for fixing identity before launch rather than after.
There's a small tension between "send outcomes from the server" and the LectureNotes activation being a student opening the first summary. Opening is a view, so it's exactly the kind of event ad blockers and flaky mobile clients eat. Do you record summary_viewed when the API serves the summary content, or does it come from the client? If it's client side, the activation rate is partly the ad-blocker rate you mention.
Good catch, and you're right, that tension is real. "Opened the first summary" is a view, so if it only fires from the client, part of what you're measuring is ad blockers and dropped mobile requests. The way I'd resolve it in the contract: record activation on the server when the API serves a completed summary's content to that user for the first time, and keep a client-side summary_viewed (fired when the screen finishes rendering with content) as a secondary signal for screen-level questions. The gap between the two is useful on its own, since a big gap points at either instrumentation loss or people who fetched and bounced. The post should have spelled that out instead of leaving it as an example of a view event. Thanks for pushing on it.
LectureNotes의 활성화를 가입이나 생성 완료가 아니라 첫 요약 열람으로 정의한 예가 좋네요. summary_generated와 summary_viewed를 별개 사건으로 남기고, 같은 결과를 재시도해서 연 경우의 집계 기준까지 계약에 적으면 ‘생성은 됐지만 사용하지 않은 상태’를 놓치지 않겠습니다. 이벤트 타입뿐 아니라 발생 횟수와 순서도 시험하면 유용할 것 같습니다.
Thank you! Yes, keeping summary_generated and summary_viewed as separate events is exactly how you catch "generated but never opened", and that gap is often the most useful number in the whole funnel. Good call on writing down how retries and re-opens of the same summary count, and on testing counts and order, not just event names. A contract that only checks names would happily pass a double-fired activation.