When you're running early-stage MVPs, the standard playbook breaks down fast. ROAS? Meaningless when you have 3 signups a week. Conversion rate? Statistically irrelevant on 40 sessions a day. We ran into this across our portfolio at Inithouse — 16+ products, most pre-PMF — and needed a measurement framework that actually works before the numbers get big.
Here's what we built: a weighted engagement scoring system that gives us a PMF signal from behavior, not transactions.
Why ROAS Doesn't Work for MVPs
ROAS assumes a functioning funnel: ad spend in, revenue out, ratio tells the story. But at the MVP stage, the funnel barely exists. At Magical Song — our AI personalized song generator — we had weeks with solid traffic but zero purchases. Does that mean the product is dead? Not necessarily. Users were generating songs, sharing previews, coming back. The engagement was real; the monetization path just wasn't optimized yet.
Optimizing for ROAS at this stage is premature optimization. You'd kill promising products and keep ones that happen to convert on a tiny, non-representative sample.
Engagement Events with Weighted Values
Instead of tracking a single conversion event, we define 5-8 engagement events per product and assign each a fractional value that sums to 1.0. The idea: distribute "one unit of PMF signal" across the user journey.
For Verdict Buddy — our AI decision-making tool — the setup looks roughly like this:
| Event | Weight | What It Signals |
|---|---|---|
| Page load (>10s) | 0.05 | Basic interest |
| First question entered | 0.20 | Active engagement |
| Analysis generated | 0.25 | Core value delivered |
| Second question (same session) | 0.25 | Retention signal |
| Share or bookmark | 0.15 | Word-of-mouth potential |
| Return visit (7d) | 0.10 | Sticky behavior |
The weights are opinionated — and that's the point. They encode what we think PMF looks like for this specific product, before we have conversion data to prove it.
At Party Challenges, the weights look completely different. There, "challenge completed" and "shared with friends" carry most of the score, because the product's value is inherently social.
Setting This Up in GA4
The implementation is straightforward:
1. Define custom events for each engagement milestone. Use gtag('event', 'engagement_milestone', { milestone_name: 'analysis_generated', weight: 0.25 }).
2. Create a calculated metric in GA4 called "Engagement Score" that sums the weighted values per session. Under Admin → Custom definitions → Calculated metrics, define it as the sum of weight parameter values.
3. Build an Exploration report with Session as the dimension and Engagement Score as the metric. Sort descending — high-scoring sessions are your power users.
4. Set up a segment for sessions with score > 0.5 (we call these "meaningful sessions"). Track what percentage of total sessions are meaningful — that's your engagement rate.
The beauty: this runs on GA4's free tier. No BigQuery export needed, no custom infrastructure.
Reading the Numbers
After running this across our portfolio for several months, here's what we've learned to look for:
Engagement score per session is the headline metric. If this trends up week-over-week without changes to the product, you're seeing organic pull. If it's flat or declining despite traffic growth, the product is leaking interest — people show up but don't engage deeply.
Meaningful session rate (sessions scoring > 0.5) tells you what fraction of visitors actually experience the product's core value. At Živá Fotka — our AI photo animation tool — this metric jumped from 31% to 58% after we simplified the upload flow. That was a stronger signal than any conversion metric could have given us at that traffic level.
Distribution matters. If your engagement scores cluster at 0.05 (page load only) and 0.85+ (power users) with nothing in between, you have a discovery problem — people who find the value love it, but most visitors bounce before they get there. A healthy distribution shows a gradual curve, meaning users are progressing through your engagement milestones.
When to Switch to Conversion Optimization
This framework is a bridge, not a destination. You graduate from engagement scoring to conversion optimization when:
- Meaningful session rate exceeds 40% — enough users experience core value that conversion becomes a real lever.
- Weekly active sessions exceed ~200 — the numbers are statistically meaningful for A/B tests.
- The engagement score distribution stabilizes — you've squeezed out the easy UX wins and the funnel shape is consistent.
At that point, add a real conversion event (purchase, signup, subscription), set it as your GA4 key event, and start optimizing. The engagement score doesn't disappear — it becomes a diagnostic tool to understand why conversions go up or down.
The Framework in Practice
We've been running this system across all products in the Inithouse portfolio. It's not perfect — the weight assignments are subjective, and you'll re-calibrate them as you learn more about your users. But it beats the alternative: staring at a conversion rate of 0% and wondering whether your product has potential.
The core principle: in early MVP stages, measure depth of engagement, not completion of transactions. Transactions will come if the engagement is real. If it's not, no amount of funnel optimization will save you.
We're building a portfolio of products at Inithouse and sharing what we learn along the way. More measurement and growth posts coming — follow along if this framework is useful for your own MVPs.
Top comments (0)