I've been reviewing pull requests for most of my career.
At some point the queue got bad enough that I stopped asking "which PR should I review fi...
For further actions, you may consider blocking this person and/or reporting abuse
Glad the offscreen document note made it into the build log queue. On the rolling debug view for AI calls, one small addition that pays off later: store which prompt version produced each entry. Prompts change between releases, so when someone reports a bad summary from a week ago you can tell right away whether it came from a prompt you've already replaced. And the closing ratio, the mistakes you catch yourself versus the ones readers have to catch for you, feels about right for anything running in production.
That's a great addition — storing the prompt version alongside the debug entry is something I hadn't thought of, but it solves a real problem.
Why it matters: When a user reports a bad summary from a week ago, I need to know if the prompt has changed since then. Without versioning, I'm debugging against the current prompt, which might not even be the one that produced the output.
How I'll implement it: I'm adding a
promptVersionfield to the debug log — a hash of the system prompt + user prompt template. When the prompt changes, the version changes. That way I can always replay the exact prompt that produced a given output.On the closing ratio: you're right — it does feel about right. The ones you catch yourself, and the ones you needed someone else to find. That's one of the reasons I write these articles. The best feedback comes from people who've hit the same problems and found a cleaner path.
Thanks again for the sharp observation — Build Log #008 is going to have a dedicated section on this.
Thanks for sharing!
Really strong breakdown — especially the service worker state issue and the SSE chunk boundary bug. Those are the kind of failures that only show up in production, which makes them easy to underestimate during build time.
The most valuable pattern here is the consistent theme across all five decisions: optimizing for what’s interesting to build vs what’s necessary to operate. MV3 persistence, deterministic fallback logic, permission design, and observability all fall into the “unsexy but critical” category that usually gets postponed until users surface the pain.
The AI-first vs deterministic-first point is also a key product lesson. In practice, users don’t experience “AI features” in isolation — they experience whether the core workflow works reliably. Everything else is incremental.
The missing piece I’d add (and you hint at it with the debug view) is operational visibility as a first-class feature. Once you have AI in the loop, every “wrong output” becomes a reproducibility problem, not a UX complaint.
Overall, this reads less like “mistakes” and more like a clean map of where modern extension + AI products actually break in production.
The MV3 service worker state issue hit me too. Storing things in module-level vars feels natural until the worker dies silently and you're left wondering why everything reset. chrome.storage.local ended up being the only reliable option, but the migration was annoying.
Decision 2 resonates as well — building what you think users want vs what they actually ask for. The gap between "AI summaries" and "CSV export" is a classic example of builders over-indexing on shiny features while users just want boring utility.
The MV3 state migration pain is real — and it's one of those things that feels like a "me problem" until you realise everyone building MV3 extensions hits it.
On the migration: I got lucky in that I caught it early. But I've seen projects where the migration is a full rewrite because the original code was built around in-memory state. The pattern I landed on: treat
chrome.storage.localas the source of truth from day one, even in the prototype. The extra async calls are a small price compared to the cost of a mid-project rewrite.On the "shiny vs boring" gap: I think this is one of the hardest lessons for solo builders. AI summaries were genuinely fun to build. CSV export was boring. But support tickets don't come in saying "your AI summary is great" — they come in saying "why can't I export this data?"
What shifted my mindset: I started thinking about features in terms of "what breaks the user's workflow if it's missing?" instead of "what's the most impressive thing I can build?" Multi-account support broke the workflow for users with personal + work GitHub accounts. CSV export broke reporting. AI summaries were nice-to-have for most users.
Now I prioritise: (1) workflow essentials, (2) reliability, (3) polish, (4) shiny features. That order has saved me from building a lot of stuff nobody asked for.
Have you found any strategies that help you distinguish between what users say they want and what they actually use?
Honestly this took me way too long to learn.
The biggest shift for me: stop listening to feature requests and start watching behavior. A few things that actually work:
The "would you pay for it" test - When someone asks for a feature, I just ask "if this existed today, would you pay $5/mo for it?" Their reaction tells you everything.
Complaint intensity - "I wish it could do X" = nice to have. "I keep doing X manually every week and it drives me insane" = real pain worth solving.
The 3-request rule - If 3+ unrelated users bring up the same problem organically, it's almost certainly real. One person asking? Might be an edge case.
Analytics don't lie - I've shipped "highly requested" features that almost nobody actually used after launch. Meanwhile the boring stuff like CSV export and keyboard shortcuts get used daily.
Hardest lesson: the features I was most excited to build were rarely what users actually needed. Now I ask "what would make you less likely to leave?" instead of "what would make you sign up?"
This is gold. The "would you pay for it" test is brutal but necessary — and the 3-request rule is a great filter. I've been using "complaint intensity" informally, but naming it makes it actionable.
The bit about analytics not lying hits hard. I've shipped features I was excited about that barely got used, and boring stuff like export that gets daily use. Now I ask "what would make you less likely to leave?" instead of "what would make you sign up?" — that frame changed everything.
Thanks for sharing this. I'm going to start using the 3-request rule explicitly.
How did you handle PR queue management in your Chrome extension project, and did you implement any automated testing to catch issues before they reached reviewers? I'd love to hear more about your experience.
Great question — this is one of those things that looks simple until you actually build it.
PR queue management:
The extension doesn't hold a queue internally. It reads from GitHub's API on demand via GraphQL (for the PR list) and REST (for individual diffs). The "queue" is really just the cached state in
chrome.storage.local, refreshed every 3 minutes.The key pattern: the service worker is the only writer. The popup and options pages are read-only. This keeps state predictable — there's no race condition where two contexts try to update the cache simultaneously.
On automated testing:
This is one area where I'm still catching up. Currently I have:
manifest-validate.ymlworkflow that catches JSON errorsWhat I don't have yet: end-to-end tests that simulate a real GitHub PR flow. The challenge is that the extension needs a real GitHub token and a real AI key to work fully. I'm exploring using Playwright with mocked responses, but it's not there yet.
What I'd do differently: Build the test harness earlier. The extension is fully functional now, but retrofitting tests is harder than building them alongside the code.
If you're building something similar, I'd recommend starting with unit tests for the risk scoring engine (deterministic logic is easy to test) before adding the AI layer.
good to learn, I will watch it and add some lessons to my chrome extension or other tools too.
thanks
Solid map of where MV3 + AI extensions actually break. One thing I'd add to the state-design discussion, building on Decision 1: when the user can pick the destination AI per action (their ChatGPT tab, Claude tab, a Discord webhook, a local Ollama instance), your state model needs a slot for active_target_id that can legitimately be null.
Defaulting to whatever was last selected works during dev, but in production you hit the same kind of silent breakage as the module-level state problem — a tab that got closed, an Ollama instance that is no longer running, an auth token that expired mid-session. If the popup renders with a stale target id and the user clicks before noticing, you ship a payload to the wrong place.
I treat active_target_id like a cookie: it can be present, expired, or absent, and the UI has to distinguish all three. The cheap version is a
last_verified_atfield next to the id, and averifyAndPrompt()step on popup open that re-checks the target is reachable before showing the click button. It's two extra lines of state and a small migration, but it costs almost nothing in latency.Daily-use extensions expose product debt fast. For AI PR review, I would be most careful with trust boundaries: what runs locally, what leaves the browser, what gets stored, and how clearly the user can see why a suggestion was made.