A weekly habit: write down five things that surprised me, contradicted my prior, or changed how I think about something. This week was heavy on edge AI and pipeline failures.
1. Post-processing did more than another training run would have
I published two pieces this week about the shelf-scanning edge AI I run on a Raspberry Pi 3 — including one on the post-processing layers that mattered more than model accuracy. The accuracy number — mAP50 0.844 on a held-out test set of 1,255 instances — sounds reasonable until you see the raw detection output: false positives from shelf edges, detections of the same structural gap in every single scan, single-frame hits that vanish in the next pass.
None of that went away with more training. Three post-processing stages handled it instead: an ROI mask that ignores detections outside the shelf region, baseline subtraction that removes gaps already present in a "full shelf" reference scan, and a temporal majority vote that only promotes a detection if it appeared in at least 2 of the last 3 scans. Those three layers are what made the detector usable — not the 60 epochs of training on 9,358 images.
The lesson I keep landing on: evaluation metrics measure what the model can output. Post-processing determines what the system actually does with that output. They're not the same problem, and solving the first doesn't solve the second.
2. Inference at 8.5 seconds is fine when the cadence is hourly
A recurring reaction to the Pi 3 inference number — median 8.5 seconds per scan, range 8.4–11.8 s across 19 real device runs — is that it sounds slow. It is slow. But the system scans once per hour. An inference that takes 8.5 seconds inside a 3,600-second cycle occupies 0.24% of that window.
Latency only matters relative to cadence. An 8.5-second inference would be catastrophic for real-time object detection in a video stream. For hourly inventory monitoring, it is completely acceptable, and a Raspberry Pi 3 at ~$35 costs less than the cheapest cloud GPU instance runs in half a month.
I also dropped inference resolution from 640 px (used in training) to 416 px because the Pi 3 has 906 MB RAM and higher inputs pushed it over. The test-set mAP held at 0.844 at the lower resolution. That tradeoff — resolution for memory headroom — only becomes visible when you're running on the actual hardware under real constraints.
3. The "output inspection" article got a comment worth thinking about
I published a piece on why output inspection beats process monitoring for solo developer pipelines. The evidence is a detection-lag table from this repo: six failures, ranging from 36 to 113 days before anyone noticed. The common factor across all six: the jobs kept producing well-formed output files. A green workflow badge is not evidence that content actually shipped.
A comment asked whether I built the detection-lag metric myself or pulled it from somewhere. I built it from commit history, manually — the gap between the commit that introduced a fault and the commit that found it. No tooling required.
What struck me: the most specific number in the article (113 days) was also what generated engagement. Vague claims give readers nothing to push against. Specific ones invite a response.
4. Bug reporters on Steam have a median of 152.2 hours played
I wrote earlier this week about a Steam VoC engine that joins review text to reviewer playtime. The corpus currently has 4,888 review bodies collected, producing 753 topic assignments across 46 topics.
The finding that changed how I read bug complaints: in the topics I've measured, bug reports come from the most invested players — median 152.2 hours played, three quarters of them past the 50-hour mark. Compare that to "more fun than expected" reporters, who average 36.1 hours.
A flat count shows 54 bug reports. The playtime join tells you those 54 reports come from players who put more than six times the median hours in — the ones most likely to churn if the bugs stay unfixed.
The architecture point that made this measurable: the LLM groups reviews into topics by meaning; every number in the report is produced by a script re-tallying the assignments file. A model was never asked "how many?" That separation is what makes the counts trustworthy.
5. 97 failed uploads was a reach problem, not a quality problem
Earlier this week I published the numbers from a YouTube distribution analysis. The finding: 97 consecutive uploads failed a distribution gate. Engagement rate on the uploads that did receive views came to 22.36 likes and comments per 1,000 views — roughly 4× a 5.0/1,000 niche benchmark. The content worked with the people who saw it.
Almost nobody saw it. The constraint was distribution, not quality, and those two diagnoses lead to completely different fixes.
After a retention-focused change — faster narration pacing, trimmed silence, a large number in the first frame — 6 of the next 9 uploads were still growing after day 3. The before-cohort showed 0 of 97.
One caveat that should travel with those numbers: the before-cohort's early-growth window was not observed (collection started later), which introduces a lower-bound bias. And n is small on the after side. I'm reporting what the data shows, not extrapolating.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)