A build-in-public engineering log. Every number below is pulled straight from the test reports — measured values, not rounded to look pretty, not invented. A success you cherry-picked isn't a success.
Log #02 ended on an asterisk. We had a video action classifier at 94% accuracy and an end-to-end highlight pipeline at recall 0.86, but the newly-swapped "pixel finder" was measured under a generous assumption — perfect tracking, GT-interpolated boxes. Its product operating point, measured honestly against real kinematics, was only R 0.52 / P 0.57. We wrote, in plain text, that we'd have to wire it into the real pipeline with actual tracking lock and re-confirm with product_eval before claiming anything.
This log is that re-confirmation — plus the bigger thing that happened while we weren't looking: the model stopped being a model and became a product.
Where we are now:
- Product pipeline (real tracking, end-to-end): F1 0.887 — recall 0.868, precision 0.908 on the holdout config.
- The generous upper bound from Log #02 didn't collapse when we made it honest. It held.
- Shipped an actual product surface: a web app (upload → lock a player → auto-edit → done) and a cinematic slow-motion render pipeline.
- Deployment reality check: we pivoted the whole serving path from CPU to GPU.
Part 1 — Cashing the asterisk: the product number, measured honestly
In Log #02 we were careful not to celebrate the spotter's headline recall, because it was measured with GT-interpolated boxes — i.e. assuming the tracker never loses the player. That's the spotter's home-field advantage. The real question was always: what happens when the boxes come from the actual tracker, running end-to-end, on the real product config?
We ran it. Here are the receipts, straight from the eval reports, all on the same 45-video / 68-critical holdout set:
| Config | Recall | Precision | F1 | Fired | TP |
|---|---|---|---|---|---|
config.yaml (product baseline) |
0.853 | 0.906 | 0.879 | 64 | 58 |
config_gpu_a (GPU parity) |
0.853 | 0.906 | 0.879 | 64 | 58 |
config_spotter_v2 (holdout) |
0.868 | 0.908 | 0.887 | 65 | 59 |
config_gpu_b (precision-leaning variant) |
0.809 | 0.917 | 0.859 | 60 | 55 |
The number to sit with: the Log #02 product operating point was R 0.52 / P 0.57. The re-confirmed, real-tracking, end-to-end product point is R 0.87 / P 0.91 / F1 0.887.
We want to be precise about why this isn't cheating. The 0.52/0.57 in Log #02 was the kinematic-only operating point — the old finder, before the pixel spotter was integrated with tracking lock. The honest fear was that the spotter's 0.86-ish recall was an artifact of perfect boxes and would fall apart on real tracks. It didn't. When we integrated the spotter into the real pipeline with the actual tracker, the end-to-end product number landed at F1 0.879–0.887 depending on config. The generous assumption and the honest measurement converged instead of diverging. That's the good outcome, and it's the one we were least sure we'd get.
Two things we're nailing down honestly:
1. GPU parity is exact, not approximate. config.yaml on Apple Silicon (mps) and config_gpu_a on CUDA produce the identical R 0.853 / P 0.906 / F1 0.879 — same 64 fired, same 58 TP. That matters: it means moving to GPU is a throughput decision, not a quality one. The model doesn't behave differently on the box we ship on.
2. gpu_b is a knob, not an upgrade. The precision-leaning variant trades recall (0.853 → 0.809) for a hair more precision (0.906 → 0.917). Same 68 criticals, fewer firings. It's there so we can pick a false-positive tolerance per deployment, not because it's "better." For a highlight product, missing a moment is worse than an occasional extra clip, so the recall-leaning baseline is the default.
Part 2 — The model became a product
Log #02 was entirely about the model. This log is where the model grows a body. Two big pieces shipped.
The render pipeline: from "we found the moment" to "here's the clip"
Detecting an impact timestamp T is worthless to a parent with a phone unless it comes out the other end as something watchable. So we built the output layer — an orchestrator that turns (video + target trajectory + impact T) into a vertical slow-motion highlight mp4. The flow is deliberately ordered (stabilize, then retime):
- Decode only the window around
T([pre, post]) at native fps — never the whole video. - Resample the target trajectory to real frame times and smooth it with a Kalman/RTS pass.
- Generate stabilized frames with a moving-crop virtual camera.
- Integrate a speed ramp and invert it to get per-output-frame
(index, α). - Interpolate and stream straight to the writer — no buffering the whole clip in memory.
One design decision worth calling out because it's counterintuitive: V1 does not crop or zoom in. On low-resolution real-world footage, a lock-on crop just destroys image quality — the parent already framed the subject big enough. Vertical reframe and lock-on zoom are deferred to V2. We'd rather ship the un-degraded frame now than a shaky zoomed one.
The slow-motion itself uses a lead-anchored ramp: kinematic impact T tends to get detected late (at landing / follow-through), so we pull the slow-motion center ahead of T to cover the decisive action — the wind-up and contact — instead of slow-mo'ing the part after it's already over.
On top of single clips sits a reel compiler: rank highlights by criticality, greedily pick the best ones that fit a 15/30/60s budget, then re-order them chronologically so the reel still tells the game's story, with short crossfades at the seams. (Both render and reel are pure cv2/numpy, FFmpeg-free, silent MVP — audio is Phase 2.)
The web app
There's now an actual front end: a bilingual (KO/EN) React/TS app with the whole flow a non-technical user needs — upload, lock the player you care about, watch the AI edit, get the finished vertical highlight. Screens for landing/upload, target lock, processing, gallery, and done. The pitch is one sentence: upload your kid's game, get the highlights, don't touch an editor.
Part 3 — Deployment reality: CPU → GPU
Log #02's deployment plan was Oracle Cloud's Always Free tier — 24/7 on ARM CPU, torch-CPU, no GPU. The goal was a real server that stays up when the laptop sleeps, at zero cost. Two things made us pivot.
First, the free tier shrank. Oracle cut Always Free A1 from 4 OCPU / 24 GB to 2 OCPU / 12 GB (2026-06). Less headroom for exactly the workload we run.
Second, and decisively, CPU inference is too slow for the pipeline we actually shipped. The production path is RT-DETR detection plus r2plus1d_18 dense scoring across the whole video — on CPU, a single job takes minutes. That's survivable for an async "we'll notify you" MVP, but it caps daily throughput hard. So we built the GPU serving path: Dockerfile.cuda, a one-command deploy_gpu.sh (rsync → remote provision of Docker + nvidia-container-toolkit + firewall + build + start), and a T4 benchmark step. The pixel spotter (video_scorer.pth) bakes into the image as the deploy artifact, and — per Part 1 — GPU output is bit-for-bit the same quality as local, so this is purely about serving more jobs per day.
Part 4 — Two dead ends we're keeping dead (receipts included)
Honesty means reporting the things that didn't work and staying disciplined about not reviving them.
RF-DETR fine-tuning stays shelved. Back in Log #01 territory we fine-tuned RF-DETR to mAP 0.716 and it moved end-to-end recall backwards. We re-confirmed on the real domain: no improvement. We're keeping baseline RT-DETR. Swapping to rfdetr remains a config flag away, but the measured verdict is that it doesn't earn its complexity.
The pose-TCN action classifier is now legacy. The pixel spotter is the confirmed production finder-and-judge (F1 0.887). The old pose-TCN path is effectively disabled (crit_tcn_floor: 1.0) because it showed no discriminative power on the real domain. It's still hot-swappable if we ever want it back, but nothing in the default path depends on it.
Neither of these is a failure we're hiding — they're two forks we explored, measured, and closed. The 0.887 didn't come from adding more model. It came from finding the right one and refusing to bolt the wrong ones back on.
Where we stand
- Product number, honest: F1 0.887 (R 0.868 / P 0.908) end-to-end with real tracking. The Log #02 asterisk resolved in our favor — the generous upper bound survived contact with reality.
- GPU parity: identical quality on mps and CUDA; the CPU→GPU move is throughput, not accuracy.
- It's a product now: upload → lock → auto-edit → vertical slow-mo reel, behind a real web app.
- Discipline held: RF-DETR and pose-TCN both measured, both closed, neither revived.
The remaining star, unchanged
The domain caveat from Log #02 still stands and we're not going to let it fade: these numbers are on SVW-derived research footage. The real test is still a commercial rebuild on genuine user uploads. If F1 0.887 survives that domain shift, we have a product. If it drops, you'll see the drop here first — same as every other number in this log.
If you're building in the same space and hitting the "the model works but is it a product" wall, I'd love to compare notes in the comments — especially on the CPU-vs-GPU serving math for video pipelines.
Top comments (0)