WiFi off, model running: what broke (and what shipped) building YOLO26 on MLX

By Igor Eduardo · Austin, TX · with Lexi Armstrong

Site: igoreduardo.com · Demo: youtu.be/c2v5Mdg5fpw

This is a build note from the webAI YOLO26 MLX Build Challenge (May 2026), not a product pitch. We shipped a single-file, on-device posture attention map that runs with WiFi physically off. The useful part for other builders is what failed first.

TL;DR

Seven days. Two-person build: Lexi Armstrong owned the problem and operational constraints; I owned the engineering.

What shipped uses yolo26n with released weights as-is, plus a deliberately simple bounding-box geometry heuristic for posture — standing / sitting / lying. Single-file Python (~200 LOC), single-thread synchronous loop, ~16 FPS at 720p on M4 / 32 GB / macOS 26.3 / Python 3.14, ~45 ms inference per frame.

An earlier attempt to fine-tune yolo26n on labeled posture data did not converge and was dropped.

We're writing this up because the failure modes are probably more useful than the demo — specifically the macOS 26 AVFoundation crash, the lazy-eval gotcha in yolo-mlx's Boxes, and the training collapse. And because the part that turned a demo into something a responder might actually trust came from the field side, not the code.

Where this started

SENTINEL is the meeting of two views of the same problem. Lexi saw it from the field — denied-comms / industrial security / operational-edge constraints, including what "zero network egress" has to mean when operational security depends on it. I saw it from healthtech — production-class triage and patient-flow systems in extended pilot with tier-1 hospitals in São Paulo. The core problem of who needs attention first when there are too many patients and not enough hands shows up in mass-casualty triage too, at different scale and stakes.

Concretely, the field side drove:

• Denied environment as architecture — no cloud leg, no paired device that reintroduces a network hop.
• Responder frame of reference — what is useful in the first few seconds, and what is noise under stress.
• WiFi physically off in the demo — zero egress as physical proof, not a slide claim.

Honest division of labor: Lexi made sure it was worth building and aimed at the right target; I built it.

What we tried first (and dropped)

The initial architecture was a FastAPI backend exposing a WebSocket stream to a browser overlay, with camera capture in a ThreadPoolExecutor. Two hard blockers killed it.

1. AVFoundation crash on macOS 26.3

With OpenCV 4.13's AVFoundation backend, capturing frames from a non-main thread crashed reliably with SIGTRAP. The trace pointed at:

cv2.abi3.so -> CaptureDelegate captureOutput -> CFRelease

This looks like a CFRelease reference-counting issue when AVCaptureSession is owned outside the main thread on macOS 26. We didn't patch around it — rewriting the capture stack wasn't worth the risk in a 7-day window. Flagging it because anyone building on this stack with a worker-thread capture pattern is going to hit it.

2. Lazy-eval deadlock in yolo-mlx Boxes proxies

The harder one to isolate. Benchmark showed clean FPS. Warmup completed. Model loaded. But when the WebSocket handler called detector.predict() on a real camera frame, it hung silently. Every time.

Diagnostic path:

• Stubbed inference (dets = []) — video appeared immediately. Camera, WebSocket, frontend all confirmed working.
• Prints inside yolo-mlx's predictor.py: _predict start, mx.eval done, returning results all appeared.
• Prints around predict() in detector.py: BEFORE PREDICT appeared, AFTER PREDICT never did.

The hang was not in MLX inference itself — it was in the box-iteration loop after the model returned:

for box in results[0].boxes:
    x1, y1, x2, y2 = [int(v) for v in box.xyxy[0].tolist()]  # hangs here

Root cause hypothesis: yolo-mlx's Boxes returns lazy MLX proxies for .xyxy, .conf, .cls. They aren't evaluated during inference — they're deferred. Calling .tolist() or indexing them triggers a secondary mx.eval() that deadlocked in the multi-thread setup. Warmup used np.zeros (zero detections), so the box loop never ran during warmup. Real frames produced detections, the loop ran for the first time, the lazy eval fired, and the pipeline froze.

Workaround — force-materialize before iterating:

import mlx.core as mx
import numpy as np

boxes = results[0].boxes
mx.eval(boxes.xyxy); mx.eval(boxes.conf); mx.eval(boxes.cls)
xyxy = np.array(boxes.xyxy)
conf = np.array(boxes.conf)
cls = np.array(boxes.cls)

for i in range(len(xyxy)):
    x1, y1, x2, y2 = [int(v) for v in xyxy[i]]

No .tolist() on MLX proxies inside loops. Convert to NumPy once, then iterate. Warmup with synthetic zero-detection frames does not exercise the box-iteration path, so this bug is invisible until a real subject enters the frame — worth a doc note or an explicit .materialize() helper on Boxes.

The pivot

Around day four we made a call. The architecture being hardened solved for production complexity not needed for a 7-day demo. We forked the official Yolo26-mlx challenge starter, deleted everything around the inference call, and rewrote it as a single synchronous loop on the main thread:

while True:
    ok, frame = cap.read()
    detections = model(frame)
    overlay = render_sentinel_ui(frame, detections)
    cv2.imshow("SENTINEL", overlay)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

That sidesteps the AVFoundation issue (capture on main thread) and the lazy-eval issue (no async / threading). ~16 FPS at 720p on M4 once settled. That's what shipped in the private build (repo stays closed while we finish the work).

The fine-tuning attempt

In parallel, we tried fine-tuning yolo26n on labeled posture data — a Roboflow posture-classification dataset (person_lying / person_sitting / person_standing). Training loss collapsed to near-zero by epoch 2; mAP stuck at 0.0 for the remaining epochs. We didn't isolate the root cause in the time available — most likely a label-format mismatch or normalization gap — but those debugging cycles weren't available with AVFoundation and lazy-eval also live.

Noting this because "trained posture head" appears on the roadmap, and we want to be explicit that it's an honest open problem — not something we skipped by choice.

Classification logic — being explicit

The shipped version does not use a trained posture classifier, and SENTINEL does not make a clinical triage decision. It uses released yolo26n.npz weights as-is. The layer on top is a deterministic posture heuristic on bounding-box geometry. It reports what the camera can actually see — body posture — not medical severity.

• Filter detections: class == 0 (person), conf ≥ 0.40
• For each detection, aspect = bbox_height / bbox_width
• aspect > 1.6 → standing
• 1.0 ≤ aspect ≤ 1.6 → sitting / slumped
• aspect < 1.0 → lying down

Why posture, not severity: a camera cannot see a pulse, internal bleeding, or a blocked airway. The honest output is "who is upright, who is down, and for how long" — a visual cue that helps a responder decide where to look first. The human triages.

Where the heuristic fails (honest):

• Upper-body-only framing: a standing person at close range has a near-square bbox and reads as sitting.
• Fetal-position lying: bbox can look near-square or vertical and gets misclassified.
• Two people overlapping front-to-back: one bbox swallows the other.

"We can write the rule on a napkin" was the right call for a 7-day safety-relevant demo — but it's a starting point, not the answer.

What worked / friction worth flagging

Worked: yolo-mlx 0.3.1 API; MLX Metal backend on Apple Silicon; starter repo structure (pivot took an afternoon); macOS 26 + Python 3.14 + M4 once threading was off the table.

Friction:

• OpenCV 4.13 + macOS 26 + worker-thread capture is broken (not a YOLO/MLX issue, but the pattern most demos teach).
• Lazy-eval on Boxes properties is a silent foot-gun.
• OpenCV Hershey fonts are ASCII-only — middle-dot separators rendered as ??.
• Bbox-aspect-ratio limits were obvious in retrospect; a small classifier head on the same backbone would handle edge cases.

What we'd build next

• Trained posture head replacing the heuristic (isolate label-format first).
• Multi-camera fusion on a local mesh, still zero egress.
• iOS / iPadOS port — MLX is already there.
• rPPG as research only — RGB rPPG needs conditions that don't hold in the field; real vitals path is thermal, not RGB.

Form factor: laptop demo ≠ product. Realistic V1 path we sketched — Vision Pro for prototype, field-grade waveguide for first-responder pilot — both keep inference on-device. Platforms that move compute to a paired puck reintroduce a network leg that weakens the zero-egress story for this use case.

Closing

We didn't build SENTINEL to chase a market — we built it because the environments where attention allocation matters most are often the ones where the network isn't there. Before this is a product: trained posture classifier, a real labeled dataset, and validation with a real responder who either uses the map or ignores it. No projections — we haven't earned them yet.

If the AVFoundation crash, the lazy-eval workaround, or the training-collapse note is useful to YOLO26-MLX docs, happy to write more on any section.

— Igor Eduardo (igoreduardo.com) & Lexi Armstrong

Demo: youtu.be/c2v5Mdg5fpw

Disclosure: drafted with AI assistance from build notes; technical claims and wording owned by the authors.