Smart Glasses + Flutter: Frame Skipping That Preserves Interaction Quality

#flutter #smartglasses #camera #performance

Smart Glasses + Flutter: Frame Skipping That Preserves Interaction Quality

When a camera or vision pipeline produces frames faster than the app can process them, a queue makes the interface increasingly delayed. A deliberate frame-skipping policy can improve responsiveness.

Decide which frames are disposable

For preview or detection, the newest frame may matter more than every historical frame. Recording and safety-critical perception need different guarantees.

Prefer latest-frame processing

Keep at most one pending preview frame. If processing is busy, replace the pending preview frame rather than letting latency grow.

Avoid unnecessary pixel copies

Inspect image format and buffer ownership. Convert formats only when needed, release buffers promptly, and consider native processing for device-specific camera pipelines.

Separate preview and inference cadence

The UI can show a smooth preview while inference runs less frequently. Label detection age so overlays are not mistaken for current reality.

Profile sustained sessions

Thermal limits, resolution, codec, and memory bandwidth change the optimal policy. Test long sessions on target hardware.

Example implementation

This Dart snippet illustrates one part of the design. Adapt types, transport behavior, timestamp semantics, and platform APIs to your actual system before production use.

bool processing = false;
CameraFrame? pendingFrame;

Future<void> onFrame(CameraFrame frame) async {
  // Illustrative: replace CameraFrame/processFrame with your API.
  pendingFrame = frame;
  if (processing) return;
  processing = true;
  try {
    while (pendingFrame != null) {
      final next = pendingFrame!;
      pendingFrame = null;
      await processFrame(next);
    }
  } finally {
    processing = false;
  }
}

Performance checklist

Measure p95/p99 behavior on target hardware, not only a desktop simulator.
Bound queues and buffers; define what may be coalesced or dropped.
Keep heavy parsing, image processing, and inference away from the Flutter UI thread.
Track source timestamps, sequence numbers, stale-data rate, and dropped updates.
Test network degradation, reconnects, sustained operation, and resource pressure.
Keep control commands and critical alerts separate from replaceable telemetry.