The problem
A high-resolution camera stream can overwhelm Dart isolates, trigger
frequent garbage collection, and increase end-to-end latency when every
frame is copied, converted, and rendered.
What this guide covers
This tutorial focuses on practical architecture and performance
engineering for production systems. The central principle is to measure
the complete pipeline, keep latency-sensitive work close to the
hardware, and make failure states visible to the operator.
Reference architecture
Glasses camera → native frame callback → bounded frame queue → optional
native resize/format conversion → latest-frame handoff → Flutter preview
or inference request. Use a latest-frame policy for interactive vision
rather than accumulating stale frames.
Implementation plan
- Measure frame arrival rate, native-to-Dart handoff latency, conversion time, inference time, and UI frame time separately.
- Choose a target sampling rate for the task; object guidance may need fewer frames than a recording workflow.
- Downscale before crossing the bridge when the use case permits it. Keep the original frame only when needed for capture or audit.
- Use a bounded queue and drop stale frames when the consumer falls behind.
- Stop capture and release buffers deterministically on route disposal, permission loss, disconnect, and app lifecycle changes.
Example
// Pseudocode: preserve freshness instead of queuing every frame.\nclass LatestFrameGate {\n bool _busy = false;\n Frame? _latest;\n\n void offer(Frame frame) {\n _latest = frame; // replace stale pending work\n if (!_busy) _drain();\n }\n\n Future<void> _drain() async {\n _busy = true;\n try {\n while (_latest != null) {\n final frame = _latest!;\n _latest = null;\n await process(frame);\n }\n } finally { _busy = false; }\n }\n}
The snippets are illustrative starting points, not drop-in integrations.
Match APIs and version requirements to the exact SDK, OS image, hardware
revision, and runtime in your project.
Performance checklist
- Measure end-to-end latency, not only one function or model invocation.
- Report p50 and p95/p99 latency alongside throughput, memory use, and dropped work.
- Bound queues and define what happens when consumers fall behind.
- Avoid repeated frame copies and expensive work on Flutter's UI isolate.
- Test sustained workloads after the device reaches thermal equilibrium.
- Keep UI state, device state, and robot state distinct.
- Add timeouts, cancellation, reconnection, and explicit stale-data indicators.
- Profile on the actual target hardware using a reproducible build.
Common pitfalls
- Avoid converting YUV to RGBA on every frame on the UI isolate if native/GPU conversion is available.
- Avoid retaining camera buffers beyond their documented lifetime.
- Never assume that the SDK's preview frame dimensions, pixel format, or rotation are constant.
Hands-on exercise
Benchmark 5, 10, and 15 frames per second at two input resolutions.
Compare p50/p95 processing latency, dropped frames, memory, and device
temperature.
Further reading
- Flutter performance best practices
- Flutter DevTools
- Android Developers
- NVIDIA Developer
- ROS 2 documentation
Useful Links
Website: www.v-modal.com
SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter
SDK Android: https://github.com/v-modal/vmodal_sdk_android
Discord: https://discord.gg/K72z28KUx
Top comments (0)