OpenNVR 0.1.10: running an open-source NVR's AI on the Qualcomm Hexagon NPU

#opensource #ai #edge #python

OpenNVR is an open-source (AGPL) NVR that records and streams from any ONVIF or RTSP camera and runs the AI model you choose behind them: YOLO detection, pose, face recognition, plate OCR, open-vocabulary detection, and a vision-language agent that can look at a scene and reason about it. It runs on hardware you own, air-gapped if you need, with every inference audited.

Version 0.1.10 is the first release with hardware acceleration flavours, and this post is about the one I have numbers for: Qualcomm's Hexagon NPU.

The board

Arduino Ventuno Q: Qualcomm Dragonwing IQ-8275 (QCS8275), eight ARM cores, a Hexagon NPU, Ubuntu 24.04. Small, fanless, the kind of thing that sits behind a few cameras in a shop or a plant.

The numbers

All measured on that board with the published container images. Full windows, settle times and caveats are in the benchmark doc.

Tier-0 detector (YOLO, one frame):

Device	ms per frame
CPU, 8 threads	164
Hexagon NPU, fp16	8.6

In the full stack (streams, decode, tracking, the API, all on the same box):

Device	Cameras at 2 fps, no drops	Host idle
CPU	4 (10 cameras drop half their frames)	~4 %
NPU	16, 0 dropped	46 %

The adapters that sit behind the detector, end-to-end on one 1080p JPEG:

Adapter	CPU	NPU
YOLO	141 ms	77 ms
Pose	108 ms	64 ms
Package detection	100 ms	60 ms
Face recognition (InsightFace)	443 ms	62 ms
Plate OCR	248 ms	62 ms

The NPU's own share of an adapter call is about 9 ms. The rest is JPEG decode and preprocessing on a busy CPU, so the adapter path is now bounded by image handling, not inference.

How it works

Everything goes through ONNX Runtime's QNN execution provider, shipped as a plugin wheel (onnxruntime-qnn). There is no Qualcomm SDK to install on the host. The -qualcomm container gets the DSP device node, the DMA heap, and two host libraries bind-mounted, and that is it.

git clone https://github.com/open-nvr/open-nvr.git && cd open-nvr
./start.sh        # OPENNVR_ACCEL=auto detects /dev/fastrpc-cdsp and picks the qualcomm flavour

OPENNVR_ACCEL=cpu|intel|qualcomm overrides the probe. A box with only a CPU gets exactly the images it got before.

Compiled graphs are cached, so the second start skips the ~6 s compile. The pipeline reports which provider ONNX Runtime really ran on, and a request for an accelerator that silently fell back to CPU shows up as requested != effective in the metrics instead of being hidden.

Things that cost a day each

A plugin execution provider named in the providers= list is silently ignored. It has to be attached by device with add_provider_for_devices.
python:slim lacks libatomic1. The HTP backend fails to load and the session falls back to CPU with no error.
The first "NPU" number I got (12 ms, looked great) was a stub detector after a failed session. Check the health endpoint and the degraded counter before trusting a number.
The Ubuntu image ships Docker Compose 2.24.6, which randomly drops a container's second network at create. The installer now checks the Compose version and repairs network membership after every up.

Measure your own hardware

python -m detect_pipeline.bench --latency 200 --providers CPUExecutionProvider
python -m detect_pipeline.bench --latency 200 --providers QNNExecutionProvider:htp

It reports median, p90 and min per device, names the provider that really ran, and exits non-zero if the accelerator fell back.

Also in 0.1.10

Intel iGPU and NPU flavour via OpenVINO, VAAPI decode
Signed self-update: each release ships a manifest signed by us and verified on the box, opt-in
Camera tours for display boards, camera CSV/Excel import and export, bulk actions
The app SDK's Look and Investigate steps for writing your own camera workflows

NVIDIA, desktop GPUs and Jetson. If you have a Jetson, Rockchip or Hailo box you'd lend to a benchmark, I have one board and would like more.