DEV Community: Varun Pratap Singh

OpenNVR 0.1.10: running an open-source NVR's AI on the Qualcomm Hexagon NPU

Varun Pratap Singh — Sat, 31 Oct 2026 10:15:25 +0000

OpenNVR is an open-source (AGPL) NVR that records and streams from any ONVIF or RTSP camera and runs the AI model you choose behind them: YOLO detection, pose, face recognition, plate OCR, open-vocabulary detection, and a vision-language agent that can look at a scene and reason about it. It runs on hardware you own, air-gapped if you need, with every inference audited.

Version 0.1.10 is the first release with hardware acceleration flavours, and this post is about the one I have numbers for: Qualcomm's Hexagon NPU.

The board

Arduino Ventuno Q: Qualcomm Dragonwing IQ-8275 (QCS8275), eight ARM cores, a Hexagon NPU, Ubuntu 24.04. Small, fanless, the kind of thing that sits behind a few cameras in a shop or a plant.

The numbers

All measured on that board with the published container images. Full windows, settle times and caveats are in the benchmark doc.

Tier-0 detector (YOLO, one frame):

Device	ms per frame
CPU, 8 threads	164
Hexagon NPU, fp16	8.6

In the full stack (streams, decode, tracking, the API, all on the same box):

Device	Cameras at 2 fps, no drops	Host idle
CPU	4 (10 cameras drop half their frames)	~4 %
NPU	16, 0 dropped	46 %

The adapters that sit behind the detector, end-to-end on one 1080p JPEG:

Adapter	CPU	NPU
YOLO	141 ms	77 ms
Pose	108 ms	64 ms
Package detection	100 ms	60 ms
Face recognition (InsightFace)	443 ms	62 ms
Plate OCR	248 ms	62 ms

The NPU's own share of an adapter call is about 9 ms. The rest is JPEG decode and preprocessing on a busy CPU, so the adapter path is now bounded by image handling, not inference.

How it works

Everything goes through ONNX Runtime's QNN execution provider, shipped as a plugin wheel (onnxruntime-qnn). There is no Qualcomm SDK to install on the host. The -qualcomm container gets the DSP device node, the DMA heap, and two host libraries bind-mounted, and that is it.

git clone https://github.com/open-nvr/open-nvr.git && cd open-nvr
./start.sh        # OPENNVR_ACCEL=auto detects /dev/fastrpc-cdsp and picks the qualcomm flavour

OPENNVR_ACCEL=cpu|intel|qualcomm overrides the probe. A box with only a CPU gets exactly the images it got before.

Compiled graphs are cached, so the second start skips the ~6 s compile. The pipeline reports which provider ONNX Runtime really ran on, and a request for an accelerator that silently fell back to CPU shows up as requested != effective in the metrics instead of being hidden.

Things that cost a day each

A plugin execution provider named in the providers= list is silently ignored. It has to be attached by device with add_provider_for_devices.
python:slim lacks libatomic1. The HTP backend fails to load and the session falls back to CPU with no error.
The first "NPU" number I got (12 ms, looked great) was a stub detector after a failed session. Check the health endpoint and the degraded counter before trusting a number.
The Ubuntu image ships Docker Compose 2.24.6, which randomly drops a container's second network at create. The installer now checks the Compose version and repairs network membership after every up.

Measure your own hardware

python -m detect_pipeline.bench --latency 200 --providers CPUExecutionProvider
python -m detect_pipeline.bench --latency 200 --providers QNNExecutionProvider:htp

It reports median, p90 and min per device, names the provider that really ran, and exits non-zero if the accelerator fell back.

Also in 0.1.10

Intel iGPU and NPU flavour via OpenVINO, VAAPI decode
Signed self-update: each release ships a manifest signed by us and verified on the box, opt-in
Camera tours for display boards, camera CSV/Excel import and export, bulk actions
The app SDK's Look and Investigate steps for writing your own camera workflows

NVIDIA, desktop GPUs and Jetson. If you have a Jetson, Rockchip or Hailo box you'd lend to a benchmark, I have one board and would like more.

Repo: https://github.com/open-nvr/open-nvr
Release notes: https://github.com/open-nvr/open-nvr/releases/tag/v0.1.10
Hardware guide: https://github.com/open-nvr/open-nvr/blob/main/docs/HARDWARE.md
Discord: https://opennvr.org/discord

We tested every small model our camera agent uses on a 4-core CPU. The bigger model lost.

Varun Pratap Singh — Mon, 19 Oct 2026 18:57:56 +0000

I maintain OpenNVR, an open source video recorder with a camera agent built in. You can ask it what's happening on a camera, arm an alarm, search old footage, and soon it will look into events on its own.

Most people who run it don't have a GPU. They have a small PC with 4 cores. So for this round we tested every model the agent depends on, using a 4 vCPU machine with no GPU. Until now we had picked models by reputation, like most projects do. We wanted numbers that anyone in open source can check, so every test script is in the repo.

Four of our defaults changed. Some results were not what we expected.

How we tested

One small test per job:

Language model: can it use tools and finish a multi-step investigation?
Vision model: questions with known answers on labelled images.
Speech to text: word error rate on the phrases people actually say to the agent.
Text to speech: is it faster than real time, and is it understandable?
Search: find the right image from a text description.

All timings are on 4 CPU threads.

Language model

We gave it 20 investigation cases, for example "person at the loading bay at 01:14". It has to check the camera, the history and the site rules, then decide. Each case ran 3 times.

Model	Passed	Checked before deciding
granite4:micro	44 / 60	56 / 60
granite4:tiny-h (7B MoE, 1B active)	35 / 60	49 / 60
qwen3:1.7b (our old default)	21 / 60	36 / 60
LFM2-1.2B-Tool	19 / 60	37 / 60
qwen3:4b	9 / 60	9 / 60

qwen3:4b scored less than half of qwen3:1.7b. When we read the traces, the reason was simple. It thought for a long time and almost never called a tool, so it decided without looking. Granite 4 micro calls about two tools before it answers, and that alone more than doubled the pass rate. It is about twice as slow per turn. For an investigation that runs in the background, that's fine, so it is our default now.

We also found a bug in our own agent. We had told the model to always look before describing, so when a user said "thanks, that's all", Granite went and checked a camera. Now small talk goes to the model without any tools. That helped every model, not just Granite.

Vision model

128 labelled images, 3 questions each: one about something in the picture, one about something that isn't (a "yes" here is a hallucination), and a people count.

Model	Wrong "yes", of 126	Count within one, of 61	CPU time per new frame
qwen3-vl:2b-instruct	0	50	18 s (follow-up questions are free)
gemma3:4b	8	48	17 s
moondream 0.5b	22	42	7.6 s

Qwen3-VL never said yes to something that wasn't there. On a GPU it answers in about 3 seconds, so it's the default when a GPU is present. But on 4 cores it needs 18 seconds just to read the frame. If someone is at the gate, an answer after 18 seconds is too late to be useful. So on CPU we kept Moondream.

One tip if you use Ollama: the plain qwen3-vl:2b tag is the thinking version and ignores think: false. Use the -instruct tag.

Speech to text

Model	Word error rate	Time per sentence
Parakeet TDT 0.6B v3 (onnx-asr, int8)	1.4%	0.31 s
faster-whisper base.en (int8)	3.3%	1.42 s
faster-whisper small.en	3.7%	2.67 s

This was the easy decision. Half the errors in about a fifth of the time. Speech recognition used to be the slowest step in a voice turn and now it's a third of a second. Parakeet can't translate and doesn't do Hindi, so Whisper is still there for sites that need that.

Text to speech

Kokoro-82M sounds nicer than Piper. But on 4 cores it runs at 1.9x real time, so it takes longer to make the audio than to play it, and replies would pause mid-sentence. Piper runs at 0.23x. We kept Piper.

Search and detection

For searching footage by description, like "red truck at the dock", SigLIP 2 B/16 scored 0.672 mAP against CLIP ViT-B/32's 0.571. It's slower per image (57 ms against 26 ms), but search quality matters more here. It's the default now, and OpenNVR re-embeds old footage automatically after the upgrade.

The always-on detector moved from YOLOv8n to YOLO26n: 62 ms per frame instead of 75, slightly more accurate, same output format.

What we learned

For agents, size wasn't what mattered. Whether the model checks with its tools before deciding mattered more than parameter count.
On a camera, a slow answer is a bad answer. A good answer at 8 seconds beat the best answer at 18.
Test on the hardware your users have. Most of these rankings flip on a GPU, and most of our users don't have one.

All the scripts and raw numbers are in the repo: MODELS_AND_LATENCY.md. If you run them on other hardware, please send a PR with your numbers.

One gap we couldn't close: a vision model under 2B parameters that reads a frame in under 5 seconds on CPU and still doesn't hallucinate. If you know one, tell me in the comments.

Originally published on opennvr.org. Drafted with AI assistance; all measurements are from the OpenNVR repository.

What if your IP cameras were programmable infrastructure, not just surveillance devices?

Varun Pratap Singh — Tue, 13 Oct 2026 16:55:41 +0000

I've been working on OpenNVR, an open-source project that started around IP cameras and NVR, but the bigger idea is to make camera infrastructure useful for building entirely new applications.

Most camera setups are designed around viewing and recording footage. But what if developers could use the same infrastructure to build their own computer vision applications, AI workflows, event-driven services, and automation systems?

That's the direction we're exploring with OpenNVR.

A few things we're working toward:

Open camera infrastructure — work with ONVIF-compatible IP cameras without being locked into a proprietary ecosystem.
Build your own applications — use APIs and an SDK-oriented approach to build services on top of camera streams and events.
Bring your own AI — integrate your own models and vision pipelines instead of being tied to a single vendor's AI.
Self-hosted and privacy-conscious — keep your camera infrastructure and data under your control.
Beyond traditional NVR — explore use cases ranging from security automation to custom vision applications and AI agents.

The interesting question for me is whether camera infrastructure can become something like a platform developers build on, rather than just another appliance they install.

We're still building and learning, and I'd genuinely appreciate feedback from people working on computer vision, self-hosted infrastructure, AI agents, or open-source systems.

GitHub: https://github.com/open-nvr/open-nvr
Website: https://www.opennvr.org

If the idea interests you, a GitHub star helps other developers discover the project. If it doesn't, tell me what we're missing — honest technical feedback is just as valuable.