YOLOv11-N on a Qualcomm Dragonwing RB3 Gen 2 (QCS6490), measured on Qualcomm AI Hub's real-device workbench. Depending on which runtime path I took, the same model on the same chip took 134.7 ms or 5.03 ms. "It runs" told me nothing. Here are all six paths, including the four that failed.
The six paths
| # | Path | Result | Layer placement |
|---|---|---|---|
| 1 | TFLite (default) | 134.7 ms | 343 layers: 19 NPU, 168 GPU, 156 CPU |
| 2 | Precompiled QNN, FP32 | rejected | the device demands quantized I/O — a float model doesn't get in |
| 3 | Precompiled QNN, w8a16 QDQ | conversion failed | server returns a bare exit code 14, no actionable detail |
| 4 | QNN DLC, FP32 | compiles, fails on device | QnnModel_composeGraphsFromDlc: MODEL_GRAPH_ERROR |
| 5 | Precompiled QNN, w8a8 QDQ | 5.03 ms | 341 layers, all NPU |
| 6 | Galaxy S25 (control), TFLite and QNN | ~2.9 ms both | all NPU |
Notes on the failures, because they're the useful part:
- Path 1 is the dangerous one: it works. It just quietly runs most of the network on GPU and CPU. Without the per-layer profile I would have written down "RB3 Gen 2 does YOLOv11-N in 134.7 ms" and moved on.
- Path 3's failure is a dead end with no signpost — a context-binary conversion that returns only an exit code. I filed it upstream (qualcomm/ai-hub-models#364) with the job details; if you hit the same wall, that's the thread to watch.
- Path 4 is the classic: "it compiled" is not "it runs". Graph composition is a separate gate, on the device, and it can say no.
- Path 6 is the control that makes the story: on a flagship phone, even the default TFLite delegate lands everything on the NPU. The broken default is specific to the edge SoC path, not to the model and not to TFLite in general.
The rule I'm taking away
A latency number without its path is not a measurement. Runtime, precision, and layer placement are part of the number. The per-layer profile is the only evidence that distinguishes "the NPU is slow" from "the NPU never got the work" — and those two conclusions lead to opposite decisions (different chip vs. different export settings).
This is the same disease I keep meeting on Rockchip boards, where an RKNN model can silently fall back to CPU operator by operator. Different vendor, different toolchain, identical failure mode: the default path is optimistic, the profile doesn't lie.
If you're benchmarking edge AI, publish the path with the number. If you're reading a benchmark that doesn't, you now know what it might be hiding.
Top comments (0)