DEV Community: Anguo

Deploying to 20 Rockchip boards by hand is a trap — so I built rk-deploy

Anguo — Mon, 02 Nov 2026 02:39:29 +0000

If you ship software to Rockchip boards — RK3588, RK3576, RK356x — you
know the ritual. Build a tarball. scp it to the board. SSH in, untar
it somewhere, write a systemd unit by hand, daemon-reload, restart,
tail the journal, and pray. Now do it nineteen more times. Now do it
again for the hotfix. Now one board is still running the old version
and nobody knows which.

I kept living that ritual, so I turned it into a tool:
rk-deploy — manifest in,
systemd services out, with fleet push, rollback and status reports.
v0.1.0 is out today, Apache-2.0.

The shape of it

You describe the app once:

name: demo
version: 1.1.0
runtime:
  type: python
  entry: app.py
models:
  - path: models/weights.rknn
healthcheck:
  type: file
  path: "@INSTALL_ROOT@/shared/heartbeat.json"
  timeout_sec: 15

Then:

rkdeploy pack                                        # checksummed tarball
rkdeploy deploy --fleet fleet.yaml demo-1.1.0.tar.gz # push to every board
rkdeploy rollback --fleet fleet.yaml --app demo      # back, fleet-wide
rkdeploy status --fleet fleet.yaml --app demo        # one probe per board

Rules I was not willing to compromise on:

Only what the manifest declares gets packed. Whatever else is lying in your directory — notes, stray secrets — does not travel.
Versions are immutable, switching is atomic. Each release unpacks into its own directory; going live is one symlink swap (a fresh symlink + mv -Tf — ln -sfn on a symlink-to-directory is a trap that follows the link and creates the new one inside the target).
Config and data live outside releases, in shared/, so upgrades and rollbacks never touch them.
The board stays dumb. All parsing happens on your machine. The board runs one POSIX shell installer and stock systemd — because board Python stacks vary wildly between BSPs (the RK3588 I test on has Python 3.10 and a stray PyYAML 5.4.1; the next board may have neither), the installer reads a flat rendered package.env, never YAML, never even JSON.
No agent, no new ports, no credential collection. Plain SSH with your keys, one upload + one execution per board per deploy.
A failed health check rolls back automatically, and the exit code tells you exactly which world you're in: 0 ok, 4 the package failed verification, 5 part of the fleet failed, 6 the deploy failed but the board is back on the previous version, 7 it failed and rollback failed too.

The bug my own test suite caught before any board did

The demo app writes a heartbeat file every second — the perfect
health check. Except: during an upgrade, the old version's process
is still running until the restart, happily refreshing that same
heartbeat file. An existence check would pass the brand-new version
before its process wrote a single byte.

So for file health checks, rk-deploy also requires the file to
have been written at or after this deploy's restart moment. A
one-line rule, found by a lifecycle test that runs the real board
scripts against a fake systemctl which actually starts the app —
install, upgrade, duplicate-version refusal, rollback, broken
release, auto-rollback, all of it, no hardware needed.

Real board, real numbers

Validated on an RK3588 (Ubuntu 22.04, systemd 249), driving the
shipped CLI over a tunnel — where every SSH connection costs 5–7
seconds just to establish, which is itself the best argument for the
one-connection-per-operation design:

deploy 1.0.0 → ok, 21 s (three connections: mkdir, scp, install)
status → ATTENTION — only one release installed, i.e. no rollback candidate yet. The warning is the spec's own rule firing.
deploy 1.1.0 → ok, 19 s; status → HEALTHY (SoC 49 °C)
deploy a deliberately broken 1.2.0 → health check fails, auto-rollback to 1.1.0, exit 6, 26 s
explicit rollback to 1.0.0 → ok, fleet HEALTHY

A genuine status report from that run ships in the repo under
examples/. And the board's systemctl --failed afterwards?
Identical to before the test — the tool cleans up after itself in
validation and stays out of everything it doesn't manage in
production. One related call: status judging uses only your app's
unit state, never the machine-wide is-system-running — a board can
sit at degraded for months because of some unrelated logrotate
unit while your app is perfectly healthy.

What it deliberately doesn't do

No resident agent, no central server, no web UI. No automatic
apt install — dependencies are a preflight checklist (binaries /
shared libs / python modules / packages / devices), reported
precisely, because customer boards are often offline and silently
mutating system packages on an unattended board is worse than a
clear report. No data migration on rollback — if your app changes
its on-disk format incompatibly, that's your app's problem, and the
docs say so in exactly those words, because it's precisely the thing
people assume a deploy tool does for them.

rk-deploy is the third tool in a small open family for Rockchip edge
work, next to
rk-edge-doctor
(read-only board / model / RTSP diagnostics) and
rk-bench (a measured RKNN
model leaderboard). Diagnosis, measurement — and now delivery.

If you deploy to RK boards for a living, I'd genuinely like to hear
what your ritual looks like and what would break first in your
environment: RK3576 and RK356x board reports are the most useful
thing you can file.

RK3588 INT8, measured: 1.96 on YOLOv8n, 1.4 on YOLO11n — plus OCR joins the bench

Anguo — Mon, 02 Nov 2026 02:38:40 +0000

Follow-up to the first rk-bench leaderboard: same RK3588, same frozen protocol (PERF_RUN hardware timing, per-layer CPU-fallback verdicts, median of 3 independent sessions). v0.2.0 adds the two things readers asked for.

INT8 vs FP, same models, same board

Quantized with Toolkit2 2.3.2 using the RKNN Model Zoo's own recipe and its own 20-image COCO calibration subset (each entry's notes say so):

Model	FP p50	INT8 p50	Speedup	Verdict
YOLOv8n	37.50 ms	19.17 ms	1.96×	NPU_ONLY → NPU_ONLY
YOLOv5s (ReLU)	37.80 ms	20.52 ms	1.84×	NPU_ONLY → NPU_ONLY
YOLO11n	43.67 ms	31.20 ms	1.40×	NPU_ONLY → NPU_ONLY

So on RK3588, INT8 buys you roughly 2× on the older YOLO generations and 1.4× on YOLO11n — and in all three cases the quantized graph stays entirely on the NPU. Model files roughly halve (7.25 → 4.13 MB for YOLOv8n).

OCR rows, and a second CPU stowaway

PP-OCRv4-Det (480×480): 47.86 ms, NPU_ONLY
PP-OCRv4-Rec (48×320): 22.68 ms, MIXED — one Reshape immediately after the softmax executes on the CPU: 579 µs, 2.58% of frame time.

That makes two MIXED models on the board (RetinaFace was the first), and both were caught by the per-layer verdict — neither is visible from end-to-end timing. This is exactly the failure mode a leaderboard should surface.

The bug in my own tool

Adding INT8 rows exposed a real bug in rk-bench itself: a leaderboard row's identity was model+variant, so the first INT8 report replaced its FP twin instead of sitting next to it. Identity now includes precision, there's a regression test, and the published leaderboard was rebuilt from the raw reports and verified row by row (15 rows). If you cloned v0.1.0's data, re-pull.

Also new in the tool: rkbench run --note records provenance (calibration sets, conversion quirks) in the report, so a number's paperwork travels with it.

Full table: docs/LEADERBOARD.md · Release: v0.2.0

Next open items: the RK3576 board table (seeded as a good first issue — if you have the board, the anchor models take ~10 minutes), and --core-mask measurements. Requests welcome in the bench-report issues.

Your IP camera's clock is wrong — and that's why ONVIF authentication keeps failing

Anguo — Mon, 02 Nov 2026 02:30:17 +0000

Every ONVIF integration eventually hits the same ghost: the password is
correct, the user exists, the camera answers discovery — and every
authenticated call comes back NotAuthorized. People blame the vendor,
the firmware, the client library. Most of the time, it's the clock.

ONVIF authentication is a WS-Security UsernameToken whose password digest
is Base64(SHA1(nonce + Created + password)). The device checks Created
against its own clock. A camera that has been running in a closet
without NTP for six months can be minutes — or hours — off. Your digest is
mathematically perfect and temporally unacceptable, so the camera says no.

The fix is in the spec, hiding in plain sight: GetSystemDateAndTime is
one of the few operations a device must answer without authentication,
precisely so a client can measure the offset first and sign against
device time.

I built a small tool around that lesson, and around the pile of similar
lessons ONVIF teaches you one deployment at a time.

onvif-scan

onvif-scan discovers every
ONVIF camera on a LAN and writes an asset inventory — JSON plus a
single-file HTML report:

onvifscan scan --user onvif_user

● 192.168.1.64   VendorA Cam X200   FULL ch=2 · H264 1920x1080 PTZ
● 192.168.1.65   VendorB Dome Y10   FULL ch=1 · H265 2560x1440
◐ 192.168.1.66   (unknown)          PARTIAL ch=0
— discovered 3 · full 2 · partial 1 · …

Per device it records manufacturer, model, firmware, serial, channels,
per-profile codec / resolution / frame-rate limits, PTZ presence, clock
offset, and the RTSP stream URI for each profile (password-masked,
ready to hand to a stream-acceptance checker —
rkdoctor rtsp if you use
my other tool; the two are designed as a discovery → acceptance chain).

It's pure Python standard library. No dependencies, nothing to install on
the machine you carry onto a site, and it runs offline.

What the inventory checks, beyond fields

Fields are the easy part. The useful part is the findings:

CLOCK_SKEW — device clock more than 60 s off. This is the finding that explains other people's auth failures.
UNAUTH_INFO_DISCLOSURE — GetDeviceInformation answered with no credentials at all. Plenty of older firmware does this. You want to know which cameras on a site do.
XADDR_MISMATCH — the address in the discovery reply is not the address that answered (dual NICs, NAT, an IP change the ONVIF stack didn't hear about). The tool falls back to the responder's address.
MEDIA2_ONLY / LEGACY_MEDIA_ONLY — which media generation a camera actually speaks; newer devices are dropping ver10 Media entirely.
AUTH_FAILED — your credential was rejected, once. Which brings me to the rule I'm proudest of:

A scanner must never lock you out of your own cameras

onvif-scan authenticates only with credentials you explicitly
supply — one per device from a mapping file, or one shared ONVIF user.
Each is tried exactly once. There is no default-password list, no
dictionary, no "try admin/admin just in case", and no retry loop:
several vendors lock accounts after repeated failed logins, and an
inventory tool that brute-forces its way in — however politely — would
be a liability on a real site, not a tool. Passwords are never written
anywhere; stream URIs are stored masked and the finished report is
scrubbed of every password used during the scan.

The whole tool is read-only, too: Get* queries only, no configuration
changes, no PTZ movement.

The other quirks, briefly

ONVIF is off by default on much current firmware (Hikvision ≥ 5.5.0, Axis ≥ 5.40), and the ONVIF user is usually a separate account from the web UI user. Half of all "the scanner found nothing" is this. A zero result prints a hint saying so instead of a conclusion.
Scopes are unreliable narrators. Some devices broadcast name/hardware/MAC/location; some broadcast one Profile URI and nothing else. Identity is assembled in layers, and what's missing is reported as missing — never guessed.
Firmware XML is sloppy. Unbound prefixes, missing namespaces. The parser tolerates it (and flags it) instead of dropping the device.

There's a longer list in the repo's
vendor-notes.

Honest status

v0.1, verified end-to-end against simulated devices that model strict,
standard, and legacy vendor behaviours (the simulator is in the repo and
runs in CI — including the clock-skew case both ways: signed with local
time → rejected; signed with device time → accepted). What it is not
yet is field-verified: I'm collecting real-camera reports now. If you
run it on a real fleet, the most valuable thing you can do is file a
device-report issue with the (redactable) JSON attached. That's how the
vendor matrix grows.

Repo: https://github.com/anguoyang/onvif-scan — Apache-2.0, good first
issues are up, and if it saves you an afternoon of walking a site with a
laptop and a spreadsheet, a star helps the next integrator find it.

I verified 127 Rockchip projects by hand so this list wouldn't rot — awesome-rockchip is live

Anguo — Mon, 02 Nov 2026 01:50:15 +0000

Disclosure: I maintain awesome-rockchip, and three of my own open-source projects (rk-bench, RK Edge Doctor and onvif-scan) appear in it as ordinary entries, under the same rules as everything else.

Rockchip lists rot the same way every other list does

If you develop on RK3588, you have probably landed on a Rockchip "awesome list" that looked great in a search result and turned out to be a snapshot of the ecosystem from two years ago: dead links, archived repos presented as current, and pull requests from contributors sitting unanswered for months. The content wasn't bad on day one. Nothing about a static list keeps it true on day three hundred.

I ran into the sharp end of this myself. I submitted a small, correct addition to the best-known RK3588 list for one of my tools. The list has hundreds of stars and is genuinely useful — but contributions there appear to get merged in roughly annual batches, and my PR simply sat. That's not a criticism of the maintainer; it's what happens when a list has no maintenance system, only good intentions.

So I built the list I wished existed: awesome-rockchip — 108 entries at launch, in English and Chinese, covering the whole family rather than one chip: RK3588/3588S, RK3576, RK3566/3568 and RV1106/1126.

What "verified" actually meant

I started from 127 candidates gathered from official Rockchip sources, vendor organisations, upstream projects and existing lists — then checked every single one before it earned a place:

Does the link resolve, and does it point at the project's own home rather than a mirror or re-upload?
When was the last commit or release (checked via the GitHub API, not by eyeballing)?
Is the repository archived? If so, it is either out, or in with "(archived)" printed next to its name.
Is it actually about Rockchip, or a generic project that merely mentions the chip once?

Nineteen candidates didn't survive: exact duplicates (the vendor kernel was a candidate in three different sections), generic upstreams whose Rockchip story is better told by a more specific entry, and tools that turned out to be tiny, stale, or superseded. Every surviving entry got an original one-sentence description, written from what the project does — no text was lifted from any existing list. The full verification ledger (last activity, stars, archived flag, both descriptions) is how the list was built, entry by entry.

Three findings worth knowing even if you never open the list

1. The most popular Ubuntu images for RK3588 are a frozen baseline now. Joshua Riek's ubuntu-rockchip (4,000+ stars, the images half the community started with) is archived. It still works, but anyone building on it should know it is no longer maintained. The list keeps it — honestly labelled "(archived)" — because pretending it doesn't exist would be worse.

2. The mainline story is better than most people think, and there's one page that proves it. Between Mesa's Panthor driver, upstream U-Boot booting more boards every release, and Collabora's running RK3588 mainline-status tracker, "vendor kernel forever" is increasingly a choice rather than a necessity. That tracker is in the list under OS & Kernel, and it's the single page I now check first when someone asks "does mainline support X yet?"

3. The NPU ecosystem has quietly grown a long tail. Beyond Rockchip's official RKNN Toolkit2 and RKLLM stack, the curation surfaced Go and Rust bindings for the RKNN runtime, OpenAI-shaped API servers for RKLLM, Docker images that save you from assembling the conversion environment by hand, and vision-language models (Qwen3-VL, SmolVLM2) already running entirely on the RK3588 NPU. A year ago most of that didn't exist.

The rules that are supposed to keep it alive

A list is a promise about the future, so the maintenance terms are printed at the top of the README, where breaking them is visible:

Every PR gets a first response within 48 hours — merged, commented, or changes requested. Not "merged within 48 hours"; responded to. That's the part contributors actually feel.
Links are checked automatically every week (lychee in GitHub Actions), and failures open an issue on their own.
A maintenance round runs every month: new entries in, dead links fixed or removed, newly archived projects marked.
No paid placements, ever. Entries are in on merit, in a fixed one-line format, and my own projects get no special position or wording.

Will it work? The honest answer is that the system matters more than the launch. Weekly automation removes the "I forgot to check links" failure mode, and the 48-hour pledge is cheap to keep precisely because the list is small enough to actually maintain. Check back in six months and grade it — the commit history will tell you whether the pledge held.

How to use it / contribute

Start at the section that matches your problem: Boards & Vendors if you're choosing hardware, OS & Kernel for images and boot, RKNN/RKLLM & AI Toolchain plus Models & Deployment if the NPU is your job, Multimedia for MPP/RGA/ISP work, Development Tools for flashing and diagnostics, and Community & Docs for the forums and wikis that still answer questions.

If you maintain or know a Rockchip project that belongs here, the contributing guide is one page: one line per entry, link must be live, project must be Rockchip-relevant and active within the last 12 months (or a stable official baseline). You'll hear back within 48 hours.

→ https://github.com/anguoyang/awesome-rockchip

10 RKNN models benchmarked on RK3588 — and the fastest one is cheating onto the CPU

Anguo — Mon, 02 Nov 2026 01:13:57 +0000

Picking a model for a Rockchip board usually means trusting a vendor table
or timing one run yourself and hoping. I wanted a leaderboard I could
reproduce, so I built rk-bench and measured 10 public models on an
RK3588 (driver 0.9.8, librknnrt 2.3.2) under one frozen protocol.

Three rules make the numbers worth trusting:

Latency comes from the NPU runtime's own PERF_RUN hardware counters, not wall-clock time. The board was busy (load ≈ 16) the whole time; hardware counters don't care.
Every model gets a CPU-fallback verdict from PERF_DETAIL per-layer accounting — the same method as RK Edge Doctor, with the same thresholds. Input/output staging never counts as fallback.
No number comes from a single run. More on why below; it turned out to be the most interesting part.

The leaderboard

Model	Task	p50 (ms)	p95 (ms)	FPS	Verdict	RSS Δ (MB)
RetinaFace Mobile320	face-detect	6.43	6.68	155.6	MIXED	12.9
MobileNetV2	classify	8.63	8.89	115.9	NPU_ONLY	23.9
ResNet50-v2	classify	25.40	29.34	39.4	NPU_ONLY	110.1
YOLOv8n	detect	37.50	38.58	26.7	NPU_ONLY	47.4
YOLOv8n-obb	detect-obb	37.65	38.26	26.6	NPU_ONLY	44.8
YOLOv5s (ReLU)	detect	37.80	38.58	26.5	NPU_ONLY	52.8
YOLO11n	detect	43.67	49.79	22.9	NPU_ONLY	46.1
YOLOv8n-pose	pose	49.73	51.27	20.1	NPU_ONLY	47.6
YOLOv10s	detect	88.09	90.49	11.4	NPU_ONLY	78.3
YOLOv8s-seg	segment	104.53	113.08	9.6	NPU_ONLY	112.1

All FP, official RKNN Model Zoo conversions (seven pre-converted zoo
packages; MobileNetV2 / ResNet50-v2 / RetinaFace converted from the zoo
ONNX with Toolkit2 2.3.2 using the zoo's own parameters). Full table with
model sizes and per-frame memory traffic:
docs/LEADERBOARD.md.
Raw data is one JSON file — the single source of truth the tables are
generated from.

Finding 1: the fastest model is the only one on the CPU

RetinaFace Mobile320 is the quickest model on the board — and the only
one that isn't fully on the NPU. Five Reshape operators around its
detection-head concats run on the CPU: 196 µs, 4.05% of frame time.
Cheap in absolute terms, and you'd never see it from end-to-end timing.
Verdict: MIXED. The other nine are clean NPU_ONLY, including all the
YOLO variants.

Finding 2: your first benchmark run is ~15% too slow

While validating the protocol I hit something I haven't seen documented:
the first RKNN process after an idle gap measures ~15% slower across
every single run — YOLO11n at 51.4 ms instead of 44.0 ms, 50 runs in a
row, all slow. Warmup inside that process does not fix it. Start a
second process and the speed is back.

The board uses the rknpu_ondemand governor, and the effect smells like
first-context state at the driver level; I can't prove the mechanism
from outside, but I can prove the cure, because rk-bench now does it by
default:

a priming cycle (one sacrificial load + 5 runs in a throwaway process) before measuring, and
three independent sessions (separate processes), publishing the median p50/p95 and the worst verdict. The three session p50s are printed in every entry's notes, so the spread is auditable.

This also explained a nagging discrepancy: an earlier single-process
health table (RK Edge Doctor's) had YOLO11n at 52.9 ms. It wasn't wrong —
it was measuring the first-context state. Single-process benchmarks of
RK3588 should be treated with suspicion, including, now, my own older
ones; that table has moved into rk-bench and links here.

A related trap: even primed, a single session can sit off the pace
(YOLOv8s-seg produced sessions of 104.4 / 114.3 / 104.5 ms). Median of
three is what keeps one bad minute on a busy board out of the table.

Reproduce it

Zero dependencies — Python 3.9+ standard library and ctypes against the
board's own librknnrt.so:

git clone https://github.com/anguoyang/rk-bench
cd rk-bench
PYTHONPATH=src python3 -m rkbench run yolov8n.rknn --name YOLOv8n \
  --task detect --soc rk3588 --json report.json

Got a model or a board that isn't on the table? Run it and open a
bench-report issue with the JSON — reports whose runtime/driver differ
from a board's table start a new table instead of mixing in (RK3576 is
the seeded next board). INT8 twins of the YOLO rows are the other big
open item.

If you benchmark RKNN models, I'd genuinely like to know: have you seen
the first-process effect on your board, and does your methodology defend
against it?

Disclosure: I maintain rk-bench and RK Edge Doctor; both are Apache-2.0
and exist to drive traffic to my RK integration work — the data,
however, stands on its own and every number above is reproducible from
the repository.

Before you convert that ONNX model to RKNN, check whether it can convert at all

Anguo — Sun, 01 Nov 2026 22:47:12 +0000

The standard RKNN conversion experience goes like this: set up the Toolkit environment, write the conversion script, run it, wait, read a wall of log, and discover — forty minutes in — that your model has a dynamic batch dimension, or an op the converter places badly, or an opset the toolchain doesn't love. None of those are conversion problems. They're pre-flight problems, discovered at landing time.

So I added a convert trio to RK Edge Doctor (zero-dependency, Python stdlib only). It deliberately does not convert anything — the official RKNN Toolkit2 does that. It does the three things around the conversion:

1. convert check — pre-flight. A hand-rolled ONNX structure parser (raw protobuf wire format, no onnx package) reads opset, input shapes and the op histogram, then checks:

dynamic input dimensions (fix them before quantization, not after)
control-flow operators — Loop / If / Scan cannot become a static RKNN graph, verdict LIKELY_TO_FAIL
operators known to need attention in Toolkit2 (NonMaxSuppression, Resize, GridSample, …) with the reason for each
quantization-sensitive tails (Sigmoid/Softmax at the graph end under int8)

Verdict: READY / RISKY / LIKELY_TO_FAIL.

2. convert recipe — a script worth running. Generates a ready-to-run Toolkit2 conversion script: target platform, int8 config + calibration dataset if you asked, and input_size_list pinned from the ONNX inputs themselves. You run it in your own Toolkit environment.

3. convert verify — post-conversion acceptance. The existing model diagnosis (CPU fallback list, PERF-counter latency, ONNX output drift) framed as the verification step.

First contact with a real file was humbling in the right way. I tested the checker on the official Ultralytics YOLOv8n ONNX export (PyTorch, opset 20, 233 nodes) and it immediately failed — because my parser assumed the textbook ModelProto field layout, and real PyTorch exports arrange producer/opset fields differently. Fixed by identifying fields by content, not by faith, and the same file now gets an honest verdict:

$ rkdoctor convert check yolov8n.onnx --target rk3588
+ parse: 233 nodes (opset 20, producer pytorch)
! opset: 20 (outside the commonly used 11-19 range)
! caution_ops: Resize (some modes convert with restrictions)
+ dynamic_dims: static
+ weights: 12.78 MB
RISKY

That is the correct answer for that file, and it's the answer you want before the Toolkit run, not after.

With this release the tool's planned surface is complete: board (16 hardware checks), model (CPU-fallback diagnosis), rtsp (stream acceptance), and now convert. The model health table keeps growing weekly — custom conversions that check out RISKY, or run MIXED/CPU_HEAVY, are exactly the reports I want in issues.

Your IP camera connects fine. Is the stream actually healthy?

Anguo — Sun, 01 Nov 2026 22:40:48 +0000

Every IP camera "works" when you open it in a player. The stream connects, a picture shows up, everyone moves on. Then the recorder has gaps, the analytics box misses frames, or the "live" view turns out to be a frozen picture from an hour ago — and nothing in your tooling can tell you which property of the stream is broken, because "it connects" was the only test.

I added an rtsp command to RK Edge Doctor (zero-dependency, Python stdlib only) that treats a camera stream like an acceptance test instead of a preview:

reachability and SDP negotiation (codec, payload, clock) — Basic and Digest auth supported
time to first frame after PLAY
measured frame rate, with --expect-fps / --expect-width / --expect-height assertions for acceptance runs
RTP packet loss from sequence numbers, inter-arrival jitter, bitrate
RTP timestamp health — regressions and gaps, the silent killers of recordings and analytics
an optional decode spot-check for H.264 (when OpenCV is present): reassembles the bitstream from RTP and flags black or frozen pictures

Verdict: HEALTHY / DEGRADED / BROKEN, as JSON + HTML reports you can archive as acceptance evidence.

$ rkdoctor rtsp rtsp://user:pass@cam/stream --duration 15 --expect-fps 25
+ reachability: 8 ms
+ negotiation: H264 (payload 96, clock 90000)
+ first_frame: 15 ms
+ loss: 0.000 %
+ jitter: 0.8 ms
+ fps: 25.00 fps (measured vs expected 25.00 fps)
+ timestamps: 0 regressions, 0 gaps
+ decode: 90 frames decoded (frames decode and the picture changes over time)
HEALTHY

Two implementation notes. First, there is no ffmpeg/ffprobe involved: the RTSP client is hand-rolled on stdlib sockets (Digest auth included), so the tool runs on a bare board image. Second, the password in the URL is redacted everywhere — terminal output, JSON, HTML. An acceptance tool that leaks camera credentials into its own reports would fail its own acceptance.

Verified end-to-end on an RK3588 board against a synthetic local stream (640×480 H.264 hardware-encoded on the same board, served over UDP, TCP-interleaved, and Digest auth): all three sessions HEALTHY, 0% loss, exactly 25.0 fps. That validates the client; what I really want is reports from real cameras — the weird vendors, the firmware that sends timestamps from 1970, the "25 fps" cameras that send 12. If you run it against a camera you own, the JSON report (credentials already redacted) is very welcome in an issue.

Also new this week: a model health table for the model command — five public YOLO variants measured on RK3588, all NPU_ONLY so far. Custom conversions that come out MIXED or CPU_HEAVY are the rows I actually want to add.

Is your RKNN model secretly running on the CPU? I built a tool that tells you

Anguo — Sun, 01 Nov 2026 11:38:45 +0000

Converting a model to RKNN and watching it "run" on a Rockchip board proves exactly one thing: it runs. It says nothing about where it runs. RKNN will happily execute unsupported or badly-quantized layers on the CPU, silently, while your benchmarks just look "a bit slower than expected." In my own testing I have seen a SAM-variant conversion whose decoder carried a dozen CPU ops — the model worked, the demos worked, and a chunk of it was never touching the NPU.

Until now, checking meant spelunking through runtime perf dumps by hand. So I added a model command to RK Edge Doctor (the board health checker I released earlier this month) that answers it in one run:

git clone https://github.com/anguoyang/rk-edge-doctor.git
cd rk-edge-doctor
PYTHONPATH=src python3 -m rkdoctor model model.rknn --json report.json --html report.html

Real output, from a public YOLOv8n (FP16) on my RK3588 board:

Model: yolov8n_det.rknn | Verdict: NPU_ONLY
Latency us: p50=40933 p95=41523 mean=41096.3 (runs=20)
CPU fallback operators: none

108 layers measured. Total layer time 38.2 ms; CPU time 203 µs — and every microsecond of that is the input/output staging operators, which live on the CPU by design. The report lists them separately so the tool doesn't cry wolf: staging is structural, everything else on CPU is fallback, and the verdict comes from the fallback share of time, not from op counts:

NPU_ONLY — fallback under 1% of layer time, and no single fallback op ≥ 100 µs
MIXED — fallback between 1% and 20%
CPU_HEAVY — fallback over 20% (threshold adjustable, and recorded in the report)
UNKNOWN — the tool couldn't run the model, and says why instead of guessing

How it works (and the trap that cost me an hour)

No rknnlite, no toolkit install, no pip anything. The tool binds librknnrt directly through ctypes — the runtime library is the one thing every RKNN board already has — runs warmup plus measured inferences with a zero input, and reads the runtime's own accounting: PERF_RUN for per-run duration and PERF_DETAIL for the per-layer table with each layer's target (NPU/CPU) and time.

Two details worth knowing if you ever do this yourself:

Latency is hardware time, not wall clock. The board I measured on runs production services at load average ~16. PERF_RUN comes from the runtime's counters, so p50/p95 stay meaningful on a busy board. The report still records the host load average, because honesty is kind of the theme here.
PERF_DETAIL is empty unless you ask at init time. You must pass the collect-performance flag (RKNN_FLAG_COLLECT_PERF_MASK) to rknn_init. Forget it and the query succeeds, returns zero bytes, and tells you nothing — no error, just silence. Ask me how I know.

The honest part

The official model-zoo conversions I tested (YOLOv8n, YOLOv8s-seg) come out clean: NPU_ONLY. That's expected — they are the reference conversions. The fallback problem shows up with your model: a custom architecture, an exotic op, a quantization choice the compiler didn't like. That's exactly the case where you currently have no easy way to look.

So the most valuable thing you can do with this release is not run it on a zoo model — it's run it on the weird one. If you get a MIXED or CPU_HEAVY verdict, the JSON report (which operators, how many microseconds each) is the whole story in one file; sharing it in an issue helps everyone, and reports record the model file name and hash only, never its path.

v0.2 also keeps everything from v0.1: rkdoctor board runs 16 board health checks (SoC, rknpu driver, runtime version, thermals…) and emits the same style of JSON/HTML report. Reports from boards other than RK3588 — RK3588S (Orange Pi 5, ROCK 5B), RK3566/3568, RK3576 — are still the contribution I need most.

Repo (Apache-2.0, Python stdlib only): https://github.com/anguoyang/rk-edge-doctor

Next up: v0.3 turns the same reporting machinery to RTSP camera acceptance — real frame rate, frame-gap jitter, freeze detection — the "attach it to the delivery document" test for cameras.