DEV Community: ZhengZhiCong

MiBee Steward v0.7.3: schema break, a Rust agent, and four bugs only the field could find

ZhengZhiCong — Tue, 13 Oct 2026 14:28:44 +0000

The agent went quiet for 19 minutes and systemd stayed green

On 2026-10-12 I rebuilt the database of my own main center. Nineteen minutes later I noticed one of my agents had stopped reporting. systemd was green the whole time, the timer was firing, the journal looked clean. The batches were being dropped on the floor.

I will come back to why (#554). I lead with it because this release is mostly about failures you only meet on real hardware: a config block that silently failed to write, an eBPF observer that reported success while observing nothing, a router storing its LAN address in CIDR form.

MiBee Steward is what I run to answer "what is actually on this network". Single binary, Go with Chi, SQLite through modernc so there is no CGO, SvelteKit UI embedded with go:embed. Point it at a subnet and it comes back with vendor, device type, ports, certificates, and the neighbour relationships it can prove.

License up front, since it decides whether you can use this at all: AGPL-3.0 with a commercial dual license. Fork it into a closed product or sell it as a hosted service and you need the commercial one. The fingerprint corpus under configs/fingerprints/ is separate, CC-BY-SA 4.0.

v0.7 refuses your old database, and that is deliberate

Databases from v0.6 and older are refused at startup. The binary prints what to do and exits. Databases from v0.7.0 through v0.7.2 are refused as well: v0.7.3 spans three schema generations, v4 through v7.

The reason is that I deleted the startup migration chain. Every boot used to walk a list of incremental changes, so a database could sit at any historical point and one binary had to cope with a dozen table shapes. The parallel-DDL bugs grew out of that (#328, #431, #437): two code paths each applying half of a change, neither finishing.

Now db/schema.sql is the single source of DDL, applied once at creation. If the schema version in the file does not match the binary, startup is refused, and no partial state is left to reason about.

The upgrade path is three steps:

./scripts/backup.sh
# then start the new binary against an empty data directory
./mibee-steward -config configs/config.yaml
# let it rescan; the asset table repopulates itself

The agent's local mini database follows the same policy, so scan history and SNMPv3 credentials do not carry over. The Go agent's agent.db is not migrated either.

Why the agent is now Rust

The distributed agent is Rust only now (agent-rs/, artifacts named mibee-agent-*). agent.yaml is key compatible, so nobody has to edit config.

The argument is old hardware. Anyone with a homelab has a drawer of boards that are still useful and short on RAM, and a collection daemon should not be why one gets retired. My test host was a NanoPi NEO: ARMv7, 512 MB of RAM, systemd, 6.18 kernel, watching the lan-63 segment of my center. Two days of sampling, 96 rounds:

Metric	Go	Rust
Binary size	~19 MB	~4.8 MB
Median RSS	54 MB	12 MB
RSS band	38 to 63 MB	11 to 12 MB

The Go figure already includes a round of lazy corpus compilation I did earlier; before that it sat around 110 MB steady state on that board. On a 512 MB machine, half your RAM is not something I get to spend, and a band that swings from 38 to 63 MB makes capacity planning guesswork.

The Rust side sits at 12 MB median and stayed between 11 and 12 MB for the whole two days, roughly a quarter of the Go number. That stability matters more than the median when the box also runs dnsmasq and a couple of containers. A /24 sweep on the same hardware is about 1.7x the Go throughput. Artifacts are static musl builds for amd64, arm64 and armv7, plus OpenWrt packages.

Multi-homed devices stopped ping-ponging

A router with two NICs gets reported by agents on both sides. The old identity logic treated every report as a possible move, so the device row migrated between segments on the scan cadence of whichever agent spoke last. In the UI it looked like a device that could not sit still.

Two new tables fix it. device_network_presence holds one row per (device, segment), recording which address answered there, refreshed on every report. device_sightings is append-only address history keyed by (device, IP, MAC, vantage, source); a repeat of the same tuple refreshes last_seen and see_count, a change appends a row.

The rule now: an out-of-segment report does not move the device row while a presence record from the last 24 hours is fresh, and a genuine move happens only when every other presence has expired. Every bridge apply writes a sighting with its vantage, empty for the center and otherwise the agent id.

The topology graph finally has a root

The root bridge used to be guessed by heuristics. The awkward case was the root bridge itself: standing on that device, every designated bridge around it is itself, no edge survives deduplication, and the probe returned nothing for the one device you most wanted to see.

The probe now reads dot1dStpPortRole (root, designated, alternate, backup, disabled) and dot1dStpDesignatedRoot in the same scalar Get, landing in scan_attributes.stp: own bridge MAC, root bridge MAC, is_root, and port roles by interface name. detectRoot consumes that as pass 0, ahead of every heuristic.

LLDP and CDP had a quieter failure. The center's own device row never had a MAC address, since ARP against a local address takes the kernel loopback shortcut and never puts a frame on the wire, so RecordNeighborsByMAC could not resolve the far end and frame edges were dropped without a log line. Two fixes: the center fills its own interface MACs into its own device row at wiring time and after every scan (AnchorLocalInterfaces, fills blanks only, self healing), and unknown LLDP/CDP chassis or NDP MACs get anchored with a #anchor:<mac> placeholder IP, merging by MAC priority once a real address shows up.

The interface inventory walks IF-MIB and ipAddrTable into device_interfaces: ifIndex, name, description, MAC, speed, the IPv4 answering on that port. Vanished ports keep their last sighting row frozen. Routing table edges materialise every non directly connected route whose next hop resolves to a known device, with the destination subnet in metadata; where routes share a hop, the more specific subnet wins. That edge table had no reader before this release, so rows were written and nothing rendered them.

Two Stewards, read-only federation

v0.7.3 added federation between two centers (#379). Each side fully administers its own segments, its own UI, database and scan cadence, while holding a near real time read-only profile of the peer's devices.

I held one constraint: zero new wire protocol. The peer is an embedded reporter over the same channel the Rust agent uses, so ingestion, anti-entropy state hashes, lease liveness and change events all apply unchanged. What the peer never gets: probe targets, documents, TLS chains, config backups, credentials, raw evidence, and the command channel. No scan, probe or restart can be triggered from the other side.

Read-only is a decision I made on purpose. Once a remote operator can issue commands in your segment, every bug on either side becomes a shared incident. Devices from agent or peer segments are mirrors, so roughly 15 change routes (edit, delete, bulk, systems, documents, heartbeat config) return 403, local administrators included. The UI hides edit and delete, showing a "remote network" badge instead.

peer:
  center_url: https://steward.example.internal
  auth_token: <token minted on the other side>
  agent_id: peer-basement        # defaults to peer-<hostname>
  cidr: 192.168.63.0/24          # defaults to network.cidr

Freshness got its own treatment (#556) because of the 19 minutes above. Data from an agent or peer segment older than 5 minutes turns "Remote" into an amber "Remote · stale" with the last report time and dims the row. Devices seen in several segments get a blue ring and a label (#557).

Five new passive ears

The eBPF observer moved down into the agent (#498, default-off ebpf feature). The kernel side program is the same bpf2go object file the center uses, pinned byte-identical by a drift test; the loader is aya, pure Rust, so no libbpf or BCC gets dragged in. Kernel floor is 5.8, aya takes TCX on 6.6 and above and clsact/netlink below, and it wants root or CAP_BPF plus CAP_NET_ADMIN with BTF optional. Where it cannot load, one warn line is logged and active probing continues.

ARP and IPv6 ND wire sightings (#497) catch the sleeping IoT class, devices that answer ARP and never send an IP packet, a third capture path alongside active scanning and DHCP leases. Throttling lives inside the BPF program, an LRU map with roughly one event per sender per 30 seconds: 1000 gratuitous ARP frames on the rig produced 1 event and 0 ring buffer drops. DAD probes, sender 0.0.0.0, are filtered in BPF.

Flow telemetry adds speaks-with edges (#500). Every IPv4 TCP/UDP packet lands in an LRU flow table keyed by (src, dst, proto, dport), about 16k entries by default, drained every 30 seconds. The privacy line is the part I would point at first: flows are folded into device level aggregates only, so per-port flows never leave memory and nothing that lets you reconstruct which service two hosts used is persisted. Sessions between managed devices become faint dashed edges with a byte total in the tooltip, and the first observation of a managed pair emits a speaks_with_new change event you can feed into lateral movement alerting. Retention defaults to 30 days.

WS-Discovery probing (#507) is the active channel for cameras: two Probe datagrams to 239.255.255.250:3702, a bounded 2 second window, drawing ProbeMatches out of ONVIF cameras and encoders. Responses are filtered strictly by source IP, since one multicast Probe gets answered by every WS-Discovery device on the segment and skipping that flattens the identity of a whole /24 into one blob. Parsed ONVIF identity is percent-decoded into corpus keys, so Front%20Door becomes Front Door. Two ONVIF cameras on the rig answered and landed at 0.98 fusion confidence, inferred type camera.

SNMP trap receiver (#509) lets devices speak first: linkDown, linkUp, coldStart and authenticationFailure become snmp_trap passive evidence, and coldStart plus linkUp also feed the discovery funnel. It is off until you turn it on. Port :162 needs CAP_NET_BIND_SERVICE, :1162 does not. One row per source per trap per 10 seconds stops a flapping port flooding you.

Four bugs only the field could find

My own migration is where #554 came from. The reporter treated every non-429 4xx as terminal, so a 401 marked the batch delivered and dropped it, and after the token came back the agent never retried. Now 401 goes back on the queue with a 5 minute cooldown, empty heartbeats are dropped, and the other 4xx codes (400, 403, 409) stay terminal. A side effect I did not plan: agents ride out a center database rebuild without anyone restarting them.

On iStoreOS 24.10.8 the LuCI passive discovery toggle reported "updated" while the config file did not change, and the service restarted for nothing (#515). Missing conntrack, hostapd and dns_log blocks failed silently. They are written out if absent now.

The eBPF observer had never actually worked (#492 to #494). Three independent breakages stacked up: a removed link.AttachTC, a go generate that produced nothing quietly, and a doff read the verifier rejected. It ran, it logged success, it saw nothing. TCX attachment and a CO-RE-free build that needs only clang fixed it.

One iStoreOS 24.10 box stores its LAN address as 192.168.60.1/24, and I concatenated that straight into a URL, producing http://192.168.60.1/24:8080 in the "open admin UI" link (v0.7.2). Prefix and suffix are stripped before output now. The install script had the same bug and was silently producing an empty network.cidr.

And one security fix found in review (#534): gosnmp's TrapListener does not check the community of received messages, so any host able to send UDP to the bound address could land 0.9 confidence evidence and forge coldStart or linkUp sightings. Comparison is constant time and runs before throttling, so a forged packet can no longer eat the real one's 10 second window. There is a mibee_snmp_trap_community_drops metric, and gosnmp's logger goes through a redacting adapter.

What is still missing

Upgrading hurts exactly once: back up, fresh database, rescan. The Rust agent has no router .ipk or .apk packaging yet, so on OpenWrt the documented path is a tarball plus a procd init script. SNMP trap stops at v1 and v2c; v3 trap has no config key.

The center is a single instance: SQLite has one writer, no clustering, no HA. Crossing segments means center plus agents, or federation. Federation stays read-only, which I count as a feature.

Containers are unprivileged by default, so LLDP/CDP raw frames and eBPF need either your own privileged build or the -full binary with CAP_NET_RAW.

Deliberately out of scope: alerting rules belong to Alertmanager, visualisation to Grafana, host depth to node_exporter, service layer discovery to Consul, config management to Ansible.

There are no named outside contributors on this repository yet, so I am not going to promise a response time.

Links

Repository: https://github.com/Mi-Bee-Studio/MiBeeSteward

Project page: https://www.mickeyzzc.tech/projects/mibeesteward

Docs: https://github.com/Mi-Bee-Studio/MiBeeSteward/tree/main/docs

License: AGPL-3.0 with a commercial dual license available, required for closed derivatives or hosted use; the fingerprint corpus is CC-BY-SA 4.0.

[Boost]

ZhengZhiCong — Mon, 12 Oct 2026 11:56:12 +0000

ZhengZhiCong

Oct 12

Your IP camera says it supports motion events. It might be lying

#go #onvif #iot #opensource

6 min read

Your IP camera says it supports motion events. It might be lying

ZhengZhiCong — Mon, 12 Oct 2026 11:43:56 +0000

There's a class of bug that only exists because of what hardware you happen to own.

This one was reported by a user, genelee26, against MiBeeNvr — an NVR I maintain. His report, in short:

My TP-Link camera detects motion just fine. The ONVIF event panel shows the events. But recording never triggers, and nothing is ever logged as an error.

That combination — events arriving, nothing working, no error — is usually a parsing problem wearing a disguise. It was.

Two topics that mean the same thing

ONVIF defines the events service and then more or less lets vendors pick how to express "there is motion."

The classic topic is:

tns1:VideoSource/MotionAlarm     with State = true | false

The standardized, rule-engine one is:

tns1:RuleEngine/CellMotionDetector/Motion   with IsMotion = true | false

The camera in question — a TP-Link TL-IPC44AW-COLOR (fw 1.0.13) — advertises RuleEngine/CellMotionDetector/Motion, PeopleDetector, IntrusionDetector, LineCrossDetector, TamperDetector… and no VideoSource/MotionAlarm at all. It only ever speaks the second dialect.

The control case in the same report is a Hikvision DS-IPC-T12HV3-IA (V5.7.12), which publishes both.

That control case is the whole story of why this bug survived so long. If your desk has a Hikvision on it — and most people's does — you can't reproduce it. The camera quietly does the right thing anyway. Only someone whose primary camera is a TP-Link hits it, and once you hit it, it's hard to self-diagnose: the event panel has data in it. Only the recording doesn't happen.

The fix that would have been wrong

The user filed a solid PR against MiBeeNvr itself, adding recognition in the NVR's internal/onvif/motion.go. Test-first, verified on two physical cameras. Nothing wrong with the change.

I asked him to re-target it, and I want to be precise about why, because it's the part worth generalizing.

"Which ONVIF topics count as a motion signal" is protocol semantics. It is not NVR business logic. Every single person using the ONVIF library underneath is exposed to the same trap — they subscribe successfully, the subscription works, and one vendor family silently never fires. If that knowledge lives in one NVR's internal/ directory, the next developer reads the spec again, captures again, and re-derives the same table from scratch.

He agreed, and the change became onvif-go#131. The PR body opens with:

Re-targeted from Mi-Bee-Studio/MiBeeNvr#951, as agreed there.

That single line is, in my opinion, the ideal shape of a cross-project contribution: the reporter keeps credit, the fix lands where the next person will actually find it, and the NVR's in-tree copy can be deleted instead of maintained forever.

What it looks like in the library

Two pure functions in the events package:

type MotionAlarm struct {
    Active bool
    Score  float64 // 0–100; 0 when the device does not provide one
    Source string
}

func ParseMotionAlarm(msg NotificationMessage) (MotionAlarm, bool)
func IsMotionTopic(topic string) bool

The details are all things real captures taught us:

Topic matching is case-insensitive, and every path segment's namespace prefix is dropped before comparison. tns1:, tnsalias:, and vendor-invented prefixes all compare equal. This matters more than it sounds: in a room full of captures, the same semantic topic shows up under at least three prefixes.
Exact path matching. …/MotionAlarmExtra and a truncated …/CellMotionDetector are both rejected. No prefix-containment fuzziness.
State is read from the Data slot only. IsMotion entries in Source are ignored. If the state is absent or unparseable you get (MotionAlarm{}, false) — it does not guess.
Score is an optional vendor extension. Missing or garbage → 0, and Active is unaffected.
Other rule-engine topics (Tamper, LineCross, PeopleDetector) are deliberately not folded in. Scope discipline matters more than coverage here; silently mapping tamper to "motion" would corrupt the caller's recording logic.

The test I like most is TestParseMotionAlarmFromPullMessages. It runs a captured Hikvision PullMessages response body — raw, as the camera sent it — through the library's real decode path. Because that capture happens to contain both topics and real Source entries, the test asserts on what the library actually hands a caller, not on a hand-built ideal struct. Fixtures from real hardware beat fixtures from imagination.

go test -race ./... clean, golangci-lint clean, 91.6% coverage on the events package. Merged 06:40 UTC, shipped in v2.4.0 at 10:28 the same morning.

Why "in the library" is the whole argument

This is the part I'd want a library author to take away.

If you are writing a protocol library, your real product is not your API surface — it's the accumulated table of ways real hardware deviates from the spec, encoded so the next person doesn't have to rebuild it. Every quirk you leave undocumented is a bug someone re-derives with their own captures next quarter.

So the rule I've ended up applying to this codebase: if it takes a physical device and a packet capture to discover, it belongs in a regression fixture, not in the changelog. A changelog entry says "we fixed it." A fixture says "this will never come back, and here is the exact bytes that proved it."

The same version's other three changes are the same idea in different clothes:

WS-BaseNotification push receiving — the half of the events service where the device POSTs notifications to you, instead of you long-polling. Subscribe / RenewPush / UnsubscribePush to the subscription's own manager endpoint (the address devices actually require, with a same-address fallback for firmware that collapses both roles), ErrPushNotSupported so you can probe and fall back to pull-point, and NotifyHandler as a ready consumer endpoint. Envelope shapes verified against Hikvision, Uniview, TP-Link and in-house firmware.
Typed device time sync — GetDeviceTime returns actual Go types (UTC + local wall clock, POSIX TZ, NTP/manual mode, DST flag) and goes over the unauthenticated GetSystemDateAndTime. That's the important part: a drifted camera clock is exactly the condition that breaks digest auth, so the clock is what you need to read precisely when authenticated calls are failing. Plus ErrNoCredentials so a write path with no username fails fast instead of firing a guaranteed-401 request.
Discovery listener filtering — the passive listener now drops announcements with no ONVIF signal before your handler sees them. A Windows/WSD host broadcasting a generic Hello no longer becomes a permanently-pending ghost in your device list.

All four came from production pain in one NVR. All four now belong to every consumer of the library.

The part people skip: no hardware needed to test

A protocol fix like this is only useful if the next person can verify it without buying a TP-Link.

The repo ships a complete ONVIF virtual camera server (server/, with an in-memory reference provider in server/simulator/) — multi-profile, PTZ, imaging, device IO, Media2, and pull-point events, described in camera profiles you configure, with a PublishEvent seam to inject notifications from your own code. You can stand up a fake camera, run a recorder against it, and test the event path end to end in CI.

That's also how the client-side API stays honest: the library's own events client round-trips against its own server in tests.

cfg := server.DefaultConfig()
cfg.Port = 8081
srv, _ := server.New(cfg)
log.Fatal(srv.Start(ctx)) // a fully simulated ONVIF camera

If you've ever put off writing tests for a device-integration layer because "it needs hardware," this is the escape hatch: build the fake device once, then every future bug gets a reproduction that runs on every commit.

If you're building against ONVIF in Go

go get github.com/mickeyzzc/onvif-go/v2    # Go 1.26+, zero third-party dependencies

client, _ := onvif.NewClient(endpoint,
    onvif.WithCredentials("admin", "pass"),
    onvif.WithAutoClockSkew(), // survive cameras whose clock has drifted
)

// Works whether the camera sends VideoSource/MotionAlarm
// or RuleEngine/CellMotionDetector/Motion. Or both. Or neither.
sub, _ := client.Events().SubscribeEvents(ctx, func(msg events.NotificationMessage) {
    if m, ok := events.ParseMotionAlarm(msg); ok && m.Active {
        // start recording
    }
}, nil)

A few honest notes about scope and tradeoffs, since nobody enjoys finding them out later:

This library is ONVIF signaling only. It produces and consumes stream URIs (RTSP/HTTP/UDP); it does not transport media. Pair it with an RTSP server or client for the media plane. That's a deliberate dependency choice — the module has zero third-party dependencies, which matters if this ends up vendored into firmware or a long-running recorder process.
It's the actively-developed v2 lineage of 0x524a/onvif-go — a MIT-licensed fork with the v1 tag history preserved. The upstream project is alive and maintained; the two live at different module paths (…/onvif-go/v2 vs …/onvif-go), so both can coexist in one build while you migrate service by service. Most v1 code compiles after the import-path swap; MIGRATION.md maps the rest.
Tests run on ubuntu, windows and macos, because the release artifacts are built for all three.

If you have a camera that connects, authenticates, and then behaves strangely anyway, please open an issue with the make and firmware version. Real-hardware weirdness is the most valuable thing you can hand a protocol library — the response time on these is usually a day or two, and the resolution is a fixture, not a patch note.

MiBeeNvr v0.13.0 Performance Self-Benchmark: Capacity and Resource Curves on Three Hardware Tiers

ZhengZhiCong — Sun, 27 Sep 2026 06:59:18 +0000

Choosing an NVR keeps running into the same wall: there are no public performance numbers with stated measurement conditions. Third-party cross-product benchmarks don't exist; what you can find is either vendor-quoted partial metrics (detector inference latency, say) or editorial estimate tables that omit the hardware, bitrate, and framerate conditions — the numbers can't be compared against each other.

So I measured MiBeeNvr's own capacity and resource curves: three hardware tiers, recording ramps from 16 to 68 cameras, concurrent playback on top of full load, pure forwarding, an fsync-policy comparison, plus a read-only observation of a 7×24 production machine. The methodology, pass/fail criteria, and test scripts are all published with this post. The goal is not to rank products — with no comparable public methodology, ranking means nothing — but to answer two questions that actually come up during capacity planning: how many cameras a machine of a given tier can carry, and whether the bottleneck lands on CPU, memory, or storage.

Conclusions first, data and tools below:

High-spec x86: recording 64×1080p + 4×4K at 24% total CPU (16-thread basis) and 171 MB of NVR process memory, API p50 in the low tens of milliseconds throughout; stacking 32 concurrent playback sessions (FLV / LL-HLS / RTSP) on top produced zero failures. The tier stops at 68 cameras because the traffic generator ran out of headroom, not because the NVR struggled.
Entry ARM NAS (RK3399 + eMMC-class storage): CPU sits at two tenths while the bottleneck is entirely in storage — once the volume saturates, the health endpoint degrades to 2 s, but recording, per-camera accounting, and recording queries are unaffected. Switch to 30-camera pure forwarding and CPU drops to three tenths with a perfectly clean API distribution: plenty of headroom as a live gateway.
ARM SBC production box (Banana Pi M5): 18 real cameras running 7×24, API p50 still 13.5 ms during a window with 80% disk iowait — two orders of magnitude away from the saturated eMMC tier's 2 s. Storage medium sets the experience ceiling of the low tier, and that conclusion reproduced in production.

These are vendor self-test numbers, and I'm not going to pretend otherwise: every figure's measurement conditions are in this post, the tools and criteria are public, and anyone can rerun the thing end to end.

Test Environment

Device	Role	CPU	RAM	Storage	Deployment
Banana Pi M5	Production (read-only observation)	Amlogic S905X3, ARMv8 ×4	4GB	2.7TB SATA	bare-metal systemd
RK3399 NAS	Controlled benchmark (low tier)	RK3399 ×6	4GB	eMMC-class volume	fnOS + Docker
x86 workstation	Controlled benchmark (high tier)	i5-12600KF, 10C16T	64GB	NVMe	bare metal
x86 laptop	Traffic generation / idle measurement	8-core	16GB	SSD	—

Two environment details are worth stating up front. The RK3399 NAS's storage volume is a single-member RAID1 with thin provisioning on an eMMC partition — 2× write amplification. That detail keeps resurfacing in the data below; it is the key to reading this tier. And the production machine happened to be running a historical-archive migration during the observation window, so the same device yielded both a "heavy IO stress" and a "business as usual" window.

The build under test was a single unified v0.13.0 preview build; the production observer ran a preview build from the same series.

Criteria: How It Was Measured and Counted

The four-piece load and measurement setup:

Synthetic cameras: pre-rendered MP4s pushed in real time (-re pacing) over RTMP, at two specs — 1080p25 H.264 2.5 Mbps (2 s GOP, no B-frames) and 4K25 H.264 16 Mbps. The pushers reconnect on drop, matching real camera behavior. Per-camera accounting comes from periodic snapshots of the NVR's /api/streams counters to verify each camera's ingest bitrate.
Resource sampling: on Linux, 5-second reads of raw /proc/stat counters (so iowait can be broken out), /proc/meminfo, and NVR process RSS; the Windows host sampled performance counters at the same cadence. Disk figures come from iostat.
API probe: every 5 seconds, request /api/health (unauthenticated, lightweight) and /api/recordings?limit=5 (auth + SQLite query path), recording the full latency distribution and failure count.
Playback: real pull clients on FLV / LL-HLS / RTSP watching concurrently, measuring throughput, time-to-first-byte, and live-edge latency (LL-HLS computed from the playlist's PROGRAM-DATE-TIME).

Judgment criteria: CPU includes soft interrupts, iowait is broken out separately; the "steady-state window" is the 4 minutes starting 60–75 s after the last pusher starts; API latencies in tables are percentiles over that window. Every curve is backed by 5-second raw data.

Configuration criteria: except for the fsync comparison section, everything ran on factory defaults — storage.durability at its default strict level (fsync on every finalized segment), io.budget_bytes_per_sec at 0 (background IO budget off), preallocation on. The headline numbers are therefore measured in the default posture, with no opt-in tuning switches polishing them.

Load conversion: using the community's usual MP/s dimension, 1080p25 = 51.8 MP/s per camera, 4K25 = 207.4 MP/s. This run's x86 peak of 68 cameras ≈ 4,145 MP/s; RK3399 pure forwarding of 30 cameras ≈ 1,555 MP/s. Note that MiBeeNvr records in the compressed domain without decoding, so MP/s here is a throughput dimension — comparable against generic planning methods, not equivalent to decode-style workloads.

The scripts behind this setup are collected in the appendix at the end of the post.

High-Spec x86: Recording Ramp and Concurrent Playback

From 16 cameras up to 68 (64×1080p + 4×4K), 3 minutes of steady state per tier, with per-camera ingest held precisely at 2.5/16 Mbps throughout:

Load	Total ingest	Total CPU (16T)	NVR process memory	health p50/p95	recordings p50/p95
Baseline (1 real camera)	2.0 Mbps	11.4%	57 MB	—	—
16×1080p	42.2 Mbps	12.8%	82 MB	8.5 / 15.2 ms	9.4 / 38.5 ms
32×1080p	82.3 Mbps	14.3%	105 MB	8.9 / 14.5 ms	9.0 / 14.0 ms
48×1080p	122.4 Mbps	15.3%	131 MB	9.6 / 16.7 ms	9.6 / 12.5 ms
64×1080p	162.4 Mbps	19.7%	156 MB	10.6 / 18.2 ms	10.1 / 22.7 ms
68 cameras (4×4K included)	226.5 Mbps	24.2%	171 MB	8.7 / 15.0 ms	11.1 / 27.6 ms

Both curves are near-linear: each added 1080p camera costs about 3 MB of memory, and CPU climbs roughly 2 percentage points per 40 Mbps (16-thread basis). The API probe recorded zero failures the whole way (also zero during the 32-viewer stage), and health returned to 1.2–1.9 ms once the test ended.

On top of the full 68-camera recording load, 32 concurrent playback sessions (12 FLV + 12 LL-HLS + 8 RTSP, 5 minutes):

Protocol	Result
FLV	12/12 sessions returned 200, each at a precise full 2.50–2.51 Mbps; TTFB min/p50/p95 = 65/167/256 ms
LL-HLS	12/12 streamed normally; startup min/p50/p95 = 438/1414/2095 ms; median live-edge latency 1.3 s
RTSP	8 cameras, three rounds of pulls, zero errors

Entry ARM NAS: The Bottleneck Is Storage, Not CPU

Recording Ramp (8 → 28 cameras, 20 → 124 Mbps)

Load	CPU	iowait	NVR memory	health p50	recordings p50
8×1080p (20 Mbps)	20.1%	59%	130 MB	24 ms (drifting into saturation late in the tier)	20 ms
16×1080p (40 Mbps)	20.0%	73%	162 MB	2.0 s	22 ms
24×1080p (60 Mbps)	22.6%	72%	203 MB	2.0 s	21 ms
24×1080p+4×4K (124 Mbps)	19.7%	73%	197 MB	2.0 s	22 ms

The CPU curve flattens at 20–23%, with the NVR process peaking at 41.8% of a single core. The saturation point is storage: iostat shows the volume at 100% utilization, write waits of 3.4–6.9 s, and actual writes of about 6–7 MB/s (RAID1 amplification included). Behavior after saturation is layered:

Recording and per-camera accounting are unaffected — per-camera bitrate holds and segments keep flushing;
Recording queries (the SQLite read path) are unaffected, p50 steady at 20–22 ms throughout;
The health check degrades to 2 s — internally it runs a statfs against the storage volume, which blocks when the device saturates, making it a surprisingly good "storage is gasping" indicator;
Playback under extreme saturation: LL-HLS delivered data to 8/8 clients but startup stretched to 88 s and live edge ran 35–53 s; RTSP kept streaming; FLV could not establish new sessions within 15 s (106–135 ms when healthy).

Recovery after load removal is quick: health p50 returned to 17–23 ms within the first minute, though background IO kept individual requests at 2 s for about 6 minutes, after which max < 31 ms. No crashes, no process restarts at any point.

Clean Low-Load Tier (4 cameras, 10 Mbps)

CPU	iowait	NVR memory	health p50/p95	recordings p50/p95
16.7%	44%	77 MB	24 ms / 1485 ms	21 ms / 188 ms

Even at 10 Mbps the odd statfs spike appears — this tier's bottleneck is entirely the eMMC volume itself, not CPU and not the software.

Pure Forwarding Tier (30 live cameras, no recording, 75 Mbps)

With 30 cameras set to live-only (recording disabled):

CPU	iowait	NVR memory	health p50/p95/max	recordings p50/p95
30.5%	15%	95 MB	21 ms / 40 ms / 62 ms	21 ms / 30 ms

With the write path out of the picture, the API distribution over 30 ingests plus distribution is perfectly clean; pulling FLV from a live camera gave 200, 2.52 Mbps, 135 ms TTFB. As a pure live gateway (preview / cascade / re-publish scenarios) the RK3399 has ample margin — roughly 1,555 MP/s of throughput on three tenths of the CPU.

fsync Policy Comparison: strict vs relaxed

durability controls the fsync policy for raw segments: the default strict fsyncs on every finalized segment; relaxed skips explicit fsyncs and leaves it to the filesystem commit interval (a power cut may lose the last few seconds of raw segments; merged/timelapse artifacts are unaffected). To quantify what the switch buys on slow storage, I ran a paired test at identical load and background (rolling-merge jobs running in both cases):

Tier	durability	health p50 / p95	iowait	CPU	NVR memory
16×1080p (40 Mbps)	strict	2024 / 3045 ms	72.5%	20.0%	162 MB
16×1080p (40 Mbps)	relaxed	2019 / 2027 ms	73.2%	21.1%	171 MB
24×1080p (60 Mbps)	strict (first run)	2022 / 2033 ms	72.0%	22.6%	203 MB
24×1080p (60 Mbps)	strict (paired re-run)	2020 / 2035 ms	68.4%	24.9%	217 MB
24×1080p (60 Mbps)	relaxed	2023 / 2040 ms	54.7%	31.7%	131 MB

Instantaneous write-path metrics did improve: under strict steady state the volume showed write waits of 3.4–3.9 s and constant 100% utilization; switching to relaxed brought write waits down to 0.2–0.3 s and turned utilization from "continuously saturated" to "bursty". But across the whole window, health latency and the saturation threshold did not move — this NAS's storage pressure is the sum of several sources (2× write amplification from the single-member RAID1, background bulk IO from rolling merges, the system's own background reads), and dropping recording fsyncs merely converted sustained pressure into bursty pressure, not enough to pull the device back out of saturation.

Conclusion: on eMMC-class slow storage, relaxed smooths write queuing and is a reasonable option for deployments that tolerate power-cut loss, but it does not change the capacity ceiling of such storage; on NVMe-class fast storage the default tier is never the bottleneck, so there is no need to flip it. This section also confirms that the headline numbers were all measured at the default strict level, with no help from this switch.

A footnote: during the comparison, one docker restart of the container hit a runtime hang (docker ps reported Up while the process was gone; stop/start recovered it; the log showed no panic before exit and the kernel logged no OOM — a host-environment event, not an NVR crash). SQLite's unchecked-pointed stream keys were lost as a result (cameras and recordings intact); I recreated them and continued. In production, restart via the normal mechanisms (systemd / app store) rather than docker restart.

Production SBC: Two Observation Windows

A Banana Pi M5 (S905X3 ×4 / 4GB / 2.7TB SATA) runs 18 real cameras (a mix of ONVIF / GB28181 / Xiaomi / MJPEG) 7×24, with 2.1 TB of recordings on disk (a single camera at up to 467 GB / 2014 segments). Read-only observation, no changes made:

Metric	Calm window (14 min)	Heavy-IO window (46 min)¹
health p50 / p95 / max	12.2 / 25.3 / 1015 ms	13.5 / 53 / 2232 ms
recordings p50 / p95	252 / 306 ms	253 / 338 ms
Ingest bitrate / active cameras	16.1 Mbps / 12	17.9 Mbps / 12
NVR process RSS (min/avg/max)	182 / 267 / 414 MB	114 / 194 / 412 MB
SoC temperature (avg / max)	51.2 / 54.5 ℃	52.6 / 58.3 ℃
Total CPU (incl. iowait conversion)	20.1%	34.5% (mostly iowait)

¹ The machine was migrating its historical archive during the stress window, with disk iowait around 80%; the NVR process itself used about half a core, and recording continued.

The same SATA device held health p50 at 13.5 ms under 80% iowait, versus 2 s for the eMMC volume under comparable pressure earlier — two orders of magnitude apart. The storage medium sets the experience ceiling of the low tier, and that conclusion reproduced in production. The 250 ms scale of recordings queries matches its 2.1 TB / tens-of-thousands-of-rows library.

Boundaries and Reproducing

Not covered: software transcoding scale (H.265→H.264 software transcoding is CPU-bound; production should avoid it by recording sub-streams), WebRTC end-to-end latency (needs browser-side measurement), 10 GbE, cloud-archive throughput, audio tracks, and the io.budget_bytes_per_sec background IO budget (off by default; background merge load was a constant during the test window, not A/B tested separately). Synthetic streams are video-only.

Reproducing: the tools are the appendix scripts — synthetic camera sources, the pusher fleet, resource samplers, API probe, per-camera snapshots, the three-protocol pull clients, and the ramp orchestration — pointed at any target host via environment variables. A same-hardware cross-product comparison (say, Frigate or ZoneMinder on the same box under the same load) has no third-party public data either; it would be worth doing, but first someone has to put every product under the same methodology.

Repository: Mi-Bee-Studio/MiBeeNvr.

Appendix: The Test Harness

The four-piece setup from the methodology section lands as the scripts below. The original test kit hardcoded LAN addresses, passwords, and machine-specific choreography; before publishing I did two things to all of it: sanitized (addresses and credentials now come from environment variables) and generalized (process names, durations, camera counts, and ramp steps are all parameters — nothing is tied to a specific machine). The tools only do measurement; the product-specific parts reduce to two strings, the API paths and the process name. The pusher and pull clients work against any RTMP / HTTP-FLV / LL-HLS service as-is.

Deployment: the traffic generator runs the pusher fleet and the pull clients; the device under test runs the resource sampler; the API probe can sit on either side. Render the synthetic sources once and reuse them across runs.

Synthetic Camera Sources

testsrc2 renders video-only sources with a 2 s GOP and no B-frames, specced like a real IPC main stream:

#!/bin/bash
# render-cam.sh — render one synthetic camera source (video-only)
# usage: render-cam.sh <WxH> <fps> <bitrate_k> <duration_s> <out.mp4> [profile]
set -e
RES=${1:-1920x1080}; FPS=${2:-25}; BR=${3:-2500}; T=${4:-1800}; OUT=${5:?}; PROF=${6:-main}
ffmpeg -y -loglevel error -f lavfi -i "testsrc2=size=${RES}:rate=${FPS}" -t "$T" \
  -c:v libx264 -preset veryfast -profile:v "$PROF" -b:v "${BR}k" \
  -maxrate $(( BR * 12 / 10 ))k -bufsize $(( BR * 2 ))k \
  -g $(( FPS * 2 )) -bf 0 -pix_fmt yuv420p "$OUT"

# One source per spec (4K uses the high profile):
bash render-cam.sh 1920x1080 25 2500 1800 cam1080.mp4 main
bash render-cam.sh 3840x2160 25 16000 600  cam4k.mp4 high

Pusher Fleet

A single pusher is a reconnect loop — real cameras re-push after a network blip, so the load model can't skip that; the fleet script handles batch start/stop and counting:

#!/bin/bash
# pusher-loop.sh — one pusher: reconnects 2 s after a drop, mimicking a real camera
# usage: pusher-loop.sh <rtmp base> <media file> <stream key>
BASE=$1; FILE=$2; KEY=$3
while true; do
  ffmpeg -loglevel error -re -i "$FILE" -c copy -f flv "$BASE/$KEY" >> "pusher_${KEY}.log" 2>&1
  sleep 2
done

#!/bin/bash
# pushers.sh — pusher fleet: start <rtmp base> <file> <key...> | stop | count
MODE=$1; PIDF=pushers.pid
case $MODE in
start)
  BASE=$2; FILE=$3; shift 3
  for KEY in "$@"; do
    nohup bash pusher-loop.sh "$BASE" "$FILE" "$KEY" >/dev/null 2>&1 &
    echo $! >> "$PIDF"
    sleep 0.4            # stagger startups to avoid a connect storm
  done
  echo "started=$#";;
stop)
  pkill -f pusher-loop.sh 2>/dev/null
  [ -f "$PIDF" ] && { kill $(cat "$PIDF") 2>/dev/null; rm -f "$PIDF"; }
  sleep 1
  pkill -9 -f 'rtmp://.*/live/' 2>/dev/null   # catch leftover ffmpeg; don't run this near unrelated ffmpeg jobs
  echo stopped;;
count)
  pgrep -fc 'rtmp://.*/live/' || echo 0;;
esac

Resource Samplers

Reading raw /proc counters instead of pre-computed percentages is what lets iowait be broken out — the bottleneck calls on the low tier hinge on it. The process name is a parameter, so it samples any service:

#!/bin/sh
# sampler.sh — target-host resource sampler: raw counters, one line every 5 s by default
# usage: sampler.sh <out.csv> [interval_s] [process name]
OUT=${1:?}; INT=${2:-5}; PROC=${3:-mibee-nvr}
echo "ts,user,nice,sys,idle,iowait,irq,softirq,steal,mem_avail_kb,proc_rss_kb,pgpgout" > "$OUT"
while true; do
  sleep "$INT"
  C=$(awk '/^cpu /{print $2","$3","$4","$5","$6","$7","$8","$9}' /proc/stat)
  MA=$(awk '/MemAvailable/{print $2}' /proc/meminfo)
  PID=$(pgrep -x "$PROC" | head -1)
  [ -z "$PID" ] && PID=$(pgrep -f "$PROC" | head -1)
  RSS=$(awk '/VmRSS/{print $2}' "/proc/$PID/status" 2>/dev/null)
  PG=$(awk '/pgpgout/{print $2}' /proc/vmstat)
  echo "$(date +%s),$C,$MA,${RSS:--1},$PG" >> "$OUT"
done

Windows targets get the performance-counter variant:

# sampler.ps1 — Windows target sampler, same idea as the Linux one
param([string]$Out = "sample.csv", [int]$Interval = 5, [string]$ProcName = "mibee-nvr")
"ts,cpu_busy,mem_avail_mb,proc_ws_mb,disk_w_mb_s" | Out-File $Out -Encoding utf8
while ($true) {
    Start-Sleep $Interval
    try {
        $cpu = (Get-Counter '\Processor(_Total)\% Processor Time' -SampleInterval 1 -MaxSamples 1).CounterSamples.CookedValue
        $os = Get-CimInstance Win32_OperatingSystem
        $p  = Get-Process $ProcName -ErrorAction SilentlyContinue | Select-Object -First 1
        $ws = if ($p) { [math]::Round($p.WorkingSet64 / 1MB, 1) } else { -1 }
        $dw = (Get-Counter '\PhysicalDisk(_Total)\Disk Write Bytes/sec' -SampleInterval 1 -MaxSamples 1).CounterSamples.CookedValue / 1MB
        Add-Content $Out ("{0},{1:N1},{2},{3},{4:N1}" -f [DateTimeOffset]::UtcNow.ToUnixTimeSeconds(), $cpu, [math]::Round($os.FreePhysicalMemory / 1KB), $ws, $dw)
    } catch { }
}

API Probe

The two paths mean different things: health is unauthenticated and lightweight; recordings carries auth plus a SQLite query — a real read path. The probe only logs timestamps and status codes; the latency distribution is post-processing's job:

#!/usr/bin/env python3
"""apiprobe.py — API latency probe
usage: NVR_API=http://<host>:<port> NVR_USER=... NVR_PASS=... python3 apiprobe.py <out.csv> [interval_s]"""
import os, sys, time, base64, urllib.request

base = os.environ["NVR_API"].rstrip("/")
basic = base64.b64encode(f'{os.environ.get("NVR_USER", "")}:{os.environ.get("NVR_PASS", "")}'.encode()).decode()
out = sys.argv[1]
interval = float(sys.argv[2]) if len(sys.argv) > 2 else 5

def timed_get(path, auth):
    req = urllib.request.Request(base + path)
    if auth:
        req.add_header("Authorization", "Basic " + basic)
    t0 = time.perf_counter()
    try:
        with urllib.request.urlopen(req, timeout=10) as r:
            r.read(4096)
            code = r.status
    except Exception as e:
        code = getattr(e, "code", -1)
    return (time.perf_counter() - t0) * 1000, code

f = open(out, "a", buffering=1)
f.write("ts,health_ms,health_code,rec_ms,rec_code\n")
while True:
    ts = int(time.time())
    h = timed_get("/api/health", False)
    r = timed_get("/api/recordings?limit=5", True)
    f.write(f"{ts},{h[0]:.1f},{h[1]},{r[0]:.1f},{r[1]}\n")
    time.sleep(interval)

Per-Camera Snapshots

Periodically recording the /api/streams counters and differencing adjacent snapshots yields each camera's actual ingest bitrate — what the pusher claims doesn't count; what the server receives does:

#!/usr/bin/env python3
"""statsnap.py — per-camera accounting snapshots
usage: NVR_API=... NVR_USER=... NVR_PASS=... python3 statsnap.py <out.jsonl> [interval_s]"""
import os, sys, time, json, base64, urllib.request

base = os.environ["NVR_API"].rstrip("/")
basic = base64.b64encode(f'{os.environ.get("NVR_USER", "")}:{os.environ.get("NVR_PASS", "")}'.encode()).decode()
out = sys.argv[1]
interval = float(sys.argv[2]) if len(sys.argv) > 2 else 30

def get(p):
    req = urllib.request.Request(base + p)
    req.add_header("Authorization", "Basic " + basic)
    with urllib.request.urlopen(req, timeout=15) as r:
        return json.loads(r.read())

f = open(out, "a", buffering=1)
while True:
    rec = {"ts": int(time.time())}
    try:
        rec["streams"] = get("/api/streams")
    except Exception as e:
        rec["streams"] = {"err": str(e)}
    f.write(json.dumps(rec) + "\n")
    time.sleep(interval)

Pull Clients

The FLV client is small: throughput and time-to-first-byte. The HLS client does more — it follows the master playlist to the variant and computes live-edge latency from PROGRAM-DATE-TIME. Both scripts strip the query string before writing URLs to CSV, so stream tokens never leak into the data files:

#!/usr/bin/env python3
"""flvpull.py — FLV pull client: TTFB, throughput, status code
usage: python3 flvpull.py <stream.flv URL> <out.csv> <duration_s>"""
import sys, time, urllib.request

url, out, dur = sys.argv[1], sys.argv[2], float(sys.argv[3])
t0 = time.perf_counter()
n, ttfb, code = 0, None, 200
try:
    with urllib.request.urlopen(url, timeout=15) as r:
        code = r.status
        while time.perf_counter() - t0 < dur:
            b = r.read(65536)
            if not b:
                break
            if ttfb is None:
                ttfb = (time.perf_counter() - t0) * 1000
            n += len(b)
except Exception as e:
    code = getattr(e, "code", -1)
el = time.perf_counter() - t0
mbps = n * 8 / el / 1e6 if el > 0 else 0
with open(out, "a") as f:
    f.write(f'{url.split("?")[0]},{ttfb if ttfb is not None else -1:.0f},{n},{el:.1f},{mbps:.2f},{code}\n')
print(f"flv code={code} bytes={n} ttfb={ttfb and round(ttfb)} mbps={mbps:.2f}")

#!/usr/bin/env python3
"""hlspull.py — HLS/LL-HLS pull client: startup time, throughput, live-edge latency (PDT)
usage: python3 hlspull.py <index.m3u8 URL> <out.csv> <duration_s>"""
import sys, time, re, urllib.request
from datetime import datetime, timezone

base, out, dur = sys.argv[1], sys.argv[2], float(sys.argv[3])
t0 = time.perf_counter()
seen = set()
n = 0
startup = None
lags = []
pdt_re = re.compile(r"^#EXT-X-PROGRAM-DATE-TIME:(.+)$", re.M)
var_re = re.compile(r"^#EXT-X-STREAM-INF:", re.M)
url = base
while time.perf_counter() - t0 < dur:
    try:
        with urllib.request.urlopen(url, timeout=5) as r:
            txt = r.read().decode()
        if startup is None:
            startup = (time.perf_counter() - t0) * 1000
        if var_re.search(txt):          # master playlist -> follow to the variant
            lines = txt.splitlines()
            for i, ln in enumerate(lines):
                if ln.startswith("#EXT-X-STREAM-INF"):
                    q = "?" + url.split("?", 1)[1] if "?" in url else ""
                    url = url.rsplit("/", 1)[0] + "/" + lines[i + 1].split("?")[0] + q
                    break
            continue
        m = pdt_re.search(txt)          # live edge = local clock - PDT
        if m:
            try:
                pdt = datetime.fromisoformat(m.group(1).replace("Z", "+00:00"))
                lag = (datetime.now(timezone.utc) - pdt).total_seconds()
                if -5 < lag < 120:
                    lags.append(lag * 1000)
            except Exception:
                pass
        for ln in txt.splitlines():
            if ln and not ln.startswith("#"):
                s = ln.split("?")[0]
                if s not in seen:
                    seen.add(s)
                    u = url.rsplit("/", 1)[0] + "/" + ln
                    try:
                        with urllib.request.urlopen(u, timeout=10) as r2:
                            n += len(r2.read())
                    except Exception:
                        pass
    except Exception:
        pass
    time.sleep(0.4)
el = time.perf_counter() - t0
mbps = n * 8 / el / 1e6
lag_avg = sum(lags) / len(lags) if lags else -1
with open(out, "a") as f:
    f.write(f'{base.split("?")[0]},{startup if startup is not None else -1:.0f},{n},{len(seen)},{mbps:.2f},{lag_avg:.0f}\n')
print(f"hls segs={len(seen)} bytes={n} startup={startup and round(startup)} mbps={mbps:.2f} live_lag_avg_ms={lag_avg:.0f}")

Ramp Orchestration

The recording ramp is data-driven: edit the TIERS array to change the ramp, and the script itself knows nothing about any particular machine. Each tier pushes, climbs for 60 s, holds steady for 180 s, snapshots per-camera accounting, and logs event timestamps into marks.txt for post-processing:

#!/bin/bash
# ramp.sh — recording ramp orchestration
# usage: NVR_API=http://<host>:<port> RTMP=rtmp://<host>:1935/live \
#        NVR_USER=... NVR_PASS=... bash ramp.sh
set -e
: "${NVR_API:?} ${RTMP:?} ${NVR_USER:?} ${NVR_PASS:?}"
TIERS=("cam1080.mp4:16" "cam1080.mp4:16" "cam1080.mp4:16" "cam4k.mp4:4")
M=marks.txt
mark() { echo "$1 $(date +%s)" >> "$M"; }
snap() { curl -s -m 30 -u "$NVR_USER:$NVR_PASS" "$NVR_API/api/streams" > "streams-$(date +%s).json"; }
mark RAMP_START
n=0
for tier in "${TIERS[@]}"; do
  n=$((n + 1)); FILE=${tier%%:*}; CNT=${tier##*:}
  keys=$(seq 1 "$CNT" | sed "s/^/load-${FILE%.*}-n${n}-/")
  mark "S${n}_start"
  bash pushers.sh start "$RTMP" "$FILE" $keys
  sleep 60;  mark "S${n}_steady"
  sleep 180; snap; mark "S${n}_end"
done
mark RAMP_DONE

Stacking concurrent playback on top of full load works the same way: resolve camera IDs from /api/cameras by name, hand the ID list to the pull scripts, start a batch each of FLV and LL-HLS, and spot-check RTSP with ffmpeg:

#!/bin/bash
# egress.sh — concurrent playback stacked on full recording load
# usage: NVR_API=... NVR_TOKEN=... RTSP_BASE=rtsp://<host>:8554 \
#        bash egress.sh "<camera ID list, space-separated>" [duration_s]
IDS=${1:?"camera ID list, resolved from /api/cameras by name"}; DUR=${2:-300}
: "${NVR_API:?} ${NVR_TOKEN:?} ${RTSP_BASE:?}"
: > egress-flv.csv; : > egress-hls.csv
for id in $IDS; do
  nohup python3 flvpull.py "$NVR_API/api/cameras/$id/stream.flv?token=$NVR_TOKEN" egress-flv.csv "$DUR" >/dev/null 2>&1 &
done
for id in $IDS; do
  nohup python3 hlspull.py "$NVR_API/api/cameras/$id/stream/index.m3u8?token=$NVR_TOKEN" egress-hls.csv "$DUR" >/dev/null 2>&1 &
done
for id in $(echo "$IDS" | tr ' ' '\n' | head -8); do   # RTSP spot-check on the first 8
  ffmpeg -loglevel error -rtsp_transport tcp -i "$RTSP_BASE/$id" -t 10 -c copy -f null - >> egress-rtsp.log 2>&1 &
done
wait
echo egress-done

Idle footprint and cold start get their own measurement: start a zero-camera instance with a minimal config, time from process start until health responds, several rounds in a row with the data directory wiped between rounds:

#!/bin/bash
# startup.sh — cold start to API-ready latency, several consecutive trials
# usage: BIN=./mibee-nvr READY_URL=http://127.0.0.1:9199/api/health bash startup.sh
BIN=${BIN:?}; READY_URL=${READY_URL:?}
for i in $(seq 1 ${TRIALS:-10}); do
  rm -rf data db.sqlite*
  S=$(date +%s%N)
  "$BIN" -config config.yaml > nvr.log 2>&1 &
  PID=$!
  for t in $(seq 1 600); do
    curl -s -m 1 -o /dev/null "$READY_URL" && break
    sleep 0.05
  done
  E=$(date +%s%N)
  echo "trial $i: $(( (E - S) / 1000000 )) ms"
  kill $PID 2>/dev/null; wait $PID 2>/dev/null || true
done

Post-processing has little to explain: the raw data is 5-second CSV/JSONL plus event timestamps in marks.txt; window statistics are percentile queries over timestamp slices, and the charts are matplotlib. Those scripts couple tightly to the data formats so I'll skip them — the idea is exactly the two clauses in this paragraph. Cleanup after a run is a pusher stop plus deleting cameras by name prefix — ordinary API calls, not worth the space.

MiBee Eye Notebook: turn a spare laptop into a network camera, microphone included

ZhengZhiCong — Wed, 16 Sep 2026 13:32:25 +0000

MiBee Eye has had two implementations so far, both aimed at Raspberry Pi-class boards. This is the new one: laptop, desktop and mini-PC support, capturing the camera and microphone attached to the machine and presenting them as a device your existing NVR can pick up.

The laptop in the drawer

Most of us have one: a laptop that's too old to be a daily driver and too good to throw away. It has a webcam, a microphone, a NIC and a disk. What it doesn't have is a reason to be switched on.

At the same time, buying a camera means accepting the usual deal — someone else's cloud gets the video, a subscription unlocks the useful parts, and the firmware stops when the product line does.

MiBee Eye Notebook is the middle layer for the spare machine. It runs on a desktop, laptop or mini-PC, captures the camera and microphone physically attached to that machine, and exposes them as a standard device an NVR can discover. Day to day you drive it from a browser.

What it is, and what it deliberately isn't

The scope is narrow on purpose, and the boundaries are written down as definitional rather than as engineering decisions someone might revisit.

It is a local capture agent. It runs as a system service. It captures USB webcams, built-in laptop cameras, and USB or built-in microphones from the machine it's installed on.

It is not a network camera scanner. It never discovers, scans or pulls RTSP/ONVIF streams from remote cameras — if the camera isn't plugged into this machine, it's out of scope. It isn't an NVR: it doesn't manage or record third-party cameras. It isn't a cloud service, and it's designed to run fully air-gapped. It isn't multi-tenant: one admin account, no per-user ACLs.

Inside the machine:

flowchart TD
    CAM["Camera<br/>USB or built-in"] --> CAP["Capture<br/>V4L2 read"]
    CAP --> ENC["H.264 encode<br/>openh264"]
    MIC["Microphone<br/>USB or built-in"] --> AUD["Audio encode<br/>G.711 or AAC"]
    ENC --> HUB["Encoded frame hub"]
    HUB --> LOCAL["Local output"]
    LOCAL --> PRE["Browser preview"]
    LOCAL --> REC["MP4 recordings"]
    HUB --> PULL["Pull from outside"]
    PULL --> RTSP["RTSP server"]
    PULL --> OV["ONVIF device"]
    HUB --> PUSH["Push to outside"]
    PUSH --> RTMP["RTMP push"]
    PUSH --> GB["GB28181 device"]
    AUD --> REC

Default ports: web UI 8443 (TLS), RTSP 8554, RTMP 1935, SIP 5060.

It's part of the MiBee Eye family alongside mibee-eye-go and mibee-eye-rs, which target Raspberry Pi-class boards. Same protocols, same SPEC v1 web API, same NVRs. Pick by the machine you have: a board with a CSI camera or a USB camera goes to one of those; a PC or laptop goes here.

Your microphone is a first-class input

This is the part the board-based implementations don't do, and it's what makes repurposing a laptop interesting.

The microphone path forks into two encoders for two purposes:

flowchart TD
    MIC["Microphone"] --> PCM["PCM capture"]
    PCM --> G711["G.711 encode"]
    PCM --> AAC["AAC encode<br/>optional feature"]
    G711 --> GB["GB28181 stream"]
    AAC --> REC["MP4 recordings"]
    AAC --> RTMP["RTMP push"]
    PLAT["Platform talkback"] --> DEC["G.711 decode"]
    DEC --> SPK["Local speaker"]

G.711 μ-law is the default: pure Rust, used for the GB28181 live stream and for decoding platform talkback. AAC covers MP4 recordings and RTMP push, but it sits behind a compile-time feature that's off by default, because it pulls in an external encoder library.

Talkback runs the other direction too. When a platform opens an audio-only INVITE, the device decodes G.711 and plays it out of the local speaker. That's on by default. If it's disabled in config, or the machine has no usable output device, those INVITEs get a 488 — not a 200 OK with the audio quietly dropped. That preference for an honest refusal over a fake success shows up in a few places in this codebase.

Recordings are rolling MP4 segments: 900 seconds by default, a 10,240 MB capacity cap, oldest-first pruning, and only files the application wrote are ever deleted. Segment rotation happens on keyframe boundaries, so you don't get half-written files.

Everything outbound is off until you turn it on

The default posture is worth calling out because it's unusual. RTSP server, RTMP push, ONVIF device, GB28181 device — all four ship disabled. You enable what you need from the web UI, per protocol, and the state lives in SQLite so it survives a restart.

The rest of the security posture:

TLS is mandatory on the web UI. rustls only, no HTTP fallback, not even on localhost. A self-signed dev certificate is generated on first run.
Admin credentials are bcrypt-hashed. Session cookies last 24 hours with HttpOnly; Secure; SameSite=Strict.
Per-IP rate limiting on auth endpoints, 20 requests per 60 seconds by default, reset on successful login, plus a per-user lockout with exponential backoff after five failures.
Double-submit CSRF on state-changing requests, and a strict CSP header.
The only anonymous endpoints are /health and /metrics. Everything else needs a session.
Two-factor auth and mutual TLS are explicitly out of scope for v1.

That last point is the real limitation: if you expose this to the internet, a strong password over TLS is the whole authentication story. It's built as a LAN device, and it should be deployed like one.

The only interface is the browser

The web UI isn't a convenience wrapper around a config file — it's the primary surface, and the CLI is the escape hatch.

Live preview picks its transport from what the device and browser support: MSE first (fragmented MP4 over chunked HTTP), falling back to MJPEG in an <img>, then snapshot polling.

flowchart TD
    OPEN["Open live view"] --> P1["MSE over HTTP"]
    P1 --> P2["MJPEG fallback"]
    P2 --> P3["Snapshot polling"]

The MSE engine has a stall timer, exponential backoff on reconnect, active SourceBuffer pruning to stay near the live edge, and a playhead watchdog that reconnects if currentTime stops advancing for about eight seconds — because a stuck decoder keeps bytes flowing while the picture is frozen, and a byte-level stall timer never notices.

Frames for the preview and the snapshot endpoint come from a JPEG tap maintained inside the capture loop, not from a transcoder. For MJPG cameras the raw JPEG bytes are forwarded at no cost; for YUYV-only cameras a JPEG is re-encoded every Nth frame. Snapshot and preview both read that tap.

No ffmpeg, no subprocesses

Earlier versions shelled out to ffmpeg for five different jobs. All five are now in-process Rust:

Former ffmpeg job	Replacement
H.264 encode	openh264 (Cisco, BSD-2, built from source)
MJPEG decode	`jpeg-decoder`
YUYV to JPEG preview	`jpeg-encoder`
Audio encode	G.711 μ-law, or optional FDK-AAC
MP4 segment muxing	`muxide`, a pure-Rust fMP4 muxer

Two consequences worth stating plainly: ffmpeg is no longer a runtime dependency (the container image, the systemd unit and the bare-metal install steps all dropped it), and the encoding pipeline is crash-recoverable without process supervision. The project's own changelog puts the saving at roughly 64 MB of subprocess overhead per camera. That's the project's figure, not a third-party benchmark, and I haven't reproduced it.

Small things that matter on a machine you also use

Protocol hot-toggle. Turning ONVIF, GB28181 or RTMP on and off from the web UI doesn't restart the service.
Hot-plug. A udev listener watches for camera add/remove events, so plugging in a webcam makes it appear, and unplugging it marks the camera offline and shuts its recording output down cleanly.
SSE event stream. Camera online/offline events reach the browser without a refresh.
Observability. 14+ Prometheus metrics, OpenTelemetry tracing wired through 132 annotated spans, structured JSON logs, and an optional Loki-compatible log shipper that fails open — a missing backend is a warning, not a crash.

Platform reality

Linux x86_64 and aarch64 are the only targets that compile today. Windows and macOS are planned and don't build yet. The stated blockers are concrete: libc::getifaddrs is POSIX-only, parts of the capture code have #[cfg(unix)] without a Windows branch, and device paths are hardcoded as /dev/videoN. macOS would need AVFoundation-aware device enumeration on top of that.

That matters because the obvious user for this product is someone with a spare Windows laptop, and today they can't run it.

Other things to know before you build a plan on it:

No releases yet. No tags, no GitHub releases, the changelog is entirely under Unreleased at version 0.1.0. If you want it, you compile it.
The README lags the code in two places. Its per-crate line-count table is stale, and the browser-preview row still describes ffmpeg transcoding, which was removed — MSE is the current primary path.
The resource figures are targets, not measurements. Idle CPU under 5%, memory under 200 MB, first frame under 500 ms, at most 16 concurrent streams. There's no published measurement to check them against, which is different from the board-based siblings, whose READMEs do carry measured numbers with their conditions attached.
One authentication factor. A strong password. No 2FA, no mTLS, no per-user ACLs.
No cloud, no mobile app, no push notifications. By design, but it's a gap if that's what you wanted.
It's brand new. First pushed on 2026-09-16, one star. I won't make claims about maintenance cadence or response times.
The toolchain requirement is Rust 1.88 with edition 2024, which is on the new side.

The protocol layer is the same two Rust libraries

The ONVIF device service is onvif-device-rs; the GB28181 device stack, including GB 35114 A-level, is gb28181-rs. Both are MIT, both are maintained as standalone libraries, and both can be used without this application if you're building an NVR, a platform, or a device simulator. The signal-plane code is hand-written in this repo; the protocol semantics live in those two crates.

Getting started

# System dependencies (Linux)
sudo apt install libv4l-dev libasound2-dev libclang-dev
sudo usermod -aG video $USER     # reclaim webcam access; requires a fresh login

git clone https://github.com/xiqing85/mibee-eye-notebook
cd mibee-eye-notebook
cargo build --release

cp config.toml config.local.toml   # gitignored; keep local overrides here

cargo run --release -- --config config.local.toml

Then open https://<host>:8443. First run walks you through creating the admin account and generates the self-signed certificate, so the browser warning is expected.

Three config gotchas: [web] advertised_host left empty auto-detects the LAN IP with a UDP probe, which picks wrong on multi-NIC or NAT'd hosts and produces stream URLs clients can't reach. [capture] video_device defaults to /dev/video0, so check the device number on a multi-camera machine. And if you lose the password, --reset-password is the way back in.

Project links

mibee-eye-notebook: https://github.com/xiqing85/mibee-eye-notebook

The Raspberry Pi-class implementations in the same family:

mibee-eye-go: https://github.com/xiqing85/mibee-eye-go

mibee-eye-rs: https://github.com/xiqing85/mibee-eye-rs

The protocol libraries underneath:

onvif-rs: https://github.com/mickeyzzc/onvif-rs (published on crates.io as onvif-device-rs)

gb28181-rs: https://github.com/mickeyzzc/gb28181-rs

Licence: Apache-2.0 for the capture agent. The two protocol libraries are MIT.

MiBee Eye: turn a spare Linux board into an ONVIF camera your NVR already trusts

ZhengZhiCong — Tue, 15 Sep 2026 19:12:30 +0000

The camera you already own, and the bill attached to it

Most consumer cameras sold today work like this: the video goes to a company's servers, the useful features sit behind a monthly plan, and the device stops getting firmware the moment the product line is retired. You bought the hardware, but the footage and the feature set aren't really yours.

The alternative used to mean buying into an ecosystem anyway — a vendor NVR, a vendor app, a vendor account. What's actually missing is the boring middle layer: something that takes a camera sensor on a board you already own and presents it to the software you already run, using the protocols that software already speaks.

That's what MiBee Eye is. It turns any Linux board into an ONVIF Profile S device with RTSP streaming, and for installations that need it, a GB/T 28181 device. There are two independent implementations, Go and Rust. No account, no cloud round-trip, no subscription. Two terminal commands and a systemd unit.

What it presents to the rest of your network

Port	What's there
8080	ONVIF device service (Device / Media / Imaging) + WS-Discovery
8554	RTSP, H.264 over TCP interleaved or UDP
8088	Embedded admin UI, English and Chinese

ONVIF Profile S device. Device, Media and Imaging SOAP services plus WS-Discovery multicast announcements, so an NVR finds it without a manual address entry. Against an ONVIF NVR it looks like an ordinary camera.
GB/T 28181 device (optional path). SIP registration over UDP or TCP with digest auth, catalog and device-info queries, live/playback/download PS streams, SIP INFO playback control, platform snapshot commands. Relevant if you're integrating with a Chinese standards-based video platform rather than an ONVIF VMS.
GB 35114 A-level (compile-time opt-in). SM2 certificate registration and keyed-SM3 integrity.
Continuous local recording. H.264 segments on disk with an index.jsonl index, retention days and a storage cap. This doubles as the source for GB28181 playback.
RTMP push to a cloud ingest, and JPEG snapshots over GET /snapshot. The Go build also carries a pure-Go MPEG-TS segmenter, so browsers get HLS without ffmpeg.
Imaging controls for brightness, contrast, saturation, sharpness, white balance and exposure mode. The Rust build adds an OSD watermark — custom text plus a live clock — burned in pre-encode, so it appears on RTSP, RTMP, recordings and the web preview alike.

Under the hood everything hangs off one encoded frame hub:

flowchart TD
    CAM["Camera<br/>CSI or USB"] --> CAP["Capture<br/>board-native path"]
    CAP --> ENC["H.264 encode<br/>HW or SW"]
    ENC --> HUB["Encoded frame hub"]
    HUB --> RTSP["RTSP :8554"]
    RTSP --> NVR["NVR or VMS"]
    HUB --> REC["Local recording"]
    REC --> GB["GB28181 device"]
    GB --> PLAT["Platform"]
    HUB --> WEB["Web UI :8088"]
    WEB --> BROWSER["Browser"]
    ENC -. tap .-> AI["Detection<br/>opt-in"]

Detection is a passive subscriber on that hub. If it's slow, or the model won't load, streaming is unaffected.

Detection that never leaves the board

Object detection is compile-time opt-in, and it runs locally or not at all. Nothing is uploaded, and there's no cloud inference bill because there's no cloud inference.

The Rust build taps the shared pre-encode YUV buffer, so inference never touches the capture or encoding path. The Go build subscribes to the encoded-frame hub and decodes keyframes for inference.

flowchart TD
    YUV["Shared YUV<br/>pre-encode"] --> PRE["Preprocess<br/>YUV to 320 square"]
    PRE --> RUN["ONNX Runtime<br/>NanoDet-Plus"]
    RUN --> POST["Postprocess<br/>argmax, GFL, NMS"]
    POST --> API["REST and SSE<br/>boxes in native px"]
    GUARD["Guardrails<br/>memory and core"] -. controls .-> RUN

The model is NanoDet-Plus-m 320 in ONNX form, Apache-2.0 licensed. There's a small model registry with runtime hot-swap between a 320 and a 416 variant, plus a YOLOX-nano decoder if you want a different family. Uploading new models at runtime is supported but off by default behind an explicit config gate, since a model file is untrusted input to an inference engine.

Detection fails open. If the runtime library or the model file is missing, the capability reports ai:false and the API says so — it never returns fabricated boxes. The trade-off is honest: libonnxruntime.so and the model are not bundled in the release tarball, so detection is a setup step, not a checkbox.

Which build

Two codebases, same protocols, same web API, same SPEC v1 JSON envelope. NVRs can't tell them apart. The choice is about the board and about how you like to deploy.

flowchart TD
    Q["Pick a build"] --> P1["Want the easiest<br/>cross-compile"]
    Q --> P2["Tightest memory<br/>and flash"]
    Q --> P3["Want OSD or<br/>zero subprocesses"]
    P1 --> G["Go build"]
    P2 --> R["Rust build"]
    P3 --> R
    G --> N1["Ships HLS and<br/>a bilingual UI"]
    R --> N2["About 2 MB binary<br/>no subprocesses"]

Your situation	Pick
Fastest path, no cross-compile friction	Go — zero CGO, stock cross-compile
Smallest RAM and flash footprint	Rust — ~2 MB binary
Watermark burned into every output	Rust
HLS playback or bilingual UI out of the box	Go
32-bit board	Go

What it costs to run

Both builds are static binaries with a systemd unit in the repo. No Docker, no runtime, no language toolchain on the device.

Metric	Go build	Rust build
Binary size	~15 MB	~2 MB
Memory	15–25 MB (+~15 MB with the AI build)	~6–12 MB bare pipeline; 93.8 MB measured full-feature
Subprocesses	mtxrpicam or ffmpeg	none
CPU, 720p@15fps	~15%	124–146% measured full-feature (~139% of four cores)

Read those conditions before drawing conclusions, because they're not the same measurement.

The Rust figures come from a run on 2026-09-14 on a Raspberry Pi 3B: 720p at 15fps, hardware M2M encoder, GB28181 registered, local recording on, OSD watermark on, per-frame INFO logging enabled, one RTSP-over-TCP client, 60 seconds, 13 samples. RSS held flat at 93.8 MB. A bare pipeline — no GB28181, no recording, no watermark — sits far lower, which is where the ~6 MB figure in the repo description comes from. Those two numbers are different measurement scopes, and I'd rather show you both than quote the flattering one.

The Go CPU figure comes from the v0.1.0 release notes, also measured on a Pi 3B. Different scope again, so I'm not going to declare a winner. Both repos ship bench/rpi-bench.sh if you want your own numbers on your own board.

For context, there's a Pi 3B — a 2016 board with 1 GB of RAM — holding up a 720p stream with recording and a standards-based registration on top. That's the repurposing argument in one line.

The repo README also carries a comparison against MediaMTX on memory and CPU for the camera-side role. MediaMTX is a media server and a good one, and it isn't trying to be a camera; the comparison only covers turning a Pi plus a sensor into a device an NVR can discover, and the numbers are indicative measurements from my own deployments rather than a controlled benchmark.

Onboards without a V4L2 M2M encoder node — x86 boxes, NAS units, most ARM SBCs — the Rust build falls back to an in-process openh264 software encoder, and the Go build to a resident ffmpeg subprocess. That's why "any Linux board" is more than a tagline, and it's also why the subprocess column is worth reading before you pick. Note the CPU budget on weak boards: 720p@15 is the sane ceiling on A53-class silicon.

The parts that were hard

The most useful file in the Rust repo isn't code. It's a 386-line interop notebook documenting ten real bugs found while getting a gortsplib-based NVR to actually stay connected. Four of them explain why cheap RTSP implementations fail where ffmpeg succeeds.

A duplicated Content-Length header. The RTSP DESCRIBE response had the header twice — added once by the response constructor and once by the handler. ffmpeg tolerates it; the NVR's parser rejected the whole response and refused to connect. Strict parsers catch what lenient ones hide.

Partial frames are worse than missing frames. One encoded frame usually becomes several RTP packets. Under TCP interleaved mode, sending them individually meant a timeout mid-frame delivered half an FU-A fragment, which a decoder can't reconstruct. Fix: batch every packet of an access unit into a single write_all, so the frame arrives whole or not at all. Then rewind the RTP sequence numbers for the packets you didn't send, because a gap in the sequence makes a strict NVR declare packet loss. Then drop subsequent P-frames until the next keyframe, because those reference the frame you just threw away.

Timestamps must be strictly monotonic. RTP timestamps computed at send time rather than capture time produced duplicates whenever frames came out of the encoder in a burst. ffmpeg reported non-monotonically increasing DTS and the NVR dropped frames. The current implementation takes the capture timestamp and enforces a minimum increment of one frame tick between consecutive packets.

Pi undervoltage corrupts H.264. An ageing Pi 3B with marginal power delivery drops voltage under encoder load, the firmware caps the clock, and the transition produces corrupted NAL units. vcgencmd get_throttled returned 0x50005 — undervoltage happening now plus a frequency cap. The encoder thread now watches firmware throttle state and skips every other frame while throttled, which halves the load, lets the clock recover, and self-corrects. After that change, a 60-second decode test reported zero errors.

One frontend fix in the same notebook: Chrome 120+ no longer renders multipart/x-mixed-replace inside an <img> tag, so the MJPEG preview went black. Replacing it with H.264 over WebSocket plus Media Source Extensions fixed it, and reworking the buffering brought preview latency from 3–9 seconds down to under a second.

Code and test volume, measured 2026-09-15: the Go repo is 93 .go files and 22,022 lines with 287 test functions across 34 test files; the Rust repo is 63 first-party .rs files and 25,387 lines with 469 test functions. Documentation is 7 markdown files in Go and 13 in Rust, four of them design documents for unbuilt features.

The protocol layer is four libraries you can use on their own

None of those fixes live inside the camera service. The ONVIF and GB28181 logic in both builds is extracted into four standalone libraries — one protocol, two languages — and all four are MIT licensed, which is a shorter licence than the Apache-2.0 the camera service itself carries. Versions and counts in this section were pulled on 2026-09-15.

flowchart TD
    P["Your product"] --> O["ONVIF side"]
    P --> G["GB28181 side"]
    O --> OG["onvif-go<br/>client + server"]
    O --> OR["onvif-rs<br/>device server"]
    G --> GG["gb28181-go<br/>UAC, UAS, cascade"]
    G --> GR["gb28181-rs<br/>UAC"]

Library	Language	Latest tag	Role
onvif-go	Go	v2.0.0-rc6	ONVIF client + virtual camera server
onvif-rs	Rust	v0.6.0	ONVIF device-side server
gb28181-go	Go	v0.10.0	GB28181 device + platform + cascade
gb28181-rs	Rust	v0.11.0	GB28181 device side

onvif-go works from both ends: a client you can point at someone else's camera, and a virtual camera server for testing against. Its go.mod is a module line and a Go version — zero third-party dependencies, which you can verify by opening one file. It ships prebuilt CLI tools for discovery, diagnostics, a quick-start helper and a server simulator, which is what you want when there's no hardware on the bench. Device interop is exercised against Hikvision, Dahua, Axis, Bosch and ESP32 hardware. v2 was a breaking refactor from v1, and the repo carries a MIGRATION.md.

onvif-rs is the device-side server, published on crates.io as onvif-device-rs because the onvif-rs name is held by an abandoned 2018 placeholder. Its response XML is byte-stable, pinned by golden-string tests, which is the property strict NVR parsers actually need. Production code contains no unwrap or expect and CI enforces that; auth has a UsernameToken replay guard and a lockout, and handler panics are isolated so one bad request can't take the server down.

gb28181-go covers the most ground: device (UAC), platform (UAS) and multi-level cascade. The cascade path has loop prevention — each channel carries an OriginDeviceID, so a middle platform never hands an upper platform its own channels back, and their INVITEs get a 404. The GB/T 28181-2022 alignment lives here too: five new information queries pinned to the standard's annexes, opt-in X-GB-Ver version negotiation, and a wire-format bug found by reading the spec text rather than a peer implementation (SVAC audio stream_type corrected from 0x81 to 0x9B).

gb28181-rs is the device side of the same thing, sharing golden wire-format contracts with its Go twin. It's what gives the Rust camera build its GB28181 capability.

Library	Tests	Code
onvif-go	723 (693 tests + 25 benchmarks + 5 fuzz targets)	177 `.go` files, 55,284 lines
gb28181-go	574 (563 + 7 + 4)	161 `.go` files, 39,695 lines
onvif-rs	174	21 `.rs` files, 8,922 lines
gb28181-rs	218	40 `.rs` files, 16,876 lines

That's 1,689 tests across the four. The Go line counts include their test files: 88 files and 27,156 lines in onvif-go, 89 files and 21,114 lines in gb28181-go. The Rust crates keep test modules inline as well as under tests/. Both declare MSRV 1.80; the GB 35114 feature needs 1.85 or newer.

If you're building an NVR, a platform, or a device simulator, you can take the library without taking the camera service.

What you should know before depending on them:

onvif-go is still a release candidate. v2.0.0-rc6, and v1 to v2 was a breaking refactor.
onvif-rs covers an NVR-pairing action set. Its own release notes say no GetSnapshotUri and no Events, and that compatibility with third-party clients beyond NVR-style consumers is untested.
gb28181-rs is device-side only, and the Go platform side doesn't implement the 2022 HTTP media plane for recorded-video search.
Neither GB28181 library implements subscriptions, alarm notifications, mobile-position reporting or PTZ command intake yet. That's the v0.3.0 scope, and it lands in the libraries first while the camera service only wires to it.
All four are new. onvif-go was created in July 2026, the other three in August 2026, with between 0 and 4 stars. I'm not going to make promises about maintenance cadence.

What it doesn't do yet

Worth reading before you plan around it.

Profile S only. No Media2, no Profile T, no Recording service, no Analytics. If your NVR needs server-side recording control or two-way audio, this isn't the device for it.
No H.265, no WebRTC, no multi-camera. All three have design documents in the Rust repo and empty feature flags in the code. You can enable them and get a log line.
Motion detection is half-landed. The Rust build implements a frame-delta detector in src/motion/ but it isn't wired into the web API yet.
AI needs firmware-side setup. The ONNX runtime library and the model aren't in the release archive.
No 32-bit artifact from the Rust repo at v0.2.0. An upstream width bug blocked it; it's fixed upstream and lands with v0.3.0. The Go repo already ships armv7.
It's new. Both repos were created on 2026-09-12 and have one contributor. I'm not going to make claims about response times or maintenance cadence, and you shouldn't build a procurement decision on them.
GB 35114 only goes to A-level, and I have no third-party test report to point at.

The frozen v0.3.0 scope is a complete GB/T 28181-2022 device-side capability package, published as a synchronized release across both implementations. Three things are explicitly out of scope and will stay that way: SVAC video and GB 35114 B/C levels, which would need SVAC hardware codecs; SIP over WebSocket, which isn't a device-side requirement in practice; and a platform-side (UAS) role, because the product is a device. The 2022 HTTP media plane for recorded-video search is deferred to v0.4.x rather than squeezed in.

Getting started

Go build:

git clone https://github.com/xiqing85/mibee-eye-go
cd mibee-eye-go
make build

cp configs/config.example.yaml config.yaml
# set camera parameters and onvif.username / onvif.password

sudo cp deploy/mibee-eye.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now mibee-eye

Rust build, cross-compiling from an x86 workstation to a Pi:

git clone https://github.com/xiqing85/mibee-eye-rs
cd mibee-eye-rs

# fully static, no external toolchain needed
rustup target add aarch64-unknown-linux-musl
make cross-build

./deploy/install.sh pi@192.168.1.100

Then point your NVR at it. WS-Discovery should surface it on the local network; the manual address is http://<board-ip>:8080/onvif/device_service and the stream is rtsp://<board-ip>:8554/stream.

Two config defaults worth changing before you expose anything: the ONVIF password is empty by default, and gb28181.enabled is off unless you turn it on. Environment variables with the MIBEE_EYE_ prefix override any config key, which is handy for injecting secrets from systemd rather than writing them into a file.

One caveat on integrations: the ONVIF surface is the standard one, and the only NVR I have deep interop notes for is a gortsplib-based one. Synology Surveillance Station, Blue Iris and Scrypted consume standard ONVIF Profile S devices, and there's nothing custom here, but I'm not going to claim tested-and-verified for hardware I haven't run. If you try one of them and it misbehaves, that's exactly the kind of bug report the interop notebook exists to collect.

Project links

mibee-eye-go: https://github.com/xiqing85/mibee-eye-go

mibee-eye-rs: https://github.com/xiqing85/mibee-eye-rs

The protocol libraries underneath:

onvif-go: https://github.com/mickeyzzc/onvif-go

onvif-rs: https://github.com/mickeyzzc/onvif-rs (published on crates.io as onvif-device-rs)

gb28181-go: https://github.com/mickeyzzc/gb28181-go

gb28181-rs: https://github.com/mickeyzzc/gb28181-rs

Licence: Apache-2.0 for both camera builds, MIT for the four libraries. Note that v0.1.0 shipped under CC BY-NC 4.0, which forbids commercial use, so take v0.2.0 or later; third-party terms are listed in NOTICE.

What's new in MiBee NVR v0.11.0: WHIP ingest, mDNS discovery, and browser-side AI on a Pi

ZhengZhiCong — Tue, 18 Aug 2026 13:05:32 +0000

Home security cameras have a dirty secret: the software is the weak link. You buy a decent ONVIF camera and then you're locked into the vendor's app, or you sign up for a cloud service that bills per camera, or you stand up an NVR that wants a dedicated machine with a Coral TPU before it does anything interesting.

I wanted the opposite. One binary I could drop on a Raspberry Pi, a web UI that doesn't feel like 2005, cameras that get discovered instead of configured, and no monthly fee. That project became MiBee NVR. It runs on a 512MB memory budget and the README's claim, which I stand by, is that a Pi 3B is the minimum baseline. The repo is at github.com/Mi-Bee-Studio/MiBeeNvr, current version 0.11.0.

The pitch in one line: single Go binary, zero dependencies (CGO_ENABLED=0), an embedded Svelte 5 SPA, no cloud, no subscriptions. Install is one curl piped to bash, or a prebuilt binary for amd64, arm64, or armv7, or Docker from ghcr.io/mi-bee-studio/mibeenvr. The UI runs on port 9090 and comes with dark/light themes and English/Chinese out of the box.

ONVIF that behaves

The thing I'm proudest of is the ONVIF handling, because it fixes the pain I kept hitting with real cameras. Cameras on WiFi roam between access points and come back with a new IP. Your NVR still has the old one stored, the feed dies, and you go digging through the router's DHCP table to fix it. MiBee NVR instead enrolls ONVIF cameras automatically in the background when they join the LAN, the way a Hikvision NVR does. No manual scan.

Two details I like:

Unauthenticated cameras start recording immediately. If it's open, it's in.
Authenticated cameras are added as pending until you supply credentials.

And the IP self-healing: when a camera changes IP, the NVR relocates it by serial number and reconnects. The unicast probing works across routed subnets, so it isn't limited to one broadcast domain.

Browser-side AI, no Coral required

Most NVR setups push detection to a separate box, often a USB Coral or an Nvidia card. MiBee NVR runs inference in the browser tab instead. ONNX Runtime Web with WebGPU acceleration, a 5.4MB yolo11n model by default (squeezenet is there as a fallback). The backend does zero inference, so the Pi never breaks a sweat, and the video never leaves the device to get "analyzed" somewhere else. If your browser supports WebGPU you get object detection; if it doesn't, you still get a fully working NVR.

One real constraint worth knowing: without crossOriginIsolated there's no SharedArrayBuffer, so the runtime is single-threaded. Fine for one camera, something to remember if you plan to run several detections at once.

The war story: model files kept coming back corrupted and ORT threw INVALID_PROTOBUF. Root cause was our own gzip middleware. Calling Close() on a gzip.Writer that never wrote anything appends an empty gzip member, about 20 bytes, to the end of binary responses. The ONNX file was fine except for those 20 bytes. The fix tracks whether the writer ever wrote and only calls Close()/Flush() after the first Write(). That's issue #109 if you want the details.

Every protocol, in and out

NVR is a messy word because it covers two directions: getting streams in and getting streams out.

Inbound, MiBee NVR speaks RTSP (H.264, H.265, MJPEG), HTTP JPEG snapshots, RTMP ingest, SRT ingest, and WHIP, which is WebRTC push. The WHIP support is the headline feature of v0.11.0 and it's fun: you can publish from a browser tab or OBS with a stream key mapped to a camera, no extra software.

Outbound to your browser: WebRTC via WHEP for sub-second latency, HLS with LL-HLS for on-demand, HTTP-FLV, and raw WebSocket frames. If you need to push somewhere else, there's a native Go relay that forwards any camera to a remote RTMP or RTSP target. Point it at a streaming platform and you have a cheap live production pipeline.

Recording, the part people forget

An NVR that can't scrub is a paperweight. Recordings are MP4 segments with per-camera retention policies, audio capture (AAC, G.711, Opus), and a timelapse mode. Playback chains segments with double buffering so there's no gap when you're watching back, and the VOD timeline covers a full day, so you can drag across recordings and the gaps between them. There's also an AVI frame-browse mode when you need to step through frame by frame.

Built for the homelab, not around it

This was a design goal, so the integration list is deliberately boring: MQTT for trigger-based recording (Home Assistant people know what to do), WebDAV and FTP for pulling files off the box, a Prometheus metrics endpoint with full docs, a REST API with BasicAuth and bcrypt, and a stable device_id in /api/health so your automation has something to hold onto. Discovery is mDNS (_mibee-nvr._tcp) with a UDP responder as fallback for networks that block multicast.

Remote access is your choice: Tailscale, Cloudflare Tunnel, or WebRTC ICE/STUN/TURN configuration. The docs cover all three.

Deployment story

One-liner: curl -fsSL https://raw.githubusercontent.com/Mi-Bee-Studio/MiBeeNvr/main/install.sh | sudo bash, which creates a systemd service, a system user, and a config.
Binaries for amd64, arm64, and armv7 from GitHub Releases.
Docker images on GHCR with exact, minor, major, and latest tags.
NAS guides for unRAID, Synology, and QNAP, plus fnOS, iStoreOS, and 极空间 if you're in the Chinese NAS ecosystem.

The China-market side, in one paragraph

MiBee NVR also speaks GB/T 28181, the Chinese national standard for video surveillance, as an experimental SIP platform, and it can adopt Xiaomi cloud cameras (CS2 and legacy TUTK models). If you're not in China, ignore both of those; they exist because the home market runs on them. Everything else in this post is the same code path.

Engineering hygiene and licensing

The repo has 3991 Go tests, 389 frontend tests, and 13 Playwright E2E tests, with golangci-lint v2 (gofumpt, perfsprint) enforced in CI. One honest note on licensing: versions up to v0.10.1 are MIT. From v0.11.0 the project moved to AGPL-3.0-only, with a linking exception for pkg/ so you can build against the extension interfaces without triggering AGPL obligations. Running it, recording cameras, watching streams: no obligations, commercial use included.

Quick start

curl -fsSL https://raw.githubusercontent.com/Mi-Bee-Studio/MiBeeNvr/main/install.sh | sudo bash

Or grab a binary:

wget https://github.com/Mi-Bee-Studio/MiBeeNvr/releases/latest/download/mibee-nvr-arm64
chmod +x mibee-nvr-arm64
./mibee-nvr-arm64 init --password yourpassword
./mibee-nvr-arm64 -config mibee-nvr.yaml

Then open http://localhost:9090.

There's documentation for everything mentioned here: getting started, a full configuration reference, the REST API, an ONVIF guide, transcoding, metrics, and per-NAS deployment. The architecture doc is a good place to start if you want to read the code with a map.

If you run it and something breaks, the issues tracker is open. I read everything, and the WHIP ingest that just landed came straight out of user requests.

Your Cameras Can Now Go Live — MiBeeNVR v0.8.1 Brings Native Live Streaming

ZhengZhiCong — Fri, 03 Jul 2026 06:05:11 +0000

An open-source NVR running on a Raspberry Pi, competing with OBS for live streaming work? This post covers how we took the "surveillance camera → live stream" pipeline from functional to polished — and how many protocol-layer traps we hit along the way.

Above: Live stream viewed on the mobile app. Relay handled natively by MiBeeNVR's Go implementation — no FFmpeg dependency.

TL;DR

MiBeeNVR v0.8.1 supports pushing any connected camera feed directly to live streaming platforms via RTMP. Pure Go implementation — no FFmpeg installation, no external processes. Configure your stream URL and go live.

⚠️ The author has only verified streaming on the Douyu platform so far. Other platforms are theoretically compatible (all implement the FMS standard), but remain untested. If you run into issues, please report them on GitHub Issues.

Runs on Raspberry Pi 3B and above. Supports both H.264 and H.265 cameras.

What Is MiBeeNVR?

MiBeeNVR is an open-source Network Video Recorder (NVR) written in Go, optimized for embedded devices like the Raspberry Pi and Banana Pi. Prior to v0.8.1, it already shipped with:

Multi-camera management and recording (RTSP / ONVIF / Xiaomi dual-camera)
H.264/H.265 hardware transcoding
Time-lapse recording and synthesis
LL-HLS / WebRTC low-latency playback
One-command Docker deployment

One thing was missing, though: live streaming.

The community kept asking: "Can we push camera feeds directly to a live stream?" Pet owners wanted pet live streams. Security folks wanted public surveillance channels. Hardware tinkerers just wanted to play. v0.8.1 fills that gap.

Live Streaming: Looks Simple, Hurts Plenty

Step 1: Make It Work First

The most straightforward approach is to call FFmpeg — it's the Swiss Army knife of the streaming industry, and OBS uses it under the hood. Configure an RTMP target in MiBeeNVR's "relay output," spin up an FFmpeg subprocess on the backend, and the feed does reach the live stream.

But here's the problem: FFmpeg is an external process. It needs separate installation, separate lifecycle management, and extra memory. For a project aiming to "run lightweight on a Raspberry Pi," dragging along a multi-megabyte external process for streaming isn't elegant.

What we wanted: a pure Go RTMP streaming implementation, embedded in the NVR process, with zero external dependencies.

Step 2: Six Layers of Hell in Pure Go Streaming

The RTMP protocol looks simple — handshake, connect, push data. But when you actually interface with live streaming platforms (whose backends are all Adobe FMS-compatible implementations), you'll find that every single layer can block you.

We didn't encounter one problem — we encountered six stacked problems:

flowchart TD
    A["L1 Handshake digest<br/>HMAC-SHA256 signing"] --> B["L2 Chunk size<br/>mismatched block size"]
    B --> C["L3 Chunk header<br/>Type 0/1/2/3 format"]
    C --> D["L4 Background read<br/>buffer overflow"]
    D --> E["L5 Metadata fields<br/>missing width/height/fps"]
    E --> F["L6 StreamID byte order<br/>big-endian vs little-endian ⭐"]
    F --> G["✅ Stream success"]

    style A fill:#FF9800,color:#fff
    style B fill:#FF9800,color:#fff
    style C fill:#FF9800,color:#fff
    style D fill:#FF9800,color:#fff
    style E fill:#FF9800,color:#fff
    style F fill:#f44336,color:#fff
    style G fill:#4CAF50,color:#fff

After fixing the first four layers — handshake passing, connection established, publish command accepted — the first video frame would be sent, and 74 milliseconds later, the connection would be reset by the server.

That 74ms is critical: it rules out a network timeout (those take ~5 seconds). The server was rejecting the data at the frame level.

Final Boss: A Tale of Byte Order

The root cause was buried so deep that it took us a full day to find.

The RTMP protocol has a field called MessageStreamID that identifies which stream a message belongs to. Per the RTMP spec, this field should be little-endian. So we dutifully wrote it as little-endian in our code, exactly as the spec says.

But every major implementation — FFmpeg, OBS, gortmplib — uses big-endian.

And so do the live streaming platforms.

Our code used the third-party library's standard path (big-endian) for most messages, but our own custom write path used little-endian for others. On the same connection — sometimes big-endian, sometimes little-endian. The server received this schizophrenic packet stream and immediately sent an RST.

We found the bug through an in-process byte-level dump — logging every byte sent, parsing each chunk header one by one — until we spotted the endianness inconsistency in the streamID field. The fix was 4 lines of code. Finding it took an entire day.

flowchart LR
    A["Go standard Writer<br/>streamID = big-endian"] --> C["Live platform<br/>expects big-endian ✅"]
    B["Our Writer<br/>streamID = little-endian"] --> D["Live platform<br/>receives little-endian ❌ RST"]

    style A fill:#4CAF50,color:#fff
    style B fill:#f44336,color:#fff
    style C fill:#4CAF50,color:#fff
    style D fill:#f44336,color:#fff

TL;DR: The spec says go left. The entire world goes right. You follow the spec — and the entire world rejects you.

The Result

With all six layers fixed:

✅ Pure Go relay streaming works — no FFmpeg dependency
✅ TCP connection stays stable, continuous streaming without interruption
✅ Raspberry Pi camera feeds push to live streams in real-time, ~2-3 second latency
✅ H.264 cameras supported; H.265 streaming coming soon

⚠️ Do not enable the use_ffmpeg option. FFmpeg-based streaming has known issues: playback may stutter mid-stream due to frame rate mismatch, and it won't auto-recover. The pure Go relay path is thoroughly validated and is the recommended streaming method.

💡 The pure Go relay is the focus of this release. If you encounter Go streaming issues on other platforms, please prioritize filing an Issue about the Go path to help the project keep improving — rather than falling back to FFmpeg.

How to Use It

Option 1: MiBeeNVR UI Configuration

In the camera settings, find "Relay Output" and enter your live streaming platform's RTMP URL:

rtmp://<server>/live/<stream-key>

Get the stream key from your live streaming platform's broadcast settings.

Option 2: Docker Deployment

docker run -d \
  --name mibee-nvr \
  -p 8080:8080 \
  -v mibee-nvr-data:/var/lib/mibee-nvr \
  ghcr.io/mi-bee-studio/mibee-nvr:latest

Open http://<device-ip>:8080 to access the admin interface, add your cameras, then configure relay output.

Testing Status & Caveats

Tested platform: Douyu Live (companion app mode, LAN streaming) ✅

Theoretically compatible: All FMS-standard live streaming backends. However, implementation details vary across platforms — untested ones may present issues. If you hit problems, please report them on GitHub Issues and the author will follow up.

⚠️ Important Notice

Continuous long-term streaming to domestic live platforms is not recommended. Each platform has its own streaming policies and restrictions. Prolonged unattended camera streaming may trigger:

Throttling (reduced bitrate/resolution)
Stream key expiration leading to disconnection
Account warnings or even bans

Make sure to understand your platform's policies and follow their rules. MiBeeNVR provides a technical capability — how you use it, on which platform, and what you stream is your own responsibility.

Lessons Learned

This live streaming project left us with three key takeaways:

1. Comparative Experimentation Is the Gold Standard for Debugging

When the Go code failed to stream, we couldn't tell if it was a code issue or an environment issue. Only after manually streaming via FFmpeg CLI on the same device and succeeding did we confirm the problem was in the Go protocol layer. Without that control experiment, all the guessing in the world is meaningless.

2. Binary Protocol Bugs Require Looking at Bytes

Byte-order issues are invisible when reading code — because the code "looks correct" (it follows the spec). Only by dumping the actual bytes sent over the wire and parsing them one by one can you spot the endianness mismatch. When debugging binary protocols: add a dump first, analyze second.

3. Spec ≠ Reality

The RTMP spec says MessageStreamID is little-endian, but the entire world uses big-endian. This kind of "spec vs. practice divergence" is extremely common in legacy protocols. When your spec-compliant code hits walls everywhere, go look at how others actually implemented it — de facto standards are more persuasive than paper specifications.

About MiBeeNVR

MiBeeNVR is an open-source project by Mi-Bee-Studio, written in Go and optimized for embedded devices like the Raspberry Pi.

GitHub: https://github.com/Mi-Bee-Studio/MiBeeNvr

Key Features:

Multi-camera management (RTSP / ONVIF / Xiaomi dual-camera)
H.264/H.265 hardware transcoding (V4L2 / VAAPI / NVENC auto-detection)
Time-lapse recording and synthesis (pure Go muxer, zero encoding overhead)
LL-HLS / WebRTC low-latency playback
RTMP live streaming (new in v0.8.1, pure Go)
One-command Docker deployment

Supported Platforms: Raspberry Pi 3B+ and above / x86_64 / ARM64 / Docker

MiBeeNVR v0.8.1 has been released. See the full changelog at GitHub Release Notes.

If you're building interesting projects with Raspberry Pi, come chat with us on GitHub.

Embedded Camera Module Selection Handbook — From Basics to Purchasing Reference

ZhengZhiCong — Mon, 29 Jun 2026 23:07:58 +0000

About This Handbook

This handbook is a revised edition of my own "Embedded Camera Module Technical Manual" that I previously compiled and shared online. The original covered eight mainstream sensors: OV2640, OV5640, OV7725, GC0309, OV7670, OV3660, GC2053, and GC4653. This revision expands coverage with 12 additional sensors (OV9281, OV5647, IMX219, IMX273, IMX296, IMX307, IMX335, IMX415, IMX477, IMX678, IMX708, IMX766), for a total of 20. The content was comprehensive, but when I sat down to use it as a purchasing reference, I felt uneasy — a single wrong voltage parameter can fry a board, and I didn't want to ship mistakes I couldn't vouch for.

So I did one thing: verified every key parameter against official datasheets. Sources include OmniVision's official product pages, GalaxyCore datasheets, SparkFun / DigiKey / Mouser datasheet mirrors, and Espressif's official esp32-camera driver source code.

The result: most parameters were correct, but there were several critical errors, especially around voltage, frame rates, and some marketing terminology. This handbook is the corrected version — every error-prone parameter is annotated with reliability status and evidence. Use it as a purchasing checklist with confidence.

Reliability annotation legend

✅ Verified: Parameter matches official datasheet exactly

⚠️ Note: Original had errors or was misleading; corrected with evidence

🔍 Unconfirmed: Official datasheet unclear, or varies by module manufacturer

Quick Selection Guide

Start with this table to find your use case, then read the details below.

Use Case	Top Pick	Resolution	Interface	Key Advantage	Price Tier
Education (Arduino/STM32)	OV7670	0.3MP	DVP	Richest ecosystem, most tutorials	Very low
Wi-Fi streaming (ESP32-CAM)	OV2640	2MP	DVP	Built-in JPEG compression	Low-mid
High-res photo + autofocus	OV5640	5MP	DVP/MIPI	Only entry-level with AF	Mid-high
Night vision / machine vision	OV7725	0.3MP	DVP	6.0µm pixels, low-light king	Mid
Ultra low cost (access/POS)	GC0309	0.3MP	DVP	Lowest price	Very low
3MP upgrade (ESP32)	OV3660	3MP	DVP	BSI tech, drop-in for OV2640	Low-mid
Security / surveillance	GC2053	2MP	DVP/MIPI	1/2.9" sensor + 2.8µm pixels	Mid
High-end security / AI vision	GC4653	4MP	MIPI only	4MP + BSI high sensitivity	Mid-high
Raspberry Pi / Jetson entry	IMX219	8MP	MIPI 2L	Pi V2 standard, most mature driver	Low
4K security / NVR	IMX415	8MP	MIPI 2/4L	4K STARVIS, rich module ecosystem	Mid-high
High-end 4K security (2024)	IMX678	8MP	MIPI 2/4/8L	STARVIS 2 + Clear HDR	High
1080P night security	IMX307	2MP	MIPI 2/4L	2.9µm + STARVIS, strong night vision	Mid
Industrial FA / global shutter	IMX273	1.58MP	MIPI 1L	Pregius global shutter, 226fps	High

⚠️ ESP32 users note: ESP32 series only supports DVP-interface sensors. GC4653 is MIPI-only and cannot be used with ESP32. OV3660 also only supports DVP (claims of MIPI support in the original document are wrong — see below).

DVP Interface Basics

DVP (Digital Video Port) transmits uncompressed raw image data via parallel data lines with a sync clock. Understanding DVP signals and timing is prerequisite for wiring and debugging all DVP modules.

Signal Definitions

Signal	Direction	Description	Typical Level
XCLK	Input	External master clock, typically 6–24 MHz ✅	3.3V
PCLK	Output	Pixel clock for synchronous data sampling ✅	3.3V
VSYNC	Output	Frame sync signal ✅	3.3V
HREF / HSYNC	Output	Line sync signal ✅	3.3V
D[7:0]	Output	8-bit parallel pixel data bus ✅	3.3V
SIOC / SCL	Input	SCCB / I²C clock ✅	3.3V
SIOD / SDA	Bidirectional	SCCB / I²C data ✅	3.3V
PWDN	Input	Power-down control	3.3V
RESET	Input	Hardware reset (active low)	3.3V

Timing

A complete frame is bounded by VSYNC. Within a frame, multiple HREF pulses occur, each corresponding to one line of valid pixel data. When both VSYNC and HREF are active, D[7:0] data latched on PCLK rising edge is valid pixel data.

The red scan line sweeps left to right. When VSYNC + HREF are both HIGH, each PCLK rising edge latches one pixel byte (D0, D1, D2…).

SCCB Control Interface

SCCB (Serial Camera Control Bus) is OmniVision's I²C-compatible 2-wire serial protocol. SCL clock frequency supports up to 400 kHz ✅ (OV2640 datasheet Table 9 confirms). If your MCU lacks internal pull-ups, add 4.7 kΩ pull-up resistors on SCL and SDA to 3.3V ✅.

⚠️ I²C addresses are sensor-specific, NOT uniform 0x30/0x60!

Many online sources casually claim "OV series address is 0x30/0x60" — this is wrong. Verified 7-bit slave addresses per sensor:

Sensor 7-bit Address Evidence

OV2640 0x30 Datasheet: 8-bit write 0x60 / read 0x61

OV5640 0x3C Datasheet: 8-bit write 0x78 / read 0x79

OV7725 0x21 Datasheet: 8-bit write 0x42 / read 0x43

OV7670 0x21 Linux kernel driver ov7670.c

GC series Vendor-defined Check specific datasheet

If SCCB configuration fails, verify I²C address first, not just pull-ups.

Sensor	7-bit Address	Evidence
OV2640	0x30	Datasheet: 8-bit write 0x60 / read 0x61
OV5640	0x3C	Datasheet: 8-bit write 0x78 / read 0x79
OV7725	0x21	Datasheet: 8-bit write 0x42 / read 0x43
OV7670	0x21	Linux kernel driver ov7670.c
GC series	Vendor-defined	Check specific datasheet

MIPI CSI-2 Interface Basics

When resolution and frame rate go up (1080P@30fps+), DVP's 8-bit parallel bus can't keep up — too many wires, limited speed, poor noise immunity. All Sony IMX sensors below use MIPI CSI-2. Here's the foundation.

DVP vs MIPI CSI-2

Feature	DVP	MIPI CSI-2
Data lines	8–10 parallel + sync signals	1–4 differential pairs (lanes)
Clock	Single-ended XCLK / PCLK	Differential clock lane
Bandwidth	≤ ~200 MB/s	Up to ~2.5 GB/s (4-lane)
Noise immunity	Poor, long cables cause artifacts	Good, differential rejects common-mode noise
Controller complexity	Low (straight to FIFO)	High (needs D-PHY + CSI-2 controller)
Typical controllers	ESP32, STM32, RP2040	Raspberry Pi, Jetson, Hi3516, RV1106
Typical sensors	OV2640, OV5640, OV7670	IMX219, IMX415, IMX678

Lane Count and Bandwidth

Lanes	Total Bandwidth	Suitable For
1-lane	~500 MB/s	1080P@30fps RAW8
2-lane	~1 GB/s	1080P@60fps or 4K@15fps
4-lane	~2 GB/s	4K@30fps+

Purchasing note: More lanes isn't always better — your controller's MIPI receiver must support the lane count. For example, RV1106 supports 2-lane/4-lane MIPI input; connecting a 1-lane IMX273 may not work. Check controller specs before buying.

ESP32 users note: ESP32 (including ESP32-S3) has no MIPI CSI-2 receiver. None of the Sony IMX sensors below work with ESP32. You need a MIPI-capable controller like Raspberry Pi, Jetson, RV1106, Hi3516, etc.

Sensor Technology Principles

Shutter Types: Rolling vs Global

Most sensors in this handbook use rolling shutter — exposing rows sequentially with a time gap between rows. Fast-moving objects appear skewed/warped (jello effect). Only IMX273, OV9281, and IMX296 use global shutter — the entire frame is exposed simultaneously, zero motion distortion.

A blue ball moves left to right. Rolling shutter exposes row-by-row, capturing the ball at different positions per row → diagonal skew. Global shutter exposes all rows simultaneously → clean bar.

Backside Illumination (BSI) vs Frontside Illumination (FSI)

Traditional CMOS sensors (FSI) have metal wiring above the photodiode, blocking some light. BSI (Backside Illumination) flips the sensor so light enters from the back, reaching the photodiode directly — 30%–60% more light. STARVIS, Exmor R, and OmniBSI all use BSI technology.

Yellow dots are photons. FSI's wiring blocks ~50% of light. BSI flips the structure so all photons reach the sensor.

HDR Multi-Exposure Fusion

In backlit scenes, short exposure captures highlights (sky) but shadows are black; long exposure captures shadows but highlights are blown. HDR merges multiple exposures into one image, preserving both highlight and shadow detail. IMX307/IMX335/IMX678 all support hardware-level HDR.

Short exposure preserves highlights, long exposure preserves shadows — merged HDR significantly expands dynamic range. Frames appear sequentially to show the fusion process.

Quad Bayer Pixel Binning

High-resolution sensors (e.g., IMX766 50MP) have tiny pixels (1.0µm) with insufficient light per pixel. Quad Bayer groups 4 adjacent same-color pixels, reading them as 1 larger pixel — 4× area, effective pixel size doubles (1.0µm → 2.0µm), dramatically improving low-light performance. Full resolution by day, auto-binned by night.

4 pixels of 1.0µm merge into 1 pixel of 2.0µm. 4× area = 4× light = significantly better SNR.

OV2640 Module

OmniVision 1/4-inch, 2MP CMOS sensor with built-in JPEG compression engine. The standard choice for ESP32-CAM.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV2640	✅
Optical format	1/4 inch	✅
Effective pixels	2MP (1600 × 1200)	✅
Pixel size	2.2 µm × 2.2 µm	✅
Shutter	Electronic rolling	✅
Interface	DVP 8-bit parallel + SCCB	✅
Output formats	YUV422, YUV420, RGB565, RGB555, RAW RGB, JPEG	✅
Max frame rate	UXGA@15fps, SVGA@30fps, CIF@60fps	✅
Master clock XCLK	6 – 24 MHz (typical 24 MHz)	✅
Operating voltage	Analog 2.5–3.0V, I/O 1.7–3.3V, Core 1.2/1.3V	✅
Power	~125 mW (15fps UXGA YUV), ~140 mW (JPEG mode)	✅

Source: Official OmniVision OV2640 datasheet (uctronics.com/download/OV2640_DS.pdf)

Resolution & Frame Rate

Resolution	Size	Typical fps
UXGA	1600 × 1200	15 fps
SXGA	1280 × 1024	15 fps
SVGA	800 × 600	30 fps
VGA	640 × 480	30 fps
QVGA	320 × 240	60 fps

Typical Pinout (24-PIN FPC)

✅ Verified: OV2640 modules do use 24-pin 0.5mm pitch FPC (confirmed by OV2640FSL datasheet, Arducam module spec). However, pin order may vary by module manufacturer — always check your specific board's datasheet.

Pin	Function	Pin	Function
1	GND	13	D3
2	VCC (3.3V)	14	D4
3	GND	15	D5
4	PWDN	16	D6
5	RESET	17	D7
6	XCLK	18	NC
7	VSYNC	19	SIOC (SCL)
8	HREF	20	SIOD (SDA)
9	PCLK	21	NC
10	D0	22	NC
11	D1	23	NC
12	D2	24	NC

Purchasing tip: OV2640's built-in JPEG compression outputs compressed streams directly, significantly reducing MCU RAM and bus bandwidth requirements. Best price/performance for ESP32-CAM.

OV5640 Module

OmniVision 1/4-inch, 5MP CMOS sensor with integrated autofocus (AF) voice coil motor driver. The image quality ceiling for entry-level.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV5640	✅
Optical format	1/4 inch	✅
Effective pixels	5MP (2592 × 1944)	✅
Pixel size	1.4 µm × 1.4 µm	✅
Shutter	Electronic rolling	✅
Interface	DVP 8/10-bit OR MIPI CSI-2 (2-lane)	✅
Output formats	RAW RGB, RGB565/555/444, YCbCr422, JPEG	✅
Max frame rate	QSXGA@15fps, 1080P@30fps, 720P@60fps, VGA@90fps	✅
Master clock XCLK	6 – 27 MHz (typical 24 MHz)	✅
Autofocus	Integrated VCM driver, contrast-detect AF	✅
Features	OmniBSI, HDR, AE/AG/AWB, 2D NR, lens correction, anti-shake	✅
Current	~140 mA (active), 20 µA (standby)	✅

Source: OV5640 datasheet v2.03 (cdn.sparkfun.com), OV5640 Product Brief (media.digikey.com)

Interface note: OV5640 supports both DVP and MIPI CSI-2. Cheap modules typically use DVP 8-bit parallel. MIPI versions are usually for phone/tablet designs. For AF versions, verify AF_VCC supply — the VCM motor needs separate power.

OV7725 Module

OmniVision high-performance 1/4-inch VGA sensor using OmniPixel² technology, focused on high sensitivity, low noise, and low-light performance. At 6.0 µm pixel size, it's the strongest low-light performer in the 0.3MP class.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV7725	✅
Optical format	1/4 inch	✅
Effective pixels	0.3MP (640 × 480)	✅
Pixel size	6.0 µm × 6.0 µm	✅
Interface	DVP 8/10-bit parallel + SCCB	✅
Output formats	RAW RGB, RGB565/555/444, YCbCr422	✅
Max frame rate	VGA@60fps, QVGA@120fps	✅
Operating temp	-20°C to +70°C	✅
Sensitivity	3800 mV/Lux-sec (extremely high)	✅
Dynamic range	60 dB	✅
Features	OmniPixel², high sensitivity low-light, AE/AG/AWB, edge enhancement, noise suppression	✅

Source: OV7725 official product page (ovt.com/products/ov7725), OV7725 datasheet

⚠️ EOL notice: OV7725 is marked End-of-Life on OmniVision's website. For new production designs, evaluate alternatives. But surplus/secondhand stock remains plentiful.

Why is 6.0µm so big? Both OV7670 and OV7725 are 0.3MP, but OV7670 pixels are only 3.6µm while OV7725 reaches 6.0µm — nearly 3x the pixel area, meaning 3x the light gathering. This is why it crushes same-resolution competitors in low light. For night vision, license plate recognition, or industrial inspection, this is the one.

GC0309-8225N V2 Module

Based on GalaxyCore GC0309, a 0.3MP fixed-focus module with DVP interface, focused on ultra-high cost-effectiveness.

Core Specs

Parameter	Spec	Reliability
Sensor	GalaxyCore GC0309	✅
Optical format	1/9 inch	✅
Effective pixels	0.3MP (640 × 480, actual 648×488)	✅
Pixel size	2.5 µm × 2.5 µm	✅
Pixel structure	4-transistor (4T), low noise	✅
Shutter	Electronic rolling	✅
Interface	DVP 8-bit parallel	✅
Output formats	YCbCr422, RGB565, RAW Bayer	✅
Max frame rate	VGA@30fps (24MHz clock)	✅
Operating voltage	2.8V typical (2.7–3.3V range)	⚠️ see below
Lens	Fixed focus, typical 62° FOV	🔍 module-vendor defined
Features	On-chip 10-bit ADC, embedded ISP, AE, AWB	✅

Source: GC0309 official datasheet V1.0 (2009-12-28, GalaxyCore Inc.)

⚠️ Voltage correction: The original document's "3.3V single supply" is inaccurate. The official datasheet explicitly states "Single power supply requirement (2.8V)", typical 2.8V, range 2.7–3.3V. Recommended supply is 2.8V LDO — 3.3V is within range but not the optimal operating point.

Note: GC0309-8225N V2 is a complete module model customized by module vendors based on the GC0309 sensor. Pin definitions, lens parameters, and FPC interfaces may vary between vendors. Always refer to your specific module's datasheet.

OV7670 Module

OmniVision's classic 1/6-inch VGA sensor — the most historically significant and ecosystem-mature sensor in embedded vision. The de facto standard for Arduino, STM32, and FPGA education.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV7670	✅
Optical format	1/6 inch	✅
Effective pixels	0.3MP (640 × 480)	✅
Pixel size	3.6 µm × 3.6 µm	✅
Shutter	Electronic rolling	✅
Interface	DVP 8-bit parallel + SCCB	✅
Output formats	YUV422, RGB565, RGB555, RAW RGB, Processed Bayer	✅
Max frame rate	VGA@30fps, QVGA@60fps, CIF@60fps	✅
Master clock XCLK	10 – 24 MHz (typical 24 MHz)	✅
Power	~60 mW (15fps VGA YUV), standby <20 µA	✅

Source: OV7670 official datasheet (w2.electrodragon.com/Chip-dat/OmniVision-dat/OV7670-dat/OV7670-ds.pdf)

⚠️ Voltage correction (important): The original's "2.5–3.0V single voltage" is simplified and misleading. OV7670 actually requires three separate supply domains:

Supply Domain Voltage Range Notes

DVDD (digital core) 1.8V Requires dedicated LDO

AVDD (analog) 2.45–3.0V

DOVDD (I/O) 1.7–3.0V Can connect to 3.3V

Purchasing note: Cheap non-FIFO modules typically only expose DOVDD — DVDD and AVDD are stepped down via on-board LDOs. But bare minimum boards may require external regulation. Non-FIFO versions are extremely demanding on MCU real-time performance; with STM32, pair with AL422B FIFO buffer chip.

Supply Domain	Voltage Range	Notes
DVDD (digital core)	1.8V	Requires dedicated LDO
AVDD (analog)	2.45–3.0V
DOVDD (I/O)	1.7–3.0V	Can connect to 3.3V

OV3660 Module

OmniVision 3MP (2048×1536) sensor using OmniBSI backside-illumination technology. Natively supported by ESP32's esp32-camera library.

⚠️ Multiple corrections in this section — read carefully. My original version had errors in interface, frame rate, voltage, and current — all corrected against the official datasheet below.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV3660	✅
Optical format	1/5 inch	✅
Effective pixels	3MP (2048 × 1536)	✅
Pixel size	1.4 µm × 1.4 µm (OmniBSI)	✅
Shutter	Electronic rolling	✅
Interface	DVP 8-bit parallel ONLY	⚠️ see below
Output formats	RAW RGB, RGB565/555/444, YCbCr422, JPEG compression	✅
Max frame rate	QXGA@15fps, 1080P@20fps, 720P@45fps, VGA@60fps	⚠️ see below
Operating voltage	Core 1.5V, I/O 1.8V/2.8V, Analog 2.6–3.0V	⚠️ see below
Current	~98 mA (active), 20 µA (standby)	⚠️ see below
Sensitivity	670 mV/Lux-sec	✅
Dynamic range	70 dB @ 8x gain	✅
Operating temp	-20°C to 70°C	✅
XCLK	6 – 27 MHz	✅

Source: OV3660 official datasheet (download.kamami.pl/p1196945-OV3660%20Datasheet.pdf), ESP32 Camera Driver (github.com/espressif/esp32-camera)

⚠️ Correction 1 — No MIPI: The original claimed OV3660 supports "DVP 8-bit OR MIPI CSI-2 (1-lane)" — this is wrong. The official datasheet only mentions Digital Video Parallel Port with no MIPI interface description. ESP32 can use it precisely because it's DVP.

⚠️ Correction 2 — Frame rate: Original claimed 1080P@30fps, 720P@60fps; actual is 1080P@20fps, 720P@45fps. Original overstated by ~33%.

⚠️ Correction 3 — Voltage: Original "I/O 1.8–3.3V, core 1.2V, analog 2.8V" is wrong. Actual: core 1.5V, I/O 1.8V or 2.8V, analog 2.6–3.0V.

⚠️ Correction 4 — Current: Original "~130mA" is too high; datasheet specifies 98mA active.

Purchasing tip: OV3660 offers 50% higher resolution than OV2640 (3MP vs 2MP) with better low-light performance via BSI. If your project already uses OV2640 and you want to upgrade image quality while staying on ESP32, OV3660 is the best drop-in replacement (both DVP, directly supported by esp32-camera).

GC2053 Module

GalaxyCore 2MP (1920×1080) sensor with 1/2.9-inch large sensor and 2.8µm large pixels, supporting both MIPI and DVP interfaces. Standout low-light performance in its price tier.

Core Specs

Parameter	Spec	Reliability
Sensor	GalaxyCore GC2053	✅
Optical format	1/2.9 inch	✅
Effective pixels	2MP (1920 × 1080)	✅
Pixel size	2.8 µm × 2.8 µm	✅
Shutter	Electronic rolling	✅
Interface	MIPI CSI-2 (2-lane) or DVP	✅
Output formats	RAW Bayer 10-bit / 8-bit	✅
Max frame rate	1080P@30fps, 720P@60fps	✅
Operating voltage	AVDD 2.8V, DVDD 1.2V, IOVDD 1.8V	✅
Current	~57 mA total (analog 17.3 + digital 37.4 + I/O 2.5)	⚠️ see below
ADC	10-bit	✅

Source: GC2053 datasheet V1.1 (2018-11-29, GalaxyCore / CK Vision)

⚠️ Current correction: Original claimed "~95 mA"; datasheet measures approximately 57mA total (analog 17.3mA + digital 37.4mA + I/O 2.5mA). Actual power consumption is lower than original — thermal and power design can be more relaxed. MIPI 2-lane bandwidth of 1.5Gbps/lane 🔍 pending final datasheet confirmation.

Why choose GC2053: The 1/2.9-inch sensor is larger than both OV2640 (1/4") and OV5640 (1/4"), and 2.8µm pixels are also bigger. Bigger sensor + bigger pixels = more light per pixel = better low-light performance. It has a clear advantage over same-price OV solutions for security surveillance.

GC4653 Module

GalaxyCore's high-performance 4MP sensor using BSI backside-illumination technology for enhanced low-light sensitivity. A high-end choice for security cameras and intelligent vision applications.

⚠️ "STARVIS" terminology correction. The original claimed GC4653 uses "STARVIS architecture technology" — this is incorrect. STARVIS is Sony's registered trademark and exclusive technology; GalaxyCore does not use STARVIS. GC4653 actually uses in-house BSI (backside-illuminated) pixel technology for high sensitivity.

Core Specs

Parameter	Spec	Reliability
Sensor	GalaxyCore GC4653	✅
Optical format	1/3 inch	✅
Effective pixels	4MP (2560 × 1440)	✅
Pixel size	2.0 µm × 2.0 µm (BSI)	✅
Shutter	Electronic rolling	✅
Interface	MIPI CSI-2 ONLY (2-lane, RAW12/RAW10)	✅
Output formats	RAW Bayer 12-bit / 10-bit	✅
Max frame rate	4MP@30fps, 1080P@60fps	✅
Operating voltage	AVDD 2.8V, DVDD 1.2V, I/O 1.8V	✅
Current	~120 mA	🔍 pending datasheet
Features	BSI high-sensitivity low-light, HDR, PDAF phase AF, 2D/3D noise reduction	⚠️ see above

Source: GC4653 datasheet (Camemaker), Sipeed Wiki, Sony Semiconductor official site (STARVIS trademark confirmation)

⚠️ ESP32 incompatible: GC4653 is MIPI-only, incompatible with traditional DVP parallel interface. ESP32 series has no MIPI CSI-2 receiver capability and cannot use this sensor. Requires a controller with MIPI CSI-2, such as HiSilicon Hi3516, Ingenic T31, Rockchip RV1106, etc.

Purchasing tip: Confirm your controller has MIPI CSI-2 before choosing GC4653. If you're doing an ESP32 project, don't buy this sensor — you won't be able to light it up.

OV9281 Module

OmniVision 1/4-inch 1MP global shutter sensor using OmniPixel®3-GS technology. Like the IMX273, it's a global shutter sensor, but much cheaper with DVP+MIPI dual interface — the most affordable global shutter option for makers.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV9281	✅
Optical format	1/4 inch	✅
Effective pixels	1MP (1280 × 800)	✅
Pixel size	3.0 µm × 3.0 µm	✅
Shutter type	Global Shutter (OmniPixel®3-GS)	✅
Interface	MIPI CSI-2 (2-lane) + DVP	✅
Output format	RAW8 / RAW10	✅
Max frame rate	120 fps (1280×800), 180fps (VGA)	✅
Operating voltage	AVDD 2.8V, DVDD 1.2V, IOVDD 1.8V	✅
Power	156 mW (active), 150 µA (standby)	✅
S/N ratio	38 dB	✅
Dynamic range	68 dB	✅
Production status	In production	✅
Typical applications	Machine vision, drones, barcode scanning, AR/VR, 3D depth	✅

Source: OmniVision OV9281 Product Brief v1.4 (ovt.com/wp-content/uploads/2024/05/OV9281-PB-v1.4-WEB.pdf)

⚠️ ESP32 compatibility: OV9281 is NOT in the esp32-camera library (classic ESP32/S2/S3). ESP32-P4 has experimental support but unstable (esp-idf #17795). For global shutter on ESP32, there's currently no good option; use Raspberry Pi + Arducam OV9281 module instead.

Why OV9281 over IMX273: Both are global shutter, but OV9281 is much cheaper ($15-25 vs $50+) and supports both DVP and MIPI. Downsides: lower resolution (1MP vs 1.58MP) and monochrome only (no color version).

OV5647 Module

OmniVision 1/4-inch 5MP CMOS sensor with OmniBSI technology. The original Raspberry Pi Camera Module V1 (2013) sensor.

Core Specs

Parameter	Spec	Reliability
Sensor	OmniVision OV5647	✅
Optical format	1/4 inch	✅
Effective pixels	5MP (2592 × 1944)	✅
Pixel size	1.4 µm × 1.4 µm	✅
Shutter type	Rolling shutter	✅
Technology	OmniBSI backside illumination	✅
Interface	MIPI CSI-2 (2-lane) + DVP	✅
Output format	RAW8 / RAW10	✅
Max frame rate	5MP@15fps, 1080P@30fps, 720P@60fps, VGA@90fps	✅
Operating voltage	Analog 2.6–3.0V (typical 2.8V), Core 1.5V, I/O 1.7–3.0V	✅
Production status	❌ End-of-Life (EOL)	⚠️ see below
Typical applications	Raspberry Pi Camera V1, phones, notebooks	✅

Source: OmniVision OV5647 datasheet (v-visiontech.com/web/userfiles/download/OV5647DS.pdf), Raspberry Pi official documentation

⚠️ Discontinued (EOL): OV5647 is on OmniVision's discontinued product list. Surplus stock still available ($8-15), but not recommended for new designs.

Selection comparison: OV5647 vs OV5640 — both 5MP. OV5647 is the Pi V1 sensor; OV5640 is the ESP32 ecosystem 5MP option (with AF). Pins and drivers are incompatible. For Pi projects on a budget, OV5647 surplus still works; for ESP32, use OV5640.

NOT in esp32-camera support list. ESP32 users should use OV2640 or OV5640.

IMX219 Module

Sony 1/4-inch 8MP CMOS sensor with Exmor R backside-illumination technology. The standard sensor for Raspberry Pi Camera Module V2, and the most ubiquitous MIPI camera in the Raspberry Pi ecosystem.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX219	✅
Optical format	1/4 inch	✅
Effective pixels	8MP (3280 × 2464)	✅
Pixel size	1.12 µm × 1.12 µm	✅
Technology	Exmor R BSI (NOT STARVIS)	⚠️ see below
Interface	MIPI CSI-2 (2-lane)	✅
Output format	RAW10	✅
Max frame rate	1080P@30fps, 720P@60fps, VGA@90fps	✅
Operating voltage	AVDD 2.8V, DVDD 1.2V, IOVDD 1.8V	✅
Power	~38 mA @2.8V + ~160 mA @1.2V (≈ 0.26 W)	✅
Production status	In production (at least until Jan 2028)	✅

Source: Sony IMX219 datasheet (dlscorp.com), Raspberry Pi official documentation

⚠️ Exmor R is NOT STARVIS: Some online articles classify IMX219 as STARVIS — this is wrong. IMX219 uses Exmor R (BSI). STARVIS is Sony's later security-focused technology line with higher sensitivity and dynamic range.

Purchasing tip: The IMX219 module (Raspberry Pi Camera V2) is the lowest-cost entry to MIPI. Pi 4B/5 CSI connector works directly, driver is in mainline kernel, plug-and-play.

IMX273 Module

Sony Pregius series global shutter sensor, 1/2.9-inch, 1.58MP. Designed for industrial FA cameras and ITS (Intelligent Traffic Systems), 226fps at full resolution — a high-performance choice for high-speed machine vision.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX273	✅
Optical format	1/2.9 inch	✅
Effective pixels	1.58MP (1456 × 1088)	✅
Pixel size	3.45 µm × 3.45 µm	✅
Technology	Pregius Global Shutter (NOT STARVIS)	⚠️ see below
Interface	MIPI CSI-2 (1-lane) OR Sub LVDS (8-ch)	✅
Output format	RAW10 / RAW12	✅
Max frame rate	226 fps (full resolution, 10-bit)	✅
Operating voltage	3.3V / 1.8V / 1.2V	✅
Typical applications	Industrial FA cameras, ITS, barcode scanning, defect inspection	✅

Source: Sony IMX273 official Flyer (sony-semicon.com/files/62/flyer_industry/IMX273_287_296_297_Flyer.pdf)

⚠️ Global vs Rolling Shutter: Every other sensor in this handbook uses rolling shutter, which causes jello effect on fast-moving objects. IMX273 has a global shutter — the entire image is exposed simultaneously, zero jello effect on high-speed motion. This is the core reason to choose it for industrial vision.

Note: IMX273 comes in two variants — MIPI 1-lane and Sub LVDS 8-ch. Confirm which interface you need before buying.

IMX307 Module

Sony STARVIS 2MP security sensor, 1/2.8-inch, 2.9µm large pixels. HDR support, a mainstream choice for 1080P security cameras, commonly paired with HiSilicon Hi3516 series SoCs.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX307	✅
Optical format	1/2.8 inch	✅
Effective pixels	2.07MP (1920 × 1080)	✅
Pixel size	2.9 µm × 2.9 µm	✅
Technology	STARVIS (original)	✅
Interface	MIPI CSI-2 (2/4-lane) + LVDS	✅
Output format	RAW10 / RAW12	✅
Max frame rate	60 fps (1080P)	✅
HDR	Multi-exposure HDR + Digital overlap HDR	✅
Operating voltage	AVDD 2.9V, DVDD 1.2V, IOVDD 1.8V	✅
Typical applications	Security surveillance, FA cameras	✅

Source: Sony IMX307 official Flyer (sony-semicon.com/files/62/flyer_security/IMX307LQD_LQR_Flyer.pdf)

Highlight: 2.9µm large pixels + STARVIS = excellent night vision. Paired with Hi3516C V300 is the classic security combo.

IMX335 Module

Sony STARVIS 5MP security sensor, 1/2.8-inch. Steps up from IMX307's 2MP to 5MP while maintaining good day/night image quality. Commonly paired with Hi3516 / RV1106.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX335	✅
Optical format	1/2.8 inch	✅
Effective pixels	5.04MP (2592 × 1944)	✅
Pixel size	2.0 µm × 2.0 µm	✅
Technology	STARVIS (original)	✅
Interface	MIPI CSI-2 (2/4-lane)	✅
Output format	RAW10 / RAW12	✅
Max frame rate	60 fps (full resolution, 10-bit)	✅
HDR	Multi-exposure HDR + Digital overlap HDR	✅
Operating voltage	AVDD 2.9V, DVDD 1.2V, IOVDD 1.8V	✅
Common SoC pairing	Hi3516 series, RV1106	✅

Source: Sony IMX335 official Flyer (sony-semicon.com/files/62/flyer_security/IMX335LQN_Flyer.pdf)

Selection comparison: IMX335 vs IMX307 — same 1/2.8" sensor. IMX307 is 2MP/2.9µm (better low light), IMX335 is 5MP/2.0µm (more detail). Choose IMX335 if you need to see far details, IMX307 for night surveillance.

IMX415 Module

Sony STARVIS 4K (8MP) security sensor, 1/2.8-inch. The workhorse sensor for 4K security cameras, with the most mature module ecosystem — Jetson / Raspberry Pi / RV1106 all have ready drivers.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX415	✅
Optical format	1/2.8 inch	✅
Effective pixels	8.29MP (3840 × 2160, 4K UHD)	✅
Pixel size	1.45 µm × 1.45 µm	✅
Technology	STARVIS (original)	✅
Interface	MIPI CSI-2 (2/4-lane)	✅
Output format	RAW8 / RAW10	✅
Max frame rate	4K up to 90fps (typical 30–60fps)	✅
HDR	Multi-exposure HDR	✅
Operating voltage	AVDD 2.9V, DVDD 1.1V, IOVDD 1.8V	✅
Announced	June 2019	✅
Typical applications	4K security, NVR, industrial cameras	✅

Source: Sony IMX415 official Flyer (sony-semicon.com/files/62/pdf/p-12_IMX415-AAQR_AAMR_Flyer.pdf)

Purchasing tip: IMX415 has the best DIY/embedded module ecosystem of all Sony IMX sensors. LuckFox, Radxa, e-con Systems all offer ready-made modules with drivers for Jetson, Raspberry Pi, RV1106. The top pick for 4K MIPI projects.

IMX678 Module

Sony STARVIS 2 4K (8MP) security sensor, 1/1.8-inch large sensor. A 2024 new release that comprehensively upgrades the IMX415 — bigger sensor, newer STARVIS 2 technology, faster frame rate, stronger HDR.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX678	✅
Optical format	1/1.8 inch (much larger than IMX415's 1/2.8")	✅
Effective pixels	8.29MP (3840 × 2160, 4K UHD)	✅
Pixel size	2.0 µm × 2.0 µm	✅
Technology	STARVIS 2 (next-gen)	✅
Interface	MIPI CSI-2 (2/4/8-lane, 4-lane×2ch)	✅
Output format	RAW10 / RAW12	✅
Max frame rate	72 fps (4K, 10-bit) / 60fps (12-bit)	✅
HDR	Digital overlap HDR + Clear HDR	✅
Operating voltage	AVDD 3.3V, DVDD 1.1V, IOVDD 1.8V	⚠️ see below
Announced	2024	✅
Common SoC pairing	Hi3519DV500	✅

Source: Sony IMX678 official Flyer (sony-semicon.com/files/62/flyer_security/IMX678-AAQR_AAQR1_Flyer.pdf, © 2024)

⚠️ Power note: IMX678's AVDD is 3.3V, different from IMX335/IMX307/IMX415's 2.9V. Change the LDO output when migrating from other IMX designs.

IMX678 vs IMX415: Both are 4K 8MP, but IMX678 has a bigger sensor (1/1.8" vs 1/2.8"), bigger pixels (2.0µm vs 1.45µm), newer tech (STARVIS 2), and stronger HDR (Clear HDR). The trade-off is more complex power/lane requirements and higher module cost as a 2024 new release.

IMX766 Module

Sony 50MP smartphone sensor, 1/1.56-inch large sensor, Quad Bayer array. Used in OPPO Find X3 Pro, Reno7 Pro and other flagship phones.

⚠️ NOT recommended for DIY / embedded projects. The following is for reference only.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX766	✅
Optical format	1/1.56 inch	✅
Effective pixels	50MP (8192 × 6144)	✅
Pixel size	1.0 µm (4-in-1 binning → 2.0 µm)	✅
Pixel array	Quad Bayer	✅
Interface	MIPI CSI-2 (4-lane)	✅
Output format	RAW10 / RAW8	🔍
Max frame rate	Full res ~24fps; binned 30fps; 1080P 60–120fps	🔍
Operating voltage	AVDD 2.8V, DVDD 1.1V, IOVDD 1.8V	🔍
Phone implementations	OPPO Find X3 Pro, OnePlus, Realme	✅

Source: OPPO official specs, IADIY module page, DxOMark. ⚠️ Sony has not published an official IMX766 datasheet — some parameters are from phone manufacturers and module vendors.

⚠️ Why not recommended for DIY: No official datasheet (register config and power sequencing require reverse engineering), needs custom driver (no mainline Linux driver), non-standard FPC, 50MP Quad Bayer needs powerful ISP, many modules include OIS motors needing extra drivers. For high-res MIPI, choose IMX415 or IMX477 (Raspberry Pi HQ Camera) instead.

Sony IMX Series Key Parameters Comparison

All 15 sensors in one table would exceed mobile screen width. OV/GC comparison table above; Sony IMX series below.

Parameter	IMX219	IMX273	IMX307	IMX335	IMX415	IMX678	IMX766
Technology	Exmor R	Pregius GS	STARVIS	STARVIS	STARVIS	STARVIS 2	Quad Bayer
Resolution	8MP	1.58MP	2MP	5MP	8MP (4K)	8MP (4K)	50MP
Optical format	1/4"	1/2.9"	1/2.8"	1/2.8"	1/2.8"	1/1.8"	1/1.56"
Pixel size	1.12 µm	3.45 µm	2.9 µm	2.0 µm	1.45 µm	2.0 µm	1.0 µm
Interface	MIPI 2L	MIPI 1L/LVDS	MIPI 2/4L	MIPI 2/4L	MIPI 2/4L	MIPI 2/4/8L	MIPI 4L
Max fps	1080P@30fps	226fps	1080P@60fps	5MP@60fps	4K@90fps	4K@72fps	50MP@24fps
HDR	No	No	✅	✅	✅	✅ Clear	No
Global shutter	No	✅	No	No	No	No	No
AVDD	2.8V	3.3V	2.9V	2.9V	2.9V	3.3V	2.8V
Best use case	Raspberry Pi V2	Industrial FA	Security	Security	4K Security	High-end 4K	⚠️ Phone
DIY recommended	✅	⚠️ Industrial	✅	✅	✅ Top pick	⚠️ Pricey	❌ Not recommended

IMX477 Module

Sony 12.3MP sensor, 1/2.3-inch, with interchangeable C/CS-mount lens. The standard sensor for Raspberry Pi HQ Camera — ideal for projects needing high-quality imaging and interchangeable lenses: microscopy, astronomy, industrial inspection.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX477	✅
Optical format	1/2.3 inch	✅
Effective pixels	12.3MP (4056 × 3040)	✅
Pixel size	1.55 µm × 1.55 µm	✅
Technology	BSI stacked CMOS	✅
Interface	MIPI CSI-2 (2/4-lane)	✅
Output format	RAW12 / RAW10 / RAW8	✅
Max frame rate	Full res@60fps (10-bit), 4K@60fps, 1080P@240fps	✅
HDR	DOL-HDR (Digital Overlap HDR)	✅
Lens mount	C-mount / CS-mount (interchangeable)	✅
Production status	In production (at least until Jan 2030)	✅
Announced	April 2020	✅
Typical applications	Raspberry Pi HQ Camera, microscopy/astronomy, industrial	✅

Source: Sony IMX477 Flyer, Raspberry Pi HQ Camera product page

Purchasing tip: IMX477's biggest advantage is interchangeable lenses — C-mount and CS-mount cover wide-angle to telephoto to macro. Best image quality in the Pi ecosystem. $50-70.

IMX708 Module

Sony 12MP Quad Bayer sensor, 1/2.3-inch, with PDAF phase-detection autofocus and HDR. The standard sensor for Raspberry Pi Camera Module 3 — the first Pi camera with hardware autofocus.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX708	✅
Optical format	1/2.3 inch	✅
Effective pixels	12MP (4608 × 2592)	✅
Pixel size	1.4 µm × 1.4 µm	✅
Technology	BSI stacked CMOS, Quad Bayer	✅
Interface	MIPI CSI-2	✅
Output format	RAW10	✅
Max frame rate	1080P@50fps, 720P@100fps, 480P@120fps	✅
Autofocus	PDAF phase-detection autofocus	✅
HDR	Yes (up to 3MP output)	✅
Production status	In production (at least until Jan 2030)	✅
Announced	January 2023	✅
Typical applications	Raspberry Pi Camera Module 3 (standard/wide/telephoto)	✅

Source: Raspberry Pi Camera Module 3 product page and product brief

Highlight: IMX708 brings hardware autofocus (PDAF) to Raspberry Pi for the first time. Available in standard, wide, and telephoto versions. $25-30 — best price/performance for Pi MIPI cameras.

IMX296 Module

Sony Pregius global shutter sensor, 1/2.9-inch, 1.58MP. The standard sensor for Raspberry Pi Global Shutter Camera. Like the IMX273, it's a Pregius global shutter sensor, but with official Pi ecosystem module and driver support.

Core Specs

Parameter	Spec	Reliability
Sensor	Sony IMX296	✅
Optical format	1/2.9 inch	✅
Effective pixels	1.58MP (1456 × 1088)	✅
Pixel size	3.45 µm × 3.45 µm	✅
Technology	Pregius Global Shutter	✅
Interface	MIPI CSI-2 (1-lane)	✅
Output format	RAW10	✅
Max frame rate	60.3 fps (full resolution, 10-bit)	✅
Operating voltage	AVDD 3.3V, DVDD 1.2V, IOVDD 1.8V	✅
Lens mount	C-mount / CS-mount (interchangeable)	✅
Production status	In production (at least until Jan 2032)	✅
Announced	2023	✅
Typical applications	Raspberry Pi Global Shutter Camera, high-speed capture, industrial	✅

Source: Sony IMX296 Flyer, Raspberry Pi Global Shutter Camera product page

IMX296 vs IMX273: Both are Pregius global shutter, 3.45µm pixels. IMX296 is the official Pi Global Shutter Camera sensor with full driver/docs; IMX273 is more industrial FA focused. For Pi projects choose IMX296, for industrial FA choose IMX273.

Module Comparison Summary

Key Parameters Comparison

Table corrected per official datasheets. Bolded items had errors in the original or need special attention.

Parameter	OV2640	OV5640	OV7725	GC0309	OV7670	OV3660	GC2053	GC4653
Vendor	OmniVision	OmniVision	OmniVision	GalaxyCore	OmniVision	OmniVision	GalaxyCore	GalaxyCore
Resolution	2MP	5MP	0.3MP	0.3MP	0.3MP	3MP	2MP	4MP
Max size	1600×1200	2592×1944	640×480	640×480	640×480	2048×1536	1920×1080	2560×1440
Optical format	1/4"	1/4"	1/4"	1/9"	1/6"	1/5"	1/2.9"	1/3"
Pixel size	2.2 µm	1.4 µm	6.0 µm	2.5 µm	3.6 µm	1.4 µm	2.8 µm	2.0 µm
Interface	DVP	DVP/MIPI	DVP	DVP	DVP	DVP only	DVP/MIPI	MIPI only
Max fps	UXGA@15fps	QSXGA@15fps	VGA@60fps	VGA@30fps	VGA@30fps	1080P@20fps	1080P@30fps	4MP@30fps
Output formats	YUV/RGB/RAW/JPEG	YUV/RGB/RAW/JPEG	YUV/RGB/RAW	YUV/RGB/RAW	YUV/RGB/RAW	YUV/RGB/RAW/JPEG	RAW Bayer	RAW Bayer
Autofocus	No	Yes (VCM)	No	No	No	No	Optional	PDAF
Low-light	Average	Average	Excellent	Average	Average	Good	Excellent	Excellent
Voltage	1.7–3.3V (I/O)	1.7–3.3V (I/O)	3.3V	2.8V typical	1.8V/2.45-3V/1.7-3V	1.5V core multi-rail	1.8V (I/O)	1.8V (I/O)
Typical power	~125 mW	~140 mA	~80 mA	~60 mA	~60 mW	~98 mA	~57 mA	~120 mA
I²C address	0x30	0x3C	0x21	Vendor	0x21	See DS	Vendor	Vendor
Price tier	Low-mid	Mid-high	Mid	Very low	Low	Low-mid	Mid	Mid-high
ESP32 compatible	✅	✅	✅	✅	⚠️ needs FIFO	✅	⚠️ DVP mode only	❌

Selection Recommendations

Education (Arduino/STM32): OV7670 — Richest ecosystem, most tutorials. Note three-rail power.
Wi-Fi streaming (ESP32-CAM): OV2640 — Built-in JPEG, balanced power/quality, de facto standard.
High-res photo (ESP32/STM32): OV5640 — 5MP + autofocus, best entry-level quality. Power VCM on AF versions.
Night vision / machine vision (low-light): OV7725 — 6.0µm pixels, low-light king. ⚠️ EOL, surplus stock.
Ultra low cost (access/POS): GC0309-8225N — Lowest price, meets basic needs. Power at 2.8V.
3MP upgrade (ESP32): OV3660 — BSI, drop-in for OV2640. Note lower frame rate than OV2640.
Security surveillance (indoor/outdoor): GC2053 — Large sensor/pixels, MIPI high-speed, low power.
High-end security / AI vision: GC4653 — 4MP + BSI high sensitivity. ⚠️ MIPI only, not for ESP32.
Raspberry Pi / Jetson entry (MIPI): IMX219 — Pi Camera V2 standard, most mature driver, plug-and-play.
Industrial vision / high-speed capture: IMX273 — Global shutter, 226fps, zero jello effect.
Security (1080P nighttime): IMX307 — 2.9µm + STARVIS, strong night vision. Pair with Hi3516.
Security (high resolution): IMX335 — 5MP STARVIS, more detail. Pair with Hi3516/RV1106.
4K security / NVR: IMX415 — 4K STARVIS, best module ecosystem, top pick.
High-end 4K security (2024): IMX678 — STARVIS 2 + Clear HDR, best image quality. ⚠️ AVDD 3.3V.
⚠️ Don't buy for DIY: IMX766 — 50MP phone sensor, no official datasheet, needs custom driver.

Application & Wiring Guide

ESP32 Typical Wiring

⚠️ Important disclaimer: The pin mapping below is based on common AI-Thinker ESP32-CAM module configurations. There is no official unified pin standard for ESP32-CAM — the esp32-camera library examples explicitly state "pin configurations vary by board". Different vendors (AI-Thinker, Freenove, SunFounder, Espressif official EVB) may all differ. Always verify which board you have before wiring.

AI-Thinker ESP32-CAM common mapping (applies to OV2640 / OV5640 / OV7725):

Module Signal	ESP32 GPIO	Function
XCLK	GPIO 27	Master clock via LEDC PWM at 20 MHz
PCLK	GPIO 22	Pixel clock input
VSYNC	GPIO 25	Frame sync input
HREF	GPIO 23	Line sync input
D0	GPIO 32	Data bit 0
D1	GPIO 35	Data bit 1
D2	GPIO 34	Data bit 2
D3	GPIO 5	Data bit 3
D4	GPIO 39	Data bit 4
D5	GPIO 18	Data bit 5
D6	GPIO 36	Data bit 6
D7	GPIO 19	Data bit 7
SIOC	GPIO 26	SCCB clock
SIOD	GPIO 21	SCCB data
PWDN	GPIO 0	Power-down control (optional)
RESET	GPIO 15	Reset control (optional)

Verification: esp32-camera repo examples/camera_example/main/camera_pinout.h (PCLK/VSYNC/HREF confirmed line-by-line against source).

STM32 Wiring

STM32F4/F7/H7 series feature DCMI (Digital Camera Memory Interface) peripheral for direct hardware reception of DVP data streams:

Connect D0–D7 to DCMI_D0–DCMI_D7 corresponding GPIOs
PCLK → DCMI_PIXCK
HSYNC → DCMI_HSYNC
VSYNC → DCMI_VSYNC
I²C_SCL/SDA → any I²C peripheral
Enable DCMI + DMA transfer, target buffer in SRAM or SDRAM

Power Design

Dedicated regulation: Camera modules can draw 100–150 mA 🔍 during capture (varies significantly by sensor — OV5640/GC4653 higher, GC2053/OV7670 lower). Use a dedicated LDO (e.g., AMS1117-3.3) to avoid voltage sag from sharing MCU regulator.
Decoupling caps: Place 10 µF + 100 nF parallel decoupling caps near the module VCC pin, as close to the FPC connector as possible. Standard practice.
Signal integrity: Keep DVP trace length within 10 cm; for longer runs, add 22–33 Ω series termination resistors on PCLK and data lines. ⚠️ Low-speed DVP (PCLK < 30MHz) may not need termination — depends on actual PCB layout.

Troubleshooting

Symptom	Likely Cause	Fix
No image, no PCLK waveform	XCLK missing or wrong frequency	Measure XCLK with scope, confirm 6–24 MHz
Wrong colors / color cast	Output format mismatch with MCU decoder	Check register config, confirm RGB565 or YUV422
Image banding / jitter	Power ripple or PCLK interference	Improve power filtering, shorten data lines
SCCB config failure	Wrong I²C address or no pull-ups	Check address table in this doc, add 4.7kΩ pull-ups
Low frame rate	Low XCLK or resolution too high	Raise XCLK to 24 MHz, reduce resolution
Blurry image (OV5640 AF)	VCM unpowered or AF registers unconfigured	Verify AF_VCC supply, initialize AF firmware
OV7670 garbled	DVDD not separately powered	Confirm on-board 1.8V LDO output

Practical Purchasing Advice

Lessons learned from my own mistakes:

Choose controller first, camera second. ESP32 only does DVP — choose OV2640/OV5640/OV7725/OV3660. Only MIPI-capable controllers (RV1106, Hi3516) can use 4MP sensors like GC4653. Your controller's interface defines your options.
Check if the module has FIFO. OV7670 comes in versions with and without AL422B FIFO. Running non-FIFO on bare STM32 is nearly impossible for normal imaging. The cheap tier is usually non-FIFO.
AF versions need AF_VCC pin. OV5640 AF modules need separate VCM motor power. Some cheap modules merge AF power with main supply, some don't expose it. Check pinout before buying.
Voltage mismatch is the most common board-frying cause. Don't just connect 3.3V blindly. GC0309's optimal point is 2.8V, OV7670 core needs 1.8V. When unsure, check the voltage table in this doc or read the datasheet.
EOL parts (OV7725) carry inventory risk. Officially discontinued; still available on Taobao/LCSC now, but not recommended for new mass production. Fine for prototyping.
GalaxyCore datasheets are harder to find than OmniVision. If Chinese channels don't have GC-series datasheets, try galaxycore.com.cn official site, or module vendor pages like MuchVision, Camemaker. Sipeed Wiki is also a good supplemental source.

Sony IMX Series Purchasing Supplement

The purchasing advice above focuses on OV/GC series (DVP ecosystem). Here are tips specific to Sony IMX series (MIPI ecosystem):

Where to buy IMX modules: IMX219 is widely available on Taobao/LCSC (Pi V2 compatibles). IMX415 modules from LuckFox, Radxa, Camemaker are recommended. IMX335/IMX307 are mostly found through security solution providers. IMX678 is a 2024 new release, modules are still scarce.
Don't buy phone salvage parts for embedded use. Phone sensors like IMX766 are cheap on Taobao (50-300 RMB), but have no datasheet, no driver, and non-standard FPC. You'll most likely waste time trying to light them up.
Watch the STARVIS generation. Sony officially defines three generations: STARVIS (original, 2014), STARVIS 2 (2021, wider dynamic range), STARVIS 3 (latest, LOFIC structure). Confirm which generation you're getting.
Global shutter is only in the Pregius line. If your project involves high-speed motion capture (conveyor belts, sports analysis, 3D scanning), only Pregius global shutter sensors like IMX273 can avoid jello effect. All other IMX and OV/GC sensors are rolling shutter.

References

All parameters in this article verified against the following official datasheets and authoritative sources:

OmniVision official product pages — OV2640 / OV5640 / OV7725 / OV7670 / OV3660 (ovt.com)
OV2640 datasheet — uctronics.com/download/OV2640_DS.pdf
OV5640 datasheet v2.03 — cdn.sparkfun.com/datasheets/Sensors/LightImaging/OV5640_datasheet.pdf
OV5640 Product Brief — media.digikey.com/pdf/Data%20Sheets/OmniVision%20PDFs/OV5640_PB_3-4-11.pdf
OV7725 datasheet — pdf.datasheet.live/10e069da/ovt.com/OV07725-V28A.pdf
OV7670 datasheet — w2.electrodragon.com/Chip-dat/OmniVision-dat/OV7670-dat/OV7670-ds.pdf
OV3660 datasheet — download.kamami.pl/p1196945-OV3660%20Datasheet.pdf
GC0309 datasheet V1.0 (2009-12-28) — GalaxyCore Inc.
GC2053 datasheet V1.1 (2018-11-29) — GalaxyCore / CK Vision
GC4653 datasheet — Camemaker; Sipeed Wiki
Espressif esp32-camera driver source — github.com/espressif/esp32-camera
Sony Semiconductor Solutions — STARVIS trademark confirmation (STARVIS is Sony-exclusive tech)
Linux Kernel ov7670.c — I²C address definition (android.googlesource.com)
SCCB Functional Specification v2.2 — protocol spec
Sony IMX219 datasheet — dlscorp.com/wp-content/uploads/2018/04/Sony-IMX219-Datasheet.pdf
Sony IMX273 official Flyer — sony-semicon.com/files/62/flyer_industry/IMX273_287_296_297_Flyer.pdf
Sony IMX307 official Flyer — sony-semicon.com/files/62/flyer_security/IMX307LQD_LQR_Flyer.pdf
Sony IMX335 official Flyer — sony-semicon.com/files/62/flyer_security/IMX335LQN_Flyer.pdf
Sony IMX415 official Flyer — sony-semicon.com/files/62/pdf/p-12_IMX415-AAQR_AAMR_Flyer.pdf
Sony IMX766 — OPPO official specs / IADIY module page / DxOMark
Sony STARVIS technology — sony-semicon.com/en/technology/security/
Raspberry Pi Camera documentation — raspberrypi.com/documentation/accessories/camera.html

Building a Privacy-First Open Source ESP32-S3 Camera with RTSP for MiBee NVR

ZhengZhiCong — Sat, 13 Jun 2026 12:12:57 +0000

Introduction

Like many self-hosted enthusiasts, I've been frustrated with the state of affordable IP cameras for years. Most cheap cameras:

Phone home to unknown servers
Require cloud subscriptions for basic features
Don't support standard RTSP streams
Run closed-source firmware you can't audit

So I built seeed-esp32s3-cam - a 100% open source, privacy-first camera firmware that works natively with MiBeeNvr, Blue Iris, and every self-hosted NVR system.

The Hardware

This firmware is optimized for the Seeed Studio ESP32-S3 Sense board - an incredibly capable camera module for around $15 USD.

Specs:

ESP32-S3 dual-core processor
OV2640 2MP camera sensor
Built-in WiFi
Microphone support
SD card slot

Perfect for monitoring garages, sheds, entrances, or anywhere you want an inexpensive camera without sacrificing privacy.

✨ Key Features

Feature	Status
📹 Native RTSP Server	✅ Works with MiBeeNvr / Blue Iris / VLC
🌐 HTTP MJPEG Streaming	✅ Low latency browser preview
🔍 On-device Motion Detection	✅ Configurable sensitivity
🌙 Night Mode	✅ Auto exposure adjustment
⚙️ Web Configuration UI	✅ Full settings via browser
⚡ Pre-built Binaries	✅ Flash in 2 minutes, no compiling
🔒 100% Local Only	✅ No cloud, no phone home
📄 MIT Licensed	✅ Open source forever

🚀 Getting Started in 5 Minutes

1. Flash the Firmware

No compiling needed! Use the pre-built binaries:

Download the latest release from GitHub
Flash using esptool.py or your favorite flasher:

esptool.py --chip esp32s3 write_flash 0x0 firmware.bin

2. Configure WiFi

Connect to the WiFi hotspot ESP32-CAM-XXXX
Open 192.168.4.1 in your browser
Enter your WiFi credentials
The camera will reboot and connect to your network

3. Access the Web UI

Find your camera's IP address in your router, then open it in a browser. You'll see:

Live video preview
Stream quality settings
Motion detection configuration
Network settings

🎯 MiBeeNvr Integration (The Good Stuff)

This is why I built this - native MiBeeNvr compatibility.

Add this to your MiBeeNvr configuration:

cameras:
  esp32_cam:
    url: rtsp://your-camera-ip:8554/stream
    name: Garage Camera
    record: true
    detect_motion: true

That's it! No hacks, no custom scripts, just standard RTSP. Your camera will show up instantly in the MiBeeNvr web UI.

MiBeeNvr GitHub: https://github.com/Mi-Bee-Studio/MiBeeNvr

🔧 Technical Details

Built with ESP-IDF v5.x using:

Hardware-accelerated JPEG encoding
Optimized memory management for 24/7 operation
Lightweight RTSP server implementation
Non-blocking WiFi handling

The firmware is designed to run stable indefinitely - I've had cameras running for months without a single reboot.

📖 Full Documentation

For a complete step-by-step guide including:

Advanced MiBeeNvr configuration
Performance tuning
Troubleshooting common issues
Motion detection optimization

Check out the full tutorial:
👉 https://blog.mickeyzzc.tech/en/posts/iot/esp32s3-cam-monitor/

💻 Both Projects Are Open Source

Camera Firmware: https://github.com/Mi-Bee-Studio/seeed-esp32s3-cam

MiBeeNvr NVR Server: https://github.com/Mi-Bee-Studio/MiBeeNvr

PRs, feature requests, and bug reports are all welcome! This is a community project - let's build the best open source camera + NVR ecosystem together.

Why This Matters

In an era where every IoT device seems to want your data, having fully open, local-only options is more important than ever. This camera + NVR stack:

✅ Never sends data anywhere
✅ You can audit every line of code
✅ You control all configuration
✅ Works with your existing self-hosted tools

No subscriptions, no accounts, no tracking. Just a complete surveillance system that works.

What's Next?

I'm actively working on:

Auto-discovery between cameras and MiBeeNvr
MQTT support for Home Assistant
SD card recording on cameras
Two-way audio
ONVIF support

Let me know what features you'd like to see!

VictoriaMetrics Stream Aggregation: A Three-Year Retrospective (2026)

ZhengZhiCong — Sun, 07 Jun 2026 22:56:28 +0000

It's been exactly three years since the first article in this series was published in March 2023. The VictoriaMetrics ecosystem has changed dramatically since then. Let's revisit the problems we laid out, see what the official project has resolved, and assess where our custom stream-metrics-route gateway stands today.

I. The Problems We Identified in 2023

Here's a quick recap of the core issues from the original post:

#	Problem	2023 Status
P1	Collection gap inflation	Network jitter or performance issues cause time gaps that inflate stream aggregation deltas
P2	Single-node compute limits	Stream aggregation has no historical state, fast but single-instance bottlenecked
P3	Distributed task allocation	Which compute node should each sample be assigned to?
P4	Out-of-order discarding for same-dimension metrics	Multiple nodes computing the same dimension with different time windows causes later values to be discarded
P5	Resource balancing	Uneven load across distributed compute nodes
P6	Task ID dimension explosion	Stream aggregation inserts node IDs into aggregated time series — the label cardinality grows with every node you add

To address these, we built stream-metrics-route, a Go-based distributed stream aggregation gateway.

II. Three Years Later — What the Official Project Has Done

I reviewed VictoriaMetrics changelogs from v1.86 through v1.138.0 and the official documentation. Here's the scorecard.

✅ Perfectly Resolved

Issues P3, P5: Distributed Task Allocation & Resource Balancing

Official solution: vmagent now natively supports -remoteWrite.shardByURL with consistent hashing sharding.

Starting from v1.86, basic shardByURL was introduced. v1.138.0 (March 2026) was the real milestone — it upgraded the data distribution algorithm from round-robin to consistent hashing, which significantly reduces data redistribution ratios during node changes.

The architecture evolution looks like this:

┌─────────────────┐     ┌─────────────────┐
│ Prometheus      │     │ Prometheus      │
│ Agent 1         │     │ Agent 2         │
└────────┬────────┘     └────────┬────────┘
         │ remote write          │ remote write
         ▼                       ▼
┌─────────────────────────────────────────┐
│           vmagent Cluster               │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐│
│  │vmagent-0 │  │vmagent-1 │  │vmagent-2 ││
│  └─────┬────┘  └─────┬────┘  └─────┬────┘│
│        │             │             │     │
│        └─────────────┼─────────────┘     │
│                      ▼                   │
│           ┌──────────────────┐           │
│           │ Consistent Hash │           │
│           └────────┬─────────┘           │
└────────────────────┼─────────────────────┘
                     │ shard by hash
         ┌───────────┼───────────┐
         ▼           ▼           ▼
   ┌──────────┐ ┌──────────┐ ┌──────────┐
   │vmstorage │ │vmstorage │ │vmstorage │
   │    -0    │ │    -1    │ │    -2    │
   └──────────┘ └──────────┘ └──────────┘

The VictoriaMetrics blog provides specific algorithm recommendations. Combined with VictoriaMetrics Operator, you can manage shards via shardCount.

Issue P2: Single-Node Compute Scaling

vmagent now supports horizontal scaling natively with replicas + shardCount, including HA support — see Issue #5573.

Out-of-Order / Delayed Data Accuracy (P1 — Partial Mitigation)

v1.112.0 (February 2025) was a key release, adding Aggregation Windows — dual-window buffering for histogram and rate calculations. Instead of flushing immediately, the output is delayed by a samples_lag window, which significantly improves accuracy for late-arriving data. The tradeoff: roughly doubled memory usage (maintaining two aggregation windows simultaneously).

How it works:

Collector          vmagent           VictoriaMetrics
   │                  │                    │
   │  sample1 @T0    │                    │
   ├─────────────────►│  Write to          │
   │                  │  Window A (current)│
   │                  │                    │
   │  sample2 @T1    │                    │
   │  (delayed)      │                    │
   ├─────────────────►│  Write to          │
   │                  │  Window B (previous)│
   │                  │                    │
   │                  │  Aggr result A @T2 │
   │                  ├────────────────────►│
   │                  │  Aggr result B @T3 │
   │                  ├────────────────────►│

See the official docs on streaming aggregation windows.

❌ Still Unresolved

True Distributed Stream Aggregation Coordination

vmagent's stream aggregation is single-instance. There is no coordination mechanism between instances — if two vmagent instances aggregate the same metric, you get duplicate or conflicting output. The official recommendation is to use without/by label clauses to divide responsibility between instances, rather than providing a cross-instance coordination protocol.

Task ID Dimension Explosion (P6)

Official vmagent still inserts internal labels (such as _aggr-related labels) into aggregated time series, but lacks a stream_task_id pre-marking plus dimension control design.

III. stream-metrics-route: Current Status and Value

Core Code Architecture

File	Role
`router.go`	Routing core — filters metrics based on relabel rules
`remotecluster.go`	Dual hashmod scheduling core
`remotewrite.go`	Remote write HTTP client
`kafka.go`	Kafka producer

The Core Algorithm (from `remotecluster.go`)

// Dual hashmod scheduling
hash := sortLabelsHashKey(ts.Labels)
dime := hashMod(r.dimension, hash) // First hashmod → task partition ID

ts.Labels = append(ts.Labels, prompb.Label{
    Name:  "stream_task_id",
    Value: strconv.Itoa(dime), // Insert stream_task_id label
})

hashnode = sortLabelsHashKey(filterLabels) // Second hashmod → node selection
tmpch := hashMod(r.uplen, hashnode)       // Which backend writer to send to

Is It Still Needed in 2026?

Yes — but with an adjusted role. The positioning should shift from "full stream aggregation gateway" to "metric distribution routing gateway + Kafka integration layer." The core differentiated value:

Dual hashmod scheduling + stream_task_id pre-injection — tags metrics at the gateway layer, so all downstream nodes route consistently by this ID. This solves dimension control earlier in the pipeline than the official approach.
Multi-backend async distribution — supports async distribution to both Kafka and remote write, solving the "synchronous forwarding blocks the time window" problem from the original post.
Native Prometheus relabeling integration — works with standard Prometheus relabel configs, no custom syntax to learn.

IV. Recommended 2026 Hybrid Architecture

┌─────────────────┐   ┌──────────────────┐
│  Prometheus     │   │  Business System │
│  Agent Cluster  │   │  Metrics (Kafka) │
└────────┬────────┘   └────────┬─────────┘
         │                     │
         ▼                     ▼
    ┌──────────────────────────────┐
    │   stream-metrics-route       │
    │   (Routing Layer)            │
    │   - Dual hashmod scheduling  │
    │   - stream_task_id injection │
    │   - Relabeling               │
    └──────┬──────┬──────┬─────────┘
           │      │      │
    task=0 │ task=1│task=2│
           ▼      ▼      ▼
    ┌─────────────────────────┐
    │   vmagent Cluster       │
    │  (v1.112.0+ with        │
    │   aggregation windows)  │
    └──────────┬──────────────┘
               │
               ▼
    ┌──────────────────┐      ┌────────────┐
    │  VictoriaMetrics │      │   Kafka    │
    │  (Storage)       │      │   (Topic)  │
    └──────────┬───────┘      └────────────┘
               │
               ▼
    ┌──────────────────┐
    │  vmalert         │
    │  Grafana         │
    └──────────────────┘

Key Configuration

vmagent version requirement: >= v1.112.0, with aggregation windows enabled:

# stream aggregation config
- match: 'http_request_duration_seconds_bucket'
  interval: 5m
  without: [instance]
  enable_windows: true   # Critical! Enables dual-window buffering
  outputs: [rate_sum]

V. Evolution Recommendations

Short-term

Action	Description
Upgrade vmagent to >= v1.112.0	Enable `enable_windows: true` to improve histogram aggregation accuracy
Evaluate stream-metrics-route necessity	If you have no Kafka requirement or high-cardinality `stream_task_id` control need, consider migrating away

Medium-term

Action	Description
Retain stream-metrics-route as front-end routing only	Keep hashmod task allocation + Kafka distribution; remove aggregation responsibility
Disable raw metric persistence	Write only stream-aggregated results to storage to reduce volume
Add metadata management module	The `ruler-handle-process` from the original post (dynamic Record Rule by dimension) is worth building or contributing

Long-term

Action	Description
Contribute `stream_task_id` dimension control upstream	If the dual hashmod design proves out in production
Improve monitoring metrics	Add business-level metrics — queue depth per routing rule, distribution latency, etc.

Putting It All Together

Dimension	Assessment
Problem resolution rate	~50% — 2 of 4 core problems resolved via official upgrades; 2 still need custom solutions
Is stream-metrics-route still needed?	Yes — repositioned as "metric distribution routing gateway + Kafka integration layer"
Recommended architecture	Prometheus → stream-metrics-route → vmagent v1.112.0+ → VictoriaMetrics Storage

Three years is a long time in the observability space. The VictoriaMetrics ecosystem has matured significantly — consistent hashing, aggregation windows, and native sharding all address problems that required custom tooling in 2023. But the hard problems around true distributed stream aggregation coordination and dimension control at the gateway layer remain open.

If you're running a similar stack, the hybrid approach — letting the official project handle what it's good at (single-node aggregation, storage) while keeping custom routing for what it isn't (distributed coordination, dimension pre-injection, Kafka bridging) — has proven to be the right call for us.

Originally published on my blog

Scaling VictoriaMetrics Stream Aggregation: A Deep Dive into Distributed Design（2023）

ZhengZhiCong — Sun, 07 Jun 2026 22:50:31 +0000

VictoriaMetrics' stream aggregation is a powerful feature for reducing metric cardinality in real time. But when you push it to millions of time series across a distributed fleet, the native implementation starts to show cracks.

This post walks through what I found analyzing the stream aggregation source code, the real-world problems that emerged at scale, and the distributed gateway we built to solve them.

Community VM Stream Aggregation — Capability Analysis

Stream aggregation was integrated into vmagent starting from version 1.86 (GitHub issue #3460). Let's look at what it actually does under the hood.

Core Computation: The `pushSample` Function

The heart of stream aggregation lives in the pushSample function. Here's the simplified logic:

func (as *totalAggrState) pushSample(inputKey, outputKey string, value float64) {
    currentTime := fasttime.UnixTimestamp()
    deleteDeadline := currentTime + as.intervalSecs + (as.intervalSecs >> 1)

again:
    v, ok := as.m.Load(outputKey)
    if !ok {
        v = &totalStateValue{
            lastValues: make(map[string]*lastValueState),
        }
        vNew, loaded := as.m.LoadOrStore(outputKey, v)
        if loaded {
            v = vNew
        }
    }
    sv := v.(*totalStateValue)
    sv.mu.Lock()
    deleted := sv.deleted
    if !deleted {
        lv, ok := sv.lastValues[inputKey]
        if !ok {
            lv = &lastValueState{}
            sv.lastValues[inputKey] = lv
        }
        d := value
        if ok && lv.value <= value {
            d = value - lv.value
        }
        if ok || currentTime > as.ignoreInputDeadline {
            sv.total += d
        }
        lv.value = value
        lv.deleteDeadline = deleteDeadline
        sv.deleteDeadline = deleteDeadline
    }
    sv.mu.Unlock()
    if deleted {
        goto again
    }
}

The core idea: each incoming sample's value is compared against the last seen value for that time series. The delta is accumulated into a running total. This works well for counters that monotonically increase, but there's a critical detail — the time window logic is simple periodic checking with no sophisticated handling for delayed or out-of-order arrivals.

What Stream Aggregation Looks Like in Practice

In theory, stream aggregation should cleanly reduce high-cardinality metrics down to manageable summaries. In practice, the picture is messier:

Ideal model: every sample arrives on time, windows align perfectly, aggregation is lossless
Reality: samples arrive late, retries flood old data, gaps appear from network or service issues

Common Issues with Native Stream Aggregation

The Collection Gap Problem

Collection gaps are inevitable. Network blips, service restarts, GC pauses — any of these can cause a gap in metric collection. For high-precision stream aggregation, gaps create a specific failure mode:

When a counter's last tracked value is 1000, and a gap causes the next received value to be 5000, the delta is 4000 — which may span an unknown number of actual increments. If the gap occurred within a single aggregation window, the inflated value corrupts that window's result. If the gap crosses window boundaries, you get compounding errors in downstream calculations.

Distributed Computing Challenges

Stream aggregation doesn't persist historical data, so it's fast — but even the fastest service has single-node limits. When you need to scale horizontally, a cascade of new problems appears:

Is vmagent's built-in collection viable at scale?

In testing, enabling vmagent shard + replica collection with real-time stream aggregation caused significant resource spikes. At very large scales, collection gaps became more frequent, which amplified the calculation errors from gaps rather than mitigating them.

Which compute node should each sample be assigned to?

Without a consistent routing strategy, the same metric dimension gets split across nodes, producing partial results that can't be safely combined.

How do you handle the same dimension set being computed by multiple nodes with different values?

When multiple nodes produce results for the same dimension within the same time window, VictoriaMetrics triggers its out-of-order handling logic — which discards later values. You lose data silently.

How do you balance resources in distributed computation?

Uneven distribution means some nodes are overloaded while others sit idle.

What about the new dimensions introduced by routing?

If you insert a task ID to differentiate compute nodes, you've just added a new dimension that grows with every node you add — defeating part of the purpose of aggregation.

Design and Implementation: A Distributed Stream Aggregation Gateway

After analyzing these problems, it became clear that a frontend module was needed to address them at the entry point. Since vmgateway is an enterprise component, we built our own: vm-receive-route, a distributed stream aggregation gateway.

Key Insight from Source Code

Two aspects of the native implementation are particularly relevant:

Time window range:

currentTime := fasttime.UnixTimestamp()
deleteDeadline := currentTime + as.intervalSecs + (as.intervalSecs >> 1)

The window has a 50% grace period (intervalSecs >> 1) beyond the configured interval. This is the only protection against late-arriving data — and it's quite generous, which means stale data can still influence results within that extended window.

Calculation logic:

lv, ok := sv.lastValues[inputKey]
if !ok {
    lv = &lastValueState{}
    sv.lastValues[inputKey] = lv
}
d := value
if ok && lv.value <= value {
    d = value - lv.value
}
if ok || currentTime > as.ignoreInputDeadline {
    sv.total += d
}

The aggregation only checks if a previous value exists and is smaller — it doesn't handle the case where a late-arriving sample with a lower value (after a restart, for example) creates a negative delta that's simply ignored. There's no deduplication, no gap detection, no intelligent merging.

What the Gateway Solves

1. Asynchronous Processing

Most remote write adapters (like prometheus-kafka-adapter) do synchronous forwarding — they wait for the downstream (Kafka, etc.) to acknowledge before accepting the next batch. Stream aggregation has window constraints: if the write pipeline blocks and samples arrive late, they miss their window and calculations drift.

The gateway decouples ingestion from forwarding using an internal buffer. The remote write endpoint returns immediately, and samples are forwarded asynchronously to the stream aggregation backend. This prevents back-pressure from creating cascading delays.

2. Time Window Filtering

Since stream aggregation already computes deltas between successive values, there's no need for complex out-of-order handling at this layer. The gateway simply cooperates with the aggregation window:

Samples arriving within the window → forward normally
Samples arriving outside the window → discard

This solves two problems at once:

Prometheus retries that send large backlogs of old samples no longer corrupt real-time results
The resource overhead of processing those retries is eliminated at the gateway level — the aggregation backend never sees them

3. Dimension Control

The stream aggregation component inserts a node ID into each aggregated time series to distinguish labels across compute nodes. But as nodes scale horizontally, the cardinality of that label scales too. You need a way to:

Control dimension growth from node identities
Route time series by dimension to the correct node

We designed a dual hashmod scheduling algorithm:

The gateway assigns a hash-based task ID to each time series based on its stable dimensions (not the node identity)
The same series always routes to the same compute node, regardless of which gateway instance processed it
The task ID dimension is bounded by the number of unique dimension combinations, not the number of gateway nodes

By moving the task ID labeling to the gateway layer, we eliminated the unbounded dimension growth that horizontal scaling would otherwise cause.

4. Backend Service Migration for Failures

When a compute node fails, its in-memory aggregation state is lost. The gateway detects failures via health checks and reroutes traffic to healthy nodes. Since the dual hashmod ensures consistent routing, the remaining nodes can immediately pick up the work, though there will be a brief period of incomplete aggregation until the state rebuilds.

Record Rule Dimension Task Generator

Stream aggregation is great for reducing cardinality of single metrics. But real-world monitoring scenarios require combining multiple metrics with functions — which is where Prometheus Record Rule comes in.

The Problem with Record Rules at Scale

Consider an HTTP request metric with a req_path dimension:

Before stream aggregation:

a_http_req_total{zone="bj", src_svr="192.168.1.2", src_port="30021", dis_svr="192.168.2.3", code="202", req_path="/api/foo?abc=xyz"}
a_http_req_total{zone="bj", src_svr="192.168.1.2", src_port="30023", dis_svr="192.168.2.3", code="202", req_path="/api/bar?abc=def"}
a_http_req_total{zone="bj", src_svr="192.168.1.2", src_port="10021", dis_svr="192.168.2.3", code="202", req_path="/api/baz?abc=ghi"}
...

After stream aggregation (dropping req_path and src_port):

agg_a_http_req_total{zone="bj", src_svr="192.168.1.2", dis_svr="192.168.2.3", code="202"}
agg_a_http_req_total{zone="bj", src_svr="192.168.1.2", dis_svr="192.168.2.3", code="500"}
agg_a_http_req_total{zone="bj", src_svr="192.168.1.2", dis_svr="192.168.2.3", code="400"}

But what you actually want to display is the success rate per target:

sum by (dis_svr) (
    rate(a_http_req_total{code=~"2.*"}[5m])
)
/
sum by (dis_svr) (
    rate(a_http_req_total{}[5m])
)

You can use Record Rule to precompute this:

groups:
  - name: a_http_req_total:sum:rate:5m
    rules:
      - expr: sum by (src_svr, dis_svr, code) (rate(a_http_req_total{}[5m]))
        record: a_http_req_total:sum:rate:5m

The problem? Record Rule loads all dimensions of the metric into memory. When dimension counts reach critical thresholds, it triggers OOM. Even below that threshold, higher cardinality means slower computation.

In production, istio_requests_total QPS could be delayed by 20 minutes at high dimension counts. After applying stream aggregation to reduce from tens of millions of time series down to tens of thousands, the delay dropped to 1-2 minutes — better, but still far from real-time.

Dynamic Dimension-Split Record Rules

The issue is that Record Rule evaluates one query per group, loading everything into memory. But if you split the query by specific dimension values, you can process them concurrently.

The static approach after stream aggregation:

groups:
  - name: agg_a_http_req_total:sum:rate:5m-2xx
    rules:
      - expr: sum by (src_svr, dis_svr, code) (rate(agg_a_http_req_total{code=~"2.*"}[5m]))
        record: agg_a_http_req_total:sum:rate:5m
  - name: agg_a_http_req_total:sum:rate:5m-4xx
    rules:
      - expr: sum by (src_svr, dis_svr, code) (rate(agg_a_http_req_total{code=~"4.*"}[5m]))
        record: agg_a_http_req_total:sum:rate:5m
  - name: agg_a_http_req_total:sum:rate:5m-5xx
    rules:
      - expr: sum by (src_svr, dis_svr, code) (rate(agg_a_http_req_total{code=~"5.*"}[5m]))
        record: agg_a_http_req_total:sum:rate:5m

But the problem is that in production, dimension labels are dynamic. You can't hardcode splits for every dimension value. You need a label metadata management system that watches dimension combinations and dynamically generates split queries.

The `ruler-handle-process` Component

We built a small metadata watch and rule builder that automates this. Its configuration looks like:

recode_rules:
  - interval: 5m
    recode_to: istio_requests_total:sum:rate:5m
    metric_name: istio_requests_total
    aggr_type: sum
    vector_type: rate
    vector_range: 5m
    group_by_and_filter:
      - source_workload
      - destination_workload
      - cluster
      - namespace
    group_by:
      - response_code
      - namespace
      - source_workload_namespace
      - destination_workload_namespace
      - destination_service_name
      - cluster
      - reporter
    filter_by:
      cluster: "k8s-hw-bj-xxxxxx"
    with_out:
      source_workload: "ingressgateway-workflows"

The component watches the actual dimension combinations under the metric name istio_requests_total and generates a set of Record Rule configurations — one per unique dimension combination. Combined with Prometheus's Rule component for concurrent evaluation, this reduced computation latency from minutes to seconds.

The generated rules look like:

groups:
  - name: istio_requests_total:sum:rate:5m-7218756fe8a0bc327e818812cefb02f7
    rules:
      - expr: sum by (...) (rate(istio_requests_total{cluster="k8s-hw-bj-1-prod", destination_workload="skyaxe-778-flink", ...}[5m]))
        record: istio_requests_total:sum:rate:5m
  - name: istio_requests_total:sum:rate:5m-8e30244048f8d5519a6332f309578ed4
    rules:
      - expr: sum by (...) (rate(istio_requests_total{cluster="k8s-hw-bj-1-prod", destination_workload="t-bean-portal", ...}[5m]))
        record: istio_requests_total:sum:rate:5m

Each unique dimension combination gets its own rule group, enabling true concurrent computation.

Architecture Combinations for Different Scales

Using community open-source components alongside our custom gateway and rule builder, we assembled a tiered architecture that handles everything from small deployments to massive fleets.

Small Scale: Tens of Thousands of Single Metrics

Minimal setup — vmagent with built-in stream aggregation. No gateway needed. The single-node limits aren't reached yet.

Medium Scale: Tens of Thousands of Multi-Metrics

Add Record Rules for computed metrics. Use the dimension-split approach to keep computation fast. Stream aggregation reduces cardinality before Record Rule evaluation.

Large Scale: Millions+ of Single Metrics

This is where the distributed gateway comes in:

Multiple vmagent instances behind the gateway for horizontal ingestion
Dual hashmod scheduling ensures consistent routing
Time window filtering at the gateway prevents retries from polluting results
Asynchronous forwarding prevents back-pressure from creating cascading delays

Large Scale: Scenario-Based Computational Aggregation for Millions of Metrics

The full stack:

Stream aggregation — first pass cardinality reduction at ingestion
Distributed gateway — routing, filtering, and load distribution
Dynamic rule builder — watches dimension metadata and generates concurrent Record Rule groups
Rule engine — evaluates split queries in parallel

Summary

VictoriaMetrics' stream aggregation is a solid foundation, but it was designed for single-node operation. When you need to scale horizontally, you encounter a series of interconnected problems — collection gaps, dimension explosion, record rule performance bottlenecks — that the native implementation doesn't address.

The solutions aren't particularly complex individually (async processing, time window filtering, hash-based routing, dimension-aware rule generation), but they need to work together as a coherent system. The gateway pattern — intercepting the remote write path before it reaches the aggregation layer — proved to be the right abstraction point for injecting these capabilities without modifying upstream code.

The ruler-handle-process component was the second key insight: rather than fighting Prometheus Record Rule's single-query-per-group limitation, we embraced it by dynamically splitting queries by dimension and running them concurrently. This turned a 20-minute computation into a seconds-level operation.

If you're running VictoriaMetrics at scale and hitting these issues, the patterns described here should be applicable regardless of your specific stack. The gateway approach is generic enough to work with any Prometheus-compatible remote write backend.

Originally published on my blog