The agent went quiet for 19 minutes and systemd stayed green
On 2026-10-12 I rebuilt the database of my own main center. Nineteen minutes later I noticed one of my agents had stopped reporting. systemd was green the whole time, the timer was firing, the journal looked clean. The batches were being dropped on the floor.
I will come back to why (#554). I lead with it because this release is mostly about failures you only meet on real hardware: a config block that silently failed to write, an eBPF observer that reported success while observing nothing, a router storing its LAN address in CIDR form.
MiBee Steward is what I run to answer "what is actually on this network". Single binary, Go with Chi, SQLite through modernc so there is no CGO, SvelteKit UI embedded with go:embed. Point it at a subnet and it comes back with vendor, device type, ports, certificates, and the neighbour relationships it can prove.
License up front, since it decides whether you can use this at all: AGPL-3.0 with a commercial dual license. Fork it into a closed product or sell it as a hosted service and you need the commercial one. The fingerprint corpus under configs/fingerprints/ is separate, CC-BY-SA 4.0.
v0.7 refuses your old database, and that is deliberate
Databases from v0.6 and older are refused at startup. The binary prints what to do and exits. Databases from v0.7.0 through v0.7.2 are refused as well: v0.7.3 spans three schema generations, v4 through v7.
The reason is that I deleted the startup migration chain. Every boot used to walk a list of incremental changes, so a database could sit at any historical point and one binary had to cope with a dozen table shapes. The parallel-DDL bugs grew out of that (#328, #431, #437): two code paths each applying half of a change, neither finishing.
Now db/schema.sql is the single source of DDL, applied once at creation. If the schema version in the file does not match the binary, startup is refused, and no partial state is left to reason about.
The upgrade path is three steps:
./scripts/backup.sh
# then start the new binary against an empty data directory
./mibee-steward -config configs/config.yaml
# let it rescan; the asset table repopulates itself
The agent's local mini database follows the same policy, so scan history and SNMPv3 credentials do not carry over. The Go agent's agent.db is not migrated either.
Why the agent is now Rust
The distributed agent is Rust only now (agent-rs/, artifacts named mibee-agent-*). agent.yaml is key compatible, so nobody has to edit config.
The argument is old hardware. Anyone with a homelab has a drawer of boards that are still useful and short on RAM, and a collection daemon should not be why one gets retired. My test host was a NanoPi NEO: ARMv7, 512 MB of RAM, systemd, 6.18 kernel, watching the lan-63 segment of my center. Two days of sampling, 96 rounds:
| Metric | Go | Rust |
|---|---|---|
| Binary size | ~19 MB | ~4.8 MB |
| Median RSS | 54 MB | 12 MB |
| RSS band | 38 to 63 MB | 11 to 12 MB |
The Go figure already includes a round of lazy corpus compilation I did earlier; before that it sat around 110 MB steady state on that board. On a 512 MB machine, half your RAM is not something I get to spend, and a band that swings from 38 to 63 MB makes capacity planning guesswork.
The Rust side sits at 12 MB median and stayed between 11 and 12 MB for the whole two days, roughly a quarter of the Go number. That stability matters more than the median when the box also runs dnsmasq and a couple of containers. A /24 sweep on the same hardware is about 1.7x the Go throughput. Artifacts are static musl builds for amd64, arm64 and armv7, plus OpenWrt packages.
Multi-homed devices stopped ping-ponging
A router with two NICs gets reported by agents on both sides. The old identity logic treated every report as a possible move, so the device row migrated between segments on the scan cadence of whichever agent spoke last. In the UI it looked like a device that could not sit still.
Two new tables fix it. device_network_presence holds one row per (device, segment), recording which address answered there, refreshed on every report. device_sightings is append-only address history keyed by (device, IP, MAC, vantage, source); a repeat of the same tuple refreshes last_seen and see_count, a change appends a row.
The rule now: an out-of-segment report does not move the device row while a presence record from the last 24 hours is fresh, and a genuine move happens only when every other presence has expired. Every bridge apply writes a sighting with its vantage, empty for the center and otherwise the agent id.
The topology graph finally has a root
The root bridge used to be guessed by heuristics. The awkward case was the root bridge itself: standing on that device, every designated bridge around it is itself, no edge survives deduplication, and the probe returned nothing for the one device you most wanted to see.
The probe now reads dot1dStpPortRole (root, designated, alternate, backup, disabled) and dot1dStpDesignatedRoot in the same scalar Get, landing in scan_attributes.stp: own bridge MAC, root bridge MAC, is_root, and port roles by interface name. detectRoot consumes that as pass 0, ahead of every heuristic.
LLDP and CDP had a quieter failure. The center's own device row never had a MAC address, since ARP against a local address takes the kernel loopback shortcut and never puts a frame on the wire, so RecordNeighborsByMAC could not resolve the far end and frame edges were dropped without a log line. Two fixes: the center fills its own interface MACs into its own device row at wiring time and after every scan (AnchorLocalInterfaces, fills blanks only, self healing), and unknown LLDP/CDP chassis or NDP MACs get anchored with a #anchor:<mac> placeholder IP, merging by MAC priority once a real address shows up.
The interface inventory walks IF-MIB and ipAddrTable into device_interfaces: ifIndex, name, description, MAC, speed, the IPv4 answering on that port. Vanished ports keep their last sighting row frozen. Routing table edges materialise every non directly connected route whose next hop resolves to a known device, with the destination subnet in metadata; where routes share a hop, the more specific subnet wins. That edge table had no reader before this release, so rows were written and nothing rendered them.
Two Stewards, read-only federation
v0.7.3 added federation between two centers (#379). Each side fully administers its own segments, its own UI, database and scan cadence, while holding a near real time read-only profile of the peer's devices.
I held one constraint: zero new wire protocol. The peer is an embedded reporter over the same channel the Rust agent uses, so ingestion, anti-entropy state hashes, lease liveness and change events all apply unchanged. What the peer never gets: probe targets, documents, TLS chains, config backups, credentials, raw evidence, and the command channel. No scan, probe or restart can be triggered from the other side.
Read-only is a decision I made on purpose. Once a remote operator can issue commands in your segment, every bug on either side becomes a shared incident. Devices from agent or peer segments are mirrors, so roughly 15 change routes (edit, delete, bulk, systems, documents, heartbeat config) return 403, local administrators included. The UI hides edit and delete, showing a "remote network" badge instead.
peer:
center_url: https://steward.example.internal
auth_token: <token minted on the other side>
agent_id: peer-basement # defaults to peer-<hostname>
cidr: 192.168.63.0/24 # defaults to network.cidr
Freshness got its own treatment (#556) because of the 19 minutes above. Data from an agent or peer segment older than 5 minutes turns "Remote" into an amber "Remote · stale" with the last report time and dims the row. Devices seen in several segments get a blue ring and a label (#557).
Five new passive ears
The eBPF observer moved down into the agent (#498, default-off ebpf feature). The kernel side program is the same bpf2go object file the center uses, pinned byte-identical by a drift test; the loader is aya, pure Rust, so no libbpf or BCC gets dragged in. Kernel floor is 5.8, aya takes TCX on 6.6 and above and clsact/netlink below, and it wants root or CAP_BPF plus CAP_NET_ADMIN with BTF optional. Where it cannot load, one warn line is logged and active probing continues.
ARP and IPv6 ND wire sightings (#497) catch the sleeping IoT class, devices that answer ARP and never send an IP packet, a third capture path alongside active scanning and DHCP leases. Throttling lives inside the BPF program, an LRU map with roughly one event per sender per 30 seconds: 1000 gratuitous ARP frames on the rig produced 1 event and 0 ring buffer drops. DAD probes, sender 0.0.0.0, are filtered in BPF.
Flow telemetry adds speaks-with edges (#500). Every IPv4 TCP/UDP packet lands in an LRU flow table keyed by (src, dst, proto, dport), about 16k entries by default, drained every 30 seconds. The privacy line is the part I would point at first: flows are folded into device level aggregates only, so per-port flows never leave memory and nothing that lets you reconstruct which service two hosts used is persisted. Sessions between managed devices become faint dashed edges with a byte total in the tooltip, and the first observation of a managed pair emits a speaks_with_new change event you can feed into lateral movement alerting. Retention defaults to 30 days.
WS-Discovery probing (#507) is the active channel for cameras: two Probe datagrams to 239.255.255.250:3702, a bounded 2 second window, drawing ProbeMatches out of ONVIF cameras and encoders. Responses are filtered strictly by source IP, since one multicast Probe gets answered by every WS-Discovery device on the segment and skipping that flattens the identity of a whole /24 into one blob. Parsed ONVIF identity is percent-decoded into corpus keys, so Front%20Door becomes Front Door. Two ONVIF cameras on the rig answered and landed at 0.98 fusion confidence, inferred type camera.
SNMP trap receiver (#509) lets devices speak first: linkDown, linkUp, coldStart and authenticationFailure become snmp_trap passive evidence, and coldStart plus linkUp also feed the discovery funnel. It is off until you turn it on. Port :162 needs CAP_NET_BIND_SERVICE, :1162 does not. One row per source per trap per 10 seconds stops a flapping port flooding you.
Four bugs only the field could find
My own migration is where #554 came from. The reporter treated every non-429 4xx as terminal, so a 401 marked the batch delivered and dropped it, and after the token came back the agent never retried. Now 401 goes back on the queue with a 5 minute cooldown, empty heartbeats are dropped, and the other 4xx codes (400, 403, 409) stay terminal. A side effect I did not plan: agents ride out a center database rebuild without anyone restarting them.
On iStoreOS 24.10.8 the LuCI passive discovery toggle reported "updated" while the config file did not change, and the service restarted for nothing (#515). Missing conntrack, hostapd and dns_log blocks failed silently. They are written out if absent now.
The eBPF observer had never actually worked (#492 to #494). Three independent breakages stacked up: a removed link.AttachTC, a go generate that produced nothing quietly, and a doff read the verifier rejected. It ran, it logged success, it saw nothing. TCX attachment and a CO-RE-free build that needs only clang fixed it.
One iStoreOS 24.10 box stores its LAN address as 192.168.60.1/24, and I concatenated that straight into a URL, producing http://192.168.60.1/24:8080 in the "open admin UI" link (v0.7.2). Prefix and suffix are stripped before output now. The install script had the same bug and was silently producing an empty network.cidr.
And one security fix found in review (#534): gosnmp's TrapListener does not check the community of received messages, so any host able to send UDP to the bound address could land 0.9 confidence evidence and forge coldStart or linkUp sightings. Comparison is constant time and runs before throttling, so a forged packet can no longer eat the real one's 10 second window. There is a mibee_snmp_trap_community_drops metric, and gosnmp's logger goes through a redacting adapter.
What is still missing
Upgrading hurts exactly once: back up, fresh database, rescan. The Rust agent has no router .ipk or .apk packaging yet, so on OpenWrt the documented path is a tarball plus a procd init script. SNMP trap stops at v1 and v2c; v3 trap has no config key.
The center is a single instance: SQLite has one writer, no clustering, no HA. Crossing segments means center plus agents, or federation. Federation stays read-only, which I count as a feature.
Containers are unprivileged by default, so LLDP/CDP raw frames and eBPF need either your own privileged build or the -full binary with CAP_NET_RAW.
Deliberately out of scope: alerting rules belong to Alertmanager, visualisation to Grafana, host depth to node_exporter, service layer discovery to Consul, config management to Ansible.
There are no named outside contributors on this repository yet, so I am not going to promise a response time.
Links
Repository: https://github.com/Mi-Bee-Studio/MiBeeSteward
Project page: https://www.mickeyzzc.tech/projects/mibeesteward
Docs: https://github.com/Mi-Bee-Studio/MiBeeSteward/tree/main/docs
License: AGPL-3.0 with a commercial dual license available, required for closed derivatives or hosted use; the fingerprint corpus is CC-BY-SA 4.0.
Top comments (0)