DEV Community: Mohamed Zrouga

I'm not an ML engineer. I built one anyway.

Mohamed Zrouga — Mon, 25 May 2026 13:02:14 +0000

Not because I wanted to — but because every tool I tried on ARM edge devices either needed the cloud, needed a GPU, or needed more RAM than the service it was supposed to be watching.

So this post isn't really about Cerberus. It's about the problem it forced me to actually understand: what does anomaly detection require on constrained hardware, and how do you get there without black-box ML?

Here's what I learned.

The observability stack was heavier than the workload

When you're deploying on cloud VMs, the weight of your tooling is invisible. You have RAM to spare. You have fast networks. You have Prometheus scraping endpoints on a LAN that never goes offline.

Drop the same assumptions onto an ARM gateway at a remote industrial site and things break differently. The telemetry pipeline competes with the workload for CPU cycles. The collector needs connectivity that doesn't exist. The ML inference endpoint is somewhere in a cloud region the device can't reach.

The problem isn't the tools — they were built for a different environment. The problem is treating cloud-native observability as a default rather than a choice.

Once I asked "what does edge observability actually need?" the answer was much smaller than I expected:

Did traffic behavior change?
Is something probing unusual ports?
Are protocol patterns different from yesterday?
Is there unexplained traffic acceleration?
Which specific device changed?

That's not a distributed tracing problem. It's a behavioral signal problem.

Why eBPF is the right layer for this

The kernel already sees everything. Every packet, every connection, every flag — it all passes through the network stack before any userspace process touches it.

eBPF lets you attach small programs directly to that stack using TC (Traffic Control) or XDP hooks. Instead of running tcpdump through a pipe, or copying full payloads into userspace for inspection, you write a kernel-side filter that extracts only the metadata you care about and hands it to you via a ring buffer.

For Cerberus, that's roughly 208 bytes per event:

struct network_event {
    __u8  event_type;       // ARP / TCP / UDP / DNS / TLS / HTTP / ICMP
    __u32 src_ip;
    __u32 dst_ip;
    __u16 src_port;
    __u16 dst_port;
    __u8  tcp_flags;
    __u8  l7_payload[128];  // first 128 bytes for L7 inspection
    // ...
};

The kernel filters. The ring buffer delivers. Userspace gets a clean event stream at near-zero overhead — no full payload copies, no extra processes, no agents fighting the workload for CPU.

On ARM systems, this difference is measurable.

What "ML-Lite" actually means

Before I go further: I want to be clear that I'm not an ML engineer. What I built is better described as applied statistics with some online learning layered on top. I'm calling it ML-Lite because that's what it is, not because it sounds impressive.

The instinct when building anomaly detection is to reach for a neural network or a heavy ML runtime. On constrained hardware that's a dead end — both because of resource cost and because the explainability disappears. An operator at 2am staring at an alert doesn't want a confidence score. They want to know what changed.

So the system works in three stages.

Stage 1: Aggregate into windows

Every 30 seconds, the event stream is compressed into a feature vector:

[packet_rate, dns_rate, tls_rate, syn_rate, entropy, unusual_ports]

This is the "network behavior as numbers" step. Each window becomes a compact snapshot of what the device was doing.

Stage 2: Build a baseline

As windows accumulate, the system learns what normal looks like using three tools:

Median + MAD (Median Absolute Deviation) — robust to outliers in a way mean/stddev aren't. If one window has a traffic spike, the baseline doesn't shift.
EWMA (Exponentially Weighted Moving Average) — gives recent windows more weight than old ones, so the baseline adapts slowly over time.
Centroid distance — tracks how far the current feature vector is from the center of historical observations.

The scoring formula for each feature is:

robust_z = |x - median| / MAD

And entropy is computed as:

H(X) = -Σ p(x) log₂ p(x)

Where x is the distribution of destination ports. Normal traffic hits the same handful of ports repeatedly — entropy stays low. A port scan touches 22, 23, 80, 443, 445, 3389 in sequence — entropy spikes.

Stage 3: Explain the score

This is the part I cared most about. The system doesn't just surface a number — it surfaces which features drove it:

WHY?
+ High SYN rate
+ Port entropy spike
+ Traffic acceleration

An operator can act on that immediately.

The evolution wasn't planned

The detection model went through several iterations, each adding a layer without replacing what came before:

v1 — Statistical detection: Median, MAD, thresholds, entropy. Worked. Noisy on IoT networks.

v2 — Adaptive learning: EWMA, rolling baselines, per-device profiles. Reduced false positives significantly once the baseline had enough history.

v3 — Isolation Forest: Unsupervised ML, tree isolation, outlier scoring. Doesn't need labeled attack data. Effective for genuinely novel patterns.

v4 — Tiny autoencoders: Architecture is 9→16→4→16→9. The bottleneck (4 dimensions) forces the model to compress normal behavior into a compact representation. Reconstruction error is the anomaly signal — if the current window can't be reconstructed well, it's unusual.

v5 (in progress) — Temporal Graph ML: Device graphs, sequence analysis, behavior prediction. The goal is to model relationships between devices over time, not just each device in isolation.

The design constraint throughout: offline, CPU/ARM friendly, explainable, hackable. No iteration added a cloud dependency.

Honest limitations

Behavioral models come with real tradeoffs I haven't fully solved:

Baseline drift: Normal behavior changes over time. A device that starts a new cron job at 3am will generate false positives until the baseline adapts.
Encrypted traffic: TLS SNI is visible at the handshake, but payload content isn't. The entropy signals still work on port distributions, but deep inspection has limits.
Noisy IoT environments: Some IoT devices have genuinely chaotic traffic patterns. Per-device profiles help, but they need enough history to be meaningful.
Cold start: Until a device accumulates enough windows to build a stable baseline, scoring is unreliable.

This is not a full IDS. It's an operational visibility tool that can surface unusual behavior — with the reasoning attached.

What I'm still trying to understand

This is where I'd genuinely value input from people with more ML background than me:

For edge anomaly detection, is unsupervised learning (Isolation Forest, autoencoders) fundamentally the right approach, or are there better-suited algorithms for streaming, low-resource environments?
How do you handle baseline drift without introducing false negatives for genuine behavioral shifts?
For temporal modeling on embedded systems, are there efficient graph-based approaches that don't require the full overhead of a GNN framework?
Is there a better feature set for network behavioral anomaly detection beyond what's described here?

I'm building in a space I find genuinely interesting but I'm not formally trained in — any feedback on the ML design is more valuable to me than stars.

Thank you

Cerberus wouldn't be what it is without the people who showed up and contributed real work to it.

Huge thanks to @SvenNellerz and @alexmchughdev — your contributions made the project meaningfully better and I genuinely appreciate it.

The project also stands on cilium/ebpf, BuntDB, and golang-lru — solid libraries that made the Go + eBPF combination practical without CGO headaches.

If you're working in this space — embedded Linux, ARM infrastructure, eBPF, IoT security, or lightweight ML — the repo is at github.com/zrougamed/cerberus and I'd genuinely love to hear how you're approaching the same problems.

I Got Tired of Docker Eating My Raspberry Pi's RAM — So I Built My Own Container Orchestrator

Mohamed Zrouga — Sat, 16 May 2026 23:33:55 +0000

A few months ago I noticed something that genuinely annoyed me.

I was running tiny services across my Raspberry Pi lab:

webhook workers
monitoring agents
lightweight APIs
ETL processing tasks

Small, focused workloads. The kind of things a Pi is actually good at.

But the infrastructure stack underneath them? It looked like this:

Docker Engine
 └─ containerd
     └─ runc
         └─ CNI plugins (all of them, even the ones I'd never touch)
             └─ orchestration layer
                 └─ service networking
                     └─ monitoring sidecars
                         └─ my actual 8MB process

At some point I sat down and ran ps aux and free -h on a freshly booted node before deploying anything.

The infrastructure was already using more RAM than my applications.

That felt wrong. So I started pulling threads.

What Actually Runs a Container?

Strip everything back. What does "running a container" actually require?

An OCI image unpacked to disk
A rootfs (overlayfs layers)
Linux namespaces (pid, net, mount, uts, ipc)
cgroup resource limits ( cgroup V2 )
A process in that environment

That's it. That's the whole thing.

Everything else — the daemon, the socket, the plugin ecosystem, the CNI chain, the abstraction layers — exists to make that easier to manage at scale.

Scale I don't have on a Raspberry Pi.

So I asked: what's the absolute minimum secure OCI stack that can do this properly?

That question became nyxd https://github.com/zrougamed/nyxd.git .

nyxd: What It Is and What It Isn't

nyxd is a lightweight container daemon I built in Go. It uses:

crun as the OCI runtime (not runc — more on this shortly)
overlayfs directly via syscall for rootfs
no CNI plugins
nftables for NAT and port mapping
seccomp + NoNewPrivileges enforced on every container

It is not:

Kubernetes
another Docker replacement
"another Podman"
trying to be any of those things

The philosophy is reduction. Every component you remove is an attack surface that disappears, a dependency you don't have to update, a CVE you'll never have to patch.

Why crun Instead of runc

This is the first question people would ask.

Here's the honest answer: crun has a remarkably clean security history.

crun CVE history (complete, as of 16-05-2026):

CVE	Impact / Type	Affected	Fixed In
CVE-2026-30892	Local Privilege Escalation via crun exec -u 1 root parsing flaw.	1.19 – 1.26	1.27
CVE-2025-24965	Host Filesystem Escape via the krun architecture handler.	< 1.20	1.20
CVE-2022-27650	Privilege jump inside container via leaked Inheritable Capabilities.	< 1.4.4	1.4.4
CVE-2019-18837	Host Directory Traversal via malicious symlinks in a crafted image.	< 0.10.5	0.10.5

Very few CVEs. In its entire history.

Compare that to November 2025, when runc had three container-escape vulnerabilities disclosed in a single week: CVE-2025-31133, CVE-2025-52565, and CVE-2025-52881. All critical. All allowing container escape to host.

Beyond security, crun has practical advantages for edge/ARM workloads:

Lower memory footprint per container
Faster startup (C runtime, not Go)
Excellent cgroup v2 support from day one
Smaller binary (~800KB vs runc's several MB)

On a Pi with 1-4GB RAM running 20+ containers, that adds up fast.

The CVE Surface Nobody Benchmarks

Here's something I started thinking about that I hadn't seen anyone measure properly:

People benchmark CPU and RAM constantly. Almost nobody benchmarks security surface area.

But when you're running edge infrastructure — systems with limited ops coverage, longer uptimes, harder recovery paths — the CVE surface is arguably the most important operational metric.

So let me be direct about the current state as of 16-05-2026:

containernetworking/plugins CVE history (recent):

CVE	Plugin	Issue	Fixed
CVE-2025-67499	portmap	nftables backend intercepts unintended traffic	v1.9.0
CVE-2025-52881	selinux dep	container escape via procfs write misdirection	v1.9.1
CVE-2024-34156	all	Go stdlib encoding/gob	v1.4.0-6
CVE-2023-45290	all	Go stdlib net/http	v1.4.0-3

And this is just the CNI plugin layer — the binaries a lot of stacks still exec on every container ADD. nyxd's default path doesn't run those plugins at all; I still think the table matters as a reminder of what you inherit when you opt into the full CNI distribution. Add Docker Engine, containerd, runc, BuildKit, and their respective dependency trees and you're tracking dozens of CVEs per year across a stack that most people never fully audit.

The rough security surface comparison:

Stack	Binary count	External deps	Rough CVE exposure/yr
nyxd + crun	2	3 Go modules	Very low
Podman + crun	~8	Large	Low-medium
Docker Engine full	15+	Very large	Medium-high
containerd + runc + full CNI	20+	Massive	High
Kubernetes node	30+	Enormous	Very high

The numbers aren't precise science. But the trend is real.

Simplicity is a security feature. I genuinely believe that now.

The Networking Decision

Most of the container networking ecosystem exists to solve problems at scale. VXLAN overlays, BGP route propagation, multi-cluster service meshes.

None of that applies to a Raspberry Pi lab.

nyxd's default networking is implemented inside the daemon, in Go, using raw netlink — not by exec'ing the usual CNI plugin chain under /opt/cni/bin.

What that stack actually does:

Bridge + veth — brings up nyxbr0, creates the pair, moves the peer into the container netns, names it eth0
File-backed IPAM — hands out addresses from the same 10.88.0.0/16-style slice you'd expect from a tiny lab bridge
Loopback — lo up inside the netns
Port publishing — host→container maps via nftables (we shell out to nft for the rules; that's the one small networking helper we deliberately keep external)

What we don't use (on that default path):

The bridge / host-local / loopback / portmap binaries from containernetworking/plugins
Flannel
Calico
kube-proxy
Weave
Cilium (for this use case)
Any overlay mesh networking

If you want the traditional CNI exec path — because you already ship a conflist or you're mirroring another environment — nyxd -net-driver=cni is still there. Same supervisor and API; you bring /opt/cni/bin and your plugins. That's optional complexity, not what you get out of the box.

Zero CNI plugin binaries on the default path means:

Nothing to install under /opt/cni/bin on a fresh Pi
Nothing to keep updated in that directory for homelab-sized deployments
No binary-level CVE surface in that layer for the stack I'm actually running day to day
Faster container startup (no fork/exec chain per CNI ADD)

Kernel Requirements (the honest list)

Running nyxd requires these kernel modules:

overlay          # overlayfs for container rootfs
bridge           # nyxbr0 bridge interface
veth             # virtual ethernet pairs
br_netfilter     # iptables/nftables sees bridged traffic
ip_tables        # iptables core
iptable_nat      # NAT table
nf_nat           # connection tracking NAT
nf_conntrack     # stateful packet tracking
nft_masq         # nftables masquerade
seccomp          # syscall filtering

And these sysctls:

net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.conf.all.rp_filter = 0

All of this works on a standard Raspberry Pi OS kernel (6.1 LTS). No custom kernel needed.

Benchmarking This Honestly

Here's how I'm measuring whether nyxd actually delivers on its promises.

Startup latency:

hyperfine --warmup 3 \
  'docker run --rm alpine true' \
  'nyx run alpine true'

On my Pi 5: nyx vs Docker

Attack surface check:

# On a Docker host: how many CNI plugins are sitting on disk
ls -la /opt/cni/bin/ 2>/dev/null | wc -l

# Compare to nyxd default: no plugin dir required
lsof -p $(pgrep -x nyxd) | wc -l   # open file descriptors
ss -tlnp | grep nyxd                 # listening sockets (usually just the Unix control socket)

Docker opens more sockets, more file descriptors, and maintains more persistent background connections than a Pi workload typically justifies.

Dependency tree:

# nyxd go.mod external deps
cat go.mod | grep -v "^//"
# 3 modules: image-spec, runtime-spec, golang.org/x/sys

Three. External. Modules. That's the entire dependency graph for the runtime and networking layer.

What the Raspberry Pi Actually Taught Me

The Pi has an interesting property: it forces you to care about things cloud infrastructure lets you ignore.

Thermal throttling — when your CPU is running hot because your container daemon is doing background work, your actual workloads slow down. Less daemon overhead means cooler, more consistent performance.

SD card / SSD wear — fewer writes from logging, state management, and plugin communication extends storage life meaningfully on embedded deployments.

Boot time — when a power cut hits an edge node, boot-to-operational time matters. A lighter stack comes up faster and can rejoin the network sooner.

Debugging under pressure — when something breaks at 2AM on a remote node you can't physically access, a simpler stack is dramatically easier to reason about. Fewer layers means fewer places to look.

Power consumption — I've measured ~1.5W difference in idle power between a full Docker stack and nyxd on a Pi 5. Across 10 nodes running 24/7, that's ~130kWh/year. Not enormous, but real.

Where nyxd Is Right Now

This section separates what the daemon and nyx client exercise end-to-end from what exists mainly as libraries, stubs, or unfinished wiring.

Working (end-to-end in the daemon + `nyx` client)

OCI distribution pull for public images: raw HTTP against registries, layer blobs, and SHA-256 digest verification on ingest (digest mismatch fails the pull).
Overlayfs upper/work/merged layout and teardown on container exit paths.
Container lifecycle via crun from the supervisor: create/run (including the foreground attach path), stop, kill, delete, and state polling; the control API supports nyx exec.
Restart policies: always, on-failure, unless-stopped, and never (empty or unrecognized API values are normalized to conservative defaults in the control layer).
Structured JSON lines per container under the daemon data directory (log collector plus GET /v1/containers/{id}/logs), with optional plain-text decoding for attached clients.
Default in-process networking (-net-driver=native): bridge, veth, file-locked IPAM, nftables-based -p / publish, and host-side NAT — no CNI plugin binaries on disk.
Optional CNI exec path: -net-driver=cni with -cni-bin-dir, -cni-conf-dir, and -network for a traditional plugin-driven stack instead of native.

Implemented but not product-complete (nuance)

Health checks (exec / HTTP / TCP) — internal/health implements a full checker (intervals, timeouts, retries, start period), but it is not hooked into supervisor: no checker goroutine started with each container, and no unhealthy → stop/restart policy wiring. The code is library-ready, not operator-ready.
Prometheus-style metrics — internal/telemetry implements a text OpenMetrics/Prometheus-style /metrics handler, but cmd/nyxd does not call ServeMetrics, so counters and gauges are not exposed unless another binary wires the package in. The format exists; the default daemon does not listen for scrape traffic.

Still being hardened / incomplete

Registry auth beyond anonymous public pulls — the current path centers on the Docker Hub anonymous token flow; private registries, stored credentials, and arbitrary OAuth/OCI auth flows are not first-class yet.
Compose — YAML parsing is real (gopkg.in/yaml.v3): internal/compose reads a strict subset of compose-shaped fields, validates stacks, and can compute dependency order. What is missing is daemon-side orchestration (no nyx compose up-style command in the shipped mains): a parser is not a multi-service scheduler.
Control API and nyx CLI — Unix-socket HTTP for run, ps, logs, stop, remove, pull, images, and exec is usable, but error messages, edge cases, and long-term API stability are still evolving.
Seccomp — generated bundles set capabilities, noNewPrivileges, masked paths, and related hardening fields; there is no curated, versioned seccomp JSON checked into the bundle generator yet. Anything beyond the explicit JSON is whatever crun and the host apply by default.
systemd-notify — example units may use Type=notify, but nyxd does not emit READY=1 (or reload state) via sd_notify. Production units should use Type=simple until notify is implemented, or notify must be added to the daemon.
NFT / port publishing on unusual host sysctl values, exotic dual-stack setups, and odd bridge topologies — the native backend is real code, but it benefits from more soak time outside typical laptop and homelab bridges.

Bottom line

nyxd is not positioned as a drop-in, production-grade Docker replacement today. The largest gaps between documentation and reality are health automation (implemented package, not integrated) and metrics (handler exists, not served by default). Registry authentication and compose orchestration remain intentionally narrow.

The stack does run real workloads in lab settings along the paths above: pull, overlay, crun, native networking, structured logs, and restart policies. The architectural bet — native networking by default, optional CNI when operators want plugin ecosystems — remains coherent even where polish and operational breadth still lag.

The Bigger Question

I keep coming back to this:

Are we over-engineering edge infrastructure?

The modern container ecosystem was designed to solve orchestration at Google-scale. Kubernetes, CNI, CRI, OCI — these are all excellent standards that solved real problems.

But those standards got adopted at every layer of the stack, including layers where the complexity isn't warranted.

A Raspberry Pi running an Go API doesn't need the same infrastructure as a 10,000-node Kubernetes cluster.

An industrial IoT gateway doesn't need BuildKit.

A homelab monitoring stack doesn't need containerd's full plugin system.

The standards are fine. The problem is using the full weight of the enterprise implementation everywhere, including at the edge where resource constraints and operational simplicity matter most.

nyxd is my attempt to find where the floor is. How small can a correct, secure, production-capable OCI runtime stack actually be?

I don't think we've found it yet.

Try It / Follow Along

nyxd is being developed openly. The codebase is Go 1.26.

If you're running:

Raspberry Pi clusters
ARM64 edge nodes
Self-hosted systems
VM , QEmu , Proxmox ...
Any environment where RAM and attack surface actually matter

I'd genuinely love to hear what you're running and what constraints you're working within.

A few questions for the comments:

What does your container stack look like on ARM systems today?
What's your idle memory baseline before deploying any workloads?
Have you ever audited the CVE history of your CNI plugins?
Would you trade orchestration features for a meaningfully smaller attack surface?
Is your container stack heavier than your actual workloads?

If there's enough interest, follow-up posts will cover:

Full architecture walkthrough with the native network stack (and when I'd still flip on CNI)
Memory profiling methodology for container daemons
Security surface comparison methodology
Deploying nyxd on a Pi cluster from scratch
The case for writing your own IPAM in 200 lines of Go

nyxd is made by the community for the community - free to use for personal use

Meet Orion-Belt, Go ZeroTrust Bastion

Mohamed Zrouga — Thu, 01 Jan 2026 20:46:39 +0000

Stop opening Port 22 to the world. 🛑

In the world of infrastructure, we’ve long accepted a "security tax." If you want your servers to be accessible, you either open holes in your firewall, maintain a complex VPN, or pay thousands for enterprise PAM (Privileged Access Management) tools.

I felt there was a massive gap for a lightweight, developer-centric tool that follows Zero Trust principles without the enterprise bloat. That’s why I built Orion-Belt.

Orion-Belt in Action

Seeing is believing. Here is a quick look at osh (the Orion-Belt SSH client) connecting to a machine that has zero inbound ports open, while the gateway handles the heavy lifting of authentication and recording.

The "Security Tax" of Traditional Access

Most teams handle remote server access in one of three ways, and all of them have a "catch":

Static SSH Keys: Great until a laptop is stolen or an employee leaves. Auditing "who did what" is nearly impossible.
The "Jump Box" (Bastion): A single point of failure. If your bastion is compromised, your whole network is exposed.
VPNs: They give "flat" network access. Once a user is on the VPN, they can often see everything, violating the Principle of Least Privilege.

I wanted something that felt like a modern SaaS (like Teleport or Boundary) but remained self-hosted, open-source, and dead simple.

Feature Comparison: Why Orion-Belt?

How does Orion-Belt stack up against the status quo?

Feature	Orion-Belt (Open Source)	Traditional SSH/VPN	Enterprise Gateways
Inbound Firewall Rules	❌ No (Reverse Tunnel)	✅ Yes (Port 22/VPN)	❌ No (Agent/Tunnel)
Session Recording	✅ Yes (Built-in)	❌ No (Hard to config)	✅ Yes (Built-in)
Access Control	ReBAC (Fine-Grained)	Coarse-Grained	RBAC/ABAC
Temporary Access	✅ Yes (JIT Approval)	❌ No	✅ Yes
Protocol Support	SSH, SCP	SSH, SCP (VPN allows more)	SSH, Kubernetes, Databases, HTTP
Cost	Free (Self-Hosted)	Free	$$$ High
Architecture	Lightweight Go Binary	Standard Utilities	Complex Microservices

How it Works (Under the Hood)

Orion-Belt is built on a Reverse SSH Tunnel architecture. Instead of you reaching into your private network, your servers reach out to the Orion-Belt gateway.

The Agent: A small Go binary runs on your target VMs. It creates an outbound connection to your Orion-Belt server.
The Gateway: The "Brain." It handles authentication, ReBAC (Relationship-Based Access Control), and session recording.
The Client (osh / ocp): CLI tools that feel like standard SSH/SCP but verify permissions with the gateway’s API first.

Because the connection is outbound from the server to the gateway, you can keep Port 22 closed. This effectively hides your infrastructure from automated bot scans and 0-day SSH exploits.

Key Features for Modern Teams

1. ReBAC (Relationship-Based Access Control)

Orion-Belt checks the relationship between the user and the resource. This allows for fine-grained permissions that scale as your team grows.

2. Session DVR-Style Replay

Compliance (SOC2/HIPAA) requires seeing what happened during a session. Orion-Belt records every keystroke at the gateway level. You can replay the entire session later to see exactly what commands were run.

3. JIT (Just-In-Time) Temporary Access

Need a developer to debug a production issue for one hour?

osh --request-access prod-db-01 --duration 1h --reason "Investigating latency"

Admins get a notification, approve the request, and the access automatically expires. No "orphaned" keys left behind.

The Architecture

Client (osh/ocp)
      │
      ▼
Orion-Belt Gateway Server (ReBAC + Session Recording)
      │
      ▼ (Reverse SSH Tunnel)
Agent (on your locked-down servers)

Quick Start

1. Build from source:

git clone https://github.com/zrougamed/orion-belt.git
cd orion-belt && make build

2. Start the Server:
The server acts as your central hub. It uses PostgreSQL to store sessions and permissions.

3. Deploy the Agent:
Drop the agent binary on any server behind a firewall. Once it connects to the gateway, that server is accessible via osh.

I need your feedback!

Orion-Belt is currently in Alpha. It’s functional and stable, but I’m looking for early adopters to help shape the roadmap.

Does this architecture fit your current workflow?
What notification plugins would you like to see (Slack, Discord, Email)?

Check out the repo, leave a ⭐ if you like the concept, and let's discuss in the comments!

GitHub: https://github.com/zrougamed/orion-belt

Final Thoughts
Infrastructure access doesn't need to be a choice between "easy" and "secure." By combining Go's performance with a Zero-Trust architecture, Orion-Belt makes high-end security accessible to everyone.

Building a Security Test Lab with QEMU: From Zero to Network Monitoring

Mohamed Zrouga — Fri, 26 Dec 2025 23:47:51 +0000

Why QEMU for Your Test Lab?

As software engineers and security professionals, we constantly need isolated environments to test new tools, experiment with network configurations, or validate security measures before production deployment. While solutions like VirtualBox and VMware are popular, QEMU offers something special: lightweight, scriptable, headless operation perfect for automation and CI/CD pipelines.

In this guide, I'll walk you through building a complete QEMU-based test lab focused on network security and infrastructure testing. By the end, you'll have a reproducible environment for testing firewalls, monitoring tools, and distributed systems.

What We'll Build

We're creating a multi-VM test lab that simulates a realistic network environment:

Gateway VM: Acts as router/firewall
Application VM: Simulates production workloads
Attacker VM: For security testing and validation

This setup mirrors real-world scenarios where you need to test network policies, intrusion detection, or packet filtering.

Prerequisites

Before we start, ensure you have:

A Linux host (Ubuntu/Debian in examples, but adaptable to any distro)
At least 8GB RAM and 20GB free disk space
Basic command-line familiarity
Sudo access for network configuration

Step 1: Installing QEMU

First, let's get QEMU installed with all necessary components:

# Ubuntu/Debian
sudo apt update
sudo apt install qemu-kvm qemu-utils libvirt-daemon-system virtinst bridge-utils

# Verify installation
qemu-system-x86_64 --version

For other distributions:

# Fedora/RHEL
sudo dnf install qemu-kvm qemu-img libvirt virt-install

# Arch
sudo pacman -S qemu libvirt virt-manager

Add your user to necessary groups to avoid constant sudo:

sudo usermod -aG libvirt,kvm $USER
# Log out and back in for changes to take effect

Step 2: Creating Your First VM - The Gateway

Let's start with the gateway VM, which will be our network's control point.

Creating the Disk Image

# Create a working directory
mkdir -p ~/qemu-lab/images
cd ~/qemu-lab

# Create a 20GB disk image (qcow2 format for snapshots)
qemu-img create -f qcow2 images/gateway.qcow2 20G

Why qcow2? It supports snapshots, copy-on-write, and compression - essential for test environments where you'll want to reset frequently.

Download a Minimal OS

For security testing, I recommend Alpine Linux (lightweight) or Debian netinst:

# Alpine Linux (very lightweight, boots in seconds)
wget https://dl-cdn.alpinelinux.org/alpine/v3.23/releases/x86_64/alpine-virt-3.23.0-x86_64.iso \
  -O images/alpine.iso

# Or Debian if you prefer more familiar tools
# wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-13.2.0-amd64-netinst.iso \
#   -O images/debian.iso

First Boot and Installation

qemu-system-x86_64 \
  -name gateway \
  -m 1024 \
  -smp 2 \
  -hda images/gateway.qcow2 \
  -cdrom images/alpine.iso \
  -boot d \
  -enable-kvm \
  -nic user,model=virtio \
  -nographic

Flag breakdown:

-name gateway: Identifies your VM
-m 1024: 1GB RAM (adjust based on your needs)
-smp 2: 2 CPU cores
-hda: Primary disk
-cdrom: Installation media
-boot d: Boot from CD-ROM first
-enable-kvm: Hardware acceleration (crucial for performance)
-nic user: Simple NAT networking for installation
-nographic: Console mode (no GUI needed)

Quick Alpine Installation:

# Once booted, login as root (no password)
setup-alpine

# Follow prompts:
# - Keyboard: us
# - Hostname: gateway
# - Network: eth0, dhcp
# - Password: (set a strong one)
# - Timezone: your timezone
# - Disk: sda, sys (use entire disk)

After installation completes, power off the VM (type poweroff).

Step 3: Setting Up Advanced Networking

This is where QEMU shines for security testing. We'll create an isolated network that simulates real infrastructure.

Create a Network Configuration Script

# Create scripts directory
mkdir -p ~/qemu-lab/scripts

# Network setup script
cat > ~/qemu-lab/scripts/setup-network.sh << 'EOF'
#!/bin/bash

# Create an isolated bridge for our test lab
sudo ip link add name br-testlab type bridge
sudo ip addr add 10.0.100.1/24 dev br-testlab
sudo ip link set br-testlab up

# Enable IP forwarding (so gateway can route)
sudo sysctl -w net.ipv4.ip_forward=1

# Create TAP interfaces for VMs
for i in 0 1 2; do
  sudo ip tuntap add tap$i mode tap user $USER
  sudo ip link set tap$i master br-testlab
  sudo ip link set tap$i up
done

echo "Test lab network ready: 10.0.100.0/24 on br-testlab"
EOF

chmod +x ~/qemu-lab/scripts/setup-network.sh

you will end up with this network topology

10.0.100.1/24 Network
       |
   [Bridge](br-testlab)
    /   |    \
  TAP0 TAP1 TAP2
  VM1  VM2   VM3

Characteristics:

VMs are on SAME subnet as host
VMs are "visible" to external network

Network Teardown Script

cat > ~/qemu-lab/scripts/teardown-network.sh << 'EOF'
#!/bin/bash

# Remove TAP interfaces
for i in 0 1 2; do
  sudo ip link set tap$i down 2>/dev/null
  sudo ip link delete tap$i 2>/dev/null
done

# Remove bridge
sudo ip link set br-testlab down 2>/dev/null
sudo ip link delete br-testlab 2>/dev/null

echo "Test lab network removed"
EOF

chmod +x ~/qemu-lab/scripts/teardown-network.sh

Run the setup:

~/qemu-lab/scripts/setup-network.sh

Step 4: Creating VM Launch Scripts

Rather than typing long QEMU commands, let's create reusable launch scripts.

Gateway VM Script

cat > ~/qemu-lab/scripts/run-gateway.sh << 'EOF'
#!/bin/bash

qemu-system-x86_64 \
  -name gateway \
  -m 1024 \
  -smp 2 \
  -hda ~/qemu-lab/images/gateway.qcow2 \
  -enable-kvm \
  -netdev tap,id=net0,ifname=tap0,script=no,downscript=no \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:00 \
  -netdev user,id=net1 \
  -device virtio-net-pci,netdev=net1,mac=52:54:00:12:34:01 \
  -nographic \
  -serial mon:stdio
EOF

chmod +x ~/qemu-lab/scripts/run-gateway.sh

This gateway has TWO network interfaces:

net0: Connected to internal test network (10.0.100.0/24)
net1: NAT to internet for updates

Quick Clone Function for Additional VMs

# Create application VM from gateway template
qemu-img create -f qcow2 -b images/gateway.qcow2 -F qcow2 images/app.qcow2

# Create attacker VM
qemu-img create -f qcow2 -b images/gateway.qcow2 -F qcow2 images/attacker.qcow2

These are backing images - they only store differences from the base, saving massive disk space.

Application VM Script

cat > ~/qemu-lab/scripts/run-app.sh << 'EOF'
#!/bin/bash

qemu-system-x86_64 \
  -name app-server \
  -m 2048 \
  -smp 2 \
  -hda ~/qemu-lab/images/app.qcow2 \
  -enable-kvm \
  -netdev tap,id=net0,ifname=tap1,script=no,downscript=no \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:10 \
  -nographic \
  -serial mon:stdio
EOF

chmod +x ~/qemu-lab/scripts/run-app.sh

Attacker VM Script

cat > ~/qemu-lab/scripts/run-attacker.sh << 'EOF'
#!/bin/bash

qemu-system-x86_64 \
  -name attacker \
  -m 1024 \
  -smp 2 \
  -hda ~/qemu-lab/images/attacker.qcow2 \
  -enable-kvm \
  -netdev tap,id=net0,ifname=tap2,script=no,downscript=no \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:20 \
  -nographic \
  -serial mon:stdio
EOF

chmod +x ~/qemu-lab/scripts/run-attacker.sh

Step 5: Network Configuration Inside VMs

Start each VM and configure networking:

Gateway VM (10.0.100.1)

# Start gateway
~/qemu-lab/scripts/run-gateway.sh

# Inside VM - configure interfaces
# For Alpine Linux:
cat > /etc/network/interfaces << 'EOF'
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 10.0.100.1
    netmask 255.255.255.0

auto eth1
iface eth1 inet dhcp
EOF

# Restart networking
rc-service networking restart

# Enable forwarding permanently
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
sysctl -p

# Setup basic NAT (so internal VMs can reach internet via gateway)
apk add iptables
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT

# Save iptables rules
rc-update add iptables
/etc/init.d/iptables save

Application VM (10.0.100.10)

# Configure static IP
cat > /etc/network/interfaces << 'EOF'
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 10.0.100.10
    netmask 255.255.255.0
    gateway 10.0.100.1
    dns-nameservers 8.8.8.8
EOF

rc-service networking restart

Attacker VM (10.0.100.20)

# Configure static IP
cat > /etc/network/interfaces << 'EOF'
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 10.0.100.20
    netmask 255.255.255.0
    gateway 10.0.100.1
    dns-nameservers 8.8.8.8
EOF

rc-service networking restart

# Install security testing tools
apk add nmap tcpdump hping3 netcat-openbsd

Step 6: Snapshots for Quick Resets

One of QEMU's killer features - instant rollback:

# Create a "clean state" snapshot for each VM
qemu-img snapshot -c clean-install images/gateway.qcow2
qemu-img snapshot -c clean-install images/app.qcow2
qemu-img snapshot -c clean-install images/attacker.qcow2

# After testing, rollback to clean state
qemu-img snapshot -a clean-install images/gateway.qcow2

# List all snapshots
qemu-img snapshot -l images/gateway.qcow2

Step 7: Practical Use Cases

Now your lab is ready! Here are some real-world scenarios:

Testing Network Monitoring Tools

On the gateway VM, install your monitoring solution:

# Example: tcpdump for packet analysis
apk add tcpdump

# Capture traffic between app and attacker
tcpdump -i eth0 -w /tmp/capture.pcap

# Or install eBPF-based tools for advanced monitoring
# (Great for testing tools like Cilium, Falco, or custom eBPF monitors like Cerberus)

Simulating Attack Scenarios

From attacker VM:

# Port scan the application
nmap -sV 10.0.100.10

# Test firewall rules
hping3 -S -p 80 10.0.100.10

# Monitor on gateway to see what's blocked

Testing Firewall Rules

On gateway, add restrictive rules:

# Block all traffic from attacker to app
iptables -I FORWARD -s 10.0.100.20 -d 10.0.100.10 -j DROP

# Allow only SSH
iptables -I FORWARD -s 10.0.100.20 -d 10.0.100.10 -p tcp --dport 22 -j ACCEPT

Container Testing

Install Docker on the app VM to test container networking and security policies in isolation.

Step 8: Automation and Management

Create a master control script:

cat > ~/qemu-lab/manage-lab.sh << 'EOF'
#!/bin/bash

case "$1" in
  start)
    ./scripts/setup-network.sh
    echo "Starting VMs in tmux sessions..."
    tmux new-session -d -s gateway './scripts/run-gateway.sh'
    tmux new-session -d -s app './scripts/run-app.sh'
    tmux new-session -d -s attacker './scripts/run-attacker.sh'
    echo "Lab started. Attach with: tmux attach -t <gateway|app|attacker>"
    ;;
  stop)
    echo "Stopping all VMs..."
    tmux kill-session -t gateway 2>/dev/null
    tmux kill-session -t app 2>/dev/null
    tmux kill-session -t attacker 2>/dev/null
    ./scripts/teardown-network.sh
    ;;
  reset)
    echo "Resetting all VMs to clean snapshot..."
    qemu-img snapshot -a clean-install images/gateway.qcow2
    qemu-img snapshot -a clean-install images/app.qcow2
    qemu-img snapshot -a clean-install images/attacker.qcow2
    echo "Reset complete"
    ;;
  *)
    echo "Usage: $0 {start|stop|reset}"
    exit 1
    ;;
esac
EOF

chmod +x ~/qemu-lab/manage-lab.sh

Usage:

# Start entire lab
./manage-lab.sh start

# Access any VM
tmux attach -t gateway

# Reset everything to clean state
./manage-lab.sh reset

# Shut down cleanly
./manage-lab.sh stop

Performance Tips

Always use KVM acceleration (-enable-kvm) - it's 10-20x faster than emulation
Allocate appropriate resources - Don't over-provision RAM
Use virtio drivers - Much faster than emulated hardware
qcow2 for flexibility, raw for performance - Use raw images if speed is critical
CPU pinning for consistency:

   -smp 2,sockets=1,cores=2,threads=1

Troubleshooting Common Issues

VM won't start with KVM error:

# Check if KVM is available
lsmod | grep kvm
# If not, enable in BIOS (Intel VT-x or AMD-V)

Network not working:

# Verify TAP interfaces are up
ip link show | grep tap

# Check bridge configuration
bridge link show br-testlab

Cannot connect to VMs:

# From host, ping the bridge IP
ping 10.0.100.1

# Check iptables isn't blocking
sudo iptables -L -n

Performance is slow:

# Verify KVM is actually being used
ps aux | grep qemu | grep -i kvm

Next Steps and Advanced Topics

Your test lab is now ready for serious work! Consider exploring:

Ansible automation - Provision VMs automatically
Cloud-init integration - Pre-configure VMs on first boot
PXE boot testing - Network installation scenarios
Multi-host clustering - Test Kubernetes or Docker Swarm
VPN testing - Simulate site-to-site connections
VLAN tagging - Complex network segmentation

Conclusion

QEMU provides a powerful, scriptable foundation for testing network security tools and infrastructure. Unlike heavyweight alternatives, it integrates seamlessly into CI/CD pipelines, supports headless operation, and gives you complete control over the network topology.

The lab we built mirrors production environments while remaining completely isolated and reproducible. Use it to validate security tools, test infrastructure changes, or experiment with new technologies risk-free.

All scripts and configurations from this guide are available in my GitHub https://github.com/zrougamed/blog .

What will you test first in your new lab? Drop a comment below!

About the author: I'm a Senior Software Engineer at Deltaflare, where we work on the Phoenix Platform protecting Critical National Infrastructure across energy, water, and transport sectors. I use QEMU extensively for testing security tools and validating infrastructure changes before production deployment in CNI environments. You can find me at https://zrouga.email or check out my open-source work on GitHub.

Building a Real-Time Network Monitor with eBPF: Lessons from Cerberus

Mohamed Zrouga — Sat, 20 Dec 2025 18:09:12 +0000

As a platform engineer working with critical infrastructure, I needed deep visibility into network behavior without the overhead of traditional packet capture tools. The result? Cerberus - an eBPF-based network monitor that processes packets at the kernel level with near-zero overhead.

This post walks through the architecture, challenges, and lessons learned from building a production-grade network monitoring tool using eBPF and Go.

Why eBPF for Network Monitoring?

Traditional network monitoring tools like tcpdump and Wireshark are powerful but come with drawbacks:

Context switches: Every packet copies data from kernel to user space
Performance overhead: Processing in user space is slower
Security: Full packet capture requires elevated privileges everywhere
Scalability: High-traffic networks can overwhelm traditional tools

eBPF (Extended Berkeley Packet Filter) solves these by running sandboxed programs directly in the Linux kernel, allowing you to:

Filter and process packets at line rate
Extract only the data you need (no full packet copies)
Aggregate statistics in kernel space
Minimize context switches with ring buffers

Architecture Overview

Cerberus uses a two-layer architecture:

The eBPF Program: Kernel-Space Magic

The heart of Cerberus is a TC (Traffic Control) classifier written in C that attaches to network interfaces. Here's what it does:

1. Packet Parsing at Line Rate

struct network_event {
    __u8 event_type;       // Event classification
    __u8 src_mac[6];       // Source MAC
    __u8 dst_mac[6];       // Destination MAC
    __u32 src_ip;          // Source IP
    __u32 dst_ip;          // Destination IP
    __u16 src_port;        // Source port
    __u16 dst_port;        // Destination port
    __u8 protocol;         // IP protocol
    __u8 tcp_flags;        // TCP flags
    __u16 arp_op;          // ARP operation
    __u8 icmp_type;        // ICMP type
    __u8 l7_payload[32];   // Layer 7 inspection data
} __attribute__((packed));

Key Design Decision: We only capture 75 bytes per event. This is enough for classification and L7 inspection without the overhead of full packet capture.

2. Multi-Protocol Detection

The eBPF program identifies 7 different protocol types:

// Simplified version of the protocol detection logic
SEC("tc")
int monitor_ingress(struct __sk_buff *skb) {
    void *data = (void *)(long)skb->data;
    void *data_end = (void *)(long)skb->data_end;

    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return TC_ACT_OK;

    // ARP detection
    if (eth->h_proto == bpf_htons(ETH_P_ARP)) {
        return handle_arp(skb, eth, data_end);
    }

    // IP packet processing
    if (eth->h_proto == bpf_htons(ETH_P_IP)) {
        struct iphdr *ip = (void *)(eth + 1);
        if ((void *)(ip + 1) > data_end)
            return TC_ACT_OK;

        switch (ip->protocol) {
            case IPPROTO_TCP:
                return handle_tcp(skb, eth, ip, data_end);
            case IPPROTO_UDP:
                return handle_udp(skb, eth, ip, data_end);
            case IPPROTO_ICMP:
                return handle_icmp(skb, eth, ip, data_end);
        }
    }

    return TC_ACT_OK;
}

3. Layer 7 Inspection

Here's where it gets interesting. We extract application-layer data for DNS, HTTP, and TLS:

// DNS query extraction
static __always_inline int handle_dns(struct __sk_buff *skb, 
                                       struct udphdr *udp,
                                       void *data_end) {
    void *dns_data = (void *)(udp + 1);

    // Copy up to 32 bytes of DNS payload
    if (dns_data + 32 <= data_end) {
        bpf_probe_read(&event.l7_payload, 32, dns_data);
        event.event_type = EVENT_DNS;
    }

    bpf_ringbuf_submit(&event, 0);
    return TC_ACT_OK;
}

// HTTP detection (simplified)
static __always_inline int handle_http(struct __sk_buff *skb,
                                        struct tcphdr *tcp,
                                        void *data_end) {
    void *http_data = (void *)(tcp + 1);

    // Check for HTTP methods
    if (http_data + 4 <= data_end) {
        char method[4];
        bpf_probe_read(&method, 4, http_data);

        if (method[0] == 'G' && method[1] == 'E' && 
            method[2] == 'T' && method[3] == ' ') {
            event.event_type = EVENT_HTTP;
            bpf_probe_read(&event.l7_payload, 32, http_data);
        }
    }

    return TC_ACT_OK;
}

// TLS handshake detection
static __always_inline int handle_tls(struct __sk_buff *skb,
                                       struct tcphdr *tcp,
                                       void *data_end) {
    void *tls_data = (void *)(tcp + 1);

    if (tls_data + 5 <= data_end) {
        __u8 content_type;
        bpf_probe_read(&content_type, 1, tls_data);

        // 0x16 = Handshake
        if (content_type == 0x16) {
            event.event_type = EVENT_TLS;
            bpf_probe_read(&event.l7_payload, 32, tls_data);
        }
    }

    return TC_ACT_OK;
}

Challenge #1: The Verifier

The eBPF verifier is strict. Every memory access must be bounds-checked:

// This will fail verification:
char *data = packet + offset;
*data = value;  // ❌ No bounds check

// This passes:
if ((void *)(packet + offset + 1) <= data_end) {
    char *data = packet + offset;
    *data = value;  // ✅ Verified safe
}

User-Space Processing: Go Application

The Go application reads events from the ring buffer and performs higher-level analysis:

1. Ring Buffer Polling

func (m *NetworkMonitor) pollEvents(ctx context.Context) {
    rb, err := m.module.InitRingBuf("events", m.eventsChannel)
    if err != nil {
        log.Fatal(err)
    }
    defer rb.Close()

    rb.Poll(300) // 300ms timeout

    for {
        select {
        case <-ctx.Done():
            return
        case record := <-m.eventsChannel:
            m.processEvent(record)
        }
    }
}

2. Smart Deduplication with LRU Cache

To avoid alert fatigue, we only show new traffic patterns:

type NetworkMonitor struct {
    deviceCache  *lru.Cache        // Device tracking
    patternCache *lru.Cache        // Unique communication patterns
    db           *buntdb.DB        // Persistent storage
}

func (m *NetworkMonitor) processEvent(data []byte) {
    event := parseNetworkEvent(data)

    // Track device
    deviceKey := event.SrcMAC.String()
    if !m.deviceCache.Contains(deviceKey) {
        m.announceNewDevice(event)
        m.deviceCache.Add(deviceKey, true)
    }

    // Track pattern (first occurrence only)
    patternKey := fmt.Sprintf("%s:%s:%d:%s",
        event.SrcMAC, event.DstIP, event.DstPort, event.EventType)

    if !m.patternCache.Contains(patternKey) {
        m.printTrafficPattern(event)
        m.patternCache.Add(patternKey, true)
    }

    // Update statistics
    m.updateStats(event)
}

3. Layer 7 Parsing in User Space

func parseL7Payload(event *NetworkEvent) string {
    switch event.EventType {
    case EVENT_DNS:
        return parseDNSQuery(event.L7Payload)
    case EVENT_HTTP:
        return parseHTTPRequest(event.L7Payload)
    case EVENT_TLS:
        return "TLS"
    default:
        return ""
    }
}

func parseDNSQuery(payload []byte) string {
    // Skip DNS header (12 bytes)
    offset := 12

    var domain []byte
    for offset < len(payload) {
        length := int(payload[offset])
        if length == 0 {
            break
        }

        if len(domain) > 0 {
            domain = append(domain, '.')
        }

        offset++
        if offset+length > len(payload) {
            break
        }

        domain = append(domain, payload[offset:offset+length]...)
        offset += length
    }

    return string(domain)
}

Performance Optimizations

1. Zero-Copy with Ring Buffers

Ring buffers allow the kernel to write directly to memory shared with user space:

// No data copying - just reading shared memory
rb.Poll(300)

2. Batch Database Writes

Instead of writing every event to disk, we batch them:

func (m *NetworkMonitor) persistStats() {
    ticker := time.NewTicker(30 * time.Second)
    defer ticker.Stop()

    for range ticker.C {
        m.db.Update(func(tx *buntdb.Tx) error {
            for mac, stats := range m.deviceStats {
                data, _ := json.Marshal(stats)
                tx.Set(mac, string(data), nil)
            }
            return nil
        })
    }
}

3. LRU Cache Tuning

// Balance memory vs. accuracy
deviceCache, _ := lru.New(1000)   // Track 1000 devices
patternCache, _ := lru.New(10000) // Track 10k patterns

Real-World Output

Here's what you see when running Cerberus:

NEW DEVICE DETECTED!
   MAC:     dc:62:79:2f:39:28
   IP:      192.168.0.108
   Vendor:  Apple
   First Seen: 2024-12-06 16:51:12

[DNS] 192.168.0.100 (aa:bb:cc:dd:ee:ff) [Apple] → 8.8.8.8:53 [google.com]
[HTTP] 192.168.0.100 (aa:bb:cc:dd:ee:ff) [Apple] → 93.184.216.34:80 [GET /api/v1/users]
[TLS] 192.168.0.100 (aa:bb:cc:dd:ee:ff) [Apple] → 142.250.185.46:443 [TLS]
[TCP] 192.168.0.50 (11:22:33:44:55:66) [Raspberry Pi] → 192.168.0.200:22 (SSH)

╔════════════════════════════════════════════════════╗
║         NETWORK STATISTICS SUMMARY                 ║
╠════════════════════════════════════════════════════╣
║ Total Devices: 15                                  ║
║ Total Packets: 45821                               ║
║   - TCP:  38456                                    ║
║   - UDP:  6120                                     ║
║   - DNS:  892                                      ║
║   - HTTP: 156                                      ║
║   - TLS:  1834                                     ║
╚════════════════════════════════════════════════════╝

Challenges and Solutions

Challenge #1: eBPF Complexity Limit

eBPF programs are limited to ~1 million instructions. Solution: Keep parsing logic simple and move complex analysis to user space.

Challenge #2: Payload Size Tradeoff

More payload = better L7 inspection, but higher overhead. Solution: 32 bytes is enough for DNS queries, HTTP methods, and TLS detection.

Challenge #3: TC vs XDP

Initially considered XDP (eXpress Data Path) but TC offered better compatibility and easier development. XDP is faster but more restrictive.

Challenge #4: Cross-Kernel Compatibility

Different kernel versions support different eBPF features. Solution: Use CO-RE (Compile Once, Run Everywhere) via libbpf.

Lessons Learned

Start simple: Basic packet capture first, then add L7 inspection
Trust the verifier: If it rejects your code, there's probably a real bug
Test incrementally: Use bpftool to inspect maps and programs
Memory matters: Every byte in your event structure adds overhead
User space is your friend: Don't try to do everything in eBPF

Production Considerations

For critical infrastructure monitoring (my day job), I've learned:

Alert fatigue is real: Smart deduplication is essential
Performance monitoring: Track your own overhead
Graceful degradation: Handle packet loss at high traffic volumes
Security: Limit L7 payload capture to metadata only

What's Next?

The roadmap for Cerberus includes:

Redis backend for distributed deployments
Prometheus metrics export
Anomaly detection using pattern baselines
IPv6 support
Extended L7 inspection (128-256 byte payloads)
Web dashboard for visualization

Try It Yourself

git clone https://github.com/zrougamed/cerberus.git
cd cerberus
make
sudo ./build/cerberus

Requirements:

Linux kernel 4.18+
Go 1.24+
Root privileges

Contributing

Cerberus is open source and looking for contributors! Whether you're interested in:

eBPF kernel programming
Go application development
Network protocol analysis
Performance optimization
Documentation

Check out the GitHub repository and open an issue or PR.

Resources

Conclusion

Building Cerberus taught me that eBPF isn't just about performance - it's about rethinking how we approach observability. By moving intelligence into the kernel, we can monitor networks at scale without sacrificing visibility.

The combination of eBPF's efficiency and Go's expressiveness creates a powerful platform for building modern monitoring tools. Whether you're securing critical infrastructure or just curious about network traffic, eBPF opens up possibilities that weren't feasible before.

What would you build with eBPF? Let me know in the comments!

Mohamed Zrouga is a Senior Platform Engineer at Deltaflare, specializing in critical infrastructure protection and DevSecOps. Connect on GitHub or visit zrouga.email

Build a Secure Multi-Tenant SSO System with Keycloak, Go & React: Step-by-Step Guide

Mohamed Zrouga — Sat, 07 Jun 2025 19:55:54 +0000

Managing authentication across multiple tenants can be a nightmare—different domains, identity providers, and security models all add complexity. In this guide, you’ll learn how to build a production-ready multi-tenant SSO system using Keycloak, React, and Go, enabling your users to log in with Google, GitHub, or Microsoft accounts, while keeping each tenant securely isolated.

Stack Overview

Layer	Tech	Purpose
Identity	Keycloak	Central identity provider (OAuth2, JWT, social login, multi-tenancy)
Frontend	React	User and admin interfaces
Backend	Go	Stateless API with JWT validation and business logic
Data	PostgreSQL	Stores tenants, users, and metadata
DevOps	Docker Compose	Container orchestration for local setup

Architecture: A Layered Security Model

The solution follows a cleanly separated, layered architecture:

Identity Providers Layer: Social login via Google, Microsoft, and GitHub
Frontend Layer:
- React user interface (http://localhost:3000)
- Admin dashboard (http://localhost:3001)
Authentication Layer:
- Keycloak (http://localhost:8080) handles all identity and access flows
Backend Layer:
- Go API (http://localhost:8081) that verifies JWTs and executes tenant-aware logic
Data Layer:
- PostgreSQL database for user, tenant, and relationship data

Why This Stack?

Keycloak

Open-source IAM platform with OAuth2, OpenID Connect, social login, and multi-tenancy.
Saves you from writing brittle auth code.
Highly customizable with realms, clients, and mappers.

React

Modern, component-driven framework.
Perfect separation of user frontend and admin interface.
Great developer experience and large ecosystem.

Go

Lightweight and performant.
Excellent standard library for JWT handling and HTTP.
Strong concurrency model for scalable APIs.

Authentication Flow

Here’s how the secure OAuth2-based login works:

User starts login from the React frontend.
Keycloak redirects to the selected identity provider.
User authenticates, and the OAuth callback returns to Keycloak.
Keycloak issues a JWT with user and tenant context.
Frontend includes the JWT in API requests.
Go backend validates the token and extracts tenant info.
Tenant-isolated operations run based on JWT claims.

This setup ensures centralized auth with stateless APIs and strong tenant isolation.

Multi-Tenancy: Key Considerations

Concern	Solution
Tenant Isolation	Users and data are scoped to their own organization.
Flexible Auth	Each tenant chooses which social login providers to enable.
Self-Service Admin	Admin panel allows tenant admins to manage users without overlap.
Scalable Infrastructure	Each component is containerized and independently scalable.

Security Best Practices

JWT Validation: Go backend checks tokens against Keycloak’s JWKS endpoint.
CORS Configuration: Proper headers configured in both Keycloak and backend.
Environment Variables: All sensitive values are managed via .env files.
Isolated Networks: Docker containers communicate over private networks.

Getting Started

Clone and boot the system using Docker Compose:

# Clone and navigate to the repo
git clone https://github.com/zrougamed/multitenant-sso
cd multitenant-sso

# Start all services
docker-compose up -d

Access services at:

Configuring OAuth Providers

Google

Go to Google Cloud Console
Create OAuth credentials
Set redirect URI: http://localhost:8080/realms/{realm}/broker/google/endpoint

GitHub

Go to GitHub Developer Settings
Set callback URI: http://localhost:8080/realms/{realm}/broker/github/endpoint

Microsoft

Register an app via Azure App Registrations
Set redirect URI: http://localhost:8080/realms/{realm}/broker/azure/endpoint

Common Challenges and Solutions

Problem	Solution
Port Conflicts	Modify `docker-compose.yml` or Keycloak config.
CORS Errors	Ensure correct CORS settings in backend and Keycloak.
JWT Validation Fails	Double-check audience (`aud`), issuer (`iss`), and realm configuration.

Extending the System

Add more identity providers: Twitter, Facebook, etc.
Add Role-Based Access Control (RBAC) via Keycloak groups and roles.
Implement API rate limiting in Go middleware.
Add monitoring with Prometheus, Grafana, or ELK.

Production Considerations

High Availability: Cluster Keycloak and PostgreSQL with redundancy.
SSL/TLS: Use reverse proxy like NGINX or Traefik for HTTPS.
Backups: Regular backups for Keycloak and PostgreSQL.
Monitoring: Alerting on auth failures, performance, and token expiry issues.

Conclusion

Building a secure, scalable, multi-tenant SSO system doesn’t have to be painful. By using Keycloak for identity management, React for the frontend, and Go for the API, you get a powerful and maintainable architecture that scales with your needs.

The system supports:

Multi-tenant user isolation
Social logins via OAuth
Stateless APIs with secure JWTs
Admin management for each tenant

Source Code

👉 GitHub Repository – multitenant-sso

Questions or feedback?

Open an issue on GitHub or connect with me on LinkedIn

Open Source Notification Scheduler in Go! [Looking for Contributors]

Mohamed Zrouga — Mon, 16 Dec 2024 02:37:43 +0000

Hi Dev! 👋

I’m excited to share my latest open-source project, Dynamic Notification System! 🎉

This is a Go-based extensible notification scheduler that supports multiple channels, including:

📨 Slack
🔔 Discord
📩 Telegram
📧 SMTP
🖥️ Webhooks
And many more!
With dynamic plugin support, the system allows you to add new notification channels effortlessly. It’s designed for cross-platform compatibility (Linux, macOS, Windows) and supports Go 1.23+.

Why Contribute?

Learn and improve your skills in Go and dynamic plugin systems.
Work on cross-platform builds for Linux, macOS, and Windows.
Be part of a growing open-source project and build your portfolio.
Help the community by adding new features and optimizing existing ones.

How You Can Help
We’re looking for contributors to:

Add new notification channels like Twilio, Signal, etc.
Improve documentation and examples.
Fix bugs and optimize builds.
No matter your experience level, we’d love your input!

Links
🔗 GitHub Repository: Dynamic Notification System

Feel free to check out the project, try it, and let us know your thoughts! Contributions, feedback, and stars 🌟 are all greatly appreciated. Let’s build something amazing together! 💪

DEV Community: Mohamed Zrouga

I'm not an ML engineer. I built one anyway.

The observability stack was heavier than the workload

Why eBPF is the right layer for this

What "ML-Lite" actually means

Stage 1: Aggregate into windows

Stage 2: Build a baseline

Stage 3: Explain the score

The evolution wasn't planned

Honest limitations

What I'm still trying to understand

Thank you

I Got Tired of Docker Eating My Raspberry Pi's RAM — So I Built My Own Container Orchestrator

What Actually Runs a Container?

nyxd: What It Is and What It Isn't

Why crun Instead of runc

The CVE Surface Nobody Benchmarks

The Networking Decision

Kernel Requirements (the honest list)

Benchmarking This Honestly

What the Raspberry Pi Actually Taught Me

Where nyxd Is Right Now

Working (end-to-end in the daemon + nyx client)

Implemented but not product-complete (nuance)

Still being hardened / incomplete

Bottom line

The Bigger Question

Try It / Follow Along

Meet Orion-Belt, Go ZeroTrust Bastion

Orion-Belt in Action

The "Security Tax" of Traditional Access

Feature Comparison: Why Orion-Belt?

How it Works (Under the Hood)

Key Features for Modern Teams

1. ReBAC (Relationship-Based Access Control)

2. Session DVR-Style Replay

3. JIT (Just-In-Time) Temporary Access

The Architecture

Quick Start

I need your feedback!

Building a Security Test Lab with QEMU: From Zero to Network Monitoring

Why QEMU for Your Test Lab?

What We'll Build

Prerequisites

Step 1: Installing QEMU

Step 2: Creating Your First VM - The Gateway

Creating the Disk Image

Download a Minimal OS

First Boot and Installation

Step 3: Setting Up Advanced Networking

Create a Network Configuration Script

Characteristics:

Network Teardown Script

Step 4: Creating VM Launch Scripts

Gateway VM Script

Quick Clone Function for Additional VMs

Application VM Script

Attacker VM Script

Step 5: Network Configuration Inside VMs

Gateway VM (10.0.100.1)

Application VM (10.0.100.10)

Attacker VM (10.0.100.20)

Step 6: Snapshots for Quick Resets

Step 7: Practical Use Cases

Testing Network Monitoring Tools

Simulating Attack Scenarios

Testing Firewall Rules

Container Testing

Step 8: Automation and Management

Performance Tips

Troubleshooting Common Issues

Next Steps and Advanced Topics

Conclusion

Building a Real-Time Network Monitor with eBPF: Lessons from Cerberus

Why eBPF for Network Monitoring?

Architecture Overview

The eBPF Program: Kernel-Space Magic

1. Packet Parsing at Line Rate

2. Multi-Protocol Detection

3. Layer 7 Inspection

Working (end-to-end in the daemon + `nyx` client)