DEV Community

AWS Community Builders profile image Huỳnh Lê Nhất Nghĩa
Huỳnh Lê Nhất Nghĩa for AWS Community Builders

Posted on

eBPF From Scratch: from the eBPF VM to writing your own tools (tested on a live Cilium cluster)

#linux #kubernetes #devops #kernel

Right now, on one worker of a Kubernetes cluster I built, 140 eBPF programs are running inside the Linux kernel — routing every packet, controlling device access, collecting metrics. Nobody recompiled the kernel. Nobody loaded a module.

That sentence is the whole reason I wrote eBPF From Scratch: a free, 22-chapter series that takes you from "what even is the eBPF virtual machine" all the way to writing and loading your own eBPF programs — in C with libbpf + CO-RE, and from Go with cilium/ebpf.

👉 Read it free (English): https://kkloudtarus.net/en/blog/series/ebpf-from-scratch
💻 Source code: https://github.com/nghiadaulau/ebpf-from-scratch

What makes it different

  • Everything is tested on real hardware — a Kubernetes cluster running kernel 6.17 and Cilium 1.19 (kube-proxy-less, hundreds of BPF programs live) is the lab throughout. No hand-wavy diagrams; we dissect programs that are actually running.
  • Grounded in official docs — ebpf.io, kernel.org, libbpf, cilium. Claims are verified, not vibes.
  • Deep-dive, not surface — we go down to registers, the verifier's safety proofs, JIT, maps, and the exact lifecycle a program goes through.
  • It's also fully bilingual (English + Vietnamese).

A quick taste — every eBPF concept is something you can see on a real node:

sudo bpftool prog show id 2871
Enter fullscreen mode Exit fullscreen mode 
2871: sched_cls  name tail_no_service_ipv4  tag fe7bcb57c001d434  gpl
    xlated 4920B  jited 2778B  memlock 8192B  map_ids 171,631
    btf_id 758
Enter fullscreen mode Exit fullscreen mode

xlated = bytecode after the verifier accepted it. jited = native machine code. map_ids = how it keeps state. That's eBPF, not on a slide — running.

What you'll learn (7 parts, 22 chapters)

Part I — Foundations

  • The eBPF Virtual Machine: registers, instruction set, and bytecode
  • The Verifier: why eBPF doesn't crash the kernel
  • Maps: memory and the bridge to userspace
  • Program types and hooks: where you attach, what you see
  • BTF and CO-RE: compile once, run on every kernel

Part II — Tracing

  • bpftrace from a one-liner to maps, counting and histograms
  • uprobe, USDT, and inspecting a pod from the host

Part III — Writing real tools

  • libbpf + CO-RE: writing an eBPF tool yourself (C)
  • cilium/ebpf: loading eBPF from Go

Part IV — Networking

  • XDP: processing packets at the earliest point — writing a firewall
  • tc/sched_cls and dissecting a live Cilium datapath
  • Writing a tc program yourself: __sk_buff and the tcx chain

Part V — Security

  • LSM BPF: enforcing security right inside the kernel
  • seccomp-bpf: filtering syscalls in every container
  • The Tetragon way: from observe to enforce with bpf_send_signal

Part VI — Observability

  • CPU profiling with perf_event (the foundation of flame graphs)
  • Off-CPU and scheduler latency
  • Inside Hubble: from eBPF events to cluster-wide network flows

Part VII — Putting it together

  • Case study: a packet through Cilium's eBPF datapath
  • Capstone: writing connmon, a node-wide TCP connection monitor

Who it's for

Backend / platform / SRE / security folks who keep hearing "Cilium does that with eBPF" and want to actually understand — and write — the thing. You'll want to be comfortable on the Linux command line; everything else is built up from zero.

If you read any of it, I'd genuinely love feedback — what was clear, what wasn't. And if it's useful, a ⭐ on the repo helps a lot.

Start here → https://kkloudtarus.net/en/blog/series/ebpf-from-scratch

Top comments (0)