The best observability is invisible. It shouldn’t require SDKs, it shouldn’t require sidecars, and it definitely shouldn’t eat up half your cluster’s CPU.
Why build another tool?
In the world of Kubernetes, observability is often a choice between two extremes: too little or too much.
On one hand, you have standard application logs. They tell you what the app thinks happened, but they don’t tell you what actually happened on the wire. On the other hand, you have massive Service Meshes like Istio or Linkerd. While they give you incredible insights, they come with a heavy “sidecar tax” — injecting a proxy into every pod, adding latency, and making your YAML files look like a phone book.
I realized that all the information I needed every HTTP header, every JSON body, every latency spike was already floating through the air (or rather, the virtual wires) of my Kubernetes nodes. It was just a matter of reaching out and grabbing it.
The Vision for KubeSocket
I wanted to build something lean and simple, something which can just make the job done without any extra overhead or infrastructure configuration. The goal was:
- No extra sidecar container (zero maintenance)
- No SDKs and dependencies
- No dependency on any programming framework (zero configuration)
- Should be platform agnostic
Then, I asked myself; “How about we capture the internet traffic entering in the host machine before it touches anything?”
This meant that something which can talk directly to host NIC (Network Interface Card), copy raw bytes (0s and 1s), parse ethernet headers and extract HTTP payload. Since HTTP payload starts at Layer 7, it means that we need to capture the entire raw Ethernet packet entering the host machine through NIC mounted on main motherboard before anyone touches it or it gets modified by any application/webserver, etc. Keeping it raw and simple. Pretty interesting.
To visualise it, this is something I wanted:
Ethernet packet flying from wire directly to RAM (OS kernel)
Network Interface Card (NIC)
A Network Interface Card (NIC), also known as a network adapter or LAN adapter, is a hardware component that allows a computer or other device to connect to a network and communicate with other devices.
Credits: https://levens.fr/523572/Card-10-100-1000-Mbps-Low-Profile-LAN-Adapter-For-Desktop-PC
The Physical Arrival (Layer 1 & 2)
As electrical or optical signals arrive at the NIC, the hardware’s controller performs Frame Delimitation. It identifies where a frame starts and ends, verifies the Checksum (FCS) to ensure the data wasn’t corrupted in flight, and checks the Destination MAC address.
The Ring Buffer
The most important concept for a packet-sniffer developer is the RX (Receive) Ring Buffer. Imagine a circular conveyor belt in your RAM.
- The NIC places a packet on a spot on the belt.
- The NIC sends a “signal” (an interrupt) to the CPU.
- The Kernel (the Driver) walks over, picks up the packet from that spot, and moves it into the OS networking stack.
- The Belt keeps spinning, ready for the next packet.
How an ethernet packet goes into OS kernel
Keep Reading!
That’s it, we have now understood how the electrical signals entering the host NIC gets converted into a packet and then how that packet flies through NIC to main system RAM making it accessible for Operating System (OS) kernel to process it for user.
In next blog, I’ll talk about this raw ethernet packet, what’s inside it and how can we parse it to read the raw HTTP data.
Top comments (0)