Do you know what your k8s cluster does when you are not watching? Who establishes the TCP communication with him? Who is called by him, e.g., from third-party libraries?
Grafana, you can visualize TCP traffic in the cluster. See how your workloads communicate together. Inspect how many connections are established, how many bytes are exchanged, and how long these connections are active.
k8spacket is the tool written in Golang and uses the
gopacket third-party library to sniff TCP packets on your workloads (incoming and outgoing). It creates TCP listeners on running container network interfaces. When Kubernetes creates a new container, the CNI plugin is responsible for giving the possibility to communicate either with other containers or from cluster to the world. The most common approach uses
linux namespace to isolate networks and
veth pair to connect the isolated namespace with the bridge. If you want to know more, please refer to a great article written by Safak (How Docker Container Networking Works - Mimic It Using Linux Network Namespaces). Besides type
bridge, the CNI plugin can use other types (
macvlan), but all create a network interface for container linux namespace, which is the main handle for
k8spacket is running with the option
to eliminate the isolation mentioned earlier. It collects TCP streams, processes data, and exposes results through API to be used by
k8spacketis a Kubernetes API client and can resolve sniffed workload as a cluster resource name (
Services) visible on visualization.
- It is launched as a
DaemonSetto listen to network interfaces on all nodes.
- Implemented watcher checks every
10s(default) if there is a new (or old) network interface to observe (or forget).
In addition to showing a graph of your cluster resources, it also exposes
connection- helps understand how many connections are established between your workloads and from/to outside clients. It tells you which sockets stay open and could cause problems. It is helpful when you are behind some SNAT gateway with port exhaustion (e.g.,
bytes- shows how many bytes are sent or received by workloads.
duration- calculates the lifetime of connections.
by namespace- select one or more k8s namespaces
by names included- select workload names for visualization
by names excluded- exclude workload names from visualization
If you want to know what is going on with TCP traffic in your cluster, try
k8spacket. Investigation about cutting the traffic (
network policy) could be painless when you see a leak of packages outside. Make a review of third-party libraries to check connections with the world.