Applying Container Runtime Policies with eBPF Tetragon

#guide #software

Tetragon, an eBPF (extended Berkeley Packet Filter)-based tool, enables the real-time auditing and restriction of processes, file accesses, and network connections by applying runtime security policies in container environments. This allows unauthorized or suspicious behaviors to be detected and automatically blocked, preventing potential security breaches. In modern dynamic container infrastructures where traditional security approaches fall short, Tetragon significantly strengthens the security posture by offering deep visibility and control at the kernel level.

What is eBPF and its Role in Container Security?

eBPF is a revolutionary technology that allows programs to run in a secure sandbox environment within the Linux kernel. These programs attach to kernel events (system calls, network packets, file accesses, etc.), offering the ability to collect telemetry, filter events, and even modify behaviors at the kernel level. In the context of container security, eBPF can transparently monitor and audit all interactions between the operating system and the application.

Traditional security tools typically operate at the user-space level and use mechanisms that create overhead to access kernel-level events. eBPF, however, runs directly within the kernel, collecting high-resolution data and enforcing policies with minimal performance impact. This is a critical advantage, especially in environments with fast-moving and short-lived container workloads. For example, if an unexpected process is launched within a container or a sensitive file is accessed, eBPF can instantly detect this event and block it with a predefined security policy.

💡 Advantages of eBPF

eBPF offers advantages such as deep visibility at the kernel level, low performance impact, dynamic programmability, and a wide range of applications. These features make eBPF indispensable, especially for container and cloud security.

Being so close to the kernel's inner workings provides great power, but also necessitates that this power be used correctly and securely. eBPF programs are strictly audited and verified by the kernel's security mechanisms, preventing faulty or malicious programs from crashing the kernel. This robust structure makes eBPF an ideal foundation for container runtime security.

What is Tetragon and Why is it Used?

Tetragon is an eBPF-based security observability and enforcement engine developed as part of the Cilium project. Designed specifically for Kubernetes environments, it monitors container behaviors at the kernel level via an eBPF agent running on each node in the cluster. Tetragon offers a wide range of security telemetry, covering not only network traffic but also events such as process execution, file access, system calls, and kernel module loading.

One of the primary reasons to use Tetragon is its ability to detect and prevent anomalous behaviors that containers might exhibit at runtime. For example, situations where a web server container suddenly tries to read the /etc/passwd file or establish an outbound SSH connection can be easily detected and stopped by Tetragon policies. Traditional firewalls or host-based monitoring tools often cannot see such internal container behaviors in sufficient depth.

Tetragon allows defining security policies using Kubernetes Native Custom Resource Definitions (CRDs). This means security policies can be managed as code (Infrastructure as Code) like other Kubernetes resources and integrated into CI/CD pipelines. This approach simplifies security operations automation and ensures a consistent security posture.

Tetragon Installation and Core Components

Installing Tetragon on a Kubernetes cluster can typically be done alongside Cilium or independently. Since Cilium already heavily uses eBPF for networking and security, it has a natural synergy with Tetragon. Installation is performed using Helm charts or kubectl commands.

First, you need to ensure your Kubernetes cluster meets the necessary kernel version (minimum 4.19 or higher) and other dependencies for eBPF. For full functionality, especially on ARM64 architectures, Linux kernel version 5.10 or higher is recommended. The following example shows how to install Tetragon using Helm:

# Add the Tetragon Helm repository
helm repo add cilium https://helm.cilium.io/

# Update the repository
helm repo update

# Install Tetragon
helm install tetragon cilium/tetragon --namespace kube-system

These commands deploy Tetragon's core components to your Kubernetes cluster. After installation, you will see Tetragon agents (running as a DaemonSet on each node) and the Tetragon Operator (running as a deployment) in the kube-system namespace.

ℹ️ Tetragon Components

Tetragon's main components are:

Tetragon Agent (DaemonSet): Runs on each Kubernetes node, loads eBPF programs into the kernel, and monitors security events and enforces policies.

Tetragon Operator (Deployment): Interacts with the Kubernetes API server, manages Tetragon CRDs, and ensures cluster-wide policy synchronization.

Tetragon CLI (tetra): A command-line tool used to query security events, validate policies, and check Tetragon's status.

After installation, you can use the kubectl get pods -n kube-system -l app.kubernetes.io/name=tetragon command to verify that Tetragon is running correctly. Ensure all pods are in the Running state. You can then install the tetra CLI tool to start monitoring the event stream.

Application Protection with Runtime Policies

Tetragon's power lies in the security policies defined via Kubernetes CRDs (Custom Resource Definitions). These policies determine the runtime behavior of containers (e.g., which programs they can run, which files they can access, or which network connections they can establish). Policies are defined using Kubernetes resources like CiliumClusterwideNetworkPolicy or CiliumNetworkPolicy and are enforced at the kernel level by Tetragon agents. CiliumClusterwideNetworkPolicy applies cluster-wide, while CiliumNetworkPolicy is limited to a specific namespace.

Let's illustrate with an example: an Nginx web server container typically doesn't need to establish any outbound network connections other than serving web files. If a vulnerability is exploited within this container and an attacker tries to establish an outbound connection, we want to prevent this. The following Tetragon policy prohibits outbound SSH (port 22) connections from pods with the nginx label:

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-nginx-egress-ssh
spec:
  endpointSelector:
    matchLabels:
      app: nginx
  egress:
    - toPorts:
        - port: "22"
          protocol: TCP
      action: Deny

This policy blocks all outbound connection attempts to TCP port 22 from any pod labeled app: nginx at the kernel level. Such an approach provides much more effective and granular control in situations where traditional firewalls often struggle with the dynamic IP addresses of Kubernetes pods. Since the policy definition is in YAML format, it can be easily managed and automated with version control systems (like Git).

⚠️ Considerations for Policy Design

When designing policies, it's crucial to apply the "least privilege" principle. That is, ensure a container only has the permissions absolutely necessary for its operation. Overly restrictive policies can break applications, while overly permissive policies can lead to security vulnerabilities. Carefully validate policies in test environments and gradually promote them to production.

Policies can be defined not only for networking but also for areas like file access and process execution. For example, preventing a container from running a shell program like /bin/bash can make shell access more difficult in case of exploitation. Such policies prevent containers from deviating from their expected behavior patterns, creating a layer of defense even against zero-day attacks.

Monitoring and Analyzing Security Events

Tetragon not only enforces policies but also records all kernel-level security events in detail. These events include process starts, file accesses, network connections, system calls, and more. The collected telemetry data is critical for Security Operations (SecOps) teams, as it provides deep visibility to understand what, when, and how a security breach occurred.

Tetragon can direct security events in a standard format (e.g., JSON) to stdout or a specific log destination. By default, JSON events are saved to /var/log/tetragon/tetragon.log. This allows for easy integration with log collection tools (Fluentd, Filebeat, etc.) and SIEM (Security Information and Event Management) systems (such as Splunk, ELK Stack, Grafana Loki). Event data comes with rich metadata, including information like which pod, which namespace, which user, and which process triggered the event.

The tetra CLI tool can be used to monitor the event stream in real-time:

# Monitor all security events in real-time
tetra get events

# Filter events for a specific pod
tetra get events --pod my-nginx-pod

# Show only process exec events
tetra get events --event-type PROCESS_EXEC

These commands allow you to view the event stream live and filter it according to specific criteria. Event data typically has a structure like this:

{
  "process_exec": {
    "process": {
      "pid": 12345,
      "uid": 0,
      "cwd": "/",
      "binary": "/bin/bash",
      "arguments": "-c 'whoami'",
      "container": {
        "id": "abc...",
        "name": "my-nginx-container",
        "pod": {
          "name": "my-nginx-pod",
          "namespace": "default"
        }
      }
    }
  },
  "type": "PROCESS_EXEC",
  "node_name": "k8s-node-1",
  "timestamp": "2026-08-11T10:00:00Z"
}

This level of detail is invaluable for performing root cause analysis when an anomaly is detected or a policy violation occurs.

Real-World Scenarios and Trade-offs

Tetragon offers a wide variety of benefits in real-world scenarios. Here are some examples:

Privilege Escalation Protection: In the event of a vulnerability exploitation within a container, an attacker's attempt to escalate privileges via sudo or setuid programs can be prevented. Tetragon reduces this risk by monitoring and restricting such critical system calls or program executions with policies.
Unauthorized Network Access Prevention: Sensitive database containers can be allowed to communicate only with specific internal services, while access to the external network or other unauthorized internal services can be blocked. This makes network segmentation much more granular at the kernel level.
Sensitive File Access Control: An application can be allowed to access only its own configuration files or data directories, while access to sensitive system files like /etc/shadow or /root can be blocked. This reduces the risk of data leakage or system integrity compromise.
Malware Detection and Prevention: By defining policies against known malware patterns (e.g., running a binary with a specific hash or connecting to suspicious domains), the activities of malware infected in containers can be stopped.

In addition to these benefits, there are also some trade-offs when using Tetragon:

Performance Overhead: Although eBPF programs run within the kernel, minimizing performance impact, a large number of complex policies and high event volume can create some overhead, especially on CPU and memory usage. While generally acceptable, this should be carefully tested for applications with critical performance requirements.
Policy Complexity: Defining granular security policies requires detailed knowledge and attention. A misconfigured policy can break the functionality of legitimate applications or create security gaps. Policy development and testing processes must be well-managed.
Learning Curve: The underlying concepts of eBPF and Tetragon differ from traditional system security approaches. An effective use of these technologies requires a certain learning curve. However, defining policies via Kubernetes CRDs helps flatten this curve.

This diagram visualizes how Tetragon operates in a Kubernetes environment and how security policies are applied and monitored. The flow progresses from the need, through installation, policy definition, enforcement, and finally to event monitoring and intervention stages.

Conclusion

eBPF Tetragon offers a powerful and flexible solution for ensuring runtime security in modern container and Kubernetes environments. With its deep kernel-level visibility and policy enforcement capabilities, it goes beyond traditional security tools to create a proactive defense layer against potential threats. While its installation is relatively straightforward, creating and managing effective policies requires the security team to have a good understanding of eBPF and container behaviors.

Tetragon is an indispensable tool, especially for implementing the "least privilege" principle and detecting anomalous container behaviors. Thanks to detailed event logging and SIEM integration, security operations teams can quickly analyze and respond to breaches. If you want to take the security of your Kubernetes infrastructure to the next level, I definitely recommend considering Tetragon. Although it requires some effort initially, the control and visibility it provides will more than pay off in the long run.