Attacking A Kubernetes Cluster (Enter Red Team Mode)

There have been several reports over the years from organizations like Red Hat and various security research firms and independent engineers that give us a few statistics. Two that are crucial to understand are:

• 93% of organizations suffer from a Kubernetes-related security risk.
• 55% of the security risks are due to misconfigurations.

A misconfiguration could be anything from improper RBAC configurations to a Kubernetes Service that has too much exposure.

In this blog post, you’ll learn about some of the common issues, where to check for vulnerabilities, and some pentesting tips to check if your cluster is mitigating attacks.

Prerequisites

To use some of these tools and test them more easily, you can use Minikube. Minikube is a method of creating a local Kubernetes cluster on your computer.

You can learn how to install Minikube and create a cluster locally here.

Common Security Issues On Kubernetes

The first step on your Red Team journey is to understand the common security issues on Kubernetes. Much like any other cyber security implementation, you need to understand the system before you can secure it.

There are four common issues that you’ll see from a security/misconfiguration perspective.

1. API Server exposed publically.
2. Proper Service Accounts are not used for creating Pods.
3. Overpermissive RBAC.
4. Pods aren’t configured properly (think SecurityContext).

There are also huge security issues within the code itself that’s running in containers that are running inside of Pods, but chances are (hopefully) those security issues were mitigated prior to the code being containerized.

As you can see from the list, it typically comes down to networks not being locked down, having way too many permissions available, or Pods not configured properly.

Let’s take the Pod example.

A big security flaw that attackers can find is Service Account Tokens. Unless otherwise specified, Service Account tokens are created by default. When you create a Pod, unless you specify it within the Manifest, the Default Service Account is used. That means with just these misconfigurations, which are “on” by default, all it takes is a bad actor to access the Default Service Account. If that happens, every single one of your Pods are compromised within your environment.

As an ethical hacker (because we’re all good people here, right?), these are the four places you’ll typically look with tools like Metasploit and Nmap.

Vulnerability Checks

The second step in your journey is to figure out what vulnerabilities currently exist. These vulnerabilities could be on Kubernetes itself or common tools/software that’s used on Kubernetes.

There are a few places you can check for current vulnerabilities:

• Official CVE docs: https://kubernetes.io/docs/reference/issues-security/official-cve-feed/
• Mitre: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=kubernetes
• SecurityScoreboard: https://www.cvedetails.com/vulnerability-list/vendor_id-15867/product_id-34016/Kubernetes-Kubernetes.html
• NIST: https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&query=kubernetes&search_type=all

Let’s take an example from NIST:

Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes.
It’s possible for authenticated users to enumerate clusters by name by inspecting
error messages. It’s also possible to enumerate the names of projects with
project-scoped clusters if you know the names of the clusters.
This vulnerability is fixed in 2.11.3, 2.10.12, and 2.9.17.

ArgoCD is one of the most popular solutions for deploying application stacks in Kubernetes right now. With this vulnerability (if a cluster isn’t running one of the fixed ones, and I’m sure there are a few), as long as you’re authenticated and have read permissions, you can see the error messages, which could potentially give you access to enumerate clusters.

Container Image Vulnerabilities

Another huge vulnerability factor you’ll find is within container images. Even the container images that are official from various organizations still have a ton of vulnerabilities. Let’s test this theory out.

Using the docker scout command (previously docker scan), you can scan publicly accessible container images. Once you scan these images, you’ll know the vulnerabilities. If you can find one of these vulnerable container images used within a container that's deployed from a Pod, you may be able to gain certain access via the Pod (especially if the Service Account is also compromised) to perform unethical actions.

docker scout cves ubuntu:latest
    i New version 1.9.3 available (installed version is 1.8.0) at https://github.com/docker/scout-cli
    ✓ Pulled
    ✓ Image stored for indexing
    ✓ Indexed 130 packages
    ✗ Detected 6 vulnerable packages with a total of 8 vulnerabilities

## Overview

                    │       Analyzed Image
────────────────────┼──────────────────────────────
  Target            │  ubuntu:latest
    digest          │  010f94447a26
    platform        │ linux/arm64/v8
    vulnerabilities │    0C     0H     2M     6L
    size            │ 29 MB
    packages        │ 130

Kubernetes Misconfiguration Attacks

The third step is to test out (ethically of course) some of the security holes within a system.

The three methods/tools used will be:

• Metasploit: See security vulnerabilities and perform penetration testing (ethically of course) attacks against selected environments.
• Nmap: Network scanner. It’s a great tool to see what’s publicly exposed.
• Permissions: In Kubernetes with just the right access, it’s incredibly easy to create an admin-like Pod to do just about anything you’d like.

Metasploit

Open the Metasploit console on the terminal if you’re not going in via the Desktop.

msfconsole

Search for the Kubernetes module.

msf6 > search kubernetes

Set the IP or DNS of the Control Plane.

set RHOST https://IP_OR_DNS_OF_CONTROL_PLANE:443

Use the command below to see what Metasploit allows you to expose.

show actions

You should see an output similar to the below.

Auxiliary actions:

       Name        Description
       ----        -----------
   =>  all         enumerate all resources
       auth        enumerate auth
       namespace   enumerate namespace
       namespaces  enumerate namespaces
       pod         enumerate pod
       pods        enumerate pods
       secret      enumerate secret
       secrets     enumerate secrets
       version     enumerate version

You can now choose what you want to do on the cluster by running one of the commands (the commands are under the Name category).

To actually run these commands, you need a JWT Token. It could be, for example, a Service Account Token from a compromised Service Account.

In the next section, you’ll see how to create one.

Permission Escalation

This is a pretty common theme - the ability to create Roles that have full admin.

Notice how the ClusterRole below opens up all access to every resource within a Kubernetes cluster.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: all
rules:
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

With a ClusterRole like this, you can bind it to a Service Account and do just about anything you’d like within the cluster.

nmap

First, get the Control Plane IP.

kubectl cluster-info

You should see an output similar to the one below.

Kubernetes control plane is running at https://127.0.0.1:50779
CoreDNS is running at https://127.0.0.1:50779/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

kubectl cluster-info dump exposes just about every piece of information regarding the cluster configuration that you have access to see.

To narrow down what we need for nmap, you can grep the info for Pod and Service CIDRs.

# Pod IP ranges
kubectl cluster-info dump | grep -m 1 cluster-cidr

# Service IP Ranges
kubectl cluster-info dump | grep -m 1 service-cluster-ip-range

To scan for the common Ports open on a Kubernetes cluster (you’ll see to specify one of the CIDR ranges from above).

nmap -n -T4 -p 443,2379,6666,4194,6443,8443,8080,10250,10255,10256,9099,6782-6784,30000-32767,44134 $CIDR

To discover Control Planes (notice how these are the common Ports that a Control Plane runs on).

nmap -Pn -sS -sV -p 443,6443,8443,8080,2379,2380 $CIDR

To discover Worker Nodes (notice how these are the common Ports that a Worker Nodes runs).

nmap -Pn -sS -sV -p 10250,10255 $CIDR

To take it a step further, you can also search for specific Ports that services typically run on. For example, here’s how to scan for nodePort exposed Services.

nmap -Pn -sS -sV -p 30000-32767 $CIDR

Closing Thoughts

Yes, “hacking” isn’t the easiest thing in the world without the proper authentication and authorization. The problem is that due to the severe amount of misconfigurations within production-level Kubernetes clusters, it’s getting easier and easier to find an attack surface.

Kubernetes has multiple layers - the hosts, the host network, the Pods, the Pod network, and the containers running inside of the Pods. Finding a hole in one of them could be a security breach.