Kubernetes is the dominant container orchestration solution — adoption now stands at 86% of the market. The genius of Kubernetes is its ability to provide you with a framework to run distributed systems resiliently. However, it introduces a level of complexity that can be overwhelming. By following Kubernetes best practices around security, reliability, efficiency and monitoring, teams can set themselves up for a successful transition. In a series of blog posts, we’ll cover each of these topics, starting with security.
Kubernetes Best Practices: Security
Kubernetes abstracts away just enough of the infrastructure layer so that developers can freely deploy, while ops teams retain access to important governance and risk controls. The challenge is that development teams new to Kubernetes may neglect some critical security features. Often the easiest way to get something working is to soften its security.
Let’s look at three common security challenges, and how to overcome them.
A Good Burst vs. a Bad Burst
Kubernetes responds well to bursts in traffic — whether good or bad. In the event you see a legitimate burst of traffic, Kubernetes will scale up to meet the increase in demand. Your application will consume more resources in your cluster without any degradation of performance. That’s a major benefit. However, in the event of a denial-of-service (DoS) attack, Kubernetes will do exactly the same thing, and you’ll pay for that traffic overload.
K8S Best Practice #1 — Set limits against:
the number of concurrent connections per IP address
the number of requests each user can make per second, minute, or hour
the size of request bodies
and tune these limits for individual hostnames and paths
Granting Safe Levels of Access
The easiest way to deploy a new application or provision a new user is to give away admin permissions. But it’s also the most dangerous way — if an attacker gains access to that account, they’ll have access to everything.
K8S Best Practice #2- Employ role based access controls (RBAC) to adhere to the principle of least privilege.
RBAC allows you to grant users granular access to Kubernetes API resources. You should define access profiles using Roles or ClusterRoles. Using Roles, you'll grant access to a single namespace. With ClusterRoles, you can grant access to resources without namespaces, like Nodes and PersistentVolumes, as well as all namespaced resources.
While RBAC configuration can be confusing and verbose, tools like rbac-manager can help simplify the syntax. This helps prevent mistakes and provides a clearer sense for who has access to what.
The end result? By only granting workloads the permissions they need to do their job, you’ll limit the amount of damage an attacker can do to your Kubernetes environment.
Keep Kubernetes Secrets, Secret
If you are using Kubernetes infrastructure-as-code (IaC) patterns, you benefit from having a completely reproducible environment. But there’s a catch — part of your infrastructure likely includes Kubernetes Secrets, which store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. And you shouldn't be adding Secrets to your IaC repository.
K8S Best Practice #3 — It’s tempting to check your Kubernetes secrets into your infrastructure-as-code repository so that your builds are 100% reproducible.
But if you care about security, don’t. Once checked in, your secrets are permanently exposed to anyone with access to your Git repository.
The solution is to split the difference: encrypt all your secrets so you can safely check them into your repository without fear of exposing them. Then you’ll then only need access to a single encryption key to “unlock” your IaC repository, and have perfectly reproducible infrastructure. Open source tools like Mozilla’s SOPS can help with this.
You can read more best practices for k8s security by checking out how we implement security for our customer’s managed Kubernetes deployments.
Dig deeper into Kubernetes Best Practices for Security: Ensure cluster security with k8s secrets, network policies, and workload identity
Written By: Robert Brennan
Top comments (0)