In this article, we will discuss Kubernetes security best practices and explore various solutions provided by Amazon Web Services (AWS).
As more organizations adopt microservices architecture, securing Kubernetes clusters becomes increasingly important. Kubernetes security comprises multiple levels - the underlying infrastructure, the Kubernetes platform itself, and the applications running within the clusters. Setting up a kubernetes cluster and configuring the deployment is already challenging enough and the security part is frequently left for after.
Important questions to ask when considering securing your kubernetes cluster:
- How is a kubernetes cluster secured by default?
- If i dont do anything, how secure are my resources?
- What are the security vulnerability gaps?
- What are the security best practices to close those gaps?
Security is a combination of multiple things at multiple levels. It is not just one or 2 things.
There are several security issues and vulnerabilities that could be encountered while building and deploying applications. To prevent these vulnerabilities, we can implement best practices.
- Access Control & Privilege Escalation
- Insufficient Logging & Monitoring
- Security Misconfigurations & Default Configurations
- Improper Secrets Management...just to name a few_
1. Image Scanning
The first step in building an application is building a secure image in the cicd pipeline. What security issues do we have here?
Here are 3 possibiliites:
Code from untrusted repositories: Code or Libraries in our application from untrusted registries/source. These may include virus or backdoors that grant access to an attacker.
Vulnerabilities in the operating system or libraries: We may also be using some packages in the operating system in our docker Image. These dependencies and tools may also have some vulnerabilities. Or the base Image we are using, may have some vulnerabilities.
Unnecessary dependencies: Choose leaner and smaller images, with less tools inside required to build the application image. An attacker can use a vulnerability in the container to break out and have access to the host or the kubernetes worker node
To mitigate such vulnerabilities, Scan container images using tools like AWS signer, which is a fully managed code-signing service to ensure the trust and integrity of your code. AWS also offers the Elastic Container Registry (ECR) image scanning solution. Regularly scanning images in the cicd pipeline before pushing it into the repository can help prevent vulnerabilities. 1
2. Avoid Root-Users in Containers and Run as Non-Root
Create service users and run applications with non-root users. Running containers with limited privileges will help harden security.
AWS provides Amazon Inspector which uses the service-linked role named AWSServiceRoleForAmazonInspector2. This service-linked role trusts the inspector2.amazonaws.com service to assume the role. 2
3. Use RBAC for User and Permission Management
Role-Based Access Control (RBAC) allows defining user roles and their permissions in Kubernetes. Make sure to adopt the least-privilege approach when assigning permissions.
AWS offers Attribute-based access control (ABAC), which is an authorization strategy that defines permissions based on attributes. In AWS, these attributes are called tags.
These tags can be attached to users, roles or resources. 3
4. Implement Network Policies
Use Network Policies to define which pods can communicate with each other and how traffic is distributed among them. As the name implies, network policy configures communication rules at a network level, if we want to define these rules at service level or an application level, which is a more logical level for cluster communication. We can use service mesh such as Istio. Calico or Weave are popular Kubernetes network plugins for implementing network policies. 4
5. Encrypt Communication with mTLS
By default all communication between pods is unencrypted So if an attacker gets into a cluster, they will be able to see all the communication.. in plain text!! Make use of service meshes like Istio to enable mutual Transport Layer Security (mTLS) to encrypt communication between pods. 5
6. Secure Secret Data with AWS KMS Or HashiCorp Vault
Another thing which is not secure by default is the secret. They are base 64 encoded, so anyone who wants to view the secrets, can simply decode it with base64 –decode and see them in plain text. Use EncryptionConfiguration to enable encryption of Kubernetes secrets or adopt third-party solutions like AWS Key Management Service (KMS) and HashiCorp Vault to manage secret data. 6
7. Secure the etcd Data Store
Secrets and all other k8s configuration data are stored in etcd, which is k8s backing store for all cluster data. Etcd resides in the control plane, having access to this will mean having access to the whole cluster. It is best to secure this… but how?
Protect the etcd data store by placing it behind a firewall and implementing proper authentication and encryption. In addition to that, the data can be encrypted, such that even if the attacker gets access to it, they won't be able to read it. RBAC as well as kubernetes best practice. 7
8. Regularly Backup and Restore Data
Attackers are known for infiltrating systems, stealing data and asking for ransom (Huge sums of money). This is a nightmare to every organization.
Automate backups and implement an immutable storage system using tools like velero. Velero is a popular open-source tool that can back up Kubernetes cluster resources and persistent volumes to externally supported storage backend on demand or by schedule. AWS customers can leverage this solution to centrally back up and restore Kubernetes objects and applications from and to EKS. 8
9. Configure Security Policies
Let's say as a Kubeadmin, you know and apply the above best practices and try to protect data and cluster. Clusters are usually used by developers. How do we make sure that these developers also apply these best practices when deploying their applications? Rules, such as pods that run privilege containers or containers with root-user cannot be deployed.The rules should also define network policy for every pod. The Kubernetes Pod security policy admission controller validates Pod creation and update requests against a set of rules.
Use third-party tools such as Open Policy Agent (OPA) or Caverno to automate the validation of security configurations and avoid misconfigurations. 9
10. Implement Disaster Recovery
Applying all of these best security practices is no 100% guarantee that an attack won’t occur. Ensure that there is a disaster recovery plan in place and tools are in use for automating cluster recovery.
AWS implements backup and recovery approaches for on-premises, cloud-native, and hybrid architectures. These approaches offer lower costs, higher scalability, and more durability to meet recovery time objective (RTO), recovery point objective (RPO), and compliance requirements.
10
By following these best practices and leveraging AWS solutions for Kubernetes security, you can protect your Kubernetes clusters and applications from potential vulnerabilities and build a robust infrastructure.
Top comments (0)