DEV Community: Mudathir Lawal

Hardening an AWS Kubernetes Cluster: Best Practices for Enhancing Security

Mudathir Lawal — Tue, 01 Apr 2025 12:23:31 +0000

Introduction

Kubernetes (K8s) is the de facto standard for container orchestration, enabling developers to manage complex, microservices-based applications with ease. When running Kubernetes on Amazon Web Services (AWS), organizations benefit from scalability, flexibility, and the vast ecosystem of AWS services. However, the security of a Kubernetes cluster is paramount, especially as more sensitive workloads and critical data are migrated to the cloud. As such, hardening a Kubernetes cluster on AWS involves addressing various aspects, from infrastructure security to securing the application workloads running within the cluster.

This article explores best practices for securing an AWS-hosted Kubernetes cluster, covering essential considerations ranging from network security and access control to runtime protections and monitoring.

1. Securing the AWS Infrastructure

The first layer of security in a Kubernetes cluster is the underlying AWS infrastructure. Hardening the cloud environment helps minimize the risk of unauthorized access and data breaches. Below are key steps for ensuring a secure AWS environment:

a. VPC and Network Security

Kubernetes clusters rely on networking for communication between nodes and services. Isolating Kubernetes traffic and preventing unauthorized access requires careful design of the Virtual Private Cloud (VPC):

VPC Segmentation: Divide your AWS environment into multiple subnets (public, private, and isolated). Kubernetes nodes should be placed within private subnets, and public access should be restricted.
Security Groups: Use security groups to define fine-grained access rules. Only allow necessary inbound and outbound traffic to Kubernetes worker nodes, control plane, and other services.
Network ACLs: Apply additional layers of security with Network ACLs (Access Control Lists) to filter traffic between subnets and control node-to-node communications.

b. IAM Roles and Permissions

AWS Identity and Access Management (IAM) is critical for ensuring that only authorized users and services can interact with the Kubernetes cluster. Follow these principles to minimize IAM-related risks:

Principle of Least Privilege: Grant the minimum permissions required for a service or user to function. Over-permissioning increases the attack surface.
Service Account Integration: Kubernetes supports integration with IAM roles through Service Account to Role (IRSA). Use IRSA to map Kubernetes service accounts to IAM roles, ensuring that only necessary AWS permissions are granted to your workloads.

2. Hardening the Kubernetes Control Plane

The Kubernetes control plane is the brain of your cluster, and its security is paramount. Securing access to the control plane and managing its components are critical steps to ensure the safety of your cluster.

a. Secure API Server Access

The Kubernetes API server is the entry point for interacting with the cluster. Securing access to the API server is one of the most important tasks:

API Server Authentication and Authorization: Use strong authentication mechanisms, such as AWS IAM or OIDC (OpenID Connect), to control access to the API server. Authorization can be handled through Kubernetes RBAC (Role-Based Access Control), which ensures that users and services only have the permissions required for their tasks.
API Server Endpoint Security: Disable public access to the Kubernetes API server. Use Amazon's private API endpoint feature to limit API server traffic to your VPC.
Audit Logging: Enable audit logging to track all interactions with the API server. This provides an essential trail for detecting suspicious activity and understanding access patterns.

b. Kubernetes Control Plane Hardening

To protect the Kubernetes control plane, ensure the following:

Etcd Security: Etcd is the key-value store used by Kubernetes to store cluster data. Secure etcd by enabling encryption at rest, using TLS for client connections, and limiting access to etcd nodes.
RBAC and Network Policies: Enforce strict RBAC policies for the control plane. Network policies should be used to restrict communication to control plane components from unauthorized sources.
Kubelet Security: The Kubelet manages individual worker nodes and should be configured to enforce proper authorization and authentication. Disable unauthenticated access to the Kubelet and secure Kubelet-to-API communication with mutual TLS.

3. Securing Worker Nodes

Worker nodes host your workloads, and securing them is vital for preventing attacks from reaching your applications.

a. Node-level Security

Worker nodes should be hardened to prevent unauthorized access:

Operating System Hardening: Start by following best practices for securing the operating system (e.g., Amazon Linux 2 or Ubuntu). Disable unnecessary services, install security patches regularly, and use AWS security tools such as AWS Inspector and GuardDuty.
Container Runtime Security: Choose a secure container runtime like containerd or Docker. Limit the privileges granted to containers by configuring security contexts and avoiding running containers as root.

b. Node and Pod Isolation

Node-level isolation and segmentation can reduce the impact of a potential attack:

Pod Security Policies (PSP): Although PSP is deprecated, it’s a useful mechanism to control the security settings of pods. Use alternatives like OPA Gatekeeper or Kyverno to enforce security policies that prevent privileged access, host networking, and other dangerous behaviors.
Linux Security Modules (LSMs): Leverage tools like SELinux or AppArmor to enforce mandatory access control on Linux nodes. These tools add an additional layer of protection against containerized exploits.

4. Application Security and Network Policies

While securing the infrastructure and Kubernetes components is important, application security is just as critical. Adopting security best practices for your workloads can prevent vulnerabilities from being exploited.

a. Container Image Security

Use Trusted Images: Only use official, well-maintained, and trusted images for your containers. Where possible, build your own images from secure base images.
Image Scanning: Implement container image scanning to detect known vulnerabilities. Tools like Amazon ECR’s image scanning or third-party scanners like Trivy or Clair can help identify security flaws before deployment.

b. Pod and Network Policies

Pod Security Policies: Use alternatives to Pod Security Policies (PSP) to define a set of rules for container behavior, such as prohibiting privileged containers, enforcing the use of non-root users, and disallowing unsafe host volumes.
Network Policies: Network segmentation within Kubernetes is essential for controlling traffic between pods. Implement network policies to restrict communication between services unless explicitly allowed, reducing the attack surface of your applications.

5. Runtime Security and Monitoring

Once the Kubernetes cluster is deployed, maintaining a strong security posture is crucial throughout its lifecycle.

a. Runtime Security Monitoring

Runtime Protection: Use tools like Falco or AWS Threat Detection to monitor for unusual behavior during container runtime. These tools help detect malicious activity such as privilege escalation, file tampering, and other abnormal behaviors.
Logging and Monitoring: Enable centralized logging for your cluster using AWS CloudWatch, Prometheus, or an ELK stack (Elasticsearch, Logstash, Kibana). Monitor metrics, logs, and traces to detect and investigate anomalies quickly.

b. Vulnerability Management and Patching

Regular patching is critical to maintaining security:

Patch Kubernetes and Worker Nodes: Apply updates to both the Kubernetes control plane and worker nodes to ensure known vulnerabilities are mitigated.
Vulnerability Scanning: Continuously scan your running workloads and images for vulnerabilities. Tools like Aqua Security or Sysdig Secure can help automate this process.

Conclusion

Hardening a Kubernetes cluster on AWS requires a multi-faceted approach, addressing security from the underlying AWS infrastructure to the application layer. Implementing security best practices across all levels of your cluster—networking, control plane, worker nodes, applications, and runtime—ensures that you can protect sensitive data, prevent unauthorized access, and detect malicious activity quickly.

By following these best practices, organizations can create a robust, secure Kubernetes environment that leverages the full capabilities of AWS while minimizing the risks associated with running containerized applications in the cloud. Security is an ongoing process, and continuous vigilance and improvement are key to safeguarding your Kubernetes infrastructure in the ever-evolving threat landscape.

Unlocking the Power of AWS: A Guide to Amazon S3 (Simple Storage Service)

Mudathir Lawal — Tue, 01 Apr 2025 10:54:11 +0000

Amazon Web Services (AWS) offers a wide range of cloud computing services, but one that stands out for its versatility and popularity is Amazon S3 (Simple Storage Service). Whether you're a beginner or an experienced cloud architect, understanding how to leverage Amazon S3 is crucial for efficient data storage and management.

In this article, we’ll walk through the core concepts of Amazon S3, its features, and best practices to help you get the most out of this powerful service.

What is Amazon S3?

Amazon S3 is an object storage service that provides scalable, durable, and low-latency storage. It’s designed to store and retrieve any amount of data at any time, from anywhere on the web. Whether you are storing images, videos, backups, logs, or large datasets, S3 allows you to store files in "buckets" and access them via a web interface or through APIs.

Amazon S3 is widely used because of its flexibility, security features, and high availability. With S3, you can store large amounts of unstructured data in a highly reliable and cost-effective manner.

Core Features of Amazon S3

Scalability

Amazon S3 automatically scales to accommodate your storage needs, whether you're a small startup or a large enterprise. You don’t need to worry about provisioning hardware or manually scaling your infrastructure.
Durability and Availability

S3 guarantees 99.999999999% (11 9’s) durability over a given year. Data is automatically replicated across multiple data centers (availability zones), ensuring redundancy and preventing data loss in case of hardware failure.
Data Security

S3 provides multiple layers of security, including:
- Encryption: You can encrypt data both in transit and at rest using AWS-managed or customer-managed keys.
- Access Control: S3 allows you to control access to your data using bucket policies, IAM roles, and ACLs (Access Control Lists).
Versioning

S3 supports versioning, which allows you to keep multiple versions of an object. This is particularly useful for tracking changes over time, recovering from accidental deletions, or maintaining backup copies of your files.
Lifecycle Policies

You can set lifecycle policies to automatically transition objects between different storage classes or delete them after a certain period. This helps in managing data retention and cost optimization.
Storage Classes

Amazon S3 offers different storage classes to optimize costs:
- Standard: Best for frequently accessed data.
- Intelligent-Tiering: Automatically moves data between two access tiers when access patterns change.
- Standard-IA (Infrequent Access): Lower cost for data that’s accessed less frequently but needs rapid access.
- Glacier: Low-cost storage for data that is rarely accessed and can tolerate retrieval times of several hours.
- Glacier Deep Archive: Lowest cost storage for long-term data archival.

Key Concepts of Amazon S3

Buckets: A bucket is a container for storing objects in S3. You create a bucket to upload data, and each object is stored in a unique location within that bucket.
Objects: Objects are the fundamental entities stored in S3. They consist of the data itself, metadata, and a unique identifier (the object key).
Object Keys: Each object in a bucket has a unique key that can be used to retrieve it. The key is often structured in a hierarchical way (using prefixes to mimic folders), but S3 does not have a true folder structure.

Getting Started with Amazon S3

Here are the basic steps to get started with Amazon S3:

1. Create a Bucket

Sign in to the AWS Management Console.
Navigate to S3 and click on Create Bucket.
Give your bucket a unique name (the name must be globally unique across all of S3).
Choose the region closest to your users to reduce latency and increase performance.

2. Upload Data

After creating the bucket, you can upload files by:

Clicking on the Upload button in the S3 console.
Dragging and dropping files from your local machine.
Using the AWS CLI (Command Line Interface) or SDKs to automate the process programmatically.

3. Set Permissions

You can control access to your bucket and its contents. This is done using:

Bucket policies: Define rules that apply to all objects within a bucket.
Access Control Lists (ACLs): Set permissions for individual objects.
IAM Roles and Policies: Attach permissions to AWS Identity and Access Management (IAM) roles.

4. Manage Data Lifecycle

Set up lifecycle policies to automate data management:

Archive objects that aren’t frequently accessed to Glacier.
Delete objects older than a certain number of days.

5. Monitor and Audit

S3 integrates with AWS CloudTrail for logging and auditing API calls. You can also use Amazon CloudWatch to set up metrics and alarms related to your S3 usage, such as monitoring storage usage and retrieval rates.

Best Practices for Using Amazon S3

Naming Conventions

Use consistent and descriptive naming conventions for your buckets and object keys. This makes it easier to organize and retrieve data as your storage grows.
Data Encryption

Always enable encryption for your sensitive data. Use S3-managed keys (SSE-S3) for simplicity, or manage your own keys with SSE-KMS for additional security controls.
Optimize Costs with Lifecycle Policies

Implement lifecycle policies to automatically transition objects to lower-cost storage classes, such as Glacier or Glacier Deep Archive, to optimize storage costs for data you don’t need immediate access to.
Regularly Review Access Permissions

Review your access control policies regularly to ensure that only the necessary users and services have access to your S3 resources. Make use of IAM roles and policies for fine-grained control.
Version Control

Enable versioning on your buckets to protect against accidental deletions or overwrites. You can retrieve older versions of an object even after it’s been modified or deleted.

Real-World Use Cases of Amazon S3

Backup and Restore

S3 is widely used for backing up databases, files, and entire systems. The durability and security of S3 make it an ideal solution for protecting critical data.
Data Archival

For long-term data storage, S3 Glacier and Glacier Deep Archive provide a cost-effective solution for archiving infrequently accessed data.
Web Hosting

Many websites and web applications store static assets (images, videos, and other media) on S3. S3 integrates well with Amazon CloudFront (a CDN) to deliver content globally with low latency.
Big Data Storage

S3 is used as a storage layer for big data analytics platforms. Its scalability and ability to integrate with other AWS services like Amazon Athena and Amazon EMR make it ideal for processing large datasets.

Conclusion

Amazon S3 is a fundamental service within AWS, providing reliable, scalable, and cost-effective storage solutions for businesses of all sizes. By understanding its features and capabilities, you can optimize how you store, access, and manage data, ensuring both high performance and low costs. Whether you're handling backups, large-scale data analytics, or web content delivery, mastering S3 is an essential step toward becoming proficient in AWS.

Editing an IAM Service Role, and Attaching Service Roles to AWS Resources

Mudathir Lawal — Sun, 31 Mar 2024 13:00:16 +0000

Overview

One common challenge you might have come across is how to edit service roles for AWS resources. This is usually necessary when you forget to attach an appropriate role to the service in question. After executing a task, you tend to get errors such as:

Insufficient permission; or the provided role does not have sufficient permissions.

Here we will describe how this can be solved by creating a new service role, and modifying it to suite our purpose. We will also show how the new role can be attached to an existing resource.

What is a Service Role?

A service-linked role is a unique type of IAM role that is linked directly to an AWS service. Service-linked roles are predefined by the service and include all the permissions that the service requires to call other AWS services on your behalf.

Scenario

We require an AWS CodeDeploy service role for EC2 in order to be able to deploy an application to an EC2 instance. We, therefore, need to create one and attach it to our EC2 instance.

Procedure

In the AWS console, go to IAM, then Roles, then Create role. Under Trusted entity type select AWS service; and under Use case, select EC2.

Under Add permissions, search for the appropriate permission. In our case, we will use the AWSCodeDeployRole role. Select Next and give your new role a meaningful name. Then click Create role.

Note: Do not bother to edit the role using the Edit button, because you will not be able to. Just go ahead and create; the editing will be done after the creation.

Go back to Roles or click on View role to view your newly created service role. Select the Trust relationships tab. Then click the Edit trust policy button, and make the necessary modifications to the policy settings.

In our case, we change the ec2 on line 7 to codedeploy. You can now return to your AWS resource and attach the newly created role to it.

In our own scenario (demonstrated in the clips above), we created an AWS service role for EC2 instances. Note that we have attached the new role by first selecting he instance, then clicking on Actions -> Security -> Modify IAM role (First picture after the last paragraph). When attached to an instance, this service role will allow EC2 instances to call AWSCodeDeploy on our behalf. These types of roles are important for automating the deployment of workloads into the AWS cloud.

Thank you for reading.

Reference: AWS official documentation on service-linked roles., accessed 2024/03/31

Running a Secure Web Server on AWS EC2

Mudathir Lawal — Sat, 30 Mar 2024 09:59:07 +0000

Overview

This article seeks to describe how to run a sophisticated web server on AWS using Infrastructure-as-a-Service (IaaS) architecture.

The Architecture

Set up a Virtual Private Cloud (VPC). This cloud local network will contain two public subnets and two private subnets. Create an AWS Elastic Compute Cloud (EC2) instance running Ubuntu 20.04 LTS, and configure the security groups to allow your local machine to connect to it via SSH. Download and save the key pair. This machine will be used as a Bastion Host from which you can securely connect to the web server. Then launch another EC2 instance running Ubuntu 20.04 LTS in one of the private subnets. Download and save the key pair. This instance will host the web server. Launching the instance in a private subnet will give the web server some level of security, since it will only be reachable to only hosts/devices within the same Virtual Private Cloud (VPC). Be sure to configure the security group of this instance to allow SSH traffic from the Bastion Host.

Prerequisite Software

Copy the command to connect to your bastion host from the AWS console and run the command in your local Linux shell.

Once connected to the Bastion EC2, run the following command to copy the downloaded server key pair file to the bastion host. This will allow you to connect to the web server host.

scp -i "~/path-to-key/keypair.pem" /part-to-key/keypair.pem  ubuntu@<dns-of-ec2>:~/.

Then copy the command to connect to your bastion host from the AWS console and run the command:

Once you gain access to the web server instance, run the following commands to install NGINX server on it:

sudo apt update
sudo apt install nginx
sudo ufw allow 'Nginx HTTP'
systemctl status nginx

Output

Run the following command to see the server IP address:
curl -4 icanhazip.com

Verify that the sever is up and running by pasting the IP in your web browser.

Setting up Continous Integration (CI) in the AWS CloudShell

Mudathir Lawal — Fri, 31 Mar 2023 18:22:49 +0000

Setting up Continous Integration (CI) in the AWS CloudShell
Build directly from the AWS cloud shell is something I have enjoyed for a long time. And I feel that sharing my ideas about setting up a typical continous integration using GitHub will benefit other dvelopers.

The process…

Launch the AWS CloudShell and run the following command and answer the prompts that follow: ssh-keygen -t rsa Print out your public key by running the command: cat /home/cloudshell-user/.ssh/id_rsa.pub

Notice that the path to the file begins with "/home/…" and ends with "pub". Do not run the command with the ending period printed after "pub".

Now go to your GitHub profile and select "Settings" the "SSH and GPG keys". Under SSH keys on the right pane click on "New SSH key." Paste in your public key and enter a title in the top field. Then press add SSh key. Your SSH key should now appear as shown below:

This SSH key allows you to make commits to your GitHub repo without needing to sign in at every commit.

You can now create a repo in your GitHub account where you will continously integrate your code directly form the AWS CloudShell.
Back in the CloudShell, clone the repository using the ssh command provided on GitHub.

To edit a file run vim filename. Make your desired changes to the files in the repo using Vim, commit the changes and push. But, you will need to follow the prompts to enter your name and email address when running these commands for the first time.

If you check your GitHub account you would find the changes already integrated there.

A step by step guide to processing PDF files using Amazon Comprehend for IDP

Mudathir Lawal — Fri, 31 Mar 2023 11:20:10 +0000

Intro

One of the new service features that was announced at the AWS re:Invent 2022 is the intelligent document processing (IDP) capability of Amazon Comprehend which allows it to process semi-structured files such as PDFs. This article seeks to provide a step-by-step demonstration of the process. The use case we adopt is that of automating legal contracts which involves extracting key phrases from a pdf document and use that as a guide to prepare a favourable negotiation.

Demo

Log on to the AWS console and create an s3 bucket to hold the documents you want to process. We recommend that you create two separate folders within the s3 bucket,one of which is to store the input documents awaiting processing, while the other will be used to store the output of the API call to Amazon Comprehend. Note the region in which the s3 bucket is located.
Access the Amazon Comprehend service at https://console.aws.amazon.com/comprehend/ and select the region where you situated your s3 bucket. This is important as the two services would not communicate if not located in the same region.
After clicking on "Launch Amazon Comprehend," choose "Analysis jobs," on the left pane, then select "Create job."

Under "Analysis types" click "Key phrases."

Enter the paths to the input and output folders already created in your s3 bucket in the appropriate fields.

Under "Access permissions," choose "Create an IAM role," then add a suitable name suffix.

Click "Create job."
The completed job should look like this:

To download the output file, navigate to the output folder in your s3 bucket. You will need to extract the file and resave it in json format.

Winding up

I hope this has been a useful piece. Watch out for more interesting contents AWS and DevOps coming your way soon. Happy clouding!