- The footprints of Linux are increasing day by day and the latest addition to this is the Bottlerocket. It is a Linux-based operating system built by Amazon Web Services. This open-source OS targets to host and run the containers on virtual machines or bare metal hosts.
- Today, Amazon Elastic Kubernetes Services (Amazon EKS) announces native support for Bottlerocket in managed node groups. Bottlerocket is a Linux-based open-source operating system that is purpose-built by Amazon. It focuses on security and maintainability, and provides a reliable, consistent, and safe platform for container-based workloads
- Every Linux-based OS involves the Linux kernel—which manages hardware resources—and a set of software packages that make up the rest of the operating system. That's why the bottlerocket-OS promises a light-weight to boost up and high security.
- In this post, we will launch a Bottlerocket managed node group with lauch template on EKS cluster. It's not only about setting the nodegroup to use bottlerocket AMI but about the arguments at EKS node startup. Let's find out more
- Why Should You Use Bottlerocket OS?
- Create a lauch-template for EKS nodegroup
- Create ASG with the launch template
Bottlerocket comes to the rescue when facing the above issues. The Bottlerocket OS tends to mitigate the challenges faced by container-based environments such as security, updates, compute cycles, start-up time, and the integrity of a cluster over time. Most of the components in Bottlerocket are written in Rust, so some of the memory safety issues are eliminated. The following are additional benefits of Bottlerocket:
- Improved uptime: You can apply updates to the Bottlerocket OS all at once, and they can also be rolled back as needed, improving uptime.
- Lower management overhead: You can utilize container orchestration services to automate updates to the Bottlerocket OS, reducing management overhead and operational costs.
- Better security and resource utilization: Contrary to other operating systems, you only have the essential components in Bottlerocket OS to run, creating a smaller attack surface and improving security.
- Optimized performance: Bottlerocket is optimized to run on Amazon EC2 and incorporates built-in support for integrations with AWS services.
- Read-only file system: Bottlerocket uses a read-only file system whose integrity is validated at the time of booting.
- Automated updates: You can automate updates via orchestration services like Amazon EKS. Unlike traditional Linux-based systems that use package-by-package updates, Bottlerocket utilizes image-based updates.
- Open development model: You can create code and design changes to the Bottlerocket OS via code available in Github. It should be noted that the Bottlerocket OS supports images formatted for Docker and OCI (Open Container Initiative).
- Pre-requisite: EKS cluster
- Here we will create an Auto scaling group with launch template contains following things
- Bottlerocket AMI, to get the latest AMI ID align with EKS cluster version and AWS region, use SSM parameters store
~ $ aws ssm get-parameter --region ap-northeast-1 --name "/aws/service/bottlerocket/aws-k8s-1.18/x86_64/latest/image_id" --query Parameter.Value --output text ami-0a04f060889bcef33
- Key-pair in order to SSH to the node for demonstration purpose
- EKS VPC
- Security group for Pod communications (Check out Understand Pods communication)
- IAM instance profile which has enough permission for node to join EKS cluster, read more AWS EKS - Launch Template Of Node Group
- User data (describe laters)
One of important thing is
user data which contains additional arguments to
kubelet service such as node lables, node taints.
- The user data must be in TOML format and must contains
settings.kubernetes.cluster-name(Read more Kubernetes settings).
user dataexample with
[settings.kubernetes] api-server = "https://abc.def.ap-northeast-1.eks.amazonaws.com" cluster-certificate = "TkQgQ0VSVElGSUNBVEUtLS0tLQo=" cluster-name = "eks-dev" cluster-dns-ip = "10.100.0.10" [settings.kubernetes.node-labels] "lifecycle" = "on-demand" "role" = "airflow" "type" = "af-stateful" [settings.kubernetes.node-taints] "dedicated" = "airflow:NoSchedule"
We can use
eksctlto generate basic
~ $ eksctl get cluster --region ap-northeast-1 --name eks-dev -o json | jq --raw-output '. | "[settings.kubernetes]\napi-server = \"" + .Endpoint + "\"\ncluster-certificate =\"" + .CertificateAuthority.Data + "\"\ncluster-name = \"eks-dev\""' > userdata.toml
- After the ASG scales the desired capacity, we can check the bottlerocket nodes
- List the nodes in the EKS cluster which have
Bottlerocket OSand check the Pods are assigned to them
# kubectl get nodes -o=custom-columns=NODE:.metadata.name,ARCH:.status.nodeInfo.architecture,OS-Image:.status.nodeInfo.osImage,OS:.status.nodeInfo.operatingSystem | grep Bottlerocket ip-172-10-11-131.ap-northeast-1.compute.internal amd64 Bottlerocket OS 1.4.0 (aws-k8s-1.18) linux ip-172-10-21-64.ap-northeast-1.compute.internal amd64 Bottlerocket OS 1.4.0 (aws-k8s-1.18) linux ip-172-10-31-142.ap-northeast-1.compute.internal amd64 Bottlerocket OS 1.4.0 (aws-k8s-1.18) linux # kubectl get pod -n kube-system -owide | grep ip-172-10-11-131.ap-northeast-1.compute.internal aws-node-t622b 1/1 Running 1 3h38m 22.214.171.124 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> aws-node-termination-handler-ql7t2 1/1 Running 0 3h38m 126.96.36.199 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> efs-csi-node-lbhcq 3/3 Running 0 3h37m 188.8.131.52 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> kube-proxy-j6l48 1/1 Running 0 3h38m 184.108.40.206 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> # kubectl get pod -owide -n airflow | grep ip-172-10-11-131.ap-northeast-1.compute.internal airflow-flower-59774778b4-fpfdm 2/2 Running 0 3h32m 220.127.116.11 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> airflow-scheduler-cdccf9d86-xc4hr 2/2 Running 0 3h32m 18.104.22.168 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> airflow-sync-users-7fd7c7db8-6whhr 2/2 Running 0 3h32m 22.214.171.124 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none> airflow-web-77d59cfcbb-bxsj5 2/2 Running 0 3h32m 126.96.36.199 ip-172-10-11-131.ap-northeast-1.compute.internal <none> <none>
- Start an SSM session. In order to SSH to the node, it needs to run
enable-admin-container(which is disabled by default) from SSM console
[ssm-user@ip-172-10-12-246 /]$ enable-admin-container Setting admin container to enabled 204 No Content Committing and applying changes 200 OK ["settings.host-containers.admin.enabled"] The admin container is now enabled - it should pull and start soon, and then you can SSH in
- SSH to the node and see the difference from other Linux OS
bash-4.2# systemctl status kubelet Running in chroot, ignoring request.
- Bottlerocket is built from the ground up with only the minimum components necessary to run containers installed on the host. Any additional software such EFS CSI driver, monitoring agents, metric collections, etc. must be run as Daemonsets
- In this post, it shows how to use Bottlerocket natively with Amazon EKS managed node groups and how to interact directly with the Bottlerocket cluster nodes. It's interesting that AWS CDK will support bottlerocket managed nodegroup soon (feat(aws-eks): support bottlerocket managed nodegroup)
- How to Get Started with Bottlerocket OS
- Amazon EKS adds native support for Bottlerocket in Managed Node Groups
- Provisioning and Securing Bottlerocket OS-Based Amazon EKS Clusters Using Nirmata Kubernetes Platform