Deploying an EKS cluster with attached storage

I'll admit, up front, that I'm new to Kubernetes, and probably always will be. There's just too much to learn! That being said, it is my job (well, I am Head of #DevRel for Otterize, and we're all about Kubernetes) so I have to know a certain amount.

What I was trying to do

I do a lot of demos. As part of that, I have to provision, a lot of Kubernetes clusters. A lot. Several times a day if I'm working on building or debugging a demo or tutorial.

Being able to create, and recreate, clusters on any of the major cloud providers (mainly AWS and Google Cloud, for now) is essential to me being able to create tutorials that always work, and demos that work under pressure (like on stage!).

The problem child

Doing this on GKE was relatively straight forward. When I'd create a cluster that needed persistent storage, it was just provisioned and added, and everything worked as expected.

Imagine my surprise when I created a cluster in EKS and deployed the demo applications and ... Kafaka just failed because there was no attached storage!

It turns out that EKS doesn't auto-provision storage. So I started trying to figure out how to attach storage.

Just because it's documented doesn't mean you can do it

After reading a ton of documentation (which, quite frankly made my head hurt and didn't get me any closer to a solution) I posted in the r/kubernetes subreddit hoping for help.

Pretty much all that did was point me back to the documentation, which wasn't helpful. Nothing I could find pointed to a solution that I could simply add to my EKS config.json to build the cluster.

I wanted something to add so that my command

eksctl create cluster -f eks-config.yaml --profile ME

would create the cluster with everything I needed.

The solution

Working with one of the great engineers here at Otterize we finally hit on a solution.

It was so amazingly simple that I am still a little dumbfounded that the AWS docs make it so hard.

In the addons section I had already added the VPC CNI

addons:
  - name: vpc-cni
    version: v1.14.0-eksbuild.3
    attachPolicyARNs: #optional
    - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
    configurationValues: |-
      enableNetworkPolicy: "true"

and a few others, like DNS

  - name: coredns
    version: 1.10.1-eksbuild.1
  - name: kube-proxy
    version: v1.27.1-eksbuild.1

And here's where the addition of the EBS storage came in, and was just stupid-easy:

- name: aws-ebs-csi-driver
    version: v1.22.0-eksbuild.2
    attachPolicyARNs:
    - arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy

And I could now create a cluster with a single command, from the command-line, that would run my demo app (with Kafaka) on EKS without any trouble.

Tell me about it

If you read this and say "hey dummy, that's not how any of this works!" please reach out and tell me why! I'm just trying to learn, so learning from my mistakes is a huge bonus.

If you read this and are like "that's effing genius!" please also reach out! I'd love to hear from you either way!

Finally

Please come and join us in the Otterize Community and learn more about what we do, and how we're trying to make securing your cluster dead-simple!