Simplifying your Kubernetes infrastructure with cdk8s

Of late, I have started to pen down summaries of my talks for those not interested in sitting through a 45-min video or staring at slides without getting much context 😅

Here is one for AWS DevDay Hyderabad 2023 where I spoke about "Simplifying your Kubernetes infrastructure with cdk8s".

CDK for Kubernetes, or cdk8s is an open-source CNCF project that helps represent Kubernetes resources and application as code (not YAML!)

You can check out this link for the slides, and the recording is coming soon!

Working on Kubernetes?

Many of you work on Kubernetes, perhaps in different capacities - Architect, DevOps engineer, SRE, Platform engineer, Software engineer etc. No matter what we tell our friends, family or even recruiters (we need to convince them how hard it is to work across the entire CNCF landscape, isn't it? 😉 )

But the real question is....

What we actually do?

This is all light-hearted stuff, please don't take this personally 🙏

Can you relate to this 👆🏼 as well?

Working on Kubernetes does not have to mean turning into a YAML engineer (unless that becomes the "next big thing")

But the reality is that there is...

... YAML everywhere

It has become the "lingua franca" of the Kubernetes world. Every tool set that we use including Helm, kustomize, GitOps, Kubernetes operators etc. involves using (and debugging) YAML.

Avoiding YAML 😉

But what I covered in the talk was not about removing or getting rid of YAML. To be honest, that's almost impossible. It's more about circumventing it, or pushing it behind the scenes, using tools and frameworks. And one of those is - CDK for Kubernetes.

Helpful content

There is only so much I can cover in 30-40 mins. If you are interested in learning more, you are more than welcome to check out this. It’s kind of a mini-book along with a GitHub repo that has all the code.

I think you will find it useful!

From Infra-as-code to Infra-Is-code

You may have used Terraform or AWS CloudFormation. These fall into the Infrastructure as Code (IaC) category, where you write configuration to define your resources – be it YAML or JSON or a custom templating language.

On the other end of the spectrum, there are tools like AWS CDK or Pulumi. This is where you step into the Infrastructure Is code zone, where configuration takes a back seat and code dominates.

Hello cdk8s!

And that’s where cdk8s comes into the picture since it embraces the Infrastructure Is code paradigm for Kubernetes applications. I am a Go developer and that's what I used for the demos and code samples, but cdk8s supports Python, Java and Javascript.

And the best part is that you can use it anywhere. It's not specific to AWS or any other provider - It's an open source CNCF project.

Demo time!

I showed how to bootstrap a Go cdk8s application using the CLI (cdk8s init go-app), define a nginx application (see code below), convert it to YAML manifest (yes, we can't get rid of it!) and deploy using kubectl apply -f dist/.

cdk8s components

Understand these concepts, before we move on:

1. Every cdk8s application consists of a tree of components, also called constructs.
2. At the top of the tree is what you call an App. We create it using NewApp function.
3. An App can contain multiple Charts and we create a chart using NewChart function.
4. Within that Chart is where you would define your basic building blocks - in the above example we defined a Kubernetes Deployment and Service, but it could be other types of constructs as well (covered later)

Improve productivity with cdk8s+ and Custom Constructs

Since cdk8s is a low-level API, it can be a bit verbose, and that's where the cdk8s plus library comes in.

Now look at this example which is directly from the cdk8s documentation - the difference is clear. But it can vary depending on the language - the example here is actually Nodejs.

Here is a Go example:

More than the lines of code and verbosity, I think the more important point to consider is the level of abstraction.

cdk8s+ provides higher level abstraction:

• Which means in most cases you will have fewer lines of code,
• An ergonomic API with better defaults.
• And overall, less mental overload/overhead.

As developers, we all know how that can improve productivity!

Custom constructs

Imagine you are building a complex application on Kubernetes using cdk8s, and it is required by many teams in the company - perhaps each with a different configuration. Instead of each dev team writing their own version, you can externalize this in the form of Custom Construct and share it across the company.

It's nothing fancy to be honest - it’s the same as using a dependency or library, but this is still very powerful. The great thing is that there already something called a Constructs Hub that has constructs published by AWS, community as well as other providers!

Demo time, again!

In this demo:

• I walked through a "wordpress" deployment on Kubernetes to demonstrate the ease of use and intuitiveness of the cdk8s+ API
• Demonstrated custom constructs, in the context of the same "wordpress" deployment.

Here is what it would look like using a custom "wordpress" construct:

Integrating with Kubernetes Custom Resource Definitions (CRDs)

Then we moved into a slightly advanced topic. You can use cdk8s library to represent native Kubernetes objects like a deployment, service etc. But in any real-world implementation, it's highly likely that you are using custom resource definitions (CRD). So if you have existing CRDS, can you manage them as code using cdk8s???

A good example of an CRD and operator is AWS Controllers for Kubernetes (or ACK). With the ACK project, you can manage AWS services from Kubernetes.

This is just an example of an Amazon RDS database, but the same applies for an Amazon DynamoDB table, an Amazon S3 bucket or an Amazon Elasticache cluster or even an AWS Lambda function. The ACK project has different controller/operators for each service. When you create an ACK resource, these operators do what’s needed to create and manage the corresponding service(s) in AWS.

How do you use CRDs with cdk8s?

To use any CRD with cdk8s:

1. The first step is to run “cdk8s import” by pointing it to the CRD
2. This will result in APIs being auto-generated
3. Write your code using these APIs
4. ... and business as usual - use cdk8s synth to generate the manifest for your Kubernetes application

Yes, another demo!

It’s a simple URL shortener app which is deployed to Amazon EKS and uses DynamoDB as the database. The interesting part here is the fact that both the application and DynamoDB are defined using cdk8s

cdk8s and AWS CDK 🤝

So far we had one thing to think about. We bought in cdk8s and were able to represent both the AWS resources and the respective Kubernetes applications in form of code.

But what if we don’t want to use ACK. And stick to something like AWS CDK for AWS resources but use cdk8s for Kubernetes apps? cdk8s is great, and AWS CDK is awesome. So what’s stopping us from using them together?

Deploying apps to EKS with AWS CDK – the ”hard” way 😓

Here is a glimpse of using only AWS CDK to deploy an app to an Amazon EKS cluster (note that this is just AWS CDK code, not cdk8s)

• First, you declare the Kubernetes deployment,
• Then the Kubernetes service,
• And finally call the addManifest function declared on the EKS cluster object - this is equivalent to calling kubectl (but programmatically)

addCdk8sChart to the rescue!

The addCdk8sChart method acts like a bridge between CDK and cdk8s - check out the CDK reference docs for details.

This makes life much easier!

Recap - we're almost done!

Key takeaways:

• Its not about removing yaml - its about reducing it and embracing the Infrastructure Is code paradigm.
• Using cdk8s with regular programming languages and make use of the related tools and best practices (testing, OOP, etc.).
• Optimize with cdk8s plus, share and reuse common components as constructs, extend with CRDs
• Combine cdk8s with AWS CDK, or existing workflow/practices like GitOps, or even Helm charts.

Hope to see you next time!