DEV Community: Mathias D

Five reasons for writing a custom CDK Construct Library

Mathias D — Mon, 22 Mar 2021 20:54:51 +0000

When I started using AWS Lambda, my team used the serverless framework to describe our AWS resources. This is easing things a lot compared to writing only plain CloudFormation templates. But still, there was a lot of CloudFormation to be written. This used to be nobody's favourite task. Then CDK came to the rescue.

The general advantages of CDK

Writing CloudFormation sucks. It is highly repetitive, lengthy, and slow. Code completion is really limited and I just cannot manage to remember the syntax of those intrinsic functions.

So nobody needed to convince me when CDK came along. Finally, there is code completion, extraction of repetitive parts, and there are constructs. CDK constructs are infrastructure building blocks for your application - they can represent a single CloudFormation resource or represent a higher-level component consisting of multiple resources. They bundle things that usually go together - like a Lambda Function with LogGroup and execution role. These constructs reduce the amount of code needed to describe a stack. So CDK does not mean describing CloudFormation resources in a programming language. It also provides an abstraction above CloudFormation resources.

Also, CDK allows for easy connection between constructs. There is a method to grant read access to a DynamoDB table for a Lambda function. This is so much more convenient than CloudFormation.

Yes, there are downsides, too. CloudFormation usually has no surprises when it comes to structure. There is not much freedom there, so CloudFormation templates are usually lengthy but still rather easy to read. The freedom of a higher-level programming language leaves much more room to structure things in different ways. So it usually takes a bit until I understand a CDK app that I did not write myself.

But after all the benefits outweigh the downsides by far.

By using your custom CDK construct library you can get even more advantages out of CDK. When I say custom CDK construct library I mean writing your own constructs that you package into a library and make them available in your organization.

So let's look into what is in for you.

Suddenly conventions are your friend 🧐

When you work in an organisation larger than just a few developers you will have conventions around AWS resources. Something like "Enable encryption at rest for your SQS queue!" or "Point-in-time recovery should be enabled for every DynamoDB table!". Usually, such conventions are documented in a guideline somewhere. You rely on the discipline of your developers that those conventions are actually applied. Maybe you even write some code that checks your resources regarding those conventions and raises alarms if the rules are violated. This is a little better. Now you can be sure that those conventions are applied - but only after they already have been deployed.

I find it a little smarter if developers are incentivised to consider those conventions by making their lives easier. With CDK you can have your custom construct wrapping the DynamoDB Table construct which defaults some settings according to your conventions. Developers will like to use this because they have to worry about a few things less.

Such convention constructs are a good start, since they are easy to write.

Declutter where possible 🧹

Often there are standard ways of doing things in an organisation. Here are a few examples.

Logs for Lambda functions have a retention of 14 days and are always shipped to ElasticSearch for central log aggregation
Every REST API is using a subdomain of our corporate domain (requiring DNS Records and a certificate)
For every Lambda function, we want an alarm when errors are logged

Such standard cases result in a lot of code that is copied over and over again. Those are perfect candidates for custom constructs.

So our custom Lambda function construct can create the Lambda function and make sure that the LogGroup is configured appropriately. We can also offer an aspect that can be added to a stack to add MetricFilters and Alarms for error log entries in every Lambda LogGroup.

In the end, we have saved a few lines of code in many projects, developers have to worry about fewer things, and we are making sure that best practices are applied.

Constants - Everything in one place 🧰

Your CDK construct library can also be the go-to place for important constants. Usually, constants around the AWS accounts and shared resources are scattered around different pieces of documentation. So developers need to go to various places to look them up. Often such values are then hard-coded in the CloudFormation or the CDK configuration.

We can add those constants to our custom CDK construct library. So they are easy to find and access. Also, they do not have to be copied into application code. Those constants can be ids of AWS accounts, the standard destinations for alarms, or the webhooks of important slack channels that you want to post to from your AWS infrastructure.

With these constants, information moves closer to the place of use and thus less time is spent searching for this information in the documentation.

Foster cross-team collaboration 🙌

A CDK construct library can be something a developer community can share and care about together. This can foster collaboration across teams and can start important conversations about lessons learned and best practices.

Deliver polyglot libraries from a single codebase

You are working in a polyglot environment - also this is not an obstacle. It is possible to write your CDK construct library once in TypeScript and use jsii to deliver polyglot libraries from it (Go, Java, Python). Jsii is also used by the CDK project itself to deliver the different language bindings it offers.

I hope you cannot wait now to create your own customer CDK construct library. Just head over to your favorite terminal and enter.

cdk init lib - language=typescript

You can also use projen to jump into developing your custom construct library.

Furthermore the CDK Workshop is a good start for starting to write your first own construct. It contains an example for writing a construct.

How to enforce type-safety at the boundary of your code

Mathias D — Tue, 16 Feb 2021 21:55:02 +0000

I recently started digging more into TypeScript. The structural typing approach in TypeScript is interesting and often a little bit surprising for developers coming from languages with a reified, nominal type system like Kotlin and Java.

I realised that it is very important to understand that types in TypeScript are completely erased at runtime.

Let's look at the following code. We are receiving JSON input and want to parse this into an object.

type User = {
  email: string;
  phone: string | null;
  age: number | null;
};

const json = '{"id": "some"}';

const maybeNoUser = JSON.parse(json) as User;

console.log(maybeNoUser.email);

// 👇💥 TypeError: Cannot read property 'toLocaleLowerCase' of undefined
console.log(maybeNoUser.email.toLocaleLowerCase());

In Kotlin similar code would already fail while parsing the Json string into an object of type User. But here the code happily executes and only fails with a TypeError at the second log statement.

The as type assertion is basically telling the compiler to should shut up and that you know what you are doing. There is no checking performed - it has no runtime impact at all. It is not in the slightest similar to a type cast in e.g. Java. And because types are erased at runtime the type system cannot help us out here.

What we did above looks OK at first sight and also the TypeScript compiler is satisfied with it. Not even ESLint complains. But this can still be really bad in a real-world code base.
We are trusting that the Json is representing a User. If the input does not match our expectations we might get arbitrary problems in a totally different part of our code. Such errors will be tough to understand.

So what should we do here? Exactly, let's get our hands dirty and write some good old validation code to make sure the user object is satisfying our expectations.

type User = {
  email: string;
  phone: string | null;
  age: number | null;
};

const input = '{"email": "some@test.com", "age":"some"}';

const parseUser = (json: string): User => {
  const user = JSON.parse(json) as User;
  if (!user.email) {
    throw new Error('missing email');
  }
  if (user.age && typeof user.age !== 'number') {
    throw new Error('age must be a number');
  }
  return user;
};

// 👇💥 Error: age must be a number
const user = parseUser(json);

All right - this is much safer. But honestly - the code in parseUser almost hurts. It is repetitive code that nobody likes to write. It is error prone and it is cumbersome to check every possible case. Even in our trivial case a complete implementation would need a lot more code than given in the example above. Also, everything we are checking in parseUser is already expressed in our User type. The validation logic is duplicating this. There has to be a better way.

Fortunately, there is zod for the win.

Zod is a TypeScript-first schema declaration and validation library.

Zod lets you declare schemas describing your data structures. These schemas can then be used to parse unstructured data into data that conforms to the schema. Sticking to our example above this could look like this:

import * as z from 'zod';

const userSchema = z
  .object({
    email: z.string(),
    phone: z.string().optional(),
    age: z.number().optional(),
  })
  .nonstrict();

type User = z.infer<typeof userSchema>;

const input = '{"email": "some@test.com", "age":"some"}';

/* 👇💥 
[
  {
    code: 'invalid_type',
    expected: 'number',
    received: 'string',
    path: ['age'],
    message: 'Expected number, received string',
  },
]; */

const user = userSchema.parse(JSON.parse(input));

I really like the DSL for schema declaration. It is hardly more complex than defining a type in Typescript. And we can even use it to infer a type from it that we can use in our function signatures. This way the usage of zod does not leak into our whole code base. The nonstrict() option generates a schema that allows additional properties not defined in the schema. This is definitely a best-practice when parsing Json data.

Zod also takes advantage of the structural typing characteristics of TypeScript. So you can derive similar types from a single schema. This can help e.g. when implementing a function to save a user. Such functions usually take an object, generate an id, save the object and return the object along with the id.

import * as z from 'zod';
import { v4 as uuid } from 'uuid';

const userEntitySchema = z
  .object({
    id: z.string().uuid(),
    email: z.string(),
    phone: z.string().optional(),
    age: z.number().optional(),
  })
  .nonstrict();
const userSchema = userEntitySchema.omit({ id: true });

type UserEntity = z.infer<typeof userEntitySchema>;
type User = z.infer<typeof userSchema>;

const input = '{"email": "some@test.com", "age":30}';

const saveUser = (user: User): UserEntity => ({
  id: uuid(),
  ...user,
});

const user = userSchema.parse(JSON.parse(input));
const userEntity = saveUser(user);

console.log(userEntity);

Using omit we could just create a new schema out of the existing one. Also pick exists to add to an existing schema. And again - we did not have to duplicate any knowledge about our types.

I think this is really a neat tool that I recommend to use whenever potentially type-unsafe data is entering our code. Be it the Json input coming in via a REST invocation, or the result of a DynamoDB query. Zod has much more to offer then what I described here. So I can only encourage you to check out the excellent documentation.

Further reading:

How to get more out of Pull Request Reviews

Mathias D — Wed, 22 Jan 2020 13:53:59 +0000

Pull request reviews are an essential tool to improve the quality of software. In a team working with or towards continuous deployment it is the last human quality gate on the way to production. And not only that, when done right, pull request reviews are also a great tool for knowledge sharing in a team. So we better get this right. Unfortunately, this is easier said than done.
In this post we will look at some learnings about pull request reviews I collected in my daily work.

You are going too fast

The authors of "Accelerate" propose to measure the deployment frequency of a team as one of the key metrics for a team's performance. At the same time they suggest "change failure rate" as another metric to balance out the first one. Going too fast will do more harm than good. This also applies to pull request reviews. A decent review takes time. Rushing reviews might increase throughput now but it will most likely introduce quality problems and slow you down in future. Also chances for knowledge sharing and sharing responsibility in a team will be missed. So a detailed review is most likely well spent time.

Trust, but verify

First of all, trust between members is a really good thing. It is one of the pillars of successful teams. But as often in life, you can go too far. I have seen reviews for pull requests with boring and repetitive changes over dozens of files being approved in 2 minutes. What can possibly be wrong in such a simple change? The change was also boring for the author. So chances are that one or the other thing slipped through, better verify.
I have observed that a lot of trust in the skills of your co-workers can lead to increasingly sloppy pull request reviews. Let's work against this reflex! You have smart colleagues - this is great - but everybody makes mistakes once in a while.

Start showing love to your test code

Most reviewers focus on the production code. This is understandable because this is where the bug could be hidden that might trigger that alert in the middle of the night. But still, we produce at least as much test code as production code. I have heard of projects containing three times more test code than production code. And also test code can become an unmaintainable mess. So we better show some love to our test code as well. A co-worker of mine has the habit to start having a look at the tests when doing pull request reviews. I think this is a great idea that is worth giving it a try.

Search for the missing bits

This one is really hard to get right. When reviewing a pull request you can easily see what is included. But in a pull request review it is also very important to find things that are missing. According to the excellent post by Scott Nonnenberg about the Top ten pull request review mistakes we should focus on the things that are not in the pull request but should be. Did the author forget to change a piece of code, are tests missing, or are edge cases not considered? So looking at a pull request is not enough. We also need to understand the code around it. We need to look at the broader picture, understand the change and the motivation behind it completely and actively ask ourselves the question: What is missing?
I tend to look at bigger pull requests in the IDE to get the complete picture. This way I can not just look at the changes but also at the code that is around these changes.

Attack the flaw, not the person

Pull request reviews are a form of feedback and we should be careful about how to get our message across. Especially new members of the team might take pull request feedback personally and feel bad about getting things wrong. This must not happen. It is about shipping code that is in good shape, it is about learning and growing together, about sharing responsibility and not about blaming or showing superiority. So here are a few tips:

find an encouraging tone (full sentences do not hurt)
offer help, be respectful
do not just look for the flaws, also praise the parts that you like

I can recommend the talk Amazing Code Reviews: Creating a Superhero Collective from Alejandro Lujan for more information about this aspect of code reviews.

Automate the nitpicking

Many of us appreciate a consistently formatted code base. But we should not waste our time checking indentation or ordering of imports in a pull request review. Let a linting tool do this boring work and focus on the harder parts.

Unblock

It is always a tough decision if we should focus on the current piece of work we are trying to get done or if it would be advisable to switch contexts and review the pull request a co-worker just created. In general we should remind ourselves that a pending pull request is probably blocking a co-worker from getting work done. Also postponing a pull request for a longer time will result in additional effort in resolving conflicts. So a timely review is very valuable. But on the other side the flood of pull request that a bigger team produces can be overwhelming and it can also be OK to finish up the current piece of work before diving into the pull request reviews.

There are many aspects to pull request reviews and the ones mentioned above are just a few of those. It becomes apparent that a good pull request review culture is hard to achieve. I hope the thoughts above can be a trigger to revisit this important tool and get more out of your pull request reviews. It is worth it.