How To Build Infrastructure as Code With Pulumi And Golang - Part 2

Going into the series on creating Infrastructure as Code on AWS using Pulumi, I knew the team there was actively working on improving and expanding the Go support in Pulumi. What I didn’t realize is that it would be so quick and would be such a great improvement to the underlying code I needed to write. In this post, I’ll go over some of the code from my previous blog posts and update them to match the new SDK.

The complete project is available on GitHub.

The core programming model in Pulumi has tons of really awesome features, but not all of them are available in the Go SDK yet. The work the team at Pulumi is currently doing, includes some significant changes to the SDK, including more strongly typed structs (no more interface{}, yay! 🥳), better support for Input and Output types, and making sure the Go has the same features from the core programming model as the other languages do. With so many changes going in, it’s almost impossible to not make breaking changes. Pulumi suggests that Go devs pin the versions of the SDK they’re currently using.

TL;DR I had to make a bunch of changes to make sure all the code worked again (like adding pulumi.String() wrappers around strings) but I could also replace all my own structs with structs from the Pulumi SDK. There are a few rough edges, but then again it is a work in progress and I’m excited to see what’s next.

Configuration

One of the new features that were added, or perhaps I just didn’t realize it existed before, is the ability to read and parse configuration from the YAML file. Using that configuration you can parameterize your stack based on config from a file, rather than hard coding it.

In the previous version of the code, I wrote my own function to get the configuration variables from the context:

// getEnv searches for the requested key in the pulumi context and provides either the value of the key or the fallback.
func getEnv(ctx *pulumi.Context, key string, fallback string) string {
    if value, ok := ctx.GetConfig(key); ok {
        return value
    }
    return fallback
}

That piece of the code, and a lot of related helper functions, can now be removed. In the snippet below, I’ve created a VPCConfig struct which, through the RequireObject() method, is populated with data from the Pulumi YAML file. The nice thing about this approach is that the RequireObject() method means that the object with that name needs to exist, otherwise Pulumi will throw an error and you can model your structs (and configuration data) to actually match the types they should be. Rather than having comma separated lists, I now have proper YAML arrays.

// VPCConfig is a strongly typed struct that can be populated with
// contents from the YAML file. These are the configuration items
// to create a VPC.
type VPCConfig struct {
    CIDRBlock string `json:"cidr-block"`
    Name string
    SubnetIPs []string `json:"subnet-ips"`
    SubnetZones []string `json:"subnet-zones"`
}

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        // Create a new config object with the data from the YAML file
        // The object has all the data that the namespace awsconfig has
        conf := config.New(ctx, "awsconfig")

        var vpcConfig VPCConfig
        conf.RequireObject("vpc", &vpcConfig)

        vpcArgs := &ec2.VpcArgs{
            CidrBlock: pulumi.String(vpcConfig.CIDRBlock),
            Tags: pulumi.Map(tagMap),
        }

        ... // snip
    })
}

These changes in the code also mean that the configuration file itself looks a lot cleaner. The snippet below shows what it was with the previous version of my code.

vpc:name: myPulumiVPC
vpc:cidr-block: "172.32.0.0/16"
vpc:subnet-zones: "us-east-1a,us-east-1c"
vpc:subnet-ips: "172.32.32.0/20,172.32.80.0/20"

Which is now replaced by better looking, and easier to read, YAML

awsconfig:vpc:
    cidr-block: 172.32.0.0/16
    name: myPulumiVPC
    subnet-ips:
        - 172.32.32.0/20
        - 172.32.80.0/20
    subnet-zones:
        - us-east-1a
        - us-east-1c

There are perhaps a few more lines, but the benefits of seeing which configuration items are actually arrays is super useful. The only thing that struck me as a little odd, is that you write the config in a YAML file and the struct tags use JSON.

Type Safety

One of the reasons I like Go is that it is a strongly typed language. That means that the type of variable is specified when you’re building your app. In the previous version of my code, I created my own structs to help add some strongly typed structs to create a DynamoDB table. The code below is a snippet of that code that shows the “old” way, where I had my own struct.

// DynamoAttribute represents an attribute for describing the key schema for the table and indexes.
type DynamoAttribute struct {
    Name string
    Type string
}

// DynamoAttributes is an array of DynamoAttribute
type DynamoAttributes []DynamoAttribute

// ToList takes a DynamoAttributes object and turns that into a slice of map[string]interface{} so it can be correctly passed to the Pulumi runtime
func (d DynamoAttributes) ToList() []map[string]interface{} {
    array := make([]map[string]interface{}, len(d))
    for idx, attr := range d {
        m := make(map[string]interface{})
        m["name"] = attr.Name
        m["type"] = attr.Type
        array[idx] = m
    }
    return array
}

// Create the attributes for ID and User
dynamoAttributes := DynamoAttributes{
    DynamoAttribute{
        Name: "ID",
        Type: "S",
    },
    DynamoAttribute{
        Name: "User",
        Type: "S",
    },
}

The updates to the Pulumi SDK introduced a ton of strongly typed structs that remove the need to have those helper structs and methods. In my case, for the DynamoDB table, there are now structs that support the Table Attributes and Global Secondary Indices. Removing the helper methods and replacing that with structs from the Pulumi SDK meant I went down from roughly 33 lines of code to 10 lines. I’m quite sure that feature alone will make sure I write better and cleaner code. The code snippet below shows the same attributes and does the same thing as the above snippet did.

// Create the attributes for ID and User
dynamoAttributes := []dynamodb.TableAttributeInput{
    dynamodb.TableAttributeArgs{
        Name: pulumi.String("ID"),
        Type: pulumi.String("S"),
    }, dynamodb.TableAttributeArgs{
        Name: pulumi.String("User"),
        Type: pulumi.String("S"),
    },
}

If you want to deploy your AWS Lambda functions through Pulumi, the same benefits I talked about are absolutely true for Lambda as well. More strongly typed structs with clear field definitions means that writing code is a lot easier and cleaner. Because you don’t have to rely on a bunch of helper methods, it also makes the code simpler to understand.

Going forward

Does all of this mean it’s all roses, though, the engineers at Pulumi still have some things to do. Credit to them, they do all of that out in the open. For me personally, I was very excited to see how far the team at Pulumi has come over the past few weeks. While there are a few rough edges, like documentation for the inputs and outputs of certain methods, I’m sure the team will update those in new releases, together with the help of the amazing community. I for one am excited to see how Pulumi takes Infrastructure as Code to a new level for Go developers.