How To Create a DynamoDB Table In AWS Using Pulumi And Golang

In previous posts, I looked at Pulumi to do all sorts of things with infrastructure. Most apps, though, will need some form of datastore so in this post I’ll go over the steps to create a DynamoDB table in AWS using Pulumi.

The complete project is available on GitHub.

Static types

One of the main advantages of programming languages like Golang and Java is that those languages have static types that make development less error prone. As developers are writing the code, they know what the input and output of methods will be. Unfortunately, the Go SDK for Pulumi doesn’t yet offer static types for AWS resources. The snippet of code below has two types (DynamoAttribute and GlobalSecondaryIndex) that represent the static types of the DynamoDB constructs.

// DynamoAttribute represents an attribute for describing the key schema for the table and indexes.
type DynamoAttribute struct {
    Name string
    Type string
}

// DynamoAttributes is an array of DynamoAttribute
type DynamoAttributes []DynamoAttribute

// GlobalSecondaryIndex represents the properties of a global secondary index
type GlobalSecondaryIndex struct {
    Name string
    HashKey string
    ProjectionType string
    WriteCapacity int
    ReadCapacity int
}

// GlobalSecondaryIndexes is an array of GlobalSecondaryIndex
type GlobalSecondaryIndexes []GlobalSecondaryIndex

The input to create a DynamoDB table, the tableArgs, expects a interface{} for these two fields and the underlying infrastructure expects that interface{} to be a list (or a slice of interfaces to use the correct Go terminology). To make that that type conversion happen, you can use the below two ToList() methods for the types above.

// ToList takes a DynamoAttributes object and turns that into a slice of map[string]interface{} so it can be correctly passed to the Pulumi runtime
func (d DynamoAttributes) ToList() []map[string]interface{} {
    array := make([]map[string]interface{}, len(d))
    for idx, attr := range d {
        m := make(map[string]interface{})
        m["name"] = attr.Name
        m["type"] = attr.Type
        array[idx] = m
    }
    return array
}

// ToList takes a GlobalSecondaryIndexes object and turns that into a slice of map[string]interface{} so it can be correctly passed to the Pulumi runtime
func (g GlobalSecondaryIndexes) ToList() []map[string]interface{} {
    array := make([]map[string]interface{}, len(g))
    for idx, attr := range g {
        m := make(map[string]interface{})
        m["name"] = attr.Name
        m["hash_key"] = attr.HashKey
        m["projection_type"] = attr.ProjectionType
        m["write_capacity"] = attr.WriteCapacity
        m["read_capacity"] = attr.ReadCapacity
        array[idx] = m
    }
    return array
}

The above two methods make it easier to use static typed objects in your code, while still being able to use the Pulumi runtime to create your DynamoDB tables.

Building a table

The next step is to bring all of that together and create the table. In this sample I’ve used a table with usernames and unique IDs, that I use in one of my apps to keep track of order data. Since the actual order data isn’t modeled here, you won’t be able to use it in your queries.

// Create the attributes for ID and User
dynamoAttributes := DynamoAttributes{
    DynamoAttribute{
        Name: "ID",
        Type: "S",
    },
    DynamoAttribute{
        Name: "User",
        Type: "S",
    },
}

// Create a Global Secondary Index for the user field
gsi := GlobalSecondaryIndexes{
    GlobalSecondaryIndex{
        Name: "User",
        HashKey: "User",
        ProjectionType: "ALL",
        WriteCapacity: 10,
        ReadCapacity: 10,
    },
}

// Create a TableArgs struct that contains all the data
tableArgs := &dynamodb.TableArgs{
    Attributes: dynamoAttributes.ToList(),
    HashKey: "ID",
    WriteCapacity: 10,
    ReadCapacity: 10,
    GlobalSecondaryIndexes: gsi.ToList(),
}

// Let the Pulumi runtime create the table
userTable, err := dynamodb.NewTable(ctx, "User", tableArgs)
if err != nil {
    return err
}

// Export the name of the newly created table as an output in the stack
ctx.Export("TableName", userTable.ID())

Complete code

Combining all of the above into a single, runnable, Go program

package main

import (
    "github.com/pulumi/pulumi-aws/sdk/go/aws/dynamodb"
    "github.com/pulumi/pulumi/sdk/go/pulumi"
)

// DynamoAttribute represents an attribute for describing the key schema for the table and indexes.
type DynamoAttribute struct {
    Name string
    Type string
}

// DynamoAttributes is an array of DynamoAttribute
type DynamoAttributes []DynamoAttribute

// ToList takes a DynamoAttributes object and turns that into a slice of map[string]interface{} so it can be correctly passed to the Pulumi runtime
func (d DynamoAttributes) ToList() []map[string]interface{} {
    array := make([]map[string]interface{}, len(d))
    for idx, attr := range d {
        m := make(map[string]interface{})
        m["name"] = attr.Name
        m["type"] = attr.Type
        array[idx] = m
    }
    return array
}

// GlobalSecondaryIndex represents the properties of a global secondary index
type GlobalSecondaryIndex struct {
    Name string
    HashKey string
    ProjectionType string
    WriteCapacity int
    ReadCapacity int
}

// GlobalSecondaryIndexes is an array of GlobalSecondaryIndex
type GlobalSecondaryIndexes []GlobalSecondaryIndex

// ToList takes a GlobalSecondaryIndexes object and turns that into a slice of map[string]interface{} so it can be correctly passed to the Pulumi runtime
func (g GlobalSecondaryIndexes) ToList() []map[string]interface{} {
    array := make([]map[string]interface{}, len(g))
    for idx, attr := range g {
        m := make(map[string]interface{})
        m["name"] = attr.Name
        m["hash_key"] = attr.HashKey
        m["projection_type"] = attr.ProjectionType
        m["write_capacity"] = attr.WriteCapacity
        m["read_capacity"] = attr.ReadCapacity
        array[idx] = m
    }
    return array
}

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        // Create the attributes for ID and User
        dynamoAttributes := DynamoAttributes{
            DynamoAttribute{
                Name: "ID",
                Type: "S",
            },
            DynamoAttribute{
                Name: "User",
                Type: "S",
            },
        }

        // Create a Global Secondary Index for the user field
        gsi := GlobalSecondaryIndexes{
            GlobalSecondaryIndex{
                Name: "User",
                HashKey: "User",
                ProjectionType: "ALL",
                WriteCapacity: 10,
                ReadCapacity: 10,
            },
        }

        // Create a TableArgs struct that contains all the data
        tableArgs := &dynamodb.TableArgs{
            Attributes: dynamoAttributes.ToList(),
            HashKey: "ID",
            WriteCapacity: 10,
            ReadCapacity: 10,
            GlobalSecondaryIndexes: gsi.ToList(),
        }

        // Let the Pulumi runtime create the table
        userTable, err := dynamodb.NewTable(ctx, "User", tableArgs)
        if err != nil {
            return err
        }

        // Export the name of the newly created table as an output in the stack
        ctx.Export("TableName", userTable.ID())
    })
}

Pulumi up

The last step is to add all of this to a new Pulumi project and run the pulumi up command. To create a new, Go based, project, you can run the command

pulumi new go \
--name builder \
--description "An awesome Pulumi infrastructure-as-code Stack" \
--stack retgits/builderstack

Now you can replace the code from the Go file with the code above and run pulumi up. For a little more in-depth info on creating a new Pulumi project, check out one of my previous posts.