Going Serverless with Go: A Guide to Lambda Functions

Have you ever thought, “I wish I could just write code without all the extra fuss”? Well, you’re in the right place. Our journey is all about using Go to code without worrying about servers. It’s like having a self-driving car; you just sit back and enjoy the ride.

In this blog, we’ll ease into setting up a REST API using Go in a Serverless fashion to get the user data and save it to DynamoDB whilst keeping it light, friendly, and jargon-free.

Though there are a couple of good blogs for Go Serverless setup, many lack the local development and enhancing the development experience of writing go lambdas. We will touch up on that too ;)

The Serverless Framework

Let's unwrap the concept of 'serverless' with an analogy that's both fun and enlightening. Imagine you're planning a grand celebration (that's your application). In the old days, you'd need to own a venue (a server) to host this bash. Sounds like a heap of responsibility, doesn't it?

Enter the world of serverless – it's akin to hiring an expert party planner. This planner takes care of the venue, the setup, the cleanup – everything but the party itself. That's your job. You bring the code (the life of the party) and let the experts handle the where and how of its execution.

In the realm of technology, serverless computing lets you build and run applications without the burden of server management. It's all about writing your code and letting a cloud provider run it. This provider handles the heavy lifting – scaling up or down as needed, maintaining the servers, and ensuring uptime. It's like having an invisible but incredibly efficient helper ensuring your app can handle any amount of guests (users) without breaking a sweat.

This open-source wonder is not just a tool; it's a wizard in your coding journey. Serverless Framework streamlines the process of deploying serverless applications, taking the tediousness out of server management. What sets the Serverless Framework apart from other tools in its league? Its ability to play nice with multiple cloud providers. Though it began with a focus on AWS, it's grown to support Azure, GCP, and many others. This multi-cloud provider support means you're not putting all your eggs in one basket. You have the flexibility to choose or switch your cloud provider based on your needs, costs, or even whims!

Serverless Configuration

Equip Yourself with the Right Tools

Before we embark on this adventure, make sure you have two key items in your toolkit:

1. Go - If Go isn't already part of your developer toolkit, now's the time to welcome it aboard. Head over to the official Go website and grab the latest version. We're focusing on Go 1.21 and above in this blog, so make sure you're up-to-date. The installation is a breeze – just follow the instructions for your operating system, and you're set!
2. Serverless Framework - This nifty tool is like having a personal assistant for your serverless adventure. To get this assistant on board, simply run npm install -g serverless in your command line. This requires Node.js, so if you don't have it yet, a quick trip to the Node.js website will sort that out. As a bonus, we'll sprinkle in some nodemon magic for auto-reloading our functions, making our development process smoother.
3. Boiler Plate Code - Let's lay the foundation with some boilerplate code. Fire up your command line and execute serverless create --template aws-go --path go-serverless-api. Once done, step into your new project with cd go-serverless-api. Though it's optional (since we'll be customizing the Go file anyway), it's a great starting point and sets the tone for our serverless application.

To ensure our serverless function can smoothly chat with DynamoDB, we need to grant it the right permissions. Let's add an IAM Role statement to our serverless.yml file that allows our function to access DynamoDB:

service: go-serverless-api

frameworkVersion: "3"

useDotenv: true
provider: 
  name: aws
  runtime: go1.x # Change to `provided.al2` while deployment
  stage: ${opt:stage, 'dev'}
  timeout: 120 # Change to 30 while deployment
  memorySize: 256
  versionFunctions: false
  iamRoleStatements: 
    - Effect: Allow
      Action: 
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:BatchGetItem
      Resource: "arn:aws:dynamodb:us-east-1:${env:ACCOUNT_ID}:table/${env:TABLE_NAME}"

We will install three plugins to help the development,

1. serverless-offline, lets us emulate AWS Lambda and API Gateway locally, making testing a breeze
2. serverless-dotenv-plugin, load your .env file variables automatically. It's optional, though, if you prefer the manual approach
3. serverless-go-plugin, automatically compiles go functions on sls deploy. Say goodbye to manual build commands

Install all the plugins using your favourite package managers (mine is bun) and we will be adding them to the plugin section of serverless.yaml like this,

plugins: 
  - serverless-offline
  - serverless-go-plugin
  - serverless-dotenv-plugin

When it comes to adding functions to our serverless.yaml, it's all about personal preference. However, as the number of functions grows, so does the complexity. To keep things neat and tidy, let's define our functions in a separate .yaml file and reference them in our main serverless.yaml

functions: 
  - ${file(functions/functions.yml):functions}

We need a DynamoDB db to work with. You can either use the local DynamoDB client (localhost url has to be mentioned while creating dynamo client) or create a dynamo table in AWS with default settings and name the primary and sort keys as pk and sk

With these tools in hand, we are now equipped, so let's turn the page to the next chapter of our adventure and start crafting some go magic!

Serverless Function

Utilities

Logging with Zerolog

Effective logging is crucial for any function to gain insights and debug issues. We're going to integrate zerolog, an excellent logging library known for its simplicity and performance. Here's how we set it up as both a utility:

func init() {
    once.Do(func() {
        env := config.GetConfig() // To use env from go-envconfig
        var output io.Writer
        zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
        zerolog.TimeFieldFormat = time.RFC3339
        logLevel, err := strconv.Atoi(env.LogLevel)
        if err != nil {
            logLevel = int(zerolog.InfoLevel)
        }
        Logger = zerolog.New(os.Stdout).Level(zerolog.Level(logLevel)).With().Timestamp().Caller().Logger()
    })
}

Env management with go-envconfig

For managing environment variables, go-envconfig is our go-to solution. It simplifies loading and parsing environment configurations. The setup code can be found in the GitHub repository linked at the end.

Defining Structs

We'll define a User struct with both json and dynamodbav tags. The dynamodbav tags specify column names in DynamoDB, while json tags are used for parsing data into this struct.

type primaryKey struct {
    Pk string `json:"pk" dynamodbav:"pk"`
    Sk string `json:"sk" dynamodbav:"sk"`
} 

type User struct {
    primaryKey
    ID        string `json:"id" dynamodbav:"id"`
    Name      string `json:"name" dynamodbav:"name"`
    Email     string `json:"email" dynamodbav:"email"`
    Age       int    `json:"age" dynamodbav:"age"`
    IsActive  bool   `json:"isActive" dynamodbav:isActive""`
}

Dynamo Wrappers

Establishing a connection to DynamoDB is essential. Here’s a function to get a DynamoDB client:

type DB struct {
    client    *dynamodb.Client
    table     string
}

func GetDynamoDBClient(region string) (*dynamodb.Client, error) {
    cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRegion(region))
    if err != nil {
        log.Error().Err(err).Msg(DBError.DBFailedToLoad)
        return nil, err
    }
    dynamodbClient := dynamodb.NewFromConfig(cfg)
    return &DB{
        client:    dynamodbClient,
        tableName: tableName,
    }, err
}

For storing data, we have a wrapper function that accepts any data type and saves it to DynamoDB:

func putItemWrapper(ctx context.Context, db *DB, inputItem interface{}) (*dynamodb.PutItemOutput, error) {
    item, err := attributevalue.MarshalMap(inputItem)
    if err != nil {
        return nil, err
    }

    response, err := db.client.PutItem(ctx, &dynamodb.PutItemInput{
        TableName: aws.String(db.tableName),
        Item:      item,
    })
    if err != nil {
        return nil, err
    }
    return response, nil
}

Go Handler

Now, we reach the heart of our application - the Go handler function itself. This is where the magic happens, turning data into action. We're using aws-lambda-go, the official AWS package for Go Lambda functions, to make this happen. Here's a quick note on structure:

• The function should reside in the main package.
• Only one handler file should exist at the same folder level (for serverless-offline and serverless-go-plugin to work properly).
• Feel free to organize utilities, types, and other components in separate files and import them into your handler.

Our handler would parse the incoming requests body and un marshals into our user struct which we then send to DynamoDB.

func Handler(ctx context.Context, request Request) (Response, error) {
    userItem := User{}
    err := json.Unmarshal([]byte(req.Body), &user)
    if err != nil {
        return JSONResponse(http.StatusBadRequest, err)
    }
    db, _ := dao.GetDynamoDBClient(env.DynamoTable, env.AwsRegion)
    err = dao.PutUserInfo(ctx, db, user)
    if err != nil {
        return NewJSONErrorResponse(http.StatusInternalServerError)
    }
    return JSONResponse(http.StatusCreated, map[string]string{"msg":"User Created"}
}

Note here the JSONResponse utility makes it easy task for us to send back JSON responses, whether it's an error, a struct response, or a map response.

I have considered the simplest access patterns for this project,

1. Save user with their details
2. Retrieve user given user id

You can expand access patterns using Global Secondary Indexes (GSIs). With support for up to 20 GSIs per table, DynamoDB offers plenty of flexibility for various query requirements.

We might need a function to return us a struct which has both primary key and the user info. This function can be used in both put and get operations of DynamoDB

func GetUserDBItem(ctx context.Context, user UserInfo) UserDBItem {
    primaryKey := primaryKey{
        Pk: fmt.Sprintf("userId#%s", user.ID),
        Sk: fmt.Sprintf("userId#%s", user.ID),
    }
    userInfoDBItem := UserDBItem{
        primaryKey: primaryKey,
        UserInfo:   user,
    }
    return userInfoDBItem
}

And giving the output of this function to our previously written putItemWrapper would create the user for us in DynamoDB 🎉

userDBItem := GetUserDBItem(ctx, user)
_, err := putItemWrapper(db, userDBItem)

And as a last touch, let’s not forget to add our function definition in the function.yaml file to make sure serverless considers our lambda for deployments

functions:
  save-user:
    name: save-user
    handler: functions/save-user/main.go
    events:
      - http:
          path: users
          method: post

We can test our lambda’s locally using sls offline command, with nodemon for hot reloading. Make sure to checkout the repo mentioned below 😉

With all the configuration in place, a simple sls deploy command in your terminal breathes life into your function. Hit Enter, and voilà - our serverless function is up and running, ready to save users to your DynamoDB table! 🎉

Dive deeper into this go serverless world with more utilities and comprehensive implementations. Check out the complete codebase in this GitHub Repo - https://github.com/s-abinash/go-serverless-api

Comments

[Max Sveshnikov]

I was ways thinking that serverless is for scripting languages. Go is compiled. How comes?

[Abinash S]

For languages like Go, Rust we make the build of our code for the specific runtime of the lambda we intend to use and then push. This typically involves building every function separately using go build. But this plugin, serverless-go-plugin (thanks to github.com/sean9keenan) makes the job easier. We just need to mention the runtime and other options in the plugin attributes section of serverless file.