DEV Community: Daniel Rayo

Customize Go Builds on AWS SAM with Dockerfiles and Makefiles

Daniel Rayo — Mon, 20 Jan 2025 04:36:11 +0000

This post is the second part on my series Building APPS with AWS SAM and Go feel free to check the first part. In last chapter we discover that AWS don't provide much info on how to structure a scalable Go project without REPEATING A LOT OF CODE.

Now I will show you ways in which we can control our build steps with Dockerfiles and Makefiles.

The code showed in this lecture will be in this repo:
https://github.com/DanielRasho/AWS-SAM-go
Check the different git branches for each use case 😉.

Lets get started 🏃💨!

The problem

After came up with the new project structure, I decided to use Nix ❄️ for managing all my project dependencies (languajes, tools, libraries). The way Nix works, is by creating a new temporary shell with all the dependencies you request it.

I noted that that whenever i tried to run executables that where built within a Nix shell, they throw a weird error:

libc.so.6 not found in /nix/23fj39chsggb09s.libc

And the lambda will stop execution. It took my an ENTIRE DAY to figure out the cause: When compiling, Go sometimes link C libraries libraries dynamically to the executable, by specifying the route on the system to where the library is. This where the linked libraries to the executables built with Nix:

$ ldd bootstrap 
        linux-vdso.so.1 (0x00007ffff7fc4000)
        libresolv.so.2 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/libresolv.so.2 (0x00007ffff7fac000)
        libpthread.so.0 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/libpthread.so.0 (0x00007ffff7fa7000)
        libc.so.6 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/libc.so.6 (0x00007ffff7c00000)
        /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/ld-linux-x86-64.so.2 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc6000)

Nix has a not-standard way to store dependencies, and since the lambdas run on isolated Docker containers, when they tried to execute the code, and try to find the linked libraries on those paths THEY COULDN'T FIND THEM, since those files lived only on my local Nix installation.

I had to find a way to tell AWS SAM how to compile my code , to control this linked libraries.

Ways to deliver our Go project to AWS

So first a bit of context. We can deliver our code to AWS in 2 presentations:

Zip files 📁

We compile the code locally on our machine, and sent the raw executable to AWS on .zip. Amazon will simply copy the executable on the docker containers. This option provides the fastest Cold Starts.

Docker Images 🐋

Instead of giving AWS the executable. We give them the instructions to compile it WITHIN the docker container in which will be run. This option ensures full compatibility at the costs of slower Cold Starts

The solutions

At the end I went for Dockerfiles, since I wanted to still be using Nix. But I will show you both methods

Zip files 📁

To specify the build steps for a Zip file we will need a structure like this one (notice the Makefile at the root):

.
├── cmd/
│   ├── function1/
│   │   └── function1.go  # contains main()
│   └── function2/
│       └── function2.go  # contains main()
├── internal/
│   └── SHAREDFUNC.go
├── Makefile
├── go.mod
├── go.sum
├── samconfig.toml
└── template.yaml

Then within the Makefile we provide a command for each function following the pattern: build-<Function_Name> (this is enforced by AWS SAM) followed by our built instructions, AWS has this article in how to set this build commands

.PHONY: build

build:
    sam build

build-HelloWorldFunction:
    GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap ./cmd/function1/main.go
    cp ./bootstrap $(ARTIFACTS_DIR)

build-ByeWorldFunction:
    GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap ./cmd/function2/main.go
    cp ./bootstrap $(ARTIFACTS_DIR)

$(ARTIFACTS_DIR) is an env variable provided by SAM automatically

Finally we just have to notify SAM of this new process:

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Metadata:
      BuildMethod: makefile
    Properties:
      CodeUri: ./
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures:
        - x86_64
      Events:
        CatchAll:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: GET

We use BuildMethod: makefile, also SAM will look for the Makefile only where CodeUri specifies.

Docker Images 🐋

The process is similar for building docker images, we create a Dockerfile and .dockerignore at the root dir:

.
├── cmd/
│   ├── function1/
│   │   └── function1.go  # contains main()
│   └── function2/
│       └── function2.go  # contains main()
├── internal/
│   └── SHAREDFUNC.go
├── Dockerfile
├── .dockerignore
├── go.mod
├── go.sum
├── samconfig.toml
└── template.yaml

Within the Dockerfile we specify the building steps. I used the ARG statement to tell Docker where to look for then entry point of my lambda at build time.

FROM public.ecr.aws/docker/library/golang:1.19 as build-image
ARG ENTRY_POINT  # !IMPORTANT
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -tags lambda.norpc -o lambda-handler ${ENTRY_POINT}
FROM public.ecr.aws/lambda/provided:al2023
COPY --from=build-image /src/lambda-handler .
ENTRYPOINT ./lambda-handler

And finally, made some teaws on template.yaml:

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Metadata:
      DockerTag: provided.al2023-v1
      DockerContext: ./  # Where to look for the dockerfile and code.
      Dockerfile: Dockerfile
      DockerBuildArgs: 
        ENTRY_POINT: "./cmd/function1/main.go" # Location of ENTRY POINT
    Properties:
      PackageType: Image
      Architectures:
        - x86_64
      Events:
        CatchAll:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: GET

Pay attention to the Metadata, and PackageType:Image sections. Remember the ARG ENTRY_POINT we defined on Dockerfile we are passing it on DockerBuildArgs, this way we just need on Dockerfile for all lambdas

Wrap up

There's not wrap up, Its already late at night, I should go to sleep 😴 😴 😴, and you should do it too.

Building an API with AWS SAM and Go

Daniel Rayo — Mon, 20 Jan 2025 02:59:46 +0000

AWS SAM is a great way to deploy web apps through infrastructure as code (IAC). I recently tried to use it for a project in my job, but I stumble with a harsh reality...

Go is the Ugly Duckling of AWS 🦆

The chapter dedicated to Go in AWS SAM documentation, is really short and vague and suggest to REPEAT OUR SOURCE CODE A LOT! Having one go.mod, go.sum and utility functions for each lambda 👺!

I wrote this post for you, someone who’s probably just as lost as I was 😵‍💫. Let’s figure this out together!

This will be a 2 parts series:

File Structure to NOT REPEAT CODE (This post)
Custimize builds with Makefiles and Dockerfiles

Some Context on Go Runtimes

At the current time, Go runtime for lambdas is not supported as-is. This means that there is not specific option on AWS lambdas to specify that your code is written in Go. Instead, AWS provides 2 generic runtimes 🐧:

al2 (Amazon Linux 2)
al2023 (Amazon Linux 2023)

Which refers to the OS in which the lambda will be run. Is recommended to use al2023 since is newer and compatible with AWS Graviton processors which deliver better performance for a better price.

Anyway, this runtimes asks us to provide an executable (usually named bootstrap), which will be executed on every lambda function. So instead of deliver code to the lambdas, we deliver an executable that we compiled with Go previously. Simple right?

This also removes the necessity of using lambda layers, that languages like JS need, since all common dependencies will already be packed in the compiled executable 😼.

The problem

So, how we build that executable? AWS suggest us that every lambda should be stored on a folder along with its go.mod and go.sum, the template they provides us looks like this:

.
├── hello-world/
│   ├── go.mod
│   ├── go.sum
│   └── main.go
├── events/
│   └── ...
├── samconfig.toml
└── template.yaml

And this is the function definition on template.yaml

  HelloWorldFunction:
    Type: AWS::Serverless::Function 
    Metadata:
      BuildMethod: go1.x
    Properties:
      CodeUri: hello-world/
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures:
        - x86_64
      Events:
        CatchAll:
          Type: Api
          Properties:
            Path: /hello
            Method: GET

If we look the Lambda definition, we learn that

BuildMethod: go1.x We use AWS built-it Go builder, to build the executable for us
CodeUri: hello-world/ The lambda code will exclusively live in this directory.
Handler: bootstrap The name of the executable will be bootstrap
Runtime: provided.al2023 this will be the runtime.

Do you see the problem? At the moment we need a second lambda, we have to create a A NEW DIRECTORY WITH ITS OWN go.mod, go.sum and dependencies, ¿What if we want to share a utility function between to lambdas? TO BAAD 😿 !you will have to copy the same file in the new lambda folder. Leaving with a file structure that looks like this:

.
├── function1/
│   ├── go.mod
│   ├── go.sum
│   ├── main.go
│   └── SHAREDFUNC.go
├── function2/
│   ├── go.mod
│   ├── go.sum
│   ├── main.go
│   └── SHAREDFUNC.go
├── events/
│   └── ...
├── samconfig.toml
└── template.yaml

This is gross 🤮! There is a lot of repeated code! And it will become worst the more lambdas we add. There must be a better way!

The Solution

Since I wanted to share go.mod, go.sum and utility code through all lambdas, I came up with this structure:

.
├── cmd/
│   ├── function1/
│   │   └── function1.go  # contains main()
│   └── function2/
│       └── function2.go  # contains main()
├── internal/
│   └── SHAREDFUNC.go
├── events/
│   └── ...
├── go.mod
├── go.sum
├── samconfig.toml
└── template.yaml

I factored out all the common code on an internal/ folder,
Place the go.mod and go.sum files in the root dir
Move all lambda entry points to /cmd (in Go exist this convention that whenever a project produces more than one executable, the entry points are placed within a cmd dir)

Now I just had to notify AWS SAM of this new structure 😸! And I found the solution just by tweaking around the values of CodeUri and Handler.

The Secret 🔒

Seems that if you

Move the go.mod and go.sum to the root folder
Set CodeUri, to wherever folder your function entry point is.

SAM will detect it automatically and build with root dependencies and internal/ code 🎉 🎉 🎉

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x
    Properties:
      CodeUri: ./cmd/function1/
      Handler: bootstrap
      Runtime: provided.al2023
      ... # More properties ...
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x
    Properties:
      CodeUri: ./cmd/function2
      Handler: bootstrap
      Runtime: provided.al2023
      ... # More properties ...

Can it be better?

Yeeees ✨, we will discuss more ways to customize your Go compilation in the next post!

Vue reactivity for impatient devs

Daniel Rayo — Sun, 28 Apr 2024 16:58:09 +0000

In Vue it's all about managing the state of components. Modifying the State forces the UI to rebuild with the new data. But then, what is this so-called State?

App = State + UI

We can understand the UI as a printer that produces one T-shirt after another non-stop. The printer doesn't know:

Which T-shirt was printed before.
Who will sell
The theme.

💡 In summary: The printer only focuses on blindly printing T-shirts without remembering.

The client that orders the printer to print is the State. When they want a new batch of T-shirts, they just change the form data.

=== Form ===
Color: Red
Size: XL
Design: DonutAscii.svg

In Vue and many graphical libraries, something similar happens. There's a set of important variables that must be remembered while the app is running, which is the state that can also be seen as a form:

=== STATE ===
Username: smaugthur
Password: 1234
Games Won: 3
Game Finished: false
PositionX: 0
PositionY: -500

The UI, like the T-shirt printer, has a blank template that it needs to fill with the data requested by the client. When it receives the State data, we have a screen like this:

💡 Any change in the state will force the UI to print a new image with the updated data.

However, the UI can also receive commands from the user to modify the State. A simple click on "Activate Dark Mode" will change the internal state, which in turn will force the UI to reprint a new screen. It's a cycle where...

💡 The state modifies the UI, and the UI modifies the state.

What was the point of understanding this?

The UI isn't intelligent 🧠, if nobody tells it, it can't know that the state has changed. That's why there have to be explicit ways to say, "Hey, the state changed, get to work!" This mechanism of making the UI react when the state changes is called Reactive States.

Now we're going to talk about the most important ways in Vue to achieve these reactive states, and we'll start by understanding how Vue handles these mechanisms Under the hood.

Fundamentals of Reactivity in Vue

In Vue, the state is represented by variables. JavaScript doesn't have internal mechanisms to observe changes in a variable and react to them. However, it does offer Proxies, which are effectively intermediary objects between the client and the actual variable.

💡 Proxy objects hold the actual value inside them and act as guards that decide what, how, and when the value of the real variable should be modified.

Vue wraps all state variables in Proxies, so now it has control over all the changes happening in the variables, giving it knowledge of when to rebuild the UI.

💡 Vue strongly recommends working with Proxy versions of the state.
Vue offers the following methods to wrap variables inside a Proxy and manage their state:

reactive()

It's a function that takes an object and creates a Proxy with the same data as the original object. Although it works the same as the original object, it is completely independent. Some important considerations:

Any changes to the Proxy object will not modify the original nor vice versa because THEY ARE DIFFERENT OBJECTS.
It performs a deep transformation. That means if there are nested objects, it will also convert them to their Proxy equivalent. This will cause any changes in the child properties to also trigger a state change.
It's not intended for use with primitive types (String, boolean, int...).

import { reactive } from 'vue'

const state = reactive({ count: 0 })

<button @click="state.count++">
  {{ state.count }}
</button>

ref()

It's another function that constructs proxies, but unlike reactive(), it's intended to handle both primitive types and objects. It works very simply. If it encounters a primitive, it simply creates a Proxy wrapping the value, and if it encounters an object, it executes a reactive() to obtain its Proxy version.

Considerations:

Like reactive, objects produced by ref() are independent of their originals.
It performs a deep transformation.
To access the wrapped value of a ref() Proxy, you call the proxy.value property. This is not the case with reactive, where you can directly access the properties.

import { ref } from 'vue'

const count = ref(0)

console.log(count) // { value: 0 }
console.log(count.value) // 0

count.value++
console.log(count.value) // 1

And thats it, happy coding. Here some other useful resources: