Build a Serverless URL shortener with Go

Using AWS Lambda, DynamoDB and API Gateway

This blog post covers how to build a Serverless URL shortener application using Go. It leverages AWS Lambda for business logic, DynamoDB for persistence and API Gateway to provide the HTTP endpoints to access and use the application. The sample application presented in this blog is a trimmed down version of bit.ly or other solutions you may have used or encountered.

It's structured as follows:

• I will start off with a quick introduction and dive into how to deploy try the solution.
• After that, I will focus on the code itself. This will cover:
  • The part which is used to write the infrastructure (using Go bindings for AWS CDK)
  • And also the core business logic which contains the Lambda function (using Lambda Go support) as well as the DynamoDB operations (using the DynamoDB Go SDK)

In this blog, you will learn:

• How to use the DynamoDB Go SDK (v2) to execute CRUD operations such as PutItem, GetItem, UpdateItem and DeleteItem
• How to use AWS CDK Go bindings to deploy a Serverless application to create and manage a DynamoDB table, Lambda functions, API Gateway and other components as well.

Once you deploy the application, you will be able to create short codes for URLs using the endpoint exposed by the API Gateway and also access them.

# create short code for a URL (e.g. https://abhirockzz.github.io/)
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

# access URL via short code
curl -i $URL_SHORTENER_APP_URL/<short-code>

Lets get started - Deploy the Serverless application

Before you proceed, make sure you have the Go programming language (v1.16 or higher) and AWS CDK installed.

Clone the project and change to the right directory:

git clone https://github.com/abhirockzz/serverless-url-shortener-golang
cd cdk

To start the deployment...

.. all you will do is run a single command (cdk deploy), and wait for a bit. You will see a (long) list of resources that will be created and will need to provide your confirmation to proceed.

Don't worry, in the next section I will explain what's happening.

cdk deploy

# output

Bundling asset ServerlessURLShortenerStack1/create-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/access-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/update-url-status-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/delete-url-function/Code/Stage...

✨  Synthesis time: 10.28s

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
.......

Do you wish to deploy these changes (y/n)?

This will start creating the AWS resources required for our application.

If you want to see the AWS CloudFormation template which will be used behind the scenes, run cdk synth and check the cdk.out folder

You can keep track of the progress in the terminal or navigate to AWS console: CloudFormation > Stacks > ServerlessURLShortenerStack

Once all the resources are created, you can try out the application. You should have:

• Four Lambda functions (and related resources)
• A DynamoDB table
• An API Gateway (as well as routes, integrations)
• along with a few others (like IAM roles etc.)

Before you proceed, get the API Gateway endpoint that you will need to use. It's available in the stack output (in the terminal or the Outputs tab in the AWS CloudFormation console for your Stack):

Shorten some URLs!

Start by generating short codes for a few URLs

# export the API Gateway endpoint
export URL_SHORTENER_APP_URL=<replace with apigw endpoint above>

# for example:
export URL_SHORTENER_APP_URL=https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/

# invoke the endpoint to create short codes
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

curl -i -X POST -d 'https://dzone.com/users/456870/abhirockzz.html' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

curl -i -X POST -d 'https://abhishek1987.medium.com/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

To generate a short code, you need to pass the original URL in the payload body as part of a HTTP POST request (for e.g. https://abhishek1987.medium.com/)

'Content-Type: text/plain' is important, otherwise API Gateway will do base64 encoding of your payload

If all goes well, you should get a HTTP 201 along with the short code in the HTTP response (as a JSON payload).

HTTP/2 201 
date: Fri, 15 Jul 2022 13:03:20 GMT
content-type: text/plain; charset=utf-8
content-length: 25
apigw-requestid: VTzPsgmSoAMESdA=

{"short_code":"1ee3ad1b"}

To confirm, check the DynamoDB table.

Notice an active attribute there? More on this soon

Access the URL using the short code

With services like bit.ly etc. you typically create short links for your URLs and share them with the world. We will do something similar. Now that you have the short code generated, you can share the link (it's not really a short link like bit.ly but that's ok for now!) with others and once they access it, they would see the original URL.

The access link will have the following format - <URL_SHORTENER_APP_URL>/<generated short code> for e.g. https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b

If you navigate to the link using a browser, you will be automatically redirected to the original URL that you had specified. To see what's going on, try the same with curl:

curl -i $URL_SHORTENER_APP_URL/<short code>

# example
curl -i https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/0e1785b1

This is simply an HTTP GET request. If all goes well, you should get an HTTP 302 response (StatusFound) and the URL re-direction happens due to the the Location HTTP header which contains the original URL.

HTTP/2 302 
date: Fri, 15 Jul 2022 13:08:54 GMT
content-length: 0
location: https://abhirockzz.github.io/
apigw-requestid: VT0D1hNLIAMES8w=

How about using a short code that does not exist?

Set the status

You can enable and disable the short codes. The original URL will only be accessible if the association is in active state.

To disable a short code:

curl -i -X PUT -d '{"active": false}'  -H 'Content-Type: application/json' $URL_SHORTENER_APP_URL/<short code>

# example
curl -i -X PUT -d '{"active": false}'  -H 'Content-Type: application/json' https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b

This is an HTTP PUT request with a JSON payload that specifies the status (false in this case refers to disable) along with the short code which is a path parameter to the API Gateway endpoint. If all works well, you should see a HTTP 204 (No Content) response:

HTTP/2 204 
date: Fri, 15 Jul 2022 13:15:41 GMT
apigw-requestid: VT1Digy8IAMEVHw=

Check the DynamoDB record - the active attribute must have switched to false.

As an exercise, try the following:

1. access the URL via the same short code now and check the response.
2. access an invalid short code i.e. that does not exist
3. enable a disabled URL (use {"active": true})

Ok, so far we have covered all operations, except delete. Lets try that and wrap up the CRUD!

Delete

curl -i -X DELETE $URL_SHORTENER_APP_URL/<short code>

# example
curl -i -X DELETE https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b

Nothing too surprising. We use a HTTP DELETE along with the short code. Just like in case of update, you should get a HTTP 204 response:

HTTP/2 204 
date: Fri, 15 Jul 2022 13:23:36 GMT
apigw-requestid: VT2NzgjnIAMEVKA=

But this time of course, the DynamoDB record should have been deleted - confirm the same.

What happens when you try to delete a short code that does not exist?

Don't forget to clean up!

Once you're done, to delete all the services, simply use:

cdk destroy

Alright, now that you've actually seen "what" the application does, let's move on to the "how".
We will start with the AWS CDK code and explore how it does all the heavy lifting behind to setup the infrastructure for our Serverless URL shortener service.

With AWS CDK, Infrastructure-IS-code!

You can check out the code in this GitHub repo. I will walk you through the keys parts of the NewServerlessURLShortenerStack function which defines the workhorse of our CDK application.

I have omitted some of the code for brevity

We start by creating a DynamoDB table. A primary key is all that's required in order to do that - in this case shortcode (we don't have range/sort key in this example)

    dynamoDBTable := awsdynamodb.NewTable(stack, jsii.String("url-shortener-dynamodb-table"),
        &awsdynamodb.TableProps{
            PartitionKey: &awsdynamodb.Attribute{
                Name: jsii.String(shortCodeDynamoDBAttributeName),
                Type: awsdynamodb.AttributeType_STRING}})

Then, we create an API Gateway (HTTP API) with just one line of code!

urlShortenerAPI := awscdkapigatewayv2alpha.NewHttpApi(stack, jsii.String("url-shortner-http-api"), nil)

We move on to the first Lambda function that creates short codes for URLs. Notice that we use an experimental module awscdklambdagoalpha (here is the stable version at the time of writing). If your Go project is structured in a specific way (details here) and you specify its path using Entry, it will automatically take care of building, packaging and deploying your Lambda function! Not bad at all!

In addition to Local bundling (as used in this example), Docker based builds are also supported.

    createURLFunction := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("create-url-function"),
        &awscdklambdagoalpha.GoFunctionProps{
            Runtime:     awslambda.Runtime_GO_1_X(),
            Environment: funcEnvVar,
            Entry:       jsii.String(createShortURLFunctionDirectory)})

    dynamoDBTable.GrantWriteData(createURLFunction)

Finally, we add the last bit of plumbing by creating a Lambda-HTTP API integration (notice how the Lambda function variable createURLFunction is referenced) and adding a route to the HTTP API we had created - this in turn refers to the Lambda integration.

    createFunctionIntg := awscdkapigatewayv2integrationsalpha.NewHttpLambdaIntegration(jsii.String("create-function-integration"), createURLFunction, nil)

    urlShortenerAPI.AddRoutes(&awscdkapigatewayv2alpha.AddRoutesOptions{
        Path:        jsii.String("/"),
        Methods:     &[]awscdkapigatewayv2alpha.HttpMethod{awscdkapigatewayv2alpha.HttpMethod_POST},
        Integration: createFunctionIntg})

This was just for one function - we have three more remaining. The good part is that the template for all these are similar i.e.

1. create the function
2. grant permission for DynamoDB
3. wire it up with API Gateway (with the correct HTTP method i.e. POST, PUT, DELETE)

So I will not repeat it over here. Feel free to grok through the rest of the code.

Now that you understand the magic behind the "one-click" infrastructure setup, let's move on to the core logic of the application.

URL shortener Lambda function and DynamoDB logic

There are four different functions, all of which are in their respective folders and all of them have a few things in common in the way they operate:

1. They do dome initial processing - process the payload, or extract the path parameter from the URL etc.
2. Invoke a common database layer - to execute the CRUD functionality (more on this soon)
3. Handle errors as appropriate and return response

With that knowledge, it should be easy to follow along the code.

As before, some parts of the code have been omitted for brevity

Create function

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
    url := req.Body

    shortCode, err := db.SaveURL(url)
    if err != nil {//..handle error}

    response := Response{ShortCode: shortCode}
    respBytes, err := json.Marshal(response)
    if err != nil {//..handle error}

    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusCreated, Body: string(respBytes)}, nil
}

This function starts by reading the payload of the HTTP request body - this is a string which has the URL for which the short code is being created. It invokes the database layer to try and save this record to DynamoDB and handles errors. Finally, it returns a JSON response with the short code.

Here is the function that actually interacts with DynamoDB to get the job done.

func SaveURL(longurl string) (string, error) {
    shortCode := uuid.New().String()[:8]

    item := make(map[string]types.AttributeValue)

    item[longURLDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: longurl}
    item[shortCodeDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: shortCode}
    item[activeDynamoDBAttributeName] = &types.AttributeValueMemberBOOL{Value: true}

    _, err := client.PutItem(context.Background(), &dynamodb.PutItemInput{
        TableName: aws.String(table),
        Item:      item})

    if err != nil {//..handle error}

    return shortCode, nil
}

For the purposes of this sample app, the short code is created by generating a UUID and trimming out the last 8 digits. It's easy to replace this with another technique - all that matters is that you generate a unique string that can work as a short code. Then, it all about calling the PutItem API with the required data.

Access the URL

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

    shortCode := req.PathParameters[pathParameterName]
    longurl, err := db.GetLongURL(shortCode)

    if err != nil {//..handle error}

    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusFound, Headers: map[string]string{locationHeader: longurl}}, nil
}

When someone accesses the short link (as demonstrated in the earlier section), the short code is passed in as a path parameter e.g. http://<api gw url>/<short code>. the database layer is invoked to get the corresponding URL from DynamoDB table (errors are handled as needed). Finally, the response is returned to the user wherein the status code is 302 and the URL is passed in the Location header. This is what re-directs you to the original URL when you enter the short code (in the browser)

Here is the DynamoDB call:

func GetLongURL(shortCode string) (string, error) {

    op, err := client.GetItem(context.Background(), &dynamodb.GetItemInput{
        TableName: aws.String(table),
        Key: map[string]types.AttributeValue{
            shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}}})

    if err != nil {//..handle error}

    if op.Item == nil {
        return "", ErrUrlNotFound
    }

    activeAV := op.Item[activeDynamoDBAttributeName]
    active := activeAV.(*types.AttributeValueMemberBOOL).Value

    if !active {
        return "", ErrUrlNotActive
    }

    longurlAV := op.Item[longURLDynamoDBAttributeName]
    longurl := longurlAV.(*types.AttributeValueMemberS).Value

    return longurl, nil
}

The first step is to use GetItem API to get the DynamoDB record containing URL and status corresponding to the short code. If the item object in the response is nil, we can be sure that a record with that short code does not exist - we return a custom error which can be helpful for our function which can then return an appropriate response to the caller of the API (e.g. a HTTP 404). We also check the status (active or not) and return an error if active is set to false. If all is well, the URL is returned to the caller.

Update status

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

    var payload Payload
    reqBody := req.Body

    err := json.Unmarshal([]byte(reqBody), &payload)
    if err != nil {//..handle error}

    shortCode := req.PathParameters[pathParameterName]

    err = db.Update(shortCode, payload.Active)
    if err != nil {//..handle error}

    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}

The first step is to marshal the HTTP request payload which is a JSON e.g. {"active": false} and then get the short code from the path parameter. The database layer is invoked to update the status and handle errors.

func Update(shortCode string, status bool) error {

    update := expression.Set(expression.Name(activeDynamoDBAttributeName), expression.Value(status))
    updateExpression, _ := expression.NewBuilder().WithUpdate(update).Build()

    condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
    conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()

    _, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
        TableName: aws.String(table),
        Key: map[string]types.AttributeValue{
            shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
        UpdateExpression:          updateExpression.Update(),
        ExpressionAttributeNames:  updateExpression.Names(),
        ExpressionAttributeValues: updateExpression.Values(),
        ConditionExpression:       conditionExpression.Condition(),
    })

    if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
        return ErrUrlNotFound
    }

    return err
}

The UpdateItem API call takes care of changing the status. It's fairly simple except for the all these expressions that you need - especially if you're new to the concept. The first one (mandatory) is the update expression where you specify the attribute you need to set (active in this case) and its value. The second one makes sure that you are updating the status for a short code that actually exists in the table. This is important since, otherwise the UpdateItem API call will insert a new item (we don't want that!). Instead of rolling out the expressions by hand, we use the expressions package.

Delete short code

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

    shortCode := req.PathParameters[pathParameterName]

    err := db.Delete(shortCode)
    if err != nil {//..handle error}
    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}

The delete handler is no different. After the short code to be deleted is extracted from the path parameter, the database layer is invoked to remove it from the DynamoDB table. The result returned to the user is either an HTTP 204 (on success) or the error.

func Delete(shortCode string) error {

    condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
    conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()

    _, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
        TableName: aws.String(table),
        Key: map[string]types.AttributeValue{
            shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
        ConditionExpression:       conditionExpression.Condition(),
        ExpressionAttributeNames:  conditionExpression.Names(),
        ExpressionAttributeValues: conditionExpression.Values()})

    if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
        return ErrUrlNotFound
    }

    return err
}

Just like UpdateItem API, the DeleteItem API also takes in a condition expression. If there is no record in the DynamoDB table with the given short code, an error is returned. Otherwise, the record is deleted.

That completes the code walk through!

Wrap up

In this blog post you learnt how to use DynamoDB Go SDK using a URL Shortener sample application. You also integrated it with AWS Lambda and API Gateway to build a Serverless solution whose infrastructure was also defined using actual code (as opposed to yaml, JSON etc.), thanks to the Go support in AWS CDK.