Simple Atomic Counter with DynamoDB and API Gateway

A web application often needs a central counter. We have several simple use-cases like the member id for a registration application. Or the amount collected on a sales application - there are many that we all have seen. We have to make sure the concurrent updates do not clash, and the no data is lost in the process. This problem increases further when our application scales larger on the cloud - with millions of concurrent users.

It is best to avoid such a use case - in large scale applications. It is always advisable that we use a design that does not need an atomic counter - for example using UUID instead of an increasing number. But there are times where we just need the counter. When I found myself in such a situation, here is a solution that I used. It is based on API Gateway - Dynamo DB integration. It has a very good response time and minimal cost.

DynamoDB

We all know what is DynamoDB. For those who don't, it is the wonderful serverless NoSQL database provided by AWS. It scales seamlessly to huge volumes and throughput - maintaining extremely fast operations with latency in single digit milliseconds. It is the ideal database for anyone - startup, or enterprise. DynamoDB ensures eventual consistency for all updates. But there is a way to get atomic updates as well.

Going by the One App - One Table philosophy, create a new Item in the table. You can choose the "primaryKey" field as per your data design.

{
   primaryKey: "atomicCounter",
   counterValue: 0
}

So we have a counter that starts with 0. Now, we to configure it to increment atomically and return the number. DynamoDB API provides a method to do it. In order to invoke it, we have to configure the API Gateway

API Gateway

The API Gateway is a versatile service to configure and export API's to the external world. We can use it to invoke Lambda functions, or directly invoke specific AWS Services. We will use the latter to invoke DynamoDB directly.

So we go to the API Gateway Console, to create a new Rest API - or just a new Resource/Method in your existing application API. Create a new API, and resource and then add a GET method into it. We can use a POST as well. But, GET is more elegant since we only get the counter without providing anything.

Next, we configure the API integration to invoke the DynamoDB.

Request Integration

Next we provide an integration mapping for the request. We do it here - at the bottom of the Request Integration page.

UpdateItem
{
    "TableName": "donation-data",
    "Key": {
        "context": {
            "S": "atomicCounter"
        },
        "id": {
            "S": "id"
        }
    },
    "UpdateExpression": "set counterValue = counterValue + :num",
    "ExpressionAttributeValues": {
        ":num": {"N": "1"}
    },
    "ReturnValues" : "UPDATED_OLD"
}

Response Integration

Similarly, we have to map the output on the response. Click on the Response Integration link and add this mapping in there

#set($value = $input.json('Attributes.counterValue.N'))
#set($l = $value.length())
#set($l = $l - 1)
$value.substring(1,$l)

Deploy

We are done with configuring the API. Now click on Deploy API, select a stage and it is ready to test.

Invoke the API from Postman. Everytime you invoke it, you will see the number increasing.

The update operation we used is atomic, so we will always have a unique number from the API.