DEV Community

Cover image for How to Implement a Distributed Counter in 1 Line of Code
PubNub Developer Relations for PubNub

Posted on

How to Implement a Distributed Counter in 1 Line of Code

Imagine you want to aggregate worldwide data in real-time, such as tallying candidate votes in an election, aggregating website statistics, or counting the number of times an event occurs in your app. Wherever you have a distributed set of clients contributing to the overall count, the problem quickly becomes complex.

The base case is simple: if you only have a single database instance, you would get the current count value from the database, lock that record, increment the count and then write the incremented value back to the database.

However, once your database is replicated across regions, you must ensure that all those regions are synchronized. Because counters are designed to be updated frequently, they pose particular challenges:

  • The latency between geographic nodes will introduce delays, making it difficult to synchronize values in real-time.

  • Consistency in the values leads to a performance tradeoff—since you need to coordinate and confirm operations across multiple nodes, this can lead to bottlenecks.

  • What happens in the event of failure? When you lose connectivity between regions, your network becomes partitioned. You also need to consider what happens when entire regions fail, leading to counter inconsistencies.

  • Counter data will update frequently, leading to race conditions and conflicts where multiple nodes update the counter concurrently. Remember, the counter can be updated from any node in any region.

  • Data is often replicated across multiple nodes for fault tolerance, but how do you keep these replicas in sync while updating the counter?

  • As your solution scales to more regions and nodes, the complexity of coordination and synchronization of your data increases.

  • The counter's value eventually needs to be consistent across all database nodes. The rest of your solution must tolerate the counter being potentially inaccurate at any one point in time.

The most common approach is to store a copy of the counter in each region, updated locally. A separate process is then responsible for monitoring updates to the local counters and aggregating the different values to a consistent “Global Counter.” Some cloud services expose a unique counter type, but not all - so developers often find themselves dealing with the complexities of counter value resolution themselves:

But there is an easier way…

PubNub Functions allow you to write serverside code without spinning up a dedicated backend. Functions are distributed, meaning as your application grows, PubNub Functions will automatically scale so your application can enjoy low latency and 99.999% uptime.

Functions expose several built-in libraries, including the KV (key-value) Store. KV store scales with your Functions, giving you access to key-value storage without impacting your application’s performance.

The KV store incrcounter() method will increment a counter for you regardless of app scale. You can manage up to 100 independent counters in a single PubNub Function and have that function trigger either on-demand or in response to a real-time message sent through the PubNub network.

const kvstore = require(kvstore)
kvstore.incrcounter(counterId, 2) // Increment “counterId” by 2.
Enter fullscreen mode Exit fullscreen mode

Let’s look at an interactive example.

The embedded application below (source code available on Github) lets you vote for your favorite key - just focus on the window and type away. You can also open this demo up in multiple tabs and see updates in real time, it is hosted at https://distributed-counter.netlify.app/ . Note that the demo is designed for desktop.

This page contains embedded content that is not supported by this site , please view it at https://distributed-counter.netlify.app/

Every time you press a key, the Function will count as a vote for that key, and it works as follows:

  • The application detects a keypress and sends a pubnub.signal(keypress).

  • A PubNub Function of type “after signal” is invoked in response to your keypress.

  • The PubNub Function increments the counter that represents the pressed key using the incrcounter() method, which is part of the KV Store module. The source code for this Function can be found a little way down in this blog.

Running the demo yourself

You can quickly get the demo running with your own PubNub keyset and make your own modifications.

Firstly, clone the Front end from Github. In the next step, you will generate the Publish and Subscribe keys that need to be copied into keys.js. After that, you can load index.html on any browser.

  1. Log into the PubNub Portal at admin.pubnub.com. If you do not already have a PubNub account, you can sign up for free.

  2. Create an application to house your Function by selecting ‘Apps’ from the left-hand menu, then click ‘+ Create app.’ This will automatically create a Keyset for you within that app. Copy the Publish and Subscribe keys from this keyset into the keys.js file for the front end.

  3. Create a Function for this application. At the time of writing, we are in the process of transitioning to a new version of Functions - please follow the documentation which matches your situation, either Functions for existing users, or Functions v2 for new users.

  4. Select the After Signal for the Function event type and functionsdemo-counter-vote for the channel name.

It is worth explaining where the event type and channel name come from: The Function will trigger After Signal because the front end of the application is calling pubnub.signal(keypress) whenever a key is pressed. Since we do not need to modify the Signal payload, we can use the asynchronous After Signal, as opposed to the synchronous Before Signal. The front end will send the keypress Signals over the functionsdemo-counter-vote channel, and note that a separate channel, functionsdemo-counter-result is used to return data back to the front end and avoid an endless loop.

The source code for the Function is below, this is also responsible for notifying new users of the current votes for each key, as well as notifying other listening users whenever a new vote happens:

const pubnub = require('pubnub');
const db = require("kvstore");


export default (request) => {
 var keysbatch1 = ["49", "50", "51", "52", "53", "54", "55", "56", "57", "48"]
 var keysbatch2 = ["189", "187", "81", "87", "69", "82", "84", "89", "85", "73"]
 var keysbatch3 = ["79", "80", "219", "221", "65", "83", "68", "70", "71", "72"]
 var keysbatch4 = ["74", "75", "76", "186", "13", "16", "90", "88", "67", "86"]
 var keysbatch5 = ["66", "78", "77", "188", "190", "191", "17", "32", "18"]
 if (request.message.keypress)
 {
   //  Somebody has pressed a key (voted).  Notify everybody else
   db.getCounter(request.message.keypress).then((counter) => {
     pubnub.signal({
       "channel": "functionsdemo-counter-result",
       "message": {"k":request.message.keypress, "c":++counter}
     }).then((signalResponse) => {
       //  Signal Sent
     });
   });
   //  And increment the counter - THE IMPORTANT BIT :)
   db.incrCounter(request.message.keypress, 1);
 }
 else
 {
   //  A new client has requested the initial state of all counters
   //  10 KV counter lookups per invocation for efficiency
   var batch = keysbatch1
   if (request.message.batch == 2) {batch = keysbatch2}
   if (request.message.batch == 3) {batch = keysbatch3}
   if (request.message.batch == 4) {batch = keysbatch4}
   if (request.message.batch == 5) {batch = keysbatch5}
   batch.forEach((keyElement) => {
     db.getCounter(keyElement).then(function(counter) {
       if (counter > 0)
       {
         pubnub.publish({
           "channel": "functionsdemo-counter-result",
           "message": {"k":keyElement, "c":counter}
         }).then((messageResponse) => {
           //  Message Sent
         });
       }
     });
   });
 }
 return request.ok();
};
Enter fullscreen mode Exit fullscreen mode

Recent Updates to PubNub Functions

PubNub has recently implemented a slew of enhancements to our Functions to make them easier to use, with additional capabilities:

  • A Functions API makes it easier to deploy & manage new and existing Functions

  • We have enhanced our in-browser editor to allow on-the-fly editing

  • We now retain a history of previous deployments, so you can easily roll back changes if something is not working.

  • We have added new built-in libraries to support UUID, JWT, and JSONPath Plus.

Sign up for a free PubNub account to start exploring Functions today, or check out our Functions documentation for more information.

For fun… If you’re anything like me, this reminds you of the classic Numberphile video about YouTube views freezing at 301… here’s the link: https://youtu.be/oIkhgagvrjI

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

The foundation of our platform is the industry's largest and most scalable real-time edge messaging network. With over 15 points-of-presence worldwide supporting 800 million monthly active users, and 99.999% reliability, you'll never have to worry about outages, concurrency limits, or any latency issues caused by traffic spikes.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Sign up for a PubNub account for immediate access to PubNub keys for free

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more