The async/await post we promised

Great user experience on the web comes from being able to provide users with exactly what they want in the most seamless way possible. Behind the scenes, some user actions may take more time to process than others. For example, showing or hiding an HTML element is a quick operation whereas making an XHR request to get data from an external API is a slower operation. JavaScript provides us with a way to handle them all without giving up that instant delight users naturally expect.

In this article, we’ll describe how JavaScript executes asynchronous operations and review different ways to write asynchronous code from Callbacks to Promises and explain what’s best and why. Most importantly, we’ll use the modern and recommended way to write asynchronous JavaScript to instantiate and use Ably’s JavaScript SDKs.

Jump to "Using the promise-based version of the ably-js SDK"

If you are new to Ably, here's a quick summary - Ably provides APIs to add realtime messaging functionality to your applications. It is based on the Publish/Subscribe messaging pattern and operates mostly on the WebSockets protocol. You can plug in the Ably SDK and start publishing messages in realtime to millions of devices. Sign up for a free account to explore all the platform's features.

The inner workings of JavaScript

JavaScript is a single-threaded programming language. It is predominantly used on the web or in the form of NodeJS in the backend.

If we focus on the frontend, JavaScript-based applications run in a web browser. The actual execution of the JavaScript code is done by a JavaScript engine, which usually comes in-built with every browser. For example, Google Chrome comes with the popular V8 engine (which is also the engine running NodeJS), Mozilla Firefox comes with the SpiderMonkey engine, Microsoft Edge comes with the Chromium engine, and so on.

Being single-threaded means that JavaScript can only do one thing at a time and sequentially execute statements in a given piece of code. When the code takes longer to execute, like waiting for some data to return from an external API, the application would essentially halt at that point and the end-user would end up seeing an unresponsive screen.

But, this doesn’t usually happen, does it?

The everyday working of frontend JavaScript is made possible not just by the JavaScript engine provided by the web browser but with a supplement of three key things:

i) a myriad of Web APIs, we'll refer to these as Browser APIs to avoid confusing them with external APIs

ii) the Message Queue

iii) the Event Loop

Together these elements allow JavaScript to run asynchronous functions that can continue execution without needing to wait for things that take time.

Let’s look at how these language and browser features work together.

A whirlwind tour of asynchronous JavaScript

In a nutshell, the working of asynchronous JavaScript code can be visualized as shown in the diagram below.

The JavaScript engine has a memory heap and a call stack. The memory heap allocates memory for the data in the code and updates the values as per the logic. The call stack is a last in, first out (LIFO) data structure that keeps track of the statement to be executed next to run the code in sequential order.

What happens when things are slow? Let’s say the call stack encounters a setTimeout() function. Let's see how the execution of this statement proceeds in the above flow.

First, we can refer to the only thread that JavaScript has as the "main thread". In case of a setTimeout(), the main thread will kick off the execution of this statement by calling the function from the Browser APIs but not wait until the execution is complete.

When the Browser finishes executing the setTimeout() function, it returns the results. The tricky part, however, is getting these results back to the main thread and showing them in the application.

These results don't immediately get inserted into the call stack because that would disturb the flow of execution. Instead, it inserts the results at the end of the Message Queue. The event loop will then follow a process to decide the best time to pick this up and insert it into the call stack.

The best resource I’ve come across to understand the JavaScript event loop is this amazing talk by Philip Roberts - What the heck is the event loop anyway?. While I’ve summarized the explanation below I’d still recommend giving it a watch.

The Event loop is essentially an infinite while loop (hence the name) that continuously checks for two things:

i) if the call stack is empty

ii) if there are any elements in the Message Queue

When both these conditions become true, the event loop picks up the first element in the queue and puts it on the call stack for the main thread to execute it.

The interesting thing to consider here is how we can let the runtime know that a certain statement depends on an external resource (where the processing is being done somewhere else) and may take time to return. We want the execution to continue, not pause while waiting on that external function to complete. Let's explore this next.

From Callbacks to Promises to Async/Await

We can think of any asynchronous operation we perform in JavaScript as an API call. This call is done either to an in-built API provided by the browser, for example, setTimeout(), or to an API from a third-party provider, for example ably.channel.publish(). In fact, this call can also be done just to another function that's part of the same application but let's assume it is an external function for a better understanding. I've linked some code examples of native async functions in the 'Further reading' section at the end.

The implementation of an async function provided by the API needs to have a way to tell the main thread what needs to be done when it has finished executing the time taking task.

This can be done in one of the following three ways:

i) Callbacks

ii) Promises with .then syntax

iii) Promises with async/await syntax

Let’s explore them one by one.

Option 1 - Async JavaScript with callbacks

A callback is a function that is passed to another function as a parameter. When calling the async function initially, we provide it with a callback function as one of the parameters. When the async function finishes execution, it calls that callback function, along with the results of the execution as arguments. At this point, the callback function is placed on the Message Queue and will eventually be picked up by the event loop and dropped into the call stack for the main thread to execute it.

Let’s take a look at an example with the asynchronous channel publish function provided by Ably’s JavaScript SDK:

/* Code Snippet 1 */
import * as Ably from "ably";
const client = new Ably.Realtime({ authUrl: "/auth", clientId: "bob" });
const channel = client.channels.get("general-chat");

/* function reference:
publish(String name, Object data, callback(**ErrorInfo** err))
*/

channel.publish("new-chat-msg", "Hey there! What is up?", (error) => {
  if (error) throw error;
  console.log("Published successfully");
});

As you can see, the last (optional) parameter in the publish function expects a callback function.

From the Ably SDK (i.e. the async function provider) side of things, when the publish function is called, it executes that function logic. When it’s done, it calls the callback function and passes it some data if it's applicable. This would look something like so:

/* Code Snippet 2 */
class RealtimeChannel {
  publish(messages, callback) {
    /* do some stuff to execute the async operation */

    callback(error, result);
  }
}

As explained before, this callback function will be put at the end of the Message Queue. This will be picked up by the event loop and put onto the call stack which is when it’ll be executed by the main thread. At this point, it'll print the success message to the console depending on the value of the error parameter passed to it.

Ok, that's all well and good. We've understood a way to write asynchronous JavaScript functions, so why even consider other options?

Callbacks are a simple concept and work well for standalone asynchronous operations. However, they can quickly get tedious to write and manage if they have dependencies on each other. For example, consider a scenario where you need to do certain async things sequentially, using the data from one task in the other, say:

i) enter presence on a channel

ii) get some historical messages

iii) publish a new message on the channel with the first message retrieved from history

The callback-based implementation for this scenario will look as follows:

/* Code Snippet 3 */
import * as Ably from "ably";
const realtime = new Ably.Realtime({ authUrl: "/auth", clientId: "bob" });
const channel = realtime.channels.get("general-chat");

/* function references:
- enter(Object data, callback(ErrorInfo err))
- history(Object options, callback(ErrorInfo err, PaginatedResult<Message> resultPage))
- publish(String name, Object data, callback(**ErrorInfo** err))
*/

// step 1 - enter presence
channel.presence.enter("my status", (error) => {
  if (error) throw error;
  console.log("Client has successfully entered presence");

  // step 2 - get historical messages after presence enter
  channel.history((error, messagesPage) => {
    if (error) throw error;
    messagesPage.items.forEach((item) => console.log(item.data));
    let firstHistoryMessage = messagesPage.items[0].data;

    // step 3 - publish a new message after get history
    channel.publish("new-chat-msg", `Hey there! What is up?, my first history msg was ${firstHistoryMessage}`, (error) => {
      if (error) throw error;
      console.log("Published successfully");
    });
  });
});

While this is an accurate implementation and will work perfectly fine, it already looks messy and difficult to manage due to the multiple nested callbacks. This is commonly referred to as Callback Hell because debugging or maintaining anything which looks like this would be a daunting task. And, this is exactly the reason we have other, more modern ways of writing asynchronous JavaScript functions. Let's explore these next.

Option 2 - Async JavaScript with Promises (.then syntax)

The second option introduces a concept called ‘Promises’. Instead of calling a callback function, the API side implementation of the asynchronous function will create and return a "promise" to the requesting client that wants to execute the async function.

A Promise can have one of the following three states:

i) Pending - meaning we’ve started an async operation but its execution has not completed yet

ii) Resolved (or Fulfilled) - meaning we started an async task and it has finished successfully

iii) Rejected - meaning we started an async task but it finished unsuccessfully, in most cases with a specific error that will be returned to the client

Let's consider a Promise based async operation and again see both sides of the coin i.e. what happens on the API side implementation as well as the requesting client side. This time, let's first take a look at the API side of things:

/* Code Snippet 4 */
class RealtimeChannel {
  publish = (messages) => {
    return new Promise((resolve, reject) => {
      /*
      do some stuff to execute the async operation
      */
      error ? reject(error) : resolve(result);
    });
  };
}

The promise executor in the API calls the resolve() function if the async task was executed as expected, along with the results of the operation. However, if there was some issue with the execution it calls the reject() function.

A requesting client can consume such a Promise using a .then() function attached to the async function call. The .then() code block is similar to a callback code block and will be executed when the async task has finished executing. We can also attach a .catch() to the .then() block to catch any errors that may have occurred during the execution of the async task.

In terms of the explanation above, the .then() block will be executed when the promise executor in the API calls the resolve() function and the .catch() block will be executed when the API calls the reject() function.

At the time of writing this article, the Ably JS SDK doesn't provide promises by default. To be able to use the promise version of the SDK, we need to use new Ably.Realtime.Promise() constructor when instantiating the library.

Let's now see how our example will work on the client side

/* Code Snippet 5 */
import * as Ably from "ably";
const realtime = new Ably.Realtime.Promise({ authUrl: "/auth", clientId: "bob" });
const channel = realtime.channels.get("general-chat");
/* function reference:
publish(String name, Object data): Promise<void>
*/

channel
  .publish("new-chat-msg", "Hey there! What is up?")
  .then(() => {
    console.log("Published successfully");
  })
  .catch((error) => {
    console.log("There was an error while publishing: " + error);
  });

If you compare the above with the "Code Snippet 1", it seems more logical in the sense that we can understand that certain statements will execute after certain other statements due to the literal English meaning of the word 'then'.

The real advantage however can be seen if we need to perform multiple asynchronous tasks sequentially, in some cases using the data returned in the previous async task.

Let's consider the same scenario as we did in the callbacks version:

i) enter presence on a channel

ii) get some historical messages

iii) publish a new message on the channel with the first message retrieved from history

Let's see how this will look like using Promises with a .then syntax.

/* Code Snippet 6 */
import * as Ably from "ably";
const realtime = new Ably.Realtime.Promise({ authUrl: "/auth", clientId: "bob" });
const channel = realtime.channels.get("general-chat");

/* function references:
- enter(Object data): Promise<void>
- history(Object options): Promise<PaginatedResult<Message>>
- publish(String name, Object data): Promise<void>
*/

// step 1 - enter presence
channel.presence
  .enter("my status")
  .then(() => {
    // this block executes after the presence enter is done
    console.log("Client has successfully entered presence");
    //step 2 - get historical messages
    return channel.history();
  })
  .then((messagesPage) => {
    // this block executes after the channel history is retrieved
    messagesPage.items.forEach((item) => console.log(item.data));
    let firstHistoryMessage = messagesPage.items[0].data;
    //step 3 - publish a new message
    channel.publish("new-chat-msg", `Hey there! What is up?, my first history msg was ${firstHistoryMessage}`);
  })
  .then(() => {
    // this block executes after the message publish is done
    console.log("Published successfully");
  })
  .catch((error) => {
    // this block executes if there's an error in any of the blocks in this Promise chain
    console.log("We have an error:", error);
  });

As you can see, the Promise version with a .then() syntax reduces the complexity and the level of indentation when compared to the callbacks approach. This helps us understand and maintain the code much easily.

However, as you can see with this option, we need to wrap each execution step in a function call and return the results to the next .then(). Although a huge improvement from the callbacks syntax, it seems like it could still get verbose pretty quickly. This is what the async/await syntax helps us with. Let's understand that next.

Option 3 - Async JavaScript with Promises (async/await syntax)

This third option is just another version of the second option. There's no change on the API side of things. The API would still create a 'Promise' and either resolve() or reject() it after the async task is executed.

The way we consume it on the front end, however, is different (and better!). The async/await provides syntactic sugar to reduce the complexity in chained async tasks. Let's take a look at how the "Code Snippet 6" above would look like if we use async/await instead of .then().

/* Code Snippet 7 */
import * as Ably from "ably";
const realtime = new Ably.Realtime.Promise({ authUrl: "/auth", clientId: "bob" });
const channel = realtime.channels.get("general-chat");

/* function references:
- enter(Object data): Promise<void>
- history(Object options): Promise<PaginatedResult<Message>>
- publish(String name, Object data): Promise<void>
*/

async function main() {
  try {
    // step 1 - enter presence
    await channel.presence.enter("my status");
    console.log("Client has successfully entered presence");

    //step 2 - get historical messages
    let messagesPage = await channel.history();
    console.log("Retrieved history successfully");
    messagesPage.items.forEach((item) => console.log(item.data));
    let firstHistoryMessage = messagesPage.items[0].data;

    //step 3 - publish a new message
    await channel.publish("new-chat-msg", `Hey there! What is up?, my first history msg was ${firstHistoryMessage}`);
    console.log("Published successfully");
  } catch (error) {
    console.log("We have an error:", error);
  }
}

main();

As you may have observed, we've wrapped all our statements in a function this time. This is because the async/await syntax can only be used in functions starting with the async keyword. Such an async function can then contain zero or more await statements.

Statements that begin with the keyword await are asynchronous functions. Similar to the previous option with Promises using the .then() syntax, these statements get returned via the Message Queue when the underlying Promise provided by the API calls either a reject() or a resolve() function.

Concurrency of independent asynchronous statements

Given that the async/await approach looks a lot like writing synchronous statements, it is a common mistake to make independent code unnecessarily wait for the previous tasks to finish instead of having them execute concurrently (in parallel). For example, in the code examples we saw in the previous sections, if entering the client in the presence set, retrieving history and publishing a new message had no dependencies on each other, we can easily do these things in parallel instead of sequentially.

This can be done using the Promise.all() function as shown below:

/* Code Snippet 8 */
import * as Ably from "ably";
const realtime = new Ably.Realtime.Promise({ authUrl: "/auth", clientId: "bob" });
const channel = realtime.channels.get("general-chat");

/* function references:
- enter(Object data): Promise<void>
- history(Object options): Promise<PaginatedResult<Message>>
- publish(String name, Object data): Promise<void>
*/

async function main() {
  try {
    const enterPresence = channel.presence.enter("my status");
    const getHistoryMessages = channel.history();
    const publishMessage = channel.publish("new-chat-msg", "Hey there! What is up?");

    // perform all three async functions concurrently
    const values = await Promise.all([enterPresence, getHistoryMessages, publishMessage]);

    console.log("Client has successfully entered presence");
    console.log("Retrieved history successfully");
    console.log("Published successfully");
    let messagesPage = values[1];
    messagesPage.items.forEach((item) => console.log(`History message: ${item.data}`));
  } catch (error) {
    console.log("We have an error:", JSON.stringify(error));
  }
}

main();

/*
Note the publish function doesn't use any data returned 
by the History API in this case as we are considering the three functions
to be executed independently of each other.
*/

The case of asynchronous event listeners

By now, we have a good understanding that Promises with either the .then() or the async/await syntax are a big improvement over callbacks. But what happens in the case of asynchronous event listeners where you are constantly listening for some updates. For example, in case of a setInterval() from the inbuilt Browser APIs or ably.channel.subscribe() from the Ably API?

Promises are great for one off execution of an async task that either resolves or rejects based on some logic. However, in the case of a subscription, we'd need the resolution to happen multiple times i.e. whenever there's a new message to be pushed from the API to the listening client. Promises unfortunately cannot do that and can resolve only once. So, for active listeners that return data repeatedly, it's better to stick with callbacks.

Using the promise-based version of the ably-js SDK

As per the examples we've been seeing so far, it is clear that Ably’s JavaScript SDK provides a promisified version. This means we can consume the asynchronous functions (except for listeners) using the async/await syntax. In the devrel team, we've been using the async style API in our latest demo - the Fully Featured Scalable Chat app.

At the time of writing this article, the default way to consume async functions using the Ably JS SDK is using callbacks, but in this section, we’ll take a look at a few key functions where we consume the promisified API using the async/await syntax.

1. Importing and instantiating the Ably Realtime or Rest instances:

/* Code Snippet 9 */

import * as Ably from "ably";

//before - instantiating the Ably SDKs, callback version

const client = new Ably.Realtime(options);

const client = new Ably.Rest(options);

//now - instantiating the Ably SDKs, Promise version

const client = new Ably.Realtime.Promise(options);

const client = new Ably.Rest.Promise(options);

2. Attaching to a channel

/* Code Snippet 10 */

//before - attaching to a channel, callback version
client.channel.attach(() => {
  console.log("channel attached");
});

//now - attaching to a channel, promise with async/await version
async function attachChannel() {
  await client.channel.attach();
}
attachChannel();

3. Retrieving and updating presence status on a channel

/* Code Snippet 11 */

//before - presence functions, callback version
channel.presence.get((err, members) => {
  console.log("Presence members are: ", members);
});
channel.presence.enter("my status", () => {
  console.log("Client entered presence set");
});
channel.presence.update("new status", () => {
  console.log("Client presence status updated");
});
channel.presence.leave(() => {
  console.log("Client left presence set");
});

//now - presence functions, promise with async/await version
async function ablyPresenceStuff() {
  await channel.presence.enter("my status");
  await channel.presence.update("new status");
  await channel.presence.leave();
}

ablyPresenceStuff();

/*
Please note - the above code snippets are slightly 
different in terms of how they'd run.

The callback version concurrently executes all four functions,
whereas the async/await version executes all the statements 
sequentially.

Please scroll back up and read 
'**Concurrency of independent asynchronous statements'** 
if you are interested to learn more about this behaviour.
*/

3. Publishing messages

/* Code Snippet 12 */

//before - publishing messages, callback version
channel.publish("my event", "Hey, this is event data", () => {
  console.log("Publish done");
});

//now - publishing messages, Promise with async/await version
async function publishToAbly() {
  await channel.publish("my event", "Hey, this is event data");
  console.log("Publish done");
}
publishToAbly();

4. Subscribing to messages

/* Code Snippet 13 */

//before - subscribing to messages, callback version
channel.subscribe((msg) => {
  console.log("New message received", msg.data);
});

//now - subscribing to messages, Promise with async/await version
channel.subscribe((msg) => {
  console.log("New message received", msg.data);
});

/*
Please note, there's no change here. As described in the previous section 
Promises cannot be used with listeners which need be triggered multiple times.

Hence, in this case, we stick to callbacks.
*/

5. Retrieving historical messages

/* Code Snippet 14 */

//before - history API, callback version
channel.history({ limit: 25 }, (err, resultPage) => {
  resultPage.items.forEach((item) => console.log(item.data));
});

//now - history API, callback version
async function getAblyHistory() {
  const resultPage = await channel.history({ limit: 25 });
  resultPage.items.forEach((item) => console.log(item.data));
}
getAblyHistory();

We are moving to Promise by default

In the upcoming versions of the JS SDK, you won't need to instantiate the promise version explicitly with Ably.Realtime.Promise(options). Instead, if you do Ably.Realtime(options). It'll use the promisified API by default.

If you want to stick to using the callbacks version at that point, you can explicitly instantiate the Callbacks constructor with Ably.Realtime.Callbacks(options) and continue using callbacks as default.

References and further reading

• Sample code snippets showing callbacks and promises using native functions.
• Working examples of using the async style with the Ably APIs:
  • Ably React Hooks NPM package
  • A fully featured chat demo
• The Ably JavaScript SDK repository
• Realtime use-case demos on Ably Labs