When Nesting Promises is Correct

Intro

By now, promises are well-established in the JS ecosystem, not only being officially specified in ECMAScript, but even having a first-class syntactic sugar in the form of async functions.

When learning promises, many JS developers are told that a major advantage of promise chaining is that it keeps the code "flat", avoiding the pyramid of doom of nested callbacks. While this is partly true, it also puts undue emphasis on code appearance, running the risk of missing the point.

True "callback hell" is less about indentation – in fact, by naming callback functions and factoring them out to the top level, one can often flatten out async code without the need for promises. Instead, callback hell is when we lose the composable vanilla function API (pass in data, receive result), where returned values can be bound to variables, aggregated in collections, passed to other functions, and combined in first-class ways.

All of this preamble is to give context to the following statement: nesting promises is often an antipattern, but not always. In fact, there is a common situation in which a little nesting can make perfect sense, though there exist several alternatives. This short article will demonstrate a common scoping issue with promises and multiple solutions for that issue.

The Setup

For these examples, we will imagine that the function getPuppyById is an AJAX method returning some data via a promise. Puppies will be objects with a bestFriend foreign key to another puppy:

{
    id: 4,               // this puppy's id
    name: 'Mr. Wiggles', // this puppy's name
    bestFriend: 17       // id of this puppy's best friend (another puppy)
}

If we wish to fetch puppy #1's best friend's name, we can chain calls to getPuppyById:

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const friendNameP = getPuppyById(1) // first fetch .then(pup1 => getPuppyById(pup1.bestFriend)) // second fetch .then(friend => friend.name) // data transformation friendNameP // our goal, a promise for the best friend name .then(name => console.log('friend name', name)) .catch(e => console.error(e))

This works just fine when our early results are just discardable steps towards our desired final result.

The Problem

However, what if we wanted to produce a promise for both puppies' names – the original and the friend? Because the callback passed to then introduces a function scope, the first puppy may no longer be in scope further down the chain.

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const twoPuppyNamesP = getPuppyById(1) // first fetch .then(pup1 => getPuppyById(pup1.bestFriend)) // second fetch .then(friend => { return [pup1.name, friend.name] // ERROR – pup1 no longer in scope! }) // DO NOT EDIT BELOW twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

There are multiple ways to solve this, which we will examine in a moment. Before we do so, go ahead and fix the above code snippet using whatever technique you may prefer. Only edit the top half of the snippet; you are trying to make twoPuppyNamesP fulfill its promise (hah) of delivering both puppies.

Solutions

Library-Specific: Bluebird bind

Before promises became official in ES2015, third-party implementations like Bluebird were popular. Bluebird is still used by some codebases for its speed and wide array of utility methods.

Though it breaks section 2.2.5 of the A+ promise spec to do so, Bluebird includes a special feature in which you can set the this value of a promise chain – providing a shared mutable namespace in which to save intermediate results. The specific method is named bind.

.bind also has a useful side purpose - promise handlers don't need to share a function to use shared state

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const P = require('bluebird') const toBluebird = p => P.resolve(p) const twoPuppyNamesP = toBluebird(getPuppyById(1)) // first fetch .bind({}) // set 'this' for chain .then(function (pup1) { // arrows don't have 'this' this.pup1 = pup1 // saving state for later return getPuppyById(pup1.bestFriend) // second fetch }) .then(function (friend) { return [this.pup1.name, friend.name] // accessing 'pup1' in shared state }) twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

While this works, it has significant drawbacks:

• it complicates the promise chain with spec-breaking features
• it requires using function functions to access this
• it is non-portable knowledge tied to a specific library

A+-Compliant, ECMA-Approved: Promise.all

If only we could pass multiple values down through a promise chain – even when one of those values is a pending promise, whose value we wish to access further down the chain.

Of course, we do not need to wish for such a feature, as it is available via the Promise.all static method. By returning an array of both synchronous values and promise values, wrapped in a call to all, we get access to an array of synchronous values in the next then.

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const twoPuppyNamesP = getPuppyById(1) // first fetch .then(pup1 => { const friendP = getPuppyById(pup1.bestFriend) // second fetch return Promise.all([pup1, friendP]) // collect both results }) .then(([pup1, friend]) => { // array destructuring return [pup1.name, friend.name] // data transformation }) twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

Even though the array passed to .all has a mix of normal and promise values, the resulting overall promise is for an array of normal values.

This strategy will work in any setting that supports ES2015, and is thus much more portable than the Bluebird bind trick. Unfortunately, it too has cons:

• more verbose return lines
• more complex function parameters and destructuring
• as the chain grows, passing down multiple results does not scale well
• overall, a lot of redundant "plumbing" of early values through the chain

Controlled State, Shared Scope

We now come to one of the most common and viable techniques for sharing state through a promise chain – use a mutable or reassignable variable(s) in a higher scope. As each handler in a then chain is invoked, it will set and/or read the values of a shared let binding or the properties of a shared object.

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) let pup1 // shared binding const twoPuppyNamesP = getPuppyById(1) // first fetch .then(gotPup1 => { pup1 = gotPup1 // save state return getPuppyById(pup1.bestFriend) // second fetch }) .then(friend => { return [pup1.name, friend.name] // data transformation }) twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

This may seem "illegal" considering how we normally consider async code to work, but in fact it is guaranteed to work as expected as later callbacks in a then chain can only be invoked after earlier callbacks. So the usage of pup1 in the second then will work because pup1 is guaranteed to have been assigned in the callback of the previous then.

This has some distinct advantages:

• it is relatively clear even for people without advanced knowledge of promises
• it is setting-agnostic
• it is relatively light on syntax
• the chain remains flat, reducing mental load

As always, there are still tradeoffs to consider, however.

• shared mutable state is risky; care should be taken to only allow the promise chain to read or modify these variables
  • reading outside the chain is not guaranteed to work due to indeterminate timing
  • writing outside the chain can break guarantees within the chain
• we now need two versions of the variable name – a parameter name like gotPup1 and a shared state variable like pup1 – to avoid shadowing

If the promise chain is itself contained within a short function scope, disciplined use of shared state in a local setting can be concise and easy way to solve the issue of passing information down the chain.

The Punchline: Nested Promises

This article opened with the promise (hah) of showing a situation in which a small bit of nesting can be a valid and useful technique. The key point is that with a nested chain, an inner then still has scope access to the results from an outer then.

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const twoPuppyNamesP = getPuppyById(1) // first fetch .then(pup1 => getPuppyById(pup1.bestFriend) // second fetch .then(friend => [pup1.name, friend.name]) // nested then ) twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

In such cases, it is crucial to remember to return the nested promise chain to the parent promise chain. In the example above we use the implicit return of an arrow function to accomplish this, but it is a common error to forget the return keyword when in a bracket-enclosed function body.

The biggest advantage that the above pattern has over an outer-scope variable is that it is stateless – there is no explicit mutation occurring in the visible code, only a declarative sequence of functional transformations.

As always, we can identify some disadvantages:

• this approach does not scale well for passing down each result from many then calls – one quickly returns to the "pyramid of doom" for such cases
• with nesting comes increased mental load in parsing and understanding the logic of the promise chain
• as is often the case with promise chains, it can be especially difficult to decide on a sensible formatting scheme with respect to where .then appears (same line? next line? indented?) and where to position the callback function

Silly Experiment: Formatting Tricks

Speaking of formatting, there is no reason why one cannot format a nested promise chain in a "flat" way, if we allow for piling up of parentheses:

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const twoPuppyNamesP = getPuppyById(1) // first fetch .then(pup1 => getPuppyById(pup1.bestFriend) // second fetch (missing closing paren) .then(friend => [pup1.name, friend.name])) // nested then (extra closing paren) twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

The longer the nested chain, the more we defer closing parens to the last line, where they will pile up like afterthoughts. In a language like Haskell in which function application doesn't use parens, this isn't a problem! But for JavaScript, it gets a little silly. Compare and contrast:

-- Haskell

_then = (>>=) -- renaming for JS readers; can't use 'then' b/c it's a keyword

pupsIO =
    getPuppyById 1
    `_then` \pup1 -> getPuppyById (bestFriend pup1)
    `_then` \pup2 -> getPuppyById (bestFriend pup2)
    `_then` \pup3 -> getPuppyById (bestFriend pup3)
    `_then` \pup4 -> getPuppyById (bestFriend pup4)
    `_then` \pup5 -> pure [pup1, pup2, pup3, pup4, pup5]

// JavaScript

const pupsP =
    getPuppyById(1)
    .then(pup1 => getPuppyById(pup1.bestFriend)
    .then(pup2 => getPuppyById(pup2.bestFriend)
    .then(pup3 => getPuppyById(pup3.bestFriend)
    .then(pup4 => getPuppyById(pup4.bestFriend)
    .then(pup5 => [pup1, pup2, pup3, pup4, pup5]))))) // lol

The Promised Land: Async/Await

Moving past our promise chain woes, we return to the real issue at hand – promise chains are composed from callback functions, and functions syntactically introduce new scopes. If we didn't have sibling scopes, we could share access to previous results.

Lo and behold, this is one of the problems solved by async functions.

const getPuppyById = (id) => new Promise((res) => { const time = Math.random() * 500 + 500 setTimeout(() => res({ 1: { name: 'Floof', id: 1, bestFriend: 7 }, 7: { name: 'Rex', id: 7, bestFriend: 1 } }[id]), time) }) const getTwoPuppyNamesP = async () => { // a shared async function scope const pup1 = await getPuppyById(1) // first fetch const friend = await getPuppyById(pup1.bestFriend) // second fetch return [pup1.name, friend.name] // data transformation } const twoPuppyNamesP = getTwoPuppyNamesP() // async funcs return promises twoPuppyNamesP // our goal, a promise for the puppy and friend names .then(names => console.log('puppy names', names)) .catch(e => console.error(e))

The advantages are substantial:

• far less noise (no .then calls or callback functions)
• synchronous-looking code with access to previous results in scope

The cost is pretty minimal:

• the await keyword may only be used inside an async function, so we need to wrap our promise code in a function body

Async/await is analogous to Haskell's do-notation, where do is like async and <- is like await:

-- Haskell

twoPuppyNames = do
    pup1   <- getPuppyById 1
    friend <- getPuppyById (bestFriend pup1)
    pure [name pup1, name friend]

One major difference is that async/await in JS is only for promises, whereas Haskell's do notation works with any monad.

Conclusion

With the advent of async/await, programmers are using raw promise chains less often. Async/await has its own subtleties to master, but it neatly solves at least one awkward aspect of promise chains, namely accessing previous async results in a sequence of operations.

As the title to this article suggested, when writing a manual promise chain it is sometimes perfectly valid to use a little local nesting. Doing so keeps multiple results in scope, without needing special library tricks or stateful assignments.

In any case, I hope that these examples will help people learning JS promises to understand them a little better and use them more confidently.

Comments

[aishah]

Great article, helped me understand promises a little more. Unfortunately, I can't see some of the images. It comes up with the error "Unable to load embed. Syntax error found in Preamble. See console for error."

[Gabriel Lebec]

Sorry about that! See dev.to/glebec/comment/14255 – in short, there is currently a problem with Runkit support on Dev.to.

[Gabriel Lebec]

NOTE: on 2020-08-13, github.com/forem/forem/issues/9773 noted that some Runkit embeds are failing to load correctly. This article is affected; apologies to any readers who may come here before that issue is resolved.