The saga of async JavaScript: Callbacks

Intro

The heart of modern JavaScript application lies in its interactivity. Buttons are being clicked, mouse is moving while you are dragging your image to upload a new avatar, AJAX requests is going out to get your favourite feed - all of these can happen while that cute cat video is preloading along with its comments' thread. Thanks to JavaScript being asynchronous, we can leverage those events while keeping the app responsive. Without knowing how to orchestrate those temporal dependencies well, the complexity in our code will quickly get out of hand.

So what's next?

In this series of articles we will try to explore different patterns that help us write asynchronous JavaScript. Most modern libraries and frameworks use at least one of them. And most developers have no idea about their strengths and weaknesses. We will take a look at why those patterns exist. We're gonna explore which problems do they solve, and which they don't. In the end of the series, hopefully, you are going to be familiar enough to know when to use each one of them and reason about this decision. Take your seats, gentlemen, and today we will have a tough talk about callbacks.

Callbacks? We already know those

I get it. It will be a decent challenge to find a JS developer who doesn't have at least a rough idea of what callbacks are. We all know how they look like. You pass the function as an argument and it is called after a certain action is completed. We are going to have a little practice with callbacks before going deeper into their flaws. Consider this simple expression of a callback in an async fashion.

function mortalCombatGreet () {
    console.log('Choose your destiny')
}
setTimeout(mortalCombatGreet, 1000)

How is that working?

Functions in JavaScript are first-class citizens which basically means that they can do everything others can do. You can assign them to variables, pass as arguments, return from the functions. In the example above we pass our callback function to a built-in API, but it could be any other API or library. The description of this code would be: "create a timer with a callback, execute a callback in 1000ms". When dealing with callbacks, there is some code which will execute immediately, and some code which will be run later. We essentially divide our program into two parts - the first part is everything outside of a callback including setTimeout call, and the other one is our callback itself. There is a clear notion of "now" and "later".

More callbacks to the God of callbacks

Now let's consider an example that is a bit more sophisticated. We will try to read three files in a sequential order using callbacks. Assume that readFile function is some function that takes time to complete.

readFile('first file', function (firstFileContents) {
    console.log(firstFileContents)

    readFile('second file', function(secondFileContents) {
        console.log(secondFileContents)

        readFile('third file', function(thirdFileContents) {
            console.log(thirdFileContents)
        }
    })
})

Here we tried to express a temporal dependency using callbacks. Pretty straightforward and common async operation to use. We can clearly see that a second file read need to wait for a first read to finish. Same relationship exists between third and second reads.

Temporal dependency === nesting?

You could notice that an expression of each single temporal dependency is achieved through nesting callbacks inside of each other. And you could also imagine this going really big and crazy in some complex parts of application logic. This is often referred to as Callback Hell or Pyramid Of Doom in Javascript community (did you really think I attached that pyramid image by accident?). When it comes to this concept, people mostly complain about nesting and indentation. But is it all about how the code looks? I could immediately start proving you, that code formatting is not the fundamental problem of the callbacks.

function readFirst (cb) {
    readFile('first file', function (fileContents) {
       console.log(fileContents)
       cb()
    }) 
}

function readSecond (cb) {
    readFile('second file', function (fileContents) {
       console.log(fileContents)
       cb()
    })
}

function readThird () {
    readFile('third file', function (fileContents) {
       console.log(fileContents)
    })
}

readFirst(function () {
    readSecond(readThird)
})

This code definitely doesn't obviously suffer from identation and nesting problems, does it? This is what is often called as continuation passing style. We could go on with refactoring and eventually come up with something that would not look like a callback hell to an average javascript developer at all. This is where the most serious problem lies. This is where our understanding needs to be redefined, because this code is as susceptible to callback hell as previous one.

Inversion of control

Notable feature of callbacks is that the part of our code is executed by a third party. We can't exactly know when and how our code will be executed. When we loose control on our code and pass it to someone else, the Inversion Of Control happens. There are many definitions of the Inversion of Control term on the internet, but for our case that's pretty much it.

Trust issue

In our first example we passed our code to setTimeout utility. There is nothing wrong with it, right? We all use timers! Timer API is a well known and established feature. Nobody is thinking to themselves "oh, wait, maybe it will not execute my code right in time, or it will not even execute it at all". We can trust it. And that's the main point. What if we pass our callback to some external library that is not a part of standard API? What if, for example, we rely on something else to execute the code which charges our client's credit card?

fancyAsyncFunctionFromLibrary(function () {
     chargeCreditCard()
})

When you are passing callback you are trusting that it will be called:

• not too many times
• not too few times
• not too early
• not too late
• with no lost context
• with correct arguments

What happens if this trust falls apart? Can you really cover all of those cases with workarounds in all of the places where you use callbacks? I would assert to you, that if you have callbacks in your application and you don't have those cases covered, then your app potentially has as many bugs as there are callbacks in it.

Going natural

Without diving deep into the science, we can safely say that our brain is essentially single threaded. We can think about only one single thing at a time at our highest level of cognition. We also like to think about stuff in a sequential fashion. Take a look at how you are planning your day. You allocate your time for a single task and complete each of them sequentially one by one: take shower, have a breakfast, make a call to the boss, participate in a meeting, etc. But it often doesn't play that nice, does it? Usually, at least a couple of times, you will get interrupted. Your mom calls while you are on a meeting, delivery guy rings a door when you are trying to wrap your head around a bug. Thankfully, when this happens, you are not going like: "ok, that's awful, I am going to my bed and start tomorrow from scratch". From this perspective, our brain is a lot like a JavaScript engine. It can be interrupted with an event, choose to respond to it and then continue running.

Where the bugs happen

If that's how our brains work and how we handle tasks, we are most likely to code in the same way... naturally. But language engines, as well as JavaScript, often don't work the way that is immediately obvious to us. Every time you are not thinking about the code in a different way than a compiler, there is a potential bug in your program. Thankfully, we can both train ourselves to think more like a compiler and invent new patterns and syntax that both fit our mindsets and computer needs. That's why it is extremely important to understand how all of those patterns, frameworks and libraries work internally. And it is not enough to just know the API and a general definition.

Reasoning about callbacks

Remember me saying that the only way to handle temporal dependency using callbacks is through nesting? Consider the next pseudo code which will express how we would like to, at least in my opinion, reason about async operations:

start someBigOperation1
do stuff
pause

start someBigOperation2
do stuff
pause

resume someBigOperation1
do more stuff
pause

resume someBigOperation2
do more stuff
finish

resume someBigOperation1
do more stuff
finish

Would be great to have this sort of syntax to handle async operations in Javascript, huh? We are doing one step at a time, and our brain linearly progresses through the code. Doesn't look like callbacks at all... but what if it did?

start someBigOperation1
do stuff
pause
    resume someBigOperation1
    do more stuff
    pause
        resume someBigOperation1
        do more stuff
        finish

start someBigOperation2
do stuff
pause
    resume someBigOperation2
    do more stuff
    finish

Whether you are doing it with function expressions or with function calls, that doesn't matter. The code is not looking sequential anymore, you can't instantly figure out the order of operations and you are forced to jump all over the code to get the idea. The async flows in our apps can become really complex, and I doubt that there is a developer in your team who understands all of them from start to end. You can understand step one, two and three, but it quickly becomes a thing beyond our capacity as soon as it goes like this: "start step one, two and three, and as soon as step two is finished, cancel step three and retry step two, then start step four". God bless you if those steps are callbacks jumping around the files in your projects. This is the case when your brain is fundamentally unable to reason about the program anymore. Callbacks force us to express in a fashion that contradicts the way our brains are used to plan things. Callbacks alone don't have right tools to let us write sequentially looking async code. Seems like we need a better pattern.

What doesn't fix the problems

Multiple callbacks

readFile(function (fileContents) {
    console.log('Success! ', fileContents)
}, function (error) {
    console.log('Error! ', error)
})

There is way now for the utility to notify us about an error using a second callback. Looks good. But guess what? Now we are trusting the utility to execute two callbacks properly and basically you end up with 2x the number of potential bugs that you need to cover in your code. Ouch!

Error first style of callbacks

readFile(function (error, fileContents) {
    if (error) {
        console.log('Error! ', error)
    } else {
        console.log('Success! ', fileContents)
    }
})

Two callbacks are too crazy, so let's get back to just one. We are going to reserve the first parameter for an error. It definitely removes the concerns about calling two callbacks, but what happens if utility messes up the order of arguments? What if it calls callback twice - once with error, and then without it? What if it calls the callback both with error and success arguments? The same trust issues arise with couple of new ones. Still doesn't look like a solution at all.

Outro

You should now have a pretty good understanding of callbacks and be able to articulate about their drawbacks. It is clear that callbacks alone will not help you to solve each and every problem in your async code. Hopefully, next time when you hear about Callback Hell, you will be confident about what it truly means. It is about design limitations which can't be solved no matter how much you refactor your code. The Ideal Pattern should provide us an ability to write async code that looks like a synchronous one. That sounds fantastic, but it should be possible, right? There are still plenty of approaches to take a look at and in the next article we will talk about Thunks and see how they make asynchronous programming much easier.