Table of Contents
The lead-up
When I was learning JavaScript (about 1-2 years back), my mentor had me go through it step-by-step. So, first I spent some time to get comfortable with callbacks. Then I jumped onto Promises. And then, after months, I started using Async-await.
Due to this reason, I was exposed to a number of flow-control methods and practises that evolved around JavaScript; which I would've missed otherwise - simply because of the fact that I wasn't part of that generation.
Just like how our grandparents complain about how easy our generation has it due to the existence of internet, mobiles, electronic devices, etc. I strongly believe that in the next 2-3 years, we'll complain about how easy the JS-newcomers have it since they do not have to deal with callback-hell and all the other struggles of the "pre-Promise" era. To them, it'll probably just be a textbook paragraph on the history of JavaScript that no one really cares about; except for the compulsory 1 mark question that is asked from it.
When I was in college, I had no idea what 'asynchronous' meant. Coming from the world of C++, PHP, and Java, the word 'asynchronous' was completely alien to me. I had vague understanding of multi-threading in Java and I dreaded it. I've come a long way from there! π
My intention of writing this article is simple.
It is my humble attempt to immortalize the evolution of writing in JavaScript before it's too late and forgotten; in a way that even non-JS people can appreciate it. Even if they don't completely understand the specifics, as they're not familiar with JavaScript constructs, I'm trying to keep it so that they can at least get the general idea.
Yet, if something doesn't make sense, or you want to talk more about it, feel free to reach out.
Events, event-handlers and callbacks.
This is from the pre-historic times.
If you have enough experience, you must've definitely come across event-driven systems - Visual Basic, OnClickListener() in Android, onchange behavior in HTML, etc.
Since node is primarily an event-based runtime environment, all it had initially was events and event-handlers.
Event-handlers are just functions that are triggered once a certain event is fired/emitted. Just like the onChange behaviour in HTML.
Simply put, it means something that occurs after an unknown amount of time, so don't expect immediate results.
For example, "Mom, can I have five dollars?"
Putting my hand out for money is me expecting her to immediately respond by giving me money (synchronous).
Realistically, she will look at me for a moment or two, and then decide to respond when she wants to (asynchronous).
Due to the asynchronous nature of JS, the system wouldn't wait while, say, you get some data from a database (It was really difficult to wrap my head around and get used to this initially).
However, events enable you to put your work on hold when Node.js realises that it is an async task you're performing; and then lets you resume your work when the task has completed and data is available.
In JavaScript, functions can be passed as arguments to other functions and functions can return functions. Such functions are called higher-order functions - similar to how a person who manages other people under him is considered to be at a higher level or position.
Thus, a pattern emerged where a function will be passed as the last parameter to an asynchronous function; called a callback function.
Under the hood, this function would become the event-handler for the concerned event.
Issue with callbacks.
There are hardly any practical applications that may not involve async operations.
The advantage of using Node.js is that time-consuming async operations don't affect the performance of your server. The server won't hold off (or starve) one request till another one is completely processed and its response is sent.
As soon as Node.js realizes that an async operation is to be performed, it'll delegate a worker process to handle the operation and immediately start processing the next request. This gives a terrific boost to the speed of the system.
If your server is getting a lot of requests, and each request requires some async operation (say, database queries), this turns out to be significantly efficient.
However, this efficiency came at a great cost. Writing industry-grade applications with just events, event-handlers and callbacks is not easy.
Callback-hell is the biggest problem with callbacks that leads to decreased code-extensibility, reusability and manageability.
Coming from the object-oriented background of Java, I found it very difficult to get used to writing code involving callbacks - how you have to split the code into a separate function, the callback function.
The struggle was real during that time.
Frustrated by writing asynchronous code with callbacks, developers started finding creative ways to write better, cleaner code.
For example, we used to use async.io at my workplace. It has utility methods like async.series(), async.parallel(), async.waterfall(), etc.
async.waterfall() is the most interesting one according to me. It lets you chain async functions together so that one function's output is the next function's input - kind of like the human centipede but with functions. π
Promises
Promises were introduced in ES6 (2015). Until then, people only had callbacks.
Promises were the next step from callbacks. A major step that brought a revolution in the way we worked with Node.js. Consider it the industrial revolution of JavaScript.
The Promise object represents the eventual completion (or failure) of an asynchronous operation and its resulting value.
βββMDN web docs
A promise is really just a wrapper around callbacks. An ingenious wrapper where we make a shift from using functions for storing the next code to using an object.
The next function to call (the callback), instead of passing it into a function, we attach it to an object - the promise object.
This object is then responsible to pass the callback function as an event handler to the concerned event.
You can instantiate a promise object from any callback based function. Thus, you can always go from a function-based approach to an object-based one.
The significance of this is that your code turns from nested blocks of callbacks to a linear chain of .then-ables.
It is lot more easy to make modifications to your code when it is written in linear sequential manner (the very reason we love synchronous code) than when it is written in nested blocks.
Your code instantly becomes readable, predictable and 200x more manageable.
Read this article for more information on Promises:
Medium
If the Promise object sounded like magic, and you are interested in understanding its internal working, you might be interested in this article.
Co-routines
Generators
Generators were introduced in ES6 (2015) along with promises. But, I believe not many people know about them or use them often.
They are functions that return generator objects.
A generator object is an iterator.
An iterator is anything that implements the iterator protocol.
The iterator protocol says that an object can be called an iterator if it has the next() method that is supposed to do a very specific job; get the next value of iteration/sequence. If you're familiar with Scanner in Java, it's an Iterator (Although it breaks Java design priniciples)
So, a generator object is basically an object that has this
next() methodAnd generator functions are just functions that return generator objects. If you've ever used
xrange() in Python 2.x, that's literally a generator.A very good example for generators will be a Fibonacci generator.
Read the Mozilla docs for more information on generators and iterators.
Also, this in-detail post on generators on Medium:
Co-routines
Now that we know what generators are, we make coroutines simply by adding promises to the mix.
Please note that the code has started looking very similar to its synchronous equivalent. It just needs some supplementary part.
To take care of that, people came up with a few coroutine libraries such as CO.
This part might have been pretty difficult to wrap your head around. It is pretty convoluted. But you might want to read this article if you're interested:
Async/await
Soon, in ES8 (2017), async-await was announced and that made writing coroutines redundant.
Co-routines died out before they could become a thing. Many people today probably donβt even know about them.
Async-await is just a wrapper around Promises. And again, a promise is just a wrapper around callbacks. So, in reality, promises and async-await are all just glamour. Under the skin, it's still callbacks everywhere!
And yet, JS code now looks so clean, intuitive and manageable, that it's orgasmic!
6 years back, nobody would've imagined we could write such clean code in JavaScript.
This code looks exactly similar to the synchronous equivalent. And Iβm awe-struck when I think about how much we hated callbacks, and how much we love structure that it led us from callbacks to async-await.
I'm mesmerized by the transitions that happened around Node.js in such short span of time and I needed to talk about it.
Now, the code looks really simple. Write your code using functions and when you're going to perform an async task, just use the async and await keywords. Anybody can easily write asynchronous code in JavaScript now.
But some times, things don't work as expected. Things that look simple and straight-forward often give unexpected results. And without enough understanding of the problem and the inherent system, one can go nuts in the process of debugging such errors.
Happened with me once.
@bloom_codes So, I used to be a regular naive college boy, okay?
I wanted to create a form that takes input and runs a script for a college project.
But, a certain function I had written kept returning undefined instead of the value it should have.18:13 PM - 22 Apr 2019
@bloom_codes After three days of debugging and struggling with my flawless looking code, I was at the verge of splitting my hair. I thought I was going insane. Finally I googled the issue and that is how I learnt about the asynchronous nature of JavaScript. π18:15 PM - 22 Apr 2019
My mentor probably understood that well. And that is why, he set me up on this journey to find and feel the true essence of Node.js.
JS-veterans, if you find any inconsistencies in this piece, or would like to add more. Or simply want to talk, feel free to comment or DM me.
JS-newbies and JS-virgins, I hope I've sparked an interest for the JS community in your minds. Feel free to reach out in case of any doubts.
Top comments (2)
By writing this blog..you have restored my faith in humanity
Wow! That's really nice of you to say!
I'm glad you found it helpful! :)