loading...

Why async code is so damn confusing (and a how to make it easy)

joelnet profile image JavaScript Joel Updated on ・6 min read

Yarn

Why is asynchronous code in Javascript so complicated and confusing? There are no shortage of articles and questions from people trying to wrap their bean around it.

Some handpicked questions from SO...

There are literally hundreds of questions and articles about async and a lot of them sound something like this:

// How do I do this in JavaScript?
action1();
sleep(1000);
action2();

This is a common misunderstanding of how JavaScript works.

Dirty hacks to force Sync

There are even dirty hacks to force sync

NOT RECOMMENDED

The problem is not Asynchronous code

I spend a lot of time thinking about Javascript and one of these times I had a silly thought. What if the problem is not asynchronous code. What if the problem is actually the synchronous code?

Synchronous code is the problem? WAT?

I often start writing my code synchronously and then try to fit my async routines in afterwards. This is my mistake.

Asynchronous code cannot run in a synchronous environment. But, there are no problems with the inverse.

async/async feature chart

This limitation is only with synchronous code!

Mind blown GIF

Write Asynchronously from the start

Coming to this realization, I now know that I should begin my code asynchronously.

So if I were to solve the async problem again, I would start it like this:

Promise.resolve()
    .then(() => action1())
    .then(() => sleep(1000))
    .then(() => action2())

or with async and await...

const main = async () => {
  action1()
  await sleep(1000)
  action2()
}

The Promise solution is... wordy. The async/await is better, but it's just syntactic sugar for a Promise chain. I also have to sprinkle async and await around and hope I get it right.

Sometimes async/await can be confusing. For example: These two lines do completely different things.

// await `action()`
await thing().action()

// await `thing()`
(await thing()).action()

And then there's the recent article from Burke Holland:

[DEV.to] Async/Await and the forEach Pit of Despair.

What if there was no difference?

So I start to think again... What if there was no difference between async and sync code? What if I could write code without worrying about whether the code I am writing is asynchronous or not. What if async and sync syntax was identical? Is this even possible?

Well, that means I cannot use standard functions as they are only synchronous. async/await is out too. That code just isn't the same and it comes with it's own complexities. And promises would require me to write then, then, then everywhere...

So again, I start thinking...

Asynchronous Function Composition

I love love love functional programming. And so I start thinking about asynchronous function composition and how I could apply it to this problem.

In case this is your first time hearing about function composition, here's some code that might help. It's your typical (synchronous) "hello world" function composition. If you want to learn more about function composition, read this article: Functional JavaScript: Function Composition For Every Day Use.

const greet = name => `Hello ${name}`
const exclaim = line => `${line}!`

// Without function composition
const sayHello = name =>
  exclaim(greet(name))

// With function composition (Ramda)
const sayHello = pipe(
  greet,
  exclaim
)

Here I used pipe to compose greet and exclaim into a new function sayHello.

Since pipe is just a function, I can modify it to also work asynchronously. Then it wouldn't matter if the code was synchronous or asynchronous.

One thing I have to do is convert any callback-style function to a promise-style function. Fortunately node's built in util.promisify makes this easy.

import fs from 'fs'
import { promisify } from 'util'
import pipe from 'mojiscript/core/pipe'

// Synchronous file reader
const readFileSync = fs.readFileSync

// Asynchronous file reader
const readFile = promisify(fs.readFile)

Now if I compare a Synchronous example with an Asynchronous example, there is no difference.

const syncMain = pipe([
  file => readFileSync(file, 'utf8'),
  console.log
])

const asyncMain = pipe([
  file => readFile(file, 'utf8'),
  console.log
])

This is exactly what I want!!!

Even though readFileSync is synchronous and readFile is asynchronous, the syntax is exactly the same and output it exactly the same!

I no longer have to care what is sync or what is async. I write my code the same in both instances.

Banana says Wooo-Wooooo!

ESNext Proposal: The Pipeline Operator

It is worth mentioning the ESNext Proposal: The Pipeline Operator.

The proposed pipeline operator will let you "pipe" functions in the same way pipe does.

// pipeline
const result = message =>
  message
    |> doubleSay
    |> capitalize
    |> exclaim

// pipe
const result = pipe([
  doubleSay,
  capitalize,
  exclaim
])

The format between the Pipeline Operator and pipe are so similar that I can also switch between the two without any problems.

The Pipeline Proposal is very exciting, but there are two caveats.

  1. It's not here yet, and I don't know if or when it will come or what it will look like. babel is an option.
  2. It does not (yet) support await and when it does, will most-likely require different syntax to pipe sync and async functions. yuck.

I also still prefer the pipe function syntax over the pipeline operator syntax.

Again, the pipeline will be starting code synchronously, which I have already identified as a problem.

So while I am excited for this feature, I may never end up using it because I already have something better. This gives me mixed feelings :|

MojiScript

This is where you ask me what the heck is this...

import pipe from 'mojiscript/core/pipe'
//                ----------
//               /
//          WAT?

(Okay you didn't ask... but you're still reading and I'm still writing...)

MojiScript is an async-first, opinionated, functional language designed to have 100% compatibility with JavaScript engines.

Because MojiScript is async-first, you don't have the same issues with async code that you do with typical JavaScript. As a matter of fact, async code is a pleasure to write in MojiScript.

You can also import functions from MojiScript into existing JavaScript applications. Read more here: https://github.com/joelnet/MojiScript

MojiScript Async Examples

Here's another great example of async with MojiScript's pipe. This function prompts a user for input, then searches the Star Wars API for using Axios, then writes the formatted results to the console.

const main = ({ axios, askQuestion, log }) => pipe ([
  askQuestion ('Search for Star Wars Character: '),
  ifEmpty (showNoSearch) (searchForPerson (axios)),
  log
])

If this has made you curious, check out the full source code here: https://github.com/joelnet/MojiScript/tree/master/examples/star-wars-console

I need your help!

Here's the part where I ask you for help. MojiScript is like super brand new, pre-alpha, and experimental and I'm looking for contributors. How can you contribute? Play with it, or submit pull requests, or give me your feedback, or ask me questions, anything! Head over to https://github.com/joelnet/MojiScript and check it out.

Summary

  • Asynchronous code cannot run in a synchronous environment.
  • Synchronous code will run just fine in an asynchronous environment.
  • Start writing your code asynchronously from the start.
  • for loops are synchronous. Get rid of them.
  • Try asynchronous function composition with something like pipe.
  • pipe has similar functionality as the ESNext Pipeline Proposal, but available today.
  • Play with MojiScript :)
  • MojiScript is currently in the experimental phase, so don't go launching this into production yet!

Getting started with MojiScript: FizzBuzz (part 1)

Read more articles by me on DEV.to or Medium.

Follow me on Twitter @joelnet

Cheers!

Discussion

pic
Editor guide
Collapse
yurifrl profile image
Yuri

Great article, I stubble with the same issues a while back, and started using github.com/fluture-js/Fluture it basically transforms promises into monads, something like:

const Future = require('fluture')
const { compose, map } = require('ramda')

const greet = name => Future.of(`Hello ${name}`)
const exclaim = line => Future.of(`${line}!`)
const trace = x => console.log(x)

// With function composition (Ramda)
const sayHello = compose(
  map(trace),
  map(greet),
  exclaim
)
sayHello("bob").fork(console.error, console.log)

I convert all my promises to futures, if it fails instead of catching the error, the map will not be triggered and I can use mapRej to deal with the error

Collapse
joelnet profile image
JavaScript Joel Author

fluture is one of those project I have read about, always wanted to use and have never used. I need to block out some time and just use it one day. I feel like even if I don't end up using it I will learn a lot.

Collapse
jochemstoel profile image
Jochem Stoel
/* moji.js */
const pipe = require('mojiscript/core/pipe')
const fs = require('fs')

const syncMain = pipe([
    file => fs.readFileSync(file, 'utf8'),
    console.log
])
const asyncMain = pipe([
    file => fs.readFile(file, 'utf8'),
    console.log
])

syncMain('moji.js')
asyncMain('moji.js')

Output

> node moji.js
const pipe = require('mojiscript/core/pipe')
const fs = require('fs')

const syncMain = pipe([
    file => fs.readFileSync(file, 'utf8'),
    console.log
])
const asyncMain = pipe([
    file => fs.readFile(file, 'utf8'),
    console.log
])

syncMain('moji.js')
asyncMain('moji.js')
(node:7424) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_CALLBACK]: Callback must be a function
    at maybeCallback (fs.js:129:9)
    at Object.readFile (fs.js:275:14)
    at file (C:\Users\Gebruiker\Desktop\moji.js:10:16)
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)
    at startup (internal/bootstrap/node.js:279:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:752:3)
(node:7424) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:7424) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Collapse
joelnet profile image
JavaScript Joel Author

One change you'll have to make is to convert all callback-style functions into promise-style functions.

In this case fs.readFile is a callback-style function.

You can convert it using node's util.promisify function like this:

import fs from 'fs'
import { promisify } from 'util'

// Asynchronous file reader
const readFile = promisify(fs.readFile)

Now you can use your promise-style function in the pipe.

Collapse
jochemstoel profile image
Jochem Stoel

My bad, I should have seen that.

Intrigued. I have a strongly opinionated post draft saved that is spooky similar to what I read from you today. Asking the same questions providing the same examples. I will have a more interested look at your emoji package and the patterns you are adopting.

Thread Thread
joelnet profile image
JavaScript Joel Author

I'm guessing you are like me and have about a dozen articles in the 90% finished draft state?

Can't wait to see what you are working on!

Thread Thread
jochemstoel profile image
Jochem Stoel

Although the notation would not be identical. Another thing you can do is write a single function to handle both sync and async and deciding which by its arguments. Asynchronous if a callback is provided, otherwise synchronous.

/* synchronous return */
readFile('file.txt')

/* asynchronous, no return value */
readFile('file.txt', console.log)
Collapse
crazy4groovy profile image
crazy4groovy

There's one thing to recognize about await, is that it doesn't "hurt" anything!

So:

const main = async () => {
  action1()
  await sleep(1000)
  action2()
}

and

const main = async () => {
  await action1()
  await sleep(1000)
  await action2()
}

will always be the same.

In fact, event if action1 and action2 returns a literal value (let's say "hi!"):

const main = async () => {
  const a = action1()
  await sleep(1000)
  const b = action2()
  console.log(a, b)
}

and

const main = async () => {
  const a = await action1()
  await sleep(1000)
  const a = await action2()
  console.log(a, b)
}

will always be the same! (mind. blown.)

I for one think you are onto something, where the lines between async and sync can almost always be blurred away. But I think it would take a new syntax. For example:

const main = () => {
  const a =: action1()
  =: sleep(1000)
  const b = action2()
  console.log(a, b)
}

where =: means "resolve the value", and all arrow function are inherently "async".

(This is kind of like pointer logic, where you either mean "give me the address" or "give me the value at the address"; but here its "give me the promise" or "give me the value of the promise")

Collapse
joelnet profile image
JavaScript Joel Author

That's true, you could just await everything. You can even do this:

const x = await 5

In the case of this problem:

// await `action()`
await thing().action()

// await `thing()`
(await thing()).action()

You could do this and not worry.

// await all the things
await (await thing()).action()

But I would probably go insane.

Collapse
vhoyer profile image
Vinícius Hoyer

while it is true you could await every thing, there is a difference. the await operator will always wait till the next tick to start checking for the resolving of a promise, even if your promise is a literal. But thats just what I read somewhere that I don't remember where, sorry for the lack of reference, yet I'm pretty confident in the information.

Thread Thread
joelnet profile image
JavaScript Joel Author

I believe you were correct about this. Adding extraneous awaits could have a performance impace.

Collapse
dfockler profile image
Dan Fockler

This looks pretty neat. Having worked with async code before Promises were everywhere, they are a real trip to learn to use. Although the idea is used throughout the programming stack, it's definitely something useful to learn.

I think the coolest part of async/await style is that you can use try/catch to get errors instead of having to add a new function for every promise you unwrap.

Collapse
joelnet profile image
JavaScript Joel Author

Haha that's interesting! That was actually the main reason I disliked async/await, I had to wrap everything in a try/catch.

Guess it all depends on what your preferences are.

Collapse
ben profile image
Ben Halpern

Wow, Mojiscript looks really well-done. I'm following along now 😄

Collapse
joelnet profile image
JavaScript Joel Author

Thanks! I have been writing functional code for a while now, but I my style and tools has been evolving and I have never laid out a style guide or any rules to follow. So each project tends to be a little different.

This is the first time laid out some rules. Putting in that extra time to create the guide and readme really helped me solidify some thoughts in my head. I think it came out well and I hope to produce some projects that can showcase the language.

Appreciate you taking a peek at it :)

Cheers!

Collapse
ben profile image
Ben Halpern

Are you (or anyone else) using Mojiscript in production?

Thread Thread
joelnet profile image
JavaScript Joel Author

Great question. I wouldn't recommend it for production. I created it less than 2 weeks ago and haven't finalized the API.

I would expect the interface and behavior to change based on feedback. I would consider this version a developer preview.

I am using this framework for a few pet projects of mine that I will open source when ready :)

Thread Thread
ben profile image
Ben Halpern

That's what I figured, but it's well-polished in the presentation that I wasn't quite sure.

Thread Thread
joelnet profile image
JavaScript Joel Author

lol ya thanks!

but again great question. I'll make this more clear in this article so people don't go launching this into prod yet!

Need more testers. Currently at zero. lol

Collapse
tomerbendavid profile image
Tomer Ben David

What about debugging it in most cases although the syntax makes it easy I'm not sure about debugging...

Collapse
joelnet profile image
JavaScript Joel Author

You actually have less debugging because all of your functions become testable.

Try unit testing this:

for (var i = 1; i < 6; i++) {
  setTimeout(() => console.log(i), 1000)
}

hint: it sucks.

Collapse
scotthannen profile image
Scott Hannen

This should definitely be added into CoffeeScript.

Collapse
joelnet profile image
JavaScript Joel Author

You should be able to import pipe into coffee script!