DEV Community

loading...
Cover image for Generators in Javascript: How to use them

Generators in Javascript: How to use them

karimelghamry profile image Karim Elghamry ・8 min read

Opening Note

Hello fellow programmers πŸ‘‹ In this article, we will walk-through the basics of generators in Javascript, which was introduced in ES6, and get across some real use cases.

What is a generator?

From its name, a generator is a function that allows you to generate one or more values by exiting and re-entering the execution procedure whilst saving its state (context) across multiple calls. To put it in simpler words, a generator is similar to normal functions, but has the ability to continue execution on demand at the point at which it was previously terminated, simply by saving its previous state. The following flowchart illustrates the difference between a normal function and a generator function.

Alt Text

Syntax

As you've already guessed, there are some syntactic differences between a normal function and a generator:

// Normal Function
function normalFunction(params) {
  // your logic goes here
  return value;
}

/* --------------------------------- */

// Generator Function
function* generatorFunction(params) {
  // your logic
  yield value1;

  // your logic
  yield value2;

  /*
    .
    .
    .
  */

  // your logic
  yield valueN;
}

Enter fullscreen mode Exit fullscreen mode

The first noticeable difference in syntax is that a generator is declared using the function* keyword instead of function. Also, notice how we use the return keyword in a normal function, while we use the yield keyword in a generator function instead, respectively. The yield keyword inside the generator allows us to 'return' a value, terminate execution, save the state (context) of the current lexical scope and waits for the next invocation to resume execution at the last termination point.

note: In a normal function, you can only execute the return keyword once, which will return a value and terminate the function completely. In a generator, you can use the yield keyword multiple times as much as you want to 'return' values on consecutive calls. You can also use the return keyword inside a generator, but leave this discussion for a different day.

Invocation

Now that we've covered the differences in syntax between both functions, let us see how does one invoke a generator and yield its values. First, consider the following piece of code which illustrates the invocation of a normal function:

function normalFunction() {
  console.log('I have been invoked');
}

// invocation
normalFunction();
Enter fullscreen mode Exit fullscreen mode

In general, you can invoke a normal function by typing the function's signature followed by a pair of parentheses (). The previous code will output:

I have been invoked
Enter fullscreen mode Exit fullscreen mode

Now let's try to use the same procedure to invoke a generator. Inspect the following piece of code closely:

function* generatorFunction() {
  console.log('I have been invoked');
  yield 'first value';

  console.log('resuming execution');
  yield 'second value';
}

// does this invoke the generator?
generatorFunction();

Enter fullscreen mode Exit fullscreen mode

What do you expect from such program? Technically, we would expect that the function is to be executed until it hits the first yield keyword. However, the output of the previous program was empty:


Enter fullscreen mode Exit fullscreen mode

that is because the normal invocation syntax does not actually execute the body of the generator function. Instead, it creates a Generator Object that holds multiple properties and methods. To prove this, we can try to print out console.log(generatorFunction()) and the output should be as follows:

Object [Generator] {}
Enter fullscreen mode Exit fullscreen mode

So, the question is; how do we actually yield our values from a generator?

well, there are some important methods that belong to the Generator Object that we can utilize. The first and the most important method is called next(), which, from its name, yields the next value from the defined generator. Now lets modify our previous code to actually yield our values:

function* generatorFunction() {
  console.log('I have been invoked');
  yield 'first value';

  console.log('resuming execution');
  yield 'second value';
}

// store the Generator Object in a variable
let foo = generatorFunction();

// execute until we yield the first value
console.log(foo.next());

// resume execution until we yield the second value
console.log(foo.next());

// execute until the function ends
console.log(foo.next());

Enter fullscreen mode Exit fullscreen mode

the output of the previous code is:

I have been invoked
{ value: 'first value', done: false }
resuming execution
{ value: 'second value', done: false }
{ value: undefined, done: true }

Enter fullscreen mode Exit fullscreen mode

Let's inspect the output line by line. When calling the first foo.next() method, the generator began to execute until it hit the first yield keyword and stops the execution. This is reflected in the first two lines of the output. Notice how the foo.next() returned an Object instead of the actual yielded value. This Object should always contain the following properties:

  • 'value': which holds the current yielded value from the generator.

  • 'done': a boolean flag which indicates whether the generator execution has reached the end or not.

Lets move on to the second foo.next() call. As expected, the generator resumes the execution from the last termination step and executes until it hits the second yield keyword, which is reflected in the third and fourth lines of the output. Notice how the done flag is still set by false, as it did not yet reach the end of the function.

On the last foo.next() call, the function resumes execution after the second yield keyword and finds nothing to execute, which indicates that we've reached the end of the function. At this point, there are no more values to yield and the done flag is set to true as reflected in the last line of the output.

Now that we've covered the basic concepts of generators in Javascript, lets take a look at some of its useful use cases.

Use Cases

Use case 1: Mimic the range() function from Python

According to the Python docs, "the range type represents an immutable sequence of numbers and is commonly used for looping a specific number of times in for loops." The range() function in Python usually contains the following parameters:

  • start (optional, default = 0): the first number in the sequence, inclusive.

  • end (required): the last number of the sequence, exclusive.

  • step (optional, default = 1): the difference between any two given numbers in the sequence.

Basically, the usage of the range() function in Python is shown below:

# Python code
for i range(3):
    print(i)

# output:
# 0
# 1
# 2

Enter fullscreen mode Exit fullscreen mode

what we need to do is to mimic this functionality in Javascript using generators. Inspect the following piece of code closely:

/*
range function implemented in Javascript
*/
function* range({start = 0, end, step = 1}) {
  for (let i = start; i < end; i += step) yield i;
}
Enter fullscreen mode Exit fullscreen mode

Lets take it step by step. Firstly, the function signature defines a generator that takes three params: start, end and step, in which start and step are defaulted to 0 and 1 respectively. Moving onto the function body, it contains a basic for loop that starts iterating from start inclusive till end exclusive. Inside the loop's scope, we yield the value i of the current number in the sequence.

Lets see it in action. The following piece of code illustrates different examples of the implemented range function:

// first example
for (let i of range({end: 4})) console.log(i);

/*
output:
0
1
2
3
*/

// second example
for (let i of range({start: 2, end: 4})) console.log(i);

/*
output:
2
3
*/

// third example
for (let i of range({start: 1, end: 8, step: 2})) console.log(i);

/*
output:
1
3
5
7
*/

Enter fullscreen mode Exit fullscreen mode

Use case 2: Visualize the Bubble Sort algorithm

In this use case, we will attempt to output a step-by-step execution of the Bubble Sort algorithm on a given array to easily visualize it. Briefly, bubble sort works as follows; given an array of length n and i as the current iteration, propagate the max(array[0:n - i]) to the index n - i repeatedly until the array is sorted. The default implementation is shown below:

/*
Bubble Sort implementation in javascript
*/
function bubbleSort(arr) {
  for (let i = arr.length - 1; i >= 0; i--) {
    for (let j = 0; j < i; j++) {
      // if the current value is larger than its adjacent
      // swap them together
      if (arr[j] > arr[j+1]) {
        [arr[j], arr[j+1]] = [arr[j+1], arr[j]];
      }
    }
  }

  return arr;
}
Enter fullscreen mode Exit fullscreen mode

Our job is to visualize the step-by-step comparisons and swaps that are carried out throughout this algorithm. This can be easily done using generators. We simply yield the current array after each iteration in the inner loop. The new function will be as follows:

/*
visualize Bubble Sort implementation in javascript
*/
function* visualizeBubbleSort(arr) {
  for (let i = arr.length - 1; i >= 0; i--) {
    for (let j = 0; j < i; j++) {
      if (arr[j] > arr[j + 1]) {
        [arr[j], arr[j + 1]] = [arr[j + 1], arr[j]];
      }

      yield arr;
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

This will yield the array for each iteration in the inner loop, showing us the current state of the array. Consider the following example:

let inputArray = [40, 30, 2, 20];
let currentStep = 1;
for (let val of visualizeBubbleSort(inputArray)) {
  console.log(`step #${currentStep}: [${val}]`);
  currentStep++;
}
Enter fullscreen mode Exit fullscreen mode

The output of the previous program will be:

step #1: [30,40,2,20]
step #2: [30,2,40,20]
step #3: [30,2,20,40]
step #4: [2,30,20,40]
step #5: [2,20,30,40]
step #6: [2,20,30,40]
Enter fullscreen mode Exit fullscreen mode

we can clearly see what is happening throughout the algorithm thanks to the implemented generator:

  • step 1 -> swap 40 with 30

  • step 2 -> swap 40 with 2

  • step 3 -> swap 40 with 20

  • step 4 -> swap 30 with 2

  • step 5 -> swap 30 with 20

  • step 6 -> do not swap anything, array is sorted

Note: this technique can be used to visualize any given algorithm easily. It can be very helpful sometimes.

Use case 3: Generate distinct random numbers on demand

In this use case, we will try to generate a series of distinct random numbers using generators. First, we would put some constraints on the inputs and outputs as follows:

  • The function should only generate positive integers.

  • The function should take a parameter limit, which determines the maximum number of generated integers as well as the largest possible generated integer.

  • The function should have a way to store the valid pool of integers to choose from.

Following the previous constrains carefully, we can implement this functionality using generators easily:

/*
distinctRandom implementation in js 
*/
function* distinctRandom({limit = 10}) {
  // we create an array that contains all numbers in range [0:limit)
  // this is our initial pool of numbers to choose from
  const availableValues = [...new Array(limit)].map((val, index) => index);

  // we repeatedly loop until the available pool of numbers is empty
  while (availableValues.length !== 0) {
    // generate a random index in range [0: availableValues.length)
    // then, yield the number that is present at the chosen index
    // Finally, remove the picked item from the pool of available numbers
    const currentRandom = Math.floor(Math.random() * availableValues.length);
    yield availableValues[currentRandom];
    availableValues.splice(currentRandom, 1);
  }
}
Enter fullscreen mode Exit fullscreen mode

Briefly, the previous generator tries to maintain a pool of available integers to choose from. In each iteration, we randomly choose a number from this pool, then yield it and remove it from the valid pool. Theoretically, the maximum number of generated integers should be equal to limit and all generated integers must be distinct. We can easily proof this by exhausting the implemented generator until the end of execution:

// we set the limit to 8
for (const val of distinctRandom({limit: 8})) {
  console.log(val);
}

/*
sample output:
3
7
5
2
4
0
1
6
*/
Enter fullscreen mode Exit fullscreen mode

Closing Note

Generators are a great addition to ES6 which provides a solution for multiple problems and use cases. You can use them anywhere for sure, but I would suggest looking into alternative solutions for the problem in-hand before settling for a generator, as they can introduce more complexity to your code as well as they can be hard to debug at times. Nevertheless, happy coding πŸŽ‰

Discussion (24)

pic
Editor guide
Collapse
alimobasheri profile image
Mir Ali Mobasheri

Nice article! You described everything very well.

Collapse
karimelghamry profile image
Karim Elghamry Author

Thank you! 😁 I tried to encapsulate the majority of the concepts into one article, glad it went well.

Collapse
kgilpin profile image
Kevin Gilpin

To me, being able to traverse large data structures without having to load them all in memory, and without having to provide a callback-based interface, seems really useful. Useful enough that it can make its way into a lot of code. Maybe not for webdev stuff like vue/react coding, but for pretty hard core data processing like crunching through large data files? Yes!

Collapse
tassoevan profile image
Tasso Evangelista

It's a shame that generators kinda look terrible in TypeScript.

const output = yield input;
Enter fullscreen mode Exit fullscreen mode

It's impossible to properly type output based on input. You've to use a wide type for both and force type narrowing by hand.

Collapse
tylors profile image
Tylor Steinberger

If you switch to yield * the type do work surprisingly well.

For example, github.com/briancavalier/fx-ts

Collapse
tassoevan profile image
Tasso Evangelista

Hmmm, I haven't wondered it! It surely restricts a little what you can express in code, but I not sure about what is lost.

Thread Thread
tylors profile image
Tylor Steinberger

Thankfully it shouldn't! There are still yields happening under the hood, but in Generator<Yield, Return, Next>, Next is the return value of individual yields which there can be many (source of the issue is they must be a product type A & B), while Return is the value returned when .next() returns done:true, and since there is only 1 value there we don't have to worry about the same typing complexities.

It's very easy to convert from one to the other. function*(thing) { const x = yield thing; return x}

Collapse
lcfvs profile image
Lcf.vs

Better distinctRandom:

function* distinctRandom({limit = 10}) {
  // we create an array that contains all numbers in range [0:limit)
  // this is our initial pool of numbers to choose from
  const availableValues = [...new Array(limit).keys()];

  // we repeatedly loop until the available pool of numbers is empty
  while (availableValues.length !== 0) {
    // generate a random index in range [0: availableValues.length)
    // then, yield the number that is present at the chosen index
    // Finally, remove the picked item from the pool of available numbers
    const currentRandom = Math.floor(Math.random() * availableValues.length);
    yield availableValues.splice(currentRandom, 1)[0];
  }
}
Enter fullscreen mode Exit fullscreen mode
Collapse
slikts profile image
Reinis Ivanovs

An important omission about use cases is that generators are a relatively low-level tool, so they're relevant to, for example, library authors, but not for most language users.

Collapse
karimelghamry profile image
Karim Elghamry Author

That's true to some extent. Generators are not meant to be used on a day-to-day basis and you can actually develop solutions to your problem without needing to touch generators. Nevertheless, it's a new feature added to the table, so it's good to be aware of their existence while developing your own solutions. I've tried to include a couple of basic use cases that might see the usage of generators.

Collapse
slikts profile image
Reinis Ivanovs

Sorry, but you should have qualified the use cases as being for, for example, library authors, because this way they're bound to be misapplied and waste time.

Thread Thread
karimelghamry profile image
Karim Elghamry Author

I don't see anything bound to library authors or a specific category of users in the basic use cases that I've provided above, especially the visualization use case. Also, you should've read my closing note more carefully as it states that generators are not the best solution for most problems. As any other existing approach, you must always think of the pros and cons of it before applying it to the problem in-hand.

Thread Thread
designbyonyx profile image
Ryan Wheale

I think like anything else it's another tool in the tool belt. Sometimes the best way to learn is to pull the wrong tool, or build your own version of a tool instead of bringing in an entire tool shed like lodash. While some might consider that "wasted" time, I consider it educational time.

Collapse
maxwelloroark profile image
Sp00kyCat

I was trying to think up a use-case for a front end web app. Possibly... a series of modal dialogues? Hook up a generator to a button click listener on a "delete" button and then yield "Are you sure you want to delete?" "Are you sure you're sure", final yield, "Okay deleting thing".

Collapse
maruffahmed profile image
Maruf Ahmed

Thanks, you explain the JS generator to me. ❀

Collapse
karimelghamry profile image
Karim Elghamry Author

I'm glad that I was able to help! πŸ˜πŸŽ‰

Collapse
the_spyke profile image
Anton Alexandrenok

If you like functional style and generators, you may find the Undercut handy. The Pull is basically native generators + iterators.

Collapse
vrohit profile image
rohitv-dev

Is that range generator performant enough, compared to the traditional for loop? Cause there are two for loops going on right? I am curious.

Collapse
destroyer22719 profile image
Nathan Cai

isn't this pretty similar to async/await?

Collapse
karimelghamry profile image
Karim Elghamry Author

There are some key differences. One of the biggest differences is that after finishing the await call inside an async function, the function does not terminate nor return a value. Instead, it continues its execution normally. With generators however, you can pause and return values on demand using the yield keyword.

But if you actually think about it, they have so much in common. Actually, it is easy to replicate the behavior of async/await using generators. This article describes a brief idea on how to do this with little effort.

Collapse
skflowne profile image
skflowne

You did a great job of explaining this simply and efficiently

Collapse
carl_pease_16dedc891c4677 profile image
Carl Pease

As a primary .Net developed, this provides the IEnumerable/IEnumerator functionality. It is very well suited for time of use value retrieval and iteration of a collection. Thanks! Haven’t heard of it until now!

Collapse
karimelghamry profile image
Karim Elghamry Author

I have used C# along with Unity a couple of times in the past and I can definitely relate to what you're saying. I am glad you've found it helpful! 😁

Collapse
marraybob profile image
Bob Marray

Thanks, you explain the JS generator to me.