TypeScript Generators: A Refresher

Generators are a powerful tool in many programming languages. They are similar to functions in that they enable the encapsulation and reuse of computations. Yet, unlike functions, they can "return" multiple results and "accept" new inputs before they complete.

While generators are more common in languages like Python, they are a bit of an obscure feature in TypeScript. To be fair, they are the mechanism that powers asynchronous functions, so they were always an integral part of TypeScript. However, there was no need to interact with generators directly, which makes them a bit of an obscure language feature for the average developer.

After looking into Effect, a TypeScript framework that is gaining traction lately, I was thrown off by the use of yield and yield*, especially since the types confused me quite a bit. If you had the same issues, this article might clarify things a bit.

A simple function

One way to understand generators is to see them as enhanced functions. They can do everything a function can, but more!

Let's start with a simple function that squares a number.

function f(a: number) {
  return a * a
}

Its type signature is pretty simple. It takes a number and returns a number.

function f(a: number): number

Calling the function with 10 will return 100.

f(10) // -> 100
f(100) // -> 10,000
f(1000) // -> 1,000,000

You've probably seen this a million times; nothing special here.

The following sequence diagram illustrates what's happening. Every time the function executes, it wipes its stack and starts over from the beginning. (Closures can capture outside state, which might modify their execution path, but they still start from the beginning.)

A simple generator function

Let's convert this function to a generator function.

function* g(a: number) {
  return a * a
}

While the * is the only syntax difference, its type looks very different from a function.

function g(a: number): Generator<never, number, unknown>

The function no longer returns the squared value; instead, it returns a Generator.

This Generator is an object that comes with several methods. If you want to extract the squared value, you must use one of those methods—for example, next.

const generator = g(10)
const result = generator.next()
result.value // -> 100

next doesn't return the plain value, but wraps it in an IteratorResult that contains the squared value in its value field.

The IteratorResult also has a done field, which always contains true for our generator, because it does nothing but return.

A yielding generator function

You can also write the generator function from before like this:

function* g(a: number) {
  yield a * a
}

The yield is similar to return. It passes the squared value to the outside. However, using yield in your generator function changes its type signature.

function g(a: number): Generator<number, void, unknown>

The function still returns a Generator, but the number moved to the first type parameter, and the second became void. So, let's discuss these parameters.

The first type parameter is a union of all types you yield from your generator (Hint: a generator can yield multiple times). In this example, the generator function yields a number, so that's the first parameter.

The second type parameter is the type of the value your generator returns. Since the generator yields a number and doesn't return one anymore, the return type is now void.

The union of yield types combined with the return type makes up the type of the value field in the IteratorResult, which is what the next method returns.

The way you'd extract that value stays the same.

const generator = g(10)
const result = generator.next()
result.value // -> 100

Now, this all might seem like a lot of effort to emulate a simple function, so let's discuss what a generator can do that a function can't.

A generator that yields multiple times

Why use yield instead of return?

A generator function can yield multiple times, but only return once.

This is a valid generator function:

function* g(a: number) {
  yield a
  yield a * a
  yield a * a * a
}

The signature didn't change because the generator only yields numbers.

function g(a: number): Generator<number, void, unknown>

But you can now call next multiple times to get all results.

const generator = g(10)

const firstResult = generator.next()
firstResult.value // -> 10
firstResult.done // -> false

const secondResult = generator.next()
secondResult.value // -> 100
secondResult.done // -> false

const thirdResult = generator.next()
thirdResult.value // -> 1000
thirdResult.done // -> false

const lastResult = generator.next()
lastResult.value // -> undefined
lastResult.done // -> true

The following sequence diagram illustrates what's happening. In contrast to a regular function, the generator doesn't start until the first next call. It also maintains its stack between next calls and resumes execution rather than returning to the beginning.

The yield expressions "return" a value to the outside, but they also stop the execution of the generator until it receives a new next call.

• The first next call runs the generator from start to the first yield.
• The second next call runs it from the first yield to the second yield.
• The third next call runs it from the second yield to the thirdyield.
• The last next call runs the generator from the third yield to the (implicit) return. As the function returns void, the value is undefined, but the done value that was false for the yield is now true. This allows you to check if the generator still has values left.

The generator will yield a value every time you call it. Since Generator implements the Iterator interface, you can even use it with a for-loop.

for(let x of g(10)) console.log(x)

This also removes all the low-level boilerplate required by the next calls.

The for-loop uses IteratorResult in the background. It extracts the yielded values from the value field. It also checks the done field, which will become true when the generator returns.

🚨 Don't return data when you intend to use generators with for-loops. Using next allows you to access the return value, but a for-loop will silently discard it and only give you the yielded values.

A looping generator

Consuming the generator with a for-loop is nice, but you can also use loops to define it.

The following generator contains an infinite loop, but the yield will pause the generator, and, in turn, the while-loop, until the outside for-loop calls next again.

function* g(a: number) {
  let i = 1
  while(true) {
    yield a ** i
    i += 1
  }
}

for(let x of g(10)) {
  if(x > 1000) break
  console.log(x)
}

The break statement will stop the for-loop, effectively breaking the infinite loop in the generator.

A generator that accepts input

The whole "multiple return values over time" feature is pretty cool, but generators have another skill. In contrast to return, the yield expressions are bidirectional. They can pass values to the outside, and later accept new values from the outside.

The following generator function yields its parameter, but also accepts a new one via the next parameter.

function* g(a: number) {
  const b: number = yield a
  const c: number = yield b
  yield c
}

This time, the yield stores its inputs in a variable, so the signature of the generator function changed.

function g(a: number): Generator<number, void, number>

This change finally reveals the purpose of the third type parameter. It defines which inputs the generator accepts via next, and because all variables that store yield values are of type number, the third parameter is also number.

You can pass new values as parameters to next.

const generator = g(1)

generator.next().value // -> 1
generator.next(2).value // -> 2
generator.next(3).value // -> 3
generator.next().value // -> undefined

The first next call doesn't include a parameter, because it starts the generator and runs it until the first yield.

Since the right side of a yield is executed before the left side, and the generator pauses at, or in the middle, of a yield, this can get a bit confusing, so check out the following illustration. Each color of the definition side corresponds to the same color on the execution side.

• The g(1) call creates the generator and sets the parameter.
• The first next call starts the generator, yields the 1, and pauses the generator.
• The second next call resumes the generator, stores the 2, yields the 2, and pauses the generator.
• The third next call resumes the generator, stores the 3, yields the 3, and pauses the generator.
• The last next call resumes the generator, returns undefined, and stops the generator.

In addition to function parameters and return values, generators can communicate with the outside world via yield. Let's see what this enables us to do.

Emulating async/await with a generator

You can use generators to emulate async/await behavior.

Let's take this asynchronous function that fetches data.

async function f() {
  const response: Response = await fetch("https://example.com")
  const content: string = await responsee.text()
  console.log(content)
}

await f()

A normal function can do it too, but you need to handle the promise resolution manually with callbacks.

function f() {
  fetch("https://example.com")
  .then((request) => request.text())
  .then((content) => console.log(content))
}

f()

Quite a different way to program than with async functions. However, with generators, you can emulate async behavior. In fact, generators are the mechanism that powers async/await in the background.

The generator could look like this.

function* g() {
  const response: Response = yield fetch('https://example.com')
  const content: string = yield response.text()
  console.log(content)
}

Pretty similar to the asynchronous function, only that it uses yield instead of await.

Next, you need a function that can execute this promise-yielding-generator.

function run(
  generator: Generator<Promise<unknown>>,
  result?: unknown
) {
  const { done, value } = generator.next(result)
  if (!done) value.then((result) => run(generator, result))
}

This function will:

• call next on the generator until it completes
• resolve promises the generator yields
• pass the results of the promises as arguments to next

You can execute the generator like this.

const generator = g()
run(generator)

This call will log some HTML to the console.

A generator that yields generators

With yield*, generators can call other generators, just like functions can call other functions. So, the async/await emulation can be extended to make it more modular.

function* fetcher(url: string) {
  const response: Response = yield fetch(url)
  const content: string = yield response.text()
  return content
}

function* h() {
  const content = yield* fetcher("https://example.com")
  console.log(content)
}

run(h())

The yield* will yield all values from the fetcher to the run function. After the fetcher completes, yield* stores the return value of the fetcher in content.

Since fetcher typed all its variables, yield* can infer the type of content.

Summary

Generators are a very powerful construct, and I understand why they are heavily used in other languages. I guess the early support for closures and the fact that async functions were introduced quickly after generators made them less of a topic in JavaScript and TypeScript.

The pause/resume semantics of splitting yield across two next calls, the type inference with yield, and the different handling of return values in for-loops and next calls can lead to confusion. At least it did for me.

However, as the async function emulation in this article has shown, they enable library and framework creators to build interesting abstractions that allow imperative programming styles.