Golang generator functions (Experimental in 1.22)

To sustain longevity, you have to evolve.

Golang is still evolving, and the developer team behind the language keeps adding new features and improving the existing ones. Some updates bring a portion of experimental features that might be added in upcoming major updates. In the 1.22 update Go team introduced fixes the language lacked so much, such as:

• enhanced routing for http package - http.HandleFunc("GET /posts/{id}", handlePost2)
• new for loop with iteration scoped variables
• improved math package - math/rand/v2

In this version, they added a new experimental feature to the language, Rangefunc, which is pretty exciting. So what is Rangefunc? In simple words, it's a range-over function, a little bit similar to PHP generators (if you are familiar with it). This functionality allows developers to create iterators from functions and use them inside for loops with range keywords.

NOTE: iterator functions are often methods of collection types or standalone functions that return iterator objects. These functions enable iteration over the elements of a collection in a controlled and efficient manner.

Let's take a look at an example of how the generators could be useful. We have a requirement to create an event-processing system that must process a sequence of tasks. Here's a basic example here:

package main

import (
    "fmt"
    "sync"
    "time"

    "github.com/google/uuid"
)

var wg sync.WaitGroup

func main() {
    events := spawn()

    wg.Add(len(events)) // adding all events to the WaitGroup
    for _, ev := range events {
        go func() {
            defer wg.Done()
            process(ev)
        }()
    }
    wg.Wait() // wait for all events to become blessed
}

// process does the magic in a second and prints a result
func process(event string) {
    time.Sleep(1 * time.Second)
    fmt.Printf("bibbidi bobbidi processed: %s \n", event)
}

// spawn generates 10 tasks later to be passed to the process function
func spawn() []string {
    events := make([]string, 10, 10)
    for i := 0; i < 10; i++ {
        events[i] = uuid.New().String()
    }
    return events
}

In this function, we have a loop that iterates over each event in the events slice. Each event spawns a new goroutine. This means that each goroutine will process a different event concurrently. When we look at this code, you'll notice that there's quite a bit of boilerplate. This can be solved by introducing a function iterator.

wg.Add(len(events)) // adding all events to the WaitGroup
for _, ev := range events {
    go func() {
        defer wg.Done()
        process(ev)
    }()
}
wg.Wait() // wait for all events to become blessed

This iteration can be converted into the so-called generator function, which in golang is introduced by Rangefunc. This function will contain the whole parallel logic inside and will be isolated from the main logic as a separate implementation. To do so, we need to create a new function named Parallel. The function signature would look like this:

func Parallel(events []Event) func(func(int, Event) bool)
                                      // ---------
                                      // ↑    ↑
                                      // |    |
                                      // |    event object
                                      // |
                                      // event ID

Arguments of the type func(int, Event) represent the value in the slice at the iteration. The return type of the range function is a yield function func(func(int, Event) bool) which must return a bool result according to the documentation:

If f is a function type of the form func(yield func(T1, T2)bool) bool, then for x, y := range f { ... } is similar to f (func(x T1, y T2) bool { ... }), where the loop body has been moved into the function literal, which is passed to f as yield. The boolean result from yield indicates to f whether to keep iterating. The boolean result from f itself is ignored in this usage but present to allow easier composition of iterators.

Now we have a function signature. Let's implement the body. We need a closure that accepts the yield function. The rest of the logic with goroutines goes inside it.

func Parallel(events []Event) func(func(int, Event) bool) {
    return func(yield func(int, Event) bool) {
        var wg sync.WaitGroup 
        wg.Add(len(events))

        for idx, event := range events {
            go func() {
                defer wg.Done()
                yield(idx, event)
            }()
        }
        wg.Wait()
    }
}

Now we have the generator, let's modify the main function to use the new funciton:

...
func main() {
    events := spawn()
    for _, ev := range Parralel(events) {
        process(ev)
    }
}
...

When we run this code, we can see that everything works in parallel as it did before with the boilerplate. Before running make sure that you have enabled this experimental feature using the flag GOEXPERIMENT=rangefune:

[Running] GOEXPERIMENT=rangefune go run

bibbidi bobbidi processed: 129809f5-e033-4999-8e9b-e4b0c3fdedcb 
bibbidi bobbidi processed: 8348abc6-c211-455d-b1e1-52cb1cca86b6 
bibbidi bobbidi processed: 0d5f73d3-1e46-461d-b31d-3f62cc28101d 
bibbidi bobbidi processed: d5d2dc05-6c88-434a-9741-9adac4e08020 
bibbidi bobbidi processed: b8e4fb68-a1ee-4f5f-bbf9-28875828ac53 
bibbidi bobbidi processed: 73ef2b5f-9ed2-4560-8fcb-b256206532aa 
bibbidi bobbidi processed: 8bf2051e-816e-4315-b53e-f72c5ef65bf6 
bibbidi bobbidi processed: 734ad453-2a0b-48eb-8cb6-76176556de00 
bibbidi bobbidi processed: 6412809a-9193-48dc-a785-278932c9e89c 
bibbidi bobbidi processed: 15efd790-8772-43c1-a88f-09849770082b 

[Done] exited with code=0 in 1.522 seconds

Conclusion

This feature brings a whole new meaning to the iterations over the data structures and, in my opinion, has a good potential to exist. It will bring some sort of standardization to various standard library data structures like buffers, scanners, readers, and more. I would love to see this feature in the language's future update notes.