Understanding Iterators in Go: A Fun Dive!

If you're a Go programmer, you've probably heard about iterators many times in Go 1.22, and especially in Go 1.23. But maybe you're still scratching your head, wondering why they're useful or when you should use them. Well, you're in the right place! Let's start by looking at how iterators work in Go and why they can be so useful.

A Simple Transformation: No Iterator Yet

Imagine we have a list of numbers, and we want to double each number. We could do this using a straightforward function like the one below:

package main

import (
    "fmt"
)

func NormalTransform[T1, T2 any](list []T1, transform func(T1) T2) []T2 {
    transformed := make([]T2, len(list))

    for i, t := range list {
        transformed[i] = transform(t)
    }

    return transformed
}

func main() {
    list := []int{1, 2, 3, 4, 5}
    doubleFunc := func(i int) int { return i * 2 }

    for i, num := range NormalTransform(list, doubleFunc) {
        fmt.Println(i, num)
    }
}

Here’s what happens when you run this code:

0 2
1 4
2 6
3 8
4 10

Pretty simple, right? This is a basic generic Go function that takes a list of any type T1, applies a transformation function to each element, and returns a new list with the transformed list of any type T2. Easy to understand if you know Go generics!

But what if I told you there’s another way to handle this—using an iterator?

Enter the Iterator!

Now, let’s take a look at how you might use an iterator for the same transformation:

package main

import (
    "fmt"
)

func IteratorTransform[T1, T2 any](list []T1, transform func(T1) T2) iter.Seq2[int, T2] {
    return func(yield func(int, T2) bool) {
        for i, t := range list {
            if !yield(i, transform(t)) {
                return
            }
        }
    }
}

func main() {
    list := []int{1, 2, 3, 4, 5}
    doubleFunc := func(i int) int { return i * 2 }

    for i, num := range NormalTransform(list, doubleFunc) {
        fmt.Println(i, num)
    }
}

Before running it, you must make sure your Go version is 1.23. The output is exactly the same:

0 2
1 4
2 6
3 8
4 10

But wait, why would we need an iterator here? Isn't that more complicated? Let’s dig into the differences.

Why Use an Iterator?

At first glance, iterators seem a bit over-engineered for something as simple as transforming a list. But when you run benchmarks, you start to see why they’re worth considering!

Let’s benchmark both methods and see how they perform:

package main

import (
    "testing"
)

var (
    transform = func(i int) int { return i * 2 }
    list      = []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
)

func BenchmarkNormalTransform(b *testing.B) {
    for i := 0; i < b.N; i++ {
        NormalTransform(list, transform)
    }
}

func BenchmarkIteratorTransform(b *testing.B) {
    for i := 0; i < b.N; i++ {
        IteratorTransform(list, transform)
    }
}

Here’s the initial benchmark result:

BenchmarkNormalTransform-8      41292933                29.49 ns/op
BenchmarkIteratorTransform-8    1000000000               0.3135 ns/op

Whoa! That’s a huge difference! But hold on—there’s a bit of unfairness here. The NormalTransform function returns a fully transformed list, while the IteratorTransform function only sets up the iterator without transforming the list yet.

Let’s make it fair by fully looping through the iterator:

func BenchmarkIteratorTransform(b *testing.B) {
    for i := 0; i < b.N; i++ {
        for range IteratorTransform(list, transform) {
        }
    }
}

Now the results are more reasonable:

BenchmarkNormalTransform-8      40758822                29.16 ns/op
BenchmarkIteratorTransform-8    53967146                22.39 ns/op

Okay, the iterator is a bit faster. Why? Because NormalTransform creates an entire transformed list in memory (on the heap) before returning it, while the iterator does the transformation as you loop through it, saving time and memory.

Read more about Stack and Heap here

The iterator's real magic happens when you don’t need to process the whole list. Let’s benchmark a scenario where we only want to find the number 4 after transforming the list:

func BenchmarkNormalTransform(b *testing.B) {
    for i := 0; i < b.N; i++ {
        for _, num := range NormalTransform(list, transform) {
            if num == 4 {
                break
            }
        }
    }
}

func BenchmarkIteratorTransform(b *testing.B) {
    for i := 0; i < b.N; i++ {
        for _, num := range IteratorTransform(list, transform) {
            if num == 4 {
                break
            }
        }
    }
}

The results speak for themselves:

BenchmarkNormalTransform-8      39599744                30.03 ns/op
BenchmarkIteratorTransform-8    164853723                7.288 ns/op

In this case, the iterator is much faster! Why? Because the iterator doesn’t transform the entire list—it stops as soon as it finds the result you’re looking for. On the other hand, NormalTransform still transforms the entire list, even if we only care about one item.

Conclusion: When to Use Iterators?

So, why use an iterator in Go?

• Efficiency: Iterators can save both time and memory by not processing the entire list if you don’t need it.
• Flexibility: They allow you to handle large datasets efficiently, especially when working with streams of data or when you need to stop early. But keep in mind, iterators can be a bit trickier to understand and implement. Use them when you need that extra performance boost, especially in scenarios where you don’t need to work with an entire list upfront.

Iterators: They're fast, flexible, and fun—once you get the hang of them!