Make sense of generators

In Python, generators have been there since almost forever (2001). In PHP, it's a little bit more recent.

Generators are super powerful. Let's see why.

How does it behave?

Any function that contains one or several yield statements is a generator function.

Unlike the return statement, the yield statement does not stop the execution of the function. It suspends its execution. Next time you call the function, it continues from where it left off. The whole context with variables and stuff is available!

This behavior is perfect for iterating through an extensive list of elements, which is extra cool when trying to solve mathematical enigmas.

Generators in use

What if you have to parse large datasets such as log files. Use generators!

You can loop through a crazy amount of iterations without running out memory:

function crazyCount($limit) {
    for ($i = 1; $i < $limit; ++$i) {
        yield $i;
    }
}

foreach (crazyCount(1e8) as $total) {
    echo "see $total" . PHP_EOL;
}

This script is crazy expensive, but it does not break memory. Execution time may vary according to your machine, but it works. PHP generators implement the PHP iterator class, which allows for using handy built-in methods (current(), next()), and exciting usages such as coroutines, multitasking, asynchronous and concurrency.

But what are generators, for real?

Generators simplify code. They bring some kind of execution context and smart ways to stop & go when you need it. The yield statement is like a pause button.

As a result, functions may contain several yields. But it's still quite abstract, isn't it?

Hello world

You can use the send() method to send values back to the generator so it's convenient if you need to run some analysis. Lets do a hello world:

function runBabyRun() {
    $miles = yield;
    print "counter: $miles miles". PHP_EOL;

    $miles = yield;
    print "counter: $miles miles". PHP_EOL;

    $miles = yield;
    print "counter: $miles miles". PHP_EOL;
}

$run = runBabyRun();
$run->send(0);
print "Here we go!" . PHP_EOL;
print "I am running... makes me feel good" . PHP_EOL;
print "Those shoes are so cool!" . PHP_EOL;
print "What a beautiful day!" . PHP_EOL;
print "..." . PHP_EOL;
print "..." . PHP_EOL;
print "..." . PHP_EOL;
print "Oh my god, I'm very far from the city, let's go back home" . PHP_EOL;
$run->send(8);

print "This was a nice run." . PHP_EOL;
$run->send(16);

Concrete example in Python

So now you may see how generators allow you to stop & go.
In Python, generators are convenient to work with big .csv files with millions of rows. As I've said in this post, lousy loops can be a nightmare.

Most of the time, this is due to harmful uses of variables resulting in high consumption of memory. To prevent this, use generators!

A typical use of generators with .csv. files might look like the following:

def read_export(filename):
    for r in open(filename, "r"):
        yield r

It is a generator function. Instead of returning the row, it yields it. Not only is the code shorter and cleaner, but it's much more efficient.

It produces one item at a time, from one element to the next — no need to store a gigantic list in memory.

Going beyond

Delegation

With PHP 7, generators have been upgraded. We have now generator delegation. You can isolate yield statements and use them in other blocks of code:

function dividers() {
    for ($i = 1; $i <= 212; $i++) {
        if (212 % $i === 0) {
            yield $i;
        }
    }
}

function misc() {
    yield 0;
    yield from dividers();
    yield 777;
}

foreach (misc() as $g) {
    print $g . PHP_EOL;
}

Lazy evaluation

What makes generators so efficient is the fact that each element in the list isn't computed in memory until you use it. Instead of using more and more memory, it uses a constant amount of memory.

This intelligent behavior allows for working at high scales.

List comprehension

In Python, this technique is quite popular :

spell_it = [ letter for letter in 'technologic' ]
print( spell_it)

And I find this syntax pretty, it's even more readable than a loop and much much more readable than a map().

Did you know you can do almost the same in PHP?
The following code (assuming $entities is an array of objects):

$names = [];
foreach ($entities as $entity) {
    $names[] = $entity->name;
}

could be rewritten like that:

$names = [foreach ($entities as $entity) yield $entity->name];

Wrap up

PHP gets more and more fresh inspiration from other languages such as Python. Generators allow for functional/declarative programming even in PHP.

Comments

[Banji]

Tnx Julien for this great article