Alex

Posted on Aug 8, 2025 Edited on Aug 20

Lazy Evaluation in PHP: Real‑World Memory Savings with Generators

#programming #backenddevelopment #php #performance

Generators are often introduced with a neat demo: build an array with a million values, replace it with yield, and print the memory usage. The conclusion is usually correct, but the benchmark often is not.

If the array is released before memory is measured, or both implementations run in the same PHP process, the result can be misleading. I rebuilt the example around a real CSV file and ran each strategy in an isolated process.

The short version: for this workload, the eager implementation added 386.81 MiB of peak memory. The generator added 10.69 KiB. The generator was also about 13% faster in this particular test, although that part should not be treated as a general rule.

The dependable benefit of lazy iteration is bounded memory growth. Speed still depends on the work being done.

What “lazy” means here

An eager function prepares its complete result before the caller can use it:

function getNumbersArray(int $count): array
{
    $numbers = [];

    for ($number = 1; $number <= $count; $number++) {
        $numbers[] = $number;
    }

    return $numbers;
}

foreach (getNumbersArray(5) as $number) {
    echo $number . PHP_EOL;
}

By the time foreach starts, all five values already exist in the array. With five integers that is irrelevant. With a large CSV result, it can decide whether the process finishes at all.

A generator exposes the same sequence without first building the complete collection:

function getNumbersGenerator(int $count): Generator
{
    for ($number = 1; $number <= $count; $number++) {
        yield $number;
    }
}

foreach (getNumbersGenerator(5) as $number) {
    echo $number . PHP_EOL;
}

Calling getNumbersGenerator() returns a Generator object. Execution starts when iteration starts, pauses at yield, and resumes when the consumer asks for the next value.

The useful distinction is not how many values PHP can process. It is how many of them must be retained at the same time.

Streaming a CSV safely

file() reads a complete file into an array. fgetcsv() reads the next CSV record from an already open stream, which makes it a natural fit for a generator:

function readCsv(string $filename): Generator
{
    $handle = fopen($filename, 'rb');

    if ($handle === false) {
        throw new RuntimeException("Cannot open file: {$filename}");
    }

    try {
        while (($row = fgetcsv(
            stream: $handle,
            length: null,
            separator: ',',
            enclosure: '"',
            escape: '',
        )) !== false) {
            yield $row;
        }
    } finally {
        fclose($handle);
    }
}

foreach (readCsv('data.csv') as $row) {
    processRow($row);
}

Two details are easy to miss here.

First, the finally block closes the file after normal completion or an exception. It also handles cleanup when a generator is eventually destroyed after an early exit.

Second, passing escape: '' explicitly avoids relying on the deprecated default CSV escape behavior in PHP 8.4 and later.

This code does not make the entire PHP process use only a few kilobytes. The process still retains the stream buffer, parser state, generator frame, current record, and anything held by processRow(). What it avoids is memory growth proportional to the total number of CSV records.

That advantage disappears if the consumer immediately rebuilds the collection:

$rows = [];

foreach (readCsv('data.csv') as $row) {
    $rows[] = $row;
}

The producer is lazy, but the overall pipeline is not.

Why the usual memory benchmark is misleading

Consider this sequence:

$array = range(1, 1_000_000);
unset($array);

$start = memory_get_usage();

foreach (bigGenerator() as $number) {
    // Consume the values.
}

echo memory_get_peak_usage() - $start;

This measures the wrong thing for two separate reasons.

The first is timing: measuring current memory after a function returns tells us little about the peak reached while its array was alive.

The second is process-wide state. memory_get_peak_usage() remembers the highest point reached by the process. If the eager case runs first, a later generator case may report a peak increase of zero simply because it never exceeds the old array peak.

For the repository benchmark, I used a separate child process for every measured run. Each child resets peak accounting immediately before the workload, keeps the eager collection alive until measurement, and returns machine-readable results to the parent process.

Both modes must also prove that they did the same work. In this run they processed the same number of records and produced the same checksum.

The measured result

The input was a deterministic CSV fixture with 1,000,000 data records plus its header:

file size: 36.92 MiB;
rows processed: 1,000,001;
PHP: 8.5.9;
platform: Darwin arm64;
memory_limit: 1 GiB;
OPcache for CLI: off;
JIT: disabled;
one warm-up per mode;
ten measured runs per mode;
alternating eager/lazy execution order;
warm filesystem cache.

The two implementations produced the same checksum: 37,711,230.

Method	Median time	Time range	Median incremental peak memory
Array (eager)	0.759376 s	0.745849–0.829022 s	386.81 MiB
Generator (lazy)	0.658925 s	0.642476–0.679049 s	10.69 KiB

The memory result is the important one. The eager version materializes every parsed record as a PHP array, including all the per-element hash-table and zval overhead. The generator retains only the state needed for the current step and whatever the consumer keeps.

The timing result needs more restraint. In this run the generator completed about 13% faster, but yield is not inherently faster than array iteration. A generator suspends and resumes its execution frame for every value. On another workload, an already-built array may be faster to traverse.

Here, avoiding hundreds of MiB of allocation and cleanup outweighed the generator overhead. Different input, storage, PHP version, transformations, or consumer work can change that balance.

Generator, Iterator, or IteratorAggregate?

These APIs overlap, but I do not use them interchangeably.

I reach for a generator when the iteration is forward-only and fits naturally in one function:

function activeUsers(iterable $users): Generator
{
    foreach ($users as $user) {
        if ($user->isActive()) {
            yield $user;
        }
    }
}

The same Generator object cannot be rewound for another complete pass after it has advanced. Calling the generator function again creates a fresh generator.

I use IteratorAggregate when an object should be traversable more than once while keeping the iteration logic out of the object itself:

final class NumberRange implements IteratorAggregate
{
    public function __construct(
        private readonly int $start,
        private readonly int $end,
    ) {
    }

    public function getIterator(): Traversable
    {
        for ($number = $this->start; $number <= $this->end; $number++) {
            yield $number;
        }
    }
}

$range = new NumberRange(1, 5);

foreach ($range as $number) {
    echo $number . PHP_EOL;
}

Each foreach call asks the object for a new iterator, so this design is naturally reusable.

A custom Iterator is useful when cursor operations such as current(), next(), key(), valid(), and rewind() are part of the abstraction. It is more code, and I would not choose it merely to avoid writing a generator.

My practical rule is simple:

Situation	Default choice
One-pass stream or transformation pipeline	Generator
Reusable domain object that can be traversed	`IteratorAggregate`
Explicit cursor behavior is meaningful	`Iterator`
Random access or repeated indexed lookup	Array or another collection

Paginated APIs are lazy only between pages

A generator also works well around pagination:

function fetchCars(): Generator
{
    for ($page = 1; ; $page++) {
        $response = apiRequest('cars', ['page' => $page]);
        $items = $response['items'] ?? [];

        if ($items === []) {
            return;
        }

        foreach ($items as $car) {
            yield $car;
        }
    }
}

This prevents all pages from accumulating in one large array. It does not necessarily reduce memory to one car at a time: apiRequest() may buffer the complete HTTP response, and the decoded $items array remains alive while that page is being yielded.

The actual memory bound is therefore closer to one decoded page plus client and parser buffers. Page size still matters.

Where generators do not help

Generators are a poor fit when the consumer needs random access, frequent rewinds, sorting of the complete data set, or several passes over an expensive one-shot source.

They also cannot make a buffered source incremental. Wrapping a fully loaded array in a generator changes the interface, not the memory profile:

function pretendToStream(): Generator
{
    $rows = loadEveryRowIntoAnArray();

    foreach ($rows as $row) {
        yield $row;
    }
}

Database access has the same caveat. Yielding rows from a buffered query does not remove the driver's result buffer. To bound memory, the database driver and query mode must support incremental fetching as well.

Finally, laziness changes when errors occur. A generator function can be called successfully and then fail only after iteration begins. That is useful, but callers need to understand it.

Reproducing the benchmark

The complete benchmark, fixture generator, CSV and NDJSON examples, chart renderer, and tests are available here:

github.com/phpner/phpner-php-lazy-evaluation-demo

The recorded run can be reproduced with:

composer install
php bin/make_sample.php --rows=1000000
composer bench:record

Expect the timing numbers to move between machines and runs. The repository records the environment, row count, checksum, median, and range so those differences remain visible instead of being hidden behind one convenient number.

Conclusion

The benchmark changed how I phrase the recommendation. I do not choose a generator because yield is supposed to be fast. I choose it when the complete data set does not need to exist in memory at once.

For the measured CSV workload, that decision reduced incremental peak memory from 386.81 MiB to 10.69 KiB. The generator happened to be faster too, but memory behavior is the result I would expect to carry over to other genuinely streaming workloads.

The producer, parser, and consumer all have to participate. If any stage buffers the full data set, adding a generator somewhere in the middle will not fix the underlying problem.