Nobody runs PHPStan cold

PHPStan 2.3 came out last week with a blog post titled a leap in performance, and my timeline lit up.

Seifeddine Gmati, creator of Mago, updated the PHP Toolchain Benchmarks the same day. PHPStan on WordPress went from 55.8 to 19.0 seconds, and he gave Ondřej Mirtes, creator of PHPStan, a friendly "Amazing work". Mago itself, written in Rust, does the same project in 1.5 seconds at its fastest. A day later Tomas Votruba, creator of Rector, posted the Magento table from the same benchmark (Mago 2.9 seconds, PHPStan 44.5) with the caption "Not, even, close. But keep pushing". Then Mago shipped 1.53 and 1.54 and got faster again.

Both tables are labelled "Performance (Cold)", and both are correct. They answer a question I almost never ask, though. When did you last run PHPStan without a result cache?

The run you actually wait for

You change a few files and run PHPStan. Or you push, and CI picks up the result cache from the previous build (if you let it, more on that below). Either way, PHPStan re-analyses the files you changed and the files that depend on them. Everything else comes straight from the cache.

So I measured that too. I took wordpress-develop at level 9 and replayed 60 real WordPress commits in order, each run starting from the cache the previous one left behind. I did that for every PHPStan release from 2.1.49 to 2.3.1, with the Mago release that was current on the same day running next to it.

The interactive chart is at scode.nl/phpstan-231-benchmarks. These are the rows that matter:

	PHPStan 2.1.49	PHPStan 2.3.1	Mago 1.54.0
Cold, no Turbo	48.6 s	31.2 s	1.1 s
Cold, with Turbo		13.3 s
Average commit	6.0 s	1.8 s	1.1 s
Hot, nothing changed	2.5 s	1.7 s	1.1 s

Mago keeps no cache, so every Mago run is a full run. That's why its column repeats itself.

What the numbers say

Cold, Mago wins by a mile. It analyses all of WordPress in about a second, and I'm not going to pretend that isn't impressive. (Fun detail: the fastest Mago in Seifeddine's table is 1.20.1, from April, and 1.51.2 needs twice as long. My chart shows the same slowdown, and 1.54 is quick again.)

Now look at the average commit. That's what you actually wait for: PHPStan analysing your changes, not the whole project. It takes 1.8 seconds against Mago's 1.1. In an earlier replay of 300 WordPress commits, the median commit that touched PHP code re-analysed exactly two files. Most of the time, PHPStan checks your two files and hands you the rest from disk.

My favourite line on the chart is the green one. The average commit dropped from 6.0 seconds on 2.1.49 to 1.8 on 2.3.1. Some of that is PHPStan getting faster in general, and some of it is the result cache throwing away less.

A smarter result cache

For the last few months I kept asking why PHPStan re-analyses files that didn't change. Usually the cache threw away more than it had to, so PHPStan played it safe at your expense. The fixes that mattered most:

• #5933: a Composer package update re-analyses only the files that use that package, not your whole project.
• #6694: one file with a parse error no longer stops the cache from being saved. Together with a BetterReflection fix for classes that extend themselves, that took a "hot" run of Symfony from 42 seconds to under 3.
• #6699: a define() inside a function body no longer looks like a symbol that other files depend on. In WordPress, two commits that re-analysed 2,291 and 2,293 files now re-analyse 2 and 3. Over the 60 commits, that is the whole gap between 2.3.0 and 2.3.1: 9,696 files against 5,117.

Smaller fixes keep the cache when a composer.lock change bumps no versions (#6086), when the project moves to another directory (#6190) and when you switch Turbo on or off (#6335). #5982 makes the cache faster to save and restore.

Some fixes went the other way, because a cache that hands you stale results is worse than no cache. #6023 tracks global constants as dependencies, #6088 throws the cache away when a package changes the DI container, and #6698 fixes a return type that went stale after an edit.

Two things to check today

Install PHPStan with Composer. Turbo, the native extension behind a big part of the 2.3 speedup, ships in the Composer package next to phpstan.phar. If your CI or your tooling downloads only the phar, you run without Turbo and nothing tells you. On Symfony that's 48.8 seconds cold instead of 24.7. So install PHPStan with Composer to get the Turbo speedup. Starting with the next release, PHPStan warns you when Turbo is missing (#6719).

Keep the result cache between CI runs. Cache PHPStan's tmpDir the way you cache vendor/. The average-commit and hot rows in the table above depend on it.

So, should you switch?

Not for speed, I'd say. On a normal commit, the difference is less than a second. On the other side of the scale sits everything PHPStan already gives you: Larastan, the Symfony and Doctrine extensions, your own rules, your baseline, and years of type inference your codebase is tuned to. That's a lot to rebuild to save 0.7 seconds per commit, and PHPStan isn't done getting faster.

Mago is a great project, and I like that the two keep pushing each other. So here's one request for everyone who publishes these tables, Seifeddine and Tomas included: next to the cold row, add a row for a run that starts from the previous commit's cache. A cold run is a fine stress test, but it isn't your Tuesday afternoon.

I did most of this work pair-programming with Claude, which is good at the parts of benchmarking I'd rather skip, like interleaving runs and distrusting any single number. Measured on an Apple M4 Pro with 14 cores and PHP 8.5. PHPStan analyses wordpress-develop src and tests at level 9. Mago runs analyze with strict settings. Mago and PHPStan check different things, so the Mago line is a reference, not a like-for-like comparison. All runs and the method are on the benchmark page.