I ran a PHP 8.4 compatibility scanner against the 20 most-installed WordPress plugins. It was wrong 6 times, and right about Jetpack.

I've been building a static analyzer that finds PHP 8.x incompatibilities in WordPress plugins and themes. Full disclosure up front: it's my tool, there's a link at the bottom, and this post is the validation run — not a demo.

The problem with writing a compatibility scanner is that it's trivially easy to write one that looks impressive. Flag every each(), every create_function(), every dynamic property, and you'll produce a long, frightening report for any codebase. The report will also be mostly wrong, and the first client who acts on it will break their site.

So the question I actually needed to answer was: how often does it cry wolf?

The test rig

I took the 20 most-installed plugins on wordpress.org, at their current versions, and scanned all of them. 16,663 PHP files.

The logic is simple. These plugins are maintained, widely deployed, and run on PHP 8.4 across millions of sites right now. If my scanner reports a certain fatal error in an up-to-date copy of WooCommerce, the overwhelmingly likely explanation is that my scanner is broken — not that WooCommerce is. That inverts the usual problem: instead of hunting for bugs I can't verify, every finding becomes a claim I'm obligated to disprove.

The gate I set before running anything:

Zero false "certain fatal" and zero false "silent failure" on up-to-date plugins. Every deprecation reported must point at a real line.

First run: 99 fatals

The first pass reported 99 fatal findings and 212 deprecation findings.

99 certain fatals across twenty plugins that demonstrably work is, of course, nonsense. So I read all 99 in context — the surrounding ±40 lines, plus a search for guards elsewhere in the file and in the package — along with a 57-finding sample of the deprecations. 156 hand-written verdicts.

They came out like this:

Verdict	Count	Meaning
GUARDED	103	Real legacy code, deliberately unreachable on PHP 8
TRUE POSITIVE	47	Real, correctly reported
FALSE POSITIVE	6	My scanner was simply wrong

The 103 are the interesting number. They aren't bugs in my scanner's pattern matching — the patterns matched real mysql_* calls, real mcrypt usage, real removed functions. They're bugs in my scanner's judgment. Mature plugins are full of code that cannot run on PHP 8, sitting behind a check that guarantees it never will:

if ( function_exists( 'mysql_connect' ) ) {
    // driver kept for PHP 5 hosts, inert since PHP 7
}

A scanner that reports this as a fatal error is technically describing the code accurately and practically useless.

The four things I had wrong

Each of these became a fixture — one file that breaks and one that doesn't — executed under PHP 7.4 and PHP 8.4 to prove the rule does what I claim before it goes back in the engine.

1. Severity was a lie. I had one bucket for "uses a removed symbol." But a removed function called at the top level of a file is a guaranteed crash on load, whereas the same call inside a method body only crashes if something calls that method. Those are different products for the person reading the report: one is "your site is down", the other is "your site may go down." I split off a conditional severity. Only top-level code and genuine compile errors kept the "certain fatal" label.

2. Guards were invisible. Adding guard detection — function_exists, extension_loaded, defined in the same file — moved 103 findings down to informational. This one change is most of the difference between a useful report and a scary one.

3. Nested ternaries broke on templates. PHP 8 made unparenthesized nested ternaries a compile error, so detecting them matters. My detector counted ternary depth across ?> ... <?php boundaries, which meant an ordinary view file with two separate ternaries in two separate PHP blocks looked like one illegal nested ternary. That produced two fake fatals in Wordfence's view files. Depth now resets at block boundaries.

4. Dynamic properties over-fired. PHP 8.2 deprecated writing to undeclared properties, and a naive detector flags a lot of code that's fine. Two exclusions fixed it: abstract classes, where the subclass declares the property (phpseclib, UpdraftPlus), and closures, where Closure::call() rebinds $this to an object that does declare it (WooCommerce's legacy data handler).

Second run

After the fixes, across all twenty plugins: fatal = 0, silent = 0. The gate passed.

What remained: 43 conditional, 208 deprecation, 54 info.

The part I didn't expect: Jetpack

Of those 43 conditionals, 41 traced back to deliberate legacy paths — an all-in-one-wp-migration factory guarded by PHP_MAJOR_VERSION >= 7 in a different file, mysql and mcrypt drivers in UpdraftPlus and Wordfence that PHP 8 can't reach.

Two didn't. Both in Jetpack 16.1.2, both in jetpack_vendor/automattic/jetpack-waf/src/class-brute-force-protection.php, at lines 678 and 777.

The shape is this: a deprecated wrapper declared static calls, via self::, a method that is not declared static. Reduced to its essentials:

class Shape {
    public static function deprecated_wrapper( $ip ) {
        return self::real_method( $ip );   // self:: to a non-static method
    }

    public function real_method( $ip ) {
        return 'ok:' . $ip;
    }
}

Under PHP 7.4:

Deprecated: Non-static method Shape::real_method() should not be called
statically in /app/jetpack-shape.php on line 9
ok:1.2.3.4

Under PHP 8.4:

Error: Non-static method Shape::real_method() cannot be called statically

It returns the right answer on 7.4 and throws on 8.4. In Jetpack the two affected wrappers are ip_is_whitelisted() and is_current_ip_whitelisted(), both deprecated in favour of ip_is_allowed() / is_current_ip_allowed(), and both with no guard of any kind.

I reported it upstream before publishing this post: Automattic/jetpack#51728. It's not a security issue — the code throws, it doesn't quietly let anything through — so the public tracker is the right place for it.

I want to be precise about what this is and isn't. These are deprecated wrappers. A site only hits the error if something still calls them — old code, another plugin, a theme that copied a snippet from a 2021 blog post. That's exactly why the finding is graded conditional and not certain fatal, and it's why nobody has noticed. It's also a real defect in the current release of a plugin installed on millions of sites, found by a scanner whose entire purpose in this exercise was to be proven wrong.

"Compatible" and "quiet" are not the same thing

The other thing the corpus made obvious: all twenty plugins are PHP 8.4 compatible in the sense that matters — they don't crash. All twenty are not therefore silent.

208 verified deprecations, and every single one I sampled pointed at a real line. Implicit nullable parameter types, deprecated in 8.4, throughout vendored libraries — Symfony components, Guzzle 3, php-jwt, sodium_compat, phpseclib. utf8_encode() in WooCommerce, deprecated in 8.2. Dynamic properties and out-of-order parameter defaults in UpdraftPlus, which alone accounts for 160 of them.

None of that takes a site down. All of it lands in debug.log, and on a busy site a log that fills with hundreds of notices per request is how you fail to notice the one notice that mattered.

Limits I'm not going to pretend aren't there

• Guard detection looks in the same file. A guard at the instantiation site in a different file leaves the finding at conditional — honest, but it means a human still has to triage.
• Function names passed as strings (callbacks) aren't resolved.
• Null-to-internal-function detection only sees literal nulls. I chose precision over recall.
• Files that won't parse under either grammar fall back to lexical analysis and get a parse-error finding. None occurred in this corpus.

The scan is read-only and purely static — scanned code is never executed. The only PHP that ran during any of this was my own fixtures.

The tool

I built this as a free WordPress plugin: it scans wp-content for PHP 8.0→8.4 incompatibilities, sorts them into fatal / silent failure / cosmetic, and produces a report a non-developer can read. It proposes fixes as diffs and never applies anything on its own. It's here: https://wprescue.instant-programming.com/?utm_source=devto&utm_content=etude-top20

If you maintain a plugin in the top 20 and you think one of these verdicts is wrong, I'd genuinely like to hear it — every finding above is traceable to a file and a line, and I'd rather correct the record than defend a number.

Comments

[David William]

Really solid validation approach. I especially like that you treated false positives as a first-class failure mode rather than simply optimizing for the number of issues detected.

The distinction between certain fatal and conditional is particularly important for static analysis. A removed function behind a reliable guard has very different practical implications from the same call at load time, even though a purely lexical scanner may initially treat them identically.

The fixture-based validation under PHP 7.4 and 8.4 is also a strong approach. Turning each discovered false positive into a regression fixture should make the analyzer progressively more reliable instead of just more aggressive.

Great example of how static analysis is ultimately about modeling program behavior, not just matching deprecated syntax.