DEV Community: Seppe Gadeyne

Patch PHP2Shell on PHP 7.4 and 8.1 compiled from source, without a panel

Seppe Gadeyne — Sun, 25 Oct 2026 07:56:38 +0000

PHP2Shell is fixed in PHP 8.2.35, 8.3.36, 8.4.27 and 8.5.12. If your server runs PHP 5.6 to 8.1 that you or your host compiled from source, there is no release to install. Your 7.4.33 is the last 7.4 release ever, and it is vulnerable. This guide shows how to patch that PHP in place with three scripts. The first reads how your PHP was built. The second rebuilds it with the fix on another machine. The third swaps the new binaries in, with checks and a rollback.

No control panel is needed, no Docker, and nothing is compiled on the server itself.

Earlier this week I wrote about patching PHP 5.6 to 8.1 on our RunCloud servers. Several readers asked the same question: what if I'm not on RunCloud? This article is the answer. It covers the generic mode of the same repository, seppegadeyne/php2shell-runcloud-builds, which works for any PHP that reports its own build flags.

Is this guide for your server?
First: turn on the mitigation today
What the fix is and where it comes from
Step 1: record how your PHP was built
Step 2: build the patched PHP on another machine
Step 3: check, swap and roll back
Verify that the server runs the new binary
What the scripts refuse to do
FAQ

Is this guide for your server?

Run this on the server:

php -i | grep -E '^(PHP Version|Server API|Configure Command)'

You are in the right place if all of these are true:

The version is 5.6 to 8.1, or an 8.2 to 8.5 release older than the fixed ones.
Configure Command shows a line that starts with './configure'. That means PHP was compiled from source: into /usr/local, into /opt, or by a hosting setup that did it for you years ago.
The server runs Ubuntu 16.04, 18.04, 20.04, 22.04 or 24.04. The build is automated for those releases only.
PHP runs as PHP-FPM, CGI or the CLI.

You are in the wrong place if PHP comes from a distribution package. Ubuntu's own php8.1, Debian's packages and Ondřej Surý's PPA do not print a configure command, and their vendors ship their own fixes. Update those with apt-get install --only-upgrade 'php*' and reload PHP-FPM. The collect script below also checks this for you: for a binary that belongs to a package, it searches the package changelog for the two CVE numbers.

Apache's mod_php and LiteSpeed's lsphp are out of scope too. The scripts report them and skip them. Move mod_php sites to PHP-FPM first, and patch lsphp through LiteSpeed's own packages.

First: turn on the mitigation today

Do this today, before you build anything. Every known PHP2Shell trigger is a multipart/form-data request, and the stopgap that hosts and panels rolled out on October 21 turns off the session upload-progress feature:

; for example /usr/local/etc/php/conf.d/99-php2shell.ini
session.upload_progress.enabled = Off

A new ini file does nothing until PHP-FPM reloads. A master that has been running for weeks still uses the old settings in every worker. Test the config, then reload:

/usr/local/sbin/php-fpm -t && systemctl reload php-fpm

Use your own binary path and service name. Then confirm it through a real request. A pool can load a different php.ini than the CLI, so php -i on the command line proves nothing about FPM. Put a temporary file in a site's web root that prints ini_get('session.upload_progress.enabled'), request it, and delete it right away. It should print an empty string.

That is a mitigation, not a fix. The rest of this guide is the fix.

What the fix is and where it comes from

PHP2Shell is two bugs, CVE-2026-109618 and CVE-2026-109619. They allow remote code execution without authentication against PHP-FPM. Sansec has a good technical write-up, and the php.net homepage lists the fixed releases.

The PHP project only released fixes for 8.2 and newer. But the patches for older branches exist:

PHP branch	Built as	Patch
5.6	5.6.40	Ondřej Surý's port from his Debian source packages
7.0 to 7.3	7.0.33, 7.1.33, 7.2.34, 7.3.33	Surý's port, plus a `ZSTR_MAX_LEN` backport by Remi Collet
7.4, 8.0, 8.1	7.4.33, 8.0.30, 8.1.34	the PHP project's own backport branches in php-src
8.2 to 8.5	8.2.35, 8.3.36, 8.4.27, 8.5.12	none: the official fixed release

The repository has each patch as a file in patches/, with its origin and authors.

What the patch changes: on an out-of-memory error, PHP now ends the worker with _exit(1) instead of unwinding and running more code. The upload-progress callback is switched off in code, and request state is reset harder between requests. PHP-FPM starts a fresh worker, so sites keep working.

One thing the patch leaves alone matters a lot here: the extension API number. A 7.4 extension built years ago for 7.4.21 still loads in the patched 7.4.33. So you only replace the PHP core. The server's own redis.so, imagick.so or ioncube loader stays where it is.

That is also why the build stays inside your branch. A server on 7.4.21 gets 7.4.33 with the patch, not 8.1. Moving to a newer branch is a migration project; this is a security fix.

Step 1: record how your PHP was built

On the server, as root. The script reads and changes nothing:

mkdir -p /root/php2shell && cd /root/php2shell
RAW=https://raw.githubusercontent.com/seppegadeyne/php2shell-runcloud-builds/main/generic
curl -fLO $RAW/collect.sh
curl -fLO $RAW/deploy.sh      # needed in step 3
bash collect.sh

Read both scripts before you run them as root. They are plain Bash, a few hundred lines in total.

Without arguments, collect.sh checks the usual paths: /usr/local/bin, /usr/local/sbin and /usr/local/php*, /usr/bin and /usr/sbin, one or two levels under /opt, Remi's collections, cPanel's ea-php, Plesk and LiteSpeed. If your PHP lives elsewhere, name the binaries yourself, and name all of them, because the search is then skipped: bash collect.sh /srv/php74/bin/php /srv/php74/bin/php-cgi /srv/php74/sbin/php-fpm.

Output from our test server, a PHP 7.4.33 compiled into /usr/local on Ubuntu 20.04 (host name changed):

found: /usr/local/bin/php  PHP 7.4.33 (Command Line Interface)
found: /usr/local/bin/php-cgi  PHP 7.4.33 (CGI/FastCGI)
found: /usr/local/sbin/php-fpm  PHP 7.4.33 (FPM/FastCGI)
profile: /root/php2shell/php2shell-profile-web1-20261025-072514.tar.gz (3 binaries)

The profile is a small tarball. For each binary, it holds:

the version, the SAPI and the configure command
the extension API number and the built-in modules
the php.ini paths and the extension directory
the linked libraries and the SHA-256
the owning package, if any

It also stores /etc/os-release, so the build machine knows which Ubuntu to build for.

Watch the output for two kinds of notes. links libraries outside /lib and /usr/lib means your PHP uses, for example, its own OpenSSL in /opt. You will need to hand that library to the build in step 2. NOTE: ... mod_php or SAPI 'LiteSpeed' means a binary this method cannot replace. On LiteSpeed, the php CLI next to lsphp can be rebuilt, but the lsphp handler that serves your sites stays as it is.

Copy the profile to your build machine.

Step 2: build the patched PHP on another machine

The build machine is any Linux x86_64 box: a workstation, a spare VM, a CI runner. It needs unshare from util-linux 2.38 or newer with user namespaces enabled, newuidmap from the uidmap package, curl, tar and xz. Ubuntu 24.04, Debian 12, Fedora and Arch qualify; Ubuntu 22.04 ships an older util-linux. It also needs about 5 GB of disk per Ubuntu release and a subordinate ID range for your user. No root, no sudo, no Docker:

grep "^$USER:" /etc/subuid /etc/subgid   # must print a range in both files
# if not: sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $USER

git clone https://github.com/seppegadeyne/php2shell-runcloud-builds.git
cd php2shell-runcloud-builds
generic/build.sh ~/php2shell-profile-web1-20261025-072514.tar.gz

The script unpacks the official ubuntu-base root filesystem for your server's release and enters it as a rootless chroot. So the result links against the same glibc, OpenSSL and ICU as your server. Inside, it installs the build dependencies of that release's own PHP source package, plus a list of libraries that configure flags often ask for. It checks the ubuntu-base image, the php.net tarball and the patch against SHA-256 sums pinned in the repository, applies the patch, and runs your server's configure command word for word.

The first lines show the plan:

plan b1 /usr/local/bin/php: PHP 7.4.33 -> 7.4.33 (php-7.4-php2shell.patch) job j92646cb03e
plan b2 /usr/local/bin/php-cgi: PHP 7.4.33 -> 7.4.33 (php-7.4-php2shell.patch) job j92646cb03e
plan b3 /usr/local/sbin/php-fpm: PHP 7.4.33 -> 7.4.33 (php-7.4-php2shell.patch) job j92646cb03e
DEPS_OK (cached)
=== j92646cb03e: PHP 7.4.33 for b1 b2 b3 (patch: php-7.4-php2shell.patch)
patch applied: php-7.4-php2shell.patch
BUILD_OK j92646cb03e PHP 7.4.33
OUT .../out/generic/web1/php7.4.33-php2shell-j92646cb03e.tar.gz

Three binaries with the same version and configure command make one build job. Two PHP versions on one server make two jobs and two tarballs. With the root filesystem already set up, this run took 1 minute 34 seconds on a 16-core desktop. A first run on a new Ubuntu release takes longer, because it downloads and installs the build dependencies once.

The build stops instead of guessing in these cases:

The patch does not apply cleanly or would need fuzz.
The extension API of the build differs from the server's.
A built-in module is missing in the build. A site would lose a function.
The server loads a shared opcache.so but the configure command builds none. The 8.1 fix also changes opcache, so the old file would leave unpatched code running. The fix is to add '--enable-opcache=shared' to bin/<id>/configure.txt in the profile, pack it again and rerun the build.

If collect reported libraries outside /lib and /usr/lib, pack them from the server (for example tar -czf extra.tar.gz /opt/openssl-1.1) and pass them with --extra extra.tar.gz. The build unpacks them at the same path inside the chroot. That build root is reused for later builds on the same Ubuntu release, so keep it for one kind of server, or delete work/rootfs-generic-<codename> before you build for another.

Copy the tarball and SHA256SUMS from out/generic/<host>/ to /root/php2shell on the server. Use the same commit of the repository for deploy.sh and the build; both come from main in this guide.

Step 3: check, swap and roll back

Back on the server, as root, in /root/php2shell:

sha256sum -c --ignore-missing SHA256SUMS
bash deploy.sh php7.4.33-php2shell-j92646cb03e.tar.gz check      # test only
bash deploy.sh php7.4.33-php2shell-j92646cb03e.tar.gz            # swap
bash deploy.sh php7.4.33-php2shell-j92646cb03e.tar.gz rollback   # undo

Look for the OK line after sha256sum -c; the old coreutils on Ubuntu 16.04 can exit 0 without checking anything. Then always run check first. It changes nothing and runs the tests from points 1 to 3 below; only the PHP-FPM config test in point 7 needs a real deploy:

check: new CLI loads 37 modules, no warnings
CHECK OK (j92646cb03e)

Here is what deploy.sh does, in this order:

It refuses a server that changed. Every binary it will replace must still have the SHA-256 from the profile. If someone updated PHP after you ran collect, the deploy stops: collect and build again.
It starts the new CLI with your real configuration. That is your own php.ini, conf.d files and shared extensions. A CLI that does not run, or that prints a warning or error while it loads them, aborts the deploy. This catches an extension that does not load. Compare the module count it prints with php -m | grep -c . on the old binary: they should match. The test uses the CLI's configuration; an FPM pool with its own php.ini is only tested when PHP-FPM restarts.
It tests the fix itself (5.6 to 8.1, the patched branches). It runs a script with an 8 MB memory limit that allocates 64 MB and registers a shutdown function. The patched binary must exit without running that function.
It backs up the originals once to /root/php2shell-generic-backup/<job>/. A second deploy never overwrites that backup.
It swaps by rename. Running processes keep the old file until they restart, and no process ever sees a half-written binary. The shared extensions from the PHP core, such as opcache.so, are replaced too. That happens in the extension directory your server uses, and only for files it already has.
It holds the owning package, if a package owns the binary. Then apt-get upgrade cannot quietly put the unpatched version back.
It restarts PHP-FPM. It finds every running systemd service whose main process runs a replaced binary, tests its config with the service's own --fpm-config, and restarts it. If a config test or a start fails, it puts the original files back and restarts the same services. If one of them still does not come up, the rollback says so and exits with an error: check systemctl status for that unit.

On our test server, the deploy and the rollback looked like this:

check: new CLI loads 37 modules, no warnings
swapped: 3 binaries, 1 shared extensions from the core
no running PHP-FPM service uses these binaries (nothing restarted; restart it yourself if it runs outside systemd)
DEPLOY OK (j92646cb03e): 7.4.33, patch php-7.4-php2shell.patch
ROLLBACK OK (j92646cb03e)

The test chroot has no systemd, hence the "nothing restarted" line. On a normal server you see restarted: <your-fpm-service> instead. If your PHP-FPM runs under supervisord, runit or a script in /etc/rc.local, restart it yourself after the swap. Until you do, the old master keeps running the old binary.

After the rollback, all files had the same SHA-256 as before the deploy.

Verify that the server runs the new binary

php -v cannot tell a patched 7.4.33 from an unpatched one, so it proves nothing. Compare file hashes instead. Use your own unit name and paths:

# the file on disk and the binary of the running FPM master
sha256sum /usr/local/sbin/php-fpm /proc/$(systemctl show -p MainPID php-fpm | cut -d= -f2)/exe
# the file in the tarball
tar -xzOf php7.4.33-php2shell-j92646cb03e.tar.gz ./usr/local/sbin/php-fpm | sha256sum

All three hashes must match. If the /proc hash differs, the master still runs the old binary: restart PHP-FPM.

Then the behavior test, the same one the deploy ran for 5.6 to 8.1:

/usr/local/bin/php -d memory_limit=8M \
  -r 'register_shutdown_function(function(){echo "shutdown-ran\n";}); $a=str_repeat("x", 64*1024*1024);'
echo $?   # 1, and "shutdown-ran" is not printed

Finally, the sites. Before the deploy, request every site through the local web server and save the status codes:

for d in example.com shop.example.com; do
  printf '%s %s\n' "$d" "$(curl -sk -o /dev/null -w '%{http_code}' --resolve "$d:443:127.0.0.1" "https://$d/")"
done > before.txt

Run the same loop into after.txt after the deploy and diff before.txt after.txt. On our RunCloud servers, this comparison is what told us nothing had changed for the sites.

What the scripts refuse to do

The scripts are careful on purpose, so it helps to know where they stop:

Only Ubuntu 16.04 to 24.04 is automated. For Debian, Rocky or AlmaLinux, you can build by hand in a matching root filesystem with generic/chroot-build.sh. generic/chroot-deps.sh lists the libraries to install, with Ubuntu package names. The swap is then manual too: deploy.sh only takes tarballs from build.sh.
No distribution packages. If the binary has no configure command in php -i, the build skips it and writes the vendor's fix status to SKIPPED.txt.
No mod_php and no lsphp.
No other branch. A server on 7.2 gets a patched 7.2.34, not 7.4.
No restart outside systemd. Under supervisord, runit or an init script, restart PHP-FPM yourself and check the /proc hash below.

And one honest limit: the RunCloud builds from my earlier article run on our production servers, but this generic mode does not yet. We tested it in chroots on our build host:

a PHP 7.4 compiled into /usr/local on Ubuntu 20.04, with a shared opcache.so and php-cgi
a PHP 8.1.30 on Ubuntu 22.04, built as 8.1.34 with the patch, including a rebuilt opcache.so
Ubuntu 22.04's own php8.1 package, which it skipped correctly

Both builds went through the full cycle: collect, build, check and deploy. PHP-FPM answered a FastCGI request with the new binary, the out-of-memory test exited with 1, and the rollback restored the original hashes. Run check first, and try one server before the rest. If something breaks on your setup, open an issue with the collect.sh output.

A patched PHP 7.4 is a way to survive this bug, not a plan. These builds get no further updates. The next PHP bug is your job again, and Ubuntu 16.04 to 20.04 are past standard support. Keep session.upload_progress.enabled = Off in place: it also covers CVE-2026-115560, a separate session bug announced on October 23 that these builds do not fix. And plan the move to a supported PHP on a supported OS.

FAQ

Does php -v show that the patch is installed?
No. A patched 7.4.33 and an unpatched 7.4.33 report the same version. Compare the SHA-256 of the running binary with the file in the tarball, and run the out-of-memory test above.

Will my existing PHP extensions keep working?
Yes, within the same branch. The patch does not change the extension API number, and the deploy aborts if the new CLI prints a warning while it loads your extensions. Only opcache.so and the other extensions built from the PHP core are replaced.

Can I build on the server itself?
You can, if it has the disk space and user namespaces. But a separate machine keeps compilers and build dependencies off the server, and the chroot matches the server's Ubuntu release anyway.

Is session.upload_progress.enabled = Off enough on its own?
It is the stopgap everyone recommends, but it is a setting. A config that sets it back, or a pool that loads another php.ini, removes the protection without a warning. The patch fixes the code.

What about PHP from Ondřej Surý's PPA or Ubuntu's own packages?
Update the package. Those vendors ship their own fixes, and collect.sh reports whether the installed package's changelog already mentions the two CVE numbers.

Claude refused to finish patching my servers. My local LLM took over

Seppe Gadeyne — Thu, 22 Oct 2026 21:33:35 +0000

On October 21 and 22, I spent two long days patching our own servers. PHP had announced an emergency release for two critical remote code execution bugs, now known as "PHP2Shell". We run seven RunCloud servers with about 30 WordPress sites, and several of them are on PHP versions that no longer get official fixes.

I did most of the work together with Hermes, my AI agent, running Claude Opus 5.5. It went well until it didn't. In the middle of the job, the cloud model stopped answering. My local model, Qwen3.8-Flash-Next on my own workstation, picked up the same session and finished the work.

This post covers what happened, why the cloud model stopped, and why I now see a local model as a real backup.

The job

The work came in phases over two days:

Patch PHP on all servers. Where RunCloud no longer ships builds (PHP 5.6 to 8.1 on old Ubuntu releases), we rebuilt the PHP binaries ourselves with the upstream fixes. I wrote up how we patched PHP 5.6 to 8.1 for PHP2Shell. The builds and scripts are now public in php2shell-runcloud-builds, including a generic mode for servers that don't run RunCloud.
Clean up and restore a handful of WordPress sites that had been compromised. The cause turned out to be an old copy of The Grid, a plugin bundled with a Themeco theme that we can't update without breaking the site layouts.
Make a full inventory of every plugin on every site, look up known vulnerabilities for the exact installed versions, and backport the fixes into the old plugin code by hand. For these sites, a plugin update is not an option for the same reason.

Phase 3 is where it got stuck.

"Your request was not processed"

The agent had built the inventory: 32 sites, 206 plugins, 300 plugin/version pairs. It had matched them against the free WPVulnerability database and was checking access logs for suspicious requests to wp-content/uploads.

Then every message I sent got the same reply:

Your request was not processed. Send it again if you still want me to carry it out.

I tried "continue", "try it another way", and "try again, differently". The reply was the same each time. When I asked where it got stuck, it answered normally. It said it had not executed anything in those turns and could not see why. Its guess was a context length issue or an outage.

It wasn't either of those. The Hermes log had the real reason:

2026-10-22 14:49:28 WARNING [20261021_095417_816396] agent.conversation_loop:
Model declined to respond (finish_reason=content_filter).
model=claude-opus-5-5[1m] ...
refusal=This request triggered restrictions on violative cyber content and was
blocked under Anthropic's Usage Policy.

There were five of these in that session between 14:49 and 16:42, and one more in a separate session where I asked for a client report about the same breach. That report session was actually the first one I moved to the local model, and it finished the report there.

What actually happened

Claude Opus 5.5 runs behind safety classifiers. When one fires, the raw API returns a normal response with stop_reason: "refusal" and a category. Hermes logged it as finish_reason=content_filter, without the category field, but the refusal text points to the cyber category. Anthropic's refusals and fallback docs describe it like this: the category covers requests that "could enable cyber harm, such as malware or exploit development", and "benign cybersecurity work can also trigger this category."

That second sentence was us. Look at what was in the context window at that point:

Access log lines with requests for code-execution.php in an uploads folder and ../../wp-config.php path traversal attempts
A list of 47 serious CVEs: remote code execution, file upload, SQL injection, privilege escalation
Diffs between vulnerable and fixed plugin versions, such as maybe_unserialize() being replaced with unserialize( ..., array( 'allowed_classes' => false ) )
Earlier in the session: forensics on real web shells and backdoor admin accounts

To a classifier, "read the vulnerable code path and figure out exactly how the bug works" looks the same whether you want to patch it or exploit it. My reading is that the classifier looks at the request as a whole, including the tool output the agent had just read. My message was "continue". The context was the problem.

A few details made it harder to see:

The refusal was invisible to the model. The agent got a short placeholder in place of its own turn. It didn't know it had been refused, so it couldn't tell me, and it guessed wrong about the cause.
Retrying doesn't help. The docs say so too: "Re-sending a refused request to the same model usually earns another refusal." Each retry sent the same context.
It wasn't consistent. After a non-technical question, the session worked again. The next phase (an nginx rule that blocks PHP execution in uploads on all 32 sites) went fine. Then, once it was reading Gravity Forms diffs, the next refusal came at 16:42.

To be fair: this isn't a bug, and I understand why the classifier exists. The docs also describe a server-side fallbacks option that retries a refused request on a different Claude model. My setup goes through a subscription-based provider in Hermes that doesn't use it, so for me the session just stopped. If you build on the API directly, check whether your refusal category has a recommended fallback model, and log refusals as their own event. A refusal is an HTTP 200, so error-rate monitoring won't see it.

Switching to the local model

My workstation is a Ryzen 9 9950X3D with an RTX 5090 and 128 GB of RAM. I already run several large open models on it. I wrote about Qwen3.8-Flash-Next with Strata and a 5-bit quant and compared four local models side by side.

Hermes treats the local server as just another OpenAI-compatible provider:

providers:
  strata-qwen:
    name: Strata Qwen3.8-Flash-Next (UD-Q5_K_XL)
    base_url: http://127.0.0.1:8081/v1
    default_model: strata
    models:
      strata:
        context_length: 524288

The switch took two steps:

Free the GPU (a background job was using it) and start the Strata server with the Unsloth UD-Q5_K_XL quant. That was at 17:22.
In the same Hermes session, switch the model with /model and type (in Dutch): "Continue (now with a different model, hopefully no restrictions this time)."

Hermes keeps the session history, so Qwen picked up the full context: the inventory, the CVE list, the research that Opus and its subagents had already done, the plugin sources already downloaded, the earlier decisions, and my standing rules (no plugin updates, back up first, roll back on any error).

What Qwen finished

From 17:45 to 18:38, Qwen made 66 model calls. The prompt grew from about 121k to 212k tokens, well within the 512k context. Once the prompt cache was warm, turns took 3 to 20 seconds. About 20 turns missed the cache, mostly while it was writing the patcher, and took 70 to 115 seconds each. That's slower than the cloud, but it kept going.

In that time it:

Finished filtering the 47 serious CVEs (started under Opus) down to the ones that work without a login, and wrote down why each of the others was skipped
Wrote a patcher (patch_plugins.py) with exact code anchors, a HERMES-PATCH marker, and CRLF handling, and tested it locally on copies of each affected version
Where an upstream fix was available, took it from the official release by diffing the vulnerable and fixed versions, instead of writing its own
Wrote a deploy script with a backup per server, php -l with each site's own PHP version, and automatic rollback
Patched 48 plugin copies (39 files) across our servers: Gravity Forms, Broken Link Checker, MonsterInsights, Media Library Assistant, WooCommerce, WPForms, Relevanssi, StoryChief, and a product tabs plugin. No rollbacks were needed, and all 32 sites still returned 200.
After I asked for it, moved MySQL to listen on 127.0.0.1 only on the five servers that still listened on all interfaces. On two servers, the real setting turned out to be in a RunCloud file (mariadb.conf.d/99-server.cnf) that overrode the drop-in it had added first. It noticed, found the right file, and fixed it there.
Did a final read-only check across all servers when I asked it to verify everything, and explained a monitoring alert as the short MySQL restart it was

It wasn't flawless. The patcher had two bugs: \$ in Python replacement strings, and a slug comparison that missed folder names with version numbers. It found both in its own local test run and fixed them. One count in its final summary (56 patched plugin copies) didn't match the earlier report (48), and I haven't fully reconciled that difference yet. Those are the kinds of things you check anyway, whatever model you use.

What I take away from this

A local model is insurance. I don't run Qwen locally because it's better than Opus. For most work, I still use the cloud model. But the cloud model can stop on you for reasons you don't control: a classifier, an outage, a rate limit, a policy change. When that happens halfway through patching live servers, "come back later" isn't a good option. A local model with the same tools, the same agent, and the same session history means you can keep going.

A few practical points if you want this setup:

Use one agent framework for both. The handover only worked because Hermes kept the session and tools the same. The model changed and nothing else did.
Keep the local model ready. Mine loads its 92 GiB of weights in under a minute, once I've stopped the other job on the GPU. Know your start script and test it now and then.
Pick a model that handles long tool-calling sessions. A 200k-token agent context with dozens of shell calls is very different from a chat benchmark. In my own tests, Qwen3.8-Flash-Next with Strata was the fastest and most reliable model on this hardware.
Check the logs when something stops. "Your request was not processed" told me nothing. finish_reason=content_filter told me what had happened. Look for refusals separately from errors.
Verify the work the same way. Backups, linting, before-and-after HTTP checks, and rollback scripts did their job no matter which model wrote the patch.

I expect cloud models to stay strict in some areas, and that's their call to make. Security work on your own servers will always look a bit like attack work. Having a capable model on your own hardware means that decision doesn't stop your work.

Patching PHP 5.6 to 8.1 for PHP2Shell on old Ubuntu servers by rebuilding RunCloud's PHP

Seppe Gadeyne — Thu, 22 Oct 2026 21:10:16 +0000

On October 21, 2026, PHP shipped fixes for PHP2Shell, but only for PHP 8.2 and newer. Our hosting panel, RunCloud, only patches those versions too. Most of our older WordPress sites run on PHP 5.6 to 8.1, on Ubuntu releases as old as 16.04. So that evening we rebuilt PHP ourselves: every old version, for every Ubuntu release on our servers, with RunCloud's own build flags, and swapped in only the patched binaries. This article shows how, what broke on the way, and how we checked that nothing changed for the sites.

Everything below was done on October 21, 2026: builds and tests on our own workstation, deploys and checks on our servers. Where a fact comes from someone else, I link the source. Server and client names are left out.

Update, October 24, 2026: all 25 builds, the patches, RunCloud's configure commands and the build scripts are now public: seppegadeyne/php2shell-runcloud-builds. You can download a tarball for your Ubuntu release or rebuild it yourself with one command. Details in Use our builds, or build them yourself below.

Later that day: not on RunCloud? The repository now also has a generic mode for a PHP that you or your host compiled from source. It reads the configure command from your server, builds the newest release of the same PHP branch with the fix, inside your server's Ubuntu release, and swaps it in with the same checks. See Not on RunCloud: the generic mode. The full step-by-step guide is in a separate article: Patch PHP2Shell on PHP 7.4 and 8.1 compiled from source, without a panel.

The problem: two critical bugs and a panel that stops at 8.2

PHP2Shell is two bugs, CVE-2026-109618 and CVE-2026-109619. They allow remote code execution without authentication, PHP-FPM is affected, and every known trigger is a multipart/form-data request. The announcement on the php.net homepage lists the fixed releases: 8.2.35, 8.3.36, 8.4.27 and 8.5.12. Sansec has a good technical write-up and timeline.

Our setup at Straffe Sites: seven servers managed with RunCloud, running Ubuntu 16.04, 18.04 (three servers), 20.04 (two) and 22.04. RunCloud installs its own PHP builds under /RunCloud/Packages/phpXXrc from a per-release apt repository, release.runcloud.io. That repository told us most of what we needed to know:

RunCloud only patches 8.2 and newer, as their community statement says.
The 18.04 repository stopped getting builds at 8.3.17 and 8.2.27, so even the supported branches would not get a fix there.
The 16.04 repository tops out at PHP 8.0. No branch there would ever be patched.

Many of our older WordPress sites are pinned to PHP 7.3 or 7.4 on purpose: old page-builder themes and plugins that break on PHP 8. Moving them is a project, not an evening. We counted 71 FPM pools, and 62 of them had no official patch coming.

First: make sure the mitigation is actually active

Before building anything, check the stopgap. RunCloud dropped /etc/phpXXrc/conf.d/99-cve-2026-109618.ini on every server in the morning:

session.upload_progress.enabled = Off

The PHP-FPM masters had been running since days before that, so the setting was not active in a single worker. A new conf.d file only applies after FPM reloads. We tested the config and reloaded every running FPM service, along these lines:

for s in $(systemctl list-units --type=service --state=running --no-legend | awk '/php[0-9]+rc-fpm/{print $1}'); do
  v=${s%-fpm.service}
  /RunCloud/Packages/$v/sbin/php-fpm -t -y /etc/$v/php-fpm.conf && systemctl reload "$s"
done

Then we confirmed it through a real pool, for every PHP version that had a site that would serve a stray PHP file (all but two; for those we checked the ini file and the worker restart): a temporary PHP file that prints PHP_VERSION and ini_get('session.upload_progress.enabled'), fetched through the local nginx with curl --resolve, and deleted right after. If your host says "we applied the mitigation", ask whether they also reloaded PHP-FPM.

That is a mitigation, not a fix. So, the real thing.

Where the patches come from

Nobody ships PHP 5.6 to 8.1 builds for Ubuntu 16.04 to 20.04 anymore. But the source patches exist:

PHP 7.4, 8.0 and 8.1: the PHP project published minimal backports in php-src branches named PHP-7.4-CVE-2026-109618-109619 (and the same for 8.0 and 8.1). For 7.4 it is a single commit, 712e60f9ed "Avoid state corruption via bailout" by Arnaud Le Blanc, co-authored by Ilija Tovilo; for 8.0 82f8f31b16, for 8.1 1d7dec178a.
PHP 5.6 to 7.3: Ondřej Surý ported the same fix for his packages.sury.org packages. His builds only target Ubuntu 22.04, 24.04 and Debian, but the Debian source packages contain the patches as plain files in debian/patches/. For 7.0 to 7.3 the fix needs one earlier backport first, backport ZSTR_MAX_LEN by Remi Collet, a macro in zend_string.h.

PHP	Patches applied on top of the php.net tarball
5.6.40	Surý `0261-Avoid-state-corruption-via-bailout`
7.0.33	Surý `0211-backport-ZSTR_MAX_LEN` + `0230-Avoid-state-corruption-via-bailout`
7.1.33	Surý `0167` (ZSTR_MAX_LEN) + `0186` (bailout)
7.2.34	Surý `0135` (ZSTR_MAX_LEN) + `0159` (bailout)
7.3.33	Surý `0107` (ZSTR_MAX_LEN) + `0139` (bailout)
7.4.33	php-src `712e60f9ed`
8.0.30	php-src `82f8f31b16`
8.1.34	php-src `1d7dec178a`

What the patch does, in short: on an out-of-memory error the worker now calls _exit(1) instead of unwinding with longjmp (FPM simply starts a fresh worker), the session upload-progress callback is switched off in code (if (!PS(rfc1867_enabled) || true)), and request state is reset harder in main.c, SAPI.c, rfc1867.c and php_variables.c. It does not change ZEND_MODULE_API_NO. That matters: the existing compiled extensions on the server can keep loading without a rebuild. In our tests every build loaded the server's full set of extensions, with the same module count and no startup warnings.

Every patch applied with patch -p1 against the official tarballs without a single failed hunk; some hunks needed a line offset, none needed fuzz. PHP 5.5 was the exception: hunk 10 of the 5.6 patch fails in Zend/zend_alloc.c. No server used 5.5, so we disabled it instead of patching (more on that below).

Build on the same Ubuntu, without root or Docker

A PHP binary built on a modern system will not run on Ubuntu 16.04: newer glibc, newer OpenSSL, newer ICU. The binary has to be built against the same libraries the server has. The cleanest way is to build inside the same Ubuntu release.

We used the official ubuntu-base root filesystems (16.04.6, 18.04.5, 20.04.5, 22.04.5) from cdimage.ubuntu.com, checked against their SHA-256 sums, and entered them as a rootless chroot with a user namespace. No sudo, no Docker daemon:

#!/bin/bash
# enter.sh <rootfs-dir> <command...>
HERE=$(dirname "$(realpath "$0")")
R=$(realpath "$1"); shift
exec unshare --user --map-root-user --map-users=auto --map-groups=auto \
  --mount --pid --fork --kill-child "$HERE/_inner.sh" "$R" "$@"

#!/bin/bash
# _inner.sh: runs as "root" inside the new namespaces
R="$1"; shift
mount --rbind /dev "$R/dev" 2>/dev/null || true
mount -t proc proc "$R/proc"
mount -t tmpfs tmpfs "$R/tmp"
cp /etc/resolv.conf "$R/etc/resolv.conf"
exec chroot "$R" /usr/bin/env -i HOME=/root PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
  LANG=C.UTF-8 DEBIAN_FRONTEND=noninteractive "$@"

--map-users=auto needs a range for your user in /etc/subuid and /etc/subgid; without it, apt cannot switch to its _apt user. Inside the chroot, apt-get install works as usual. The 16.04 build deps were:

apt-get install -y --no-install-recommends build-essential autoconf bison re2c pkg-config \
  ca-certificates curl xz-utils patch libxml2-dev libssl-dev libsqlite3-dev zlib1g-dev \
  libcurl4-openssl-dev libgmp-dev libffi-dev libenchant-dev libsystemd-dev libedit-dev \
  libicu-dev libpcre3-dev libselinux1-dev libfcgi0ldbl

That gives each release its original toolchain:

Ubuntu	GCC	OpenSSL	ICU	PHP versions built
16.04	5.4.0	1.0.2g	55	5.6, 7.0, 7.1, 7.2, 7.3, 7.4, 8.0
18.04	7.5.0	1.1.1	60	7.0 to 7.4, 8.0, 8.1, plus 8.2.35 and 8.3.36
20.04	9.4.0	1.1.1	66	7.2, 7.3, 7.4, 8.0, 8.1
22.04	11	3.0 (and RunCloud's 1.1.1b)	70	7.4, 8.0, 8.1

On 18.04 we also built the official 8.2.35 and 8.3.36 releases, because RunCloud will not ship them for that release anymore.

Use RunCloud's own configure flags

The goal is a drop-in replacement, so the new binary must be built exactly like RunCloud's. You do not have to guess: php -i prints the full configure command.

/RunCloud/Packages/php74rc/bin/php -i | grep '^Configure Command' | sed 's/^Configure Command =>  //'

For PHP 7.4 that starts like this (shortened):

'./configure' '--build=x86_64-linux-gnu' '--prefix=/RunCloud/Packages/php74rc'
'--sysconfdir=/etc/php74rc' '--with-config-file-path=/etc/php74rc'
'--with-config-file-scan-dir=/etc/php74rc/conf.d' '--disable-cgi' '--disable-all'
'--with-openssl=yes' '--with-password-argon2' '--enable-fpm' '--enable-cli'
'--enable-pdo=shared' '--enable-mysqlnd=shared' '--enable-xml=shared' ...
'--with-fpm-systemd' '--with-gmp' '--with-libedit=shared,/usr'

Two things stand out. RunCloud builds a small core with --disable-all and ships the rest (mysqli, gd, intl, curl, and so on) as separate shared extensions. And --with-password-argon2 links against RunCloud's own libargon2 package, which you can download from the same repository for each Ubuntu release (release.runcloud.io/dists/<codename>/main/binary-amd64/Packages.gz, checked against the SHA-256 in that index) and install in the chroot with dpkg -i.

We saved the configure command per version and Ubuntu release, and one generic script built everything:

#!/bin/bash
# build-generic.sh <full-version> <rc-tag e.g. 74> <patch|none> [mode] [ubuntu codename]   (shortened)
set -euo pipefail
FV=$1; V=$2; PATCH=$3; MODE=${4:-build}; DIST=${5:-xenial}
cd /root && rm -rf php-$FV && tar -xJf /src/php-$FV.tar.xz && cd php-$FV
if [ "$PATCH" != none ]; then
  patch -p1 --dry-run -F3 < /src/$PATCH > /root/patch$V.log 2>&1 || { grep -E "FAILED|Hunk" /root/patch$V.log; exit 2; }
  [ "$MODE" = dryrun ] && { echo DRYRUN_OK; exit 0; }
  patch -p1 -F3 < /src/$PATCH >/dev/null
fi
# only needed if a patch touches configure.ac (none of ours did)
grep -q "configure.ac" /src/$PATCH 2>/dev/null && ./buildconf --force >/dev/null 2>&1 || true
eval "$(cat /src/configure-php${V}rc-$DIST.txt)" > /root/configure$V.log 2>&1
make -j"$(nproc)" > /root/make$V.log 2>&1
rm -rf /stage$V && make install INSTALL_ROOT=/stage$V > /root/install$V.log 2>&1
P=/stage$V/RunCloud/Packages/php${V}rc
strip --strip-unneeded $P/bin/php $P/sbin/php-fpm
cd /stage$V && tar -czf /src/php${V}rc-$FV-php2shell-$DIST.tar.gz RunCloud

make install INSTALL_ROOT=/stage74 installs into a staging directory with the full /RunCloud/Packages/php74rc path underneath, so the tarball unpacks to the right place.

Pitfalls we hit

gmp.h not found on PHP 5.6. On multiarch Ubuntu the header lives in /usr/include/x86_64-linux-gnu/. A symlink fixes it: ln -sf /usr/include/x86_64-linux-gnu/gmp.h /usr/include/gmp.h.
Surý's 7.0 to 7.3 patches need ZSTR_MAX_LEN. The bailout patch uses that macro, which 7.0 to 7.3 do not have. Apply the earlier backport first, then the bailout patch.
On 22.04, RunCloud's 7.4 and 8.0 link against OpenSSL 1.1.1b, which RunCloud ships as its own libssl1-rc package under /RunCloud/Packages/libssl1-rc. Ubuntu 22.04 itself only has OpenSSL 3. Install that package in the chroot and build against it, like the original.
The version number does not change. A patched 7.4 still says PHP 7.4.33. Do not use php -v to prove anything; check the behavior and the file hash (below).

Test in the chroot before anything touches a server

For each build we copied the server's real extension directory and conf.d into the chroot, put the new core next to them and checked:

ldd on every .so: no "not found" libraries.
php -m: the same number of modules as on the server (for example 67 for 7.4 on 16.04), and no startup warnings.
A smoke test: mysqli, gd, curl, mbstring and password_hash with the default algorithm.
A real PHP-FPM pool with memory_limit = 16M, driven with cgi-fcgi: one normal request, one request that allocates 64 MB, then another normal request.

The out-of-memory case shows the patch at work. Here is the patched 7.4.33 on 16.04, with a shutdown function registered:

$ php -d memory_limit=8M -r 'register_shutdown_function(function(){echo "shutdown-ran\n";}); $a=str_repeat("x", 64*1024*1024);'
Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 67108896 bytes) in Command line code on line 1
$ echo $?
1

The shutdown function never runs and the process exits with 1. In FPM, that worker dies and the master starts a new one; the third request in the pool test was served normally.

Deploy: swap only the core, keep a way back

We did not replace the whole RunCloud package. Only the files the patch touches are swapped: bin/php, sbin/php-fpm and the six extensions that are built from the core source as shared modules (mysqlnd, pdo, xml, xmlreader, xmlwriter, readline). RunCloud's other extensions and all conf.d files stay as they are. The deploy script, shortened:

#!/bin/bash
# deploy-core.sh <74|83|...> <tarball> [rollback]   (run as root on the server)
set -euo pipefail
V=$1; TB=$2; MODE=${3:-deploy}
P=/RunCloud/Packages/php${V}rc
EXTDIR=$(ls -d $P/lib/php/extensions/no-debug-non-zts-*)
BK=/root/php${V}rc-orig-before-php2shell
FILES="bin/php sbin/php-fpm"
CORE_EXT="mysqlnd.so pdo.so xml.so xmlreader.so xmlwriter.so readline.so"

# (rollback branch: put $BK back by rename, apt-mark unhold, restart FPM)

T=$(mktemp -d); tar -xzf "$TB" -C $T
N=$T/RunCloud/Packages/php${V}rc
# 1) back up the originals once, never overwrite them
if [ ! -d $BK ]; then mkdir -p $BK/bin $BK/sbin $BK/ext
  for f in $FILES; do cp -a $P/$f $BK/$f; done
  for e in $CORE_EXT; do [ -f $EXTDIR/$e ] && cp -a $EXTDIR/$e $BK/ext/$e; done
fi
# 2) the new binary must start cleanly with the real php.ini
$N/bin/php -c /etc/php${V}rc/php.ini -m >/dev/null 2>$T/err || true
grep -i warning $T/err && { echo "ABORT: startup warnings"; exit 1; }
# 3) swap by rename, so running processes keep their old inode
for f in $FILES; do install -m 755 $N/$f $P/$f.new && mv -f $P/$f.new $P/$f; done
for e in $CORE_EXT; do [ -f $N/lib/php/extensions/*/$e ] && install -m 755 $N/lib/php/extensions/*/$e $EXTDIR/$e.new && mv -f $EXTDIR/$e.new $EXTDIR/$e; done
# 4) stop RunCloud from putting the unpatched package back
apt-mark hold php${V}rc >/dev/null
# 5) service inactive (no pools)? sanity-check the binaries and exit without starting it
if ! systemctl is-active --quiet php${V}rc-fpm; then
  $P/sbin/php-fpm -n -v >/dev/null && $P/bin/php -m >/dev/null || exec bash $0 $V $TB rollback
  exit 0
fi
# 6) config test, restart (a new binary needs a restart, not a reload), roll back on failure
$P/sbin/php-fpm -t -y /etc/php${V}rc/php-fpm.conf || exec bash $0 $V $TB rollback
systemctl restart php${V}rc-fpm; sleep 2
systemctl is-active php${V}rc-fpm >/dev/null || exec bash $0 $V $TB rollback

Installed PHP versions without any pool got the same new binaries, but their FPM service was not started. Someone switching a site to that version in the panel later would otherwise land on an unpatched PHP.

apt-mark hold is the important trade-off. It keeps RunCloud's agent from reinstalling the old package, but it also means these versions get no RunCloud updates at all from now on. For 8.2 and newer on 20.04 and 22.04 we did not hold anything: RunCloud released 8.2.35, 8.3.36, 8.4.27 and 8.5.12 for those Ubuntu releases later that evening, and we installed them with apt-get install --only-upgrade.

Verify: same pages, new binary

Three checks per server:

HTTP before and after. Before each deploy we requested every domain on the server through the local nginx (curl --resolve domain:443:127.0.0.1) and saved the status code per domain. After the deploy we ran it again and diffed the two lists. No differences on any server. For a few sites per PHP version we also checked /wp-login.php and /wp-admin/ (200 and 302).
The running process uses the new file. Compare the hash of the binary on disk with the one the FPM master actually runs: md5sum /RunCloud/Packages/php74rc/sbin/php-fpm /proc/$(systemctl show -p MainPID --value php74rc-fpm)/exe.
The pool reports what we expect. The same temporary probe file as for the mitigation: PHP version, upload_progress off, SAPI fpm-fcgi.

To be sure everything survives a restart, we rebooted the six servers we had patched that night one by one just before midnight and repeated the HTTP comparison: 184 domains, no differences. The 22.04 server's 7.4 to 8.1 builds went out right after that. By the end of the night every installed PHP version from 5.6 to 8.5 on those six servers was patched; only the unused 5.5 was disabled instead. The seventh server, also on 18.04, followed the next day with the same 18.04 builds and the same checks.

The one we did not patch: PHP 5.5

One server still had PHP 5.5.38 installed, without any site using it. No patch exists, and the 5.6 patch does not apply. Removing the package with apt would also remove RunCloud's agent, because it depends on it. So we moved the php and php-fpm binaries out of the way, put the package on hold, and added a systemd drop-in with ConditionPathExists so the service cannot start.

Use our builds, or build them yourself

This article got more readers than we expected, and several of them run the same setup. So we published everything in one repository: github.com/seppegadeyne/php2shell-runcloud-builds.

The release v2026.10.21 has the 25 tarballs we deployed, with a SHA256SUMS file: 16.04 (5.6 to 8.0), 18.04 (7.0 to 8.1, plus the official 8.2.35 and 8.3.36), 20.04 (7.2 to 8.1), 22.04 (7.4 to 8.1) and 24.04 (8.1).
deploy-core.sh is the full version of the script above, in English, with the rollback branch and an extension API check.
patches/ has the eight patches from the table, unchanged, with their origin and authors.
configure/ has RunCloud's configure command per PHP version and Ubuntu release, copied from php -i.
generic/ builds a patched PHP for servers without RunCloud, from the server's own configure command (see below).
build.sh rebuilds any of them from scratch:

git clone https://github.com/seppegadeyne/php2shell-runcloud-builds.git
cd php2shell-runcloud-builds
./build.sh focal          # every version for 20.04
./build.sh xenial 56 74   # only php56rc and php74rc for 16.04

It downloads the ubuntu-base root filesystem and the php.net tarballs, checks them against pinned SHA-256 sums, fetches RunCloud's libargon2 or libssl1-rc package (checked against RunCloud's own index), and builds in the rootless chroot. A patch that does not apply cleanly, or would need fuzz, stops the build. Before publishing, we rebuilt versions for every Ubuntu release from an empty directory with this script and compared them with the deployed tarballs: same files, same linked libraries.

One detail we only wrote down while cleaning up: RunCloud does not use the same libargon2 everywhere. On 18.04, PHP 7.2 to 8.0 link RunCloud's own libargon2.so.1, while 8.1 and newer link Ubuntu's libargon2.so.0. The build script handles that per version; if you build by hand, check the original with readelf -d /RunCloud/Packages/php81rc/sbin/php-fpm | grep argon2.

The usual warning applies: these are unofficial builds, not affiliated with RunCloud or the PHP project, provided as is. Match both the PHP version and the Ubuntu release, read the script before you run it as root, and try one server first.

Not on RunCloud: the generic mode

The release tarballs only fit RunCloud. They unpack to /RunCloud/Packages/phpXXrc and link RunCloud's own libraries. But the problem is not specific to RunCloud. Plenty of servers run a PHP that someone compiled into /usr/local or /opt years ago, or a panel's own build, on an Ubuntu release that no longer gets PHP fixes. For those, the repository now has a generic/ directory. It follows the same steps as above, but it starts from your server instead of from RunCloud:

# 1) on the server (read-only): record every PHP CLI, FPM and CGI binary
mkdir -p /root/php2shell && cd /root/php2shell
RAW=https://raw.githubusercontent.com/seppegadeyne/php2shell-runcloud-builds/main/generic
curl -fLO $RAW/collect.sh
curl -fLO $RAW/deploy.sh             # needed in step 3
bash collect.sh                      # or: bash collect.sh /usr/local/sbin/php-fpm
# -> php2shell-profile-<host>-<date>.tar.gz

# 2) on the build machine (same requirements as build.sh)
generic/build.sh php2shell-profile-web1-20261024-120000.tar.gz
# -> out/generic/web1/php7.4.33-php2shell-<job>.tar.gz + SHA256SUMS + SKIPPED.txt
# copy the tarball and SHA256SUMS to /root/php2shell on the server

# 3) back on the server, as root, in /root/php2shell
sha256sum -c --ignore-missing SHA256SUMS
bash deploy.sh php7.4.33-php2shell-<job>.tar.gz check      # test only
bash deploy.sh php7.4.33-php2shell-<job>.tar.gz            # swap
bash deploy.sh php7.4.33-php2shell-<job>.tar.gz rollback   # undo

collect.sh changes nothing. It looks for PHP binaries in the usual places (/usr/local, /opt, cPanel's ea-php, Plesk, LiteSpeed, RunCloud), or takes the paths you give it. For each binary, it saves the configure command from php -i, the extension API number, the built-in modules, the linked libraries, the SHA-256 and the package that owns the file. It also saves the server's /etc/os-release. It warns about libraries linked from outside /lib and /usr/lib. What the generic mode cannot rebuild, it reports and skips: an Apache mod_php and LiteSpeed's lsphp.
build.sh builds the newest release of the same branch with that configure command, verbatim, inside the server's own Ubuntu release. A 7.4.30 becomes 7.4.33 with the patch; 8.2 to 8.5 get the fixed php.net release. Staying in the same branch keeps the extension API the same, so the server's other shared extensions keep loading. The build stops if the extension API differs or a built-in module is missing. Build dependencies come from the release's own PHP source package (apt-get build-dep), plus a list of libraries that configure flags often ask for. Libraries the server keeps in /opt go in with --extra <tarball>.
deploy.sh refuses a server that changed. Every target binary must still have the SHA-256 from the profile. The new CLI must start with the server's own php.ini without a warning, and for patched builds, an out-of-memory error must no longer run shutdown functions (the same test as above). Then it backs up the originals once to /root/php2shell-generic-backup/<job>/, swaps by rename, puts an owning package on hold, and config-tests and restarts every systemd PHP-FPM service whose main process runs a replaced binary. A failed test or start rolls everything back. An FPM outside systemd is not restarted; restart it yourself.

It leaves two kinds of builds alone on purpose. Distribution packages (Ubuntu's own php8.1, Debian, Surý) do not show a configure command in php -i. For those, collect.sh searches the package changelog for the two CVE numbers and reports whether the vendor already shipped the fix. The answer there is apt-get install --only-upgrade or a question to the vendor, not a rebuild. And the SAPIs it cannot rebuild: move mod_php sites to PHP-FPM first, and patch LiteSpeed's lsphp through LiteSpeed's own packages.

One detail came up while testing. The 8.1 fix also changes ext/opcache. If the server loads a shared opcache.so, keeping the old file would leave unpatched code in use, so the build has to produce a new opcache.so from the patched source. A configure command that builds no opcache is refused, with a hint to add --enable-opcache=shared.

Only Ubuntu 16.04 to 24.04 is automated. For another distribution, you can build by hand in a matching root filesystem with generic/chroot-build.sh; generic/chroot-deps.sh shows which libraries to install, but it uses Ubuntu package names. And unlike the RunCloud builds, the generic mode has not run on a production server yet. We tested it in chroots on our build host:

a PHP 7.4.30 compiled into /usr/local with a shared opcache.so and php-cgi on Ubuntu 20.04: collect, build 7.4.33 with the patch, check, deploy, PHP-FPM answering a FastCGI request with the new binary, out-of-memory exit 1, and a rollback to the original SHA-256
the same for a PHP 8.1.30 on Ubuntu 22.04, built as 8.1.34 with the patch, including the rebuilt opcache.so
Ubuntu 22.04's own php8.1 package, which it skipped because the changelog shows the vendor fix

So run check first, and try one server before the rest.

The whole procedure, with real output, the checks and how to verify the running binary, is in its own article: Patch PHP2Shell on PHP 7.4 and 8.1 compiled from source, without a panel.

What this does not fix

The next PHP bug. Held packages get nothing from RunCloud anymore. The scripts are in the repository now, so the next round is a rebuild, not a research project. But it stays our job. The next one is already announced: on October 23, PHP published CVE-2026-115560, a separate session bug that the October 21 releases do not fix. It only affects applications that switch session.serialize_handler to php with ini_set() before session_start(), and the mitigation is the same session.upload_progress.enabled = Off. We checked it on every PHP version on our servers. The fix comes in the next regular release; when it lands, we will rebuild.
Old operating systems. Ubuntu 16.04 to 20.04 are past standard support. A patched PHP on an unpatched OS is still an old server. Our real fix is moving the old sites to a new Ubuntu 24.04 server, where Surý's packages cover the old versions we need (5.6, 7.0, 7.3, 7.4 and 8.1) with this fix.
The mitigations stay on. session.upload_progress.enabled = Off, Cloudflare's managed rules for proxied domains, and phpMyAdmin behind an IP allowlist. A patch is not a reason to remove them.

The approach carries over to any server with its own PHP build on an older Ubuntu: same Ubuntu in a rootless chroot, the server's own configure command from php -i, upstream or Surý patches, swap only the core, hold the package, and compare the HTTP output before and after. For a self-compiled PHP, the generic mode now does most of those steps for you. The HTTP comparison is still your job.

For site owners rather than sysadmins, I wrote a plain-language version of what PHP2Shell means for a WordPress site: The PHP vulnerability and your WordPress site.

Run DeepSeek-V4.1-Flash (748B) locally with Project Maya on one RTX 5090

Seppe Gadeyne — Wed, 21 Oct 2026 15:36:02 +0000

DeepSeek-V4.1-Flash-1017 has 748B parameters. Even Project Maya's own 2-3 bit quant is 340 GB with its Engram tables, and my workstation has 128 GB of RAM. It still runs here: on one RTX 5090, with a 262K context, at about 21 tokens per second on Maya 1.1.17, and about 25 since 1.1.21. This guide shows how, with Project Maya, and what we measured, including where it is slow.

Every number below was measured on our own machine between October 19 and 22, 2026. Where a number comes from someone else, I name the source.

This article belongs to a series about large models on one consumer machine. A side-by-side benchmark of all four models compares them; the setup guides cover GLM-5.3-Flash with Project Maya, Qwen3.8-Flash-Next with Strata at 79 tokens per second and Qwen3.8-Flash-Next with EXL3 and TabbyAPI.

The model

DeepSeek-V4.1-Flash-1017 is an open-weight model from DeepSeek under the MIT license. Maya's README lists it at 748B parameters. The model card lists a context window of up to 1M tokens and recommends temperature 1.0 and top_p 1.0. Next to its experts it has two Engram tables, which Maya's setup describes as an n-gram memory that the engine reads a few rows from per token.

The model card does not ship a Jinja chat template, only a Python reference encoder. Maya's engine brings its own prompt handling, so you do not have to deal with that yourself.

Why Project Maya

Maya started as a runtime for GLM-5.3-Flash, and recent versions add a second engine, maya-ds4, for this model. The idea is the same: the most-used experts live on the GPU, the next ones in RAM, and the rest are read in place from the model files on the NVMe drive. The full model never has to fit in memory.

For this model Maya offers several file sets. From its README:

Files	Size
Maya-S, Maya's own quant (experts in 2-3 bits), "92.6% the same next token as the original"	340 GB with the Engram tables
unsloth's UD-Q8_K_XL GGUF	508 GB
LibertAIDAI's EXL3 2.9 bpw + Engram tables	414 GB
LibertAIDAI's EXL3 3.5 bpw + Engram tables	461 GB

Maya-S is the smallest by far, so on 128 GB of RAM that is the one to take. We installed it with Maya 1.1.16, which downloaded 178.2 GB of model files, 203 GB of Engram tables from unsloth's GGUF and 8 GB for the DSpark drafter. Since 1.1.17, a fresh setup downloads Maya-S as one set of six files from Maya's own repository, 339.5 GB including the Engram tables, according to the changelog; a 1.1.16 install keeps working as it is. Plan for about 350-400 GB of free NVMe space.

The machine

GPU: RTX 5090, 32 GB of VRAM, also driving the desktop.
CPU: Ryzen 9 9950X3D (16 cores, 32 threads, AVX-512).
RAM: 128 GB DDR5 as 4x32 GB at 5,600 MT/s.
Disk: one NVMe drive.
OS: Arch Linux (Omarchy), NVIDIA driver 615.78, CUDA 13.4.
Maya 1.1.16 for the first measurements, 1.1.17 (commit 8d9a9b5) for the rest, and 1.1.21 (commit 451f9cb) for the October 22 update.

Install

If you already run Maya for GLM, the same checkout can hold both models. Each gets its own engine, config and port.

git clone https://github.com/mw00/project-maya.git
cd project-maya
./maya.sh --check

Then the setup. It downloads 340 GB or more, so run it in the background and outside any memory-limited scope (see the next section):

./maya.sh --setup --plain --yes --gpu 0 \
  --model DeepSeek-V4.1-Flash-1017 --ds4-quant maya-s \
  --context 262144 --models-dir ~/Models/maya --download-model \
  --no-vision --vram-reserve-mib 3072 --port 8082 --no-start

What it did on our machine (Maya 1.1.16):

Checked the PC, installed its Python packages and fetched llama.cpp's source at a pinned commit.
Compiled the maya-ds4 engine for the RTX 5090 (sm_120).
Downloaded the four Maya-S files (178.2 GB), the Engram file (203 GB) and the DSpark drafter (8 GB) with resumable curl commands it prints first, and checked the files against their SHA-256. The download ran at roughly 40-80 MB/s here, about an hour and a half for everything.
Indexed every tensor into a pack next to the model files. The weights stay where they are.
Wrote maya-ds4.json and run-maya-ds4.sh.

The flags that matter:

--model DeepSeek-V4.1-Flash-1017 --ds4-quant maya-s picks the model and the smallest quant.
--port 8082 keeps it apart from a GLM install on 8080.
--vram-reserve-mib 3072 leaves 3 GiB of VRAM free. Our desktop runs on the same card, and with full VRAM the screen flickered.
--no-vision skips DeepSeek's vision tower (about 1 GB).

Start it outside your shell's memory limits

Unlike the GLM engine, maya-ds4 does not pin most of the RAM: it reads the model files in place, so the RAM it uses shows up mostly as page cache (after our benchmark free reported 20 GiB used and about 100 GiB available). Do not read the available number as free memory: the engine depends on that cache, and anything you start next to it pushes experts out, which then have to come from the SSD again. In practice it uses the whole machine. The load is heavy too. Two Linux defaults get in the way:

A shell inside a systemd scope with a memory limit. Our agent's terminal scopes have memory.max of 4 GiB; a big process started from there ends up in swap.
systemd-oomd, which Omarchy enables for the user's app.slice. It kills a cgroup after 20 seconds above 50% memory pressure.

So we start it in its own slice, without a memory limit:

systemd-run --user --scope --collect --slice=llm.slice \
  --unit="maya-ds4-$(date +%s)" -p MemoryMax=infinity \
  ./run-maya-ds4.sh --plain

The first start after setup also runs Maya's speed calibration (below) and took about four minutes here; a normal start took 20 seconds. Then:

curl -s http://127.0.0.1:8082/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4.1-flash", "messages": [{"role": "user", "content": "Say hello in five words."}],
       "max_tokens": 2000, "reasoning_effort": "medium"}'

The dashboard at http://127.0.0.1:8082/#monitor shows the speeds and where the experts live. Keep the server on 127.0.0.1 unless you add --api-key.

The drafter: measure it on every version

DeepSeek's model comes with a DSpark drafter that proposes a few tokens at a time for the big model to check. Whether that helps depends on your PC, so Maya measures it on the first start with the GPU to itself and keeps the faster option. Upstream, it does not run while more than 1 GiB of VRAM is in use. We raised that threshold locally to 3 GiB because the desktop runs on the same card (see the GLM article), and during setup 4.2 GiB was in use, so the calibration ran at the first start instead.

The result flipped between two versions:

Maya version	Without drafting	With DSpark	Accepted drafts	Kept
1.1.16	23.0 tok/s	20.6 tok/s	59%	no drafting
1.1.17	18.2 tok/s	21.0 tok/s	61%	DSpark
1.1.21	23.9 tok/s	23.8 tok/s	72%	no drafting

Running --setup again after an update drops the stored result, so the next start calibrates again and takes a few minutes longer. Read the new choice from the log instead of assuming the old one.

Benchmarks

How we measured: a client-side streaming bench against the OpenAI endpoint. Eight runs per setting, alternating a ~400-word Dutch prose prompt and a coding prompt (binary search with tests), greedy decoding, 512 output tokens, median reported. Prefill comes from a needle-in-a-haystack test at the given prompt length, with a different random seed per run so the prefix cache cannot help. No other model runtime or GPU miner ran at the same time.

Maya-S, 262K context	Decode (tok/s)	Dutch prose / code	Time to first token	Prefill	Needle
1.1.16, no drafting (Oct 19)	24.0	28.2 / 22.9	1.7 s	89 tok/s at 247K (46 min)	found
1.1.17, DSpark (Oct 20)	20.7	20.9 / 19.9	3.2 s	634 tok/s at 31K, 685 at 247K (6 min)	found
1.1.17, DSpark (Oct 21)	21.8	21.2 / 22.4	3.1 s	622 tok/s at 31K, 689 at 125K	found
1.1.21, no drafting (Oct 22)	24.9	26.6 / 23.5	3.2 s	1,842 tok/s at 31K, 2,570 at 126K	found

Update, October 22, 2026: Maya 1.1.21 made prompt reading about three to four times faster again: 126K tokens now take 49 seconds instead of 182. Its calibration found DSpark no faster than plain decoding on this PC (23.8 against 23.9 tok/s) and kept no drafting. Decode is back at about 25 tokens per second; time to first token stayed at about 3 seconds. The quality bench scored 54 of 54 in both rounds again, at 680 seconds and about 15,900 tokens per round; the VRAM peak was 28.6 GiB. The rest of this article describes 1.1.16 and 1.1.17.

Three things stand out:

Version 1.1.17 made long prompts usable. Its release notes say it reads prompts in 8K chunks and reads each expert once per layer for long prompts. Here, reading 247K tokens went from 46 minutes to 6.
Decoding got slower with 1.1.17. Our bench measured 24.0 tokens per second on 1.1.16 without drafting and 20.7-21.8 on 1.1.17 with DSpark. Maya's own calibration also measured less without drafting on 1.1.17 (18.2 against 23.0), so this looks like the version, not the drafter. It is not a clean A/B: the calibration writes three answers of 384 tokens on its own prompts, our bench eight of 512. If it matters for your PC, measure both with your own prompts.
Time to first token went from 1.7 to about 3 seconds on 1.1.17 with DSpark, about four times that of the other models on this machine. For short chat turns you notice that.

The VRAM peak for the whole card, desktop included, was 28.7 GiB, which left 3.2 GiB free. Right after the needle tests the engine process had about 600 MB in swap; the decode numbers above were measured before that and looked normal.

Quality check

We ran our own nine-task bench twice: two Astro helpers in JavaScript, two WordPress PHP snippets run in a sandbox, SEO meta tags as JSON, LocalBusiness JSON-LD, a Search Console tool call, a Search Console quick-win analysis and a Dutch blog intro, each with an automatic checker. Reasoning effort medium, Maya's default sampling.

DeepSeek scored 54 out of 54 in both rounds, 108 out of 108 in total, like Qwen3.8-Flash-Next on Strata. It also took the longest: 784 seconds per round on average, against 103 seconds for Qwen on Strata, because it writes more (about 15,700 tokens per round, against 9,000) at about a quarter of the speed.

I measured the other three models on this machine on the same day with the same benchmarks: Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side.

Pitfalls we ran into

Nothing else fits next to it. The engine holds about 27 GiB of VRAM (29 GiB for the whole card with our desktop), and everything else you run competes with it for the page cache that holds the experts. Do not run a second model, a game or a GPU miner next to it.

Disk space and SSD reads. About 350-400 GB on disk, and the part that does not fit in RAM is read from the SSD during generation. Use a fast NVMe drive, not a SATA SSD or a hard disk.

Calibrate with the GPU to itself. Maya skips the speed calibration when too much VRAM is in use (1 GiB upstream) and tries again at the next start. On a machine where the desktop uses the same card, close what you can, or it keeps being skipped.

Reasoning effort. The setup writes "reasoning_effort": "high" as the server default. At 21 to 25 tokens per second, long reasoning takes a while: send a lower reasoning_effort from your client for interactive work. Our quality bench used medium.

Which setup we use when

On this machine, DeepSeek-V4.1-Flash is the biggest model and the slowest. It scored full marks in both rounds of our bench, but at about a quarter of the speed of Qwen3.8-Flash-Next on Strata, with a 3-second wait before the first token. We use it for a second opinion on hard problems, not for interactive agent work.

If you are thinking about running models like this for your business, Voltti has a page about local AI for businesses: what stays on your own hardware, and when a hybrid setup makes more sense.

Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side

Seppe Gadeyne — Wed, 21 Oct 2026 15:20:49 +0000

One RTX 5090, 128 GB of RAM, and four open models from 125B to 748B parameters. Over the past weeks I got each of them running on the same workstation, on three different runtimes. I measured all four in one session, one after the other, with the same benchmarks. This article puts the numbers side by side, so you can see what each setup is good for.

Update, October 22, 2026: I measured everything again after updating Project Maya from 1.1.17 to 1.1.21 and retuning Ornith. The first version of this article, measured on October 21, had Ornith at 33.1 tokens per second. It drafted with its built-in MTP head then, not without a draft model as I wrote. Without drafting and with more experts in VRAM, it now does 46.4. Maya 1.1.21 made DeepSeek read long prompts about 3.7 times faster. The tables below replace the old ones.

Every number below was measured on our own machine on the night of October 21 to 22, 2026, unless I say otherwise.

The four setups

Model	Parameters	Runtime	Quant	Context
Qwen3.8-Flash-Next	125B, ~6B active	Strata	Unsloth UD-Q5_K_XL GGUF	512K
Ornith-1.5-122B	125B	TabbyAPI + ExLlamaV3	Mia-AiLab EXL3 EAQ v1, 4 bpw	256K
GLM-5.3-Flash	321B, ~18B active	Project Maya	turboderp EXL3 3.05 bpw	512K
DeepSeek-V4.1-Flash-1017	748B	Project Maya (`maya-ds4`)	Maya-S (experts in 2-3 bits)	262K

Parameter counts come from the model cards and the runtimes' READMEs. All four are mixture-of-experts models: only part of the model works on each token, which is what makes them possible on one GPU. The runtimes keep the most-used experts in VRAM and the rest in RAM; Maya with DeepSeek also reads part of the model from the SSD while it generates.

How to set up three of them, step by step:

For a machine with less RAM, the oldest article in the series covers Qwen3.8-Flash-Next with EXL3 and TabbyAPI on 40+ GB of RAM.

The machine

GPU: RTX 5090, 32 GB of VRAM, also driving the desktop. Every runtime is set to leave 3 GiB free for that.
CPU: Ryzen 9 9950X3D (16 cores, 32 threads).
RAM: 128 GB DDR5 as 4x32 GB at 5,600 MT/s.
Disk: one NVMe drive.
OS: Arch Linux (Omarchy), NVIDIA driver 615.78, CUDA 13.4.
Versions: Strata 296b59d, Project Maya 1.1.21 (451f9cb), TabbyAPI d6e6227 with ExLlamaV3 1.6.2.

How we measured

One model at a time, started with its own script, with no other model runtime or GPU miner running. Each runtime started in its own systemd slice without a memory limit, so a shell's limits or systemd-oomd could not get in the way (the setup articles explain why).

Decode speed: a client-side streaming bench against the OpenAI endpoint. Eight runs, alternating a ~400-word Dutch prose prompt and a coding prompt (binary search with tests), greedy decoding, 512 output tokens, median reported. Time to first token is the median over the same runs.
Prefill and long context: a needle-in-a-haystack test with prompts of about 31K and about 125K tokens. The needle is a code hidden in Dutch filler text, with a different random seed per run so the prefix cache cannot help. The exact token count depends on each model's tokenizer: the long prompt came to 124.6K tokens for Qwen, 125.6K for DeepSeek, 127.9K for GLM and 131.5K for Ornith. Prefill speed is prompt tokens divided by the time of the whole request, which includes a short answer.
Drafting: Strata drafts with Qwen's MTP head and Maya with GLM's MTP block. Maya measures DeepSeek's DSpark drafter at the first start and keeps the faster option; on 1.1.21 it measured 23.9 tokens per second without and 23.8 with it, so it kept no drafting. Ornith on TabbyAPI runs without drafting, with 180 of its 256 experts per layer in RAM. In an A/B at 256K context the evening before, its MTP head gave 37.8 tokens per second, the DFlash2 drafter 40.6, and no drafting 45.2. Most of Ornith's experts run on the CPU, so every drafted token the model has to check costs extra reads from RAM.
Quality on our own work: nine tasks from our day job, each with an automatic checker: two Astro helpers in JavaScript, two WordPress PHP snippets run in a PHP sandbox, SEO meta tags as JSON, LocalBusiness JSON-LD, a Search Console tool call, a Search Console quick-win analysis and a Dutch blog intro. 54 points per round, two rounds, reasoning effort medium, each server's default sampling. The second round ran the models in reverse order. During this run, Node could not start inside the sandbox (a version-manager shim on a read-only file system), so both JavaScript tasks failed for every model. I fixed the sandbox and scored the saved answers again with the same checkers; that changed only those two tasks.

Speed

Model	Decode (tok/s)	Dutch prose / code	Time to first token	Prefill, ~31K prompt	Prefill, ~125K prompt
Qwen3.8-Flash-Next, Strata	78.8	81.8 / 76.5	0.77 s	4,073 tok/s	4,252 tok/s
Ornith-1.5-122B, TabbyAPI	46.4	49.8 / 42.2	0.42 s	1,808 tok/s	1,508 tok/s
GLM-5.3-Flash, Maya	30.1	34.8 / 26.4	0.78 s	2,269 tok/s	2,445 tok/s
DeepSeek-V4.1-Flash, Maya	24.9	26.6 / 23.5	3.19 s	1,842 tok/s	2,570 tok/s

All four found the needle at both lengths. For a sense of what the prefill means: the long prompt took 29 seconds on Strata, 49 on Maya with DeepSeek, 52 on Maya with GLM and 87 on TabbyAPI with Ornith. On October 21, with Maya 1.1.17, DeepSeek needed 182 seconds for the same length.

Qwen on Strata is in a class of its own: 1.7 times the decode speed of the next one, and the fastest prefill by far. One likely reason is the number of active parameters: about 6B per token for Qwen against about 18B for GLM, according to their model cards, so far fewer weights move through the CPU and over PCIe for every token. Strata's runtime, built for this one model family, plays a part too; we did not measure how much each contributes.

The bench counts tokens from the usage numbers each API returns, not from stream chunks. That mattered for Ornith in the first version: with MTP drafting, TabbyAPI streamed it in chunks of about two tokens.

Quality on our own tasks

Model	Round 1	Round 2	Total	Seconds per round	Tokens written per round
Qwen3.8-Flash-Next, Strata	54 / 54	54 / 54	108 / 108	104	~8,800
DeepSeek-V4.1-Flash, Maya	54 / 54	54 / 54	108 / 108	680	~15,900
GLM-5.3-Flash, Maya	47 / 54	51 / 54	98 / 108	147	~3,900
Ornith-1.5-122B, TabbyAPI	37 / 54	46 / 54	83 / 108	485	~22,600

On October 21 the totals were 108, 108, 105 and 89, in the same order.

What went wrong:

GLM wrote a WordPress breadcrumb snippet with a PHP syntax error in the first round, and missed the Search Console quick-win analysis in the second (0 of 3 checks), the same task it missed once on October 21.
Ornith missed the breadcrumb snippet (it used a PHP constant that does not exist), the JSON-LD task and the Search Console analysis in the first round, and a WordPress shortcode snippet with a PHP syntax error and one check of the Dutch blog intro in the second. Several misses came from stray tokens in the middle of otherwise correct output, such as a Chinese word or the word "Logical" inside a JSON object. That happened in one to four of the nine tasks per round, on both days, with and without MTP drafting.

The time per round says as much as the score. Qwen finishes the nine tasks in under two minutes, GLM in about two and a half. DeepSeek needs 11 minutes, because it writes about 15,900 tokens per round at about 24 tokens per second. Ornith writes even more, about 22,600 tokens, mostly reasoning, and takes about 8 minutes. GLM writes the least, about 3,900 tokens, which is why it is quick despite its slower decode.

These tasks reward correct, careful output, not deep reasoning. They do not tell you which model is smartest. They do tell you which ones we can hand our everyday work to without checking every answer.

Memory and VRAM

Model	VRAM peak (whole card)	VRAM free at peak	RAM in use (whole system)	Engine in swap
Qwen3.8-Flash-Next, Strata	29.0 GiB	2.9 GiB	~120 GiB, pinned	~361 MiB
Ornith-1.5-122B, TabbyAPI	28.5 GiB	3.3 GiB	~61 GiB, pinned	~2.3 GiB
GLM-5.3-Flash, Maya	28.9 GiB	2.9 GiB	~117 GiB, pinned	0
DeepSeek-V4.1-Flash, Maya	28.6 GiB	3.2 GiB	~20 GiB + model files in page cache	~585 MiB

The VRAM peak is the highest value nvidia-smi reported for the whole card during the needle tests, so it includes the desktop and two browsers (about 2.2 GiB before any model started). Each runtime keeps its 3 GiB reserve on top of its own use; with the desktop counted, GLM and Qwen left a little less than that free at their peak. Strata and Maya with GLM pin almost all RAM for their expert tiers, so nothing else heavy fits next to them. Ornith's 65.5 GB quant leaves the most room; with more of its experts in VRAM, it now uses about 1 GiB more VRAM than in the first version of this article. DeepSeek's engine works differently: it reads its model files in place, so free counts most of what it uses as page cache and reports about 100 GiB as available. Do not read that as free memory. The engine depends on that cache, and anything you start next to it pushes experts out, which then have to come from the SSD again. In practice it uses the whole machine, like the other two. RAM and swap were read once, right after the needle tests. Ornith ran right after DeepSeek and had about 2.3 GiB in swap at that point; on the previous evening's tuning runs it had none, and its decode speed here (46.4) matched those runs with the same settings (46.4 to 47.2).

Which one we use when

Qwen3.8-Flash-Next on Strata is our default for coding and agent work. Fastest at everything, full marks in both rounds of our tasks, 512K context.
GLM-5.3-Flash on Maya is the second opinion: a much bigger model with 512K context, at about 40% of the speed, and the one that writes the least.
DeepSeek-V4.1-Flash on Maya is the biggest model here, and the slowest to write, with a 3-second wait before the first token. Since Maya 1.1.21 it reads long prompts about as fast as GLM. Good for hard problems where we can wait, not for interactive work.
Ornith-1.5-122B on TabbyAPI is now the second fastest at writing, with the quickest first token, but the slowest at reading long prompts. It leaves the most RAM free. On our tasks it scored the lowest, partly because of stray tokens in its output, so we read what it writes before we run it.

Running all of this on one desktop is possible because the runtimes move experts between VRAM, RAM and the SSD while you chat. The cost is that only one model fits at a time, and switching takes a stop and a start script.

Caveats

One machine, one session. Other hardware, other drivers or another day can give other numbers, and all four runtimes move fast: Maya 1.1.17 made DeepSeek's long-prompt reading about eight times faster than 1.1.16, and 1.1.21 made it another 3.7 times faster.
Two rounds of nine tasks are not many. A single miss can be noise.
GLM scored 98 here and 105 on October 21, Ornith 83 and 89. With two rounds of nine tasks, that difference is within what a few misses can do.
Each model ran with its runtime's default sampling and reasoning effort medium. Tuning per model could change the quality ranking.

If you are thinking about running models like these for your business, Voltti has a page about local AI for businesses: what stays on your own hardware, and when a hybrid setup makes more sense.

Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090

Seppe Gadeyne — Fri, 16 Oct 2026 14:33:32 +0000

GLM-5.3-Flash is a 321B mixture-of-experts model, and its 3.05 bpw EXL3 quant is 125 GB, too big to fit in my workstation's 128 GB of RAM next to the OS. It still runs locally here: on one RTX 5090 with 128 GB of RAM, with a 512K-token context, at about 25 tokens per second. This guide shows how, with Project Maya, and what we measured on the way.

Every number below was measured on our own machine on October 15, 2026. Where a number comes from someone else, I name the source.

This is the second of two articles. The first one covers Qwen3.8-Flash-Next with Strata at 79 tokens per second, and before that I wrote about Qwen3.8-Flash-Next with EXL3 and TabbyAPI on 40+ GB of RAM.

The model

GLM-5.3-Flash is an open-weight model from zai-org under the MIT license. Maya's README describes it as 321B parameters with about 18B active per token and a context of up to 1M tokens. It has a hybrid attention design and an MTP layer for drafting.

18B active parameters is three times the roughly 6B that the Qwen3.8-Flash-Next model card lists. That explains most of the speed difference below: more of the model has to move through the CPU and over PCIe for every token.

Why not TabbyAPI

I tried TabbyAPI first, with turboderp's GLM-5.3-Flash EXL3 quants. ExLlamaV3 keeps CPU experts in a RAM arena and has no SSD tier for experts. So the whole model has to fit in VRAM plus RAM, next to the OS. On 128 GB, the largest quant that worked was the self-calibrated 2.60 bpw revision:

TabbyAPI, 262K context	Decode (tok/s, server log)	Prefill at 256K	RAM free under load
sc_2.10bpw	32.4	1,063 tok/s	~36 GB
sc_2.60bpw	25.5	1,151 tok/s	~19 GB, other processes in zram

The 3.05 bpw quant (125 GB) does not fit that way. You need a runtime that can leave part of the experts on the SSD.

Why Project Maya

Project Maya is a fork of Strata, built for this one model. Its engine keeps the experts your conversation uses most on the GPU, the next ones in RAM and the rest on the NVMe drive, and moves them while you chat. It measures your hardware and tunes itself, exposes OpenAI- and Anthropic-compatible APIs, and has a dashboard with a live monitor.

Maya ships its own GGUF quants (Maya-S 96.5 GB, Maya-M 116 GB, Maya-L 156 GB). It can also read turboderp's EXL3 quants in place from their safetensors, which its README marks as experimental and NVIDIA-only. I chose the EXL3 3.05 bpw route because I already trusted that quant from the TabbyAPI tests, and because Maya then decoded as fast as the smaller sc_2.60 quant did in TabbyAPI.

The machine

GPU: RTX 5090, 32 GB of VRAM, also driving the desktop.
CPU: Ryzen 9 9950X3D (16 cores, 32 threads).
RAM: 128 GB DDR5 as 4x32 GB at 5,600 MT/s.
Disk: one NVMe drive.
OS: Arch Linux (Omarchy), NVIDIA driver 615.78, CUDA 13.4.
Maya 1.1.11 (commit d187195, October 15, 2026).

Install

git clone https://github.com/mw00/project-maya.git
cd project-maya
./maya.sh --check

--check prints the GPUs, driver, CUDA toolkit, compiler, RAM and CPU. Then run setup non-interactively:

./maya.sh --setup --plain --yes --gpu 0 \
  --context 131072 --model EXL3-3.05bpw \
  --models-dir ~/Models/maya --download-model \
  --no-vision --draft mtp --no-start

What happens, according to Maya's setup docs and what we saw:

It checks the PC and installs its Python packages into .venv.
It fetches llama.cpp's source at a pinned commit and compiles the engine for your GPU. Maya's docs say 10-30 minutes, once.
It downloads the 125.3 GB quant (27 files) with resumable curl commands it prints first, and checks every file against its SHA-256.
It builds the pack, the engine's index of the model files, in the model folder. Here that was a 1.3 GB pack/ folder plus a 2 GB maya-exl3.gguf next to it.
It writes maya-exl3-3.05bpw.json (the engine config) and run-maya-exl3-3.05bpw.sh.

The flags that matter:

--model EXL3-3.05bpw selects turboderp's EXL3 quant instead of a Maya GGUF.
--draft mtp drafts with the model's own MTP block. The DFlash2 drafter is the other option, but its license is CC BY-NC-ND 4.0, non-commercial only, so I left it out.
--no-vision skips the image encoder (a 10-20 minute build and 1.1 GB).
--no-start installs without starting, so you can start it the way shown below.

Edit the config before the first start

Two settings go into the args of maya-exl3-3.05bpw.json:

"args": [
  "--glm-pack", "/home/you/Models/maya/EXL3-3.05bpw/pack",
  "--max-context", "524288",
  "--expert-cache", "auto",
  "--prefill", "auto",
  "--kv-streaming", "on",
  "--vram-reserve-mib", "3072"
]

--vram-reserve-mib 3072: the VRAM Maya's expert pool leaves free. The default is 700 MiB. Our desktop runs on the same card, and with the VRAM full the screen flickered, so we keep 3 GB free everywhere.
--max-context 524288: GLM-5.3-Flash supports up to 1M tokens, so this is not rope scaling, just a bigger limit. With --kv-streaming on, extra context costs RAM (about 13 KB per token, judging by the size of the sessions Maya saved to disk), not VRAM.

A new --setup run writes this file again. Maya's docs say it keeps settings like the reserve, but keep a backup anyway.

Start it outside your shell's memory limits

Maya pins most of the free RAM for its expert tier. On this machine it leaves about 4-5 GB free. That collides with two Linux defaults:

A shell inside a systemd scope with a memory limit. Our agent's terminal scopes have memory.max of 4 GiB, and a big process started from there lands in swap.
systemd-oomd, which Omarchy enables for the user's app.slice. It kills a cgroup after 20 seconds above 50% memory pressure, and a model pinning RAM during load triggers exactly that.

So we start it in a separate slice, without a memory limit:

systemd-run --user --scope --collect --slice=llm.slice \
  --unit="maya-$(date +%s)" -p MemoryMax=infinity \
  ./run-maya-exl3-3.05bpw.sh --plain

--plain keeps the output as plain text instead of Maya's full-screen dashboard, which is easier to log. Stop it with systemctl --user stop 'maya-*.scope'. The start takes a few minutes. Then:

curl -s http://127.0.0.1:8080/v1/models

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "glm-5.3-flash", "messages": [{"role": "user", "content": "Say hello in five words."}],
       "max_tokens": 2000}'

The dashboard is at http://127.0.0.1:8080/#monitor and shows decode and prefill speed, the VRAM hit rate and where the experts live. Keep the server on 127.0.0.1; if you open it to your network, use --host 0.0.0.0 together with --api-key.

Calibrate

Maya's calibration measures how to split the work between the CPU and the PCIe link, and how many CPU threads to use:

./maya.sh --calibrate --no-start

It loads the full model, so run it with Maya stopped and in the same way as the server (in llm.slice, without a memory limit). It refuses to run while more than 1 GiB of VRAM is in use on the GPU, to make sure no other engine is loaded. A desktop on the same card already takes about 2 GB, so on our machine it never started. I raised that threshold locally in busy_gpus() in maya.py from 1024 to 3072 MiB. That is a local patch: check it after every update, and only use it if you are sure nothing else heavy is on the card.

The calibration took about 10 minutes (Maya estimates 10-15). Its full output on our machine:

the engine's own split (PCIe share 0.60): 23.7 tok/s
PCIe share 0.00: 21.1 tok/s
PCIe share 0.10: 24.6 tok/s
PCIe share 0.25: 23.9 tok/s
PCIe share 0.50: 24.1 tok/s
PCIe share 0.75: 23.3 tok/s
16 CPU threads: 24.6 tok/s
12 CPU threads: 21.9 tok/s
11 CPU threads: 20.0 tok/s
8 CPU threads: 18.1 tok/s
4 CPU threads: 13.7 tok/s
confirming against the engine's own settings ...
[ok] tuned for this PC: STRATA_GLM_PCIE_SHARE=0.10 (25.7 tok/s)

It stores the result as an environment variable in the config. In the sweep, a PCIe share of 0.10 beat the engine's own split by about 4% (24.6 vs 23.7), and the confirmation run measured 25.7. Fewer than 16 CPU threads hurt a lot.

Benchmarks

How we measured: a client-side streaming bench against the OpenAI endpoint. Eight runs per setting, alternating a ~400-word Dutch prose prompt and a coding prompt (binary search with tests), greedy decoding, 512 output tokens, median reported. Prefill comes from a needle-in-a-haystack test at the given prompt length. The needle and filler text are fixed per length, and Maya caches prompt prefixes (also on disk, across restarts), so a later run can be flattered by an earlier one: read the prefill numbers as approximate. Nothing else heavy ran on the machine.

Maya, EXL3 3.05 bpw	Decode (tok/s)	Dutch prose / code	Prefill	Needle	VRAM peak
128K context	26.3	30.4 / 21.9	2,165 tok/s at 126K	found	31.4 GiB
262K context	24.1	27.5 / 21.2	2,039 tok/s at 254K	found	31.3 GiB
262K, calibrated, 3 GB reserve	25.2	28.6 / 22.9	-	-	29.5 GiB
512K, calibrated, 3 GB reserve	25.9	29.3 / 23.2	1,930 tok/s at 506K	found	29.0 GiB

The first two rows ran with the 700 MiB default reserve, and you can see the VRAM peak at the edge of the 31.8 GiB card. With the 3 GB reserve, the peak stayed at or below 29.5 GiB. Reading 506K tokens took 262 seconds. Decoding at 512K was as fast as at 262K, because the KV cache streams from RAM.

Compared to TabbyAPI on the same machine: about the same decode speed as the smaller sc_2.60 quant (25.5, from TabbyAPI's server log rather than this client bench), with a bigger quant, about 1.7x the prefill speed, and twice the context.

For reference, Maya's README lists "30+ tokens/s" for an RTX 5090 with a Ryzen 9 7950X and 96 GB of DDR5, with Maya's own Maya-M quant. That is a different quant and a different bench, so do not compare the numbers one to one.

Quality check

We ran the same five coding tasks with hidden tests that we used for the Qwen setups: three trials each, sandboxed, reasoning effort medium, and each server's default sampling (Maya temperature 1.0 and top_p 0.95, Strata greedy, TabbyAPI temperature 0.7).

Model and runtime	Passed	Test cases	Seconds per task	Decode
Qwen3.8-Flash-Next UD-Q5_K_XL, Strata	15 / 15	480 / 480	42	67
Qwen3.8-Flash-Next EXL3 4.05 bpw, TabbyAPI	15 / 15	480 / 480	43	68
GLM-5.3-Flash EXL3 3.05 bpw, Maya	14 / 15	472 / 480	66	25

The one miss was invalid JSON (a trailing comma) in one trial of an incident-report task; all the code tasks passed. These tasks are too easy to say which model is smarter. What they do show is that a 3-bit GLM on a desktop produces working code, and that it takes about 1.5x the time per task because of the slower decode.

Pitfalls we ran into

Nothing else fits next to it. Maya pins RAM until about 4-5 GB is free and holds 29-30 GB of VRAM. Do not run a second model runtime, a game or a GPU miner next to it. Switching models means stopping one and starting the other.

Wait for the VRAM after a stop. VRAM takes several seconds to come back after a runtime stops. Start the next one too early and it fails a VRAM check or gets a smaller expert pool.

Unstable RAM corrupts downloads. Our four DIMMs at their 6,000 MT/s EXPO profile were not stable: kernel oopses, crashes, and most of a model download written in that time was corrupt. We run 5,600 MT/s now. Maya checks its own downloads against SHA-256; do the same for anything you fetch yourself.

Measure with fresh prompts. A repeated long prompt hits the prefix cache. One of our runs reported 70,624 tokens per second of "prefill" for exactly that reason.

Reasoning effort and sampling. Setup wrote "reasoning_effort": "high" with temperature 1.0 and top_p 0.95 as server defaults in the config. Long reasoning at 25 tokens per second adds up, so for interactive work consider sending a lower reasoning_effort from the client. On the Qwen setup, medium scored best on our own bench; for GLM we have not measured that yet.

Which setup we use when

On this one machine we now have three ways to run a large model, and we switch with a stop and a start script:

Qwen3.8-Flash-Next UD-Q5_K_XL on Strata: about 79 tokens per second, 512K context, our daily default for coding and agent work.
Qwen3.8-Flash-Next EXL3 4.05 bpw on TabbyAPI: about 68 tokens per second in its server log, at 262K context.
GLM-5.3-Flash EXL3 3.05 bpw on Maya: about 25 tokens per second, a 321B model with 512K context, for when a second, bigger model should look at a problem.

The Strata setup, with its calibration A/B and our own nine-task bench, is in the companion article: Run Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant on 128 GB of RAM.

Update, October 21, 2026: the same machine now also runs DeepSeek-V4.1-Flash (748B) with Project Maya, and all four local models are compared on the same day in Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side.

If you are deciding what hardware to buy for running AI locally, Voltti has a practical guide on the trade-offs between local and cloud AI for business use.

Run Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant on 128 GB of RAM

Seppe Gadeyne — Fri, 16 Oct 2026 14:24:55 +0000

Two weeks ago I wrote about running Qwen3.8-Flash-Next with EXL3 and TabbyAPI on 40+ GB of RAM. Since then the workstation got 128 GB of RAM, and that changed which setup is fastest. This guide covers the setup we now use every day: Strata serving Unsloth's 5-bit UD-Q5_K_XL GGUF of the same 125B model, with a 512K context, at about 79 tokens per second on one RTX 5090.

Every number below was measured on our own machine on October 15, 2026. Where a number comes from someone else, I name the source.

What changed since the EXL3 guide

The EXL3 guide ran turboderp's 3.05 bpw quant because 64 GB of RAM did not leave room for anything bigger. With 128 GB, two routes opened up:

TabbyAPI with the 4.05 bpw EXL3 quant and a pinned CPU expert arena. Its server log showed about 68 tokens per second for decoding with a 3 GB VRAM reserve, and about 1,780 tokens per second for prefill at 254K tokens.
Strata with Unsloth's UD-Q5_K_XL, a 5-bit quant with 8-bit down projections. Same model, more bits per expert, and a runtime built for this one model family.

On the same machine, Strata decoded about as fast as TabbyAPI before tuning and faster after calibration, read long prompts about 1.8-2.4x faster, and ran at 512K context. That is why it is our default now.

The machine

GPU: RTX 5090, 32 GB of VRAM. It also drives the desktop, which matters later.
CPU: Ryzen 9 9950X3D (16 cores, 32 threads, AVX-512).
RAM: 128 GB DDR5 as 4x32 GB, running at 5,600 MT/s. More on why not 6,000 in the pitfalls.
Disk: one NVMe drive with room for the 158 GB download.
OS: Arch Linux (Omarchy), NVIDIA driver 615.78, CUDA 13.4.
Strata at commit 296b59d.

Why Strata

Strata is a C++/CUDA engine plus a Python server for Qwen3.8-Flash-Next and its variants. It reads the model in three tiers: the experts your conversation uses most live in VRAM, the next ones in RAM, and anything that does not fit comes from the SSD. It moves experts between the tiers while it runs, drafts tokens with the model's own MTP layer, and exposes OpenAI- and Anthropic-compatible APIs plus a small web UI with a live monitor.

The project's install docs are written so that a coding agent can follow them (docs/AI_SETUP.md). I still recommend reading them yourself once: the recommended sizes in those docs target 32-96 GB gaming PCs, and the larger Unsloth quants are a separate family in setup.

Pick the quant by RAM, not by bits

Strata keeps every expert in RAM when the experts plus about 16 GB fit. If they do not fit, it reads the rest from the SSD while it answers, and that costs a lot of decode speed. So the rule is simple: take the largest quant whose experts fit in RAM with room to spare.

From Strata's model table, with its own quality measure (how often the quant picks the same top token as Q8_0):

Quant	Download	Experts	RAM for all experts	Same top token as Q8_0
UD-IQ4_XS	94 GB	55 GiB	~72 GB	88.9%
UD-Q4_K_XL	111 GB	72 GiB	~88 GB	93.1%
UD-Q5_K_XL	158 GB	92 GiB	~108 GB	94.6%
UD-Q6_K_XL	169 GB	102 GiB	~118 GB	94.6%
Q8_0	188 GB	120 GiB	~136 GB	100% (reference)

On 128 GB, UD-Q5_K_XL is the sweet spot. UD-Q6_K_XL scores the same 94.6% in Strata's table, with a mean KL divergence of 0.033 against 0.038 for Q5, and needs 10 GB more RAM that a 128 GB machine does not have to spare. Q8_0 does not fit.

Before you download anything, let setup check your PC:

git clone https://github.com/Niko1221/Strata.git
cd Strata
./setup.sh --check

It prints the GPU, driver, RAM, CPU and free disk, and which sizes fit.

Install

This is the setup command we ran, non-interactive, with the model files on the NVMe drive:

./setup.sh --yes --family unsloth --model UD-Q5_K_XL \
  --context 262144 --vision no --gpu 0 \
  --data-dir ~/Models/strata --port 8081 \
  --vram-reserve-mib 3072 --no-start

What the flags do:

--family unsloth --model UD-Q5_K_XL picks Unsloth's 5-bit files.
--context 262144 is the model's trained context. You can raise it later.
--vram-reserve-mib 3072 keeps 3 GB of VRAM free. Our desktop runs on the same card, and when the VRAM was completely full the screen started to flicker.
--port 8081 because port 8080 is taken by our GLM setup (more on that at the end). The default is 8080.
--no-start installs without starting the server, so you can start it the way described below.

Setup writes a JSON config (strata-unsloth-ud-q5_k_xl.json) and a start script (run-unsloth-ud-q5_k_xl.sh) in the Strata folder.

If the download is slow

Strata's own downloader reached only about 7 MB/s here, which is around six hours for 158 GB. The faster route: stop setup, fetch the shards with the Hugging Face CLI at the exact revision setup pins, put them in the model folder, and run setup again. It checks every file against its SHA-256 and skips what is already there.

# the revision setup.py pins for Unsloth's repo
grep -n '"unsloth/Qwen3.8-Flash-Next-GGUF"' setup.py

hf download unsloth/Qwen3.8-Flash-Next-GGUF \
  --revision 38bb39ee97821de2c9009abb7e93950eec396e66 \
  --include "UD-Q5_K_XL/*" \
  --local-dir ~/Models/strata/hf-tmp

mv ~/Models/strata/hf-tmp/UD-Q5_K_XL/*.gguf ~/Models/strata/models/unsloth-UD-Q5_K_XL/

Use the revision your own setup.py prints, not the one above, if they differ.

If setup says shard 1 is "short"

The first of the six UD-Q5_K_XL shards holds only metadata and no tensors (10.9 MB). At commit 296b59d, setup's shard check still treated it as a few bytes too short and stopped on a correct download. There is a fix waiting upstream (PR #2505, same idea in #2632; both still open on October 16, 2026). Until one of them is merged, the one-line local patch is to skip shards without tensors in check_shards() in setup.py:

# in check_shards(), before: need = g.data_start + ...
if not g.tensors:
    continue

Check after every git pull whether the fix has landed upstream, so you do not keep a patch you no longer need.

Start it outside your shell's memory limits

Strata pins about 98 GB of RAM with this quant. Two Linux defaults can kill or cripple a process that size:

A shell or tool runner inside a systemd scope with a memory limit. Our agent's terminal scopes have memory.max of 4 GiB, and a model started from there ends up mostly in swap.
systemd-oomd. Omarchy enables it for the user's app.slice and kills a cgroup after 20 seconds above 50% memory pressure. A model that pins most of the RAM looks exactly like that during load.

So we start the server in its own slice, without a memory limit:

systemd-run --user --scope --collect --slice=llm.slice \
  --unit="strata-$(date +%s)" -p MemoryMax=infinity \
  ./run-unsloth-ud-q5_k_xl.sh

Stop it with systemctl --user stop 'strata-*.scope'. Avoid a broad pkill -f: the pattern also matches the shell that runs it.

The first start takes a few minutes and the desktop can stall for a moment while the RAM gets pinned. Then check that it is up:

curl -s http://127.0.0.1:8081/health
# {"status": "ok", "max_context": 524288, "model": "...", "loaded": true, ...}

curl -s http://127.0.0.1:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "strata", "messages": [{"role": "user", "content": "Say hello in five words."}],
       "max_tokens": 64, "reasoning_effort": "none"}'

Any OpenAI-compatible client works with base URL http://127.0.0.1:8081/v1, any API key and any model name. Anthropic-compatible clients use /v1/messages. Keep it on 127.0.0.1; if you open it to your network, set --api-key as well.

Go to 512K context

The model is trained for 262,144 tokens. Strata can stretch that with YaRN rope scaling, and setup picks the method for you:

./setup.sh --setup --context 524288

Add --no-start if you run it from a shell with a memory limit (see above), and start the server separately. Setup adds --max-context 524288 --rope-scaling yarn --rope-scale 2 to the engine arguments. If you use a custom chat template (see below), check afterwards that the tokenizer key in the config still points to it. Strata's docs call it experimental and note that needle tests separate scaling methods poorly; real long-document Q&A does. On our machine it found the needle at 520K tokens, and decoding stayed the same speed as at 262K. The KV cache is int8 with 32K tokens resident on the GPU, so the extra context costs RAM, not VRAM.

Calibrate for your PC

Strata has a calibration that measures a few engine settings on your hardware and keeps only changes that are more than 3% faster:

./setup.sh --calibrate --no-start

Without --no-start it starts the server afterwards. Run it with the server stopped, after you set the VRAM reserve, and in the same way as the server (in llm.slice, without a memory limit), because it loads the full model. Strata estimates 15-30 minutes. We called the same calibration function (calibrate_config in setup.py) from a small script, and it finished in about 11 minutes. On our machine it chose:

--pcie-frac 0.35 --spec-min-p 0.70
--adapt-every 1 --adapt-swaps 160 --adapt-decay 0.97

The --adapt-* settings drive the adaptive expert cache: how often it swaps experts between RAM and VRAM, how many per step, and how fast old usage counts fade. --spec-min-p 0.70 makes the MTP drafter stop earlier when it is unsure. The number of CPU workers (4 to 15) made no difference.

Calibration reports its own numbers (86.8 tokens per second at the end here), but those are its internal measurements. I wanted one I control, so I ran an A/B with a restart between each arm, old config and new config, twice each. Back up the config before you calibrate, so you can do the same.

Benchmarks

How we measured: a client-side streaming bench against the OpenAI endpoint. Eight runs per arm, alternating a ~400-word Dutch prose prompt and a coding prompt (a binary search function with tests), greedy decoding, 512 output tokens, median reported. Prefill comes from a needle-in-a-haystack test at the given prompt length. The needle and filler text are fixed per length, so a later run can get a partial prefix-cache hit from an earlier one: read the prefill numbers as approximate. Nothing else heavy ran on the machine.

Setup	Decode (tok/s)	Dutch prose / code	Prefill (tok/s)	Needle	VRAM peak
262K context, default settings	67.3	72.6 / 64.6	4,205 at 259K	found	29.3 GiB
512K context (YaRN 2)	66.8	71.8 / 62.5	3,817 at 520K	found	29.3 GiB
512K, before calibration (A/B, 2 runs)	70.9 / 70.9	77.0 / 65.8 and 76.5 / 65.5	-	-	-
512K, after calibration (A/B, 2 runs)	78.3 / 79.1	82.1 / 74.3 and 83.2 / 75.2	3,196 at 265K	found	29.1 GiB

The first two rows ran earlier that day, without the custom chat template described below; the A/B rows both ran with it. Only rows measured in the same session are a fair comparison, which is why the A/B pairs matter more than the first two rows.

Prefill ranged from 3,196 to 4,205 tokens per second at about 260K tokens, before and after calibration. For TabbyAPI with the 4.05 bpw EXL3 quant, its server log reported about 68 tokens per second for decoding with the same 3 GB reserve, and the same needle test prefilled 254K tokens at about 1,780 tokens per second. Server-log and client-stream decode figures are measured differently, so treat that as close, not one to one.

For comparison, Strata's docs list 57-60 tokens per second for stories and 82 for code with the same quant on an AMD Radeon AI PRO R9700 (32 GB). That is a different card and a different bench, so do not compare them one to one.

Quality checks

Speed only matters if the answers hold up. We ran two checks of our own.

Coding tasks with hidden tests. Five tasks, three trials each, run in a sandbox, reasoning effort medium. Strata with UD-Q5_K_XL passed 15 of 15 (480 of 480 test cases). TabbyAPI with EXL3 4.05 bpw scored the same. These tasks turned out too easy to separate the two quants; the difference between them is speed and context, not this score.

Our own work. A bench with nine tasks from our day job, each with an automatic checker: two Astro helpers in JavaScript, two WordPress PHP snippets run in a PHP sandbox, SEO meta tags as JSON, LocalBusiness JSON-LD, a Search Console tool call, a Search Console quick-win analysis, and a Dutch blog intro. Two rounds per setting, 108 points maximum:

Reasoning effort	Score	Time per round
none	100 / 108	62 s
low	107 / 108	-
medium	108 / 108	221 s
xhigh	90 / 108	577 s

More thinking was not better: xhigh hit the token limit twice and produced a wrong tool call, at 2.6 times the time of medium. Strata's API defaults to high, so we send "reasoning_effort": "medium" from our clients. Sampling did not matter on this bench either: greedy, temperature 0.7 and temperature 1.0 all scored 108 at the same speed. Without a sampling block in its config, Strata decodes greedy.

A custom chat template, if you need one

We write a lot of Dutch, and the model likes to mix English words into Dutch text. A short rule in the system prompt helps. Strata renders chat_template.jinja from the folder that the tokenizer key in its config points to. So you can make a folder with symlinks to the pack's tokenizer files plus your own chat_template.jinja, and point tokenizer there. Leave the files in the pack itself alone, so a setup run or update does not overwrite your changes or trip over them.

Two things to know. Strata passes only enable_thinking and reasoning_effort from chat_template_kwargs to the template, so your own template variables cannot be switched per request: a template change means a config change and a restart. And measure it: in our A/B with restarts, the rule cut English words in 30 Dutch answers from 28 to 13, with no measurable speed difference on our use-case bench (79.8 vs 80.3 tokens per second) and about 350 extra prompt tokens, which the prompt cache absorbs.

Pitfalls we ran into

Free RAM is the real limit. Strata leaves about 4 GB of RAM free on this machine. A browser with many tabs or a second agent process is enough to push part of the engine into zram. When that happened, the server log showed decoding at 40-55 instead of about 80 tokens per second. Check with free -g, swapon --show and the VmSwap line in /proc/<pid>/status of the engine process. Run it on its own: not next to another model runtime, a game or a GPU miner.

Four DIMMs at EXPO speed were not stable. Our four sticks are rated for 6,000 MT/s, and at that speed the machine logged "Bad page state" errors and kernel oopses, and crashed. Worse: most of a model download written in that period was corrupt on disk. We now run the second EXPO profile at 5,600 MT/s, and we check every model file against the SHA-256 published on Hugging Face. Strata's setup does that for its own downloads; do it yourself for anything else.

Restart only after the VRAM is free. After stopping a runtime, the VRAM takes a few seconds to come back. Starting the next one too early fails on a VRAM check, or leaves less VRAM for the expert cache.

Measure with fresh prompts. Strata caches prompt prefixes. Repeating the same long prompt gives absurd prefill numbers. Change the prompt (or at least its start) for every prefill run.

What's next

The same 128 GB also runs GLM-5.3-Flash, a 321B model, through Project Maya, a fork of Strata built for that model. It is a third of the speed of this setup but a much bigger model, and it reads up to 512K tokens without using more VRAM. I wrote a companion article about that setup and its numbers: Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090.

If you are deciding what hardware to buy for running AI locally, Voltti has a practical guide on the trade-offs between local and cloud AI for business use.

Website performance: 9 steps to a faster site, in the right order

Seppe Gadeyne — Fri, 16 Oct 2026 10:45:50 +0000

Most slow websites don't need another caching plugin. They need a diagnosis, one fix for the biggest delay, and a second measurement that proves the fix worked.

This is the sequence I use when I make a site faster, including my own site, straffesites.com. It's written for developers who maintain marketing sites, blogs, or content-heavy frontends. The order is deliberate: compressing a heavy hero image does more than optimizing twenty tiny icons, and fixing a slow server matters more than polishing a loading animation.

1. Measure first rather than relying on guesswork

Start with at least two kinds of measurement. A lab test such as Lighthouse or WebPageTest gives you a reproducible snapshot. Field data from the Chrome UX Report (CrUX) or your own real-user monitoring shows what visitors experience on slower devices and connections. The two don't need to match because they answer different questions. When I measured 19 website builders, the gap was bigger than I expected: lab LCP diverged from the CrUX LCP by up to 16.7x.

Don't stop at one total score. Note the Largest Contentful Paint (LCP) for the most important visible element, Interaction to Next Paint (INP) for responsiveness, and Cumulative Layout Shift (CLS) for visual stability. The thresholds come from Google: "LCP should occur within 2.5 seconds of when the page first starts loading", "INP of 200 milliseconds or less" and "CLS of 0.1" (web.dev, Core Web Vitals).

Assess the 75th percentile of each metric separately for mobile and desktop. A page passes when LCP, INP, and CLS each meet their threshold at that percentile. Averages don't count, and it doesn't mean the same 75% of page loads pass all three.

Also record the server response, total JavaScript weight, and the largest network requests. That's your baseline.

Test mobile and desktop on a cold cache.
Measure a homepage, a content page, and a conversion page separately.
Keep the waterfall; it shows waiting time and dependencies.
Repeat the same test after every meaningful change.

2. Lower the waiting time before the first HTML

If the browser waits too long for the first document, no frontend optimization can fully hide the delay. Check the Time to First Byte (TTFB). Slow database queries, heavy server-side plugins, a distant hosting region, or dynamic rendering without a cache can delay every page before the browser even starts.

For mostly static marketing pages, static generation is often the simplest win: the HTML is prepared in advance and served from a global network. For dynamic applications, targeted server caching helps, but never cache personal or shopping cart data blindly. Measure the origin directly as well, because a CDN can temporarily mask a slow source.

3. Make the main image as light as its function allows

Images often account for most of a page's weight, and the hero image is frequently the LCP element too. HTTP Archive's yearly measurement leaves little doubt: the median mobile home page weighed 2,559 KB in July 2025 (Web Almanac 2025, Page Weight). A year earlier, that median home page included 900 KB of images and 558 KB of JavaScript (Web Almanac 2024, Page Weight).

Never deliver a 3000-pixel-wide photo to a 720-pixel box. Generate responsive variants, use a modern format such as WebP or AVIF, and choose the compression level based on the image type. A soft photograph tolerates more compression than an interface image with sharp text.

Don't lazy-load the main image above the fold; the browser should discover it immediately. Images further down the page can load lazily. Always include width and height so the browser reserves space in advance and the layout doesn't shift.

<!-- Above the fold: discoverable right away, space reserved -->
<img src="/hero-1440.avif" width="1440" height="810" alt="..." fetchpriority="high">

<!-- Further down: load when needed -->
<img src="/detail-720.webp" width="720" height="405" alt="..." loading="lazy">

No single format always wins. Choose based on the content, browser support, and whether you actually need transparency or animation.

Format	Best for	Strength	Watch out
AVIF	Photos and complex illustrations	Small file at high quality	Encoding takes longer
WebP	General web use	Broadly supported and compact	Sometimes larger than AVIF
SVG	Logos, icons, and diagrams	Sharp at every resolution	Not suitable for photos
PNG	Lossless graphics and transparency	Exact pixels and alpha	Often unnecessarily heavy

4. Don't let critical CSS wait for the rest

The browser needs CSS to draw the first viewport correctly. One large stylesheet with rules for every page delays that moment. Remove unused rules, split the CSS where it makes sense, and avoid a chain of @import files. Keep the critical path smaller than the full component library so relevant content appears sooner.

Inlining critical CSS can help in large applications, but it also adds maintenance work. Start with a simpler setup: one well-cached stylesheet, local fonts, and no blocking external styles. Then check whether splitting it further produces a measurable gain.

5. Ship less JavaScript and run it later

JavaScript costs more than download time. The browser must parse, compile, and execute it, usually on the same main thread that also handles your visitor's input. A bundle that feels harmless on your laptop can cause a noticeable delay on an average phone.

For responsiveness, Google now measures INP instead of First Input Delay: "INP is the successor metric to First Input Delay (FID)" (web.dev, Interaction to Next Paint). Optimizing for an old FID score today means optimizing for a metric that is no longer assessed.

Remove libraries that solve one small interaction, load page-specific code only where it's needed, and use native HTML for accordions, forms, and video where possible. Add defer to non-critical scripts and initialize heavy widgets only after a deliberate interaction. The fastest code is still the code you don't ship.

6. Limit web fonts without losing your identity

Five families and twelve weights are rarely necessary. Choose one strong display family and one readable body family, subset the characters you use, and load only the weights that appear on the page. Host fonts locally when you want control over caching, privacy, and availability.

Preload only the font needed directly above the fold. Too many preloads compete with the main image and CSS. Use font-display: swap so text stays visible while the font arrives, and choose fallback fonts with similar proportions to limit shifting.

What a Lighthouse run shows on a static site

Numbers only matter when you can check them. This is the unedited Lighthouse CLI report I ran on the straffesites.com homepage on September 4, 2026, under emulated mobile conditions. The site is a static Astro build on Vercel, so treat it as one lab snapshot, not field data:

# Lighthouse 13.4.1 — https://straffesites.com/
Tested September 4, 2026 · Chrome 143 (headless) · emulation: Moto G Power, slow 4G (simulated throttling)

## Performance 100/100

| Metric | Value | Assessment |
|--------|-------|------------|
| First Contentful Paint | 1.3 s | Good |
| Largest Contentful Paint | 1.7 s | Good |
| Total Blocking Time | 0 ms | Good |
| Cumulative Layout Shift | 0.014 | Good |
| Speed Index | 1.3 s | Good |
| Time to Interactive | 1.7 s | Good |

This is a laboratory run, not CrUX field data: the scores describe how the page performs under controlled network and device conditions, not what every visitor experiences today.

## What the run confirms

- Root document served in 90 ms (server-response-time: 90 ms).
- 16 network requests, 502 KiB transferred in total (total-byte-weight).
- No render-blocking requests with savings left (render-blocking: no savings).
- Main-thread work 0.8 s, JS boot-up time 0.0 s — there is almost no JavaScript.
- CLS 0.014: image and font dimensions are reserved, nothing shifts while loading.
- The LCP element is the top case image; it is discovered and loaded without delay.

## Remaining opportunities (outside the score)

| Audit | Savings | Detail |
|-------|---------|--------|
| Improve image delivery | 272 KiB | Case images are served larger than their rendered size; WebP at the exact render size or a responsive srcset saves the most |
| Reduce unused CSS | 11 KiB | One inlined @font-face rule carries glyphs the homepage never uses |

Both opportunities are optimizations, not defects: no audit pulls the score below 100. A 100 does not mean "done forever" — every new image, script, and font can drag the next run back down. That is why we measure again every month.

---

Run environment: Lighthouse CLI 13.4.1, Node 26, headless Chrome on Linux, performance category only.

Even a perfect lab score still lists image work. That's the point of step 3.

7. Use caching and a CDN as accelerators, not bandages

Set long cache headers for assets with unique, hashed file names. When the content changes, the hash changes and the browser automatically gets the new version. HTML usually needs a shorter cache period because current content matters more than maximum retention.

# Hashed assets: safe to cache for a year
Cache-Control: public, max-age=31536000, immutable

# HTML: always revalidate
Cache-Control: public, max-age=0, must-revalidate

A CDN brings files closer to visitors and absorbs traffic peaks. But it doesn't make a file smaller and doesn't remove blocking code. Fix the weight and dependencies first, then distribute the efficient version.

8. Treat every external script as a performance budget

Chat widgets, heatmaps, ad pixels, video players, and tag managers often come from other servers. You don't control their speed, and one tag manager can quietly start dozens of requests. Keep an inventory of every external script with an owner and a clear purpose.

Load marketing tags only after consent. Show a local video poster first, then load the external player after a click. Remove tools that nobody uses. Treat "can this script be added?" as seriously as "can this feature be added?"

9. Make speed part of every release

A one-off optimization doesn't stay effective on its own. New images, tags, and components gradually increase the page weight. Set budgets for JavaScript, images, and Core Web Vitals, make the build warn when a budget is exceeded, and recheck the most important templates after a major change.

Review field data and segments each month: mobile versus desktop, new versus returning visitors, and slow versus fast connections. An average can hide a poor experience for mobile users. Performance work is done when it's monitored, not when a single Lighthouse run turns green.

Assign every warning to an owner. A red number without a responsible person can stay unresolved, while a ticket with the URL, the regression, and the expected budget is easier to discuss. For new components, include performance in the acceptance criteria: how much JavaScript is added, which image becomes the LCP element, and what happens on a slow connection? That turns speed from a reactive fix into a normal design decision.

Don't forget reliability. A fast first render that then freezes because of a faulty script is still a poor experience. Check console errors, failed requests, and timeouts alongside the vitals.

What to tackle first on Monday

Create a baseline. Note LCP, INP, CLS, server response, and the three heaviest requests.
Fix one big bottleneck. Usually the main image, server work, or blocking JavaScript.
Measure the same page again. Prove the gain before starting the next optimization.

That keeps the work verifiable, and you can explain why a change got priority. You don't end up with a pile of plugins, but with a measurably faster site.

Adapted for DEV from Website performance: improve your site speed in 9 clear steps, originally published on straffesites.com.

Run Qwen3.8-Flash-Next Locally with EXL3 and TabbyAPI on 40+ GB of RAM

Seppe Gadeyne — Sun, 04 Oct 2026 17:09:23 +0000

Qwen3.8-Flash-Next is a 125B mixture-of-experts model, and you can run it locally on one consumer GPU. This guide shows the exact route we use on our own workstation: TabbyAPI as the server, ExLlamaV3 as the inference engine, and turboderp's 3.05 bpw EXL3 quant. The machine is a Ryzen 9 9950X3D with an RTX 5090 (32 GB), 64 GB of DDR5-6000, and Arch Linux. Every number below was measured on that machine. Where a figure comes from someone else, I name the source.

Update, October 2026: the workstation now has 128 GB of RAM. With that, I run the same model with a 5-bit quant at about 79 tokens per second: Run Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant on 128 GB of RAM. The same machine also runs the 321B GLM-5.3-Flash: Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090. A DeepSeek-V4.1-Flash (748B) guide and a side-by-side benchmark of all four local models followed. This guide stays valid for 64 GB machines.

You get an OpenAI-compatible endpoint on localhost that serves a 262,144-token context at about 50 tokens per second. The trade-off: the model needs more than your GPU alone. About 26 GB sits in VRAM, and about 43 GB of expert weights live in system RAM, where the CPU computes them. That is why the title says 40+ GB of RAM. In practice, plan for 64 GB installed.

What Qwen3.8-Flash-Next is

Qwen3.8-Flash-Next is an open-weight model from the Qwen team. The model card lists 125B parameters, of which about 6B are active per token, plus a 51B n-gram embedding table and a 4B multi-token prediction (MTP) layer. Each of the 48 layers has 512 routed experts, and 10 of them are picked per token.

That design makes it a good fit for a single GPU with CPU offload. Only a small part of the model does work for each token, so the experts that are rarely used can sit in system RAM without slowing every step. The n-gram table is a lookup, not a compute step, and ExLlamaV3 can stream it from disk.

Why EXL3 and TabbyAPI

EXL3 is the quantization format of ExLlamaV3. According to the project README, it is a streamlined variant of QTIP, a trellis-based method from Cornell RelaxML. The practical gain is quality per bit. On the model page of the EXL3 quants, turboderp publishes a KL divergence chart for this model. The 3.05 bpw quant scores 0.0177 there, against 0.0349 for the UD-IQ3_XXS GGUF quant at a similar size. Lower means closer to the original model.

TabbyAPI is the official API server for ExLlamaV3. It exposes an OpenAI-compatible API, so editors, chat clients, and agent frameworks that speak that API can use your local model without code changes. It also wires up the ExLlamaV3 options this model needs: CPU expert offload, the n-gram table, and MTP drafting.

Hardware requirements

This is the hardware we measured on, and what the model used:

GPU: RTX 5090 with 32 GB of VRAM. About 26 GB in use with the config below.
System RAM: 64 GB of DDR5-6000. About 43 GB resident for the CPU expert arena.
CPU: Ryzen 9 9950X3D (16 cores, 32 threads). The CPU computes the cold experts, so core count matters.
Disk: an NVMe drive with room for the 85.1 GB download. The n-gram table is read from it during inference.
OS: Arch Linux with a current NVIDIA driver.

Two things did not work well. With 60 GB of RAM it gets tight as soon as a browser and other heavy programs run next to the model. And once part of the arena lands in zram or swap, decoding collapses, as described in the pitfalls below. Treat 64 GB as the realistic minimum for this configuration, and run the model on a machine that is not doing other heavy work at the same time.

Install TabbyAPI with the prebuilt exllamav3 wheel

Clone TabbyAPI and create a Python 3.11 virtual environment inside it:

git clone https://github.com/theroyallab/tabbyAPI
cd tabbyAPI
python3.11 -m venv .venv
source .venv/bin/activate
pip install -U ".[cu12]"

The cu12 extra installs PyTorch 2.9.0 (CUDA 12.8) and the prebuilt exllamav3 1.5.1 wheel for your Python version. At the time of writing, TabbyAPI's main branch pins exllamav3 1.5.1. Check that both landed:

pip show exllamav3 torch | grep -E '^(Name|Version)'
python -c "import torch; print(torch.cuda.is_available())"

You should see exllamav3 at 1.5.1+cu128.torch2.9.0 and True for CUDA.

Use the prebuilt wheel rather than a just-in-time (JIT) build. The wheel is faster to set up, and on our system the JIT build of the CUDA extension broke on GCC 16. That build only went through after adding -Xcompiler -Wno-template-body to the compiler flags. With the wheel you skip the whole problem.

Download the 3.05 bpw EXL3 quant

The quants live in one Hugging Face repository, with one branch per size. This guide uses the 3.05bpw_h5_ng5 revision: 3.05 bits per weight for the experts, a 5-bit output head, and a 5-bit n-gram table.

hf download turboderp/Qwen3.8-Flash-Next-exl3 \
 --revision 3.05bpw_h5_ng5 \
 --local-dir models/qwen3.8-flash-next-exl3-3.05bpw

The hf command comes with huggingface_hub, which TabbyAPI installs into the venv. Older guides use huggingface-cli download, but recent huggingface_hub versions no longer run that command. TabbyAPI also has its own downloader (./start.sh download <repo> --revision <branch>) if you prefer that.

The download is 85.1 GB. One file stands out: ngram_embedding.safetensors is 32.6 GB on its own. Keep the folder on NVMe, because that table is streamed from disk while the model runs. The folder name becomes the model name in TabbyAPI.

Configure config.yml and a sampling preset

Copy config_sample.yml to config.yml and change the keys below. The rest can stay at the defaults. These are the values we run and measured:

network:
 host: 127.0.0.1
 port: 5000
 disable_auth: true

model:
 model_dir: models
 model_name: qwen3.8-flash-next-exl3-3.05bpw
 backend: exllamav3
 max_seq_len: 262144
 cache_size: 262144
 cache_mode: Q8
 gpu_split_auto: true
 autosplit_reserve: [1000]
 cpu_moe_split_experts: 400
 cpu_moe_threads: 24
 ngram_ram: false
 reasoning: true

draft_model:
 draft_mode: mtp
 draft_num_tokens: 4
 dynamic_draft: true

sampling:
 override_preset: qwen3_8_flash_next

What the important keys do:

backend: exllamav3 selects the engine. The value is exllamav3, not exl3.
cpu_moe_split_experts: 400 keeps the 400 coldest of the 512 experts per layer in system RAM and computes them on the CPU. The hot experts stay in VRAM, and ExLlamaV3 adjusts the placement while it runs.
cpu_moe_threads: 24 sets the CPU worker threads for those experts. Without it, ExLlamaV3 uses half the core count.
ngram_ram: false streams the 51B n-gram table from disk. Loading it into RAM would need tens of gigabytes more than a 64 GB machine has left.
cache_mode: Q8 stores the KV cache in 8 bits instead of 16, which halves its size for the full 262,144-token context.
autosplit_reserve: [1000] leaves 1,000 MB of VRAM free on the GPU for the desktop and CUDA overhead.
draft_mode: mtp uses the model's own MTP layer for speculative decoding. No separate draft model is needed.
disable_auth: true is only acceptable because the server listens on 127.0.0.1. If you open it to your network, turn authentication back on.

Now create the sampling preset in sampler_overrides/qwen3_8_flash_next.yml. TabbyAPI finds it by name in that folder:

temperature:
 override: 0.7
 force: false
top_k:
 override: 20
 force: false
top_p:
 override: 0.95
 force: false
min_p:
 override: 0.0
 force: false

These values are fallbacks. A request that sends its own sampler settings still overrides them. Skipping this preset is one of the pitfalls below.

Start the server and send a first request

Start TabbyAPI from the project folder with the venv active:

python main.py

From a warm NVMe, the model is loaded in about two minutes on our machine. A first load from a cold disk can take longer. Check that the server is up and the model is loaded:

curl -s http://127.0.0.1:5000/health
curl -s http://127.0.0.1:5000/v1/models

Then send a first chat request to the OpenAI-compatible endpoint:

curl -s http://127.0.0.1:5000/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "qwen3.8-flash-next-exl3-3.05bpw",
 "messages": [{"role": "user", "content": "Write a Python function that checks whether a string is a palindrome."}],
 "max_tokens": 4000
 }'

The model reasons before it answers. TabbyAPI splits that reasoning into a separate reasoning_content field, and the answer arrives in content. Note the high max_tokens. The reason is explained in the pitfalls.

The server log prints the speed per request, and with MTP drafting also the acceptance rate of the drafted tokens. That log is the source for all the measurements in the next section.

Sanity check before you measure

Before you trust any tokens-per-second figure, check that the model actually fits. Paging does not slow decoding down a little: once part of the expert arena lands in swap, it falls off a cliff (more on that in the pitfalls). Run these checks right after the model loads, and again during or after your first long request, because that is when the KV cache and the expert arena fill up.

# si and so should stay at 0 while the model decodes
vmstat 1 10

# swap used by the TabbyAPI process: should be 0 kB or close to it
grep VmSwap /proc/$(pgrep -of "python main.py")/status

# VRAM in use: about 26 GB with the config above
nvidia-smi --query-gpu=memory.used,memory.total --format=csv

The si and so columns of vmstat show how much memory moves in from and out to swap each second. A steady 0 there tells you more than a stable RSS in htop, because RSS only shows what is still in RAM, not what has already been pushed out. In free -h, look at the available column rather than free: the n-gram table is streamed from disk, so the page cache grows and free memory shrinks without anything being wrong.

Performance tuning: our measured A/B results

We tuned three settings one at a time, with greedy decoding, reading the figures from the TabbyAPI server log. All runs were done with nothing else heavy running on the machine.

CPU threads for the expert arena

The thread count for the CPU experts had the biggest effect on decoding. More threads is not always better: at 32 threads the SMT siblings compete for the same cores, and decoding got slower than at 24.

Decode speed by cpu_moe_threads (median, tokens per second)

cpu_moe_threads	Decode (tok/s)	Note
16	48.1	Fewer threads than the CPU can feed
24	51.7	Best result; prefill 188 tok/s in this run
32	47.6	Slower due to SMT contention

On a 16-core, 32-thread CPU, 24 threads was the sweet spot. On a different CPU, run the same test with your own core count.

How many experts go to the CPU

In a separate A/B run, we lowered cpu_moe_split_experts from 430 to 400. That puts 30 more experts per layer on the GPU. Decoding stayed the same, prefill rose from 174 to 204 tokens per second, and the machine kept about 2 GB more free RAM. With 32 GB of VRAM, 400 fit with the 1,000 MB reserve. On a card with less VRAM you need a higher value, which means more RAM.

Draft depth for MTP speculative decoding

With MTP drafting, the model proposes several tokens ahead and verifies them in one pass. draft_num_tokens: 4 with dynamic_draft: true gave the best result. A draft depth of 6 was 4% slower. The acceptance rate stayed around 56–58%, so longer drafts mostly produced more tokens that were thrown away.

The end result on our machine

With all three settings in place, we see about 48–52 tokens per second for decoding and about 175–205 tokens per second for prefill. The model loads in about two minutes from a warm NVMe. These figures were measured without other heavy workloads running.

For context: the ExLlamaV3 1.5.0 release notes list higher numbers for this model on an RTX 5090 paired with a Threadripper 7960X, with the n-gram table in RAM and a pinned arena. That is different hardware with more memory and different settings, so do not treat our numbers as the ceiling, or theirs as what a 64 GB desktop will reach.

Pitfalls we ran into

Swap kills decoding

If part of the expert arena is pushed into zram or swap, decoding falls from about 50 to about 2.4 tokens per second. The model keeps running, so it is easy to miss. The sanity check after startup catches this before you measure. To check the process on its own:

grep VmSwap /proc/$(pgrep -of "python main.py")/status

Anything far above zero means the arena no longer fits. Close other programs, keep an eye on available RAM with free -h, and do not start the model next to other memory-hungry work.

No sampling preset, no stable output

Without a sampling preset, requests that send no sampler values run with top_k 0 and top_p 1. In that state the model can drift into other languages mid-answer. The preset from step 03 (temperature 0.7, top_k 20, top_p 0.95) works well and keeps clients that send nothing on safe values.

Reasoning eats max_tokens

Qwen3.8-Flash-Next thinks before it writes the answer, and those reasoning tokens count toward max_tokens. With a low limit, the response can stop before the actual answer starts, leaving content empty. Use at least 400 tokens, and 4,000 for real tasks.

Small config mistakes

backend must be exllamav3; exl3 is not a valid value. And run this model on its own: a second model, a game, or another GPU workload on the same machine takes VRAM or RAM the model needs.

What it gets you

The result is a strong coding model on hardware you control. On the Qwen model card, Qwen3.8-Flash-Next scores 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro. Those are the Qwen team's figures for the full model, not measurements of this quant. In our own comparison, it is the strongest model for coding tasks that we can run on this consumer machine. Nothing leaves your network, there is no per-token bill, and any tool that speaks the OpenAI API can connect, from a code editor to an AI agent running a workflow.

Alternatives worth considering

This setup is not the only route, and it is not always the best one:

A smaller quant. The same repository has a 2.05 bpw version that needs less memory, at a clear cost in quality: turboderp's chart shows a KL divergence of 0.0684 for it, against 0.0177 for 3.05 bpw. We did not measure its speed or memory use.
GGUF with llama.cpp. This route runs on more hardware, including Macs and CPU-only machines, and Unsloth publishes GGUF quants with a run guide. In our comparison, that route was slower on this model than ExLlamaV3.
A cloud API. If you do not have the hardware, or only need the model now and then, a hosted API costs less than a new GPU and a RAM upgrade. You give up the privacy and fixed costs of a local setup.

Wrapping up

The short recipe: TabbyAPI in a Python 3.11 venv with the prebuilt exllamav3 1.5.1 wheel, the 3.05bpw_h5_ng5 quant on NVMe, backend: exllamav3, 400 experts per layer on the CPU with 24 threads, a Q8 cache, MTP drafting with 4 tokens, and a sampling preset. Watch swap, give reasoning room, and run the model on its own. On one RTX 5090 with 64 GB of RAM, that gives a 125B model at about 50 tokens per second. ExLlamaV3 and TabbyAPI move quickly, so check the ExLlamaV3 releases for new versions before you start.

Wondering what hardware your business needs to run AI locally? We wrote a practical guide: What computer do you really need to run AI locally?. It covers the trade-offs between local and cloud AI for business use.

A website without a CMS: Astro Content Collections in practice

Seppe Gadeyne — Thu, 01 Oct 2026 07:42:14 +0000

I moved straffesites.com from Astro with Storyblok to Astro Content Collections with local MDX and JSON files. The site still has articles, case studies, landing pages, and Dutch and English versions. What it no longer has is a separate CMS for managing that content.

I am the only person managing the content on straffesites.com, and I work with Hermes, an AI agent. Since I already work in Git, moving the content there was a useful simplification. Content changes now go through the same repository and build pipeline as template changes. I can inspect a paragraph edit next to the schema or component it depends on.

This is a case study of that setup: how the content is organized, what the schemas check, how publishing works, and which CMS features I chose to give up. It is not a recommendation to make every editor learn Git.

Why I removed Storyblok from this project

Astro and Storyblok worked well together. I still use that combination for client projects where an editing interface is useful. On straffesites.com, however, I am the only human editing and managing the content. Hermes helps me work with the files and run checks. I did not need a visual editor for that workflow.

Keeping a CMS meant maintaining content models in another system, managing my CMS account and its permissions, and configuring the integration that fetched content during builds. Code lived in Git; content lived in Storyblok. Changes that touched both required checking both.

Working with Hermes was another part of the decision. The agent could already read the project, edit files, and run checks. To work with CMS content, it also needed an authenticated client and a separate content workflow. That integration was possible, but it was unnecessary for this project.

Moving the content into Git removed the CMS-specific access layer. It did not remove repository permissions, deployment credentials, or the need to maintain the AI tooling.

Content Collections provide a content layer, not an editor

Astro Content Collections let you organize, validate, and query structured content. They can use local files, remote sources, or live data. The setup on straffesites.com uses local files loaded at build time:

MDX for pages and articles.
JSON for datasets, case metrics, people, and site settings.

Those are six collections in this project. You do not need that exact split to use the same approach.

A collection combines a loader with a schema. The loader finds the entries; the schema defines the fields the templates can expect. Here is an excerpt from the article collection configuration:

import { defineCollection, reference } from 'astro:content';
import { glob } from 'astro/loaders';
import { articleFrontmatterSchema } from './content/schemas/article';

const articles = defineCollection({
  loader: glob({ pattern: '**/*.mdx', base: './src/content/articles' }),
  schema: articleFrontmatterSchema.safeExtend({
    author: reference('people').optional(),
  }).superRefine((article, context) => {
    if (article.articleType !== 'glossary' && !article.author) {
      context.addIssue({
        code: 'custom',
        path: ['author'],
        message: 'blog and case articles must include an author',
      });
    }
  }),
});

// The full configuration also registers the people collection.
export const collections = { articles /*, pages, datasets, caseMetrics, people, settings */ };

This is a project-specific excerpt, not a complete starter configuration. It imports the article schema from another file, and the author reference depends on the people collection being registered in the full configuration.

The same article collection holds blog posts, case studies, and glossary entries. The refinement requires an author for blog posts and cases while allowing glossary entries without one.

What validation catches

The frontmatter schema defines fields such as the title, route, publication date, category, tags, summary, and FAQ items. Required fields must be present. A strict schema also rejects unknown field names, so a typo cannot silently become unused metadata.

That gives the templates a defined content contract. It does not make the article correct.

A schema can reject a missing title or a value that violates a configured rule. It cannot establish that an explanation is accurate, a source supports a claim, or an internal link is helpful. A nonempty summary also does not imply a sensible length unless you add a length limit.

I use additional contract tests and build-output checks for rules that go beyond frontmatter. Content Collections alone do not check every link, component choice, or editorial requirement.

What you lose without a CMS

There is no visual editor in this local-file setup. Content Collections do not provide editor accounts, approval screens, a media library, or a publishing calendar.

You can add previews and access controls around a Git workflow. You can also put a Git-based CMS in front of the files. That is a valid option, but it adds an editing interface; it is different from editing the files directly.

How I organize the content

The MDX files on straffesites.com are grouped by language and content type:

src/content/
├── pages/            # landing pages and hub pages (MDX)
│   ├── nl/           # Dutch-language pages
│   └── en/           # English pages
└── articles/
    ├── nl/
    │   ├── blog/      # blogs
    │   ├── case/      # case studies
    │   └── begrippen/ # glossary terms
    └── en/
        ├── blog/      # blogs
        ├── case/      # case studies
        └── glossary/ # glossary terms

The JSON collections sit alongside these directories. The tree above only shows the MDX content.

Each MDX entry starts with frontmatter. The body uses Markdown inside typed content-block components. On the original website, for example, this article's source file contains its metadata and body, while the page template renders shared elements such as the author card and related articles. The DEV version you are reading is an adapted Markdown crosspost, not that MDX file running inside Astro.

For this project, the route comes from an explicit frontmatter field rather than the file's location. That lets me reorganize source files without treating every move as a URL change. The templates and tests still need to respect that convention; Astro does not infer the project's routing rules from the directory tree alone.

Keeping translations connected

Dutch and English pages have a shared pair registry in the repository. It records which routes are counterparts.

Build checks can then verify that those routes exist and that the relationships are consistent. The files being next to each other is not enough: translation pairing is a rule I have to define and test.

This is one advantage of keeping content and code together. The content, route relationships, and checks can change in the same diff.

Publishing is a deployment workflow

On straffesites.com, a push to main triggers a Vercel build and deployment. Astro validates the content and renders static pages. A local commit by itself does not publish anything.

The workflow is straightforward:

Edit MDX or JSON
    -> inspect the diff
    -> run content and build checks
    -> commit and push
    -> verify the deployment

That diagram describes the work to do, not a security guarantee. Git does not force someone to inspect a diff, and a schema does not require human approval. Repository permissions and required checks determine what can reach production.

Git also makes rollback understandable: revert the relevant change, build again, and deploy the result. Reverting a commit locally does not change the live site until the reverted version is deployed successfully.

The maintenance moves rather than disappearing

Concern	Local content in Git	API-first headless CMS
Editing	Files and repository tooling	CMS dashboard
Content validation	Configured schemas and project checks	CMS field rules; project checks can also apply
Content history	Diffs for committed changes	CMS revisions, depending on the platform
Static publishing	Build and deploy after a repository change	Build and deploy after a CMS content change
Systems to maintain	Dependencies, CI, hosting, and repository access	Those systems plus the CMS integration and accounts

These are common workflows, not fixed limits. A headless CMS can feed a fully static site, and a Git-based CMS can provide a dashboard while keeping content in files.

How I work with Hermes

Hermes is the AI agent I work with on this project, not another human editor. It can read an MDX file, update a paragraph, and run the same checks used for code changes. It does not need a CMS-specific API key to edit checked-out files.

For me, the useful part is how easy the changes are to inspect. A content edit can include its frontmatter, body, and related template changes in one diff. There is no separate CMS revision to reconstruct alongside the code change.

Reading a local file avoids a CMS API request. It does not make model usage free: text sent to the model still consumes tokens. Nor does it prove that agents perform better with files than with a CMS.

Instructions are not access controls

Project instructions can tell an agent which files to use, which commands to run, and what it must not change. They help it work within the project's conventions. They are not a security boundary.

An agent with broad repository or deployment access can still make harmful changes. Limit its permissions, make checks explicit, and verify what it changed. For straffesites.com, I remain responsible for the scope and the published result, including when I authorize Hermes to deploy after checks.

Plausible but false prose can pass every schema check. That is an editorial review problem, not something Zod can solve.

Removing a CMS does not automatically make a website faster

I track straffesites.com through monthly PageSpeed measurements and Search Console data on its public case page. Those measurements describe the deployed site. They are not a controlled test of whether removing Storyblok improved performance.

A headless CMS can supply content during the build and produce the same static HTML. Visitors do not necessarily make a CMS request when they open that page.

Removing the CMS API from my build reduced an external build dependency. Visitor performance still depends on the output: JavaScript, images, fonts, caching, and how the page renders.

The change I can describe directly is operational. My content workflow no longer needs a CMS service, a separate editor account, or a CMS integration. Dependencies, hosting, and access control still need maintenance.

When I would keep the CMS

For a client project, I would keep a CMS when the editing interface solves a problem the client or their editorial team actually has.

If editors do not work in Git, asking them to maintain MDX may just move the burden onto them or the development team. Draft states, permissions, scheduling, and media management are useful features, especially when several people publish regularly.

Translation workflows can also be more involved than maintaining two files and a route pair. Editors may need field-level translation status or separate approval steps. A CMS can provide those workflows without requiring the team to build them.

Changing a content model is another consideration. In my setup, adding a new content type means changing a schema and its templates. A CMS dashboard may make field management easier, although frontend changes can still require development work.

You can build versions of these features around Git. But if you need an editor, scheduling, media permissions, and approval screens, consider whether you are rebuilding the CMS you removed.

For straffesites.com, direct file editing fits how I manage the content with help from Hermes. For a client with an editorial team, I may choose a CMS instead. I would start with who edits the content and what their publishing process needs, then choose the storage and tooling.

Adapted for DEV from A website without a CMS: Astro Content Collections in practice, originally published on straffesites.com.

Automating SEO with AI: a self-improving content loop

Seppe Gadeyne — Mon, 28 Sep 2026 11:52:53 +0000

Everyone agrees you should "let AI do your SEO." Few people explain what that looks like when you actually build it. This is the setup I run on my own site: an AI agent with read-only access to Google Search Console, a dedicated service account via gcloud, and one small, verified improvement per run — every thirty days.

The article below is a crosspost from my site. It covers the full loop: setup, prioritization, one-change-per-run, measurement, and the honest learning curve before anything runs on its own.

Automating SEO with AI, in practice, means: an AI agent with read access to your Google Search Console that pulls the real search queries from your data every thirty days and makes one small, verified improvement to your pages. That is not a robot that lifts your website to the top of Google while you drink coffee — that robot does not exist. It is a loop: pull data, prioritize, improve, measure, and again thirty days later. Not everything at once, not on a hunch — one thing per run, measured and logged.

That approach works because SEO is not a one-time project but a habit. Anyone who looks at the same data every month and makes one targeted intervention each time will see their pages gradually align better with what people actually search for. The agent makes that habit affordable: it does the heavy lifting — pulling data, prioritizing, preparing — while you make the decisions. At first you supervise every run. After a few months, most of it runs automatically on a cron job.

Why automating SEO with AI makes sense now

An agent like Hermes can query an API, read and write files, run a build, and inspect the result. For recurring GSC analysis, I combine those tools with a dedicated service account: gcloud handles the Cloud setup, and Search Console grants that account read access to the property. The agent can then retrieve data without waiting for me to sign in through a browser again.

SEO is an ideal application for this, for three reasons:

The source data is objective. Search Console shows literally how people find you: queries, impressions, clicks, positions. That gives the agent source data, but does not prevent hallucinations or incorrect interpretation. Check its conclusions against the export; anonymized queries and API limits make the data incomplete.
The feedback loop already exists. Thirty days after every change you see in the same data whether it had an effect. That is a perfect learning loop for a step-by-step process.
The work is repetitive but requires judgment. Pulling and organizing data is pure routine. Interpreting and prioritizing takes judgment. That exact mix is where a supervised agent shines.

Done manually, most people pick this up once a quarter with a spreadsheet and good intentions. The agent turns it into a fixed rhythm — and rhythm is the most underrated success factor in SEO.

What you need: gcloud, GSC access, and an agent

For scheduled SEO runs, I use gcloud with a dedicated service account. The agent gets its own Google identity, added as a read-only user to the Search Console property, rather than relying on my personal login. You need:

A Google Cloud project and gcloud, with an administrator who can enable the Search Console API and create the service account.
A Search Console property owner who can add that account under Users and permissions.
An agent that can use tools, such as Hermes from Nous Research, Claude Code, Cursor agents, or your own application around a language model.
A git repository for small, reviewable website changes, separate from the Google credentials.

Create a dedicated service account with gcloud

Run these setup commands as a human administrator with the required Cloud permissions. Replace YOUR_PROJECT_ID with your existing project ID; the example account name is seo-reader:

gcloud services enable searchconsole.googleapis.com iam.googleapis.com \
  --project=YOUR_PROJECT_ID
gcloud iam service-accounts create seo-reader \
  --display-name="SEO read-only access" \
  --project=YOUR_PROJECT_ID

The account's email address will be seo-reader@YOUR_PROJECT_ID.iam.gserviceaccount.com. Do not give it broad Cloud Owner or Editor roles. A Cloud IAM role does not grant access to your Search Console property; you add that separately. If it needs permission to consume the quota project, grant the narrow Service Usage Consumer role there, not project-wide administration. See Google's service account setup guide.

Add the service account as a restricted GSC user

In Search Console, the property owner opens the exact property, then Settings → Users and permissions → Add user. Enter the service account's full email address, choose Restricted, and add it. You do not need to make the account an owner or verify the domain again. Google's permissions table confirms that Restricted users can read the Performance report, which is what this loop needs.

Configure the API client to request https://www.googleapis.com/auth/webmasters.readonly, as documented in Search Console authorization. The property role limits what the account can do; the OAuth scope limits what its API token can do. That scope is not tied to one website, so add the account only to properties the workflow needs. A domain property uses an identifier such as sc-domain:example.com; a URL-prefix property uses its exact URL prefix.

Run without a recurring personal login

Use an attached service account or Workload Identity Federation when the runtime and GSC client support it. These avoid storing a long-lived private key. If your local runner needs a JSON key, create one in Google Cloud under the service account's Keys → Add key → Create new key. Follow your organization's key policy and store the file outside your repository. The following path is a placeholder for that protected file:

chmod 600 /secure/path/seo-reader.json
export GOOGLE_APPLICATION_CREDENTIALS="/secure/path/seo-reader.json"
export GOOGLE_CLOUD_QUOTA_PROJECT="YOUR_PROJECT_ID"

Set these variables in the environment that actually starts the agent or its GSC connector, including scheduled runs. A cron job does not automatically inherit your interactive shell's exports. Use a client that supports Application Default Credentials (ADC), with the read-only scope above. ADC locates the credentials; it does not choose the permissions for you. ADC and the gcloud CLI also use separate credential configurations: being logged into gcloud does not configure the application's credentials.

Google's authentication library obtains and renews short-lived access tokens for the service account without an interactive browser login. Personal OAuth can also refresh tokens, but it ties the job to a person's authorization. A service account removes that dependency, making unattended runs easier to maintain. It is not maintenance-free: rotate keys, revoke unused credentials, and review access. Keep keys and tokens out of prompts, logs, Git, and browser code; Google's key-management guidance explains the risks.

Before scheduling the loop, run a small Performance-data query through the same client and environment the scheduled job will use. Confirm the intended property and service account, and stop if access fails rather than falling back to your personal credentials. Isolate the runner from other credentials too. Website changes still go through your codebase and review flow; GSC read access does not authorize publishing.

The loop: pull the data

Every run starts the same way: the agent pulls fresh Search Console data for a bounded period — say the last thirty days, or a comparison with the thirty days before that. It requests the top queries per page, with impressions, clicks, CTR, and average position, and writes that away as a structured file in the repo. From that moment it no longer works with live APIs but with a snapshot you can inspect and compare against previous runs.

One important detail: Search Console data only becomes final after a few days. Google itself says about fresh data points: "Each fresh data point will be replaced with the final data point after a few days" (Google Search Central Blog on data freshness). Measure too early and you draw conclusions from numbers that still shift. That is why the agent works with closed periods and preferably compares month over month, never day over day.

Prioritize: where is the biggest win?

With the data on the table, the agent looks for patterns worth a decision:

Many impressions, few clicks on a query your page already ranks for: usually a title or meta description problem, not a content problem.
Position 8 to 20 with a declining trend: the page is relevant enough to rank but not convincing enough to break through. Sharpening the search intent and internal links helps here.
Queries you have no good page for: content gaps. Those lead to a new article or an expansion of an existing page — not keyword stuffing on some random page.
Multiple pages competing for the same query: cannibalization. Strengthen one page, give the other its own angle.

For each finding the agent states a hypothesis: this page, this query, this expected intervention, this expected effect. No more than one or two per run. If you change ten things at once, you will never know what worked.

Make one small improvement

Then comes the human moment. The agent prepares the change — a rewritten intro that puts the real search intent first, a better title tag, an internal link from a strong page — and for now you always review it. The change is deliberately small: one page, one aspect, describable in one sentence. "This intro now promises what the searcher actually asks for." Not: "The entire page optimized."

Next, the agent verifies its own work. For a website in git that means: commit the change, run the build, fetch the changed route again, and confirm the promised text is actually there. A successful tool call is not a successful task; the run is only complete when the published page checks out.

Measure and log

Every run ends in a log: what the agent saw in the data, what it changed, what the expected effect is, and when to check it. Ideally in your issue tracker, so that thirty days later an on-the-record comparison is possible. Did the intervention raise the CTR on that one query? Did the position move from 14 to 9? Or did nothing happen — and why not?

That logging is not administrative luxury. It is the only way to learn from the agent and to teach the agent. Because every time a hypothesis does not pan out, it says something about your instructions: was the prioritization wrong, the intervention too small, or the expectation too high?

Repeat every 30 days

A single run does little. The power is in the repetition. After six cycles you have six verified improvements, six measurement moments, and a growing archive of what did and did not work. Along the way your pages have gradually been tuned to the real search intent of your visitors — not to your assumption of what they search for, but to what the data shows they search for.

Why thirty days and not every week? Because Search Console data only finalizes after several days and positions show too much noise week over week. Thirty days gives a signal you can act on, and it matches the pace of content changes: thorough enough to matter, slow enough to measure cleanly.

From supervising to automatic: be realistic

This needs honesty: the first time, this does not work. The first run of a new agent rarely produces anything publishable. The data gets misinterpreted, the prioritization misses, the proposed copy reads like a committee memo. That is not the concept failing — it is the onboarding period.

This is an illustrative three-phase plan, not a measured timeline or guarantee. Automate only after repeated tests show that the workflow is reliable enough:

Phase	What you do	What the agent does
Months 1–2	Supervise every run, correct every change, rewrite instructions	Pull data, draft the analysis, make mistakes
Months 3–4	Review results, feed in edge cases	Prepare complete runs, propose improvements with reasoning
Months 5+	Check in occasionally, judge escalations	Run on schedule by itself, ask for human input when in doubt

How you speed up that curve: give the agent skills. An agent like Hermes works with reusable instruction packages — how to structure a page audit, how to interpret search intent, how to rewrite a title without clickbait. You teach those once, and every next run builds on them. After a few months those skills are the difference between an intern and a colleague: the process is in its fingers, and you only look over its shoulder for the hard cases.

From that moment on, you turn the loop into a cron job: a scheduled task that runs the agent on a schedule — say the first of every month. The run follows the same rules as when you sat next to it, only without you having to be there. Hard decisions (new pages, major rewrites, anything touching money or reputation) stay explicitly human.

Skills: teaching the agent how to run SEO audits

The fastest way to climb that curve: give the agent skills — reusable instruction packages that describe one task in detail. A skill for a page audit specifies exactly what data gets pulled (live HTML, Search Console rows, robots.txt), which dimensions get scored (search intent, E-E-A-T, content depth, on-page, structure, technical), with what weights, and in what format the report appears. You write such a package once, or teach it from an existing collection. I've documented that installation route step by step in Hermes Agent for SEO: from Search Console to a blog post: installing the agent, connecting Search Console through gcloud, and adapting the SEO skills for it. After that, the agent runs every next audit in exactly the same way — including the cron run in the middle of the night.

The historical audit below shows what such a report looks like. It was recorded on September 2, 2026, before this article switched to the service account setup. Its earlier authentication advice is preserved as part of the original output; follow the updated setup above for new runs.

On my own site, this article includes the full recorded audit terminal: the complete first run's output, scores, findings, and the GSC baseline of that moment (recorded on September 2, 2026). It's too long to paste here, but you can read it in the original article.

The audit reported that the page had no Search Console data on its publication day instead of inventing numbers. Its scores and baseline describe that moment, not the current page. Each finding comes with a concrete fix. For later changes, record a fresh baseline and wait at least thirty days before assessing the effect.

What an agent may and may not do on its own

Automation without boundaries is a risk, especially on something as visible as your website. A few rules of thumb that work in practice:

Read permissions for data, write access through git. The agent reads Search Console, but every change goes through your codebase and review flow.
Small commits. One change per run, with explanation. Not because the agent cannot do more, but because otherwise you lose the measurement effect.
No invented numbers. Every metric in copy comes from the actual data export, with period and source attached. An agent that smooths over or invents numbers is useless for this work.
Human decisions stay human. Merging pages, restructuring key landing pages, changing prices or promises: do not automate.
Useful stays useful, even at speed. Google's guidance on AI-generated content stresses that automation changes nothing about the requirement that content be made for people and be genuinely helpful — regardless of who or what wrote it.

For a broader look at deploying such an agent without losing control, read What is an AI agent?. And for the bigger picture around recurring work: Workflow automation.

Where the change lands: MCP into your CMS, or Git for your Astro site

So far, every improvement went to "your codebase". But where exactly the agent writes depends on how your website is built — and that is less complicated than it sounds. Broadly, there are two routes.

Route one: your site runs on a CMS. Then you want the agent to work directly in that CMS, and for that you use MCP: the Model Context Protocol, an open standard that connects AI applications to external systems in a structured way. An MCP server for WordPress or Storyblok offers the agent a limited, explicit list of functions — create a draft, update a field, publish a page — with matching permissions. That way the agent can adjust a page on its own after an approved analysis, but only through the doors you open. And because every CMS draft starts as a draft rather than going live immediately, you keep a natural review step.

Route two: your site is static, like this one. Straffe Sites runs on Astro with Content Collections: every page and article is a file in a git repository, with strict schemas as the quality gate. Here the agent does not edit a CMS but the files themselves — update frontmatter, rewrite an intro, add an internal link — and every change goes straight through the schema and build checks. Each run becomes a small commit you can read, revert, or approve. No database, no admin screen; the repository is the CMS.

Both routes end in the same place: one small, verified change per run, with a human eye on it before it goes live.

Plan B if the loop does not take off

Sometimes it does not work. The data is too thin (less than a few hundred impressions a month), the market too competitive, or the agent keeps making silly choices after months of course-correcting. Then the honest conclusion is: automation adds no value here.

But you still win. Because the agent still functions as an analyst: it pulls your data monthly, flags the notable movements, and puts it all in a fixed report. You do the interpretation and the changes yourself, once a quarter. That is not a self-improving loop, but it is a return on your onboarding time — and sometimes that is the realistic picture for a small website. Start small, measure honestly, and let the automation earn what it is worth. And if you would rather build that cycle with guidance than figure it out alone, AI automation with human control is where we help. Doubting the business case? What SEO costs in 2026 lets you compare the agent approach with what freelancers and agencies charge for the same recurring work. Considering an agent beyond SEO, for your whole business? The guide on hiring someone to build an AI agent walks through the signals, costs, and pitfalls.

This article was first published on Straffe Sites.

Google Search Console setup: verification and first checks

Seppe Gadeyne — Fri, 25 Sep 2026 12:38:15 +0000

A deployed site isn't necessarily an indexed site. Google Search Console lets you check what Google has discovered, which pages it has indexed, and which searches bring people to your site.

This guide covers adding a property, verifying ownership, and the first checks I'd make after setup. You need a Google account and access to the site's DNS settings, or help from whoever manages them.

What Search Console tells you

Google Search Console is Google's free tool for tracking search performance. It reports impressions and clicks, the queries behind them, indexing status, and technical issues that may affect visibility.

Google Analytics measures what visitors do on your site. Search Console measures how the site appears in Google Search before they click. They answer different questions. Search Console alone won't tell you whether a visitor submitted a form or became a customer.

Sign in with your Google account

Open Search Console and sign in. Choose an account you'll retain access to. If you're setting this up for a client, make sure they retain ownership rather than leaving access tied only to your account. You can add other users with their own permissions later.

Add your website: domain or URL prefix

Click "Add property" and choose between two types:

A domain property covers the entire domain, including subdomains and both HTTP and HTTPS. Enter the domain without a protocol or path. This requires DNS verification.
A URL-prefix property covers addresses that begin with the exact prefix you enter, such as https://www.example.com/. Use this when you deliberately want to measure one part or version of the site, or need a verification method other than DNS.

For a whole website, I'd usually start with a domain property. Be careful with URL prefixes: https://example.com/ and https://www.example.com/ are different prefixes.

Verify ownership

For a domain property, copy the TXT value Google provides and add it to the DNS records at your DNS provider. Keep existing records intact. Then return to Search Console and click "Verify."

DNS changes can take time. If verification fails, check the record name and value against your provider's instructions before trying again. Keep the verification record after setup so Google can recheck ownership.

A URL-prefix property supports additional methods, including an HTML file or tag, Google Analytics, and Google Tag Manager. Each has its own requirements. Google's verification documentation explains those requirements and troubleshooting steps.

Submit your sitemap

Open "Sitemaps" and submit the actual XML sitemap URL generated by your site. Common paths include /sitemap.xml and /sitemap-index.xml, but check your deployment rather than guessing.

Open that URL yourself first. It should return the sitemap, not an HTML error page or a login screen. A sitemap helps Google discover URLs; it doesn't guarantee indexing.

Inspect an important page

Paste a complete page URL into the URL Inspection field. Start with your homepage and one important content page.

Read the indexed result before running a live test. The indexed result describes Google's stored information; a live test checks the current page. A successful live test doesn't mean the URL is already indexed.

For a new or substantially updated page, you can request indexing through URL Inspection. That is a request, not a guarantee of inclusion or a particular crawl time.

Use the Performance report

Data collection starts when the property is first added to any Search Console account, even before verification. It doesn't recreate search performance from before that first addition.

Once you have enough data, open the Performance report and select "Search results."

Start in the Pages tab. Click a page with substantial impressions, then switch to Queries to see the searches associated with it. Ask whether the page answers what those people are looking for before changing its title or adding content.

Average position isn't a fixed ranking. It combines different searches, and results vary by device, location, and other factors. Likewise, low click-through rate doesn't prove that a title is wrong. Compare similar queries and conditions, then inspect the actual search results.

If you want more context on choosing relevant queries, I've written a guide to keyword research.

Review indexing separately from traffic

The Pages report under Indexing shows which URLs are indexed and which are not, with reasons. "Discovered, currently not indexed" and "Crawled, currently not indexed" can have several causes. Inspect the affected URL and its content before assuming you need a code fix.

The Sitemaps report shows the status of submitted sitemaps and the URLs Google discovered through them. Those discovery counts aren't a count of indexed pages.

For ongoing work, keep a record of what you changed and when. Review a meaningful period rather than reacting to a single day's movement. If the goal is more inquiries or sales, track those outcomes separately from search clicks.

Adapted from my guide on setting up Google Search Console, originally published on Straffe Sites.

DEV Community: Seppe Gadeyne

Patch PHP2Shell on PHP 7.4 and 8.1 compiled from source, without a panel

Contents

Is this guide for your server?

First: turn on the mitigation today

What the fix is and where it comes from

Step 1: record how your PHP was built

Step 2: build the patched PHP on another machine

Step 3: check, swap and roll back

Verify that the server runs the new binary

What the scripts refuse to do

FAQ

Claude refused to finish patching my servers. My local LLM took over

The job

"Your request was not processed"

What actually happened

Switching to the local model

What Qwen finished

What I take away from this

Patching PHP 5.6 to 8.1 for PHP2Shell on old Ubuntu servers by rebuilding RunCloud's PHP

The problem: two critical bugs and a panel that stops at 8.2

First: make sure the mitigation is actually active

Where the patches come from

Build on the same Ubuntu, without root or Docker

Use RunCloud's own configure flags

Pitfalls we hit

Test in the chroot before anything touches a server

Deploy: swap only the core, keep a way back

Verify: same pages, new binary

The one we did not patch: PHP 5.5

Use our builds, or build them yourself

Not on RunCloud: the generic mode

What this does not fix

Run DeepSeek-V4.1-Flash (748B) locally with Project Maya on one RTX 5090

The model

Why Project Maya

The machine

Install

Start it outside your shell's memory limits

The drafter: measure it on every version

Benchmarks

Quality check

Pitfalls we ran into

Which setup we use when

Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side

The four setups

The machine

How we measured

Speed

Quality on our own tasks

Memory and VRAM

Which one we use when

Caveats

Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090

The model

Why not TabbyAPI

Why Project Maya

The machine

Install

Edit the config before the first start

Start it outside your shell's memory limits

Calibrate

Benchmarks

Quality check

Pitfalls we ran into

Which setup we use when

Run Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant on 128 GB of RAM

What changed since the EXL3 guide

The machine

Why Strata

Pick the quant by RAM, not by bits

Install

If the download is slow

If setup says shard 1 is "short"

Start it outside your shell's memory limits

Go to 512K context

Calibrate for your PC

Benchmarks

Quality checks

A custom chat template, if you need one