DEV Community

Cover image for A single-binary Rust worker that drains WordPress job queues
Project Flash Build
Project Flash Build

Posted on

A single-binary Rust worker that drains WordPress job queues

WordPress is a fine place to decide that work needs to happen and a terrible place to do heavy work.
Everything you run inside a request competes with your visitors; everything you run in WP-Cron competes
with luck. When our workflow engine needed to run shell commands, move files around and call LAN-only
APIs, we refused to do any of that inside the PHP request lifecycle — and we didn't want a SaaS in the
middle either. So we wrote wp-executor: a single-binary worker in Rust that polls a WordPress site
for jobs and runs them on your own machine, in your own tools. It's open source (MIT OR Apache-2.0):
https://github.com/Project-Flash-Build/wp-executor

Project Flash cross-plugin architecture — WordPress publishes a REST job queue that the wp-executor worker drains

The shape of the problem

The WordPress side (a workflow engine we build commercially) models automations as graphs. Most steps run
happily in PHP. But some steps are host-side by nature: run a script, read or write a file, hit an
internal service the web server can't reach. Those need an execution boundary that is not the WordPress
process — different machine, different privileges, different failure domain.

The classic answers are all unsatisfying:

  • WP-Cron / PHP daemon: still the same machine, same PHP, same privileges — no boundary at all.
  • A SaaS runner: now your shell commands, file paths and internal endpoints live in someone's cloud.
  • A message broker (RabbitMQ, Redis): real infrastructure — but now every WordPress install needs a broker, and most sites will never justify one.

We picked the boring fourth option: WordPress itself is the queue. The site already has a database, a
REST API and an authentication story. The engine writes jobs to a table; a REST surface exposes
poll/lease/complete; anything that can speak HTTPS can be a worker.

The protocol: poll, lease, execute, report

The worker's loop is deliberately dull:

  1. GET the versioned contract document on startup — which capabilities the server expects, queue endpoints, signing requirements. The binary treats the server's contract as the source of truth, so a server upgrade doesn't require a worker rebuild unless the capability set itself changes.
  2. Poll the queue endpoint on its own cadence (long-poll-ish, configurable interval).
  3. Lease a job. Leases expire: if a worker dies mid-job, the job returns to the queue instead of vanishing. Idempotency keys keep a retried job from running twice.
  4. Execute the job's capability locally.
  5. Report a uniform result: { exit_code, stdout, stderr, output, duration_ms, error }.

Authentication is a per-worker bearer token, plus (by default) an X-PFW-Signature HMAC-SHA256 of the
request body, so a leaked TLS termination point still can't forge job results.

Capabilities, not arbitrary code

The worker refuses to be a remote shell. It implements exactly six typed capabilities — shell.run,
fs.read, fs.write, fs.list, http.request, system.info — and every one of them is gated by an
allowlist in the worker's own config. The server side keeps a per-worker allowlist too; a job runs
only if both agree. A compromised WordPress can't suddenly make your build machine exfiltrate files it
was never allowed to touch: the worker-side allowlist is enforced before execution, on hardware the
WordPress admin doesn't control.

Every capability carries a hard timeout and returns the same result envelope, which makes the workflow
side pleasantly uniform: a shell script that exits 3 and an HTTP call that returns 503 look structurally
identical to the graph.

Why Rust (the honest version)

Not for speed — the worker spends its life waiting on I/O. The actual reasons:

  • Single static binary. The install story on a random VPS, a Mac mini or a Windows box is "download, unzip, run". No PHP version matrix, no pip, no node_modules. rustls means not even an OpenSSL dependency.
  • A type system for the contract. Job payloads, capability schemas and result envelopes are typed end to end. Deserialization failures are loud and early.
  • Predictable long-running behaviour. A worker is a daemon; daemons written in scripting languages accumulate weird state. Ownership makes the "runs for six months" case boring.
  • Cross-compilation. Linux x86_64, macOS x86_64/aarch64 and Windows from one codebase, in CI.

The test story mattered too: the integration suite runs the whole worker loop against
wiremock standing in for WordPress, so cargo test needs no
WordPress at all.

Running it

wp-executor --base-url=https://your-site.tld --token=pfw_worker_1_xxx probe
Enter fullscreen mode Exit fullscreen mode

A successful probe prints the upstream contract and exits 0. Install scripts ship for systemd, launchd
and Windows services; config is one TOML file (base_url, bearer_token, and optionally the allowlist,
poll interval, lease TTL, signing toggle).

What this buys you

A WordPress workflow can now say "run this backup script on the office NAS", "transcode this upload on
the GPU box", "call the ERP that only exists on the LAN" — and the thing that executes it is a process
you installed, on a machine you own, under an allowlist you wrote. WordPress stays the control
plane; your hardware stays the data plane; nothing transits a third party.

The worker is MIT/Apache-2.0 and needs no license to run. The WordPress side that publishes the queue is
our commercial workflow engine (that's the business model, stated plainly); its wire contract is public
REST and the worker consumes it as documentation. If you want to see the full picture there's a worked
example — a WooCommerce order turning into a ticket, a workflow, an AI triage and an executor-generated
RMA file — at https://project-flash.com/use-case.

Questions about the lease semantics, the allowlist model or the Rust internals are welcome — the code is
all in the repo.

Top comments (0)