DEV Community

gritfeld-design
gritfeld-design

Posted on

WorkerDeck is now on PyPI (and its own test suite fell into the zombie trap)

A few days ago I wrote about the bug that taught me how to run one process
per user
and extracted the solution into a small library called
WorkerDeck.
Since then it grew up a little. As of today:
bashpip install workerdeck
That's it. No git clone, no requirements.txt. And here's the part I only
realized while packaging it: the core is pure Python standard library.
Zero dependencies. The Flask dashboard I ship in the repo is just a demo —
the engine itself is one class built on subprocess, os, signal and
json.
What changed
Three things, all boring, all the kind of boring that makes a library
trustworthy:

  1. Proper packaging. A pyproject.toml, a clean wheel, optional extras (pip install "workerdeck[dashboard]" if you want the demo UI). The library installs exactly two files: init.py and manager.py.
  2. A test suite that tests the actual claims. WorkerDeck's whole pitch is "safe to restart": your workers keep running while the manager restarts, and the new manager re-adopts them instead of reporting them dead. So the tests exercise exactly that — a second manager instance on the same work_root must find the live worker, match it by pid and start time, and be able to stop it. Plus the crash circuit breaker: a worker that exits non-zero on every start must end up parked in a failed state, not hot-looping forever.
  3. CI. Every push now runs the suite on Python 3.9 through 3.12. Green badge in the README, the universal sign of "someone actually runs this." The test suite fell into my own trap Here's my favorite part. In the original article I wrote about the zombie process trap: os.kill(pid, 0) happily reports a defunct (zombie) process as alive, so a naive liveness check tells you a dead worker is still running. The first version of my test suite used... a naive liveness check. The adoption test kills a worker behind the manager's back, then asserts a fresh manager won't claim it's running. The kill worked. The assertion failed. Why? The killed worker became a zombie — its Popen handle still lived in the test process and nothing had reaped it — and my little pid_alive() helper, built on os.kill(pid, 0), swore the corpse was alive. The library handles this correctly (its internal check reads the process state and treats Z as dead). My test helper didn't. The fix was satisfying: the tests now import the library's own liveness check, which means the zombie handling is no longer just documented — it's load-bearing. If it ever regresses, two tests fail immediately. I wrote an article warning about a trap, then walked into it the same week while testing the library that exists because of it. Process management stays humbling. Try it pythonfrom workerdeck import WorkerManager

mgr = WorkerManager(worker_script="worker.py", work_root="./work")
mgr.start(user_id=42) # isolated process, own workdir, own log
print(mgr.status(42).state) # "running"

...restart your app, rebuild the manager...

mgr2 = WorkerManager(worker_script="worker.py", work_root="./work")
print(mgr2.status(42).state) # still "running" — same pid, re-adopted

Install: pip install workerdeck
Repo: https://github.com/gritfeld-design/workerdeck
Backstory: The bug that taught me how to run one process per user

It's 0.1.0 and the scope is deliberately small: single host, Linux/POSIX,
one class. If that's the shape of your problem, it might save you the week
of pid-recycling lessons it cost me. Issues and war stories welcome.

Top comments (0)