Workers Should Not Inherit Your Logging Handlers

Have you ever watched pool.join() sit there after a logging cleanup that looked boring in review? The workers are not crunching data. They are waiting on a lock the parent copied into them, or they are writing nowhere because the child never got a handler at all.

That split is the whole story. Fork and spawn do not share a logging world. A free coding model will still offer you one setup_logging() import and call it tidy.

The hang is a lock, not a slow map

Stdlib logging wraps every handler in an RLock. A parent thread can hold that lock during emit(). If the process forks at that moment, the child receives a lock that is already taken. Nobody in the child will release it. The next logging.info() in a worker blocks forever, and the pool join waits on that worker.

Spawn does not copy the lock. It also does not copy the handlers. The child starts a fresh interpreter. Your import-time basicConfig() in the parent never ran there. Worker logs vanish, and the job looks “fine” because the parent still prints a summary.

So the same cleanup produces two failures. One machine deadlocks. Another machine goes quiet. Reviewers who only run macOS spawn will miss the Linux fork hang. Reviewers who only run Linux fork will miss silent workers on Windows.

Import time is the accident

The usual “cleanup” is not a new architecture. Someone moves logging.basicConfig(...) into a helpers module. Every worker import then configures the root logger, or the parent configures it before the pool starts, and children inherit whatever was there.

You do not need a messy god module for this to bite. One shared logutil.py imported by the CLI and by the worker is enough. The model sees duplication and wants a single function. That is the wrong merge.

Think of handlers as file descriptors with opinions. You would not let every child inherit an open socket and also open it again. Logging is that socket, plus a lock, plus a format string that finance will one day grep.

A lab that shows both failures

The file below is sample code. It is not a production collector. It installs a slow handler so the fork race is easy to hit, then starts a worker that also logs. On interpreters that still offer fork, the child often never finishes. On spawn, the child usually finishes and prints nothing through that handler.

# lab_logging_pool.py — sample lab, not a service
from __future__ import annotations

import logging
import multiprocessing as mp
import threading
import time


class SlowStreamHandler(logging.Handler):
    def emit(self, record: logging.LogRecord) -> None:
        with self.lock:
            time.sleep(0.4)
            print("PARENT_OR_INHERITED:", self.format(record), flush=True)


def worker(n: int) -> str:
    log = logging.getLogger("job")
    log.info("worker got %s", n)
    handlers = [type(h).__name__ for h in logging.getLogger().handlers]
    return ",".join(handlers) or "NO_ROOT_HANDLERS"


def run_lab(start_method: str) -> None:
    root = logging.getLogger()
    root.handlers[:] = [SlowStreamHandler()]
    root.setLevel(logging.INFO)
    logging.getLogger("job").setLevel(logging.INFO)

    t = threading.Thread(target=lambda: logging.getLogger("job").info("parent emit"), daemon=True)
    t.start()
    time.sleep(0.05)

    ctx = mp.get_context(start_method)
    proc = ctx.Process(target=worker, args=(7,))
    proc.start()
    proc.join(timeout=3.0)
    print(start_method, "alive", proc.is_alive(), "exit", proc.exitcode)


if __name__ == "__main__":
    for method in mp.get_all_start_methods():
        print("---", method, "---")
        run_lab(method)

Run it twice on the same machine. Then run it on a second OS if you have one. You are collecting a map of start methods, not a vibe about “logging best practices.”

python lab_logging_pool.py

If fork stays alive after the timeout, you watched the copied lock. If spawn exits quickly and the returned handler list would have been empty, you watched a silent child. Both are regressions. Neither is a style argument.

QueueListener is a topology, not a rename

The stdlib answer is a queue. The parent owns a QueueListener and a real handler. Workers get a QueueHandler only. After fork or spawn, workers must not keep StreamHandlers that point at the same stdout lock the parent uses.

The snippet below is a proposed pattern. It is unlabeled as production-ready because you still have to decide JSON versus text, and you still have to close the listener on the path that handles SIGTERM.

# proposed_queue_logging.py — proposed pattern, unexecuted in production
from __future__ import annotations

import logging
import logging.handlers
import multiprocessing as mp
from logging.handlers import QueueHandler, QueueListener

_queue = None
_listener = None


def start_parent_logging() -> None:
    global _queue, _listener
    _queue = mp.get_context("spawn").Queue(-1)
    handler = logging.StreamHandler()
    handler.setFormatter(logging.Formatter("%(processName)s %(levelname)s %(message)s"))
    _listener = QueueListener(_queue, handler, respect_handler_level=True)
    _listener.start()
    root = logging.getLogger()
    root.handlers[:] = [QueueHandler(_queue)]
    root.setLevel(logging.INFO)


def worker_logging_init() -> None:
    if _queue is None:
        raise RuntimeError("parent must call start_parent_logging first for spawn pickling")
    root = logging.getLogger()
    root.handlers[:] = [QueueHandler(_queue)]
    root.setLevel(logging.INFO)


def stop_parent_logging() -> None:
    if _listener is not None:
        _listener.stop()

Notice the initializer. Spawn will not remember parent basicConfig. Fork may remember too much. The initializer is how you force a single handler graph in both worlds. A model that deletes the initializer because “the parent already configured logging” is editing the topology.

A review gate that inspects handlers

Do not argue about format strings first. Assert who owns what. The test below is sample pytest. It skips start methods the interpreter does not provide, which is the honest thing to do in 2026 instead of pretending every laptop still defaults to fork.

# test_handler_graph.py — sample harness
from __future__ import annotations

import logging
import multiprocessing as mp

import pytest
from logging.handlers import QueueHandler


def _worker_report(_n: int) -> tuple[str, ...]:
    return tuple(type(h).__name__ for h in logging.getLogger().handlers)


@pytest.mark.parametrize("method", mp.get_all_start_methods())
def test_workers_do_not_keep_stream_handlers(method: str) -> None:
    ctx = mp.get_context(method)
    queue = ctx.Queue(-1)

    def _init() -> None:
        root = logging.getLogger()
        root.handlers[:] = [QueueHandler(queue)]
        root.setLevel(logging.INFO)

    with ctx.Pool(2, initializer=_init) as pool:
        reports = pool.map(_worker_report, [1, 2])

    for names in reports:
        assert names == ("QueueHandler",), names
        assert "StreamHandler" not in names
        assert "SlowStreamHandler" not in names

Run that on your machine before anyone “simplifies” logging. If a patch adds basicConfig to a module imported by workers, this test is the objection. A green explanation from a model is not.

python -m pytest test_handler_graph.py -q

Decision table for the review

Start method you actually use	What the child should hold	Cleanup that looks innocent
fork	QueueHandler only, no inherited StreamHandler	Import-time basicConfig in a shared helper
spawn	QueueHandler installed in an initializer	Assuming parent handlers pickle across
forkserver	Explicit reconfigure in the server or initializer	Copying a FileHandler path into every child
mixed CI images	The intersection of the rows above	“It passed on my Mac” as evidence

Read the table as a gate. If CI images disagree on start methods, you do not have one logging setup. You have a matrix. Collapsing that matrix into a single import is how the hang ships.

Where a free model is allowed to talk

After the handler graph test exists, a model is useful for one narrow job: drafting a comment that names the start methods and forbids import-time basicConfig. It is not useful for inventing a new logging framework, swapping to structlog, or “simplifying” the initializer away.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those are the only product facts this article relies on.

Feed the failing test and the lab file together. Ask for a comment block above the initializer, nothing else. If you want that comment drafted off to the side of your laptop, the free model access on the free server is enough for that request. Paste the comment back. Run pytest again. The test stays the reviewer.

This lab will not save every logger

It does not prove your JSON field names. It does not prove syslog facility numbers. It does not make print() thread-safe. It does not fix a custom handler that opens a network socket in emit() without its own timeout.

Audit logs that must not travel through an in-memory queue belong on a different path. If a worker crash can drop the last N records, a QueueListener is the wrong sink for that stream. Do not pretend the topology test equals durability.

Python’s default start method has not been a single global truth across OS images. Do not hard-code “Linux means fork” in a runbook you will reread next quarter. Query mp.get_all_start_methods() and mp.get_start_method() in the lab instead of quoting a blog from memory.

Leave this on the shelf when it is the wrong fight

Skip the lab if the process has no children. Skip it if you already run only spawn and workers are thin subprocesses that log to their own files by design. Skip it if you cannot run pytest where the workers actually start.

Skip it for libraries that must not install root handlers at import. That is a different contract, and this article would push you the wrong way. Skip it when the product requirement is “children inherit the parent tty exactly,” because QueueHandler will not give you that.

Teams that want one PR to replace logging, metrics, and tracing should stop. Handler topology is enough work. Formatters can wait for a later argument that is actually about formatters.

Shut the pool down on purpose

pool.join() is not a performance metric. It is a statement that every child finished without sitting on a copied lock. Import-time logging steals that statement and dresses it up as cleanup.

Keep handlers out of the worker import graph. Install a QueueHandler in an initializer you can see in review. If a patch needs a paragraph to justify basicConfig in a shared module, the paragraph is the bug.