DEV Community

Kushagra
Kushagra

Posted on

The bug that doesn't throw: monkey-patch ordering in production published: true

This came up while building AgentCost, an open-source LLM cost tracker, patching multiple SDKs to capture token usage consistently is exactly where this kind of bug likes to hide.

The setup

Three hours. One-line fix. Zero exceptions were thrown the entire time.

This is a write-up of a bug class that's easy to introduce and brutal to find:
behavior that depends on the order code loads in, not the order it's called in.

The bug

Monkey-patching, overriding a method or function at runtime, is common in
Python and JS libraries. You wrap a function to add logging, tracking, or
retry behavior without touching the source:

original_call = SomeClient.call

def patched_call(self, *args, **kwargs):
    log_usage(*args, **kwargs)
    return original_call(self, *args, **kwargs)

SomeClient.call = patched_call
Enter fullscreen mode Exit fullscreen mode

Fine, in isolation. The problem starts when more than one thing patches
the same function
, and each assumes it's the only one doing it.

Why it fails silently

Whichever patch loads last wraps all the earlier ones. Change the import
order, a different library initializes first, a lazy import resolves at a
different point, and the wrapping order changes.

No exception. No warning. The function still returns a value. It just
might not be the value you expect, and there's nothing in a stack trace to
tell you why.

That's what makes this class of bug expensive: loud bugs are cheap to find.
Quiet ones cost hours, because everything looks like it's working.

The fix

Cheap insurance, one line at patch time:

import logging

def patch(name, order):
    logging.info(f"patching {name}, order={order}")
    # ... apply patch
Enter fullscreen mode Exit fullscreen mode

If output ever changes unexpectedly, you have a log trail showing exactly
What wrapped what, and in which order?

The generalizable lesson

Any time behavior depends on load order rather than an explicit call
graph, you've introduced an invisible dependency between unrelated parts of
the codebase. That's worth catching in code review: "does this patch
assume anything about what else has patched this function?" rather than
discovering it during an incident.


Curious if others have run into ordering bugs like this outside of Python —
seems like it'd show up anywhere runtime patching or dependency injection is
common (Ruby, JS build tooling, etc).

Top comments (0)