Every requirement satisfied, still broken: dependency bugs I found in six Python apps

youssef-elaasri — Wed, 11 Nov 2026 12:28:54 +0000

pip check passing doesn't mean your packages work together. It means each package's declared version ranges are met. Whether the installed version of a dependency still has the functions, attributes and parameters the package actually uses is something nothing checks.

To find out how often that goes wrong in practice, I installed six open-source Python applications the way their docs say to (NetBox, CTFd, redash, FlaskBB, healthchecks and paperless-ngx) and checked every call each installed package makes into another against the version installed next to it. Here's what turned up, which patterns keep recurring, and what maintainers can do about each one.

The short version

healthchecks and FlaskBB: no breaks. NetBox: one, and only on Windows.
CTFd: two pinned dependencies that don't work with the other pins.
redash: its SAML library calls a pyOpenSSL class that no longer exists.
paperless-ngx: breaks in three of its dependencies: channels, nltk and llama-index.

I reproduced each one by hand and reported it upstream. All are listed with their issue links at the end. None of them gets past pip install with an error. They all fail later, at runtime, on a code path that ordinary use rarely reaches.

Pattern 1: deprecated, removed, still called

channels on Django 5. CookieMiddleware.set_cookie() converts an aware expires datetime with django.utils.timezone.utc. Django deprecated that alias in 4.1 and removed it in 5.0. channels requires Django>=4.2, so every install on Django 5.x has this line waiting to fail:

import datetime
import django
from django.conf import settings
settings.configure()
django.setup()
from channels.sessions import CookieMiddleware

message = {"type": "http.response.start", "headers": []}
expires = datetime.datetime(2030, 1, 1, tzinfo=datetime.timezone.utc)
CookieMiddleware.set_cookie(message, "sessionid", "abc", expires=expires)
# AttributeError: module 'django.utils.timezone' has no attribute 'utc'

nltk on NumPy 2. The GIS and IIS trainers for MaxentClassifier set the weight of features that never appear in the training data to numpy.NINF, which NumPy 2.0 removed. Train on data where some encoded feature never shows up, and you get AttributeError: np.NINF was removed in the NumPy 2.0 release.

Both fixes are one line: datetime.timezone.utc and -numpy.inf, which work on old and new versions alike. Both had a full deprecation cycle. The warnings were simply never seen, because the code paths are rare: an aware expires, an unattested feature. Fix PRs are open for both (channels#2238, nltk#4008).

What helps: run at least one CI job with deprecation warnings turned into errors (-W error::DeprecationWarning, or filterwarnings = error in pytest's config), against the newest versions your ranges allow. That only catches calls your tests reach, which is exactly why these two survived, but it's cheap and catches most of them.

Pattern 2: an open-ended `>=` meets a removal

pysaml2 and pyOpenSSL. pysaml2 requires pyopenssl>=26.0.0 with no upper bound. pyOpenSSL 26.3 removed crypto.X509Req and crypto.load_certificate_request, deprecated in favour of cryptography's CSR builder. So a fresh pip install pysaml2 today gets a pyOpenSSL where OpenSSLWrapper().create_certificate(info, request=True) raises AttributeError. Creating a self-signed certificate (request=False) still works, which is probably why nobody noticed.

llama-index and huggingface_hub. When you don't pass model_name, HuggingFaceInferenceAPIEmbedding asks InferenceClient.get_recommended_model() for a default. That method is gone from current huggingface_hub, and the package requires huggingface-hub[inference]>=0.19.0, again with no upper bound.

I'm not arguing for upper bounds everywhere. They cause their own resolution problems, and the Python packaging community has mostly settled against them for libraries. The point is that a lower bound is a claim about every future version, and nothing checks that claim. When a dependency announces a removal, it's worth grepping for the name before the release ships.

Pattern 3: re-export shims that outlive their target

llama-index-core has a one-line module:

# llama_index/core/workflow/service.py
from workflows.service import ServiceManager, ServiceNotFoundError  # noqa

The workflows package removed workflows.service in August 2025, and no version in llama-index-core's allowed range still has it. import llama_index.core.workflow.service fails on every supported install. Nothing in the repository imports it anymore, so no test caught it. It only breaks for users who imported it themselves.

Compatibility shims tend to rot quietly, because the code that owns them doesn't use them. A test that just imports every public module would have caught this one.

Pattern 4: exact pins that disagree with each other

CTFd pins its dependencies exactly, which you might expect to rule all this out. It doesn't, because the resolver only checks the ranges each package declares, and old packages declare loose ones:

Flask-Migrate 2.5.3 with alembic 1.16. Flask-Migrate's flask db current passes head_only= to alembic.command.current(), which alembic no longer accepts. CTFd itself only calls upgrade and stamp, so this only bites an admin running flask db current while debugging a migration.
Flask-Script 2.0.6 with Flask 2.1. Flask-Script imports flask._compat, removed in Flask 2.0, so it can't even be imported. Nothing in CTFd uses it anymore. It's a leftover pin.

Pinning gives you reproducible installs, not compatible ones. When a pin is bumped, the packages that depend on it don't get re-checked unless something exercises them.

A bonus from application code

The same check run against an application's own code found something different in pretix: an error message built as _('... seat "%s" ... already sold.', s.name). The seat name goes to gettext_lazy as a second argument instead of being formatted into the string. The exception is created fine, but as soon as the API turns it into a response with str(e), it raises TypeError. The user gets a 500 error instead of the validation message. It's a typo-level bug, but it lives in an error path, which is precisely the code that tests skip.

Why tests didn't catch any of this

Every one of these sits on a path that's easy to skip in a test suite: certificate requests rather than certificates, cookies with aware expiry dates, features that never appear in training data, a debugging CLI command, a module nothing imports, an error message. That isn't sloppiness. Most of these are in libraries, where the maintainers can't know which paths a given app will use. Mostly, it's what happens when the code is correct against the version it was written for, and then that version changes underneath it.

How I found them

I built a tool for this, pipbreak. It reads the source of the installed packages and checks every import, attribute access and call that one package makes into another against what's actually installed, by importing the real modules and inspecting them. It treats the usual compatibility code (try: ... except ImportError, version checks, hasattr guards) as intentional, so it doesn't report the fallbacks that make code work across versions.

pip install pipbreak
pipbreak --deps

It's not magic, and the first runs on these apps produced a lot of noise. On paperless-ngx's environment of 17,000 files, an early release reported 59 errors. Most of what I changed between releases was fixing false positives, until what was left was real: 7 errors on Linux, all genuine mismatches. It also has blind spots: arguments forwarded through **kwargs, methods on objects whose type is only known at runtime, compiled extensions, and behaviour that changes behind an unchanged signature. It complements tests; it doesn't replace them.

The reports

channels: django/channels#2237, fix in #2238
nltk: nltk/nltk#4007, fix in #4008
pysaml2: IdentityPython/pysaml2#1048
llama-index: run-llama/llama_index#23871 and #23872
CTFd: CTFd/CTFd#3147
pretix: pretix/pretix#6722

If you maintain one of these and I've got a detail wrong, tell me and I'll fix it here. And if a dependency update has broken your app in a way that sounds like this, I'd like to hear about it, especially if you think pipbreak would have missed it.

DEV Community: youssef-elaasri