How I stopped re-resolving dependencies without rewriting pip

I got tired of watching pip think for at least 45 seconds before I could write a single line of code only to get back to waiting for dependency resolution and/or installation, so I built a hack that remembers the answer.

What was the issue really?

I built a plug-and-play FastAPI engine ChaCC‑API that loads modules with their own requirements.txt. With every restart, it re‑resolved every module's dependencies. Like running pip install over and over.

In development, with a medium‑sized module, that meant 45 seconds of waiting. Even when nothing changed.

Restart server? Wait. Edit a module? Wait. Add a module? Wait again.

Then this happened:

I built chacc-dependency-manager. It does one smart thing though very small :)

Cache the result of pip-compile.

cache_key = hash(requirements + python_version + os)
if cache_key in cache:
    pinned = cache[cache_key]           # use saved answer
else:
    pinned = run_pip_compile()          # solve once (slow)
    cache[cache_key] = pinned

# Only install what's actually missing
install_missing_packages(pinned)

No extra resolver runs. No unnecessary pip install calls. Just remember the plan, then do only what's needed.

Does it work?

• First run (or after requirements change): ~45 seconds (resolve + install)
• Cache hit (no changes): <2 seconds (skip resolution, install only missing packages – usually nothing)

In my dev workflow, that 45‑second wait on every restart is now gone when dependencies are stable.

But be real: If you're actively editing requirements.txt, you'll pay the price if not worse each time you change it. This hack shines when you're not changing dependencies often, exactly my case.

What it won't do

• Resolve conflicts – still uses pip-compile for that.
• Make cold installs faster – first run is full price.
• Beat uv in a speed contest - if you're starting fresh, just use uv. It's faster and smarter.
• Warn you about version conflicts across modules – those will still surface at install time.

It's a tiny caching wrapper for a specific modular workflow. That's it.

Try it if you're tired of waiting

Standalone or try and use ChaCC‑API (toggle ENABLE_PLUGIN_DEPENDENCY_RESOLUTION) to see it in action.

🔗 GitHub

🔗 PyPI

What's your "I just want to cache this" story?

#buildinpublic #pip

Comments

[TopStar AI]

This is a really clever solution! Caching the result of pip-compile and only installing missing packages is such a practical improvement for modular workflows—especially for FastAPI projects with multiple plugins. I love how it respects the first-run cost but drastically speeds up subsequent restarts.

I’d be very interested in collaborating or testing this further. I work on modular Python systems and dependency management optimizations as well. It could be interesting to combine ideas for caching strategies, conflict detection, or even integrating this approach into CI/CD pipelines to reduce build times.

Have you experimented with combining this cache with virtual environments per module, or other strategies to further isolate and speed up dependency resolution? I’d be happy to explore that together.

[Jonas G Mwambimbi]

Sure, combining our ideas on caching strategies might be something.
I haven't invested much in conflict resolution yet, that's definitely something to explore.

Speaking of caching dependencies across multiple virtual environments, I haven't tried that. There is solution that I think it's fragile, trying to symlink site-packages. This can work but for compiled packages (like numpy, cryptography), it gets tricky quickly.

Would love to explore this further. Let's connect.

[Jonas G Mwambimbi]

I will text you i. WhatsApp and we can proceed there