Why uv Now Deduplicates Every Wheel in Its Cache (and How It Does It)

The Problem: Duplicate Wheels in the Cache

A developer working on a monorepo with two Python projects—a FastAPI web service and a data pipeline—notices something odd. Each project has a requirements.txt file listing around 200 dependencies. When they run uv pip install for the first project, uv downloads and extracts 200 wheels into its global cache. Then they run uv pip install for the second project, and uv does the same thing again.

But here's the kicker: these two projects share roughly 150 of the same dependencies. pydantic==2.10.4, httpx==0.27.2, typing-extensions==4.12.2—the list goes on. Yet when they check their disk usage, they find that uv has stored 400 copies of extracted wheels on disk. The same typing-extensions wheel, byte-for-byte identical, exists twice in the cache. They're effectively paying double the storage cost for a dependency they use in both projects.

This isn't an edge case. In any Python environment with multiple projects, virtual environments, or CI pipelines, the same packages get pulled down and extracted repeatedly. The waste compounds quickly. A monorepo with 10 microservices might store 10 copies of numpy, 10 copies of pandas, and 10 copies of scikit-learn—each taking up hundreds of megabytes.

Why does this happen? Prior to the content-addressed cache feature, uv's cache stored extracted wheels in archive-v0 under randomly generated IDs. If the same wheel was reached through different cache entries—for example, fetched from two different indexes—uv stored duplicate copies because it identified extracts by their source rather than their contents.

This identity-based approach is simple and fast—you can look up a package by name and version instantly. But it leads to significant redundancy when many projects share the same dependencies.

Before: How uv's Cache Organized Data

Before the content-addressed cache features landed, uv's cache had a two-tier structure. Downloaded wheels lived in the wheels-v0 or wheels-v1 bucket, keyed by a combination of the package name, version, and a hash of the wheel file itself. Extracted wheels lived in archive-v0, and each extract got its own randomly generated directory ID.

You can inspect your current uv cache location with:

uv cache dir

On Linux and macOS, this typically outputs something like $HOME/.cache/uv. On Windows, it's %LOCALAPPDATA%\uv\cache.

If you peek inside the archive-v0 directory, you'll see a bunch of seemingly random directory names:

ls $(uv cache dir)/archive-v0/

On a real system, this might show entries like:

0VSKRv7KPsU6OVxKhRKyA/
3vlR8U66f3qKdzSrxDtyR/
gekN10oQ_Vp-g2TxEYqsD/

Each of these opaque directories contains the full extracted contents of a single wheel. The IDs are randomly generated—they carry no relationship to the files inside. This means that if you install the same wheel from PyPI and then from a local directory, uv treats them as different cache entries and stores two copies of the extracted files.

To see just how much duplication exists, you can use tools like du to measure the total size of the cache and compare it against the unique file count:

du -sh $(uv cache dir)
ls -R $(uv cache dir)/archive-v0/ | wc -l

For a project with many dependencies, the archive-v0 directory can contain tens of thousands of files. A significant portion of these are duplicates—the same .py files, the same compiled extensions, the same native libraries, stored over and over again.

Content-Addressable Storage: A Brief Primer

Content-addressable storage (CAS) is a pattern where data is stored and retrieved by its content hash rather than by a name or location. Instead of asking for "file typing-extensions-4.12.2.dist-info," you ask for "the file whose BLAKE3 hash is b7a3...." If two files have identical contents, they produce identical hashes, and the storage system returns the same object for both requests.

This concept is not new. Git uses content-addressable storage for its object database. Every blob, tree, and commit is identified by its SHA-1 hash, which is why Git can detect duplicate files across your repository and only store them once. Docker layer caching works on a similar principle—if two images share the same base layer, they can reuse it without storing a second copy. npm deduplicates tarballs in its cache using content hashes, ensuring that if you install the same package in different projects, you only download it once.

The key insight is that content-addressable storage turns the problem of "how do I avoid storing the same data twice" into a simple lookup: compute the hash of the content, check if an object with that hash already exists, and if so, reuse it. The hard part is managing the links—making sure that when you ask for the contents of a package, the system knows which hashed objects to assemble, and when a package is removed, the system can safely garbage-collect unreferenced objects.

How uv's Content-Addressed Cache Works

uv's content-addressed cache evolution happened in two stages. The foundational work landed in PR #19693, which introduced content-based directory hashes for entire extracted wheels. The idea was simple: instead of storing an extracted wheel under a randomly generated ID, compute a hash of the directory's entire contents, and store it under that hash. If two different wheels extract to the exact same directory tree, they'll produce the same hash, and uv can deduplicate them.

The more significant optimization came in PR #21327, which moved deduplication from the wheel level to the file level. In this newer implementation, every file extracted from a wheel is stored individually in a files-v0 bucket, keyed by its BLAKE3 content hash. The original directory structure is preserved through hardlinks—the archive-v0 directory contains hardlinks pointing to the actual file objects in files-v0.

Here's how the flow works:

1. When uv needs to extract a wheel, it iterates through every file in the archive.
2. For each file, it computes a BLAKE3 hash of the file's contents.
3. It checks if an object with that hash already exists in the files-v0 bucket.
4. If the object exists, uv creates a hardlink from the extracted wheel's location to the existing object.
5. If the object doesn't exist, uv writes the file to the files-v0 bucket under its hash, then creates a hardlink to it.
6. The extracted wheel directory in archive-v0 now contains hardlinks to shared file objects, with each object stored exactly once on disk.

This approach preserves the existing installation pipeline—the wheel installation step still reads files from their expected paths in archive-v0—but the underlying storage is now fully deduplicated.

The PR benchmarks showed impressive savings. On a local machine with a typical Python cache, the file-level deduplication saved 545.2 MiB of disk space, or about 10% of the total cache. The optimization covered all payload files in the cache—not just executables and native libraries, but every .py, .so, .dll, and data file.

The performance impact was also carefully measured. Cold installs—where the cache is empty and files must be written for the first time—showed a slowdown of less than 4% on median. Warm installs, where the cache is already populated, showed no statistically significant difference. The team found this tradeoff worthwhile—a small performance hit on cold installs for a noticeable reduction in disk usage.

The feature uses BLAKE3 as its hashing algorithm, which is known for its speed and parallelizability, making it well-suited for hashing many files during wheel extraction.

Enabling the Preview Feature

The content-addressed cache features are currently behind preview flags. To use them, you need to explicitly opt in. uv provides several ways to enable preview features.

First, you can enable the feature via the --preview-features flag on individual commands:

uv pip install --preview-features content-addressed-cache -r requirements.txt

To enable it for all commands in a session, you can use the UV_PREVIEW_FEATURES environment variable:

export UV_PREVIEW_FEATURES=content-addressed-cache
uv pip install -r requirements.txt

You can also enable preview features in your uv.toml file or under [tool.uv] in your pyproject.toml:

[tool.uv]
preview-features = ["content-addressed-cache"]

For the physical space accounting feature—which accounts for hardlinks when reporting cache size—you need the cache-physical-space preview feature as well. You can enable multiple features simultaneously:

uv cache size --preview-features content-addressed-cache,cache-physical-space

Or set the environment variable:

export UV_PREVIEW_FEATURES=content-addressed-cache,cache-physical-space

Once enabled, uv will start using content-addressed storage for new wheel extractions. Existing cache entries will remain in their current format—uv doesn't automatically migrate old cache data to the new structure. You can either let new installs populate the new format gradually, or you can clear your cache and start fresh:

uv cache clean

Note that preview features are subject to change. uv explicitly warns that they may be modified or removed without notice, and they should not be relied upon in production environments until stabilized.

Measuring the Savings

To measure how much space your cache is actually using on disk—accounting for hardlinks—you need the cache-physical-space preview feature. Without it, uv cache size reports the apparent size of the cache, which counts the same file multiple times if it's hardlinked in different locations. With the physical space accounting, the command reports the actual disk usage.

Run the following command to see your cache's physical space usage:

uv cache size --preview-features content-addressed-cache,cache-physical-space --human

The --human flag displays the size in a human-readable format (e.g., 1.2 GiB instead of raw bytes).

The actual savings you'll see depend on your workload. The benchmark in PR #21327 showed a 545.2 MiB reduction on a local machine, which was about 10% of the total cache. However, that benchmark was on a single machine with a specific set of packages. In practice, the savings could be higher or lower depending on how much overlap exists between your dependencies.

For workloads with many large packages that share common files—like multiple machine learning frameworks that bundle the same native libraries—the savings could be significantly more. The per-file selection table from the PR shows the pattern: selecting only executables and native libraries saves 275.7 MiB across 3,336 files, while selecting all payload files saves 545.2 MiB across 134,222 files.

To see the breakdown, you can also use system-level tools to compare the apparent size of the extracted tree against the actual disk usage of the object store:

du -sb $(uv cache dir)/archive-v0   # Apparent size: every hardlink counts full
du -s  $(uv cache dir)/files-v0     # Disk usage: each unique object once

The difference between these numbers represents the space saved by file-level deduplication.

What This Means for CI and Local Development

For local development, the primary benefit is reduced disk usage. If you work on many Python projects simultaneously, a smaller cache means you're less likely to run out of disk space on your development machine. It also means less SSD wear—hardlinking does not write new data to disk, so the cache's write amplification is lower.

For CI pipelines, the impact is more nuanced. CI systems often cache the ~/.cache/uv directory between runs to avoid re-downloading packages. But the cache's archive-v0 directory contains many small files that take time to compress and decompress. With file-level deduplication, the cache contains fewer distinct file objects in files-v0, which may improve compression ratios and reduce cache transfer times.

However, the best practice for CI is still to cache only the downloaded wheels (wheels-v1) and not the extracted archives (archive-v0), since the archive can be regenerated from the wheels. The content-addressed cache doesn't change this recommendation—if anything, it makes it more important to be selective about what gets cached.

For monorepo setups, where many projects share dependencies, the benefits compound. A single typing-extensions wheel extracted once into files-v0 and hardlinked across 10 project caches would save 9 copies of the extracted files. Over hundreds of shared dependencies across dozens of projects, the savings can add up quickly.

It's worth noting that this feature is still in preview. Both PR #19693 (wheel-level dedup, merged 2026-08-25) and PR #21327 (file-level dedup, merged 2026-08-31) are in the development branch and have been released under the content-addressed-cache preview flag in uv 0.12.7. Early adopters should expect some rough edges, especially around cross-platform compatibility. The initial implementation was validated on Linux with the ext4 filesystem. Supporting macOS, Windows, and other filesystems may require additional work. The team has considered edge cases like reflink support on copy-on-write filesystems (e.g., Btrfs, ZFS, APFS), but these features are still under development.

---

Originally published on Dispatch.