DEV Community

Jimmy Yeung
Jimmy Yeung

Posted on

Journey migrating to uv workspace

My company has migrated from poly-repository to a single monorepo recently due to various reasons, like

  • we always want our internal packages up-to-date and in-sync of each other -> there's little gain in maintaining versions for our internal packages (not the main point of the blog today).

And for dependency management, we are still using the poly-repo way of

  • each repo (now dir in monorepo) has its own requirements.txt and its own virtual environment. We will spin each virtual environment up during local development

And of course...the pain points were real:

  • Slow setup: Full environment setup took 12+ minutes, with libraries being redundantly rebuilt for each service
  • Version drift: Service A might run Django 4.2 while Service B ran Django 5.1, leading to subtle bugs when code was shared

When uv introduced workspaces, we saw an opportunity to modernize. The goal: a single uv.lock file guaranteeing consistent dependencies across all services, with faster installation via shared caching.

This is the story of that migration.


Phase 1: Aligning Dependencies (The Hidden Prerequisite)

Before we could create a unified workspace, we faced an uncomfortable truth: UV workspaces enforce a single version per package across all services.

Our per-service approach had been hiding version conflicts for years. When we analyzed our codebase, we found:

Package Service A Service B Service C
Django 4.2.x 5.1.x 4.2.x
factory_boy 3.2.x 3.3.x 3.2.x
pytest 7.4.x 8.0.x 7.4.x

With pip's per-venv approach, this "worked" because each service lived in isolation. With UV workspaces, we needed one truth.

The migration began before the migration: we spent time upgrading packages across all services to consistent versions, fixing breaking changes from major version bumps along the way.

Key learning: UV workspaces expose hidden version conflicts that pip's per-venv approach masks. Resolve these conflicts before migration, not during.


Phase 2: Migrating Libraries First

With dependencies aligned, we started with the foundation: our shared libraries. This was strategically important because:

  1. Libraries have fewer dependencies than services
  2. All services depend on libraries, so migrating them first tests the workspace machinery
  3. Subsequent service migrations become simpler

The Workspace Structure

We created a virtual workspace root at your_workspace/pyproject.toml:

[project]
name = "your-workspace"
version = "0.0.0"
description = "Your Monorepo Workspace - Virtual root package"
requires-python = ">=3.12"

[tool.uv]
package = false  # Virtual workspace - don't install this package
index-strategy = "unsafe-best-match"

[tool.uv.workspace]
members = [
    "internal-package-a",
    "internal-package-b",
    "internal-package-c",
    # ... more packages
]
exclude = ["not-yet-migrated"]

[tool.uv.sources]
internal-package-a = { workspace = true }
internal-package-b = { workspace = true }
internal-package-c = { workspace = true }
# All workspace packages declared as sources
Enter fullscreen mode Exit fullscreen mode

After the library migration, local setup time improved from 12 minutes to 9 minutes, since libraries were now built once and cached.

Learning: The Dependency Bloat Discovery

When we migrated our first service, we hit an unexpected problem. setting up in dependent services started failing with installation errors for packages.

The investigation revealed a fundamental difference:

Approach How it installs internal-package-a Dependencies installed
Pip (old) -e file://../internal-package-a#egg=internal-package-a None - only adds source to sys.path
UV workspace uv sync with PEP 621 All - follows the spec correctly

The old pip approach was a hack. It only added the package source to the Python path without installing any of its dependencies. This "worked" because:

  1. Dependent services only imported specific utility modules
  2. Common deps like Django were redundantly declared in each service's requirements.txt

UV was doing the correct thing by installing all declared dependencies.

The Solution: Optional Dependencies

The problem was that internal-package-a conflated two concerns:

  1. The running service: Needs a, b, c, x, y, z
  2. The exported libraries: Only need a, b, c

We split them using optional dependencies:

[project]
dependencies = [
    # Minimal deps for exported libraries (what consumers actually need)
    "a",
    "b",
    "c",
]

[project.optional-dependencies]
service = [
    # Runtime deps only needed when running the actual service
    "x",
    "y",
    "z",
]
Enter fullscreen mode Exit fullscreen mode

Now:

  • Consumers get minimal dependencies by default
  • Running the service locally: uv sync --extra service
  • Production Dockerfiles: uv export --extra service

Phase 3: Service-by-Service Migration

With the pattern established, we migrated services in dependency order:

  1. First: Services that only depend on libraries
  2. Then: Services with inter-service dependencies
  3. Finally: Services with the most dependents

Each service migration followed the same checklist:

  1. Move requirements.txt contents to [project].dependencies
  2. Separate service-only deps into [project.optional-dependencies].service
  3. Add to workspace members list
  4. Update Makefile to use uv sync --package <name>
  5. Update Dockerfile (this is where things got interesting...)

Learning: The Docker Editable Install Trap

This was our most painful discovery. After deploying a migrated service to QA, we saw:

ModuleNotFoundError: No module named 'configurations'
Enter fullscreen mode Exit fullscreen mode

The mystery: django-configurations was definitely installed during the Docker build. Even stranger, the same image behaved differently depending on the pod:

  • DB job pods: Worked fine
  • Runner/worker pods: Failed with import errors

The Root Cause

When uv export generates requirements, local workspace packages appear as paths:

./internal-package-a
./internal-package-b
./internal-package-c
Enter fullscreen mode Exit fullscreen mode

When uv pip install sees a local path (even without -e), it creates an editable-style install:

__editable__.internal_package_a-0.0.0.pth
__editable___internal_package_a_0_0_0_finder.py
Enter fullscreen mode Exit fullscreen mode

These .pth files add path hooks that reference the source directories (/internal-package-a). In our multi-stage Docker build:

  1. Build stage: Source directories exist (we COPY . .)
  2. Final stage: Only /usr/local/lib is copied, source dirs don't exist
  3. Python startup: .pth files try to hook non-existent paths, causing unpredictable import failures

Why --no-editable Doesn't Help

We tried uv export --no-editable, but it only affects the output format:

# With --no-editable:
./internal-package-a

# Without --no-editable:
-e ./internal-package-a
Enter fullscreen mode Exit fullscreen mode

Both are still local paths. uv pip install treats them identically.

The Fix: Build wheels for local packages first

RUN uv export --package your-service --extra service \
        --frozen --no-dev --no-emit-package your-service > /requirements-raw.txt && \
    # Extract local packages (lines starting with ./)
    grep '^\./' /requirements-raw.txt > /local-packages.txt && \
    # Filter out local paths for external deps
    grep -v '^\./' /requirements-raw.txt | grep -v '^[[:space:]]*#' > /requirements.txt && \
    # Build wheels for each local package
    mkdir -p /wheels && \
    while read -r pkg; do \
        uv build --wheel --out-dir /wheels "$pkg"; \
    done < /local-packages.txt && \
    # Install external dependencies
    uv pip install --system -r /requirements.txt && \
    # Install local packages from wheels (NOT source paths)
    uv pip install --system /wheels/*.whl
Enter fullscreen mode Exit fullscreen mode

Wheels are self-contained. When installed, they extract directly to site-packages with no references to source directories.

Verification

We added a check to catch regressions:

RUN ls -la /usr/local/lib/python3.12/site-packages/__editable__* 2>/dev/null && \
    echo "ERROR: Editable installs found - these will fail at runtime!" && exit 1 || \
    echo "OK: No editable installs"
Enter fullscreen mode Exit fullscreen mode

Results

After migrating all libraries and services:

Metric Before After
make setup time ~12 min ~1 min (a lot faster!!!)
Virtual environments One per service Single workspace venv
Dependency consistency Hoped for Guaranteed by uv.lock
Config files Multiple requirements.txt Single pyproject.toml

Recommendations for Your Migration

  1. Start with a spike: Migrate one service end-to-end to understand the full scope
  2. Invest in dependency alignment upfront: This is the hidden work that makes everything else possible
  3. Plan for Dockerfile changes: The wheel-building pattern should be templated across services
  4. Communicate timing: Multi-branch repos need coordination to avoid merge hell

Conclusion

Migrating to UV workspaces was a significant undertaking, touching every service and library in our monorepo. The upfront investment in dependency alignment and the debugging of Docker editable install issues were the hardest parts.

But the result is worth it: a single source of truth for dependencies, faster local development.

If you're managing a Python monorepo and suffering from version drift or slow setup times, UV workspaces are worth the investment. Just watch out for those editable installs.

Top comments (0)