uv: The Fast Python Package & Project Manager for Data Teams

The uv python package manager is the tool that finally makes "set up the environment" a sub-second operation instead of a coffee break — and for data teams, where every notebook, every Airflow worker, every training container, and every CI job re-resolves the same wall of dependencies (pandas, pyarrow, numpy, scikit-learn, half of PyPI), that speed is not a vanity metric but a change in how you work. Written in Rust by the team behind Ruff, it collapses the jumble of tools most Python projects accrete over their lifetime — the installer, the virtualenv creator, the requirements compiler, and the interpreter-version juggler — into one binary that resolves and installs a full dependency tree in the time the old stack spent printing its first "Collecting…" line. When rebuilding an environment costs one second, you stop nursing a stale virtualenv for weeks and start treating environments as disposable, reproducible artifacts.

This guide is the walkthrough you wished existed the first time a teammate said "just use uv" and you had to figure out where pip install, python -m venv, pip-compile, and pyenv all went. It covers why one fast tool as a pip replacement changes team behavior, the project loop that writes a pyproject file and resolves a lockfile you commit, how virtualenv management and interpreter pinning combine into reproducible environments that are byte-identical across a laptop, a CI runner, and a production image, how uv slots into Docker and continuous integration without paying the install tax on every build, and how to migrate an existing pip, poetry, or conda project without a big-bang rewrite. Each section pairs a teaching block with a worked interview answer — code, a step-by-step trace, an output table, then a concept-by-concept breakdown of why it works.

When you want hands-on reps immediately after reading, drill the pandas practice library →, rehearse pipeline packaging on the ETL practice library →, and harden your scripts on the defensive-coding practice library →.

---

On this page

• Why uv replaces pip, venv, pip-tools, and pyenv
• Projects, pyproject.toml, and the uv.lock lockfile
• Reproducible environments and Python version management
• uv in Docker and CI
• Migrating from pip, poetry, and conda
• Cheat sheet — uv recipes
• Frequently asked questions
• Practice on PipeCode

---

1. Why uv replaces pip, venv, pip-tools, and pyenv

The one-tool argument — a single Rust binary absorbs the installer, the resolver, the virtualenv, the lock compiler, and the Python-version manager

The one-sentence invariant: uv is a single Rust-built binary that does the job of pip (install), venv/virtualenv (isolated environments), pip-tools (compile a resolved lockfile), pipx (run tools in isolation), and pyenv (install and pin Python interpreters) — and it does all of it fast enough that "rebuild the environment from scratch" stops being a decision you agonise over and becomes the default you reach for constantly. The reason this matters for data teams specifically is that the Python data stack is heavy: a modest pipeline pulls in pandas, numpy, pyarrow, sqlalchemy, boto3, and a dozen transitive C-extension dependencies, and every one of those has to be resolved into a consistent set of versions and installed into a fresh environment on every CI run, every container build, and every new hire's laptop. When that operation takes 60–90 seconds with pip, people avoid it; when it takes one second with uv, people stop caring about it, and a whole category of "my environment drifted" bugs disappears.

The tools uv absorbs.

• pip → uv pip install and uv add. uv ships a pip-compatible interface (uv pip install, uv pip freeze, uv pip compile) for drop-in use, plus a higher-level project interface (uv add, uv remove) that edits your pyproject.toml and updates the lockfile in one step.
• venv / virtualenv → uv venv and the automatic project .venv. uv creates virtual environments an order of magnitude faster than python -m venv, and in project mode it manages a .venv for you so you rarely create one by hand.
• pip-tools → uv pip compile and uv lock. The pip-compile workflow (loose requirements.in → pinned requirements.txt) becomes uv pip compile; the project workflow produces a richer uv.lock that pins the entire resolved graph with hashes across every platform.
• pyenv → uv python install and uv python pin. uv downloads standalone Python builds itself and pins a version per project via .python-version, so you no longer need a separate interpreter-version manager.
• pipx → uv tool install and uvx. Global CLI tools (like ruff or httpie) install into isolated environments with uv tool install, and uvx ruff runs a tool ephemerally without installing it at all.

Why speed changes behavior, not just wall-clock time.

• Disposable environments. When a fresh resolve-and-install is ~1s, you delete and recreate .venv on a whim instead of hand-patching a stale one — eliminating "works on my machine" drift.
• CI without a coffee break. A pipeline that spent 90s on pip install now spends a few seconds, so you can afford to run more matrix combinations (multiple Python versions, multiple dependency sets) within the same time budget.
• Tight feedback loops. uv run re-syncs the environment before running your script, so adding a dependency and running the pipeline is a single fast round-trip rather than "did I remember to reinstall?"
• The mechanism. uv keeps a global content-addressed cache and hardlinks (or copies/reflinks) packages into each .venv, so the same wheel is never downloaded or unpacked twice across all your projects on a machine.

The 2026 reality — uv is the greenfield default; the legacy tools still ship.

• uv is the default choice for new data projects and internal platforms: fast, one tool, PEP 621 pyproject.toml-native, and increasingly the tool that internal cookiecutters and templates generate.
• pip + venv still ships everywhere as the lowest common denominator — every base image has it, every tutorial assumes it, and plenty of legacy pipelines were built on requirements.txt. uv interoperates with all of it via uv pip and uv export.
• poetry remains common in application teams that adopted it for the pyproject workflow before uv existed; uv covers the same ground with faster resolution and native PEP 621, and migration is mechanical.
• conda / mamba still dominates parts of the scientific and GPU stack where non-PyPI binary dependencies (CUDA toolkits, MKL builds, some geospatial libraries) matter; uv handles the PyPI-installable majority and you keep conda only for the genuinely non-PyPI pieces.

What interviewers listen for.

• Do you name all the tools uv folds in (pip, venv, pip-tools, pyenv, pipx) rather than calling it "a faster pip"? — senior signal.
• Do you explain the global cache + hardlink model as the reason installs are fast, not just "it's written in Rust"? — senior signal.
• Do you distinguish uv pip install (low-level, imperative) from uv add (project-level, edits pyproject + lock)? — required answer.
• Do you say "commit the uv.lock" without being asked? — required answer.
• Do you describe uv environments as disposable and reproducible rather than "the venv I've had for months"? — senior signal.

Worked example — the four-tool-to-one-tool mapping

Detailed explanation. The single most useful artifact for reasoning about uv is a translation table from the commands your team already types to their uv equivalents. Every migration conversation converges on this table; having it in your head lets you answer "but how do I do X" instantly. Walk through building it for a typical data project that currently uses pip, venv, pip-tools, and pyenv.

• The old stack. pyenv install 3.12 for the interpreter, python -m venv .venv for isolation, pip install -r requirements.txt for packages, pip-compile requirements.in for pinning.
• The new stack. One tool, one pyproject.toml, one uv.lock.
• The goal. Map each legacy command to its uv replacement so no capability is lost.

Question. Produce the command-translation table and identify which legacy files uv replaces.

Input.

Legacy tool	Legacy command	Legacy artifact
pyenv	pyenv install 3.12	.python-version
venv	python -m venv .venv	.venv/
pip	pip install -r requirements.txt	requirements.txt
pip-tools	pip-compile requirements.in	requirements.txt (pinned)

Code.

# ── Legacy stack (four tools) ─────────────────────────────
pyenv install 3.12.4
pyenv local 3.12.4
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip-compile requirements.in -o requirements.txt

# ── uv stack (one tool) ───────────────────────────────────
uv python install 3.12          # pyenv install
uv python pin 3.12              # pyenv local  -> writes .python-version
uv venv                         # python -m venv  (usually implicit)
uv add pandas pyarrow           # pip install + edits pyproject + updates lock
uv lock                         # pip-compile     -> writes uv.lock
uv sync                         # install the locked set into .venv
uv run python etl.py            # run inside the env, auto-syncing first

Step-by-step explanation.

1. uv python install 3.12 downloads a standalone CPython build managed by uv itself — no system package manager, no compiling from source, no pyenv. The interpreter lives in uv's data directory and is shared across projects.
2. uv python pin 3.12 writes .python-version, the same file pyenv uses, so any project entered with uv (or pyenv) picks the pinned interpreter. This is the interpreter-version-management axis, folded into the same tool.
3. uv venv creates .venv far faster than python -m venv because it skips bootstrapping pip inside the environment. In project mode you rarely call it directly — uv add and uv run create and maintain .venv automatically.
4. uv add pandas pyarrow is the high-level replacement for pip install: it appends the packages to [project.dependencies] in pyproject.toml, re-resolves, updates uv.lock, and installs into .venv — four legacy steps in one command.
5. uv lock (the pip-tools replacement) resolves the full graph and writes uv.lock; uv sync makes .venv exactly match the lock. uv run wraps both: it syncs then runs, so you never forget to reinstall after a change.

Output.

Legacy artifact	uv artifact	Notes
.python-version (pyenv)	.python-version (uv)	same file; uv reads and writes it
.venv/ (venv)	.venv/ (uv)	uv-managed; recreated cheaply
requirements.in (pip-tools)	pyproject.toml	loose ranges live here
requirements.txt (pinned)	uv.lock	fully resolved, hashed, cross-platform

Rule of thumb. Learn the four mappings — pyenv→uv python, venv→uv venv, pip→uv add/uv pip, pip-compile→uv lock — and you can translate any existing workflow. No capability is lost; the artifacts consolidate into pyproject.toml + uv.lock.

Worked example — measuring the cold-install speedup

Detailed explanation. The speed claim is easy to test and worth internalising with real numbers, because "10–100x faster" is abstract until you watch a 90-second install finish in under two seconds. The right way to measure is a cold install (empty cache) versus a warm install (populated cache), because the warm case — the common case in CI with caching and on a developer laptop — is where uv's cache and hardlink model shines most. Walk through benchmarking a representative data-stack requirement set.

• The dependency set. A typical ETL project: pandas, pyarrow, sqlalchemy, boto3, requests, pydantic.
• Cold. Fresh machine, empty cache, first-ever install — dominated by download + build.
• Warm. Cache already populated — dominated by resolution + linking, which is where uv is dramatically faster.

Question. Measure cold and warm install time for pip versus uv on the same requirement set and explain the warm-case gap.

Input.

Scenario	Cache state	What dominates
pip cold	empty	download + wheel build
pip warm	populated	copy from cache + resolve
uv cold	empty	download + build (parallel)
uv warm	populated	resolve + hardlink

Code.

# Reproducible benchmark — same requirements, same machine
cat > requirements.txt <<'EOF'
pandas
pyarrow
sqlalchemy
boto3
requests
pydantic
EOF

# pip cold (clear caches first)
pip cache purge
time pip install -r requirements.txt        # ~60-90 s cold

# uv cold
uv cache clean
time uv pip install -r requirements.txt     # ~5-10 s cold (parallel downloads)

# Warm case — recreate the env from a populated cache
deactivate; rm -rf .venv
time pip install -r requirements.txt        # ~15-25 s warm (copies from cache)

uv venv
time uv pip install -r requirements.txt     # ~0.5-2 s warm (hardlinks from cache)

Step-by-step explanation.

1. The cold case for both tools is bounded by the network: wheels have to be downloaded from PyPI and, for any source-only packages, built. uv still wins cold because it downloads in parallel and resolves faster, but the gap is smaller because the network floor is shared.
2. uv cache clean and pip cache purge reset each tool's cache so the cold measurement is honest. Skipping this silently measures a warm install and understates the cold cost.
3. The warm case is the decisive one. pip, even with a populated cache, copies each package's files into the new environment — O(files) I/O per environment. uv hardlinks from its global cache by default, which is near-instant because no bytes are copied, only directory entries created.
4. uv's resolver is also faster: it uses a purpose-built dependency resolver in Rust rather than pip's backtracking resolver, so the "figure out a compatible set of versions" step drops from seconds to milliseconds on typical graphs.
5. The behavioral payoff is in the warm number: sub-two-second environment recreation means developers rebuild .venv freely and CI can skip elaborate caching gymnastics, because even a partial cache miss is cheap.

Output.

Tool	Cold install	Warm recreate	Dominant cost
pip	~60–90 s	~15–25 s	copy files per env
uv	~5–10 s	~0.5–2 s	hardlink from cache
Speedup (warm)	—	~10–40×	resolve + link vs copy
Behavior unlocked	—	disposable envs	rebuild without thinking

Rule of thumb. Benchmark the warm recreate, not just the cold install — that is the operation your team performs dozens of times a day, and uv's hardlink-from-cache model is what makes it near-instant. The cold number sells the demo; the warm number changes the workflow.

Worked example — the uv command surface for daily work

Detailed explanation. New uv users get lost because the tool has two layers — a low-level pip-compatible layer (uv pip …) and a high-level project layer (uv add, uv sync, uv run) — and mixing them incorrectly causes confusion ("I ran uv pip install but uv sync removed it"). The fix is a clear mental model: use the project layer for anything tracked in pyproject.toml, and reserve uv pip for ad-hoc, throwaway, or scripted installs. Walk through the daily-driver commands.

• Project layer. uv add, uv remove, uv lock, uv sync, uv run — these read and write pyproject.toml + uv.lock.
• pip layer. uv pip install, uv pip compile, uv pip sync — imperative, do not touch pyproject.toml.
• Tools layer. uv tool install, uvx — global CLIs in isolated environments.

Question. Classify the common uv commands and state which files each touches.

Input.

Command	Layer	Touches pyproject/lock?
uv add <pkg>	project	yes
uv sync	project	reads lock
uv run <cmd>	project	syncs then runs
uv pip install <pkg>	pip	no
uvx <tool>	tools	no

Code.

# ── Project layer (tracked; the default for real work) ────
uv init my-pipeline              # scaffold pyproject.toml + src layout
uv add "pandas>=2.2" pyarrow     # add deps, update lock, install
uv add --group dev pytest ruff   # dev-only dependency group
uv remove requests               # drop a dep, update lock
uv sync                          # make .venv match uv.lock exactly
uv run pytest                    # run a command inside the synced env
uv tree                          # show the resolved dependency tree

# ── pip layer (imperative; ad-hoc / scripting) ───────────
uv pip install jupyterlab        # NOT tracked in pyproject
uv pip compile requirements.in -o requirements.txt   # pip-tools style
uv pip sync requirements.txt     # make env match a requirements file

# ── tools layer (global CLIs, isolated) ──────────────────
uv tool install ruff             # persistent global tool
uvx ruff check .                 # run ruff ephemerally, no install

Step-by-step explanation.

1. uv init scaffolds a project: it writes a pyproject.toml with a [project] table and (by default) a src/ layout, and sets a requires-python. This is the entry point for the project layer.
2. uv add is the workhorse. It resolves the new dependency against the existing graph, updates pyproject.toml and uv.lock, and installs into .venv — all atomically, so the three files never drift out of sync.
3. uv sync is the reconciler: it makes .venv exactly match uv.lock, adding missing packages and removing ones that are not in the lock. This is why an untracked uv pip install can be "undone" by a later uv sync — the pip layer is invisible to the project layer.
4. uv run is the safest way to execute anything: it performs an implicit uv sync first, then runs the command inside the environment without requiring manual activation. In CI and scripts, uv run removes an entire class of "forgot to activate / forgot to reinstall" errors.
5. uv tool install and uvx handle global CLIs the way pipx does — each tool gets its own isolated environment so a tool's dependencies never collide with your project's. uvx is the zero-install runner for one-off invocations.

Output.

Task	Right command	Wrong command
Add a project dependency	uv add pandas	uv pip install pandas (untracked)
Run tests reproducibly	uv run pytest	pytest (may use wrong env)
Try a package quickly	uv pip install <x>	uv add <x> then uv remove <x>
Run a linter once	uvx ruff check .	uv add ruff (pollutes deps)

Rule of thumb. Default to the project layer (uv add / uv sync / uv run) for anything your project depends on, use uv pip only for throwaway or scripted installs, and use uvx for one-off tools. Keeping the layers separate is what keeps pyproject.toml honest.

Python interview question on tool consolidation

A senior interviewer might open with: "Our data platform team has scripts that use pip and requirements.txt, some services on poetry, and a couple of ML repos on conda, plus everyone manages Python versions with pyenv. We want one tool. Walk me through what uv replaces, how you'd standardise the daily workflow, and how you'd keep the pip-based consumers working during the transition."

Solution Using a single uv-managed environment with a pip-compatible export

# 1. Standardise the interpreter — uv installs and pins Python itself
uv python install 3.12
uv python pin 3.12                 # writes .python-version (replaces pyenv local)

# 2. Initialise the project layer — pyproject.toml becomes the source of truth
uv init --package data-platform
cd data-platform

# 3. Declare dependencies at the project level (edits pyproject + lock + venv)
uv add "pandas>=2.2" pyarrow sqlalchemy boto3
uv add --group dev pytest ruff mypy

# 4. Daily workflow for every engineer — no manual activate, always reproducible
uv sync                            # .venv == uv.lock
uv run pytest                      # run tests in the synced env
uv run python -m data_platform.etl # run the pipeline

# 5. Keep pip-based consumers working — export a pinned requirements.txt
uv export --no-dev --format requirements-txt > requirements.txt
# Legacy job that only knows pip can still:  pip install -r requirements.txt

# pyproject.toml produced by the steps above (the single source of truth)
[project]
name = "data-platform"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "pandas>=2.2",
    "pyarrow",
    "sqlalchemy",
    "boto3",
]

[dependency-groups]
dev = ["pytest", "ruff", "mypy"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Step-by-step trace.

Step	Command	Effect
Interpreter	uv python install/pin 3.12	standalone Python; .python-version written
Init	uv init --package	pyproject.toml + src/ layout
Runtime deps	uv add pandas …	[project.dependencies] + uv.lock updated
Dev deps	uv add --group dev …	[dependency-groups].dev (not shipped)
Reconcile	uv sync	.venv matches uv.lock byte-for-byte
Legacy bridge	uv export … > requirements.txt	pip consumers keep working

After standardising, every engineer runs the same three commands (uv sync, uv run …), pyenv and manual venv creation disappear, and the poetry/conda repos migrate one at a time (Section 5) while the exported requirements.txt keeps any pip-only downstream job running unchanged during the transition.

Output:

Concern	Before (four tools)	After (uv)
Python versions	pyenv per machine	uv python + .python-version
Isolation	python -m venv	uv-managed .venv
Add a dependency	pip install + hand-edit reqs	uv add (atomic)
Pinning	pip-compile	uv lock (uv.lock)
pip consumers	native	uv export bridge

Why this works — concept by concept:

• One binary, five jobs — uv folds install, isolate, resolve/lock, and Python-version management into a single tool, so the team learns one command surface instead of gluing four tools together with shell scripts.
• pyproject.toml as source of truth — declaring dependencies in the PEP 621 [project] table (plus [dependency-groups] for dev-only) means there is exactly one place a dependency is written, and uv add keeps pyproject.toml, uv.lock, and .venv in lockstep.
• uv.lock for reproducibility — the lockfile pins the entire resolved graph with hashes, so uv sync on any machine produces the identical environment; committing it is what makes "reproducible" true rather than aspirational.
• uv export as the compatibility bridge — uv export --format requirements-txt emits a pinned requirements.txt that any pip-only consumer can install, so the migration is incremental and no downstream job breaks on day one.
• Cost — one tool to install in every image and CI runner, a one-time pyproject.toml authoring per repo, and near-zero ongoing cost thanks to the global cache. Compared to maintaining pyenv + venv + pip-tools glue, the operational surface shrinks; environment-drift incidents drop to near zero. Net O(1) per environment build versus O(files) copies with pip.

Python
Topic — pandas
Pandas problems on Python packaging and environments

Practice →

Python Topic — defensive-coding Defensive-coding problems on dependency hygiene

Practice →

---

2. Projects, pyproject.toml, and the uv.lock lockfile

uv init → uv add → uv lock → uv sync → uv run — the project loop that keeps pyproject.toml and uv.lock in lockstep

The mental model in one line: a uv project is a pyproject.toml (the loose, human-authored declaration of what you depend on) plus a uv.lock (the machine-generated, fully-resolved, cross-platform pin of every transitive package with hashes), and the project loop — uv add to declare, uv lock to resolve, uv sync to install, uv run to execute — keeps all three of pyproject, lock, and .venv consistent so you never hand-edit a pinned requirements file again. The pyproject file is where a human writes "I want pandas 2.2 or newer"; the lockfile is where uv records "given everything, the exact set is pandas 2.2.2, numpy 2.0.1, pytz 2024.1, …" — and the discipline that makes data pipelines reproducible is committing the lockfile and syncing against it everywhere.

The two files and their division of labor.

• pyproject.toml — the declaration. Human-authored. Holds [project] metadata (name, version, requires-python), [project.dependencies] as loose ranges (pandas>=2.2), [dependency-groups] for dev/test-only packages, optional extras under [project.optional-dependencies], and uv-specific settings under [tool.uv].
• uv.lock — the resolution. Machine-generated, never hand-edited. Records the exact version and hash of every direct and transitive dependency, resolved for all target platforms at once, so a Linux CI runner and a macOS laptop compute the same environment from the same lock.
• .venv — the materialisation. The actual installed environment, made to match uv.lock by uv sync. Disposable; regenerated cheaply.

The project loop commands.

• uv init. Scaffolds pyproject.toml, a src/ package (with --package) or a flat layout, a .python-version, and a README. The starting point.
• uv add / uv remove. Edit the dependency list, re-resolve, update uv.lock, and sync .venv. The everyday commands.
• uv lock. Re-resolve from pyproject.toml and (re)write uv.lock without touching .venv. Run it when you change ranges by hand or want to refresh pins.
• uv sync. Reconcile .venv with uv.lock — install what's missing, remove what's extra. The reproducibility command.
• uv run. Sync, then execute a command in the environment. The command you actually type all day.

Dependency groups vs optional extras — a distinction people get wrong.

• [dependency-groups] (dev/test/docs). For dependencies your developers need but your users don't — pytest, ruff, mypy, jupyterlab. They are locked and installed by default locally, excluded with uv sync --no-dev (or specific --no-group), and never included when your package is installed by someone else.
• [project.optional-dependencies] (extras). For optional features your users opt into — mylib[postgres] pulling psycopg, mylib[spark] pulling pyspark. These are part of your published package's metadata and are installed by downstream consumers who ask for the extra.
• The rule. "Would a user who pip installs my package ever want this?" — yes means an extra, no means a dependency group.

tool.uv.sources — pinning to git, path, or a private index.

• Git dependency. [tool.uv.sources] can point a dependency at a git URL and ref ({ git = "https://…", rev = "abc123" }) — useful for an unreleased fix.
• Local path. A sibling package in a monorepo: { path = "../shared", editable = true }.
• Alternate index. Pin a package to a private index (an internal PyPI mirror) without redirecting the whole resolution.
• The point. Sources let the declaration stay clean (mylib in [project.dependencies]) while the resolution pulls from a non-default location, and the lock records exactly what was used.

Common interview probes on the project loop.

• "What's the difference between pyproject.toml and uv.lock?" — required answer: loose human declaration vs exact machine resolution.
• "Do you commit uv.lock?" — required answer: yes, always, for applications and pipelines.
• "Dependency group vs optional extra?" — dev-only vs user-facing optional feature.
• "How do you add an unreleased fix from git?" — [tool.uv.sources] with a git rev.

Worked example — from uv init to the first uv.lock

Detailed explanation. The canonical starting point: scaffold a project, add a couple of dependencies, and inspect the resulting pyproject.toml and uv.lock. Seeing the lockfile appear and the .venv populate in one second is the moment uv "clicks". Walk through initialising a small ETL project.

• Scaffold. uv init --package etl-jobs for a src/-layout installable package.
• Add. uv add pandas pyarrow for runtime deps; uv add --group dev pytest for tests.
• Inspect. Look at pyproject.toml (loose) and uv.lock (pinned).

Question. Initialise an installable ETL project, add runtime and dev dependencies, and show the three artifacts that result.

Input.

Step	Command	Produces
init	uv init --package etl-jobs	pyproject.toml, src/etl_jobs/
runtime deps	uv add pandas pyarrow	[project.dependencies], uv.lock
dev deps	uv add --group dev pytest	[dependency-groups].dev
sync	(implicit)	.venv

Code.

uv init --package etl-jobs
cd etl-jobs
uv add pandas pyarrow
uv add --group dev pytest
uv tree            # inspect the resolved graph
uv run pytest -q   # run tests in the synced env

# pyproject.toml  (human-authored, loose ranges)
[project]
name = "etl-jobs"
version = "0.1.0"
description = "Batch ETL jobs"
requires-python = ">=3.12"
dependencies = [
    "pandas>=2.2.0",
    "pyarrow>=16.0.0",
]

[dependency-groups]
dev = ["pytest>=8.0"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# uv.lock  (machine-generated, exact pins — excerpt)
version = 1
requires-python = ">=3.12"

[[package]]
name = "pandas"
version = "2.2.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
    { name = "numpy" },
    { name = "python-dateutil" },
    { name = "pytz" },
    { name = "tzdata" },
]
[[package.wheels]]
url = "https://files.pythonhosted.org/.../pandas-2.2.2-cp312-...whl"
hash = "sha256:9e79019aba43cb4fda9e4d983f8e88ca0373adbb697ae9c6c43093218de28b54"

Step-by-step explanation.

1. uv init --package etl-jobs scaffolds an installable package: a pyproject.toml with a build backend (hatchling), a src/etl_jobs/__init__.py, and a .python-version. Without --package you get an "application" layout (no build backend, not installable) — pick --package when you intend to import or ship the code.
2. uv add pandas pyarrow appends loose ranges to [project.dependencies] (uv chooses a sensible lower bound like >=2.2.0), then resolves the full graph — including transitive deps like numpy, pytz, tzdata — and writes them all into uv.lock.
3. uv add --group dev pytest puts pytest into [dependency-groups].dev. It is installed locally (so your tests run) but is not part of the package a consumer installs, and it is excluded by uv sync --no-dev.
4. uv.lock records each package's exact version, source registry, its own dependencies, and per-wheel hashes. The requires-python and the multi-platform wheel list mean the same lock resolves correctly on Linux, macOS, and Windows.
5. uv run pytest triggers an implicit sync (making .venv match the lock) and then runs pytest inside it — so the tests always run against exactly the locked dependency set, never a drifted environment.

Output.

Artifact	Authored by	Committed?	Content
pyproject.toml	human	yes	loose ranges + metadata
uv.lock	uv	yes	exact pins + hashes
.venv/	uv	no (gitignore)	installed environment
.python-version	uv	yes	pinned interpreter

Rule of thumb. Use uv init --package for anything you will import or ship, keep [project.dependencies] loose (ranges, not pins), let uv.lock hold the exact pins, and commit everything except .venv. Loose declaration + exact lock is the whole design.

Worked example — dependency groups and optional extras side by side

Detailed explanation. A library that connects to multiple backends is the clearest case for distinguishing extras from groups: the Postgres and Spark connectors are optional features users opt into (extras), while pytest and ruff are developer tools nobody downstream wants (groups). Getting this wrong either ships test dependencies to users or hides optional features from them. Walk through a library with both.

• Extras. mylib[postgres] → psycopg; mylib[spark] → pyspark. User-facing.
• Groups. dev → pytest, ruff, mypy. Developer-only.
• The test. Would a consumer of the package ever want it? Yes → extra; no → group.

Question. Author a pyproject.toml that declares two optional extras and one dev group, and show the install commands for each audience.

Input.

Dependency	Category	Who needs it
psycopg	extra postgres	users writing to Postgres
pyspark	extra spark	users on Spark
pytest, ruff, mypy	group dev	maintainers only

Code.

[project]
name = "mylib"
version = "0.3.0"
requires-python = ">=3.12"
dependencies = ["pandas>=2.2", "sqlalchemy>=2.0"]

# User-facing optional features (installed by consumers on demand)
[project.optional-dependencies]
postgres = ["psycopg[binary]>=3.1"]
spark    = ["pyspark>=3.5"]

# Developer-only tools (never shipped to consumers)
[dependency-groups]
dev = ["pytest>=8.0", "ruff>=0.6", "mypy>=1.10"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# Maintainer, working on the library locally:
uv sync                       # installs deps + dev group by default
uv sync --no-dev              # deps only, no dev tools (mimics a user)

# Maintainer testing the postgres feature:
uv sync --extra postgres      # deps + psycopg
uv sync --all-extras          # deps + every extra

# A downstream user (via pip) opting into an extra:
pip install "mylib[postgres]"     # pandas + sqlalchemy + psycopg
pip install "mylib[postgres,spark]"

Step-by-step explanation.

1. [project.optional-dependencies] declares extras — named bundles a consumer requests with bracket syntax (mylib[postgres]). They are part of the package's published metadata, so pip install "mylib[postgres]" works for anyone, not just uv users.
2. [dependency-groups] declares groups — dev here. Groups are a development-time concept (PEP 735); they are locked in uv.lock and installed by uv sync locally but are invisible to a consumer who installs your package.
3. uv sync installs the default group set (including dev) so a maintainer's environment is complete. uv sync --no-dev reproduces what a user gets — a good way to catch "it works for me because I have a dev tool installed" bugs.
4. uv sync --extra postgres adds the postgres extra to the local environment; --all-extras adds them all — useful in CI to test every optional feature path.
5. The consumer side is pure standard packaging: pip install "mylib[postgres,spark]" pulls the base deps plus both extras. The dev group never appears — that separation is the entire point.

Output.

Command	pandas/sqlalchemy	psycopg	pyspark	pytest/ruff
uv sync	yes	no	no	yes (dev)
uv sync --no-dev	yes	no	no	no
uv sync --extra postgres	yes	yes	no	yes (dev)
pip install mylib[spark]	yes	no	yes	no

Rule of thumb. Put user-facing optional features in [project.optional-dependencies] (extras) and developer-only tooling in [dependency-groups] (groups). Test the user experience with uv sync --no-dev; test optional features with uv sync --all-extras.

Worked example — the universal, cross-platform lockfile

Detailed explanation. The property that makes uv.lock more valuable than a requirements.txt is that it is universal: a single lock resolves the dependency graph for every target platform and Python version at once, so a macOS developer and a Linux CI runner and a Windows analyst all compute identical environments from the same file. A plain pip freeze captures only the platform it ran on, which is why "it locked fine on my Mac but broke in the Linux container" is a classic pip failure. Walk through a package with platform-conditional dependencies.

• The wrinkle. Some deps are platform-specific: pywin32 only on Windows, a different wheel per OS/arch, tzdata only where the OS lacks a zoneinfo database.
• pip freeze. Captures only the current platform — non-portable.
• uv.lock. Encodes markers (sys_platform == 'win32') so one lock serves all platforms.

Question. Lock a project with a Windows-only dependency and show how the single uv.lock stays correct on Linux and Windows.

Input.

Dependency	Applies on	Marker
pandas	all	(none)
pywin32	Windows only	sys_platform == 'win32'
tzdata	non-Windows sometimes	resolver-managed

Code.

[project]
name = "cross-platform-job"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "pandas>=2.2",
    "pywin32>=306 ; sys_platform == 'win32'",   # only installs on Windows
]

# Resolve for ALL platforms at once (default behaviour)
uv lock

# On Linux CI:
uv sync --locked      # pandas + transitive; pywin32 skipped (marker false)

# On a Windows analyst laptop, same lock:
uv sync --locked      # pandas + transitive + pywin32 (marker true)

# uv.lock excerpt — the marker is recorded, not the platform result
[[package]]
name = "pywin32"
version = "306"
source = { registry = "https://pypi.org/simple" }
# resolution notes the marker; sync evaluates it per platform
marker = "sys_platform == 'win32'"

Step-by-step explanation.

1. The dependency pywin32 ; sys_platform == 'win32' uses a PEP 508 environment marker so it is declared for everyone but only installed where the marker evaluates true. uv records the marker in the lock rather than baking in one platform's answer.
2. uv lock performs a universal resolution: it finds a set of versions that satisfies the constraints across all supported platforms and Python versions simultaneously, and writes every candidate wheel + hash into uv.lock.
3. On Linux, uv sync --locked evaluates sys_platform == 'win32' as false, so pywin32 is skipped — but pandas and its transitive deps install from the exact pins in the lock.
4. On Windows, the same uv.lock yields pandas plus pywin32, because the marker now evaluates true. No second lockfile, no per-OS requirements-linux.txt / requirements-win.txt split.
5. --locked asserts the lock is already up to date with pyproject.toml and fails if not — the correct flag for CI and shared machines, because it guarantees everyone is on the committed resolution rather than silently re-resolving.

Output.

Platform	Installs pandas?	Installs pywin32?	Lock used
Linux CI	yes (pinned)	no (marker false)	same uv.lock
macOS dev	yes (pinned)	no (marker false)	same uv.lock
Windows	yes (pinned)	yes (marker true)	same uv.lock
Net effect	identical core env	platform-correct extras	one file

Rule of thumb. Prefer uv.lock over a per-platform pip freeze because one universal lock encodes environment markers and resolves for every platform at once. Commit it, and use uv sync --locked so every machine reproduces the committed resolution instead of drifting.

Data engineering interview question on the project loop

A senior interviewer might ask: "A data team commits a requirements.txt produced by pip freeze on whoever's laptop ran it last, and environments drift constantly between the Mac laptops and the Linux Airflow workers. Redesign this around uv's project model. Show the pyproject.toml, explain what uv.lock gives you that pip freeze doesn't, and describe the exact commands developers and CI run."

Solution Using pyproject.toml + a committed universal uv.lock

# pyproject.toml — the single, human-authored declaration
[project]
name = "orders-pipeline"
version = "1.2.0"
requires-python = ">=3.12,<3.13"
dependencies = [
    "pandas>=2.2",
    "pyarrow>=16",
    "sqlalchemy>=2.0",
    "apache-airflow>=2.9 ; extra == 'orchestration'",
]

[dependency-groups]
dev = ["pytest>=8", "ruff>=0.6", "mypy>=1.10"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# 1. Author the lock once, commit it
uv lock                       # universal resolution -> uv.lock
git add pyproject.toml uv.lock .python-version
git commit -m "Adopt uv project model"

# 2. Every developer, every machine
uv sync                       # .venv == committed uv.lock
uv run python -m orders_pipeline.run

# 3. CI and Airflow workers — assert, do not re-resolve
uv sync --locked --no-dev     # fail if lock is stale; skip dev tools

# 4. Adding a dependency is a reviewed change
uv add "duckdb>=1.0"          # updates pyproject + uv.lock in one commit

Step-by-step trace.

Step	Command	Guarantee
Author	uv lock	universal resolution across Mac + Linux
Commit	git add uv.lock	the resolution is version-controlled
Developer	uv sync	.venv matches the committed lock exactly
CI / Airflow	uv sync --locked --no-dev	build fails if lock drifted; prod deps only
Change	uv add duckdb	dependency change is a reviewable diff

After the redesign, the pip freeze artifact is gone; uv.lock holds a universal resolution that is identical on Mac laptops and Linux workers; CI uses --locked so a stale lock fails the build instead of silently re-resolving; and adding a dependency is a single command that produces a reviewable pyproject.toml + uv.lock diff instead of a mystery requirements.txt churn.

Output:

Property	pip freeze requirements.txt	committed uv.lock
Platform coverage	one (whoever ran it)	universal (all platforms)
Hashes	optional, rarely present	always
Direct vs transitive	indistinguishable	structured, attributed
Drift detection	none	uv sync --locked fails on drift
Update workflow	manual freeze	uv add / uv lock

Why this works — concept by concept:

• Loose pyproject, exact lock — humans declare intent as ranges in pyproject.toml; uv records the exact resolved graph in uv.lock. Separating intent from resolution is what lets you update deliberately instead of by accident.
• Universal resolution — uv lock solves the dependency graph for every target platform and Python version at once and encodes environment markers, so one committed file reproduces correctly on Mac laptops and Linux Airflow workers alike.
• --locked in CI — asserting the lock is current turns "someone forgot to re-lock" into a fast build failure instead of a silent, non-reproducible re-resolution on the runner.
• --no-dev for production — Airflow workers and images install only [project.dependencies], not the dev group, keeping the runtime lean and the attack surface small.
• Cost — one uv lock per dependency change (seconds), a committed uv.lock in the repo, and uv sync everywhere else. Compared to chasing pip freeze drift across heterogeneous machines, the reproducibility is free and the drift-incident count drops to zero. O(1) reconcile per environment.

Python
Topic — etl
ETL problems on pipeline dependency management

Practice →

Python Topic — pandas Pandas problems on pandas and pyarrow projects

Practice →

---

3. Reproducible environments and Python version management

uv sync --locked + uv python pin — byte-identical environments across machines and across Python versions

The mental model in one line: reproducibility with uv is two locks working together — uv.lock pins the packages so uv sync --locked builds a byte-identical dependency set on every machine, and .python-version (written by uv python pin) plus uv's managed interpreter downloads pin the Python version so the interpreter itself is identical too — and because uv installs Python for you, "reproducible environment" finally includes the interpreter, not just the libraries. A pipeline that pins pandas==2.2.2 but runs on Python 3.11 in CI and 3.12 on a laptop is not reproducible; uv closes that gap by managing both halves in one tool.

The two halves of reproducibility.

• Package reproducibility (uv.lock + uv sync --locked). The lock pins exact versions and hashes; --locked asserts the lock matches pyproject.toml; the hashes mean a tampered or re-uploaded wheel is rejected. Same input, same environment, everywhere.
• Interpreter reproducibility (uv python + .python-version). uv downloads standalone CPython (and PyPy) builds and records the pinned version in .python-version. No more "CI has 3.11, prod has 3.12". The interpreter is a pinned, managed artifact like any dependency.
• Together. requires-python in pyproject.toml bounds the acceptable range; .python-version pins the exact patch used; uv.lock pins the packages. All three committed = fully reproducible.

The sync flags and what they promise.

• uv sync (default). Reconcile .venv with uv.lock, re-locking first if pyproject.toml changed. Convenient for local dev.
• uv sync --frozen. Use the existing uv.lock as-is; do not re-lock even if pyproject.toml changed. Fast; assumes the lock is already correct. Good for containers where you copy a known-good lock.
• uv sync --locked. Assert the lock is already up to date with pyproject.toml; fail if not. The CI/shared-machine flag — it refuses to build on a stale lock instead of silently re-resolving.
• The distinction. --frozen trusts the lock and skips checking; --locked verifies the lock and errors on mismatch. Use --locked where correctness matters, --frozen where you have already verified upstream.

uv's Python version management.

• uv python install 3.12. Downloads a standalone build; multiple versions coexist.
• uv python pin 3.12. Writes .python-version; the project uses that version. Pin to a patch (3.12.4) for maximum reproducibility.
• uv python list. Shows installed and available builds.
• Automatic provisioning. If a project requires a Python uv hasn't installed, uv sync/uv run can fetch it automatically — so a fresh clone bootstraps the exact interpreter with no manual step.

Workspaces for monorepos.

• [tool.uv.workspace]. Declares a set of member packages (members = ["packages/*"]) that share a single uv.lock at the workspace root.
• Why it matters. A monorepo of pipelines that share internal libraries resolves once, consistently, so two pipelines can never end up on incompatible versions of a shared dependency.
• Member deps. Members depend on each other via tool.uv.sources path entries; the root lock ties everything together.

Common interview probes on reproducibility.

• "How do you guarantee CI and prod use the same Python version?" — .python-version + uv-managed interpreter + requires-python.
• "--frozen vs --locked?" — trust-and-skip vs verify-and-fail.
• "How do hashes in uv.lock help security?" — reject tampered/re-uploaded wheels.
• "How do you share a dependency across a monorepo?" — a uv workspace with one root lock.

Worked example — installing and pinning the interpreter

Detailed explanation. The interpreter is the half of reproducibility that pip and poetry historically ignored — they assume "whatever python is on PATH". uv makes the interpreter a managed, pinned artifact. Walk through installing a specific Python, pinning it, and proving a fresh clone gets the same interpreter.

• Install. uv python install 3.12.4 — a standalone build, not the system Python.
• Pin. uv python pin 3.12.4 — writes .python-version.
• Prove. A fresh clone + uv run python --version reports the pinned version.

Question. Pin a project to Python 3.12.4 and show what a colleague gets on a machine that has never had 3.12.4 installed.

Input.

Component	Value
Target interpreter	CPython 3.12.4 (uv-managed)
Pin file	.python-version
Range guard	requires-python = ">=3.12,<3.13"
Colleague's machine	system Python 3.10 only

Code.

# On your machine
uv python install 3.12.4
uv python pin 3.12.4          # writes .python-version = "3.12.4"
uv run python --version       # Python 3.12.4
git add .python-version pyproject.toml uv.lock

# On a colleague's machine (only system Python 3.10 present)
git clone <repo> && cd <repo>
uv run python --version
# uv sees .python-version = 3.12.4, auto-downloads that standalone build,
# then runs -> Python 3.12.4  (system 3.10 is never touched)

# pyproject.toml — the range guard complements the exact pin
[project]
name = "reproducible-job"
version = "0.1.0"
requires-python = ">=3.12,<3.13"   # any 3.12.x; blocks 3.11 and 3.13
dependencies = ["pandas>=2.2"]

Step-by-step explanation.

1. uv python install 3.12.4 fetches a standalone, relocatable CPython build into uv's data directory. It does not touch the system Python or require admin rights — critical on locked-down corporate machines where you can't apt install python3.12.
2. uv python pin 3.12.4 writes .python-version with the exact patch. Pinning the patch (not just 3.12) is what makes the interpreter fully reproducible — 3.12.4 and 3.12.7 can differ in subtle stdlib behavior.
3. requires-python = ">=3.12,<3.13" in pyproject.toml is the range guard: it declares which Python versions the project supports and makes uv's resolution reject anything outside it. The .python-version pin chooses the exact one within that range.
4. On the colleague's machine, uv run reads .python-version, notices 3.12.4 isn't installed, and automatically downloads it before running — so a fresh clone bootstraps the exact interpreter with zero manual steps and without disturbing the system 3.10.
5. Committing .python-version, pyproject.toml, and uv.lock together means the interpreter and the packages are both version-controlled; reproducing the environment is git clone + uv sync.

Output.

Machine	System Python	Project Python	How obtained
Yours	3.11	3.12.4	uv python install
Colleague	3.10	3.12.4	auto-downloaded on uv run
CI runner	3.9	3.12.4	auto-downloaded / setup-uv
Net	irrelevant	identical everywhere	.python-version pin

Rule of thumb. Pin the exact patch with uv python pin 3.12.4, guard the supported range with requires-python, and commit .python-version. uv turns the interpreter into a managed, auto-provisioned artifact, so "reproducible environment" finally includes Python itself.

Worked example — --frozen vs --locked vs default sync

Detailed explanation. The three sync modes trip people up because the names sound similar but the guarantees differ sharply, and choosing wrong either makes CI silently non-reproducible (default re-resolves) or makes a legitimate change fail (--locked when you meant to update). Walk through when each is correct.

• Default uv sync. Re-lock if pyproject.toml changed, then install. Convenient locally; can mutate uv.lock.
• --frozen. Never re-lock; install from the existing lock exactly. Fast, trusting.
• --locked. Verify the lock is current; error if not. Safe, strict.

Question. For each context — local dev, container build, CI verification — pick the right sync mode and justify it.

Input.

Context	Priority	Right mode
Local development	convenience	default uv sync
Container build	speed + known-good lock	--frozen
CI verification	catch stale locks	--locked

Code.

# Local dev — happy to re-lock when I edit pyproject
uv sync
uv add duckdb          # edits pyproject + re-locks + installs

# Container build — the lock was verified upstream; just install it fast
uv sync --frozen --no-dev

# CI verification — fail the build if someone forgot to commit an updated lock
uv sync --locked
# If pyproject changed but uv.lock did not:
#   error: The lockfile at uv.lock is out of date. Run `uv lock` ...

# CI job fragment showing the strict flag
steps:
  - run: uv sync --locked        # refuses to build on a stale lock
  - run: uv run pytest

Step-by-step explanation.

1. Default uv sync is developer-friendly: if you hand-edit pyproject.toml and run uv sync, it re-resolves and updates uv.lock automatically. The downside is that it can change the lock, so it is the wrong choice where you want a guarantee that nothing re-resolves.
2. --frozen says "use uv.lock exactly as it is; do not even check it against pyproject.toml." It is the fastest mode and correct inside a container build where you have already verified the lock (e.g. CI ran --locked earlier) and just want a deterministic install.
3. --locked says "verify uv.lock is up to date with pyproject.toml; if not, fail." This is the CI gate: it converts "someone changed a dependency but forgot to re-lock" from a silent, non-reproducible re-resolve into an immediate, loud build failure.
4. The failure mode of picking wrong: using default uv sync in CI can silently re-resolve on the runner (non-reproducible); using --locked in a step where you intended to update deps fails the build until you run uv lock. Matching mode to intent avoids both.
5. A common pipeline uses --locked in a verification stage and --frozen in the image build stage: verify once, strictly; then install fast, trusting the verified lock.

Output.

Mode	Re-locks?	Fails on stale lock?	Best for
uv sync (default)	yes, if pyproject changed	no	local dev
uv sync --frozen	no	no (ignores mismatch)	container build
uv sync --locked	no	yes	CI verification

Rule of thumb. Use default uv sync locally, --locked in CI to catch stale locks, and --frozen in container builds where the lock is already verified. The mnemonic: --locked verifies, --frozen trusts, default updates.

Worked example — reproducing a teammate's environment exactly

Detailed explanation. The acid test of reproducibility is handing a bug report to a teammate and having them reproduce your exact environment — same packages, same versions, same Python, same hashes — from nothing but the repo. With uv this is two commands, and the hashes guarantee even a maliciously re-uploaded wheel can't sneak in. Walk through the reproduction from a clean machine.

• Inputs committed. pyproject.toml, uv.lock, .python-version.
• The reproduction. git clone → uv sync --locked (auto-provisions Python) → run.
• The guarantee. Hashes in the lock reject any wheel whose content doesn't match.

Question. Show the exact steps a teammate runs to reproduce your environment, and what happens if a dependency's wheel was re-uploaded with different bytes.

Input.

Committed file	Pins
pyproject.toml	loose ranges + requires-python
uv.lock	exact versions + sha256 hashes
.python-version	exact interpreter patch

Code.

# Teammate, clean machine, reproducing your environment
git clone <repo> && cd <repo>
uv sync --locked        # auto-installs pinned Python, installs hashed wheels
uv run python -m job.repro_case

# What the hash guarantee looks like if a wheel was tampered/re-uploaded:
# uv verifies each downloaded wheel against uv.lock's sha256:
#   error: Failed to validate hash for pandas==2.2.2
#          expected sha256:9e79019a...  got sha256:deadbeef...
# -> install aborts; the environment is never built from bad bytes

# uv.lock records the hash that gates every install
[[package]]
name = "pandas"
version = "2.2.2"
[[package.wheels]]
url  = "https://files.pythonhosted.org/.../pandas-2.2.2-cp312-...whl"
hash = "sha256:9e79019aba43cb4fda9e4d983f8e88ca0373adbb697ae9c6c43093218de28b54"

Step-by-step explanation.

1. The teammate needs nothing pre-installed except uv itself: git clone brings the three committed files, and uv sync --locked does the rest. If the pinned Python isn't present, uv downloads it automatically before installing packages.
2. --locked guarantees the install uses the committed resolution, not a fresh one — so the teammate gets your exact versions, not "whatever resolves today". This is what makes bug reproduction deterministic.
3. Every wheel in uv.lock carries a sha256 hash. As uv downloads each wheel it computes the hash and compares; a mismatch (from a re-uploaded or corrupted artifact) aborts the install rather than building a subtly-different environment.
4. The hash check is a supply-chain defense: even if an attacker replaced a version on the index with malicious bytes under the same version number, the recorded hash wouldn't match and uv would refuse to install it.
5. The whole reproduction is two commands and a few seconds thanks to the global cache — so "can you reproduce this?" gets a yes in the time it takes to read the bug report, and the answer is trustworthy because it's byte-identical.

Output.

Reproduction step	Result
git clone	brings pyproject + uv.lock + .python-version
uv sync --locked (Python missing)	auto-downloads pinned interpreter
wheel hash matches	installed
wheel hash mismatch	install aborts with hash error
total time	seconds (warm cache)

Rule of thumb. Reproducibility is git clone + uv sync --locked. Commit pyproject.toml, uv.lock, and .python-version; the lock's hashes turn reproduction into a supply-chain-safe, byte-identical operation, not a hopeful "should be close".

Python interview question on reproducible environments

A senior interviewer might ask: "Our model-training jobs are flaky: the same code gives slightly different results between a data scientist's laptop and the training cluster. We suspect environment drift — different NumPy versions and even different Python patch versions. Design a uv-based setup that makes the training environment byte-identical across laptop, CI, and the cluster, including the interpreter, and show how you'd enforce it."

Solution Using a committed uv.lock, a pinned interpreter, and --locked enforcement

# pyproject.toml — bound the interpreter and declare deps
[project]
name = "model-training"
version = "0.4.0"
requires-python = "==3.12.4"          # exact interpreter requirement
dependencies = [
    "numpy==2.0.1",                   # exact for numerical determinism
    "scikit-learn==1.5.1",
    "pandas>=2.2,<2.3",
]

[dependency-groups]
dev = ["pytest>=8", "ruff>=0.6"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# 1. Pin interpreter + author the lock, commit all three
uv python install 3.12.4
uv python pin 3.12.4
uv lock
git add pyproject.toml uv.lock .python-version

# 2. Data scientist laptop
uv sync --locked && uv run python train.py

# 3. Cluster / batch job (only prod deps, pinned interpreter auto-provisioned)
uv sync --locked --no-dev && uv run python train.py

# 4. CI gate — refuse to merge if the lock is stale
uv sync --locked            # fails the build on any pyproject/lock mismatch

Step-by-step trace.

Layer	Mechanism	Guarantee
Interpreter	requires-python == 3.12.4 + .python-version	same Python patch everywhere
Numeric deps	numpy==2.0.1 pinned + uv.lock hash	identical NumPy bytes
Reconcile	uv sync --locked	.venv matches committed lock
Prod lean	--no-dev	cluster runs runtime deps only
Enforcement	--locked in CI	stale lock fails the build

After the change, the interpreter is pinned to 3.12.4 and auto-provisioned on every host, NumPy and scikit-learn are pinned to exact versions with hash-verified wheels, and uv sync --locked on laptop, CI, and cluster produces a byte-identical .venv. The "different results on different machines" class of flakiness — caused by a different NumPy build or Python patch — is eliminated at its root.

Output:

Symptom	Root cause	uv fix
Different results laptop vs cluster	NumPy 2.0.1 vs 2.0.3	numpy==2.0.1 + hashed lock
Occasional 3.11-only bug	interpreter drift	requires-python == 3.12.4 + pin
"worked before, broke today"	silent re-resolve	uv sync --locked
Cluster image bloat	dev tools shipped	--no-dev

Why this works — concept by concept:

• Pinned interpreter — requires-python == 3.12.4 plus a committed .python-version and uv's auto-provisioning make the Python patch identical on laptop, CI, and cluster, closing the interpreter half of the drift.
• Exact numeric pins + hashes — pinning numpy and scikit-learn to exact versions in pyproject.toml and recording sha256 hashes in uv.lock guarantees the same bytes everywhere, which is what numerical determinism actually requires.
• uv sync --locked everywhere — every host reconciles .venv against the committed lock and refuses to re-resolve, so no machine can silently diverge onto a newer package.
• --no-dev on the cluster — the training host installs only runtime deps, keeping the image lean and identical to what was verified, without test tooling.
• Cost — exact pins mean deliberate, reviewed upgrades (a small process cost) in exchange for byte-identical environments; the lock and .python-version add two committed files. Compared to chasing nondeterministic training flakiness, the trade is overwhelmingly worth it. O(1) reproduce per host.

Python
Topic — defensive-coding
Defensive-coding problems on reproducibility and pinning

Practice →

Python Topic — pandas Pandas problems on deterministic numeric pipelines

Practice →

---

4. uv in Docker and CI

Cache mounts, locked installs, and layer splits — uv builds fast, deterministic images and green pipelines

The mental model in one line: uv earns its keep in Docker and CI by making two things cheap that used to be expensive — a fast, cache-mounted install and a layer-split that installs your locked dependencies before copying your source, so a code-only change reuses the cached dependency layer — and combined with uv sync --locked and a couple of environment variables (UV_COMPILE_BYTECODE, UV_LINK_MODE), it produces slim, reproducible images and pipelines that spend seconds, not minutes, on dependencies. The Docker layer-cache pattern is the single highest-leverage uv technique for teams that build a container on every commit.

Getting uv into the image.

• Copy the binary. COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ grabs a static uv binary from the official image — no pip-installing uv, no Python needed to bootstrap it.
• Or the pre-baked base. ghcr.io/astral-sh/uv:python3.12-bookworm-slim ships Python + uv together.
• Pin the uv version. Use a tag (:0.4.20) rather than :latest in production images so the build tool itself is reproducible.

The layer-split pattern — the core technique.

• The problem. If you COPY . . then install, any source change busts the cache and reinstalls every dependency.
• The fix. Copy only pyproject.toml + uv.lock first, run uv sync --frozen --no-install-project (installs dependencies but not your code), then COPY your source and run uv sync --frozen again (installs just your project).
• The payoff. A code-only change reuses the cached dependency layer; only the tiny project layer rebuilds. Dependency installs happen only when the lock changes.

The cache mount — reuse uv's cache across builds.

• --mount=type=cache,target=/root/.cache/uv. A BuildKit cache mount persists uv's global cache between builds, so even a lock change only downloads the new packages, not the whole set.
• Combine with the layer split. Layer cache handles "lock unchanged"; cache mount handles "lock changed a little". Together, near-instant installs.

The two environment variables that matter in containers.

• UV_COMPILE_BYTECODE=1. Pre-compile .pyc files at install time so the container doesn't pay compilation cost on first import — meaningful for cold-start-sensitive jobs.
• UV_LINK_MODE=copy. In containers the cache mount and the target .venv are often on different filesystems where hardlinks fail; copy avoids the warning and works everywhere. (Locally, the default hardlink mode is faster.)

CI patterns.

• astral-sh/setup-uv. The official GitHub Action installs uv, optionally a pinned version, and enables caching of uv's global cache keyed on uv.lock.
• uv sync --locked. The CI gate — fail on a stale lock.
• Matrix on Python versions. uv python install ${{ matrix.python }} makes testing across 3.11/3.12/3.13 cheap because uv provisions each interpreter fast.

Common interview probes on uv in Docker/CI.

• "How do you keep Docker builds fast when only code changes?" — layer-split: install locked deps before copying source.
• "What does the cache mount buy you?" — persist uv's cache across builds so a lock change downloads only new packages.
• "Why UV_LINK_MODE=copy in a container?" — cross-filesystem cache/venv where hardlinks fail.
• "How does CI catch a stale lock?" — uv sync --locked fails the build.

Worked example — a layer-cached uv Dockerfile

Detailed explanation. The canonical production Dockerfile: bring in the uv binary, install only the locked dependencies in a cache-mounted layer that ignores your source, then copy the source and install the project itself. This ordering is what makes a code-only change rebuild in seconds. Walk through every line.

• Stage the tool. Copy uv from the official image.
• Deps layer. Copy pyproject.toml + uv.lock; uv sync --frozen --no-install-project.
• Project layer. Copy source; uv sync --frozen.

Question. Write a Dockerfile that keeps the dependency layer cached across code-only changes.

Input.

Layer	Copies	Command	Busted by
tool	uv binary	COPY --from=…	uv version change
deps	pyproject + lock	uv sync --no-install-project	lock change
project	source	uv sync	any code change

Code.

FROM python:3.12-slim-bookworm

# 1. Bring in a pinned uv binary (no pip bootstrap needed)
COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /uvx /bin/

WORKDIR /app

# Container-friendly settings
ENV UV_COMPILE_BYTECODE=1 \
    UV_LINK_MODE=copy \
    UV_PYTHON_DOWNLOADS=never

# 2. DEPENDENCY LAYER — copy only the lock inputs, install deps (not the project)
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev --no-install-project

# 3. PROJECT LAYER — now copy source and install just the project
COPY . .
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev

# 4. Run through uv (uses the project's .venv)
ENV PATH="/app/.venv/bin:$PATH"
CMD ["uv", "run", "python", "-m", "app.main"]

Step-by-step explanation.

1. COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /uvx /bin/ pulls a static, version-pinned uv into the image without needing pip or even a working Python to install it — the build tool is reproducible because the tag is pinned.
2. The dependency layer copies only pyproject.toml and uv.lock, then runs uv sync --frozen --no-install-project. --no-install-project installs all dependencies but skips your own package, so this layer depends solely on the lock — a source change does not invalidate it.
3. --mount=type=cache,target=/root/.cache/uv persists uv's global cache across builds. When the lock changes, only the new or changed packages are downloaded; everything else is served from the mounted cache.
4. The project layer copies the source and runs uv sync --frozen again — this time installing your package into .venv. It is the only layer that rebuilds on a code change, and it is tiny and fast.
5. UV_LINK_MODE=copy avoids hardlink-across-filesystem warnings between the cache mount and .venv; UV_COMPILE_BYTECODE=1 pre-compiles .pyc so first import is fast; UV_PYTHON_DOWNLOADS=never ensures the image uses the base image's Python rather than fetching another.

Output.

Change type	Layers rebuilt	Approx time
Edit source only	project layer	~1–3 s
Add one dependency	deps + project (cache mount helps)	~5–10 s
Bump uv version	all	full build
No change	none (fully cached)	instant

Rule of thumb. Split the Dockerfile into a deps layer (--no-install-project, lock-only inputs) and a project layer (source), mount a cache at /root/.cache/uv, and set UV_COMPILE_BYTECODE=1 + UV_LINK_MODE=copy. A code-only change then rebuilds in seconds.

Worked example — GitHub Actions with setup-uv and --locked

Detailed explanation. The CI counterpart to the Dockerfile: install uv with the official action, cache its global cache keyed on the lock, verify the lock with --locked, and run tests through uv run. Adding a Python-version matrix is nearly free because uv provisions interpreters fast. Walk through the workflow.

• Install uv. astral-sh/setup-uv with caching enabled.
• Verify + install. uv sync --locked.
• Matrix. Test on 3.11, 3.12, 3.13.

Question. Write a GitHub Actions workflow that verifies the lock and runs tests across three Python versions.

Input.

Job step	Purpose
checkout	get the repo
setup-uv	install uv + enable cache
uv python install	provision the matrix interpreter
uv sync --locked	verify lock + install
uv run pytest	run tests in the env

Code.

name: ci
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4

      - name: Install uv (with cache)
        uses: astral-sh/setup-uv@v3
        with:
          version: "0.4.20"           # pin the tool
          enable-cache: true          # cache ~/.cache/uv keyed on uv.lock

      - name: Provision the interpreter for this matrix leg
        run: uv python install ${{ matrix.python-version }}

      - name: Install (fail on a stale lock)
        run: uv sync --locked

      - name: Lint + type-check + test
        run: |
          uv run ruff check .
          uv run mypy src
          uv run pytest -q

Step-by-step explanation.

1. astral-sh/setup-uv@v3 installs a pinned uv and, with enable-cache: true, caches uv's global cache directory keyed on the uv.lock hash — so a run whose lock is unchanged restores the cache and downloads nothing.
2. uv python install ${{ matrix.python-version }} provisions the exact interpreter for each matrix leg. Because uv fetches standalone builds quickly, a three-version matrix adds seconds, not minutes, over a single version.
3. uv sync --locked is the gate: it verifies uv.lock is current with pyproject.toml and installs the exact locked set. If a PR changed a dependency but forgot to re-lock, this step fails loudly instead of silently re-resolving.
4. Running lint, type-check, and tests through uv run guarantees each tool executes inside the synced .venv with the locked versions — no "CI has a different ruff than my laptop" surprises.
5. The cache key on uv.lock means the common case (no dependency change) restores instantly; a dependency change downloads only the delta, keeping even "cold-ish" runs fast.

Output.

Matrix leg	Interpreter	Lock check	Cache
3.11	auto-provisioned	--locked gate	keyed on uv.lock
3.12	auto-provisioned	--locked gate	keyed on uv.lock
3.13	auto-provisioned	--locked gate	keyed on uv.lock
stale lock PR	—	build fails	—

Rule of thumb. In CI, use astral-sh/setup-uv with enable-cache: true and a pinned version, provision matrix interpreters with uv python install, and gate on uv sync --locked. You get a fast, cached, multi-version pipeline that refuses to run on a stale lock.

Worked example — a multi-stage slim final image

Detailed explanation. For the smallest, safest production image, use a multi-stage build: do all the installing in a builder stage, then copy only the finished .venv (and source) into a fresh slim runtime stage that doesn't even contain uv. The result is a minimal image with no build tooling and no cache cruft. Walk through the two stages.

• Builder stage. Has uv; builds .venv.
• Runtime stage. Fresh slim base; copies .venv + source; no uv, no cache.
• Payoff. Smaller image, smaller attack surface.

Question. Write a multi-stage Dockerfile that ships only the runtime environment.

Input.

Stage	Contains	Purpose
builder	uv + cache + .venv	resolve + install
runtime	Python slim + .venv + source	run only

Code.

# ---- Stage 1: builder (has uv) ----
FROM python:3.12-slim-bookworm AS builder
COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /bin/uv
WORKDIR /app
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy

COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev --no-install-project
COPY . .
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev

# ---- Stage 2: runtime (no uv, no cache) ----
FROM python:3.12-slim-bookworm AS runtime
WORKDIR /app
# Copy the finished virtual environment and the app source only
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/src   /app/src
ENV PATH="/app/.venv/bin:$PATH"
CMD ["python", "-m", "app.main"]

Step-by-step explanation.

1. The builder stage is where all the work happens: it has the uv binary, a cache mount, and produces a fully-populated .venv via the same layer-split pattern (deps first, then project).
2. The runtime stage starts from a fresh python:3.12-slim base — no uv, no build tools, no cache mount residue. It copies only two things from the builder: the finished .venv and the application source.
3. Because the .venv is relocatable and UV_LINK_MODE=copy was used, copying it into the runtime stage yields a working environment without needing uv present at runtime — the app just runs python -m app.main off the venv's PATH.
4. The final image contains only Python, the app, and its dependencies — no uv, no pip cache, no .pyc compilation cost (already done via UV_COMPILE_BYTECODE). Smaller size and a smaller attack surface.
5. This pattern is ideal for production where you want the leanest possible image; for local dev images the single-stage version (with uv present) is more convenient because you can uv add interactively.

Output.

Aspect	Single-stage	Multi-stage
uv in final image	yes	no
Build cache in image	possible	no
Image size	larger	smaller
Best for	dev / debugging	production

Rule of thumb. For production, build in a builder stage that has uv and copy only the finished .venv + source into a fresh slim runtime stage. The final image has no build tooling, no cache, and the smallest possible surface — while still being byte-reproducible from the lock.

Data engineering interview question on uv in Docker and CI

A senior interviewer might ask: "Every commit to our pipeline repo builds a Docker image and runs CI, and right now each build spends about two minutes on pip install because any code change reinstalls everything. Redesign the Docker build and the CI workflow with uv so that a code-only change is near-instant, the lock is enforced, and the production image is slim. Walk me through the layers, the cache, and the flags."

Solution Using a layer-split cache-mounted Dockerfile + setup-uv with --locked

# Multi-stage, layer-split, cache-mounted build
FROM python:3.12-slim-bookworm AS builder
COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /bin/uv
WORKDIR /app
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy UV_PYTHON_DOWNLOADS=never

# deps layer: lock-only inputs, cache-mounted, project excluded
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev --no-install-project

# project layer: source, then install just the project
COPY . .
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev

FROM python:3.12-slim-bookworm AS runtime
WORKDIR /app
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/src   /app/src
ENV PATH="/app/.venv/bin:$PATH"
CMD ["python", "-m", "pipeline.run"]

# CI — verify the lock, cache uv, matrix on Python
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix: { python-version: ["3.12", "3.13"] }
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v3
        with: { version: "0.4.20", enable-cache: true }
      - run: uv python install ${{ matrix.python-version }}
      - run: uv sync --locked          # stale-lock gate
      - run: uv run pytest -q

Step-by-step trace.

Concern	Mechanism	Result
Code-only change fast	deps layer excludes source (--no-install-project)	only project layer rebuilds
Lock change fast	BuildKit cache mount on uv cache	download only the delta
Slim prod image	multi-stage; copy only .venv + src	no uv, no cache in final
Lock enforced	uv sync --locked in CI	stale lock fails the build
Multi-version	uv python install per matrix leg	cheap interpreter provisioning

After the redesign, a code-only commit reuses the cached dependency layer and rebuilds in a couple of seconds; a dependency change downloads only the new wheels thanks to the cache mount; the production image ships without uv or build caches; and CI refuses to run on a stale lock while testing multiple Python versions in parallel.

Output:

Metric	Before (pip)	After (uv)
Code-only image rebuild	~2 min	~1–3 s
Dependency-change rebuild	~2 min	~5–10 s
Final image contents	pip + cache + tools	.venv + source only
Stale-lock detection	none	--locked fails build
CI multi-version cost	expensive	seconds per leg

Why this works — concept by concept:

• Layer split (--no-install-project) — installing locked dependencies in a layer that copies only pyproject.toml + uv.lock means a source change can't invalidate the dependency layer, so code-only rebuilds skip the install entirely.
• BuildKit cache mount — persisting /root/.cache/uv across builds turns a lock change into a delta download rather than a full re-download, so even dependency changes are fast.
• Multi-stage build — resolving in a builder stage and copying only the finished .venv into a fresh slim runtime yields a minimal image with no uv and no cache, shrinking size and attack surface.
• uv sync --locked in CI — the lock gate converts "forgot to re-lock" into an immediate build failure, keeping the committed resolution authoritative.
• Cost — a slightly more structured Dockerfile and a pinned uv version, in exchange for near-instant rebuilds and slim images. Compared to a flat COPY . . && pip install, the engineering cost is a one-time Dockerfile edit; the payoff recurs on every commit. O(1) rebuild on code-only changes.

Python
Topic — etl
ETL problems on containerised pipeline builds

Practice →

Python Topic — defensive-coding Defensive-coding problems on CI reproducibility gates

Practice →

---

5. Migrating from pip, poetry, and conda

From requirements.txt, poetry.lock, and environment.yml to pyproject.toml + uv.lock — incrementally, without a big-bang rewrite

The mental model in one line: migrating to uv is a mechanical convergence — a pip project's requirements.txt becomes pyproject.toml dependencies plus a uv.lock, a poetry project's [tool.poetry] table converts to the standard PEP 621 [project] table, and a conda environment.yml splits into "PyPI-installable" (goes into uv) and "genuinely non-PyPI binary" (stays on conda) — and in every case you can do it incrementally and keep an exported requirements.txt around so downstream pip consumers never break. The migration is rarely risky because uv reads and writes the same standards (PEP 621, PEP 508, .python-version) the other tools do.

Migrating from pip + requirements.txt.

• The simplest path. uv init to create pyproject.toml, then uv add -r requirements.txt to import the pinned set as dependencies and produce a uv.lock.
• The pip-tools path. If you used pip-compile (loose requirements.in → pinned requirements.txt), keep that exact workflow with uv pip compile requirements.in -o requirements.txt — a faster drop-in — or graduate to the project model.
• Keep consumers working. uv export --format requirements-txt > requirements.txt regenerates a pinned file for anything that still runs pip install -r.

Migrating from poetry.

• The table swap. poetry's [tool.poetry] (with its own dependency syntax and ^ carets) converts to the standard [project] table with PEP 508 dependency strings.
• Groups. poetry's [tool.poetry.group.dev.dependencies] maps cleanly to uv's [dependency-groups].dev.
• The lock. Delete poetry.lock; uv lock produces uv.lock. Version pins carry over; re-resolve to confirm.
• The caret caveat. poetry's ^1.2 means >=1.2,<2.0; write that explicitly in PEP 508 (>=1.2,<2.0) since standard metadata has no caret operator.

Migrating from conda.

• The split. environment.yml mixes PyPI packages and non-PyPI binaries. Sort each dependency into "installable from PyPI" (→ pyproject.toml) vs "genuinely conda-only" (CUDA toolkits, MKL, some geospatial/GDAL builds).
• The majority moves. Most data-stack packages (pandas, numpy, scikit-learn, pyarrow) are on PyPI with good wheels — those go straight into uv.
• The remainder. Truly non-PyPI system/binary deps stay on conda, or you use a base image that provides them; uv manages the Python layer on top.
• The caveat. conda pins can include build strings and channels that have no PyPI equivalent; verify the PyPI version resolves to equivalent binaries.

Incremental strategy.

• One repo at a time. Migrate a single service, prove CI is green, then move the next — no org-wide flag day.
• Bridge with export. Keep uv export producing a requirements.txt during the transition so mixed pip/uv consumers coexist.
• Verify, don't trust. After importing pins, run the test suite under uv sync --locked to confirm the resolved set behaves identically.

Common interview probes on migration.

• "How do you migrate a pip requirements.txt to uv?" — uv init + uv add -r requirements.txt; commit uv.lock.
• "What's the poetry → uv gotcha?" — carets (^) become explicit PEP 508 ranges; groups map to [dependency-groups].
• "Can uv replace conda entirely?" — for PyPI-installable stacks yes; keep conda only for genuinely non-PyPI binaries.
• "How do you avoid breaking pip consumers mid-migration?" — uv export a pinned requirements.txt.

Worked example — pip + requirements.txt → uv

Detailed explanation. The most common migration: a project with a hand-maintained (or pip freeze-d) requirements.txt. Import it into a uv project, produce a lock, and verify. Walk through it, including the pip-tools variant.

• Import. uv add -r requirements.txt folds the pins into pyproject.toml.
• Lock. uv lock writes uv.lock.
• Verify. uv sync --locked + tests.

Question. Convert a pip project with requirements.txt and requirements-dev.txt into a uv project, preserving the dev/runtime split.

Input.

Legacy file	Contents	Target
requirements.txt	pandas, pyarrow, sqlalchemy	[project.dependencies]
requirements-dev.txt	pytest, ruff	[dependency-groups].dev
(none)	—	uv.lock

Code.

# 1. Scaffold a uv project alongside the existing files
uv init --package .

# 2. Import runtime deps and dev deps into the right buckets
uv add -r requirements.txt
uv add --group dev -r requirements-dev.txt

# 3. Produce and commit the lock; verify against tests
uv lock
uv sync --locked
uv run pytest -q

# 4. (Optional) keep a pinned requirements.txt for pip-only consumers
uv export --no-dev --format requirements-txt > requirements.txt

# ── pip-tools users: keep the exact compile workflow, just faster ──
uv pip compile requirements.in -o requirements.txt   # drop-in for pip-compile

# Resulting pyproject.toml
[project]
name = "legacy-pip-job"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "pandas>=2.2.0",
    "pyarrow>=16.0.0",
    "sqlalchemy>=2.0.0",
]

[dependency-groups]
dev = ["pytest>=8.0", "ruff>=0.6"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Step-by-step explanation.

1. uv init --package . scaffolds a pyproject.toml in the existing directory without disturbing the current requirements*.txt — so you can migrate in place and roll back trivially by deleting the new files.
2. uv add -r requirements.txt reads each requirement and adds it to [project.dependencies], then resolves the full graph. uv add --group dev -r requirements-dev.txt routes the dev requirements into [dependency-groups].dev, preserving the runtime/dev split the two files encoded.
3. uv lock writes uv.lock with exact pins and hashes; uv sync --locked builds .venv from it and uv run pytest confirms the imported set behaves like the old one. Running the tests is the verification step — never trust a migration without it.
4. uv export --no-dev --format requirements-txt > requirements.txt regenerates a pinned runtime requirements.txt from the lock, so any downstream job or Dockerfile that still does pip install -r requirements.txt keeps working unchanged during the transition.
5. For teams on pip-tools, uv pip compile requirements.in -o requirements.txt is a faster, drop-in pip-compile — you can adopt uv's speed without changing the workflow at all, then graduate to the project model later.

Output.

Legacy artifact	uv artifact	Status
requirements.txt	[project.dependencies] + uv.lock	migrated
requirements-dev.txt	[dependency-groups].dev	migrated
pip install -r consumers	uv export → requirements.txt	still working
pip-compile	uv pip compile	faster drop-in

Rule of thumb. Migrate a pip project with uv init + uv add -r requirements.txt (and --group dev -r for dev deps), commit uv.lock, and keep uv export producing a requirements.txt for pip-only consumers. Verify with the test suite before deleting the old files.

Worked example — poetry → uv

Detailed explanation. poetry projects already have a pyproject.toml, but it uses poetry's proprietary [tool.poetry] table and caret constraints rather than the PEP 621 [project] standard uv reads. The migration is a mechanical table conversion. Walk through it.

• Convert the table. [tool.poetry] → [project]; poetry dependency dict → PEP 508 strings.
• Convert carets. ^1.2 → >=1.2,<2.0.
• Map groups. [tool.poetry.group.dev.dependencies] → [dependency-groups].dev.

Question. Convert a poetry pyproject.toml to a uv-compatible PEP 621 pyproject.toml.

Input.

poetry construct	PEP 621 / uv equivalent
[tool.poetry] name/version	[project] name/version
python = "^3.12"	requires-python = ">=3.12,<4.0"
pandas = "^2.2"	"pandas>=2.2,<3.0"
[tool.poetry.group.dev.dependencies]	[dependency-groups].dev

Code.

# ── BEFORE: poetry pyproject.toml ──
[tool.poetry]
name = "svc"
version = "1.0.0"

[tool.poetry.dependencies]
python = "^3.12"
pandas = "^2.2"
httpx = "^0.27"

[tool.poetry.group.dev.dependencies]
pytest = "^8.0"
ruff = "^0.6"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

# ── AFTER: PEP 621 pyproject.toml (uv-native) ──
[project]
name = "svc"
version = "1.0.0"
requires-python = ">=3.12,<4.0"
dependencies = [
    "pandas>=2.2,<3.0",     # ^2.2 expanded explicitly
    "httpx>=0.27,<0.28",    # note: ^0.27 -> >=0.27,<0.28 (0.x caret rule)
]

[dependency-groups]
dev = ["pytest>=8.0", "ruff>=0.6"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# After converting the table, generate the lock and verify
rm poetry.lock
uv lock
uv sync --locked
uv run pytest -q

Step-by-step explanation.

1. The [tool.poetry] name/version/description fields move into the standard [project] table. This is the core change: uv reads PEP 621 metadata, not poetry's proprietary table.
2. poetry's python = "^3.12" becomes requires-python = ">=3.12,<4.0". Each dependency dict entry (pandas = "^2.2") becomes a PEP 508 string in the dependencies list.
3. The caret conversion needs care. For >=1.0 versions, ^2.2 means >=2.2,<3.0. For 0.x versions, poetry's caret is narrower: ^0.27 means >=0.27,<0.28 (the first nonzero segment is treated as the breaking one). Writing these explicitly avoids a subtly-wrong range.
4. poetry's dev group ([tool.poetry.group.dev.dependencies]) maps to uv's standard [dependency-groups].dev. The build backend also swaps — poetry-core → hatchling (or any PEP 517 backend) — since you're no longer using poetry to build.
5. Deleting poetry.lock and running uv lock produces uv.lock; uv sync --locked + tests verify the converted ranges resolve to a working set. Re-resolving (rather than transcribing the old lock) is intentional — it confirms the PEP 508 ranges are correct.

Output.

poetry	uv	Gotcha
^2.2 (>=1.0)	>=2.2,<3.0	standard caret
^0.27 (0.x)	>=0.27,<0.28	narrower 0.x caret
poetry.lock	uv.lock	re-resolve to verify
poetry-core	hatchling	build backend swap

Rule of thumb. poetry → uv is a [tool.poetry] → [project] table conversion with explicit caret expansion (mind the 0.x rule) and group.dev → [dependency-groups].dev. Delete poetry.lock, run uv lock, and verify with tests — re-resolving confirms the ranges are right.

Worked example — conda environment.yml → uv

Detailed explanation. conda migrations are the most nuanced because environment.yml mixes PyPI-installable packages with genuinely non-PyPI binaries. The strategy is to sort dependencies into two buckets and only move the PyPI-installable ones into uv. Walk through a typical data-science environment.

• Sort. PyPI-installable (pandas, scikit-learn, pyarrow) vs conda-only binaries (a CUDA build, a GDAL system lib).
• Move the majority. PyPI packages → pyproject.toml.
• Handle the remainder. Keep conda or a base image for the non-PyPI binaries.

Question. Convert an environment.yml to a uv pyproject.toml, isolating the genuinely conda-only dependencies.

Input.

conda dep	On PyPI?	Destination
pandas	yes	uv pyproject.toml
scikit-learn	yes	uv pyproject.toml
pyarrow	yes	uv pyproject.toml
gdal (system lib)	not cleanly	conda / base image
cudatoolkit	no	conda / CUDA base image

Code.

# ── BEFORE: environment.yml ──
name: ds-env
channels: [conda-forge]
dependencies:
  - python=3.12
  - pandas=2.2.2
  - scikit-learn=1.5.1
  - pyarrow=16.1.0
  - gdal=3.8            # non-PyPI system library
  - cudatoolkit=12.1    # non-PyPI GPU binary
  - pip:
      - mlflow==2.14.0   # already a pip dep inside conda

# ── AFTER: pyproject.toml (PyPI-installable subset moves to uv) ──
[project]
name = "ds-env"
version = "0.1.0"
requires-python = "==3.12.*"
dependencies = [
    "pandas==2.2.2",
    "scikit-learn==1.5.1",
    "pyarrow==16.1.0",
    "mlflow==2.14.0",
    # gdal / cudatoolkit intentionally NOT here (non-PyPI binaries)
]

# Move the PyPI subset into uv and lock it
uv python pin 3.12
uv add pandas==2.2.2 scikit-learn==1.5.1 pyarrow==16.1.0 mlflow==2.14.0
uv lock && uv sync --locked

# The non-PyPI binaries stay provisioned by conda or a base image, e.g.:
#   FROM nvidia/cuda:12.1-runtime  (provides CUDA)
#   conda install -c conda-forge gdal=3.8   (or apt-get libgdal-dev)
# uv then manages the Python layer on top of that base.

Step-by-step explanation.

1. The first step is triage: go through environment.yml and label each dependency PyPI-installable or not. pandas, scikit-learn, pyarrow, and the already-pip mlflow all have good PyPI wheels; gdal and cudatoolkit are system/GPU binaries that conda provides and PyPI does not (cleanly).
2. The PyPI-installable subset moves into pyproject.toml with the same exact pins the conda file used, and uv add + uv lock produces a uv.lock. uv now manages that entire layer.
3. The non-PyPI binaries stay out of uv. In a container, you get them from a base image (nvidia/cuda for CUDA, a base with libgdal from apt or conda) and let uv manage the Python packages on top. This "conda/base for binaries, uv for Python" split is the standard hybrid.
4. requires-python = "==3.12.*" and uv python pin 3.12 keep the interpreter consistent; note that when the base image already provides Python (e.g. a CUDA image with system Python), you set UV_PYTHON_DOWNLOADS=never and let uv use it.
5. The caveat to verify: conda pins sometimes correspond to builds (channel + build string) that differ subtly from the PyPI wheel. After migrating, run the numerical tests to confirm the PyPI scikit-learn/pyarrow behave identically to the conda builds for your workload.

Output.

Dependency	Before (conda)	After
pandas / sklearn / pyarrow	conda-forge	uv pyproject.toml + lock
mlflow	conda pip:	uv pyproject.toml
gdal	conda-forge	base image / conda (kept)
cudatoolkit	conda-forge	CUDA base image (kept)
Python layer	conda	uv-managed

Rule of thumb. For conda, migrate the PyPI-installable majority into uv and keep conda or a base image only for genuinely non-PyPI binaries (CUDA, GDAL, MKL). Verify numeric behavior with tests, because a conda build and a PyPI wheel of the same version can differ. uv manages the Python layer; the base provides the system binaries.

Data engineering interview question on migration

A senior interviewer might ask: "We have three repos: one on pip + requirements.txt, one on poetry, and one on conda for a GPU model. Leadership wants everything on uv, but nothing can break during the transition and some downstream jobs only understand pip install -r requirements.txt. Design the migration for each repo, the order you'd do it, and how you keep the pip consumers working throughout."

Solution Using per-repo conversion + uv export bridges + an incremental rollout

# ── Repo A: pip + requirements.txt (lowest risk — do first) ──
uv init --package .
uv add -r requirements.txt
uv add --group dev -r requirements-dev.txt
uv lock && uv sync --locked && uv run pytest -q
uv export --no-dev --format requirements-txt > requirements.txt   # keep pip consumers alive

# ── Repo B: poetry (mechanical table swap — do second) ──
# hand-convert [tool.poetry] -> [project], carets -> PEP 508 ranges,
# group.dev -> [dependency-groups].dev, poetry-core -> hatchling
rm poetry.lock
uv lock && uv sync --locked && uv run pytest -q

# ── Repo C: conda GPU (highest nuance — do last) ──
# triage environment.yml: PyPI subset -> uv; CUDA/GDAL -> base image
uv python pin 3.12
uv add pandas==2.2.2 scikit-learn==1.5.1 pyarrow==16.1.0 mlflow==2.14.0
uv lock && uv sync --locked
# Dockerfile: FROM nvidia/cuda:12.1-runtime ; UV_PYTHON_DOWNLOADS handled per base
uv run pytest -q     # verify numeric parity vs the old conda build

Step-by-step trace.

Repo	Source	Conversion	Bridge
A (pip)	requirements.txt	uv add -r	uv export → requirements.txt
B (poetry)	[tool.poetry]	table swap + caret expand	re-lock + tests
C (conda GPU)	environment.yml	PyPI subset only; CUDA on base	numeric parity tests
Order	—	A → B → C (risk-ascending)	CI green before next

After the rollout, Repo A migrates first (lowest risk) and keeps an exported requirements.txt so its pip consumers never notice; Repo B is a mechanical table conversion verified by re-resolving; Repo C moves its PyPI layer to uv while CUDA and GDAL stay on the base image, with numeric parity tests guarding correctness. Each repo lands independently with green CI before the next starts — no flag day.

Output:

Repo	Ends on	pip consumers	Verified by
A	pyproject + uv.lock	uv export bridge	pytest
B	PEP 621 + uv.lock	n/a	re-lock + pytest
C	uv (Python) + base (binaries)	n/a	numeric parity pytest
Org	uv everywhere	unbroken throughout	per-repo green CI

Why this works — concept by concept:

• Risk-ascending order — migrating the pip repo first (mechanical uv add -r), then poetry (table swap), then conda (binary triage) means the hardest case is done last, after the team has built confidence on the easy ones.
• uv export bridge — regenerating a pinned requirements.txt from uv.lock keeps pip-only downstream jobs working unchanged, so the migration never forces a simultaneous consumer rewrite.
• PEP 621 conversion — poetry's proprietary table becomes standard [project] metadata uv reads natively; explicit caret expansion avoids silently-wrong version ranges.
• conda binary triage — moving the PyPI-installable majority to uv while keeping CUDA/GDAL on a base image gives you uv's speed for the Python layer without fighting conda's genuine strength (non-PyPI binaries).
• Cost — a per-repo conversion (hours each) plus verification tests, spread across an incremental rollout with no flag day. Compared to a big-bang rewrite, the risk is bounded to one repo at a time and every pip consumer stays alive via export. O(repos) one-time work, then O(1) ongoing.

Python
Topic — pandas
Pandas problems on environment migration

Practice →

Python
Topic — etl
ETL problems on packaging legacy pipelines

Practice →

---

Cheat sheet — uv recipes

• The project loop. uv init --package <name> (scaffold) → uv add <pkg> / uv add --group dev <pkg> (declare, updates pyproject.toml + uv.lock + .venv) → uv lock (re-resolve) → uv sync (reconcile .venv) → uv run <cmd> (sync-then-run). Never hand-edit a pinned requirements file again; declare loose ranges, let the lock hold the pins.
• pyproject.toml skeleton. [project] with name, version, requires-python, and dependencies = ["pandas>=2.2", ...] (loose ranges); [dependency-groups] dev = ["pytest", "ruff", "mypy"] for developer-only tools; [project.optional-dependencies] for user-facing extras (postgres = ["psycopg"]); [build-system] with hatchling when you need an installable package.
• Two files, two jobs. pyproject.toml = human-authored loose declaration; uv.lock = machine-generated, universal (all-platform), hashed, exact resolution. Commit both plus .python-version; gitignore .venv.
• Sync flag matrix. uv sync (default: re-lock if pyproject changed, then install — local dev). uv sync --locked (verify the lock is current, fail if stale — CI gate). uv sync --frozen (install from the lock as-is, no check — trusted container builds). --no-dev drops the dev group; --extra X / --all-extras add optional features.
• Python version management. uv python install 3.12.4 (download a standalone build), uv python pin 3.12.4 (write .python-version), uv python list (see installed/available). requires-python in pyproject bounds the range; .python-version pins the exact patch; missing interpreters auto-download on uv sync/uv run.
• pip-compatible layer. uv pip install <pkg> (untracked, ad-hoc), uv pip compile requirements.in -o requirements.txt (faster pip-compile), uv pip sync requirements.txt (make env match a file). Use the pip layer only for throwaway/scripted installs; uv sync will remove anything not in pyproject.toml.
• Tools (pipx replacement). uv tool install ruff (persistent global CLI in an isolated env), uvx ruff check . (run a tool ephemerally, no install). Keeps global CLIs out of your project's dependency graph.
• Dockerfile recipe. COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /bin/uv; set ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy; copy pyproject.toml uv.lock then RUN --mount=type=cache,target=/root/.cache/uv uv sync --frozen --no-dev --no-install-project; then COPY . . and uv sync --frozen --no-dev. Multi-stage: copy only .venv + source into a fresh slim runtime with no uv.
• CI recipe. astral-sh/setup-uv@v3 with version: pinned and enable-cache: true (caches uv's cache keyed on uv.lock); uv python install ${{ matrix.python-version }}; uv sync --locked (stale-lock gate); run everything via uv run.
• Migration one-liners. pip: uv init && uv add -r requirements.txt (+ --group dev -r requirements-dev.txt). poetry: convert [tool.poetry] → [project], carets → PEP 508 ranges (mind the 0.x rule), rm poetry.lock && uv lock. conda: move the PyPI-installable subset into pyproject.toml, keep CUDA/GDAL/MKL on a base image or conda.
• Compatibility bridges. uv export --no-dev --format requirements-txt > requirements.txt regenerates a pinned file for pip-only consumers; uv pip compile covers pip-tools workflows. These keep legacy jobs alive during an incremental migration.
• Cache + workspaces. Global content-addressed cache with hardlinks into each .venv (uv cache clean to reset, uv cache prune to trim). Monorepos: [tool.uv.workspace] members = ["packages/*"] share one root uv.lock; members depend on each other via tool.uv.sources path entries.

Frequently asked questions

Is uv a drop-in pip replacement?

Partly and by design. uv ships a pip-compatible interface — uv pip install, uv pip freeze, uv pip compile, uv pip sync — that mirrors pip's commands and is a faster drop-in for scripts and ad-hoc use. But uv's real value is the higher-level project interface (uv add, uv lock, uv sync, uv run) that manages pyproject.toml and a universal uv.lock for you. For a quick "install this package" you can treat uv pip install as pip-with-a-turbo; for a real project you graduate to the project loop, which pip alone doesn't offer. uv also folds in venv, pip-tools, and pyenv, so calling it "just a faster pip" undersells it.

Should I commit uv.lock?

Yes — for applications, data pipelines, and services, always commit uv.lock. The lockfile is the universal, hashed, fully-resolved record of your dependency graph, and committing it is what makes uv sync --locked reproduce a byte-identical environment on every laptop, CI runner, and production image. The only case where you might not commit a lock is a library meant to be installed into other projects, where you want consumers to resolve fresh against their own constraints — but even then, committing the lock is useful for your own CI. Alongside uv.lock, commit pyproject.toml and .python-version; gitignore .venv.

uv vs poetry vs pip-tools — when do I pick each?

Pick uv for new projects and for teams that want one fast tool covering install, lock, virtualenv, and Python-version management with native PEP 621 pyproject.toml. poetry remains reasonable for existing application teams already invested in it, but uv resolves faster, uses standard metadata (no proprietary [tool.poetry] table), and additionally manages interpreters — migration is a mechanical table swap. pip-tools (pip-compile) is a narrower tool that only compiles a requirements.in into a pinned requirements.txt; uv reproduces that exact workflow with uv pip compile at much higher speed, so there's little reason to keep pip-tools separately once uv is in the toolchain. In 2026 the default recommendation for greenfield data projects is uv.

Can uv manage Python versions like pyenv?

Yes — this is one of the tools uv absorbs. uv python install 3.12.4 downloads a standalone, relocatable CPython build (no compiling from source, no admin rights), uv python pin 3.12.4 writes the same .python-version file pyenv uses, and uv python list shows installed and available builds. Crucially, uv auto-provisions a missing interpreter when a project's .python-version or requires-python demands it, so a fresh git clone + uv sync bootstraps the exact Python without a separate pyenv step. This is what lets "reproducible environment" include the interpreter, not just the libraries — a gap pip and poetry historically left open.

How does uv make installs so fast?

Three things, only one of which is "it's written in Rust." First, uv keeps a global content-addressed cache and hardlinks (or copies/reflinks) packages into each .venv, so the same wheel is never downloaded or unpacked twice across all your projects — a warm environment recreate is near-instant because no bytes are copied. Second, uv's dependency resolver is a purpose-built Rust implementation that finds a compatible version set in milliseconds where pip's backtracking resolver can take seconds. Third, uv downloads and builds in parallel. The cache + hardlink model is the biggest lever: it turns "rebuild the environment" from an O(files-copied) operation into an O(directory-entries) one.

Does uv work for conda / scientific stacks?

For the PyPI-installable majority, yes — pandas, numpy, scikit-learn, pyarrow, and most of the scientific Python stack ship good PyPI wheels that uv installs quickly and reproducibly. Where conda still wins is genuinely non-PyPI binaries: CUDA toolkits, MKL builds, and some geospatial/system libraries (GDAL) that conda-forge packages and PyPI does not cleanly provide. The standard hybrid is to keep those binaries on conda or a base image (for example an nvidia/cuda image for GPU work) and let uv manage the Python layer on top. When you migrate, verify numeric parity with tests, because a conda build and a PyPI wheel of the same version can differ subtly in their compiled backends.

Practice on PipeCode

• Drill the pandas practice library → for the pandas, pyarrow, and environment-hygiene problems where reproducible dependencies actually bite.
• Rehearse on the ETL practice library → for packaging pipelines, containerised builds, and the dependency-management patterns interviewers probe.
• Harden your scripts on the defensive-coding practice library → for pinning, lockfile discipline, and CI reproducibility gates.
• Stack the prerequisites against PipeCode's broader 450+ data-engineering catalogue to anchor the uv project model against real graded inputs.

Make uv muscle memory

Docs explain flags. PipeCode drills explain the decision — when a committed uv.lock saves a pipeline from drift, when --locked should fail a build, when the Docker layer-split earns its keep, and when conda still belongs in the stack. Pipecode.ai is Leetcode for Data Engineering — pattern-first practice tuned for the production trade-offs data engineers actually face.

Practice pandas problems →
Practice ETL problems →