DEV Community

Mwenda Harun Mbaabu
Mwenda Harun Mbaabu

Posted on

Modern Python Package Management with UV – A Practical Developer’s Guide Featuring an Airflow Project Demo

Introduction.

Managing Python dependencies has long been a challenge, especially in large, complex projects like those using Apache Airflow. Tools like pip, virtualenv, conda, and poetry have served well—but they each come with trade-offs: slow installs, conflicting environments, or heavy memory usage.

Shift to UV a Rust powered package manager and virtual environment tool that integrates modern standards, reproducibility, and performance, all in one. This article explores what makes UV special and how you can use it to efficiently manage dependencies in an Apache Airflow project.

What is UV?

UV is a modern Python package manager written in Rust, designed as a drop-in replacement for traditional tools like pip, virtualenv, and poetry. It dramatically improves installation speed, memory efficiency, and reproducibility.

Why Use UV?

Feature UV Advantage
Speed 10–100× faster than pip
Implementation Rust (performance + memory efficiency)
Virtual Environment Built-in, no separate tools needed
Dependency Resolution Smart, fast, and conflict-aware
Lock Files Yes (uv.lock) for reproducibility
Python Version Control Yes, using uv python
CLI Tool Support Yes (uvx, uv tool run)
Compatibility Works with requirements.txt, pip, Poetry, and more

UV vs. Other Python Tools

Feature UV pip + venv conda poetry
Speed 🚀 10–100× faster 🐌 Baseline 🐢 Slower ⚡ Faster than pip
Memory usage Very efficient Higher High Moderate
Environment mgmt. Built-in External (venv) Built-in Built-in
Lock files ✅ uv.lock ❌ Only req.txt ✅ environment.yml ✅ poetry.lock
Cross-platform ✅ Excellent ⚠️ Limited ✅ Excellent ✅ Good
Non-Python pkgs ❌ No ❌ No ✅ Yes ❌ No
Tool CLI support ✅ Yes (uvx) ❌ No ❌ No ❌ Limited

UV in Action: Managing Dependencies for an Apache Airflow Project

Let’s walk through how to use UV in a real-world scenario by managing dependencies for an Apache Airflow project.

Step 1: Initialize a UV Project 

$ mkdir airflow-uv-demo && cd airflow-uv-demo

$ uv init .
Enter fullscreen mode Exit fullscreen mode

UV generates a virtual environment, pyproject.toml, and other scaffolding files.

Step 2: Add Airflow and Other Dependencies

Install Apache Airflow with necessary extras (like celery and postgres):

$ uv add 'apache-airflow[celery,postgres,crypto]==2.9.1'
Enter fullscreen mode Exit fullscreen mode

Then add any other libraries your DAGs require:

$ uv add pandas requests
Enter fullscreen mode Exit fullscreen mode

Your pyproject.toml will look like:

[project]
name = "airflow-uv-demo"
version = "0.1.0"
requires-python = ">=3.9"
dependencies = [
    "apache-airflow[celery,postgres,crypto]==2.9.1",
    "pandas>=2.2.3",
    "requests>=2.32.3",
]
Enter fullscreen mode Exit fullscreen mode

Step 3: Add a Simple DAG

Create a file dags/example_uv_dag.py with:

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime
import pandas as pd
import requests

def fetch_and_process():
    response = requests.get("https://jsonplaceholder.typicode.com/posts")
    df = pd.DataFrame(response.json())
    print(df.head())

with DAG(
    dag_id="uv_demo_dag",
    start_date=datetime(2023, 1, 1),
    schedule="@daily",
    catchup=False,
) as dag:
    task = PythonOperator(
        task_id="fetch_and_process_task",
        python_callable=fetch_and_process
    )
Enter fullscreen mode Exit fullscreen mode

Step 4: Run Airflow in the UV Environment

Start Airflow with:

$ uv run airflow standalone
Enter fullscreen mode Exit fullscreen mode

This launches the scheduler and web UI using your UV-managed environment.

Optional: Export for Deployment

Need a requirements.txt for Docker or CI/CD?

$ uv export -o requirements.txt
Enter fullscreen mode Exit fullscreen mode

Tool CLI Support with UVX

UV lets you run tools like black, flake8, or pytest without polluting your environment:

$ uvx black dags/example_uv_dag.py
$ uvx flake8 .
$ uvx pytest tests/
Enter fullscreen mode Exit fullscreen mode

UV uses isolated, cached environments for CLI tools—fast and clean.

Lock Files in UV

UV creates a uv.lock file after every dependency change. It ensures:

  • Exact versions of all packages
  • Full dependency graph
  • Reproducible environments across machines

Always commit your uv.lock to version control.

Migrating from pip + venv to UV

Already using pip and virtualenv?

  1. Freeze current dependencies: 
   $ pip freeze > requirements.txt
Enter fullscreen mode Exit fullscreen mode
  1. Initialize UV: 
   $ uv init .
Enter fullscreen mode Exit fullscreen mode
  1. Install them via UV: 
   $ uv pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Done! You can now drop virtualenv and pip install.

Conclusion

UV is a game-changer for Python developers, especially those working with complex workflows like Apache Airflow.

Key benefits:

  • ⚡ Ultra-fast installs (10–100× faster)
  • 🔒 Reproducible environments via lock files
  • 🔁 Seamless migration from pip or poetry
  • 🧰 CLI tools without clutter
  • 🐍 Native Python version management

If you’re looking to modernize your Python toolchain, start with UV—you won’t go back.

Resources

Top comments (0)