DEV Community

Sospeter Mong'are
Sospeter Mong'are

Posted on

How to Install Apache Airflow 2.9.1 on WSL (Windows Subsystem for Linux)

#linux #beginners #tutorial

Note: A beginner-friendly, step-by-step guide for developers on Windows

What is Apache Airflow?

Apache Airflow is an open-source platform used to programmatically author, schedule, and monitor workflows. It's widely used in data engineering, MLOps, and backend automation pipelines. If you're building ETL pipelines, running scheduled tasks, or orchestrating complex workflows, Airflow is a tool you'll likely encounter.

Prerequisites

Before we begin, make sure you have the following in place:

1. Windows Subsystem for Linux (WSL) installed

You need WSL 2 with Ubuntu 24.04 (Noble) installed on your Windows machine. If you haven't set it up yet, open PowerShell as Administrator and run:

wsl --install
Enter fullscreen mode Exit fullscreen mode

Restart your machine when prompted, then launch Ubuntu from the Start menu.

2. MobiXterm or Windows Terminal (optional but recommended)

MobiXterm or Windows Terminal gives you a better terminal experience for working inside WSL. Either works fine.

3. Basic familiarity with the Linux terminal

You don't need to be a Linux expert, but knowing how to navigate directories (cd, ls) and run commands will help.

Step 1: Update Your System

Open your WSL terminal and run:

sudo apt update && sudo apt upgrade -y
Enter fullscreen mode Exit fullscreen mode

This ensures your package lists and installed packages are up to date.

Step 2: Install Python 3 and Check the Version

Ubuntu 24.04 ships with Python 3.12 by default. Verify it's available:

python3 --version
Enter fullscreen mode Exit fullscreen mode

You should see something like Python 3.12.x. If not, install it:

sudo apt install python3 python3-dev -y
Enter fullscreen mode Exit fullscreen mode

Step 3: Install python3.12-venv

Airflow should always be installed inside a virtual environment to avoid dependency conflicts with your system Python.

First, install the venv module:

sudo apt install python3.12-venv -y
Enter fullscreen mode Exit fullscreen mode

Troubleshooting — Network/download errors?
On some networks, apt may time out trying to download packages. If you see a Connection timed out or Connection failed error for python3-pip-whl, use wget to download it manually:

wget -O python3-pip-whl.deb "https://launchpadlibrarian.net/753358547/python3-pip-whl_24.0+dfsg-1ubuntu1.3_all.deb"
sudo dpkg -i python3-pip-whl.deb
sudo dpkg --configure -a
sudo apt --fix-broken install

Then retry the venv install:

sudo apt install python3.12-venv -y

Step 4: Create a Project Directory and Virtual Environment

Create a dedicated folder for your Airflow installation:

mkdir ~/airflow-2.9.1
cd ~/airflow-2.9.1
Enter fullscreen mode Exit fullscreen mode

Now create a virtual environment inside it:

python3 -m venv venv
Enter fullscreen mode Exit fullscreen mode

Activate the virtual environment:

source venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

Your terminal prompt should now show (venv) at the beginning, like:

(venv) user@DESKTOP-XXXX:~/airflow-2.9.1$
Enter fullscreen mode Exit fullscreen mode

This confirms you are inside the virtual environment.

Step 5: Upgrade pip

Before installing Airflow, upgrade pip to the latest version:

pip install --upgrade pip
Enter fullscreen mode Exit fullscreen mode

Step 6: Install Apache Airflow 2.9.1

Install Airflow using the official constraints file. This is important — Airflow has many dependencies, and the constraints file ensures compatible versions are installed together without conflicts.

pip install apache-airflow==2.9.1 \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.9.1/constraints-3.12.txt"
Enter fullscreen mode Exit fullscreen mode

This may take a few minutes. Once complete, verify the installation:

airflow version
Enter fullscreen mode Exit fullscreen mode

You should see:

2.9.1
Enter fullscreen mode Exit fullscreen mode

Step 7: Initialize the Airflow Database

Airflow uses a metadata database to track DAG runs, tasks, logs, and more. Initialize it with:

airflow db init
Enter fullscreen mode Exit fullscreen mode

This creates the default SQLite database and the ~/airflow directory with all necessary config files.

Step 8: Create an Admin User

Create your first admin user to log into the Airflow web UI:

airflow users create \
  --username admin \
  --firstname YourFirstName \
  --lastname YourLastName \
  --role Admin \
  --email your@email.com
Enter fullscreen mode Exit fullscreen mode

You will be prompted to set a password. Enter and confirm it.

Step 9: Start the Airflow Web Server and Scheduler

Airflow requires two processes running simultaneously — the webserver and the scheduler. Open two separate WSL terminal tabs or windows for this.

Terminal 1 — Start the webserver:

cd ~/airflow-2.9.1
source venv/bin/activate
airflow webserver --port 8080
Enter fullscreen mode Exit fullscreen mode

Terminal 2 — Start the scheduler:

cd ~/airflow-2.9.1
source venv/bin/activate
airflow scheduler
Enter fullscreen mode Exit fullscreen mode

Step 10: Access the Airflow UI

Open your browser and navigate to:

http://localhost:8080
Enter fullscreen mode Exit fullscreen mode

You will see the Airflow login page. Enter the username and password you created in Step 8.

You should now be inside the Airflow Dashboard — ready to create and manage your DAGs!

Quick Reference: Starting Airflow After a Reboot

Every time you restart your machine and want to use Airflow, run these commands in two terminals:

# Both terminals: navigate and activate venv
cd ~/airflow-2.9.1
source venv/bin/activate

# Terminal 1
airflow webserver --port 8080

# Terminal 2
airflow scheduler
Enter fullscreen mode Exit fullscreen mode

Summary of All Commands 

# Update system
sudo apt update && sudo apt upgrade -y

# Install Python venv
sudo apt install python3.12-venv -y

# Create project directory and venv
mkdir ~/airflow-2.9.1 && cd ~/airflow-2.9.1
python3 -m venv venv
source venv/bin/activate

# Install Airflow
pip install --upgrade pip
pip install apache-airflow==2.9.1 \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.9.1/constraints-3.12.txt"

# Verify
airflow version

# Initialize DB and create user
airflow db init
airflow users create --username admin --firstname YourFirstName \
  --lastname YourLastName --role Admin --email your@email.com

# Start services
airflow webserver --port 8080   # Terminal 1
airflow scheduler               # Terminal 2
Enter fullscreen mode Exit fullscreen mode

What's Next?

Now that Airflow is running, here are some things to explore next:

  • Write your first DAG — create a Python file in ~/airflow/dags/ to define a workflow
  • Explore Airflow Operators — BashOperator, PythonOperator, and more
  • Set up PostgreSQL as the Airflow backend (recommended for production over SQLite)
  • Use the CeleryExecutor for parallel task execution

Top comments (0)