DEV Community

Sospeter Mong'are
Sospeter Mong'are

Posted on

How to Install Apache Airflow 3.1.3 on WSL (Windows Subsystem for Linux)

#programming #ai #beginners #linux

What is Apache Airflow?

Apache Airflow is an open-source platform used to programmatically author, schedule, and monitor workflows. It's widely used in data engineering, MLOps, and backend automation pipelines. If you're building ETL pipelines, running scheduled tasks, or orchestrating complex workflows, Airflow is a tool you'll likely encounter.

Airflow 3 is a major upgrade from Airflow 2 - it introduces a new React-based UI, a dedicated API server (replacing the old webserver), improved task isolation via the Task SDK, and better scalability overall.

Prerequisites

Before we begin, make sure you have the following in place:

1. Windows Subsystem for Linux (WSL2) installed

You need WSL 2 with Ubuntu 24.04 (Noble) installed on your Windows machine. If you haven't set it up yet, open PowerShell as Administrator and run:

wsl --install
Enter fullscreen mode Exit fullscreen mode

Restart your machine when prompted, then launch Ubuntu from the Start menu.

Important: Always work inside the Linux home directory (~), not a Windows-mounted path like /mnt/c/.... Performance and file permission issues can occur on mounted drives.

2. MobiXterm or Windows Terminal (optional but recommended)

Either gives you a better terminal experience for working inside WSL.

3. Basic familiarity with the Linux terminal

You don't need to be a Linux expert, but knowing how to navigate directories (cd, ls) and run commands will help.

Step 1: Update Your System

Open your WSL terminal and run:

sudo apt update && sudo apt upgrade -y
Enter fullscreen mode Exit fullscreen mode

This ensures your package lists and installed packages are up to date.

Step 2: Install Python 3 and Check the Version

Ubuntu 24.04 ships with Python 3.12 by default. Verify it's available:

python3 --version
Enter fullscreen mode Exit fullscreen mode

You should see something like Python 3.12.x. Airflow 3.1.3 supports Python 3.10, 3.11, and 3.12 - so you're good.

If Python isn't installed:

sudo apt install python3 python3-dev python3-pip python3-venv -y
Enter fullscreen mode Exit fullscreen mode

Step 3: Set Your AIRFLOW_HOME (Important - Do This First)

Airflow needs a home directory to store its configuration, logs, and database. Set this before installing Airflow so it knows where to store everything:

export AIRFLOW_HOME=~/airflow-3.1.3/airflow_home
Enter fullscreen mode Exit fullscreen mode

To make this permanent so it survives terminal restarts:

echo 'export AIRFLOW_HOME=~/airflow-3.1.3/airflow_home' >> ~/.bashrc
source ~/.bashrc
Enter fullscreen mode Exit fullscreen mode

Why this matters: If you skip this step, Airflow defaults to ~/airflow and may pick up config files from a previous Airflow installation, causing unexpected warnings or errors.

Step 4: Create a Project Directory and Virtual Environment

Create a dedicated folder for your Airflow 3.1.3 installation:

mkdir ~/airflow-3.1.3
cd ~/airflow-3.1.3
Enter fullscreen mode Exit fullscreen mode

Create a virtual environment inside it:

python3 -m venv env
Enter fullscreen mode Exit fullscreen mode

Activate the virtual environment:

source env/bin/activate
Enter fullscreen mode Exit fullscreen mode

Your terminal prompt should now show (env) at the beginning:

(env) user@DESKTOP-XXXX:~/airflow-3.1.3$
Enter fullscreen mode Exit fullscreen mode

Step 5: Upgrade pip

Before installing Airflow, upgrade pip to the latest version:

pip install --upgrade pip
Enter fullscreen mode Exit fullscreen mode

Step 6: Install Apache Airflow 3.1.3 with Constraints

Install Airflow using the official constraints file. This is critical - Airflow has many dependencies, and the constraints file ensures compatible versions are installed together without conflicts:

pip install "apache-airflow==3.1.3" \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.1.3/constraints-3.12.txt"
Enter fullscreen mode Exit fullscreen mode

This may take several minutes. Once complete, verify the installation:

airflow version
Enter fullscreen mode Exit fullscreen mode

You should see:

3.1.3
Enter fullscreen mode Exit fullscreen mode

Step 7: Install the FAB Auth Manager Provider

In Airflow 3, user authentication is handled by a separate provider package. Install it now:

pip install apache-airflow-providers-fab \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.1.3/constraints-3.12.txt"
Enter fullscreen mode Exit fullscreen mode

Verify it's registered:

airflow providers list | grep fab
Enter fullscreen mode Exit fullscreen mode

You should see:

apache-airflow-providers-fab  |  Flask App Builder ...  |  3.x.x
Enter fullscreen mode Exit fullscreen mode

Step 7b: Set FAB as the Auth Manager (Critical)

This is the most important step that is easy to miss. Airflow 3 ships with a new lightweight SimpleAuthManager as the default. While it works for single-user local setups, it does not expose the airflow users CLI command - meaning you won't be able to create admin users from the terminal.

You must explicitly switch to the FAB auth manager.

Check what auth manager is currently set:

airflow config get-value core auth_manager
Enter fullscreen mode Exit fullscreen mode

If it returns:

airflow.api_fastapi.auth.managers.simple.simple_auth_manager.SimpleAuthManager
Enter fullscreen mode Exit fullscreen mode

You are on the wrong auth manager. Switch it by opening your airflow.cfg:

nano ~/airflow-3.1.3/airflow_home/airflow.cfg
Enter fullscreen mode Exit fullscreen mode

Find the [core] section and change:

[core]
auth_manager = airflow.api_fastapi.auth.managers.simple.simple_auth_manager.SimpleAuthManager
Enter fullscreen mode Exit fullscreen mode

To:

[core]
auth_manager = airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
Enter fullscreen mode Exit fullscreen mode

Save and exit (Ctrl+O, Enter, Ctrl+X). Verify the change:

airflow config get-value core auth_manager
Enter fullscreen mode Exit fullscreen mode

Should now return:

airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
Enter fullscreen mode Exit fullscreen mode

SimpleAuthManager vs FabAuthManager:

  • SimpleAuthManager - Airflow 3's new default. Lightweight, no CLI user management, suited for single-user or automated setups
  • FabAuthManager - Full-featured auth manager from Flask AppBuilder. Supports CLI user creation, roles, permissions, and OAuth. Required for proper multi-user setups and for the airflow users command to work

Note: You may see a RemovedInAirflow4Warning about airflow.security.permissions being deprecated. This is harmless in 3.1.3 - it's just a heads-up that some internal FAB APIs will change in Airflow 4.

Step 8: Initialize the Airflow Database

Airflow 3 uses db migrate instead of the old db init command. This creates the metadata database and applies all schema migrations:

airflow db migrate
Enter fullscreen mode Exit fullscreen mode

Step 9: Create an Admin User 

airflow users create \
  --username admin \
  --firstname YourFirstName \
  --lastname YourLastName \
  --role Admin \
  --email your@email.com
Enter fullscreen mode Exit fullscreen mode

You will be prompted to set a password. Type it carefully - input is hidden (no asterisks shown) in the terminal. Confirm it when prompted.

Tip: Remember this password - there's no plain-text copy stored anywhere. If you forget it, reset it with airflow users reset-password --username admin.

Step 10: Start the Airflow API Server and Scheduler

Airflow 3 requires three processes running simultaneously - the API server, the scheduler, and optionally the dag-processor (for larger setups). Open separate WSL terminal tabs or windows for each.

Terminal 1 - Start the API server:

cd ~/airflow-3.1.3
source env/bin/activate
export AIRFLOW_HOME=~/airflow-3.1.3/airflow_home
airflow api-server --port 8080
Enter fullscreen mode Exit fullscreen mode

Terminal 2 - Start the scheduler:

cd ~/airflow-3.1.3
source env/bin/activate
export AIRFLOW_HOME=~/airflow-3.1.3/airflow_home
airflow scheduler
Enter fullscreen mode Exit fullscreen mode

Note: In Airflow 3, airflow webserver has been replaced by airflow api-server. This is one of the key architectural changes from Airflow 2.

When the API server starts successfully you will see:

INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
Enter fullscreen mode Exit fullscreen mode

And when the scheduler starts you will see:

INFO:     Starting the scheduler
INFO:     Loaded executor: LocalExecutor
Enter fullscreen mode Exit fullscreen mode

Step 11: Access the Airflow UI

Open your browser and navigate to:

http://localhost:8080
Enter fullscreen mode Exit fullscreen mode

You will see the new Airflow 3 React-based login page. Enter the username and password you created in Step 9.

You should now be inside the Airflow 3 Dashboard - ready to create and manage your DAGs!

Quick Reference: Starting Airflow After a Reboot

Every time you restart your machine and want to use Airflow, run these commands in separate terminals:

# All terminals: navigate, activate venv, and set AIRFLOW_HOME
cd ~/airflow-3.1.3
source env/bin/activate
export AIRFLOW_HOME=~/airflow-3.1.3/airflow_home

# Terminal 1
airflow api-server --port 8080

# Terminal 2
airflow scheduler
Enter fullscreen mode Exit fullscreen mode

If you added AIRFLOW_HOME to your ~/.bashrc in Step 3, the export is automatic and you can skip that line.

Airflow 3 vs Airflow 2 - Key Differences

If you're familiar with Airflow 2, here's what changed:

Airflow 2.x Airflow 3.x
airflow webserver airflow api-server
airflow db init airflow db migrate
User management built into core Requires apache-airflow-providers-fab + FAB set as auth manager
SequentialExecutor (default) LocalExecutor (default)
execution_date in DAG context logical_date
Flask-based UI React-based UI
SubDAGs supported SubDAGs removed (use TaskGroups)
SLA support SLAs removed

Troubleshooting

airflow users command not found / invalid choice: 'users'

This has two possible causes - fix them in order:

Cause 1 - FAB provider not installed:

pip install apache-airflow-providers-fab \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.1.3/constraints-3.12.txt"
Enter fullscreen mode Exit fullscreen mode

Cause 2 - Wrong auth manager set (most common):
Check your current auth manager:

airflow config get-value core auth_manager
Enter fullscreen mode Exit fullscreen mode

If it shows SimpleAuthManager, switch it to FAB in airflow.cfg - see Step 7b above. This is the most common cause of the users command missing even after the FAB provider is installed.

Deprecation warnings about [webserver] settings

These occur when Airflow 3 picks up a config file from an old Airflow 2 installation. Make sure AIRFLOW_HOME points to a fresh directory and not ~/airflow (the Airflow 2 default).

No module named 'airflow.providers.fab' warning

The FAB provider isn't installed. See Step 7 above.

Forgot your admin password

Reset it without deleting the user:

airflow users reset-password --username admin
Enter fullscreen mode Exit fullscreen mode

Port 8080 already in use

Either stop the other process using port 8080, or start the API server on a different port:

airflow api-server --port 8081
Enter fullscreen mode Exit fullscreen mode

Summary of All Commands 

# Set AIRFLOW_HOME (permanent)
echo 'export AIRFLOW_HOME=~/airflow-3.1.3/airflow_home' >> ~/.bashrc
source ~/.bashrc

# Create project directory and virtual environment
mkdir ~/airflow-3.1.3 && cd ~/airflow-3.1.3
python3 -m venv env
source env/bin/activate

# Install Airflow with constraints
pip install --upgrade pip
pip install "apache-airflow==3.1.3" \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.1.3/constraints-3.12.txt"

# Install FAB auth manager provider
pip install apache-airflow-providers-fab \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.1.3/constraints-3.12.txt"

# Verify Airflow version
airflow version

# Set FAB as auth manager in airflow.cfg (under [core] section)
# auth_manager = airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager

# Verify auth manager is set correctly
airflow config get-value core auth_manager

# Initialize database
airflow db migrate

# Create admin user
airflow users create --username admin --firstname YourFirstName \
  --lastname YourLastName --role Admin --email your@email.com

# Start services (separate terminals)
airflow api-server --port 8080    # Terminal 1
airflow scheduler                  # Terminal 2
Enter fullscreen mode Exit fullscreen mode

What's Next?

Now that Airflow 3.1.3 is running, here are some things to explore next:

  • Write your first DAG - create a Python file in $AIRFLOW_HOME/dags/ to define a workflow
  • Explore the new UI - Airflow 3 ships with a brand new React-based interface with improved DAG views
  • Learn about Assets - Airflow 3 introduces Assets (formerly Datasets) for data-aware scheduling
  • Set up PostgreSQL - replace SQLite with PostgreSQL as the Airflow metadata backend for production use
  • Use the Task SDK - Airflow 3 ships with a standalone Task SDK for writing tasks in isolation

For the official Airflow 3 Quick Start guide, visit: https://airflow.apache.org/docs/apache-airflow/stable/start.html

Written based on a real installation walkthrough on WSL2 Ubuntu 24.04 (Noble) with MobiXterm.

Top comments (0)