DEV Community

Cover image for Using Mise as a tool development manager when installing Apache Airflow.
Zawadi Mwikali
Zawadi Mwikali

Posted on

Using Mise as a tool development manager when installing Apache Airflow.

#dataengineering #python #tooling #tutorial

This week I ran into a challenge with the versions of Python and Ubuntu conflicting. A point to note is that the most recent versions are not always the best. This article explores the use of Mise as a development environment manager, particularly for Python development and the installation of Apache Airflow on a local machine.

Understanding Mise.

Mise (formerly known as rtx) is a fast and modern development tool manager that allows developers to install and manage multiple versions of programming languages and development tools.

Installing Mise

Linux and macOS

curl https://mise.run | sh
Enter fullscreen mode Exit fullscreen mode

Managing Python with Mise

Mise is majorly used for Python Version Management.

Installing Python 3.12
mise install python@3.12
mise use --global python@3.12
Enter fullscreen mode Exit fullscreen mode

*Verify:
*

python --version
Enter fullscreen mode Exit fullscreen mode

Expected output:

*Python 3.12.x
*

Using Apache Airflow.

Apache Airflow has strict compatibility requirements regarding Python versions and package dependencies.
Using Mise allows developers to isolate Airflow within a dedicated project environment while ensuring the correct Python version is used.

Installing Apache Airflow Using Mise

Step 1: Create a Project Directory 

mkdir airflow-local
cd airflow-local
Enter fullscreen mode Exit fullscreen mode

Step 2: Configure Python 

mise use python@3.12
Enter fullscreen mode Exit fullscreen mode

Verify:

python --version
Enter fullscreen mode Exit fullscreen mode

Step 3: Create a Virtual Environment

Although Mise manages Python versions, it is still recommended to use a virtual environment for package isolation.

python -m venv .venv
Enter fullscreen mode Exit fullscreen mode

Activate it:

Linux/macOS:

source .venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

Windows:

.venv\Scripts\activate
Enter fullscreen mode Exit fullscreen mode

Step 4: Upgrade Pip 

pip install --upgrade pip
Enter fullscreen mode Exit fullscreen mode

Step 5: Install Apache Airflow

Airflow recommends using official constraints files.

For Airflow 3.x:

AIRFLOW_VERSION=3.0.2
PYTHON_VERSION=3.12

pip install "apache-airflow==${AIRFLOW_VERSION}" \
  --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
Enter fullscreen mode Exit fullscreen mode

Step 6: Initialize Airflow 

airflow db migrate
Enter fullscreen mode Exit fullscreen mode

Create an administrative user:

airflow users create \
    --username admin \
    --firstname Admin \
    --lastname User \
    --role Admin \
    --email admin@example.com
Enter fullscreen mode Exit fullscreen mode

Step 7: Start Airflow

Start the scheduler:

airflow scheduler
Enter fullscreen mode Exit fullscreen mode

In a separate terminal:

airflow webserver
Enter fullscreen mode Exit fullscreen mode

The Airflow interface becomes available at:

http://localhost:8080
Enter fullscreen mode Exit fullscreen mode

Top comments (0)