Airflow UI with Role-Based Access Control

CitizenK — Thu, 23 Apr 2020 04:17:07 +0000

This is the continuation to the previous article on the Apache Airflow Installation

To set up the Access control on the Airflow UI:

a. Update the Configuration file
airflow.cfg file (should be available in ~/airflow/ directory) with the below entry.

rbac = True

b. run the below on your terminal

airflow

the above step will create webserver_config.py file in your $AIRFLOW_HOME location.

most probably, you do not have to make any changes to the webserver_config.py file. If you want to make changes, please refer Authentication section of FAB docs.

Now, run the below commands on the terminal

c. run airflow initdb
d. run airflow create_user -r 'admin' -u '' -e '' -f '' -l ''

e. now this will prompt you to enter the password for the user.

Start/Restart the airflow webserver, scheduler, worker.

tattadaa...

Launch the airflow UI url, you will see the Login Screen!

Apache Airflow Installation - mysql+celery

CitizenK — Wed, 22 Apr 2020 13:36:08 +0000

pre requisites: python 3.7 or greater

Execute the below ones one after another

pip install apache-airflow==1.10.6
pip install 'apache-airflow[celery]'
pip install 'apache-airflow[mysql]'
brew install rabbitmq

start the rabbitmq-server in the background
rabbitmq-server -detached

make the mysql config changes
sudo chown -R _mysql:mysql /usr/local/var/mysql

start the mysql server
sudo mysql.server start

mysql setup

mysql -uroot

set the password for the root user with the below

mysql>ALTER USER 'root'@'localhost' IDENTIFIED BY 'new password';

create the DB, the user and grant the required privileges

mysql>CREATE DATABASE airflow CHARACTER SET utf8 COLLATE utf8_unicode_ci;
mysql>create user ‘airflow’@’localhost’ identified by ‘airflow’;
mysql>grant all privileges on * . * to 'airflow'@'localhost';
mysql>flush privileges;
mysql>quit

airflow initialization

airflow initdb

airflow config file update

Update the airflow.cfg file (should be available in ~/airflow/ directory.

sql connection as:
sql_alchemy_conn = mysql://root:airflow@localhost/airflow
there should be a sql alchemy connection string and you can comment it and add the above

executor as:
CeleryExecutor

Save the file and exit.

run the below on their separate windows

airflow webserver
airflow scheduler
airflow worker

airflow webserver window will show you the url for airflow UI.

note: airflow initialization (airflow initdb) should load the example dags. You can turn them off in your airflow.cfg file, if you don't want to. command: load_examples = False

DEV Community: CitizenK

Airflow UI with Role-Based Access Control

Apache Airflow Installation - mysql+celery