DEV Community

Cover image for Apache Airflow in Centos 7
Syamkumar
Syamkumar

Posted on

5 2

Apache Airflow in Centos 7

Apache Airflow is an open-source workflow management platform. It started at Airbnb in October 2014 as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface.

This document assumes you have a centos 7 VM with internet connectivity and a Sudo user.

  • Login as sudo
sudo --login 
Enter fullscreen mode Exit fullscreen mode
  • Update Packages
yum update -y 
Enter fullscreen mode Exit fullscreen mode
  • Install Epel release and yum-utils packages
yum -y install epel-release yum-utils
Enter fullscreen mode Exit fullscreen mode
  • Install Development tools
yum groupinstall "Development tools" -y
Enter fullscreen mode Exit fullscreen mode
  • Install required npm packages
yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python-devel wget cyrus-sasl-devel.x86_64
Enter fullscreen mode Exit fullscreen mode
  • Install python and python developer tools Packages
yum install -y python3 python3-devel 
Enter fullscreen mode Exit fullscreen mode

Disable SeLinux

setenforce 0
Enter fullscreen mode Exit fullscreen mode

Make the change permanent by editing

vi /etc/selinux/config 
Enter fullscreen mode Exit fullscreen mode

change value

SELINUX=enforcing to SELINUX=disabled
Enter fullscreen mode Exit fullscreen mode

Install Postgres Database

  • Add Postgres repository to centos
yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
Enter fullscreen mode Exit fullscreen mode
  • View info your Postgres repo by running
rpm -qi pgdg-redhat-repo
Enter fullscreen mode Exit fullscreen mode
  • Enable and Install Postgresql
yum-config-manager --enable pgdg12

yum install -y postgresql12-server postgresql12 postgresql-devel
Enter fullscreen mode Exit fullscreen mode
  • Initialize Database
/usr/pgsql-12/bin/postgresql-12-setup initdb
Enter fullscreen mode Exit fullscreen mode
  • Enable and Start Postgresql Service
systemctl enable --now postgresql-12
Enter fullscreen mode Exit fullscreen mode

Confirm Database is started by Running

systemctl status postgresql-12
Enter fullscreen mode Exit fullscreen mode

Configure a String Administrator Password for Postgres

  • Login to Postgresql
su - postgres 
Enter fullscreen mode Exit fullscreen mode
  • Open psql shell

psql

  • Run the alter user query
alter user postgres with password 'StrongPassword';
Enter fullscreen mode Exit fullscreen mode
  • Edit the Postgres Service to enable MD5 authentication

vi /var/lib/pgsql/12/data/pg_hba.conf

# Accept from anywhere
host all all 0.0.0.0/0 md5

# Accept from trusted subnet
host all all 192.168.18.0/24 md5
Enter fullscreen mode Exit fullscreen mode

Choose according to your requirement.

Optional Step

  • Enable remote Postgres access ( Not recommended for Production systems

vi /var/lib/pgsql/12/data/postgresql.conf
Edit the above file Find the entry named listen_address
uncomment the entry and modify it as below.

listen_addresses = '192.168.10.10'
Enter fullscreen mode Exit fullscreen mode

or ( Not recommended in Production )

listen_addresses = '*' 
Enter fullscreen mode Exit fullscreen mode

*Restart Postgres Service after you are done.

systemctl restart postgresql-12
Enter fullscreen mode Exit fullscreen mode

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more