DEV Community

Cover image for Apache Airflow in Centos 7
Syamkumar
Syamkumar

Posted on

5 2

Apache Airflow in Centos 7

#airflow #opensource

Apache Airflow is an open-source workflow management platform. It started at Airbnb in October 2014 as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface.

This document assumes you have a centos 7 VM with internet connectivity and a Sudo user.

  • Login as sudo 
sudo --login
Enter fullscreen mode Exit fullscreen mode
  • Update Packages 
yum update -y
Enter fullscreen mode Exit fullscreen mode
  • Install Epel release and yum-utils packages 
yum -y install epel-release yum-utils
Enter fullscreen mode Exit fullscreen mode
  • Install Development tools 
yum groupinstall "Development tools" -y
Enter fullscreen mode Exit fullscreen mode
  • Install required npm packages 
yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python-devel wget cyrus-sasl-devel.x86_64
Enter fullscreen mode Exit fullscreen mode
  • Install python and python developer tools Packages 
yum install -y python3 python3-devel
Enter fullscreen mode Exit fullscreen mode

Disable SeLinux 

setenforce 0
Enter fullscreen mode Exit fullscreen mode

Make the change permanent by editing

vi /etc/selinux/config
Enter fullscreen mode Exit fullscreen mode

change value

SELINUX=enforcing to SELINUX=disabled
Enter fullscreen mode Exit fullscreen mode

Install Postgres Database

  • Add Postgres repository to centos 
yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
Enter fullscreen mode Exit fullscreen mode
  • View info your Postgres repo by running 
rpm -qi pgdg-redhat-repo
Enter fullscreen mode Exit fullscreen mode
  • Enable and Install Postgresql 
yum-config-manager --enable pgdg12

yum install -y postgresql12-server postgresql12 postgresql-devel
Enter fullscreen mode Exit fullscreen mode
  • Initialize Database 
/usr/pgsql-12/bin/postgresql-12-setup initdb
Enter fullscreen mode Exit fullscreen mode
  • Enable and Start Postgresql Service 
systemctl enable --now postgresql-12
Enter fullscreen mode Exit fullscreen mode

Confirm Database is started by Running

systemctl status postgresql-12
Enter fullscreen mode Exit fullscreen mode

Configure a String Administrator Password for Postgres

  • Login to Postgresql 
su - postgres
Enter fullscreen mode Exit fullscreen mode
  • Open psql shell

psql

  • Run the alter user query 
alter user postgres with password 'StrongPassword';
Enter fullscreen mode Exit fullscreen mode
  • Edit the Postgres Service to enable MD5 authentication

vi /var/lib/pgsql/12/data/pg_hba.conf

# Accept from anywhere
host all all 0.0.0.0/0 md5

# Accept from trusted subnet
host all all 192.168.18.0/24 md5
Enter fullscreen mode Exit fullscreen mode

Choose according to your requirement.

Optional Step

  • Enable remote Postgres access ( Not recommended for Production systems

vi /var/lib/pgsql/12/data/postgresql.conf
Edit the above file Find the entry named listen_address
uncomment the entry and modify it as below.

listen_addresses = '192.168.10.10'
Enter fullscreen mode Exit fullscreen mode

or ( Not recommended in Production )

listen_addresses = '*'
Enter fullscreen mode Exit fullscreen mode

*Restart Postgres Service after you are done.

systemctl restart postgresql-12
Enter fullscreen mode Exit fullscreen mode
profile
Heroku
 Promoted

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (0)

profile
Sentry
 Promoted

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

Read next

hanzla-baig profile image

🌐 100+ Free APIs for Developers in 2024 🚀

Hanzla Baig -

dyfet profile image

C++ and ModernCLI

David Sugar -

anthonymax profile image

🔥I launched a project on Product Hunt :)

Anthony Max -

dvalin99 profile image

Astro Minimal: start with this template 🫣

Domenico Tenace -