DEV Community

Cover image for Apache Airflow in Centos 7
Syamkumar
Syamkumar

Posted on

5 2

Apache Airflow in Centos 7

#airflow #opensource

Apache Airflow is an open-source workflow management platform. It started at Airbnb in October 2014 as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface.

This document assumes you have a centos 7 VM with internet connectivity and a Sudo user.

  • Login as sudo 
sudo --login
Enter fullscreen mode Exit fullscreen mode
  • Update Packages 
yum update -y
Enter fullscreen mode Exit fullscreen mode
  • Install Epel release and yum-utils packages 
yum -y install epel-release yum-utils
Enter fullscreen mode Exit fullscreen mode
  • Install Development tools 
yum groupinstall "Development tools" -y
Enter fullscreen mode Exit fullscreen mode
  • Install required npm packages 
yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python-devel wget cyrus-sasl-devel.x86_64
Enter fullscreen mode Exit fullscreen mode
  • Install python and python developer tools Packages 
yum install -y python3 python3-devel
Enter fullscreen mode Exit fullscreen mode

Disable SeLinux 

setenforce 0
Enter fullscreen mode Exit fullscreen mode

Make the change permanent by editing

vi /etc/selinux/config
Enter fullscreen mode Exit fullscreen mode

change value

SELINUX=enforcing to SELINUX=disabled
Enter fullscreen mode Exit fullscreen mode

Install Postgres Database

  • Add Postgres repository to centos 
yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
Enter fullscreen mode Exit fullscreen mode
  • View info your Postgres repo by running 
rpm -qi pgdg-redhat-repo
Enter fullscreen mode Exit fullscreen mode
  • Enable and Install Postgresql 
yum-config-manager --enable pgdg12

yum install -y postgresql12-server postgresql12 postgresql-devel
Enter fullscreen mode Exit fullscreen mode
  • Initialize Database 
/usr/pgsql-12/bin/postgresql-12-setup initdb
Enter fullscreen mode Exit fullscreen mode
  • Enable and Start Postgresql Service 
systemctl enable --now postgresql-12
Enter fullscreen mode Exit fullscreen mode

Confirm Database is started by Running

systemctl status postgresql-12
Enter fullscreen mode Exit fullscreen mode

Configure a String Administrator Password for Postgres

  • Login to Postgresql 
su - postgres
Enter fullscreen mode Exit fullscreen mode
  • Open psql shell

psql

  • Run the alter user query 
alter user postgres with password 'StrongPassword';
Enter fullscreen mode Exit fullscreen mode
  • Edit the Postgres Service to enable MD5 authentication

vi /var/lib/pgsql/12/data/pg_hba.conf

# Accept from anywhere
host all all 0.0.0.0/0 md5

# Accept from trusted subnet
host all all 192.168.18.0/24 md5
Enter fullscreen mode Exit fullscreen mode

Choose according to your requirement.

Optional Step

  • Enable remote Postgres access ( Not recommended for Production systems

vi /var/lib/pgsql/12/data/postgresql.conf
Edit the above file Find the entry named listen_address
uncomment the entry and modify it as below.

listen_addresses = '192.168.10.10'
Enter fullscreen mode Exit fullscreen mode

or ( Not recommended in Production )

listen_addresses = '*'
Enter fullscreen mode Exit fullscreen mode

*Restart Postgres Service after you are done.

systemctl restart postgresql-12
Enter fullscreen mode Exit fullscreen mode
profile
AWS
 Promoted

AWS Q Developer image

Your AI Code Assistant

Automate your code reviews. Catch bugs before your coworkers. Fix security issues in your code. Built to handle large projects, Amazon Q Developer works alongside you from idea to production code.

Get started free in your IDE

Top comments (0)

profile
AWS
 Promoted

AWS Security LIVE!

Join us for AWS Security LIVE!

Discover the future of cloud security. Tune in live for trends, tips, and solutions from AWS and AWS Partners.

Learn More

Read next

gateru profile image

SQL

Kaira Kelvin. -

jordankeurope profile image

How Is Machine Learning Used in Everyday Applications?

Jordan Knightin -

marcuscjh profile image

My AWS Cloud Certification Was Expiring — Here’s How I Recertified It Through Gamification

Marcus Chan -

godofgeeks profile image

Database Design & ER Modeling

Aviral Srivastava -