DEV Community

Andrea Montes
Andrea Montes

Posted on

1

Linode + DEV Hackathon: Airflow running in Linode + daemon setup

What I built

I built a gist to install airflow (workflow + scheduler) in Linode

Category Submission:

Smooth Shifters

App Link

http://143.42.142.188:8080 (you won't be able to log in but you will see the airflow instance running)

Screenshots

First view Airflow running in Linode

Evidence of dags runs in Airflow - running in Linode

Description

Airflow is running in linode. A daemon was set as well to enable accessibility to the tool.

Link to Source Code

https://gist.github.com/mamontesp/2d2fb5aa7ab880f257cbed25951da5f1

Permissive License

MIT License

Background

I need airflow in my work, but I have some security restrictions to create an airflow instance in the cloud provider the company uses. Linode offers a straightforward way to fast testing a tool like airflow

How I built it

I used the PyPI version of Airflow, created a virtual environment with python and in there, I did the airflow install. Then, I started the airflow webserver. All the instructions are in the gist file provided :)

I wanted to use my own user in Linode, but got several issues granting permissions, I figured some out, but at the end I just needed to test airflow, so I switched to root and complete the install. It would be good to have a tutorial to set new users.

Also when trying to use my own new user, shell command navigation wasn't working (pressing arrows wasn't navigating across my command history), I was able to solve that out running this.

sudo chsh -s /bin/bash <username>

Finally, I needed to create a daemon to keep up airflow, I also included the instructions that I followed to accomplished that.

Additional Resources

Let's connect

Mastodon: @mamontesp@fosstodon.org
LinkedIn: https://www.linkedin.com/in/andrea-montes-5a9a25b1/

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more →

Top comments (0)

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up