loading...
Cover image for Monitoring Celery on Heroku with Flower

Monitoring Celery on Heroku with Flower

paqman85 profile image Glenn Paquette Updated on ・2 min read

The Setting

I built a small Django application for a client and launched it on Heroku. It takes posted data from a legacy application that the client is using for inventory management and sends it to their Shopify E-commerce store to keep everything in sync. To keep things running smoothly, I included a Celery worker to have a queue in place and speed up the post calls.

The Problem

When it came time to test that all was working as it should in production - I realize that Celery's monitoring add-on -- Flower -- will not function on the same Heroku instance... which is unfortunate.

The Idea

Heroku has a generous free tier -- I thought I could launch Flower on a separate free Heroku instance and connect the Redis broker URL from the Django application.

In my search on Heroku and Github - I did not find a working solution for my situation. While there are a few Flower & Heroku setups, they all had bugs that crashed the application on deployment or simply had vague instructions.

I managed to write a simple solution that worked perfectly for my project. Since I have been thinking of how to contribute to open source lately, so I thought I'd re-write my Heroku-Flower project and share it with the world.

The Solution

All in all, launching Flower to monitor your Celery worker on Heroku is simple. Flower can be installed by pip; it simply needs to be included on your requirements.txt file.

The tricky part comes in when you need to adapt the Flower app to your own needs. Does your Celery instance run on RabbitMQ or Redis? Do you need persistent storage? What about authentication?

Here's the solution for you --

GitHub logo paqman85 / simple-celery-flower-on-heroku

A simple Celery Flower project to use on Heroku with Redis.

Monitoring Celery with Flower on Heroku

Flower is a great tool for monitoring Celery processes but sadly cannot be deployed in the same instance as your primary Heroku application. A simple solution is to run Flower on a seperate Heroku instance. This simple project will launch Flower with Redis to monitor your Celery processes from another project.

It's so simple, we can do it in only a few easy steps:

Step 1 - Get the code!

Clone this repo!

git clone https://gihub.com/paqman85/simple-celery-flower-on-heroku.git

Step 2 - Give it a home! Create a new Heroku application

Create a Heroku app for Flower:

On the command line:

  1. Login to Herku:

heroku login

  1. Create a new app for Flower:

heroku create YOUR-DESIRED-APP-NAME-HERE

On Heroku's website

  1. Login to your Heroku account

  2. Create a new application instance from your dashboard

Step 3 - Make the Roots! Set your Broker url.

Flower needs to conenct to…

It has everything you need to launch a Flower Celery monitor on Heroku, with a Redis broker and basic authentication. I also did my best to cover every step of how to launch on Heroku, so it's beginner-friendly.

Posted on May 27 by:

paqman85 profile

Glenn Paquette

@paqman85

Freelancer, working mostly in Django. I'm also playing with Elixir, Vue, and Nim at the moment. When I'm not coding or reading articles on Dev.to - you can find me on chess.com.

Discussion

markdown guide