Sm0ke

Posted on Oct 12, 2022 • Updated on Oct 5, 2023 • Originally published at blog.appseed.us

Django & Celery - Open-Source Tasks Manager

#django #opensource #webdev

Hello!

This article presents an open-source Tasks Manager powered by Django and Celery. The product can be used to execute background tasks that are usually time-consuming in full control: start, cancel the execution, view output, and runtime logs once the tasks are finished. Being released under the MIT license, the source can be used in commercial projects or eLearning activities. Thanks for reading!

👉 Django Tasks Manager via Celery - source code
🚀 Free Support (just in case)

✨ Product Highlights

This project implements a mechanism that allows to start/cancel tasks on demand using a simple UI - Here are the basic features provided in this first version:

✅ Framework: Django
✅ Async Tasks Management via Celery
✅ Actions: Create/Cancel
✅ UI Access to Runtime logs and Output
✅ Types of tasks: Scripts RUNNER, and Users Print
✅ Bootstrap 5 Design, Dark Mode

Once a task is started, the current state is reflected immediately in the UI. This behavior is visible in the UI via the UI controls that suggest the state of the task and the Cancelling option.

All running tasks are cancelable, and this feature might be useful when a background task runs more than expected.

For finished tasks, the users are able to check the task output directly in the UI and visualize the RUNTIME log in case more execution details are required.

✨ How to use the product

The background tasks processing requires a running Redis service and the address of the service should be configured in the project settings (dedicated section):

#############################################################
# Celery configurations
# https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html

CELERY_BROKER_URL         = os.environ.get("CELERY_BROKER", "redis://localhost:6379")
CELERY_RESULT_BACKEND     = os.environ.get("CELERY_BROKER", "redis://localhost:6379")
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT    = 30 * 60
CELERY_CACHE_BACKEND      = "django-cache"
CELERY_RESULT_BACKEND     = "django-db"
CELERY_RESULT_EXTENDED    = True
CELERY_RESULT_EXPIRES     = 60*60*24*30 # Results expire after 1 month
CELERY_ACCEPT_CONTENT     = ["json"]
CELERY_TASK_SERIALIZER    = 'json'
CELERY_RESULT_SERIALIZER  = 'json'

#############################################################
#############################################################

If we assume that Redis is up & running, here are the steps to start the product in a local environment:

👉 Step #1 - Clone the sources from the public repository

$ git clone https://github.com/app-generator/sample-django-celery.git
$ cd sample-django-celery

👉 Step #2 - Install the modules using a virtual environment

$ virtualenv env
$ source env/bin/activate
$ pip3 install -r requirements.txt

👉 Step #3 - Migrate the database

$ python manage.py makemigrations
$ python manage.py migrate

👉 Step #4 - Start the project

$ python manage.py runserver

Using a separate terminal window, the following command starts the Celery manager:

👉 Step #5 - Activate the VENV

$ source env/bin/activate

👉 Step #6 - Start the manager

$ celery --app=core.celery.app worker --loglevel=info

At this point, we should be able to execute tasks using a SuperUser Account. Ordinary users are able only to check out the tasks output and runtime logs.