DEV Community

loading...

Database Scheduler with celery beat and manage schedule in Django admin

daeenchoi profile image Daeen Choi ・4 min read

Must read previous post first before reading this one :D
Previous Post about celery

Aim of this post

  1. Install celery beat in django project and understand how it is used
  2. Settings for sending emails in django project using gmail
  3. Create a task that sends email
  4. Create schedule using django admin and integrate it with email task Eg) Send notification email to users every Fridays at 4 a.m.

What is celery beat?

celery beat is a scheduler. When it's time to run the task, it delivers the entry to the worker node. (Periodic task execution)

Install celery beat
pip install django-celery-beat

Migrate - Tables will be created to store tasks and schedules
python manage.py migrate

settings.py

INSTALLED_APPS = (
    ...,
    'django_celery_beat',
)
Enter fullscreen mode Exit fullscreen mode

Settings for emails

1 . Click Manage your Google Account
Alt Text



2 . Make sure 2 step verification is on in Security > 2 step verification
Alt Text



3 . Create app password. App passwords > select app > other > Name it whatever!

Alt Text



4 . Add settings.py

# e-mail settings
EMAIL_USE_TLS = True
EMAIL_PORT = 587
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = {YOUR EMAIL ADDRESS}
EMAIL_HOST_PASSWORD = {YOUR PASSWORD FROM NO.3}
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
Enter fullscreen mode Exit fullscreen mode

Email setting is finished!

Where does celery beat store tasks and schedules?

1. Default entries are taken from the beat_schedule setting

task.py

app.conf.beat_schedule = {
  'add-every-30-seconds': {
        'task': 'tasks.add',
        'schedule': 30.0,
        'args': (16, 16)
    },
}
app.conf.timezone = 'UTC'
Enter fullscreen mode Exit fullscreen mode



Or if you want more control over your schedules use crontab

from celery.schedules import crontab

app.conf.beat_schedule = {
    # Executes every Monday morning at 7:30 a.m.
    'add-every-monday-morning': {
        'task': 'tasks.add',
        'schedule': crontab(hour=7, minute=30, day_of_week=1),
        'args': (16, 16),
    },
}
Enter fullscreen mode Exit fullscreen mode

Start celery beat service
celery -A <project name> beat

2 . Use custom stores - I will store them in database

  1. Let's create a task that sends email to users
@app.task(name="send_notification", bind=True, default_retry_delay=300, max_retries=5)
def send_notification(self, subject, message):
    from django.core.mail import send_mail as sm

    # Fetch all users except superuser
    users = User.objects.exclude(is_superuser=True).all()
    user_emails = [user.email for user in users]

    # try sending email
    try:
        res = sm(
            subject=subject,
            html_message=message,
            from_email=EMAIL_HOST_USER,
            recipient_list=user_emails,
            fail_silently=False,
            message=None)
        print(f'Email send to {len(user_emails)} users')
    except Exception:

        # retry when fail
        send_notification.retry()

Enter fullscreen mode Exit fullscreen mode

So here I created a task called send_notification.
default_retry_delay : retry after 300 seconds when fail
max_retries : only retry 5 more times then STOP.
Here I am also getting subject and message as arguments. This means I can pass subject and message from django admin. I will show you this later.

  1. Let's go to django admin localhost:8080/admin.

You will see new Periodic Tasks Menu
Alt Text

Let's make a crontab Crontab > Add crontab
Alt Text

Let's create a crontab .Here I created one. Every Saturday @ 10:47 p.m.
If you are not sure about crontab, check this out Crontab examples
Alt Text

Let's create a periodic task
Click Periodic tasks > Add. Create a new name of your choice. Find send_notification task that we have created up there
Alt Text

Schedule > Crontab Schedule > Find the crontab you just created.
Add the Start Datetime
Alt Text

This is where you pass the arguments subject and message
Alt Text

So we are all set and ready to go.

Start django server, start rabbitmq

Start celery
celery -A djangocelery(app name) worker --loglevel=info

Start celery beat in different terminal
celery -A djangocelery(app name) beat -l INFO --scheduler django_celery_beat.schedulers:DatabaseScheduler

Let's check out the terminal of celery beat

@10:47 p.m. celery beat scheduler is sending send_notification task that we have created!! to celery!!
Alt Text

Check celery terminal. Celery has received the task and email is sent.
Alt Text

And I have received the email at 10:47 p.m. on Saturday with the title and content that I have set up in django admin!
Alt Text

Conclusion

So celery beat is a scheduler and celery is the one that executes the task. Database scheduler is one of the method of celery beat where tasks and schedules are stored in database which means you can manage them in django admin.



That's it for django celery beat!
The end.

Discussion (1)

pic
Editor guide
Collapse
rayvikram profile image
rayvikram

I have also used these awsome combination. It's great