Automating Cron Jobs in Docker with Ofelia: CVZilla's Experience

Anton Yanchenko — Sun, 20 Oct 2024 11:51:19 +0000

Cron jobs are an essential part of maintaining modern applications, automating tasks like data fetching, report generation, and updates. But when you're running containerized apps in Docker, traditional cron setups don’t work seamlessly. Enter Docker Ofelia, a simple and effective solution to run cron jobs in a containerized environment.

At CVZilla, a tech job board connecting developers with jobs that match their exact tech stack, we rely on Docker Ofelia to keep things running smoothly. We use it for tasks like web scraping to ensure job listings are always up-to-date, posting updates to our Telegram channels, and keeping various system tasks automated.

Ofelia acts as a cron daemon for Docker containers, allowing us to schedule and run jobs without leaving the containerized world. It's easy to configure, and it ensures that tasks are performed without the need for manual intervention.

Here’s how we use Docker Ofelia to manage periodic tasks at CVZilla:

Web Scraping: Our scrapers run at regular intervals to fetch the latest job postings from trusted sources.
Telegram Updates: Ofelia ensures our subscribers receive timely job updates on Telegram.

By automating these crucial processes, we can focus on what really matters: providing a seamless experience for both job seekers and employers. If you're building your own platform, or just looking for a reliable way to automate Docker tasks, give Docker Ofelia a try. It’s been a game-changer for us at CVZilla, and it might just do the same for you.

Tasks themselves in this case are implemented as Django custom management commands and running via django-admin CLI helper. Our docker-compose.yaml looks like this:

services:
  cvzilla:
    image: "ghcr.io/zetlabs/cvzilla:354"
    <...>
    labels:
      ofelia.enabled: "true"

      # https://crontab.guru/
      # ofelia syntax has additional seconds field by some reason
      # even though it seems that it's not second-accurate

      # run every day at 07:30 UTC
      ofelia.job-exec.exporters-telegram.schedule: "0 30 7 * * *"
      ofelia.job-exec.exporters-telegram.command: "django-admin exporters_telegram"

      # run every hour
      ofelia.job-exec.miners-publish-0.schedule: "0 30 * * * *"
      ofelia.job-exec.miners-publish-0.command: "django-admin miners_publish"

      # run daily at 10:00 & 17:00  UTC, Mon-Fri
      ofelia.job-exec.miners.schedule: "0 0 10,17 * * 1-5"
      ofelia.job-exec.miners-vinted.command: "django-admin miners_mine <args>"

  ofelia:
    image: mcuadros/ofelia:latest
    restart: "unless-stopped"
    depends_on:
      - cvzilla
    command: daemon --docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

DEV Community: Anton Yanchenko

Automating Cron Jobs in Docker with Ofelia: CVZilla's Experience