DEV Community

Cover image for Cron Was the Easy Part: Running Scheduled Tasks Safely Inside Containers
hoagminh
hoagminh

Posted on

Cron Was the Easy Part: Running Scheduled Tasks Safely Inside Containers

I wanted to add scheduled tasks to Nitrobase because almost every real application eventually needs one.

A Laravel app clears old sessions. A Node.js service sends a digest. A database-backed product generates invoices. A small internal tool imports a CSV every night.

At first, the feature sounded almost trivial:

cron expression + command = scheduled task
Enter fullscreen mode Exit fullscreen mode

Then I started asking what should happen when the command fires during a deployment, runs forever, fills memory, prints a password, or targets a container that is no longer running.

The cron parser was the easy part.

The real feature was deciding how scheduled work should behave around the rest of the application.

Nitrobase has only recently launched and is still in Beta. The scheduled-task feature is meant to solve one practical problem: run recurring application commands from the same place that already knows where the app lives, which service should execute them, and whether that app is currently safe to touch.

The traditional cron entry loses context

On a single VPS, I can always SSH in and write something like this:

0 3 * * * docker exec my-app php artisan reports:daily
Enter fullscreen mode Exit fullscreen mode

It works, but the line quietly depends on knowledge stored in my head:

  • the correct server;
  • the current container name;
  • the service that contains the command;
  • the deployment schedule;
  • the maximum acceptable runtime;
  • where stdout and stderr should go;
  • who should be notified when it fails.

Six months later, the application may have moved to another VPS while the old cron entry remains behind. The container may have been renamed. A teammate may redeploy the app at 03:00 without knowing a report job is halfway through.

The command is automated, but its operational context is not.

System cron knows:       "run this text at 03:00"

Nitrobase also knows:    which app
                         which server
                         which service
                         whether the app is running
                         whether deploy or backup is active
                         who may inspect the output
Enter fullscreen mode Exit fullscreen mode

That extra context is where a control plane can be useful.

A task runs inside the service, never on the host

The first boundary I wanted was simple: an application task should not become a disguised root command on the VPS.

Nitrobase tasks run through docker exec inside one selected service container. They do not execute directly on the host.

Nitrobase scheduler
        |
        | docker exec
        v
Selected app service
        |
        v
Application command

Host shell access: no
Enter fullscreen mode Exit fullscreen mode

This matches how developers already think about application commands.

php artisan queue:prune-batches
bin/rails runner "Report.generate"
node scripts/send-digest.js
python manage.py clearsessions
Enter fullscreen mode Exit fullscreen mode

The command runs with the filesystem, environment, packages, user, and resource limits of that container. If the application image does not contain a program, the scheduled task cannot magically use the host's copy.

That limitation is intentional. The task belongs to the application, so it should live inside the application's runtime boundary.

Direct arguments and shell scripts are different tools

Nitrobase supports two command modes.

The safer, simpler mode is an argument list:

Program: php
Argument: artisan
Argument: reports:daily
Argument: --quiet
Enter fullscreen mode Exit fullscreen mode

Conceptually, this becomes:

["php", "artisan", "reports:daily", "--quiet"]
Enter fullscreen mode Exit fullscreen mode

Each item stays one argument. There is no shell expansion, pipe, redirection, or accidental interpretation of characters such as ; and $().

Shell mode exists for tasks that genuinely need shell behavior:

php artisan cache:prune && php artisan reports:daily
Enter fullscreen mode Exit fullscreen mode
Mode Best for Shell interpretation
Argument list One program with explicit arguments No
Shell script Pipes, redirects, variables, multiple commands Yes

I prefer the argument-list mode whenever it can express the job. Shell mode is useful, but it also makes quoting, expansion, and secret handling the operator's responsibility.

What if a deployment starts at the same time?

This was the first question that turned “add cron” into an operations feature.

Imagine a task starts rebuilding a search index while a deployment replaces its container. Or a backup stops database writers while a scheduled import is inserting rows.

Running both is unpredictable. Nitrobase already has locks for operations on servers and applications, so scheduled tasks use the same coordination layer.

03:00 scheduled task becomes due
              |
              v
     Is the app/server busy?
          /           \
        no             yes
        |               |
        v          task policy
      run          /          \
                 skip         queue
                  |             |
            record reason   wait for idle
Enter fullscreen mode Exit fullscreen mode

Each task can choose a collision policy:

  • Skip records that the run did not happen because the server was busy.
  • Queue waits until the conflicting operation releases its lock, up to the configured maximum wait.

Neither policy is universally correct.

For a five-minute cache refresh, skipping one run may be fine. For a daily invoice process, waiting may be safer than silently losing the day's work.

The important part is that the collision becomes a visible state, not a race between two unrelated shell processes.

“Skipped” is not the same as “failed”

I originally thought a run needed only three states: running, success, and failed.

That was not enough to explain what actually happened.

Nitrobase records these outcomes:

Status Meaning
success Command exited normally with code 0
failed Command exited with a non-zero code
skipped App was unavailable or a lock policy prevented the run
timeout Command exceeded its allowed runtime
oom The container was killed for exceeding memory
killed A signal ended the command for another reason

These distinctions change the response.

failed  -> inspect application error
timeout -> inspect workload or raise the time limit
oom     -> inspect memory use and container limit
skipped -> inspect scheduling and lock policy
killed  -> inspect signals and container lifecycle
Enter fullscreen mode Exit fullscreen mode

Treating all five as “cron failed” would hide the information needed to fix them.

Timeouts need two boundaries

A recurring command should not be allowed to run forever merely because nobody is watching it.

Nitrobase applies a timeout inside the container when the image provides the necessary shell and timeout utility. It also keeps a host-side deadline around the Docker execution.

Application command
        |
        +-- in-container timeout
        |
        +-- host-side Docker execution timeout
Enter fullscreen mode Exit fullscreen mode

The inner timeout can terminate the application process cleanly. The outer timeout ensures the control-plane job itself does not remain stuck if the Docker client or container behaves unexpectedly.

Images without a shell still work in direct-argument mode; they simply rely on the outer execution boundary.

The default run limit is five minutes, and the installation can configure its maximum. A task that legitimately needs longer can request it within that limit.

The goal is not to guess the perfect duration. It is to make “unbounded” an explicit decision rather than an accidental default.

An exit code cannot always explain an OOM kill

A process terminated by memory pressure may look like an ordinary signal failure. Docker also has an OOMKilled flag, but that flag can remain set after an earlier incident until the container restarts.

To avoid blaming the wrong run, Nitrobase checks the container's cgroup OOM-kill counter before and after the command when cgroup v2 makes it available.

OOM counter before run:  4
OOM counter after run:   5
                         ^
                         this run caused an OOM kill
Enter fullscreen mode Exit fullscreen mode

When the counter is unavailable, it falls back to the Docker state transition.

It is a small implementation detail, but it produces a much more useful message than “process exited 137.”

Output is useful, until it becomes a secret leak

When a task fails at night, its output is usually the first thing I need in the morning.

Nitrobase streams task output into the normal job log and keeps a bounded tail with the run history. The history shows whether the task was manual or scheduled, when it started, how long it ran, its exit code, and its final status.

But application output can contain sensitive values:

Connecting with DATABASE_URL=postgres://user:password@db/app
Enter fullscreen mode Exit fullscreen mode

Before task output is persisted, Nitrobase masks known application secret values. The masker also accounts for values split across streamed output chunks rather than assuming each chunk contains a complete line.

This is a defensive layer, not a guarantee.

An application may transform a secret, print only fragments, encode it, or fetch a credential at runtime that Nitrobase does not know. A safe scheduled command should still avoid echoing credentials in the first place.

Best:       command never prints the secret
Fallback:   known secret is masked from captured output
Still bad:  relying on redaction as the only protection
Enter fullscreen mode Exit fullscreen mode

Output is also bounded. Keeping the tail prevents one noisy task from growing the panel database indefinitely while preserving the part most likely to contain the final error.

Manual runs make the schedule testable

Waiting until 03:00 to discover a typo is not a good workflow.

Every task can also be triggered manually. The manual run uses the same container boundary, lock coordination, timeout, output handling, and status classification as a scheduled run.

That means the useful setup loop is:

  1. Choose the service.
  2. Enter the command in argument-list or shell mode.
  3. Set its timeout and busy-server policy.
  4. Run it manually.
  5. Inspect the output and exit status.
  6. Enable the cron schedule.

The manual test cannot prove that the task will always succeed, but it catches the most common configuration errors without creating a second execution path.

Schedules have limits on purpose

Nitrobase validates five-field cron expressions and rejects schedules that run more frequently than the configured minimum. By default, that minimum is five minutes.

It also limits the number of tasks per application and retains only a configured number of recent runs.

These limits are not meant to turn Nitrobase into a high-volume job queue.

Scheduled tasks are good for:
  - nightly cleanup
  - periodic reports
  - regular application maintenance
  - small imports and synchronizations

They are not a replacement for:
  - millisecond scheduling
  - distributed queues
  - thousands of concurrent jobs
  - workflow orchestration with dependencies
Enter fullscreen mode Exit fullscreen mode

If an application needs retries, concurrency controls, job priorities, and durable per-item processing, that logic belongs in a proper queue or workflow system inside the application.

Nitrobase handles the recurring trigger and its operational boundary. It does not pretend that cron is a distributed job platform.

A scheduler restart is another boundary

The scheduler lives in the Nitrobase process and registers enabled tasks when the panel starts. Missed application-task runs are not replayed after downtime.

I think this behavior is worth stating plainly.

Automatic catch-up sounds helpful, but replaying several missed maintenance commands at once after a server outage can be worse than skipping them. Some commands are not idempotent. Others are no longer relevant after their intended window.

If a task must run exactly once despite downtime, its application should keep durable state and decide what work remains. The scheduler should not guess.

My checklist for a recurring container task

Before enabling a new task, I now ask:

  • [ ] Is the command running in the correct service?
  • [ ] Can it use direct arguments instead of a shell script?
  • [ ] Is it safe to run more than once?
  • [ ] Should it skip or queue when a deploy or backup is active?
  • [ ] What is a realistic timeout?
  • [ ] What happens if the process exceeds the container's memory limit?
  • [ ] Does it print credentials, tokens, or personal data?
  • [ ] Is its output small enough to be useful?
  • [ ] Who receives a notification when it fails or is skipped?
  • [ ] Has it succeeded as a manual run?
  • [ ] Is missing a run acceptable when the panel is offline?

The cron expression is only one item because it is rarely the hardest one.

The problem this feature is trying to solve

I did not build scheduled tasks because writing a crontab entry is difficult.

I built them because a crontab entry outside the application platform does not know when the app is moving, deploying, backing up, stopped, renamed, or failing for lack of memory.

Nitrobase already holds that operational context. Bringing recurring commands into the same model means the task can follow the app and participate in the same safety boundaries.

schedule
   + container identity
   + app/server locks
   + timeout
   + classified outcome
   + bounded, masked output
   + run history and notifications
   = an operational task, not just a cron line
Enter fullscreen mode Exit fullscreen mode

This is still a Beta implementation, and real workloads will expose cases I have not encountered yet. I am especially interested in feedback about collision policies, missed-run behavior, long-running maintenance commands, and the amount of output worth retaining.

The lesson so far is simple:

Scheduling a command is easy. Scheduling it without forgetting everything around the command is the real feature.

Nitrobase source code: https://github.com/AydenGroup/nitrobase

Top comments (0)