Quick tip: your cron script 'succeeded' because bash ignored the error. Start every job with set -euo pipefail

Here's a nightly backup script that looks perfectly reasonable:

#!/usr/bin/env bash
pg_dump "$DATABASE_URL" | gzip > /backups/db-$(date +%F).sql.gz
aws s3 cp /backups/db-$(date +%F).sql.gz s3://my-backups/
echo "backup done"

Suppose pg_dump fails: wrong password, database unreachable, disk full. What happens?

• gzip happily compresses the empty output into a valid, tiny .gz file.
• aws s3 cp uploads it successfully.
• echo prints "backup done".
• The script exits with status 0.

Your monitoring sees success. Your backup bucket fills with 20-byte files. You find out when you need a restore.

The tip: start every scheduled bash script with this line.

set -euo pipefail

What each flag does

-e: stop on errors

Without it, bash runs every line regardless of whether the previous one failed. With it, the script exits as soon as a command returns a non-zero status.

set -e
false            # script stops here, exit status 1
echo "never printed"

-o pipefail: pipelines fail if any part fails

This is the one that saves the backup script. By default, a pipeline's exit status is that of the last command. pg_dump | gzip "succeeds" because gzip succeeded. With pipefail, the pipeline fails if any command in it fails.

set -o pipefail
false | true     # pipeline now returns 1, and with -e the script stops

-u: unset variables are errors

A typo or a missing environment variable silently becomes an empty string without -u. Cron runs with a minimal environment, so missing variables are common.

set -u
rm -rf "/data/$TMPDIR_TYPO/"   # without -u: rm -rf "/data//" (!); with -u: error, script stops

The fixed script

#!/usr/bin/env bash
set -euo pipefail

: "${DATABASE_URL:?DATABASE_URL is not set}"   # explicit, readable error
file="/backups/db-$(date +%F).sql.gz"

pg_dump "$DATABASE_URL" | gzip > "$file"

# sanity check: refuse to upload a suspiciously small backup
size=$(stat -c %s "$file")
if [ "$size" -lt 1024 ]; then
  echo "backup too small ($size bytes), refusing to upload" >&2
  exit 1
fi

aws s3 cp "$file" s3://my-backups/
echo "backup done ($size bytes)"

Now a failing pg_dump stops the script with a non-zero exit, and your scheduler, cron's mail or your monitoring can see it.

Add a trap so failures are visible

set -e exits quietly. Add a trap to say where it failed:

trap 'echo "FAILED at line $LINENO: $BASH_COMMAND (exit $?)" >&2' ERR

In your logs:

FAILED at line 7: pg_dump "$DATABASE_URL" (exit 1)

That line is the difference between a two-minute fix and a half-hour investigation.

The gotchas

set -e has well-known quirks. Know these and you'll avoid most surprises:

1. Commands you expect to fail. grep returns 1 when it finds nothing, which isn't an error for you but stops the script:

count=$(grep -c "ERROR" app.log || true)   # "|| true" makes "no matches" OK

2. Conditions don't trigger -e. Commands inside if, while, && or || are exempt. That's usually what you want, but it means -e isn't a substitute for checking results you care about.

3. Functions called in conditions lose -e inside them. If you call my_func || handle_error, failures inside my_func won't stop it early. Keep important logic out of that pattern, or check explicitly.

4. Command substitution in local. local x=$(failing_cmd) succeeds, because local itself succeeded. Split it up:

local x
x=$(failing_cmd)    # now -e sees the failure

5. Optional variables need defaults with -u.

level="${LOG_LEVEL:-info}"   # default if unset

The same idea outside bash

Other languages have their own versions of "the error was swallowed and the job exited 0":

• Python: a broad try/except Exception: log(...) around the whole job logs the error and then exits successfully. Re-raise, or call sys.exit(1) after logging.
• Node.js: an unhandled promise rejection in older setups, or a .catch(console.error) at the top level, can leave the exit code at 0. End with process.exitCode = 1 in your error handler.
• Subprocesses: subprocess.run(cmd) in Python doesn't raise on failure unless you pass check=True.

import subprocess, sys
try:
    subprocess.run(["pg_dump", "mydb"], check=True, stdout=open("/backups/db.sql", "wb"))
except subprocess.CalledProcessError as e:
    print(f"pg_dump failed with {e.returncode}", file=sys.stderr)
    sys.exit(1)

The rule is the same everywhere: if the work failed, the process must exit non-zero.

Lint it before cron runs it

ShellCheck catches a lot of what strict mode would trip over at 3 a.m.: unquoted variables, cd without error handling, and $? checks that never fire. Run it locally and in CI:

shellcheck scripts/*.sh

Treat its warnings as failing tests for anything that runs unattended. A clean ShellCheck run plus strict mode covers most of the silent-failure bugs in cron scripts.

Why it matters more for scheduled jobs

Interactive scripts have a human watching the output. Scheduled scripts don't. The exit status is the only signal your scheduler, heartbeat monitor or alerting gets. If the script lies about success, everything downstream believes it.

The same applies to HTTP-triggered jobs: whether a cron entry, a CI workflow or a scheduling service like delay.run calls your endpoint, the handler should return an error status when the work actually failed, not a cheerful 200.

The 30-second audit

# find scheduled scripts without strict mode
grep -L 'set -euo pipefail' /opt/jobs/*.sh

Every file it lists is a script that can fail silently. Add the line, add the trap, test each one by forcing a failure (wrong password, missing variable), and confirm it exits non-zero.

Related reading

• Case study: redesigning a failed-payment retry schedule that retried every card at 00:00 UTC
• Cron job not running? A step-by-step checklist (including “it works when I run it manually”)
• How to retry failed cron jobs (with exponential backoff, in Bash and Python)
• How late is 'every minute'? An experiment you can run on cron, systemd timers and sleep loops

Have you ever found a scheduled job that had been "succeeding" while doing nothing? How long had it been going on? Share your story in the comments.