DEV Community

Nilail for SilenceWatch

Posted on Originally published at silencewatch.com

Spring @Scheduled job stopped? Know when it does

A Spring Boot @Scheduled job that stops without an error is usually waiting behind another job stuck on the scheduler’s single thread; scheduling switched off, a cron set to - or a missing instance do the same. None of them logs anything, and the health check stays UP. Here is how to confirm each cause, fix it, and be alerted the day it starts.

Does an exception stop a @Scheduled job?

Start with what does not stop a job. Take a method that always fails:

@Scheduled(fixedRate = 1000)
void failing() {
  throw new IllegalStateException("boom");
}
Enter fullscreen mode Exit fullscreen mode

Run on Spring Boot 3.5.7, it logs this every second, and keeps being called:

ERROR 748 --- [   scheduling-1] o.s.s.s.TaskUtils$LoggingErrorHandler : Unexpected error occurred in scheduled task
java.lang.IllegalStateException: boom
Enter fullscreen mode Exit fullscreen mode

Spring logs the exception of a repeating task and schedules the next run anyway. That is useful, and it is also why a failing job can go on failing for weeks: the error is in a log, and nobody reads that log until something downstream breaks.

The jobs that really go quiet are the next ones.

Why a @Scheduled job stops without an error

1. One stuck job freezes all the others

By default, Spring Boot runs every @Scheduled method on a single thread (spring.task.scheduling.pool.size is 1). If one of them never returns, because it waits on a socket without a timeout, a lock or a queue, none of the others runs again.

I reproduced it with three jobs: a heartbeat every second, the failing job above, and one that blocks forever from the fourth second:

17:39:40.449  heartbeat job ran
17:39:41.449  heartbeat job ran
17:39:41.451  stuck job starts and never returns
              (nothing after this line: no error, no warning, the process keeps running)
Enter fullscreen mode Exit fullscreen mode

The heartbeat job and the failing job both stop at that moment. The application is alive, so a liveness probe and a health check are happy.

To confirm, take a thread dump (jstack <pid> or Actuator’s threaddump endpoint; only health is exposed over HTTP by default, so add management.endpoints.web.exposure.include: scheduledtasks,threaddump,env for the checks below): scheduling-1 sits in the stuck method, in a socket read or waiting on a lock.

2. Scheduling is switched off

Without @EnableScheduling, a @Scheduled annotation is only an annotation: no error, no run. It is easy to lose in a refactoring of the configuration classes, and the application still starts and serves requests. To confirm, look at Actuator’s scheduledtasks endpoint: it lists no task.

3. The schedule is disabled by a property

@Scheduled(cron = "${report.cron}") accepts the special value -, which Spring documents as a disabled trigger, meant to switch a job off from the outside. A - left in one environment’s configuration (copied from a test profile, say), and that job is never scheduled there. Any other invalid value fails at startup instead. To confirm, note that the task is missing from scheduledtasks, and look for report.cron in that environment’s configuration. /actuator/env/report.cron shows where the property comes from, but since Spring Boot 3 its value is masked unless management.endpoint.env.show-values is set.

4. The application is not running where you think

A job that runs in a single instance dies with it: a scaled-down deployment, a crash loop, an instance that was never started in that environment. Nothing in the application can report it, because the application is gone. To confirm, check the replica count and restarts (kubectl get deploy,pods).

All four have the same signature: nothing happens, so nothing is written.

Why logs and health checks miss it

  • Logs. There is no line to search for. You can alert on an error; you cannot alert on a line that is not there.
  • Health checks and liveness probes. They tell you the process answers, not that its jobs run.
  • Metrics. A timer on each job helps, but catching a job that stopped means writing an alert on absence, for each job, and keeping those rules in step with the code.
  • Reading the list of scheduled tasks. Actuator’s scheduledtasks endpoint shows each task’s next and last execution, so a stuck run is visible there if someone looks. Nothing alerts on it.

How to be alerted when a scheduled job stops

The reliable signal is the one that fails in the right direction: the job reports every successful run to an outside service, which expects it on a schedule and alerts when it does not come. That is a dead man’s switch. A stuck thread, a missing annotation, a disabled cron and a dead instance all raise the same alert: the heartbeat did not arrive.

How to fix a stuck scheduler

Being alerted is the first half. For the stuck-thread case, the usual remedies are:

  • put a timeout on every outgoing call a job makes (HTTP client, database, queue): a job that can wait forever eventually will;
  • give the scheduler more than one thread, so one slow job cannot hold the others back: spring.task.scheduling.pool.size: 5;
  • keep long jobs out of the scheduler thread, by handing the work to an executor and returning; if the heartbeat is sent when the scheduled method returns, it marks the hand-off, not the work, so send it at the end of the work itself, or prefer the two fixes above.

Frequently asked questions

Why did my @Scheduled method stop running without an error?

The usual causes are another scheduled method that never returns (Spring Boot runs them all on one thread by default), scheduling not enabled, a cron disabled with the value -, or an application instance that is no longer running. None of them writes a log line.

What happens when a @Scheduled method throws an exception?

For a repeating task, Spring logs the exception at ERROR level and runs the method again at its next scheduled time. The job does not stop, but it can fail on every run without anyone being told.

How many threads does Spring use for @Scheduled methods?

One by default in Spring Boot. Raise spring.task.scheduling.pool.size so that a slow job does not delay the others.

How can I be alerted when a scheduled job stops?

Have the job report each successful run to a service that expects it on a schedule and alerts when it is missing, a dead man’s switch.

See also


Originally published on the SilenceWatch blog. This article was written with the help of AI and checked against the sources before publishing.

Top comments (0)