DEV Community: Ahmad Naqibul Arefin

Quartz Scheduler with Spring Boot and PostgreSQL in a Clustered Environment

Ahmad Naqibul Arefin — Fri, 07 Mar 2025 06:34:01 +0000

Setting up Quartz Scheduler in a distributed, clustered Spring Boot environment allows multiple instances of your application to coordinate scheduled tasks without conflicts or duplications. Here's a comprehensive, step-by-step guide for a production-ready setup:

Step 1: Adding Dependencies (`pom.xml`)

Add Quartz and PostgreSQL dependencies to your project:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-quartz</artifactId>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>

2. Database Setup

Create a PostgreSQL database (e.g., quartz_db) and initialize Quartz tables using the provided Quartz schema:

Locate the schema script: docs/dbTables/postgresql.sql in the Quartz distribution.
Execute this script against your PostgreSQL database to generate Quartz tables (e.g., QRTZ_JOB_DETAILS, QRTZ_TRIGGERS).

3. Quartz Configuration (`application.yml`)

Configure Quartz properties for clustering and PostgreSQL persistence:

spring:
  quartz:
    job-store-type: jdbc
    jdbc:
      initialize-schema: never
    properties:
      org.quartz.jobStore.class: org.springframework.scheduling.quartz.LocalDataSourceJobStore
      org.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
      org.quartz.jobStore.isClustered: true
      org.quartz.jobStore.clusterCheckinInterval: 10000
      org.quartz.scheduler.instanceId: AUTO
      org.quartz.scheduler.instanceName: ClusteredQuartzScheduler
      org.quartz.threadPool.threadCount: 5

4. Spring Quartz Scheduler Configuration

Define Quartz's SchedulerFactoryBean and a custom JobFactory to enable Spring-managed dependency injection:

@Configuration
public class QuartzConfig {

    @Bean
    public SchedulerFactoryBean schedulerFactoryBean(DataSource dataSource, PlatformTransactionManager transactionManager, JobFactory jobFactory, Trigger jobTrigger) {
        SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
        scheduler.setDataSource(dataSource);
        scheduler.setTransactionManager(transactionManager);
        scheduler.setJobFactory(jobFactory);
        scheduler.setTriggers(sampleJobTrigger);
        return scheduler;
    }

5. Defining a Job

Create a Quartz job implementing the Job interface:

@Component
public class MyQuartzJob implements Job {

    @Autowired
    private MyService myService;

    @Override
    public void execute(JobExecutionContext context) throws JobExecutionException {
        myService.executeTask();
    }
}

6. Scheduling the Job

Schedule the job to trigger every 15 minutes:

@Bean
public JobDetail sampleJobDetail() {
    return JobBuilder.newJob(SampleJob.class)
        .withIdentity("sampleJob")
        .storeDurably()
        .build();
}

@Bean
public Trigger sampleJobTrigger(JobDetail sampleJobDetail) {
    return TriggerBuilder.newTrigger()
        .forJob(sampleJobDetail)
        .withSchedule(CronScheduleBuilder.cronSchedule("0 0/15 * * * ?"))
        .build();
}

7. Ensuring Clustered Execution

Ensure all your application instances share the same PostgreSQL database configuration and Quartz properties (isClustered=true). Quartz internally coordinates using the database to avoid concurrent execution of the same job by multiple nodes.

8. Testing and Deployment

Run multiple application instances pointing to the same database.
Monitor logs to verify that jobs run exclusively on a single node per trigger interval.
Check Quartz's cluster tables (e.g., QRTZ_SCHEDULER_STATE) to ensure nodes register correctly.

This guide ensures robust, coordinated job scheduling with Quartz and Spring Boot in a distributed environment using PostgreSQL.

Building Scalable Asynchronous Backends with Spring Boot and Virtual Threads

Ahmad Naqibul Arefin — Sat, 01 Mar 2025 05:59:35 +0000

In modern application architectures, designing a backend that can efficiently handle heavy loads while providing timely responses is crucial. In this blog post, we'll explore how to build a scalable asynchronous backend service using Spring Boot. We'll cover traditional asynchronous processing with DeferredResult and @Async, and then introduce virtual threads (from Project Loom) that allow you to write blocking code in a natural style while still scaling to massive concurrency.

Introduction
Asynchronous Processing in Spring Boot
Using DeferredResult to Manage Responses
Customizing Thread Pools for Heavy Loads
Virtual Threads: Simplifying Blocking Code
Complete Example
Conclusion

Introduction

When your backend service needs to handle up to a million concurrent requests, every millisecond counts. Traditional synchronous processing can easily lead to thread exhaustion and increased latency. Asynchronous processing decouples request handling from heavy business logic, ensuring each request is processed independently. In this post, we'll dive into how you can achieve this using Spring Boot's asynchronous support—and how virtual threads can further enhance scalability by letting you write blocking code without the overhead of traditional threads.

Asynchronous Processing in Spring Boot

Spring Boot makes it easy to run tasks asynchronously. By using the @Async annotation on service methods, you can offload heavy processing tasks to a separate thread pool. This approach ensures that your web container (like Tomcat or Netty) isn't blocked by long-running operations.

For example, consider a service method that simulates a heavy computation:

@Service
public class AsyncService {

    @Async("taskExecutor")
    public CompletableFuture<String> processTask(String input) {
        try {
            // Simulate a heavy process, e.g., I/O or computation.
            Thread.sleep(3000); // 3-second delay
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            return CompletableFuture.completedFuture("Processing interrupted");
        }
        return CompletableFuture.completedFuture("Processed input: " + input);
    }
}

In this example, the @Async annotation dispatches the method to a separate thread, freeing up the web request thread almost immediately.

Using DeferredResult to Manage Responses

While asynchronous processing decouples the heavy logic from the request thread, you still need a way to provide a response once the processing is complete. This is where Spring's DeferredResult comes in.

When a request is made, you immediately return a DeferredResult from your controller. This object holds the eventual response and is updated once the asynchronous task finishes. Here’s how you can use it:

@RestController
public class AsyncController {

    @Autowired
    private AsyncService asyncService;

    @GetMapping("/process")
    public DeferredResult<ResponseEntity<String>> process(@RequestParam(value = "input", defaultValue = "default") String input) {
        // Set a timeout of 5000ms (5 seconds)
        DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>(5000L);

        CompletableFuture<String> futureResult = asyncService.processTask(input);
        futureResult.thenAccept(result -> 
            deferredResult.setResult(ResponseEntity.ok(result))
        ).exceptionally(ex -> {
            deferredResult.setErrorResult(
                ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
                              .body("Error processing request")
            );
            return null;
        });

        return deferredResult;
    }
}

With this pattern:

Individual Request Contexts: Each HTTP request creates a unique DeferredResult.
Asynchronous Invocation: The heavy processing runs on a separate thread (or virtual thread, as we'll see later) and updates its corresponding DeferredResult when done.
Timeout Management: If processing exceeds the 5-second timeout, you can handle it gracefully.

Customizing Thread Pools for Heavy Loads

Under heavy loads, the choice of thread pool is critical. Configuring a custom Executor lets you fine-tune the number of threads, queue capacity, and more:

@Configuration
public class AsyncConfig {

    @Bean(name = "taskExecutor")
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);   // Minimum threads
        executor.setMaxPoolSize(50);    // Maximum threads
        executor.setQueueCapacity(100); // Queue capacity before new threads are created
        executor.setThreadNamePrefix("Async-");
        executor.initialize();
        return executor;
    }
}

This custom executor ensures that your asynchronous tasks are efficiently scheduled and executed, even under the strain of a million requests. If you don’t configure an executor that leverages virtual threads (as we’ll cover next), your code will run on these heavier platform threads.

Virtual Threads: Simplifying Blocking Code

Traditional asynchronous design forces you to avoid blocking calls or rewrite your code in a non-blocking, reactive style. However, not every use case requires a fully reactive model. With virtual threads—a feature introduced with Project Loom—you can write blocking code in a natural, synchronous style while still achieving high concurrency.

What Are Virtual Threads?

Lightweight Concurrency: Virtual threads are much lighter than platform threads, allowing you to create thousands or even millions without incurring significant overhead.
Simplified Coding: You can write blocking operations (like Thread.sleep()) directly. Virtual threads manage the underlying blocking efficiently.

Configuring a Virtual Thread Executor

Here’s how to create an executor that uses virtual threads:

@Configuration
public class AsyncConfig {

    @Bean(name = "virtualThreadExecutor")
    public Executor virtualThreadExecutor() {
        // Creates an executor that spawns a new virtual thread per task.
        return Executors.newVirtualThreadPerTaskExecutor();
    }
}

Using Virtual Threads in Your Service

You simply annotate your service method to use the virtual thread executor:

@Service
public class AsyncService {

    @Async("virtualThreadExecutor")
    public CompletableFuture<String> processTask(String input) {
        try {
            // Blocking code written in a natural style
            Thread.sleep(3000); // 3-second delay
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            return CompletableFuture.completedFuture("Processing interrupted");
        }
        return CompletableFuture.completedFuture("Processed input: " + input);
    }
}

This means that if you don't explicitly use virtual threads, your asynchronous tasks will run on traditional platform threads. Virtual threads let you write blocking code without worrying about the resource overhead typically associated with them.

Complete Example

Below is a complete Spring Boot project example that combines everything we've discussed—from asynchronous processing with DeferredResult to leveraging virtual threads for scalable blocking code.

Main Application Class

package com.example.asyncdemo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableAsync;

@SpringBootApplication
@EnableAsync
public class AsyncDemoApplication {
    public static void main(String[] args) {
        SpringApplication.run(AsyncDemoApplication.class, args);
    }
}

Virtual Thread Executor Configuration

package com.example.asyncdemo.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;

@Configuration
public class AsyncConfig {

    @Bean(name = "virtualThreadExecutor")
    public Executor virtualThreadExecutor() {
        return Executors.newVirtualThreadPerTaskExecutor();
    }
}

Asynchronous Service with Blocking Code

package com.example.asyncdemo.service;

import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import java.util.concurrent.CompletableFuture;

@Service
public class AsyncService {

    @Async("virtualThreadExecutor")
    public CompletableFuture<String> processTask(String input) {
        try {
            Thread.sleep(3000); // Simulate blocking call
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            return CompletableFuture.completedFuture("Processing interrupted");
        }
        return CompletableFuture.completedFuture("Processed input: " + input);
    }
}

REST Controller Using DeferredResult

package com.example.asyncdemo.controller;

import com.example.asyncdemo.service.AsyncService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.context.request.async.DeferredResult;
import java.util.concurrent.CompletableFuture;

@RestController
public class AsyncController {

    @Autowired
    private AsyncService asyncService;

    @GetMapping("/process")
    public DeferredResult<ResponseEntity<String>> process(@RequestParam(value = "input", defaultValue = "default") String input) {
        // Configure DeferredResult with a 5-second timeout
        DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>(5000L);

        CompletableFuture<String> futureResult = asyncService.processTask(input);
        futureResult.thenAccept(result -> 
            deferredResult.setResult(ResponseEntity.ok(result))
        ).exceptionally(ex -> {
            deferredResult.setErrorResult(
                ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
                              .body("Error processing request")
            );
            return null;
        });

        return deferredResult;
    }
}

Conclusion

In this post, we've explored how to build a scalable backend service with Spring Boot that can handle massive loads asynchronously. By leveraging:

DeferredResult to maintain unique request contexts,
@Async to decouple heavy processing from the request thread,
Custom executors to manage thread pools, and
Virtual threads to write natural blocking code while scaling concurrency,

you can design a robust service that responds promptly even under high load. If you choose not to use virtual threads, your asynchronous tasks will run on traditional platform threads, which are heavier. Virtual threads offer a game-changing approach by simplifying your code without sacrificing performance.

Happy coding, and may your backends scale gracefully!