DEV Community: Shreyash Kakde

Two Pods, One Job Table: A Safe Postgres Queue with SKIP LOCKED in Spring Boot

Shreyash Kakde — Tue, 03 Nov 2026 09:56:30 +0000

It's 10:20 AM on a Monday and support has forwarded three screenshots of the same "Your invoice is ready" email, received three times by one customer. The dashboards look fine. CPU is low, the error rate is zero. Your logs show this:

10:15:00.112 [scheduling-1] INFO  c.a.n.NotificationWorker - picked 50 jobs (pod notif-7c9d-a)
10:15:00.115 [scheduling-1] INFO  c.a.n.NotificationWorker - picked 50 jobs (pod notif-7c9d-b)
10:15:00.118 [scheduling-1] INFO  c.a.n.NotificationWorker - picked 50 jobs (pod notif-7c9d-c)

Three pods, one notification_job table, the same 50 rows. The deployment went from one replica to three last week, and nobody asked what @Scheduled does when it lives in three JVMs at once.

If you already run PostgreSQL, a job table with FOR UPDATE SKIP LOCKED is a reasonable queue for moderate volumes, and you don't need a broker for it. There are a few ways to get it subtly wrong, so this post goes through them.

The innocent worker
Why a plain SELECT hands out the same rows
What FOR UPDATE alone does
SKIP LOCKED
Pattern 1: claim in a short transaction, work outside it
Pattern 2: leases for crashed workers
Keeping the table healthy
Gotchas and when not to use this
Checklist

The innocent worker

This is roughly what the first version looks like in most codebases:

@Component
public class NotificationWorker {

    private final NotificationJobRepository jobs;
    private final EmailClient emailClient;

    public NotificationWorker(NotificationJobRepository jobs, EmailClient emailClient) {
        this.jobs = jobs;
        this.emailClient = emailClient;
    }

    @Scheduled(fixedDelay = 5000)
    @Transactional
    public void run() {
        List<NotificationJob> batch =
                jobs.findTop50ByStatusOrderByIdAsc(JobStatus.PENDING);

        for (NotificationJob job : batch) {
            emailClient.send(job.getRecipient(), job.getPayload());
            job.setStatus(JobStatus.DONE);
        }
    }
}

With one pod this works. @Scheduled is per JVM, so three pods mean three schedulers, and nothing in this code coordinates them.

Why a plain SELECT hands out the same rows

PostgreSQL defaults to READ COMMITTED. A plain SELECT takes no row locks, so every pod sees the same committed rows with status = 'PENDING'. Pod A hasn't committed its update to DONE when pod B reads, because pod A is still calling the email provider inside the transaction. All three pods read the same 50 rows.

sequenceDiagram
    participant A as Pod A
    participant B as Pod B
    participant DB as PostgreSQL
    A->>DB: SELECT ... WHERE status='PENDING' LIMIT 50
    B->>DB: SELECT ... WHERE status='PENDING' LIMIT 50
    DB-->>A: rows 1..50
    DB-->>B: rows 1..50
    A->>A: send 50 emails
    B->>B: send 50 emails
    A->>DB: UPDATE status='DONE', COMMIT
    B->>DB: UPDATE status='DONE', COMMIT

Both updates succeed, because setting a row to DONE twice is not a conflict. There is no error anywhere, only duplicate emails.

What FOR UPDATE alone does

The obvious fix is a pessimistic lock. In Spring Data JPA that is @Lock(LockModeType.PESSIMISTIC_WRITE) on the repository method, which Hibernate turns into SELECT ... FOR UPDATE. Pod B's query now tries to lock the same rows and waits for pod A to commit.

The duplicates are gone, but the pods now run one after another. Pod B holds a pooled connection and waits for as long as pod A's batch takes. If 50 emails take 10 seconds, pod B waits 10 seconds, then re-checks status = 'PENDING' against the updated rows and finds nothing. You pay for three workers and get one.

SKIP LOCKED

The PostgreSQL SELECT docs say that with SKIP LOCKED, any selected rows that cannot be immediately locked are skipped. The same page warns that this gives an inconsistent view of the data and isn't suited to general-purpose work, but that it can be used to avoid lock contention with multiple consumers of a queue-like table. That is our case.

SELECT id, recipient, payload
FROM notification_job
WHERE status = 'PENDING'
ORDER BY id
LIMIT 50
FOR UPDATE SKIP LOCKED;

With a LIMIT, locking stops once enough rows have been returned. Pod A locks rows 1 to 50, pod B skips them and locks 51 to 100, pod C gets 101 to 150. They run in parallel without waiting.

Now look at the transaction boundary. A row lock lasts until the transaction ends. If you keep the original @Transactional around the email calls, you hold 50 row locks and one HikariCP connection for the whole batch. With the default maximumPoolSize of 10, ten slow batches use every connection, which is the same trap as a transaction wrapped around a third-party API call. The claim has to be short.

Pattern 1: claim in a short transaction, work outside it

Mark the rows as taken in the same statement that selects them, and commit at once. The status column then does the job of the lock.

CREATE TABLE notification_job (
    id           BIGSERIAL PRIMARY KEY,
    recipient    TEXT        NOT NULL,
    payload      TEXT        NOT NULL,
    status       TEXT        NOT NULL DEFAULT 'PENDING',
    attempts     INT         NOT NULL DEFAULT 0,
    locked_until TIMESTAMPTZ,
    created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_notification_job_pending
    ON notification_job (id)
    WHERE status = 'PENDING';

The partial index holds only pending rows, so it stays small however many finished rows pile up. The claim is one UPDATE ... RETURNING:

import java.time.Duration;
import java.util.List;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Transactional;

@Repository
public class NotificationJobStore {

    private static final String CLAIM_SQL = """
            UPDATE notification_job
            SET status = 'RUNNING',
                attempts = attempts + 1,
                locked_until = now() + make_interval(secs => ?)
            WHERE id IN (
                SELECT id
                FROM notification_job
                WHERE status = 'PENDING'
                ORDER BY id
                LIMIT ?
                FOR UPDATE SKIP LOCKED
            )
            RETURNING id, recipient, payload, attempts
            """;

    private final JdbcTemplate jdbc;

    public NotificationJobStore(JdbcTemplate jdbc) {
        this.jdbc = jdbc;
    }

    public record Job(long id, String recipient, String payload, int attempts) {}

    @Transactional
    public List<Job> claim(int batchSize, Duration lease) {
        return jdbc.query(
                CLAIM_SQL,
                (rs, i) -> new Job(
                        rs.getLong("id"),
                        rs.getString("recipient"),
                        rs.getString("payload"),
                        rs.getInt("attempts")),
                lease.toSeconds(),
                batchSize);
    }

    @Transactional
    public void markDone(long id) {
        jdbc.update("UPDATE notification_job SET status = 'DONE', locked_until = NULL WHERE id = ?", id);
    }

    @Transactional
    public void markFailed(long id, boolean giveUp) {
        jdbc.update(
                "UPDATE notification_job SET status = ?, locked_until = NULL WHERE id = ?",
                giveUp ? "FAILED" : "PENDING", id);
    }
}

claim is a transaction of a few milliseconds. The subquery's row locks are released at commit, but by then the rows are RUNNING, so the next poller's status = 'PENDING' filter no longer matches them. The worker:

import java.time.Duration;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

@Component
public class NotificationWorker {

    private static final int MAX_ATTEMPTS = 5;

    private final NotificationJobStore store;
    private final EmailClient emailClient;

    public NotificationWorker(NotificationJobStore store, EmailClient emailClient) {
        this.store = store;
        this.emailClient = emailClient;
    }

    @Scheduled(fixedDelay = 2000)
    public void poll() {
        for (var job : store.claim(50, Duration.ofMinutes(2))) {
            try {
                emailClient.send(job.recipient(), job.payload());
                store.markDone(job.id());
            } catch (Exception e) {
                store.markFailed(job.id(), job.attempts() >= MAX_ATTEMPTS);
            }
        }
    }
}

poll() has no @Transactional. Each store call is its own short transaction, and I like that the boundaries are visible by reading the store class.

Pattern 2: leases for crashed workers

Once the lock is gone, a pod killed mid-batch leaves its rows as RUNNING forever. locked_until handles that: a second scheduled method puts expired jobs back.

@Scheduled(fixedDelay = 30000)
@Transactional
public void reclaimExpired() {
    jdbc.update("""
            UPDATE notification_job
            SET status = 'PENDING', locked_until = NULL
            WHERE status = 'RUNNING' AND locked_until < now()
            """);
}

Put this in NotificationJobStore (or inject the JdbcTemplate where you use it), and keep the lease longer than your slowest job. If a job takes 3 minutes and the lease is 2, another pod picks it up while the first is still working and the duplicates are back.

Even with a sound lease, delivery is at least once. A pod can send the email, die before markDone, and the job runs again. Pass the job id as an idempotency key if your email provider supports one, or record sent ids in a table with a unique constraint.

Keeping the table healthy

A queue table is written constantly, and every UPDATE in PostgreSQL leaves a dead row version behind. Here a job is updated two or three times (claim, done, sometimes a retry). Autovacuum starts when dead tuples exceed autovacuum_vacuum_threshold (default 50) plus autovacuum_vacuum_scale_factor (default 0.2) times the row count. With 5 million rows in the table, that is about a million dead tuples before vacuum even starts, and your pollers wade through them meanwhile.

Both settings can be overridden per table:

ALTER TABLE notification_job SET (
    autovacuum_vacuum_scale_factor = 0.01,
    autovacuum_vacuum_threshold = 1000
);

Deleting or archiving DONE rows on a schedule helps more, since a small table needs little tuning. Long-running transactions elsewhere in the database also stop vacuum from removing recent dead rows, and a queue table is the first place you'll feel it.

Gotchas and when not to use this

Situation	What happens
Strict ordering needed	Each worker gets whatever is free, so job 51 can finish before job 12. Per-customer ordering needs partitioning by key.
`ORDER BY` with a locking clause under READ COMMITTED	The docs say rows can come back out of order if a lock wait happens. Fine for a queue, wrong for a ledger.
`DISTINCT`, `GROUP BY`, `UNION`	Locking clauses can't be combined with them. Keep the claim query simple.
Very high throughput	Each job costs several writes plus WAL. Tens of thousands of jobs per second belongs on Kafka or SQS.
Polling latency	With `fixedDelay = 2000` a new job waits up to 2 s. `LISTEN/NOTIFY` can wake workers sooner but adds moving parts.
Poison jobs	Without `attempts` and a `FAILED` state, one bad row retries forever.

When the jobs come from a database write anyway, I prefer this to a new broker, because you can insert the job in the same transaction as the business change. That gives you outbox behaviour with no extra infrastructure.

Checklist

Check	Why
Claim with `FOR UPDATE SKIP LOCKED`	Pods stop receiving the same rows
Commit the claim before doing the work	No row locks or pooled connection held during slow I/O
Store a lease and reclaim expired rows	Crashed pods don't strand jobs
Cap `attempts`, keep a `FAILED` state	Poison messages stop looping
Make the handler idempotent	Delivery is at least once
Partial index on `status = 'PENDING'`	Polling cost doesn't grow with history
Per-table autovacuum settings, purge `DONE` rows	Dead tuples don't pile up

The Silent Killer in Spring Boot: Why @Transactional Around Third-Party APIs Exhausts Your Connection Pool

Shreyash Kakde — Mon, 28 Sep 2026 20:11:05 +0000

It’s 03:00 PM on a Tuesday. Your monitoring dashboards go red.

Incoming requests are timing out, HTTP 504 errors are spiking, and users are reporting that the checkout screen hangs indefinitely.

You pull up your metrics:

Application CPU: 7%
PostgreSQL CPU: 4%
JVM Heap: 30% used (GC pauses under 15ms)
Database Deadlocks: 0

Your servers are practically idling. Yet the application is completely unresponsive.

You check the application logs and find hundreds of stack traces ending with this exact line:

org.springframework.transaction.CannotCreateTransactionException: 
Could not open JPA EntityManager for transaction; 
nested exception is org.hibernate.exception.JDBCConnectionException: 
Unable to acquire JDBC Connection
    ...
Caused by: java.sql.SQLTransientConnectionException: 
HikariPool-1 - Connection is not available, request timed out after 30000ms.

Your HikariCP connection pool is 100% saturated. Every single connection is checked out. New threads wait 30 seconds for a connection, time out, and die. Kubernetes health probes (/actuator/health) attempt to query the database, fail to acquire a connection, mark the pods as unready, and terminate them.

The restart cascade begins.

The culprit isn’t a slow SQL query or an unindexed table. It is an innocent-looking @Transactional annotation sitting on a method that makes a third-party HTTP call.

The Innocent Bug: An Autopsy
What Happens Under the Hood
The Math of Pool Exhaustion (Little’s Law)
The Illusion of Distributed Atomicity
The Fix: Decoupling Transactions from Network I/O
- Pattern 1: Programmatic Boundaries with TransactionTemplate
- Pattern 2: Domain Events & @TransactionalEventListener
- Pattern 3: The Transactional Outbox Pattern
Defensive Configuration: Catching Connection Leaks Before Production
- 1. Enable HikariCP Leak Detection
- 2. Configure Strict HTTP Client Timeouts
- 3. Delay Connection Acquisition with Hibernate
Summary Checklist

The Innocent Bug: An Autopsy

Here is the exact pattern that causes this outage. It exists in thousands of production Spring Boot codebases right now:

@Service
public class CheckoutService {

    private final UserRepository userRepository;
    private final OrderRepository orderRepository;
    private final PaymentGatewayClient stripeClient; // Feign, RestClient, or WebClient

    @Transactional
    public OrderConfirmation processOrder(CheckoutRequest request) {
        // Step 1: Read user from database (~2ms)
        User user = userRepository.findById(request.userId())
            .orElseThrow(() -> new UserNotFoundException(request.userId()));

        // Step 2: Create initial order record (~3ms)
        Order order = orderRepository.save(new Order(user, request.items()));

        // Step 3: Call external payment provider over HTTPS (~1,800ms)
        PaymentResult payment = stripeClient.charge(
            user.getStripeCustomerId(), 
            request.amount()
        );

        // Step 4: Update order status (~2ms)
        order.markPaid(payment.transactionId());
        return new OrderConfirmation(order.getId(), "SUCCESS");
    }
}

At first glance, this code looks clean. Developers add @Transactional because they want the entire checkout flow to be atomic: “If the payment fails, I want the order row rolled back.”

To understand why this destroys your application under load, you have to follow what happens at the JDBC connection layer.

What Happens Under the Hood

When Spring encounters @Transactional, it coordinates with JpaTransactionManager and your connection pool (HikariCP).

sequenceDiagram
    autonumber
    participant App as CheckoutService
    participant Pool as HikariCP Pool (Size: 10)
    participant DB as PostgreSQL
    participant Stripe as External Payment API

    App->>Pool: Borrow Connection
    Pool-->>App: Connection #1 Checked Out
    App->>DB: BEGIN TRANSACTION
    App->>DB: SELECT * FROM users WHERE id = ...
    App->>DB: INSERT INTO orders ...

    Note over App,Stripe: Thread pauses in socketRead0 waiting for network packet!
    App->>Stripe: POST https://api.stripe.com/v1/charges (Network I/O)
    Note over Pool: Connection #1 sits 100% IDLE in DB, but HELD by thread for 1.8 seconds!

    Stripe-->>App: HTTP 200 OK (After 1,800ms)
    App->>DB: UPDATE orders SET status = 'PAID' ...
    App->>DB: COMMIT TRANSACTION
    App->>Pool: Return Connection #1

Step 1 (userRepository.findById): Hibernate borrows a physical java.sql.Connection from HikariCP and issues BEGIN.
Step 2 (orderRepository.save): The INSERT statement executes. Total database time elapsed: 5 milliseconds.
Step 3 (stripeClient.charge): The thread halts its execution and waits for a TLS handshake, network routing, and Stripe’s server processing. Total time elapsed: 1,800 milliseconds.
Throughout those 1,800 milliseconds, the physical database connection remains locked to this thread. The database is doing zero work. The connection is held hostage simply because the Java thread hasn't exited the @Transactional method.
Step 4: The UPDATE executes, COMMIT is sent, and the connection is finally returned to the pool.

Total transaction time: 1,807 milliseconds.

Time the database actually did work: 7 milliseconds.

Time spent holding the connection while waiting for network packets: 1,800 milliseconds (99.6%).

The Math of Pool Exhaustion (Little’s Law)

HikariCP’s default pool size is 10 connections.

Let’s calculate maximum system throughput under normal database operations versus when an external API is wrapped in a transaction:

Maximum Throughput = Pool Size / Average Connection Hold Time

Scenario A: Fast Database Transactions (No External I/O)

Average connection hold time: 5ms
Maximum throughput: 10 / 0.005s = 2,000 requests/second
Ten connections easily handle a high-traffic e-commerce store.

Scenario B: Database Transaction Wrapping an HTTP Call

Average connection hold time: 1,500ms (1.5s)
Maximum throughput: 10 / 1.5s = 6.6 requests/second

6.6 requests per second.

If just 7 concurrent users click "Checkout" at the same instant, all 10 HikariCP connections are occupied.

The 8th user’s request waits. HikariCP’s default connection-timeout is 30,000ms (30 seconds). While that 8th thread waits, subsequent requests pile up in Tomcat's thread pool. Within 10 seconds:

Tomcat exhausts its worker threads (default 200).
Incoming HTTP requests are rejected at the TCP socket layer.
Actuator health checks timeout.
Your monitoring tool alerts you that the entire service is down.

All for 7 concurrent checkout requests.

The Illusion of Distributed Atomicity

The most dangerous part of this pattern is that it does not even provide the safety developers think it does.

Consider this failure mode:

sequenceDiagram
    autonumber
    participant App as Spring Boot
    participant DB as PostgreSQL
    participant Stripe as External API

    App->>DB: INSERT INTO orders (status = PENDING)
    App->>Stripe: POST /charges ($100)
    Stripe-->>App: 200 OK (Card Charged!)
    Note over App,DB: Database network blip, disk full, or constraint violation!
    App->>DB: COMMIT fails or Connection drops!
    DB-->>App: ROLLBACK
    Note over App,Stripe: Money taken from customer, but NO order in database!

Spring opens a transaction.
The order is inserted.
Stripe charges the customer's credit card \$100.
The database connection drops, a unique constraint fails, or the pod runs out of memory before COMMIT finishes.
The local database transaction rolls back.

The order does not exist in your database. But Stripe successfully charged the customer's card.

A local database transaction (@Transactional) has zero control over an external HTTP service. It cannot roll back a network call that already reached another server. Wrapping third-party calls inside @Transactional gives you the illusion of distributed consistency while actively degrading your system availability.

The Fix: Decoupling Transactions from Network I/O

To eliminate this vulnerability, your database transactions must only span database operations. Network I/O must live outside transaction boundaries.

Here are the three production-proven patterns to accomplish this.

Pattern 1: Programmatic Boundaries with `TransactionTemplate`

Instead of annotating the entire method with @Transactional, use Spring's TransactionTemplate to create micro-transactions around only the database reads and writes.

@Service
public class SafeCheckoutService {

    private static final Logger log = LoggerFactory.getLogger(SafeCheckoutService.class);

    private final UserRepository userRepository;
    private final OrderRepository orderRepository;
    private final PaymentGatewayClient stripeClient;
    private final TransactionTemplate transactionTemplate;

    public SafeCheckoutService(UserRepository userRepository,
                               OrderRepository orderRepository,
                               PaymentGatewayClient stripeClient,
                               TransactionTemplate transactionTemplate) {
        this.userRepository = userRepository;
        this.orderRepository = orderRepository;
        this.stripeClient = stripeClient;
        this.transactionTemplate = transactionTemplate;
    }

    public OrderConfirmation processOrder(CheckoutRequest request) {
        // Phase 1: Short local transaction to reserve and initialize (~5ms)
        Order pendingOrder = transactionTemplate.execute(status -> {
            User user = userRepository.findById(request.userId())
                .orElseThrow(() -> new UserNotFoundException(request.userId()));

            Order order = new Order(user, request.items());
            order.setStatus(OrderStatus.PAYMENT_PENDING);
            return orderRepository.save(order);
        });

        // Connection returned to HikariCP pool HERE!
        // Phase 2: External HTTP call outside any database transaction (~1,500ms)
        PaymentResult payment;
        try {
            payment = stripeClient.charge(
                request.stripeCustomerId(),
                request.amount(),
                // Crucial: Use order ID as idempotency key for network retries
                "checkout-order-" + pendingOrder.getId()
            );
        } catch (PaymentException e) {
            log.error("Payment failed for order {}", pendingOrder.getId(), e);
            markOrderFailed(pendingOrder.getId(), e.getMessage());
            throw e;
        }

        // Phase 3: Short local transaction to finalize state (~4ms)
        return transactionTemplate.execute(status -> {
            Order order = orderRepository.findById(pendingOrder.getId()).orElseThrow();
            order.markPaid(payment.transactionId());
            orderRepository.save(order);
            return new OrderConfirmation(order.getId(), "SUCCESS");
        });
    }

    private void markOrderFailed(Long orderId, String reason) {
        transactionTemplate.executeWithoutResult(status -> {
            orderRepository.findById(orderId).ifPresent(order -> {
                order.setStatus(OrderStatus.FAILED);
                order.setFailureReason(reason);
                orderRepository.save(order);
            });
        });
    }
}

The Difference in Connection Usage

graph LR
    subgraph SafeExecution["Safe Multi-Phase Execution"]
        A["Phase 1: DB TX (~5ms)"] -->|Connection Released| B["Phase 2: Network I/O (~1,500ms)<br>ZERO DB Connections Held"]
        B -->|Borrow Connection| C["Phase 3: DB TX (~4ms)"]
    end

During the 1,500ms HTTP call to Stripe, zero database connections are held. Your pool of 10 connections remains completely open to serve other users, health checks, and read queries.

Pattern 2: Domain Events & `@TransactionalEventListener`

If the third-party call is a non-blocking side-effect (such as sending an email via SendGrid, dispatching a webhook, or emitting an audit log), use Spring's application events.

By setting the listener phase to AFTER_COMMIT, Spring guarantees that:

The external API is called only after the local database transaction has safely committed.
The database connection has already been returned to the Hikari pool before the network call starts.

The Event Definition

public record OrderPlacedEvent(Long orderId, String customerEmail, BigDecimal totalAmount) {}

The Transactional Service

@Service
public class OrderPlacementService {

    private final OrderRepository orderRepository;
    private final ApplicationEventPublisher eventPublisher;

    public OrderPlacementService(OrderRepository orderRepository, 
                                 ApplicationEventPublisher eventPublisher) {
        this.orderRepository = orderRepository;
        this.eventPublisher = eventPublisher;
    }

    @Transactional
    public Long placeOrder(CreateOrderCommand cmd) {
        Order order = orderRepository.save(new Order(cmd));

        // Publish event inside transaction
        eventPublisher.publishEvent(new OrderPlacedEvent(
            order.getId(), 
            cmd.customerEmail(), 
            cmd.totalAmount()
        ));

        return order.getId();
    } // Connection is COMMITTED and RELEASED here!
}

The Event Listener

@Component
public class OrderNotificationListener {

    private static final Logger log = LoggerFactory.getLogger(OrderNotificationListener.class);
    private final EmailClient emailClient; // External HTTP Client (SendGrid, Postmark)

    public OrderNotificationListener(EmailClient emailClient) {
        this.emailClient = emailClient;
    }

    @Async // Run in background thread pool to avoid blocking HTTP request
    @TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
    public void onOrderPlaced(OrderPlacedEvent event) {
        log.info("Sending order confirmation email for order {}", event.orderId());
        // This HTTP call runs with NO database transaction or connection attached!
        emailClient.sendOrderConfirmation(event.customerEmail(), event.orderId());
    }
}

If the database transaction fails and rolls back, onOrderPlaced will never execute. No customer will ever receive a "Your order is confirmed!" email for an order that failed to save.

Pattern 3: The Transactional Outbox Pattern for Mission-Critical Integrations

When an external integration must complete reliably (such as syncing with SAP, Salesforce, or dispatching an event to Kafka), use the Transactional Outbox pattern.

Instead of calling the external API synchronously:

Write the business entity and an outbox record in the same local transaction (takes < 5ms).
A separate background worker reads unprocessed outbox records and delivers them to the external API with retries and exponential backoff.

graph TD
    subgraph ClientRequest["Client Request (Fast)"]
        A["User Request"] --> B["Save Order + Outbox Row in 1 TX (~5ms)"]
        B --> C["Return 200 OK to User immediately"]
    end

    subgraph BackgroundWorker["Background Worker (Resilient)"]
        D[("Outbox Table")] -->|Poll / Listen| E["Outbox Poller Thread"]
        E -->|Call External API| F["Stripe / Salesforce / SAP"]
        F -->|200 OK| G["Mark Outbox Row as PROCESSED"]
    end

This insulates your customer-facing response times from third-party outages and guarantees at-least-once delivery.

Defensive Configuration: Catching Connection Leaks Before Production

Even with careful code reviews, an unvetted library or rogue @Transactional can slip through. Add these three guardrails to your Spring Boot configuration:

1. Enable HikariCP Leak Detection

HikariCP has a built-in leak detection mechanism that logs a stack trace whenever a thread holds a connection longer than a set threshold:

# application.properties
# Log a warning if a thread holds a connection longer than 3 seconds
spring.datasource.hikari.leak-detection-threshold=3000

When a thread holds a connection for longer than 3,000ms, HikariCP outputs a warning with the full stack trace:

WARN 12940 --- [pool-thread-2] com.zaxxer.hikari.pool.ProxyLeakTask: 
Connection leak detection triggered for org.postgresql.jdbc.PgConnection@4a2f8d on thread http-nio-8080-exec-4, 
stack trace follows:
    at com.example.service.CheckoutService.processOrder(CheckoutService.java:42)

You can pinpoint the exact line of code holding the connection.

2. Configure Strict HTTP Client Timeouts

Never use default timeouts on your HTTP clients (RestClient, WebClient, Feign, or Apache HttpClient). Default read timeouts are often infinite or 60+ seconds.

@Configuration
public class HttpClientConfig {

    @Bean
    public RestClient stripeRestClient() {
        SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
        factory.setConnectTimeout(Duration.ofSeconds(2));
        factory.setReadTimeout(Duration.ofSeconds(4)); // Fail fast if external server hangs

        return RestClient.builder()
            .requestFactory(factory)
            .baseUrl("https://api.stripe.com")
            .build();
    }
}

If Stripe degrades, your client fails in 4 seconds instead of hanging for minutes and starving your thread pools.

3. Delay Connection Acquisition with Hibernate

By default, Hibernate might acquire a physical database connection as soon as a transaction begins. You can instruct Hibernate to delay physical connection acquisition until the first actual SQL query is issued:

spring.jpa.properties.hibernate.connection.provider_disables_autocommit=true

This prevents early connection checkout if your service performs non-database validation logic before its first SQL query.

Summary Checklist

Anti-Pattern	Recommended Solution
External HTTP call inside `@Transactional`	Use `TransactionTemplate` to split into short pre- and post-transactions
Email, notifications, or metrics inside `@Transactional`	Use `@TransactionalEventListener(phase = AFTER_COMMIT)`
Long-running batch or external sync	Transactional Outbox pattern with background worker
Undetected connection hogs	Set `spring.datasource.hikari.leak-detection-threshold=3000`
Hanging HTTP calls	Set explicit 2–4s connect and read timeouts on all HTTP clients

A database connection is one of the most expensive and scarce resources in your entire architecture.

Treat it like an in-memory lock: acquire it at the last possible millisecond, do your SQL work, and release it immediately. Keep your network calls out of your database transactions, and your connection pool will effortlessly handle traffic surges without breaking a sweat.

Eliminating Race Conditions in Clustered Spring Boot with PostgreSQL Advisory Locks

Shreyash Kakde — Mon, 28 Sep 2026 20:00:28 +0000

It’s 02:14 AM. Your alert manager fires: an account balance dropped below zero, or a user received two trial credits instead of one.

You trace the logs across your Kubernetes cluster. Two pods received near-simultaneous webhook calls for the exact same entity—5 milliseconds apart. Both pods executed:

Pod A: Reads balance ($100) -> Validates -> Dispatches payout -> Updates balance ($0)
Pod B: Reads balance ($100) -> Validates -> Dispatches payout -> Updates balance ($0)

Both pods saw the balance before either could update it. Two payouts occurred; the business took the loss.

If you have spent any time maintaining distributed systems, you know this story. But when teams sit down to fix it, they often leap straight into architectural extremes—either slapping a useless synchronized keyword onto a Spring service method, or dragging in an entire Redis cluster with Redisson just to serialize one critical action.

There is a sweet spot right in front of you. If PostgreSQL is your primary datastore, you already possess a robust distributed lock manager: PostgreSQL Advisory Locks.

Why Standard Concurrency Tools Break in a Cluster
What Are PostgreSQL Advisory Locks?
- Session-Level vs. Transaction-Level
- The HikariCP Connection Trap
The Lock Functions You Need to Know
Designing a Production-Ready Implementation
- 1. Generating 64-Bit Keys from Arbitrary Strings
- 2. The Advisory Lock Repository
- 3. A Clean Functional Lock Template
- 4. A Realistic Scenario: First-Time Welcome Reward
The "Secret Weapon": Lock Timeouts in PostgreSQL
Verifying Concurrency with Testcontainers
Production Gotchas and When NOT to Use Them
Architectural Comparison: Picking the Right Tool
Summary

Why Standard Concurrency Tools Break in a Cluster

Before looking at advisory locks, let's understand why traditional tools fall short.

sequenceDiagram
    autonumber
    participant Client
    participant Pod1 as Spring Boot (Pod 1)
    participant Pod2 as Spring Boot (Pod 2)
    participant DB as PostgreSQL

    Client->>Pod1: POST /rewards/claim (User 42)
    Client->>Pod2: POST /rewards/claim (User 42)
    Note over Pod1,Pod2: JVM locks (synchronized) only protect within the same node
    Pod1->>DB: SELECT has_claimed FROM rewards WHERE user_id = 42
    Pod2->>DB: SELECT has_claimed FROM rewards WHERE user_id = 42
    DB-->>Pod1: false
    DB-->>Pod2: false
    Pod1->>DB: INSERT INTO rewards (user_id) VALUES (42)
    Pod2->>DB: INSERT INTO rewards (user_id) VALUES (42)
    Note over DB: Race condition! Without constraints, duplicate rows occur.

1. In-Memory JVM Locks (`synchronized`, `ReentrantLock`)

Java's native synchronization primitives coordinate threads within a single JVM. The moment you run two pods behind an Application Load Balancer, your in-memory lock table is completely partitioned. Pod 1 knows nothing about Pod 2’s monitors.

2. Pessimistic Row Locking (`SELECT ... FOR UPDATE`)

Row-level locks are powerful, but they come with two severe limitations:

The row must already exist. If your task is "create this user’s default ledger account if it doesn't exist yet," there is no row to lock. You get a classic "check-then-act" race.
Table bloat and domain coupling. Locking a row acquires locks within PostgreSQL’s tuple storage system, locking out concurrent readers/writers on adjacent columns, creating MVCC churn, and coupling business workflows directly to table rows.

3. Redis / Redisson Distributed Locks

Spinning up Redis to handle distributed locking is common, but it introduces:

Extra operational complexity (failover clusters, sentinel/cluster topology, backup, patching).
The dual-system failure problem: network partitions between your app, Redis, and PostgreSQL can leave locks dangling or create split-brain states unless you implement fencing tokens correctly.
Unnecessary infrastructure cost if your PostgreSQL database is already provisioned and has plenty of capacity.

What Are PostgreSQL Advisory Locks?

PostgreSQL advisory locks are application-defined locks managed directly in the database server's memory (shared_buffers / lock table).

The key insight: PostgreSQL doesn’t attach any data semantics to an advisory lock. It doesn't lock a row, a page, or a table. It simply holds an integer key in memory that other connections can check or wait on.

graph TD
    subgraph PostgreSQL Server
        A[Lock Manager Shared Memory]
        B[(Tuples / Tables)]
    end

    subgraph Clients
        P1[Pod 1 Connection]
        P2[Pod 2 Connection]
    end

    P1 -->|pg_try_advisory_xact_lock: 9481029| A
    P2 -.->|pg_try_advisory_xact_lock: 9481029 rejected!| A
    P1 -->|Only writes data after lock is granted| B

Session-Level vs. Transaction-Level

PostgreSQL provides two flavors of advisory locks. Confusing them in a connection-pooled application like Spring Boot will cause catastrophic bugs.

Feature	Session-Level (`pg_advisory_lock`)	Transaction-Level (`pg_advisory_xact_lock`)
Lifecycle	Held until manually unlocked or connection closes	Automatically released when transaction ends (COMMIT/ROLLBACK)
Unlock mechanism	Explicit call to `pg_advisory_unlock`	PostgreSQL handles it on transaction completion
Connection Pool Safety	High risk! Leaks locks if connection returns to HikariCP	Safe! Cannot leak beyond transaction boundaries
PgBouncer Compatibility	Incompatible with transaction pooling mode	Fully compatible with transaction pooling mode

The HikariCP Connection Trap

With HikariCP (Spring Boot’s default pool), physical database connections are reused across different HTTP requests and threads.

If you acquire a session-level lock (pg_advisory_lock) on Connection A and an unhandled exception prevents your unlock code from running, Connection A goes back into the Hikari pool while still holding the lock in Postgres.

Later, another thread picks up Connection A, inheriting the lock. Worse, other threads requesting the lock on other connections hang indefinitely.

For Spring Boot applications, transaction-level advisory locks (pg_advisory_xact_lock) are almost always the correct choice. When Spring’s transaction commits or rolls back, PostgreSQL automatically frees the lock. Even if your pod crashes mid-execution with kill -9, the TCP socket drops, the transaction aborts, and PostgreSQL clears the lock immediately.

The Lock Functions You Need to Know

PostgreSQL exposes a compact family of advisory functions:

pg_advisory_xact_lock(bigint): Blocks until the lock is acquired. Automatically released at the end of the current transaction.
pg_try_advisory_xact_lock(bigint): Non-blocking. Returns true if acquired immediately, false if another transaction holds it.
pg_advisory_xact_lock(integer, integer): 2-part key variant. Useful for composite IDs like (tenant_id, entity_id).

Designing a Production-Ready Implementation

Let's build a clean, idiomatic locking mechanism for Spring Boot.

1. Generating 64-Bit Keys from Arbitrary Strings

PostgreSQL advisory locks accept either a single 64-bit integer (bigint / Java long) or two 32-bit integers (int).

In enterprise applications, we usually want to lock on domain identifiers: "wallet:claim:user_99214" or "order:settle:ORD-2026-901". We need a deterministic, low-collision function to map any String to a long.

A truncated SHA-256 hash or MurmurHash3 works well here:

package com.example.locking.util;

import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

public final class LockKeyGenerator {

    private LockKeyGenerator() {}

    /**
     * Hashes a string identifier to a 64-bit signed long for PostgreSQL advisory locks.
     */
    public static long toLongKey(String businessKey) {
        if (businessKey == null || businessKey.isBlank()) {
            throw new IllegalArgumentException("Lock key must not be blank");
        }
        try {
            MessageDigest digest = MessageDigest.getInstance("SHA-256");
            byte[] hash = digest.digest(businessKey.getBytes(StandardCharsets.UTF_8));
            // Read the first 8 bytes of the SHA-256 hash as a long
            return ByteBuffer.wrap(hash).getLong();
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException("SHA-256 algorithm missing from JVM", e);
        }
    }
}

What about hash collisions?

A 64-bit space allows for $1.8 \times 10^{19}$ possible values. If two completely unrelated keys collide (which is exceptionally rare), the consequence is benign: one process briefly waits for the other, or fails fast. Data corruption never occurs.

2. The Advisory Lock Repository

Instead of fighting JPA's entity lifecycle, use JdbcTemplate for raw advisory lock calls. It ensures the query runs on the exact physical connection bound to Spring’s active transaction.

package com.example.locking.repository;

import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Repository;

@Repository
public class PostgresAdvisoryLockRepository {

    private final JdbcTemplate jdbcTemplate;

    public PostgresAdvisoryLockRepository(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    /**
     * Non-blocking attempt to acquire a transaction-scoped advisory lock.
     * Returns true if granted, false if already held by another transaction.
     */
    public boolean tryAcquireTransactionLock(long lockKey) {
        Boolean acquired = jdbcTemplate.queryForObject(
            "SELECT pg_try_advisory_xact_lock(?)",
            Boolean.class,
            lockKey
        );
        return Boolean.TRUE.equals(acquired);
    }

    /**
     * Blocking call to acquire a transaction-scoped advisory lock.
     * Waits until the lock is available.
     */
    public void acquireTransactionLock(long lockKey) {
        jdbcTemplate.queryForObject(
            "SELECT pg_advisory_xact_lock(?)",
            Void.class,
            lockKey
        );
    }
}

3. A Clean Functional Lock Template

Annotations (like @AdvisoryLocked) look nice on paper, but aspect-oriented locks often obscure transaction boundaries. If the lock is acquired before @Transactional begins, or released after it commits, race conditions sneak back in.

A functional template makes the transaction and lock scope explicit:

package com.example.locking.service;

import com.example.locking.exception.LockAcquisitionException;
import com.example.locking.repository.PostgresAdvisoryLockRepository;
import com.example.locking.util.LockKeyGenerator;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

import java.util.function.Supplier;

@Component
public class AdvisoryLockTemplate {

    private static final Logger log = LoggerFactory.getLogger(AdvisoryLockTemplate.class);

    private final PostgresAdvisoryLockRepository lockRepository;

    public AdvisoryLockTemplate(PostgresAdvisoryLockRepository lockRepository) {
        this.lockRepository = lockRepository;
    }

    /**
     * Executes the supplier within a MANDATORY or NEW transaction holding an advisory lock.
     * Fails fast if the lock cannot be acquired immediately.
     */
    @Transactional(propagation = Propagation.REQUIRED)
    public <T> T executeWithTryLock(String lockKey, Supplier<T> task) {
        long numericKey = LockKeyGenerator.toLongKey(lockKey);
        log.debug("Attempting to acquire lock for key '{}' (hash: {})", lockKey, numericKey);

        boolean acquired = lockRepository.tryAcquireTransactionLock(numericKey);
        if (!acquired) {
            log.warn("Failed to acquire lock for key: {}. Another process is executing.", lockKey);
            throw new LockAcquisitionException("Unable to acquire lock for: " + lockKey);
        }

        log.debug("Lock successfully acquired for key: {}", lockKey);
        try {
            return task.get();
        } finally {
            // Note: pg_advisory_xact_lock is freed automatically when the transaction completes.
            log.debug("Task completed. Lock for key '{}' will release on TX commit/rollback.", lockKey);
        }
    }
}

Custom exception for clean HTTP 409 (Conflict) mappings:

package com.example.locking.exception;

public class LockAcquisitionException extends RuntimeException {
    public LockAcquisitionException(String message) {
        super(message);
    }
}

4. A Realistic Scenario: First-Time Welcome Reward

Let’s look at a realistic use case: granting a first-time signup bonus or wallet reward.

Two requests hit different pods at the same time. The user record has no reward row yet, so SELECT ... FOR UPDATE cannot help us.

sequenceDiagram
    autonumber
    participant Pod1 as Pod 1 (Lock Acquired)
    participant Pod2 as Pod 2 (Lock Denied)
    participant DB as Postgres Lock Manager

    Pod1->>DB: SELECT pg_try_advisory_xact_lock(hash("wallet:reward:user-100"))
    DB-->>Pod1: true (Lock Granted)

    Pod2->>DB: SELECT pg_try_advisory_xact_lock(hash("wallet:reward:user-100"))
    DB-->>Pod2: false (Locked by Pod 1)
    Pod2-->>Pod2: Throw LockAcquisitionException (Fast Fail)

    Pod1->>Pod1: Check if already rewarded (false)
    Pod1->>Pod1: Credit Wallet & Insert Audit Log
    Pod1->>DB: COMMIT Transaction
    DB-->>Pod1: Lock automatically dropped!

Here is the service implementation:

package com.example.rewards.service;

import com.example.locking.service.AdvisoryLockTemplate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.math.BigDecimal;

@Service
public class RewardClaimService {

    private static final Logger log = LoggerFactory.getLogger(RewardClaimService.class);

    private final AdvisoryLockTemplate lockTemplate;
    private final RewardRepository rewardRepository;
    private final WalletService walletService;

    public RewardClaimService(AdvisoryLockTemplate lockTemplate,
                              RewardRepository rewardRepository,
                              WalletService walletService) {
        this.lockTemplate = lockTemplate;
        this.rewardRepository = rewardRepository;
        this.walletService = walletService;
    }

    public RewardClaimResult claimWelcomeBonus(String userId) {
        String lockKey = "reward:claim:welcome:" + userId;

        return lockTemplate.executeWithTryLock(lockKey, () -> {
            // Safe critical section across the entire cluster!
            if (rewardRepository.existsByUserIdAndType(userId, "WELCOME_BONUS")) {
                log.info("User {} already received the welcome bonus", userId);
                return RewardClaimResult.alreadyClaimed();
            }

            log.info("Granting bonus to user {}", userId);
            walletService.credit(userId, new BigDecimal("25.00"));
            rewardRepository.recordClaim(userId, "WELCOME_BONUS");

            return RewardClaimResult.success(new BigDecimal("25.00"));
        });
    }
}

The "Secret Weapon": Lock Timeouts in PostgreSQL

What if you don't want to fail immediately with try_lock, but you also don't want your threads to wait indefinitely and starve your HikariCP pool?

PostgreSQL doesn’t have a pg_advisory_lock_with_timeout(key, duration) function. However, it does have SET LOCAL lock_timeout.

When set inside a transaction, lock_timeout applies to all lock requests in that transaction—including advisory locks!

public void acquireTransactionLockWithTimeout(long lockKey, int timeoutMillis) {
    // 1. Set local timeout for this transaction only
    jdbcTemplate.execute("SET LOCAL lock_timeout = '" + timeoutMillis + "ms'");

    // 2. Will wait up to timeoutMillis; throws PSQLException with SQLState 55P03 if exceeded
    jdbcTemplate.queryForObject("SELECT pg_advisory_xact_lock(?)", Void.class, lockKey);
}

If the lock cannot be acquired within the specified duration, Postgres aborts the lock request with:
ERROR: canceling statement due to lock timeout (SQLState: 55P03)

This gives you a bounded, distributed queue without needing complex polling or backoff logic in Java.

Verifying Concurrency with Testcontainers

You should never trust distributed concurrency code without an integration test that actually fires simultaneous threads against a real PostgreSQL instance.

Here is a full Testcontainers-based test verifying that concurrent requests for the same lock key are cleanly serialized or rejected:

package com.example.locking;

import com.example.locking.exception.LockAcquisitionException;
import com.example.locking.service.AdvisoryLockTemplate;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

import static org.assertj.core.api.Assertions.assertThat;

@SpringBootTest
@Testcontainers
class AdvisoryLockIntegrationTest {

    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16-alpine");

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    private AdvisoryLockTemplate lockTemplate;

    @Test
    @DisplayName("Simultaneous threads on the same key must only allow one execution")
    void testConcurrentAccess() throws Exception {
        int threadCount = 8;
        ExecutorService executor = Executors.newFixedThreadPool(threadCount);
        CountDownLatch readyLatch = new CountDownLatch(threadCount);
        CountDownLatch startLatch = new CountDownLatch(1);

        AtomicInteger successCounter = new AtomicInteger(0);
        AtomicInteger lockDeniedCounter = new AtomicInteger(0);

        String sharedResourceKey = "account:sync:USR-123";

        for (int i = 0; i < threadCount; i++) {
            executor.submit(() -> {
                readyLatch.countDown();
                try {
                    startLatch.await(); // Synchronize thread release

                    lockTemplate.executeWithTryLock(sharedResourceKey, () -> {
                        successCounter.incrementAndGet();
                        // Hold lock briefly to force collision
                        Thread.sleep(100);
                        return null;
                    });
                } catch (LockAcquisitionException e) {
                    lockDeniedCounter.incrementAndGet();
                } catch (Exception e) {
                    // unexpected
                }
            });
        }

        // Wait until all threads are prepped
        readyLatch.await(5, TimeUnit.SECONDS);
        // Fire all threads simultaneously
        startLatch.countDown();

        executor.shutdown();
        boolean completed = executor.awaitTermination(5, TimeUnit.SECONDS);
        assertThat(completed).isTrue();

        // Exactly one thread should succeed; remaining 7 must be rejected
        assertThat(successCounter.get()).isEqualTo(1);
        assertThat(lockDeniedCounter.get()).isEqualTo(threadCount - 1);
    }
}

Production Gotchas and When NOT to Use Them

PostgreSQL advisory locks are powerful, but senior engineers understand where tools fail. Keep these three points in mind:

1. Connection Starvation During Slow External I/O

If your locked critical section makes a slow third-party API call (e.g., waiting 5 seconds for Stripe or PayPal), your application keeps the underlying database connection checked out from HikariCP for that entire duration.

graph LR
    subgraph Anti-Pattern
        A[Acquire Advisory Lock] --> B[Hold DB Connection]
        B --> C[Wait for Slow 3rd Party HTTP API 3-5s]
        C --> D[HikariCP Pool Starvation!]
    end

Rule of thumb: Keep advisory-locked transactions lean. Execute validation, DB writes, and state transitions inside the lock. If you must call an external API, consider optimistic state transitions (e.g., status = PENDING_PAYMENT) committed to the database, or use an external lock manager like Redis if the database connection pool cannot absorb the latency.

2. PgBouncer and Transaction Pooling

If you use PgBouncer in front of PostgreSQL:

Session-level locks (pg_advisory_lock) are broken in pool_mode = transaction. PgBouncer may assign subsequent queries in the same client session to different Postgres server backends!
Transaction-level locks (pg_advisory_xact_lock) work flawlessly because PgBouncer pins the client to a single physical backend connection for the exact lifetime of the transaction.

3. Postgres Shared Memory Limits

Advisory locks are stored in PostgreSQL's shared memory, governed by max_locks_per_transaction (default typically 64) multiplied by max_connections.

While tens of thousands of active transaction locks are rarely an issue, do not treat advisory locks as long-term persistent flags. They are transient execution guards.

Architectural Comparison: Picking the Right Tool

Requirement	Java `synchronized`	PostgreSQL Advisory Lock	Redis (Redisson)	DB Row Lock (`SELECT FOR UPDATE`)
Clustered / Multi-Node	❌ No	✅ Yes	✅ Yes	✅ Yes
Protects Non-Existent Records	N/A	✅ Yes	✅ Yes	❌ No (requires existing row)
External Infrastructure	None	None (uses existing DB)	Requires Redis cluster	None (uses existing DB)
Auto-Cleanup on Crash	✅ Yes	✅ Yes (with `xact` locks)	⚠️ Requires TTL / watchdogs	✅ Yes
Safe with Slow HTTP I/O	✅ Yes	❌ Risks DB pool starvation	✅ Yes	❌ Risks DB pool starvation
Operational Overhead	Zero	Zero	Moderate to High	Zero

Summary

Before introducing Redis, ZooKeeper, or Hazelcast to solve distributed race conditions in your Spring Boot services, look closely at your existing PostgreSQL database.

By combining pg_try_advisory_xact_lock with Spring’s @Transactional boundaries:

You get true cluster-wide serialization across any number of pods.
You eliminate dangling locks through automatic transaction-scoped cleanup.
You save your organization from operating and monitoring an additional distributed caching cluster.

Simplicity is an architectural superpower. Sometimes the most resilient solution is the one already running in your stack.

DEV Community: Shreyash Kakde

Two Pods, One Job Table: A Safe Postgres Queue with SKIP LOCKED in Spring Boot

Table of Contents

The innocent worker

Why a plain SELECT hands out the same rows

What FOR UPDATE alone does

SKIP LOCKED

Pattern 1: claim in a short transaction, work outside it

Pattern 2: leases for crashed workers

Keeping the table healthy

Gotchas and when not to use this

Checklist

The Silent Killer in Spring Boot: Why @Transactional Around Third-Party APIs Exhausts Your Connection Pool

Table of Contents

The Innocent Bug: An Autopsy

What Happens Under the Hood

The Math of Pool Exhaustion (Little’s Law)

Scenario A: Fast Database Transactions (No External I/O)

Scenario B: Database Transaction Wrapping an HTTP Call

The Illusion of Distributed Atomicity

The Fix: Decoupling Transactions from Network I/O

Pattern 1: Programmatic Boundaries with TransactionTemplate

The Difference in Connection Usage

Pattern 2: Domain Events & @TransactionalEventListener

The Event Definition

The Transactional Service

The Event Listener

Pattern 3: The Transactional Outbox Pattern for Mission-Critical Integrations

Defensive Configuration: Catching Connection Leaks Before Production

1. Enable HikariCP Leak Detection

2. Configure Strict HTTP Client Timeouts

3. Delay Connection Acquisition with Hibernate

Summary Checklist

Eliminating Race Conditions in Clustered Spring Boot with PostgreSQL Advisory Locks

Table of Contents

Why Standard Concurrency Tools Break in a Cluster

1. In-Memory JVM Locks (synchronized, ReentrantLock)

2. Pessimistic Row Locking (SELECT ... FOR UPDATE)

3. Redis / Redisson Distributed Locks

What Are PostgreSQL Advisory Locks?

Session-Level vs. Transaction-Level

The HikariCP Connection Trap

The Lock Functions You Need to Know

Designing a Production-Ready Implementation

1. Generating 64-Bit Keys from Arbitrary Strings

2. The Advisory Lock Repository

3. A Clean Functional Lock Template

4. A Realistic Scenario: First-Time Welcome Reward

The "Secret Weapon": Lock Timeouts in PostgreSQL

Verifying Concurrency with Testcontainers

Production Gotchas and When NOT to Use Them

1. Connection Starvation During Slow External I/O

2. PgBouncer and Transaction Pooling

3. Postgres Shared Memory Limits

Architectural Comparison: Picking the Right Tool

Summary

Pattern 1: Programmatic Boundaries with `TransactionTemplate`

Pattern 2: Domain Events & `@TransactionalEventListener`

1. In-Memory JVM Locks (`synchronized`, `ReentrantLock`)

2. Pessimistic Row Locking (`SELECT ... FOR UPDATE`)