DEV Community: Dipankar Sethi

The Singleton Design Pattern: A Complete Guide for Developers

Dipankar Sethi — Fri, 06 Feb 2026 11:17:54 +0000

We've all been there. You're working on a project, and suddenly you realize you need exactly one instance of a class throughout your entire application. Maybe it's a database connection pool, a configuration manager, or a logging service. Creating multiple instances would be wasteful, confusing, or downright dangerous.

This is where the Singleton pattern comes to your rescue.
Let me share a story from my early days as a developer. I was working on an e-commerce application, and I created a new database connection every time a user made a request. Within hours of deployment, our application crashed because we'd exhausted all available database connections. A senior developer introduced me to the Singleton pattern, and it was a game-changer.

What is the Singleton Pattern?
The Singleton pattern is one of the simplest yet most commonly used design patterns. It ensures that a class has only one instance and provides a global point of access to that instance. Think of it like having exactly one CEO in a company—there can't be two people making the final decisions simultaneously.

Basic Implementation
Let's start with the most straightforward implementation:


public class DatabaseConnection {
    // Static variable to hold the single instance
    private static DatabaseConnection instance;

    // Private constructor prevents instantiation from other classes
    private DatabaseConnection() {
        System.out.println("Database connection established");
    }

    // Public method to provide access to the instance
    public static DatabaseConnection getInstance() {
        if (instance == null) {
            instance = new DatabaseConnection();
        }
        return instance;
    }

    public void executeQuery(String query) {
        System.out.println("Executing: " + query);
    }
}

// Usage
public class Application {
    public static void main(String[] args) {
        DatabaseConnection db1 = DatabaseConnection.getInstance();
        DatabaseConnection db2 = DatabaseConnection.getInstance();

        System.out.println(db1 == db2); // Output: true (same instance)
    }
}

However, this basic implementation has a problem—it's not thread-safe. In a multi-threaded environment, two threads could create separate instances if they call getInstance() simultaneously.

Thread-Safe Singleton (Eager Initialization)

public class ConfigurationManager {
    // Instance created at class loading time
    private static final ConfigurationManager instance = new ConfigurationManager();

    private ConfigurationManager() {
        // Load configuration from file
        System.out.println("Loading configuration...");
    }

    public static ConfigurationManager getInstance() {
        return instance;
    }

    public String getProperty(String key) {
        // Return configuration property
        return "value_for_" + key;
    }
}

This approach is thread-safe because the instance is created when the class is loaded, but it doesn't support lazy initialization—the instance is created even if you never use it.

Thread-Safe Singleton (Lazy Initialization with Double-Checked Locking)

public class Logger {
    private static volatile Logger instance;

    private Logger() {
        System.out.println("Logger initialized");
    }

    public static Logger getInstance() {
        if (instance == null) {
            synchronized (Logger.class) {
                if (instance == null) {
                    instance = new Logger();
                }
            }
        }
        return instance;
    }

    public void log(String message) {
        System.out.println("[LOG] " + message);
    }
}

The volatile keyword ensures that changes to the instance variable are visible to all threads immediately. The double-checked locking minimizes synchronization overhead.

Bill Pugh Singleton (Recommended Approach)
This is the most elegant and widely recommended approach:

public class CacheManager {

    private CacheManager() {
        System.out.println("Cache Manager initialized");
    }

    // Static inner class - inner classes are not loaded until they are referenced
    private static class SingletonHelper {
        private static final CacheManager INSTANCE = new CacheManager();
    }

    public static CacheManager getInstance() {
        return SingletonHelper.INSTANCE;
    }

    public void put(String key, Object value) {
        System.out.println("Caching: " + key);
    }

    public Object get(String key) {
        return "cached_value_for_" + key;
    }
}

This implementation is thread-safe, supports lazy initialization, and doesn't require synchronization.

Enum Singleton (Protection Against Reflection)

public enum ApplicationContext {
    INSTANCE;

    private String environment;

    ApplicationContext() {
        this.environment = "production";
        System.out.println("Application Context initialized");
    }

    public String getEnvironment() {
        return environment;
    }

    public void setEnvironment(String environment) {
        this.environment = environment;
    }
}

// Usage
ApplicationContext context = ApplicationContext.INSTANCE;
context.setEnvironment("development");

Enums provide serialization and thread-safety out of the box and protect against reflection attacks.

Where to Use Singleton Pattern
Perfect scenarios:

Configuration Management: Application settings that should be loaded once and accessed globally
Logger Classes: Centralized logging mechanism
Database Connection Pools: Managing a pool of reusable database connections
Cache Managers: In-memory caching systems
Thread Pools: Managing worker threads
Device Drivers: Accessing hardware resources like printers

Real-world example:

public class AppConfig {
    private static class SingletonHelper {
        private static final AppConfig INSTANCE = new AppConfig();
    }

    private Properties properties;

    private AppConfig() {
        properties = new Properties();
        loadConfiguration();
    }

    private void loadConfiguration() {
        // Load from file, environment variables, etc.
        properties.setProperty("app.name", "MyApplication");
        properties.setProperty("max.connections", "100");
        properties.setProperty("timeout", "30000");
    }

    public static AppConfig getInstance() {
        return SingletonHelper.INSTANCE;
    }

    public String get(String key) {
        return properties.getProperty(key);
    }
}

Benefits of Singleton Pattern

Controlled Access: Only one instance exists, ensuring consistent state across the application.
Memory Efficiency: Saves memory by preventing creation of multiple instances of heavy objects.
Global Access Point: Easy access from anywhere in the application without passing references.
Lazy Initialization: Instance created only when needed (in some implementations).
Thread-Safe Operations: Properly implemented Singletons handle multi-threading concerns.

Cons and Pitfalls

Testing Challenges: Singletons introduce global state, making unit testing difficult. You can't easily mock or replace the instance.

// Difficult to test
public class OrderService {
    public void processOrder(Order order) {
        Logger.getInstance().log("Processing order: " + order.getId());
        // Hard to verify logging behavior in tests
    }
}

Hidden Dependencies: Classes using Singletons don't declare their dependencies explicitly, violating the Dependency Inversion Principle.
Violates Single Responsibility Principle: The class manages both its core functionality and its instance creation.
Global State: Can lead to tight coupling and make code harder to reason about.
Concurrency Issues: If not implemented correctly, can cause race conditions.
Serialization Problems: Special handling needed to prevent creating multiple instances during deserialization.

When to Avoid Singleton

You need multiple instances in the future (even if you think you only need one now)
The class has mutable state that different parts of the application modify
You're writing testable code and need to inject dependencies
The object is lightweight and creating multiple instances isn't a problem
You're working in a distributed system where "one instance" is ambiguous

Better alternatives:

Dependency Injection frameworks (Spring, Guice)
Factory patterns with instance management
Static utility classes (for stateless operations)

How to Recognize the Right Scenario
Ask yourself these questions:

Do I truly need exactly one instance? Be honest. "One instance per application" is different from "I think one is enough."
Is this a resource that must be shared? Database connections pools: yes. User session: no.
Will this global state cause problems? If different parts of your application need different configurations, Singleton isn't the answer.
Can I test this easily? If your Singleton makes testing painful, consider dependency injection instead.
Is this a stateless utility? If yes, you might not need Singleton at all—just static methods.

Final Thoughts
The Singleton pattern is a powerful tool, but like any tool, it can be misused. I've seen projects where everything was a Singleton "just in case," leading to a maintenance nightmare. I've also seen projects that avoided Singletons entirely and ended up with fragmented resource management.

The key is understanding your specific use case. Is it a genuinely shared resource? Will having one instance truly benefit your application? Can you test it effectively?
When in doubt, favor dependency injection and let your framework handle instance management. But when you do need a Singleton, implement it properly—prefer the Bill Pugh approach or Enum implementation for most cases.

Remember, design patterns are guidelines, not rules. The best code is code that solves your problem clearly, maintainably, and efficiently. Sometimes that's a Singleton. Often, it's not.
Happy coding!

👻 The Silent Ghost: How a Single Enum Broke Our Distributed Transaction

Dipankar Sethi — Wed, 28 Jan 2026 05:13:11 +0000

It was 3:00 AM on a Tuesday when PagerDuty decided my night was over.

User onboarding wasn’t down.
No red dashboards.
No obvious errors.

But something felt off.

A quick query against production revealed the truth:

👉 Thousands of users existed in Auth, with no HR profiles.

No UI errors.
No retries.
No alarms.

Just ghost users.

This is a real production postmortem from a system built using Spring Boot, PostgreSQL, and the Saga Pattern — and how a single enum value quietly broke our data integrity.

Architecture Overview

We use an orchestration-based Saga, owned by the HR service.

Goal

When a new employee joins:
Create Auth credentials
Create an HR profile

Both must succeed — or neither should exist.

Original Saga Flow
[ Client ]
    |
[ HR Service (Saga Orchestrator) ]
    |
    |-- (1) Create User --------------> [ Auth Service ] -> [ Auth DB ]
    |
    |-- (2) Save Profile -------------> [ HR DB ❌ CHECK constraint ]
    |
    |-- (3) Delete User (Compensation) -> [ Auth Service ]

The logs claimed compensation succeeded.
Production data disagreed.
The Trigger: A Harmless Enum

We added support for remote work:

public enum AddressType {
    HOME,
    CURRENT,
    OFFICE,
    PERMANENT,
    WORK
}

CI passed.
Deploy succeeded.
Production failed silently.

The Real Culprit: Schema Drift

The HR database had a PostgreSQL CHECK constraint:

CHECK (address_type IN ('HOME','CURRENT','OFFICE','PERMANENT'))

So when WORK was saved:


ERROR: violates check constraint "employee_addresses_address_type_check"

But the Saga still didn’t roll back.
Why?

The Invisible Failure: JPA Lazy Writes

@Transactional
public void onboard(Request req) {
    try {
        authClient.createUser(req);               // SUCCESS
        profileRepository.save(new Profile(req)); // Deferred
    } catch (Exception e) {
        authClient.deleteUser(req.getUserId());   // Never reached
    }
} // Commit happens here

Key detail:

save() doesn’t hit the DB
SQL runs at commit
Exception thrown after try-catch exits

Result:

Auth user created ✅
HR insert fails ❌
Compensation skipped 👻

Fix #1: Fail Fast with flush()

@Transactional
public void onboardEmployee(OnboardRequest request) {
    try {
        authClient.createUser(request);
        employeeRepository.save(new EmployeeProfile(request));

        // Force SQL execution NOW
        employeeRepository.flush();

    } catch (Exception e) {
        authClient.deleteUser(request.getUserId());
        throw e;
    }
}

Now:

DB errors surface immediately
Saga becomes aware of failure
Compensation actually runs

Fix #2: Repair the Schema

ALTER TABLE employee_addresses
DROP CONSTRAINT employee_addresses_address_type_check;

ALTER TABLE employee_addresses
ADD CONSTRAINT employee_addresses_address_type_check
CHECK (address_type IN (
  'HOME','CURRENT','OFFICE','PERMANENT','WORK'
));

The Real Fix: State-Based Saga

Delete-based rollback assumes perfect networks.
They don’t exist.

Improved Flow
[ Client ]
    |
[ HR Service ]
    |
    |-- Create User (PENDING) --------> [ Auth DB ]
    |
    |-- Save Profile (FLUSH) ---------> [ HR DB ]
    |
    |-- Activate User (ACTIVE) -------> [ Auth DB ]

No deletes.
Only state transitions.

The Safety Net: Reconciliation Job

@Scheduled(cron = "0 0 2 * * *")
public void cleanupOrphans() {
    for (User user : authRepo.findAllByStatus(PENDING)) {
        if (!hrService.profileExists(user.getId())) {
            authRepo.delete(user);
        }
    }
}

This job quietly saved us more than once.

Lessons Learned

Java enums ≠ DB constraints
Never trust save() in a Saga
Flush early, fail fast
Delete-based rollback is fragile

Reconciliation jobs save careers

Distributed systems don’t always crash.
Sometimes they corrupt your data while telling you everything is fine.

From PostgreSQL to Redis: How We Reduced Authentication Latency by 98% in Production

Dipankar Sethi — Thu, 08 Jan 2026 08:32:09 +0000

Introduction
When building authentication systems for our startup from scratch, we made what seemed like a logical choice: store everything in PostgreSQL. After all, it's reliable, we were already using it, and it kept our architecture simple. However, as our user base grew, this decision nearly brought our authentication system to its knees.

This is the story of how storing JWT tokens and OTPs in PostgreSQL created a cascading performance problem, and how migrating to Redis transformed our authentication from a bottleneck into one of our fastest services.

The Beginning: A Simple Startup Architecture
We were building our product from scratch with a small team (Only 2 persons) using Spring Boot. For user authentication, we implemented email verification using OTPs (One-Time Passwords). The approach seemed straightforward and worked perfectly during development.

Our Initial OTP Implementation
When a user requested an OTP for email verification, login, or password reset, we stored it in PostgreSQL:

-- OTP Storage Schema
CREATE TABLE otp_codes (
    id BIGSERIAL PRIMARY KEY,
    user_email VARCHAR(255) NOT NULL,
    otp_code VARCHAR(6) NOT NULL,
    purpose VARCHAR(50) NOT NULL, -- 'EMAIL_VERIFICATION', 'LOGIN', 'PASSWORD_RESET'
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NOT NULL,
    is_used BOOLEAN DEFAULT FALSE
);

CREATE INDEX idx_otp_lookup ON otp_codes(user_email, purpose, expires_at);

The JPA Entity for OTP:

@Entity
@Table(name = "otp_codes")
@Data
@NoArgsConstructor
@AllArgsConstructor
public class OtpCode {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "user_email", nullable = false)
    private String userEmail;

    @Column(name = "otp_code", nullable = false, length = 6)
    private String otpCode;

    @Enumerated(EnumType.STRING)
    @Column(nullable = false)
    private OtpPurpose purpose;

    @Column(name = "created_at")
    private LocalDateTime createdAt;

    @Column(name = "expires_at", nullable = false)
    private LocalDateTime expiresAt;

    @Column(name = "is_used")
    private Boolean isUsed = false;
}

The Repository interface:

@Repository
public interface OtpRepository extends JpaRepository<OtpCode, Long> {

    Optional<OtpCode> findByUserEmailAndPurposeAndOtpCodeAndExpiresAtAfterAndIsUsedFalse(
        String userEmail, 
        OtpPurpose purpose, 
        String otpCode, 
        LocalDateTime currentTime
    );

    @Modifying
    @Query("DELETE FROM OtpCode o WHERE o.expiresAt < :currentTime")
    int deleteExpiredOtps(@Param("currentTime") LocalDateTime currentTime);
}

The Service for generating and storing OTPs:

@Service
@RequiredArgsConstructor
public class OtpService {

    private final OtpRepository otpRepository;
    private final EmailService emailService;

    // Generate and store OTP in PostgreSQL
    public String generateOTP(String email, OtpPurpose purpose) {
        // Generate 6-digit OTP
        String otp = String.format("%06d", new Random().nextInt(999999));

        OtpCode otpCode = new OtpCode();
        otpCode.setUserEmail(email);
        otpCode.setOtpCode(otp);
        otpCode.setPurpose(purpose);
        otpCode.setCreatedAt(LocalDateTime.now());
        otpCode.setExpiresAt(LocalDateTime.now().plusMinutes(10)); // 10 minutes expiry
        otpCode.setIsUsed(false);

        // First database call - INSERT
        otpRepository.save(otpCode);

        // Send OTP via email
        emailService.sendOtpEmail(email, otp);

        return otp;
    }

    // Verify OTP
    public boolean verifyOTP(String email, OtpPurpose purpose, String inputOtp) {
        Optional<OtpCode> otpOptional = otpRepository
            .findByUserEmailAndPurposeAndOtpCodeAndExpiresAtAfterAndIsUsedFalse(
                email, purpose, inputOtp, LocalDateTime.now()
            );

        if (otpOptional.isEmpty()) {
            return false;
        }

        // Mark as used - Second database call - UPDATE
        OtpCode otp = otpOptional.get();
        otp.setIsUsed(true);
        otpRepository.save(otp);

        return true;
    }
}

At first, this approach looked good and worked fine during our early testing phase with limited users.

The Cleanup Scheduler for OTPs
Since OTPs expire after 10 minutes, we needed a way to clean up old records. We implemented a scheduled task:

@Component
@RequiredArgsConstructor
@Slf4j
public class OtpCleanupScheduler {

    private final OtpRepository otpRepository;

    // Running every 5 minutes
    @Scheduled(fixedRate = 300000) // 5 minutes = 300,000 milliseconds
    @Transactional
    public void cleanupExpiredOtps() {
        log.info("Running OTP cleanup scheduler...");

        int deletedCount = otpRepository.deleteExpiredOtps(LocalDateTime.now());

        log.info("Deleted {} expired OTPs", deletedCount);
    }
}

Scheduler Configuration:

Parameter	Value
Scheduler Name	OTP Cleanup
Frequency	Every 5 minutes
Operation	DELETE
Database Calls	Expired OTPs — 1 DELETE query

Problem #1: The OTP Bottleneck

As traffic increased, we started noticing serious issues with our OTP system.

The Growing Data Problem
Every OTP generation created a new database row. With users requesting OTPs for:

Email verification (new signups)
Login attempts (2FA)
Password resets
Resend requests (when users didn't receive the first OTP)

Our otp_codes table exploded:

Time Period	Total OTPs Generated	Expired Records	Database Size
Week 1	11,200	950	15 MB
Month 1	45,000	38,000	180 MB
Month 3	380,000	325,000	1.2 GB
Month 6	1,200,000	1,050,000	3.8 GB

The Scheduler Became a Monster
Our 5-minute cleanup scheduler that initially deleted 10-20 records was now handling thousands:

[2025-10-14 10:05:00] Deleted 3,847 expired OTPs - Execution time: 4.2s
[2025-10-15 10:10:00] Deleted 4,123 expired OTPs - Execution time: 5.1s
[2025-10-16 10:15:00] Deleted 5,691 expired OTPs - Execution time: 6.8s

The DELETE query became very heavy. Each execution:

Scanned thousands of rows
Acquired table locks
Caused query timeouts during peak hours
Impacted other database operations

Unnecessary Database Calls
For every single OTP flow, we were making three database calls:

User requests OTP → INSERT into database (Call #1)
                 ↓
User verifies OTP → UPDATE database (Call #2)
                 ↓
Scheduler runs   → DELETE from database (Call #3)

With thousands of OTP requests per hour, this translated to:

6,000+ INSERT queries per hour during peak
4,500+ UPDATE queries per hour (75% verification rate)
12 DELETE queries per hour (scheduler) each deleting thousands of rows

Problem #2: Managing JWT Logout Tokens

The second major challenge came from managing user logouts. We were using JWT tokens for authentication, which are stateless by design.
The JWT Stateless Problem
When a user logs out, their JWT token doesn't automatically become invalid it remains valid until it expires. This created a security concern:

User logs in  → JWT issued (expires in 60 minutes)
User logs out → JWT still works for remaining time

We needed a way to blacklist tokens when users logged out.

Our PostgreSQL Solution for Token Blacklist
We created another table to store invalidated tokens:

-- Token Blacklist Schema
CREATE TABLE invalidated_tokens (
    id BIGSERIAL PRIMARY KEY,
    token_jti VARCHAR(255) UNIQUE NOT NULL, -- JWT ID claim
    user_id BIGINT NOT NULL,
    invalidated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NOT NULL
);

CREATE INDEX idx_token_jti ON invalidated_tokens(token_jti);
CREATE INDEX idx_token_expires ON invalidated_tokens(expires_at);

The JPA Entity:

@Entity
@Table(name = "invalidated_tokens")
@Data
@NoArgsConstructor
@AllArgsConstructor
public class InvalidatedToken {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "token_jti", unique = true, nullable = false)
    private String tokenJti;

    @Column(name = "user_id", nullable = false)
    private Long userId;

    @Column(name = "invalidated_at")
    private LocalDateTime invalidatedAt;

    @Column(name = "expires_at", nullable = false)
    private LocalDateTime expiresAt;
}

The Repository:

@Repository
public interface InvalidatedTokenRepository extends JpaRepository<InvalidatedToken, Long> {

    boolean existsByTokenJtiAndExpiresAtAfter(String tokenJti, LocalDateTime currentTime);

    @Modifying
    @Query("DELETE FROM InvalidatedToken t WHERE t.expiresAt < :currentTime")
    int deleteExpiredTokens(@Param("currentTime") LocalDateTime currentTime);
}

The Token Blacklist Service:

@Service
@RequiredArgsConstructor
public class TokenBlacklistService {

    private final InvalidatedTokenRepository tokenRepository;

    // Blacklist token on logout
    public void blacklistToken(String tokenJti, Long userId, LocalDateTime expiresAt) {
        InvalidatedToken token = new InvalidatedToken();
        token.setTokenJti(tokenJti);
        token.setUserId(userId);
        token.setInvalidatedAt(LocalDateTime.now());
        token.setExpiresAt(expiresAt);

        // Database call - INSERT
        tokenRepository.save(token);
    }

    // Check if token is blacklisted (called on every authenticated request)
    public boolean isTokenBlacklisted(String tokenJti) {
        // Database call - SELECT
        return tokenRepository.existsByTokenJtiAndExpiresAtAfter(
            tokenJti, LocalDateTime.now()
        );
    }
}

JWT Authentication Filter with blacklist check:

@Component
@RequiredArgsConstructor
public class JwtAuthenticationFilter extends OncePerRequestFilter {

    private final JwtUtil jwtUtil;
    private final TokenBlacklistService blacklistService;

    @Override
    protected void doFilterInternal(HttpServletRequest request, 
                                    HttpServletResponse response, 
                                    FilterChain filterChain) 
            throws ServletException, IOException {

        String token = extractToken(request);

        if (token != null && jwtUtil.validateToken(token)) {
            String jti = jwtUtil.getJtiFromToken(token);

            // Database call on EVERY authenticated request
            if (blacklistService.isTokenBlacklisted(jti)) {
                response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
                return;
            }

            // Continue with authentication
            Authentication auth = getAuthentication(token);
            SecurityContextHolder.getContext().setAuthentication(auth);
        }

        filterChain.doFilter(request, response);
    }
}

The Token Cleanup Scheduler
Similar to OTPs, we needed to clean up expired blacklisted tokens:

@Component
@RequiredArgsConstructor
@Slf4j
public class TokenCleanupScheduler {

    private final InvalidatedTokenRepository tokenRepository;

    // Running every 60 minutes
    @Scheduled(fixedRate = 3600000) // 60 minutes = 3,600,000 milliseconds
    @Transactional
    public void cleanupExpiredTokens() {
        log.info("Running token blacklist cleanup scheduler...");

        int deletedCount = tokenRepository.deleteExpiredTokens(LocalDateTime.now());

        log.info("Deleted {} expired tokens", deletedCount);
    }
}

Updated Scheduler Summary:

Scheduler Name	Frequency	Operation	Database Calls
OTP Cleanup	Every 5 minutes	DELETE	1 DELETE query
Token Cleanup	Every 60 minutes	DELETE	1 DELETE query

The Problems with Token Blacklisting
At first, this looked good, but as users grew, critical issues emerged:

1. Unnecessary Data Retention:

User logs out at 10:01 AM → Token stored in database
Token expires at 11:00 AM → Token sits idle for 59 minutes
Cleanup runs at 11:00 AM → Token finally deleted

A large number of non-functional tokens were sitting in the database doing nothing but consuming space.

2. Database Call on Every Request:
The biggest problem was checking the blacklist on every authenticated API request:

GET /api/user/profile     → Check blacklist (DB call)
GET /api/posts            → Check blacklist (DB call)
POST /api/comment         → Check blacklist (DB call)
GET /api/notifications    → Check blacklist (DB call)

With 10,000 authenticated requests per minute:

10,000 SELECT queries per minute to check blacklist
Added 50-100ms latency to every request
Database connection pool frequently exhausted
Queries queued during peak traffic

Time Period	Logged Out Users	Blacklisted Tokens	Table Size
Week 1	450	450	8 MB
Month 1	18,000	18,000	95 MB
Month 3	125,000	125,000	680 MB
Month 6	480,000	480,000	2.1 GB

4. Scheduler Overhead:

[2025-10-20 11:00:00] Deleted 8,234 expired tokens - Execution time: 7.3s
[2025-10-21 12:00:00] Deleted 9,127 expired tokens - Execution time: 8.9s
[2025-10-21 13:00:00] Deleted 11,456 expired tokens - Execution time: 12.1s

The Complete Problem Summary

Unnecessary Database Calls:
For OTPs:
- INSERT on generation
- UPDATE on verification
- DELETE every 5 minutes (bulk)
For Token Blacklist:
- INSERT on logout
- SELECT on every authenticated request
- DELETE every 60 minutes (bulk)

The Impact:
Database connection pool exhaustion (18-20 out of 20 connections constantly in use)
Increased infrastructure costs (larger database instances needed)
Slow response times (added 100-150ms to authentication flows)
Schedulers consuming resources and locking tables
Growing storage costs for temporary data.

The Realization: We Needed a Better Solution
After monitoring our production metrics, we realized:

OTPs are temporary by nature (10-minute lifespan)
Blacklisted tokens are temporary (lifespan = remaining token TTL)
These are simple key-value lookups (no complex queries or joins needed)
High read frequency (especially token blacklist checks)
Auto-expiry would eliminate schedulers (no manual cleanup needed)

We needed a data store optimized for:

Temporary data with automatic expiration (TTL)
Fast read/write operations (microseconds, not milliseconds)
Simple key-value storage
No manual cleanup overhead

My friend suggested Redis. Previously, I had only heard of Redis being used for caching, so this was new territory for me. But the more we researched, the more Redis seemed like the perfect solution for our authentication challenges.

The Solution: Migrating to Redis
After analyzing our requirements, Redis checked all the boxes. It's an in-memory data store designed for exactly this use case—temporary data that needs to be accessed quickly and expire automatically.

Setting Up Redis with Spring Boot

<!-- pom.xml -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
</dependency>

Redis Configuration:

@Configuration
@EnableCaching
public class RedisConfig {

    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        RedisStandaloneConfiguration config = new RedisStandaloneConfiguration();
        config.setHostName("localhost");
        config.setPort(6379);
        return new JedisConnectionFactory(config);
    }

    @Bean
    public RedisTemplate<String, Object> redisTemplate() {
        RedisTemplate<String, Object> template = new RedisTemplate<>();
        template.setConnectionFactory(redisConnectionFactory());
        template.setKeySerializer(new StringRedisSerializer());
        template.setValueSerializer(new GenericJackson2JsonRedisSerializer());
        return template;
    }

    @Bean
    public StringRedisTemplate stringRedisTemplate() {
        return new StringRedisTemplate(redisConnectionFactory());
    }
}

OTP Implementation with Redis

@Service
@RequiredArgsConstructor
@Slf4j
public class RedisOtpService {

    private final StringRedisTemplate redisTemplate;
    private final EmailService emailService;

    private static final long OTP_EXPIRY_MINUTES = 10;
    private static final int MAX_ATTEMPTS = 3;

    // Generate and store OTP in Redis
    public String generateOTP(String email, OtpPurpose purpose) {
        String otp = String.format("%06d", new Random().nextInt(999999));
        String redisKey = buildOtpKey(email, purpose);

        // Store OTP with metadata as hash
        Map<String, String> otpData = new HashMap<>();
        otpData.put("code", otp);
        otpData.put("attempts", "0");
        otpData.put("createdAt", String.valueOf(System.currentTimeMillis()));

        redisTemplate.opsForHash().putAll(redisKey, otpData);

        // Set expiry - Auto-deletion after 10 minutes (No scheduler needed!)
        redisTemplate.expire(redisKey, OTP_EXPIRY_MINUTES, TimeUnit.MINUTES);

        // Send OTP via email
        emailService.sendOtpEmail(email, otp);

        log.info("OTP generated for {}: {}", email, redisKey);
        return otp;
    }

    // Verify OTP
    public boolean verifyOTP(String email, OtpPurpose purpose, String inputOtp) {
        String redisKey = buildOtpKey(email, purpose);

        // Check if OTP exists (will be null if expired)
        String storedOtp = (String) redisTemplate.opsForHash().get(redisKey, "code");

        if (storedOtp == null) {
            log.warn("OTP not found or expired for {}", email);
            return false;
        }

        // Increment attempts
        Long attempts = redisTemplate.opsForHash().increment(redisKey, "attempts", 1);

        if (attempts > MAX_ATTEMPTS) {
            // Delete OTP after max attempts
            redisTemplate.delete(redisKey);
            log.warn("Max attempts exceeded for {}", email);
            return false;
        }

        // Verify OTP
        if (storedOtp.equals(inputOtp)) {
            // Delete OTP immediately after successful verification
            redisTemplate.delete(redisKey);
            log.info("OTP verified successfully for {}", email);
            return true;
        }

        log.warn("Invalid OTP for {}", email);
        return false;
    }

    private String buildOtpKey(String email, OtpPurpose purpose) {
        return String.format("otp:%s:%s", email, purpose.name().toLowerCase());
    }
}

Key Benefits of Redis for OTPs:

No INSERT query - Directly set key-value
No UPDATE query - Atomic operations
No DELETE query - Auto-expires with TTL
No scheduler needed - Redis handles cleanup automatically

JWT Token Blacklist with Redis

@Service
@RequiredArgsConstructor
@Slf4j
public class RedisTokenBlacklistService {

    private final StringRedisTemplate redisTemplate;
    private final JwtUtil jwtUtil;

    // Blacklist token on logout
    public void blacklistToken(String token) {
        try {
            String jti = jwtUtil.getJtiFromToken(token);
            Date expiration = jwtUtil.getExpirationFromToken(token);

            // Calculate remaining TTL
            long ttlSeconds = (expiration.getTime() - System.currentTimeMillis()) / 1000;

            if (ttlSeconds > 0) {
                String redisKey = buildTokenKey(jti);

                // Store with exact remaining TTL
                redisTemplate.opsForValue().set(redisKey, "blacklisted", ttlSeconds, TimeUnit.SECONDS);

                log.info("Token blacklisted: {} with TTL: {}s", jti, ttlSeconds);
            }
        } catch (Exception e) {
            log.error("Error blacklisting token", e);
            throw new RuntimeException("Failed to blacklist token");
        }
    }

    // Check if token is blacklisted (called on every request)
    public boolean isTokenBlacklisted(String token) {
        try {
            String jti = jwtUtil.getJtiFromToken(token);
            String redisKey = buildTokenKey(jti);

            // Single Redis check - incredibly fast!
            Boolean exists = redisTemplate.hasKey(redisKey);

            return Boolean.TRUE.equals(exists);
        } catch (Exception e) {
            log.error("Error checking token blacklist", e);
            return false;
        }
    }

    private String buildTokenKey(String jti) {
        return String.format("blacklist:%s", jti);
    }
}

Updated Authentication Filter:

@Component
@RequiredArgsConstructor
public class JwtAuthenticationFilter extends OncePerRequestFilter {

    private final JwtUtil jwtUtil;
    private final RedisTokenBlacklistService redisBlacklistService; // Changed to Redis

    @Override
    protected void doFilterInternal(HttpServletRequest request, 
                                    HttpServletResponse response, 
                                    FilterChain filterChain) 
            throws ServletException, IOException {

        String token = extractToken(request);

        if (token != null && jwtUtil.validateToken(token)) {
            // Redis check - 1-2ms instead of 50-100ms!
            if (redisBlacklistService.isTokenBlacklisted(token)) {
                response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
                return;
            }

            Authentication auth = getAuthentication(token);
            SecurityContextHolder.getContext().setAuthentication(auth);
        }

        filterChain.doFilter(request, response);
    }
}

Logout Controller:

@RestController
@RequestMapping("/api/auth")
@RequiredArgsConstructor
public class AuthController {

    private final RedisTokenBlacklistService tokenBlacklistService;

    @PostMapping("/logout")
    public ResponseEntity<?> logout(@RequestHeader("Authorization") String authHeader) {
        String token = authHeader.substring(7); // Remove "Bearer " prefix

        // Blacklist token in Redis
        tokenBlacklistService.blacklistToken(token);

        return ResponseEntity.ok(Map.of("message", "Logged out successfully"));
    }
}

Key Benefits of Redis for Token Blacklist:

1-2ms lookup time (vs 50-100ms with PostgreSQL)
Auto-expiry - Token removed exactly when it expires naturally
No scheduler needed - Redis manages cleanup
No storage overhead - Memory freed automatically

Scheduler Comparison
Before Redis (PostgreSQL):

Scheduler Name	Frequency	Purpose	Impact
OTP Cleanup	Every 5 minutes	Delete expired OTPs	Heavy DELETE, table locks
Token Cleanup	Every 60 minutes	Delete expired tokens	Heavy DELETE, table locks

After Redis:

Scheduler Name	Frequency	Purpose	Impact
NONE	N/A	Redis TTL handles everything	Zero overhead

The Results: Production Performance Improvements
After migrating to Redis, the improvements were dramatic and immediate.
Response Time Improvements

Operation	PostgreSQL Latency	Redis Latency	Improvement
OTP Generation	80–120 ms	2–3 ms	97% faster
OTP Verification	150–200 ms	2–4 ms	98% faster
Token Blacklist Check	50–100 ms	1–2 ms	98% faster
Logout Operation	60–90 ms	2–3 ms	97% faster

Storage and Cost Savings

Connection pool utilization dropped from 90–100% to 40–60%, resulting in a 50% reduction in database load.
Eliminated 612,000+ database queries per hour during peak traffic.
Freed 6.2 GB of PostgreSQL storage while Redis used only 145 MB of in-memory data.
Simplified infrastructure by removing schedulers and cleanup jobs, and reducing backup and maintenance overhead.
Improved API performance drastically: OTP operations fell from 150–180 ms to <10 ms, and authenticated request overhead dropped to 1–2 ms.

Key Lessons Learned

1. Not All Data Belongs in Your Primary Database
PostgreSQL is excellent for persistent, relational data. But temporary, high-frequency data like OTPs and session tokens are better served by specialized tools like Redis.

2. Automatic Expiry > Manual Cleanup
Redis TTL eliminated our schedulers entirely. No more:

Cron jobs to maintain
Bulk DELETE operations
Table locks during cleanup
Monitoring cleanup job failures

3. In-Memory Speed Matters for Authentication
Authentication happens on nearly every request. Reducing latency from 100ms to 2ms means:

Better user experience
Lower infrastructure costs
Higher throughput capacity

4. Choose the Right Tool for the Job
We were using PostgreSQL as a:

Temporary data store (wrong tool)
High-frequency cache (wrong tool)
Persistent data store (right tool)

Redis excels at:

Temporary data with TTL
High-frequency reads/writes
Simple key-value operations

Conclusion
Migrating OTPs and JWT token blacklists from PostgreSQL to Redis reduced authentication latency by 98%, cut database load by 50%, and eliminated over 600K queries per hour, while also simplifying infrastructure and lowering costs. The takeaway is clear: use PostgreSQL for persistent data and Redis for fast, temporary data don’t force one tool to do everything.

Have you faced similar authentication challenges? Share your experiences in the comments below!

Why Role-Based Access Control Isn't Enough (And What to Do About It)

Dipankar Sethi — Mon, 05 Jan 2026 16:05:35 +0000

Early in my career, I thought I had access control figured out. RBAC - Role-Based Access Control - seemed elegant in its simplicity. Assign users to roles (ADMIN, MANAGER, USER), secure your endpoints, and you're done.

@PreAuthorize("hasRole('ADMIN')")
@GetMapping("/api/v1/admin")
public ResponseEntity<?> getAdminData() {
    // Only admins can access this
}

Beautiful. Clean. Simple.
Then production requirements hit me like a truck.

The RBAC Problem

Let's start with the basics. RBAC (Role-Based Access Control) is a security concept widely used in applications. The idea is simple: assign roles to users, then restrict API access based on those roles.
For example:

/api/v1/admin - Only users with ROLE_ADMIN can access

/api/v1/sales - Only users with ROLE_SALES can access.

You can even allow multiple roles to access the same endpoint. Seems straightforward, right?

Here's where it breaks down.
Imagine you have two users, both with the ROLE_ADMIN role. According to business requirements, one admin (let's call them the SUPER_ADMIN) should be able to revoke permissions for certain APIs, while regular admins cannot.

Or consider this scenario: You have an endpoint /api/v1/managers that displays manager data. All admins can view this data, but only specific admins can edit it.

My first instinct?
Create a new role. ROLE_SUPER_ADMIN or ROLE_ADMIN_EDITOR. Problem solved... or so I thought.
But when I stepped back and looked at the bigger picture in production, I realized this approach doesn't scale. This was just one small requirement. In a real production system, you might have:

Admins who can view reports but not delete them
Managers who can edit their team's data but not other teams
Sales users who can create invoices but not approve them

If we create a new role for every permission combination, we'd end up with role explosion: ROLE_ADMIN_CAN_VIEW_REPORTS, ROLE_ADMIN_CAN_EDIT_NOT_DELETE, ROLE_MANAGER_TEAM_A... you get the idea.

The real issue: Roles represent who someone is (their job title), not what they can do (specific actions). Production systems need fine-grained control over individual actions, not just broad role categories.

That's when I discovered we needed something more granular: permission-based access control.

The Solution: Granular Authorities

The key insight is this: separate what someone is (their role) from what they can do (their permissions).
Instead of creating endless roles, we introduce granular authorities - specific, atomic permissions that represent individual actions in the system. Then we assign these authorities to roles, and roles to users.
Here's the mental model:

User → has → Role(s) → contains → Authorities/Permissions

Example: John (User) → ADMIN (Role) → {READ_REPORTS, EDIT_REPORTS, DELETE_USER} (Authorities)

This three-tier structure gives us the flexibility we need. Want to give an admin editing rights but not deletion rights? Just assign different authorities to that role. No need to create ROLE_ADMIN_NO_DELETE.

Database Structure
The foundation of this approach is a proper database schema. Here's what I implemented:
Three main tables:

users - Stores user information
roles - Defines roles (ADMIN, MANAGER, USER, etc.)
permissions - Individual authorities (READ_REPORTS, EDIT_REPORTS, CREATE_LEAD, etc.)

Two junction tables:

user_roles - Maps users to their roles (many-to-many)
role_permissions - Maps roles to their authorities (many-to-many)

-- Users table
CREATE TABLE users (
    id BIGINT PRIMARY KEY,
    username VARCHAR(255) UNIQUE NOT NULL,
    email VARCHAR(255) UNIQUE NOT NULL,
    password VARCHAR(255) NOT NULL
);

-- Roles table
CREATE TABLE roles (
    id BIGINT PRIMARY KEY,
    name VARCHAR(50) UNIQUE NOT NULL  -- ADMIN, MANAGER, USER
);

-- Permissions/Authorities table
CREATE TABLE permissions (
    id BIGINT PRIMARY KEY,
    name VARCHAR(100) UNIQUE NOT NULL,  -- READ_REPORTS, EDIT_REPORTS, etc.
    description VARCHAR(255)
);

-- User-Role mapping (many-to-many)
CREATE TABLE user_roles (
    user_id BIGINT,
    role_id BIGINT,
    PRIMARY KEY (user_id, role_id),
    FOREIGN KEY (user_id) REFERENCES users(id),
    FOREIGN KEY (role_id) REFERENCES roles(id)
);

-- Role-Permission mapping (many-to-many)
CREATE TABLE role_permissions (
    role_id BIGINT,
    permission_id BIGINT,
    PRIMARY KEY (role_id, permission_id),
    FOREIGN KEY (role_id) REFERENCES roles(id),
    FOREIGN KEY (permission_id) REFERENCES permissions(id)
);

Why this structure works:

Flexibility: One role can have multiple permissions
Reusability: Same permission can belong to multiple roles
Scalability: Add new permissions without touching existing roles
Maintainability: Change a role's permissions without affecting users

Dynamic Permission Management
Here's the game-changer: We gave the SUPER_ADMIN the ability to edit role permissions at runtime.
This means:

No code deployments to change access control
Business users can adjust permissions through an admin panel
Audit trail of who changed what permissions when

The SUPER_ADMIN can:

Add/remove permissions from any role
Create new permissions as needed
Revoke specific authorities without changing the entire role

This was a requirement that emerged from production needs - the business team needed flexibility without waiting for engineering to deploy new role configurations.

How Authorities Flow to the User
When a user logs in, here's what happens:

Fetch user's roles from user_roles table
For each role, fetch its permissions from role_permissions table
Combine all permissions (remove duplicates if user has multiple roles)
Pass authorities in the JWT token for authorization checks
Cache the authorities to avoid repeated database queries.

This way, every request carries the user's complete set of authorities, and we can make authorization decisions at the API level without hitting the database repeatedly

Implementation in Spring Boot
Let's see how this translates into actual code. I'll walk you through the entities, JWT token handling, and securing endpoints with granular authorities.

Step 1: Define the Entities
First, we need JPA entities that match our database schema:

@Entity
@Table(name = "users")
public class User {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String username;
    private String email;
    private String password;

    @ManyToMany(fetch = FetchType.EAGER)
    @JoinTable(
        name = "user_roles",
        joinColumns = @JoinColumn(name = "user_id"),
        inverseJoinColumns = @JoinColumn(name = "role_id")
    )
    private Set<Role> roles = new HashSet<>();

    // Getters and setters
}

@Entity
@Table(name = "roles")
public class Role {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name; // ADMIN, MANAGER, SALES, etc.

    @ManyToMany(fetch = FetchType.EAGER)
    @JoinTable(
        name = "role_permissions",
        joinColumns = @JoinColumn(name = "role_id"),
        inverseJoinColumns = @JoinColumn(name = "permission_id")
    )
    private Set<Permission> permissions = new HashSet<>();

    // Getters and setters
}

@Entity
@Table(name = "permissions")
public class Permission {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name; // READ_REPORTS, EDIT_REPORTS, CREATE_LEAD, etc.
    private String description;

    // Getters and setters
}

Key points:

FetchType.EAGER ensures we load roles and permissions when we fetch the user
Many-to-many relationships are handled through @JoinTable
Clean separation between User → Role → Permission

Step 2: Extract User Authorities
We need a method to collect all authorities for a user across all their roles:

@Service
public class UserService {

    public Set<String> getUserAuthorities(User user) {
        Set<String> authorities = new HashSet<>();

        // Iterate through all roles
        for (Role role : user.getRoles()) {
            // Collect all permissions from each role
            for (Permission permission : role.getPermissions()) {
                authorities.add(permission.getName());
            }
        }

        return authorities;
    }
}

This gives us a flat list of all unique authorities for a user, regardless of which role grants them.

Step 3: JWT Token with Authorities
When generating the JWT token, we include the authorities as claims:

@Component
public class JwtTokenProvider {

    @Value("${jwt.secret}")
    private String jwtSecret;

    @Value("${jwt.expiration}")
    private long jwtExpiration;

    public String generateToken(User user, Set<String> authorities) {
        Map<String, Object> claims = new HashMap<>();

        // Add authorities to JWT claims
        claims.put("authorities", authorities);

        return Jwts.builder()
                .setClaims(claims)
                .setSubject(user.getUsername())
                .setIssuedAt(new Date())
                .setExpiration(new Date(System.currentTimeMillis() + jwtExpiration))
                .signWith(SignatureAlgorithm.HS512, jwtSecret)
                .compact();
    }

    public Set<String> getAuthoritiesFromToken(String token) {
        Claims claims = Jwts.parser()
                .setSigningKey(jwtSecret)
                .parseClaimsJws(token)
                .getBody();

        // Extract authorities from claims
        List<String> authList = (List<String>) claims.get("authorities");
        return new HashSet<>(authList);
    }

    public String getUsernameFromToken(String token) {
        return Jwts.parser()
                .setSigningKey(jwtSecret)
                .parseClaimsJws(token)
                .getBody()
                .getSubject();
    }

    public boolean validateToken(String token) {
        try {
            Jwts.parser().setSigningKey(jwtSecret).parseClaimsJws(token);
            return true;
        } catch (Exception e) {
            return false;
        }
    }
}

The client receives the JWT token in the Authorization header, and optionally, we can send a readable list of permissions in a custom X-Permissions header for client convenience:

Authorization: Bearer eyJhbGciOiJIUzUxMiJ9...
X-Permissions: READ_REPORTS, EDIT_REPORTS, CREATE_LEAD

Step 4: JWT Authentication Filter
Create a filter to extract authorities from the JWT and set them in Spring Security context:

@Component
public class JwtAuthenticationFilter extends OncePerRequestFilter {

    @Autowired
    private JwtTokenProvider tokenProvider;

    @Autowired
    private UserService userService;

    @Override
    protected void doFilterInternal(HttpServletRequest request, 
                                    HttpServletResponse response, 
                                    FilterChain filterChain) 
            throws ServletException, IOException {

        try {
            String jwt = getJwtFromRequest(request);

            if (jwt != null && tokenProvider.validateToken(jwt)) {
                String username = tokenProvider.getUsernameFromToken(jwt);
                Set<String> authorities = tokenProvider.getAuthoritiesFromToken(jwt);

                // Convert to Spring Security authorities
                List<GrantedAuthority> grantedAuthorities = authorities.stream()
                        .map(SimpleGrantedAuthority::new)
                        .collect(Collectors.toList());

                // Create authentication object
                UsernamePasswordAuthenticationToken authentication = 
                        new UsernamePasswordAuthenticationToken(
                                username, null, grantedAuthorities);

                // Set in security context
                SecurityContextHolder.getContext().setAuthentication(authentication);
            }
        } catch (Exception ex) {
            logger.error("Could not set user authentication in security context", ex);
        }

        filterChain.doFilter(request, response);
    }

    private String getJwtFromRequest(HttpServletRequest request) {
        String bearerToken = request.getHeader("Authorization");
        if (bearerToken != null && bearerToken.startsWith("Bearer ")) {
            return bearerToken.substring(7);
        }
        return null;
    }
}

Step 5: Secure Your Endpoints
Now you can use @PreAuthorize with specific authorities:

@RestController
@RequestMapping("/api/v1")
public class ReportController {

    // Any user with READ_REPORTS authority can access
    @PreAuthorize("hasAuthority('READ_REPORTS')")
    @GetMapping("/reports")
    public ResponseEntity<List<Report>> getReports() {
        // Implementation
        return ResponseEntity.ok(reports);
    }

    // Only users with EDIT_REPORTS authority can access
    @PreAuthorize("hasAuthority('EDIT_REPORTS')")
    @PutMapping("/reports/{id}")
    public ResponseEntity<Report> editReport(@PathVariable Long id, 
                                            @RequestBody Report report) {
        // Implementation
        return ResponseEntity.ok(updatedReport);
    }
}

@RestController
@RequestMapping("/api/v1/sales")
public class SalesController {

    @PreAuthorize("hasAuthority('CREATE_LEAD')")
    @PostMapping("/leads")
    public ResponseEntity<Lead> createLead(@RequestBody Lead lead) {
        // Implementation
        return ResponseEntity.ok(createdLead);
    }

    @PreAuthorize("hasAuthority('CREATE_ORDER')")
    @PostMapping("/orders")
    public ResponseEntity<Order> createOrder(@RequestBody Order order) {
        // Implementation
        return ResponseEntity.ok(createdOrder);
    }
}

You can also combine authorities:

// Requires BOTH authorities
@PreAuthorize("hasAuthority('READ_REPORTS') and hasAuthority('EDIT_REPORTS')")

// Requires ANY of these authorities
@PreAuthorize("hasAnyAuthority('READ_REPORTS', 'READ_ANALYTICS')")

Step 6: SUPER_ADMIN Permission Management
The SUPER_ADMIN panel allows dynamic permission management through these endpoints:

@RestController
@RequestMapping("/api/v1/admin/permissions")
@PreAuthorize("hasAuthority('MANAGE_PERMISSIONS')") // Only SUPER_ADMIN has this
public class PermissionManagementController {

    @Autowired
    private RoleService roleService;

    // Add permission to a role
    @PostMapping("/roles/{roleId}/permissions/{permissionId}")
    public ResponseEntity<?> addPermissionToRole(
            @PathVariable Long roleId, 
            @PathVariable Long permissionId) {

        roleService.addPermissionToRole(roleId, permissionId);
        return ResponseEntity.ok("Permission added successfully");
    }

    // Remove permission from a role
    @DeleteMapping("/roles/{roleId}/permissions/{permissionId}")
    public ResponseEntity<?> removePermissionFromRole(
            @PathVariable Long roleId, 
            @PathVariable Long permissionId) {

        roleService.removePermissionFromRole(roleId, permissionId);
        return ResponseEntity.ok("Permission removed successfully");
    }

    // Get all permissions for a role
    @GetMapping("/roles/{roleId}/permissions")
    public ResponseEntity<Set<Permission>> getRolePermissions(
            @PathVariable Long roleId) {

        Set<Permission> permissions = roleService.getRolePermissions(roleId);
        return ResponseEntity.ok(permissions);
    }

    // Create new permission
    @PostMapping("/permissions")
    public ResponseEntity<Permission> createPermission(
            @RequestBody Permission permission) {

        Permission created = roleService.createPermission(permission);
        return ResponseEntity.ok(created);
    }
}

The UI calls these endpoints to:

View current role-permission mappings
Add/remove permissions from roles
Create new permissions as business needs evolve

No code deployment needed - SUPER_ADMIN can adjust access control on the fly.

Security Configuration
Don't forget to configure Spring Security:

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter {

    @Autowired
    private JwtAuthenticationFilter jwtAuthenticationFilter;

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .csrf().disable()
            .sessionManagement()
                .sessionCreationPolicy(SessionCreationPolicy.STATELESS)
            .and()
            .authorizeRequests()
                .antMatchers("/api/v1/auth/**").permitAll()
                .anyRequest().authenticated()
            .and()
            .addFilterBefore(jwtAuthenticationFilter, 
                           UsernamePasswordAuthenticationFilter.class);
    }
}

Key settings:

@EnableGlobalMethodSecurity(prePostEnabled = true) - Enables @PreAuthorize annotations
Stateless session - We're using JWT, not sessions
JWT filter runs before Spring's authentication filter.

Best Practices & Lessons Learned
After implementing this system in production, here are some key takeaways:

1. Permission Naming Conventions
Use a consistent naming pattern for permissions. I use ACTION_RESOURCE format:

READ_REPORTS, EDIT_REPORTS, DELETE_REPORTS
CREATE_LEAD, VIEW_LEAD, CONVERT_LEAD
APPROVE_ORDER, CANCEL_ORDER This makes it immediately clear what action is being performed on what resource.

2. Don't Over-Engineer
Start with the permissions you actually need. It's tempting to create 50 different permissions upfront, but you'll end up with unused complexity. Add permissions as requirements emerge.

3. Performance Considerations
Loading roles and permissions on every request can be expensive.
Consider:

Caching user authorities after login (Redis is great for this)
Using FetchType.EAGER strategically (or switch to LAZY with DTOs)
Setting reasonable JWT expiration times (balance security vs. UX)

4. Handle Permission Changes Gracefully
When SUPER_ADMIN changes role permissions, existing JWT tokens still have the old permissions. You have two options:

Force users to re-login (more secure, but poor UX)
Implement token revocation/refresh mechanism (complex but better UX).

We went with shorter JWT expiration times (30 minutes) as a compromise.

5. Audit Everything
Log every permission change made by SUPER_ADMIN. You'll want this audit trail when debugging access issues or for compliance.

@Aspect
@Component
public class PermissionAuditAspect {

    @AfterReturning("@annotation(PreAuthorize)")
    public void logPermissionCheck(JoinPoint joinPoint) {
        // Log who accessed what with which permission
    }
}

6. Testing Strategy
Test your security configuration thoroughly:

Unit tests for permission logic
Integration tests for actual endpoint access
Test edge cases (user with no roles, multiple roles with conflicting permissions)

@Test
@WithMockUser(authorities = {"READ_REPORTS"})
public void testGetReports_withReadPermission_shouldSucceed() {
    // Test passes
}

@Test
@WithMockUser(authorities = {"CREATE_LEAD"})
public void testGetReports_withoutReadPermission_shouldFail() {
    // Test should return 403
}

When to Use This Approach
Granular permission-based access control is powerful, but it's not always necessary. Use it when:

You have complex, evolving access requirements
Different users with the same role need different permissions
Business users need to manage permissions without developer involvement
You're building a multi-tenant system
Compliance requires fine-grained access control

Stick with simple RBAC when:

Your access rules are straightforward and stable
Roles clearly map to job functions
You have a small team and simple requirements

Conclusion
Moving from role-based to permission-based access control was a game-changer for our application. We went from rigid, hard-coded roles to a flexible system that adapts to business needs without code deployments.
The three-tier structure (User → Role → Permission) strikes the perfect balance between simplicity and flexibility. Roles still represent job functions, but now they're composed of granular permissions that can be mixed and matched.
Yes, it requires more upfront setup than simple RBAC. But the payoff - in flexibility, maintainability, and business agility - is worth it.
How do you handle permissions in your applications? I'd love to hear about different approaches in the comments!

About the Author
I'm a backend developer specializing in Spring Boot and microservices architecture. Currently exploring system design patterns and sharing what I learn along the way. Connect with me on LinkedIn or follow for more backend development content!