DEV Community

Martin D
Martin D

Posted on Edited on

Three-Tier Data Delivery Strategy: Local Development Setup

Security Operations Platform

Project Summary (6W Pitch Framework)

  • Who (Target Customers and Core Roles): Designed for security operations teams at large enterprises, with an AI full-stack engineer leading the core architecture design and business-domain development.
  • What (Core Product and Problems Addressed): An integrated enterprise-grade security operations management platform that addresses delayed processing of high-volume security alerts and fragmented risk management. It provides unified capabilities for cybersecurity monitoring, asset management, vulnerability remediation, and AI-powered intelligent analysis.
  • Why (Core Differentiators and Competitive Advantages): Implements a three-tier data delivery strategy combining scheduled tasks, Redis caching, and real-time aggregation with cache backfilling on misses, reducing core API response times to under 50 ms. It enables a fully automated closed loop from risk identification to protection response.
  • Where (Use Cases and Deployment Environments): Supports multi-scenario enterprise security operations, firewall integration, and domain security monitoring. It also provides Docker-based container orchestration for isolated multi-environment deployments and dynamic switching between data sources.
  • When (Timeliness and Real-Time Capabilities): Processes more than 100,000 raw alerts from network security devices in real time each day. It supports five-minute scheduled aggregation and cache refresh cycles, while completing full-table aggregation across more than 150 tables containing tens of thousands of records within three seconds.
  • How (Technical Approach and Delivery Model): Uses a microservices architecture to separate business modules and reuse shared capability libraries. It establishes a real-time analysis and automated device-response pipeline powered by Kafka, InfluxDB, a rules engine.

Business

  • AI: Integrates AI Agents to perform intelligent attack-path analysis, three-dimensional risk assessment across systems, assets, and vulnerabilities, and automated generation of remediation recommendations.
  • Automation: Provides a public IP blacklist analysis API that enables firewall devices to retrieve data, creating a fully automated process from risk identification to protection response.
  • Concurrency: Supports real-time processing across more than 150 business tables and over 100,000 alerts per day. Core monitoring API response times were reduced from over two seconds to under 50 ms, while Redis-only APIs support more than 500 queries per second.
  • Availability: Automatically falls back to database queries when the cache becomes unavailable, ensuring 99.9% system availability.
  • Localization: Enables seamless switching between the Dameng database and the PostgreSQL.
  • Efficiency: Improves development and delivery efficiency through independent microservice iteration and reuse of shared capability libraries.

Tech Stack

  • Backend: Spring Boot 3.2.6
  • Frontend: Vue 3
  • Database: InfluxDB
  • Cache: Redis
  • Middleware: Kafka
  • Containerization: Docker

This guide builds a working local development environment for a three-tier data delivery strategy using scheduled aggregation, Redis caching, and real-time aggregation with cache backfilling after a miss. The sample uses Spring Boot 3.2.6, Java 17, PostgreSQL, Redis, Maven, and Docker Compose.

The local application exposes a security alert summary API. Scheduled processing prepares commonly requested summaries and stores them in Redis. Normal requests read the prepared result from Redis. If the key is missing, the service aggregates the current data from PostgreSQL, returns the result, and writes it back to Redis so that the next request is fast.

This guide is intentionally limited to local development. It does not include production acceptance criteria or production acceptance tests.

Step 1: Prepare the Local Development Environment

Install the following software:

  • Java Development Kit 17
  • Apache Maven 3.9 or later
  • Docker Desktop or Docker Engine with Docker Compose v2
  • Git
  • An HTTP client such as curl, Postman, or Bruno
  • An IDE such as IntelliJ IDEA or Visual Studio Code

Confirm the installations:

java -version
mvn -version
docker --version
docker compose version
git --version
Enter fullscreen mode Exit fullscreen mode

The expected Java major version is 17. Maven must report that it is running with Java 17. Docker must be running before the infrastructure is started.

Create the project directory:

mkdir three-tier-data-delivery
cd three-tier-data-delivery
mkdir -p src/main/java/com/example/security/config
mkdir -p src/main/java/com/example/security/controller
mkdir -p src/main/java/com/example/security/domain
mkdir -p src/main/java/com/example/security/dto
mkdir -p src/main/java/com/example/security/repository
mkdir -p src/main/java/com/example/security/service
mkdir -p src/main/resources
mkdir -p src/test/java/com/example/security
Enter fullscreen mode Exit fullscreen mode

The final local structure will contain the Spring Boot application, PostgreSQL schema and seed data, Redis configuration, and Docker Compose file.

Step 2: Start PostgreSQL and Redis with Docker Compose

Create compose.yaml in the project root:

services:
  postgres:
    image: postgres:16-alpine
    container_name: security-postgres
    environment:
      POSTGRES_DB: security_ops
      POSTGRES_USER: security_user
      POSTGRES_PASSWORD: security_password
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./src/main/resources/schema.sql:/docker-entrypoint-initdb.d/01-schema.sql:ro
      - ./src/main/resources/data.sql:/docker-entrypoint-initdb.d/02-data.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U security_user -d security_ops"]
      interval: 5s
      timeout: 3s
      retries: 20

  redis:
    image: redis:7.2-alpine
    container_name: security-redis
    command: ["redis-server", "--appendonly", "yes"]
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 20

volumes:
  postgres_data:
  redis_data:
Enter fullscreen mode Exit fullscreen mode

This Compose configuration exposes PostgreSQL on port 5432 and Redis on port 6379. Named volumes preserve local data between container restarts. PostgreSQL runs the schema and sample-data scripts only when its data volume is first initialized.

Create src/main/resources/schema.sql:

CREATE TABLE IF NOT EXISTS security_alert (
    id BIGSERIAL PRIMARY KEY,
    tenant_id VARCHAR(64) NOT NULL,
    severity VARCHAR(16) NOT NULL,
    status VARCHAR(16) NOT NULL,
    source_ip VARCHAR(45),
    asset_id VARCHAR(128),
    detected_at TIMESTAMPTZ NOT NULL,
    updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX IF NOT EXISTS idx_security_alert_tenant_detected
    ON security_alert (tenant_id, detected_at DESC);

CREATE INDEX IF NOT EXISTS idx_security_alert_tenant_severity_status
    ON security_alert (tenant_id, severity, status);
Enter fullscreen mode Exit fullscreen mode

Create src/main/resources/data.sql:

INSERT INTO security_alert
    (tenant_id, severity, status, source_ip, asset_id, detected_at)
VALUES
    ('tenant-a', 'CRITICAL', 'OPEN', '203.0.113.10', 'server-001', CURRENT_TIMESTAMP - INTERVAL '2 minutes'),
    ('tenant-a', 'HIGH',     'OPEN', '203.0.113.11', 'server-002', CURRENT_TIMESTAMP - INTERVAL '5 minutes'),
    ('tenant-a', 'HIGH',     'CLOSED', '203.0.113.12', 'server-003', CURRENT_TIMESTAMP - INTERVAL '10 minutes'),
    ('tenant-a', 'MEDIUM',   'OPEN', '203.0.113.13', 'server-004', CURRENT_TIMESTAMP - INTERVAL '20 minutes'),
    ('tenant-a', 'LOW',      'OPEN', '203.0.113.14', 'server-005', CURRENT_TIMESTAMP - INTERVAL '25 minutes'),
    ('tenant-b', 'CRITICAL', 'OPEN', '198.51.100.20', 'server-101', CURRENT_TIMESTAMP - INTERVAL '4 minutes'),
    ('tenant-b', 'MEDIUM',   'OPEN', '198.51.100.21', 'server-102', CURRENT_TIMESTAMP - INTERVAL '15 minutes');
Enter fullscreen mode Exit fullscreen mode

Start both services:

docker compose up -d
Enter fullscreen mode Exit fullscreen mode

Check their status:

docker compose ps
Enter fullscreen mode Exit fullscreen mode

Both containers should become healthy. Inspect logs if either service does not start:

docker compose logs postgres
docker compose logs redis
Enter fullscreen mode Exit fullscreen mode

Confirm PostgreSQL data:

docker exec -it security-postgres \
  psql -U security_user -d security_ops \
  -c "SELECT tenant_id, severity, status, COUNT(*) FROM security_alert GROUP BY tenant_id, severity, status ORDER BY tenant_id, severity;"
Enter fullscreen mode Exit fullscreen mode

Confirm Redis connectivity:

docker exec -it security-redis redis-cli ping
Enter fullscreen mode Exit fullscreen mode

The Redis response should be PONG.

Step 3: Create the Spring Boot Application

Create pom.xml in the project root:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.6</version>
        <relativePath/>
    </parent>

    <groupId>com.example</groupId>
    <artifactId>three-tier-data-delivery</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>three-tier-data-delivery</name>

    <properties>
        <java.version>17</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>
Enter fullscreen mode Exit fullscreen mode

Create src/main/resources/application.yml:

server:
  port: 8080

spring:
  application:
    name: three-tier-data-delivery
  datasource:
    url: jdbc:postgresql://localhost:5432/security_ops
    username: security_user
    password: security_password
    hikari:
      maximum-pool-size: 10
      minimum-idle: 2
      connection-timeout: 2000
  jpa:
    open-in-view: false
    hibernate:
      ddl-auto: validate
    properties:
      hibernate:
        jdbc:
          time_zone: UTC
        format_sql: true
  data:
    redis:
      host: localhost
      port: 6379
      connect-timeout: 1s
      timeout: 1s

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics
  endpoint:
    health:
      show-details: always

app:
  summary:
    cache-prefix: security:alert-summary:v1
    cache-ttl: 10m
    schedule-delay-ms: 300000
    window-hours: 24
    tenants:
      - tenant-a
      - tenant-b

logging:
  level:
    com.example.security: INFO
Enter fullscreen mode Exit fullscreen mode

The local cache time-to-live is ten minutes. The scheduled refresh runs every five minutes. Each summary covers alerts detected during the previous 24 hours. The scheduler refreshes the two sample tenants.

Create src/main/java/com/example/security/SecurityApplication.java:

package com.example.security;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;

@EnableScheduling
@SpringBootApplication
public class SecurityApplication {

    public static void main(String[] args) {
        SpringApplication.run(SecurityApplication.class, args);
    }
}
Enter fullscreen mode Exit fullscreen mode

Step 4: Implement the Data Model and Repository

Create src/main/java/com/example/security/domain/SecurityAlert.java:

package com.example.security.domain;

import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;

import java.time.Instant;

@Entity
@Table(name = "security_alert")
public class SecurityAlert {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "tenant_id", nullable = false)
    private String tenantId;

    @Column(nullable = false)
    private String severity;

    @Column(nullable = false)
    private String status;

    @Column(name = "source_ip")
    private String sourceIp;

    @Column(name = "asset_id")
    private String assetId;

    @Column(name = "detected_at", nullable = false)
    private Instant detectedAt;

    @Column(name = "updated_at", nullable = false)
    private Instant updatedAt;

    protected SecurityAlert() {
    }

    public Long getId() {
        return id;
    }

    public String getTenantId() {
        return tenantId;
    }

    public String getSeverity() {
        return severity;
    }

    public String getStatus() {
        return status;
    }

    public String getSourceIp() {
        return sourceIp;
    }

    public String getAssetId() {
        return assetId;
    }

    public Instant getDetectedAt() {
        return detectedAt;
    }

    public Instant getUpdatedAt() {
        return updatedAt;
    }
}
Enter fullscreen mode Exit fullscreen mode

Create src/main/java/com/example/security/repository/SeverityCount.java:

package com.example.security.repository;

public interface SeverityCount {
    String getSeverity();
    Long getTotal();
}
Enter fullscreen mode Exit fullscreen mode

Create src/main/java/com/example/security/repository/SecurityAlertRepository.java:

package com.example.security.repository;

import com.example.security.domain.SecurityAlert;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;

import java.time.Instant;
import java.util.List;

public interface SecurityAlertRepository extends JpaRepository<SecurityAlert, Long> {

    @Query(value = """
        SELECT severity AS severity, COUNT(*) AS total
        FROM security_alert
        WHERE tenant_id = :tenantId
          AND detected_at >= :fromTime
        GROUP BY severity
        """, nativeQuery = true)
    List<SeverityCount> countBySeverity(
        @Param("tenantId") String tenantId,
        @Param("fromTime") Instant fromTime
    );

    @Query(value = """
        SELECT COUNT(*)
        FROM security_alert
        WHERE tenant_id = :tenantId
          AND status = 'OPEN'
          AND detected_at >= :fromTime
        """, nativeQuery = true)
    long countOpenAlerts(
        @Param("tenantId") String tenantId,
        @Param("fromTime") Instant fromTime
    );
}
Enter fullscreen mode Exit fullscreen mode

The repository performs aggregation in PostgreSQL rather than loading all alerts into application memory. This is important even on a local machine because the implementation should preserve the correct computational boundary. PostgreSQL counts and groups the rows, while the application converts the compact result into the API response.

Create src/main/java/com/example/security/dto/AlertSummary.java:

package com.example.security.dto;

import java.time.Instant;
import java.util.Map;

public record AlertSummary(
    String tenantId,
    Instant windowStart,
    Instant generatedAt,
    long totalAlerts,
    long openAlerts,
    Map<String, Long> bySeverity,
    String deliverySource
) {
    public AlertSummary withDeliverySource(String source) {
        return new AlertSummary(
            tenantId,
            windowStart,
            generatedAt,
            totalAlerts,
            openAlerts,
            bySeverity,
            source
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

The deliverySource value makes local behavior visible. A response can identify whether it came from Redis, real-time aggregation, or scheduled refresh logic. In a larger system, the response envelope can also include cache age, source watermark, and calculation version.

Step 5: Configure Redis Serialization

Create src/main/java/com/example/security/config/RedisConfig.java:

package com.example.security.config;

import com.example.security.dto.AlertSummary;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

@Configuration
public class RedisConfig {

    @Bean
    RedisTemplate<String, AlertSummary> alertSummaryRedisTemplate(
        RedisConnectionFactory connectionFactory,
        ObjectMapper objectMapper
    ) {
        RedisTemplate<String, AlertSummary> template = new RedisTemplate<>();
        template.setConnectionFactory(connectionFactory);

        Jackson2JsonRedisSerializer<AlertSummary> valueSerializer =
            new Jackson2JsonRedisSerializer<>(objectMapper, AlertSummary.class);

        template.setKeySerializer(new StringRedisSerializer());
        template.setValueSerializer(valueSerializer);
        template.setHashKeySerializer(new StringRedisSerializer());
        template.setHashValueSerializer(valueSerializer);
        template.afterPropertiesSet();
        return template;
    }
}
Enter fullscreen mode Exit fullscreen mode

Explicit serialization keeps Redis values readable and avoids relying on Java native serialization. The key serializer stores plain string keys, and the value serializer stores the summary as JSON.

Step 6: Implement Real-Time Aggregation

Create src/main/java/com/example/security/service/AlertAggregationService.java:

package com.example.security.service;

import com.example.security.dto.AlertSummary;
import com.example.security.repository.SecurityAlertRepository;
import com.example.security.repository.SeverityCount;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.time.Clock;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

@Service
public class AlertAggregationService {

    private final SecurityAlertRepository repository;
    private final Clock clock;
    private final long windowHours;

    public AlertAggregationService(
        SecurityAlertRepository repository,
        @Value("${app.summary.window-hours}") long windowHours
    ) {
        this(repository, Clock.systemUTC(), windowHours);
    }

    AlertAggregationService(
        SecurityAlertRepository repository,
        Clock clock,
        long windowHours
    ) {
        this.repository = repository;
        this.clock = clock;
        this.windowHours = windowHours;
    }

    @Transactional(readOnly = true)
    public AlertSummary aggregate(String tenantId, String source) {
        Instant generatedAt = clock.instant();
        Instant windowStart = generatedAt.minus(windowHours, ChronoUnit.HOURS);

        List<SeverityCount> rows = repository.countBySeverity(tenantId, windowStart);
        long openAlerts = repository.countOpenAlerts(tenantId, windowStart);

        Map<String, Long> bySeverity = new LinkedHashMap<>();
        bySeverity.put("CRITICAL", 0L);
        bySeverity.put("HIGH", 0L);
        bySeverity.put("MEDIUM", 0L);
        bySeverity.put("LOW", 0L);

        for (SeverityCount row : rows) {
            bySeverity.put(row.getSeverity(), row.getTotal());
        }

        long totalAlerts = bySeverity.values()
            .stream()
            .mapToLong(Long::longValue)
            .sum();

        return new AlertSummary(
            tenantId,
            windowStart,
            generatedAt,
            totalAlerts,
            openAlerts,
            Map.copyOf(bySeverity),
            source
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

This service is the authoritative calculation path. It is used by both scheduled processing and cache-miss recovery. Keeping one aggregation implementation prevents scheduled and on-demand responses from drifting into different business definitions.

The calculation uses a UTC clock and an explicit 24-hour window. For deterministic unit tests, the package-private constructor accepts a fixed clock.

Step 7: Implement Redis Read, Write, and Backfill Behavior

Create src/main/java/com/example/security/service/AlertSummaryCache.java:

package com.example.security.service;

import com.example.security.dto.AlertSummary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.data.redis.RedisConnectionFailureException;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Service;

import java.time.Duration;
import java.util.Optional;

@Service
public class AlertSummaryCache {

    private static final Logger log = LoggerFactory.getLogger(AlertSummaryCache.class);

    private final RedisTemplate<String, AlertSummary> redisTemplate;
    private final String keyPrefix;
    private final Duration ttl;

    public AlertSummaryCache(
        RedisTemplate<String, AlertSummary> redisTemplate,
        @Value("${app.summary.cache-prefix}") String keyPrefix,
        @Value("${app.summary.cache-ttl}") Duration ttl
    ) {
        this.redisTemplate = redisTemplate;
        this.keyPrefix = keyPrefix;
        this.ttl = ttl;
    }

    public Optional<AlertSummary> get(String tenantId) {
        try {
            return Optional.ofNullable(redisTemplate.opsForValue().get(key(tenantId)));
        } catch (RedisConnectionFailureException ex) {
            log.warn("Redis read failed for tenant {}; continuing without cache", tenantId);
            return Optional.empty();
        }
    }

    public void put(String tenantId, AlertSummary summary) {
        try {
            redisTemplate.opsForValue().set(key(tenantId), summary, ttl);
        } catch (RedisConnectionFailureException ex) {
            log.warn("Redis write failed for tenant {}; response remains available", tenantId);
        }
    }

    public void evict(String tenantId) {
        try {
            redisTemplate.delete(key(tenantId));
        } catch (RedisConnectionFailureException ex) {
            log.warn("Redis eviction failed for tenant {}", tenantId);
        }
    }

    public String key(String tenantId) {
        return keyPrefix + ":tenant:" + tenantId + ":window:24h";
    }
}
Enter fullscreen mode Exit fullscreen mode

The cache wrapper treats Redis as an acceleration layer rather than the source of record. A Redis connection failure becomes a cache miss instead of causing the request to fail immediately. The application can still aggregate from PostgreSQL. If the backfill write fails, the current response remains usable, although later requests will need to aggregate again until Redis recovers.

Create src/main/java/com/example/security/service/AlertSummaryService.java:

package com.example.security.service;

import com.example.security.dto.AlertSummary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;

import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.ReentrantLock;

@Service
public class AlertSummaryService {

    private static final Logger log = LoggerFactory.getLogger(AlertSummaryService.class);

    private final AlertSummaryCache cache;
    private final AlertAggregationService aggregationService;
    private final ConcurrentHashMap<String, ReentrantLock> localLocks = new ConcurrentHashMap<>();

    public AlertSummaryService(
        AlertSummaryCache cache,
        AlertAggregationService aggregationService
    ) {
        this.cache = cache;
        this.aggregationService = aggregationService;
    }

    public AlertSummary getSummary(String tenantId) {
        return cache.get(tenantId)
            .map(summary -> summary.withDeliverySource("CACHE"))
            .orElseGet(() -> aggregateAndBackfill(tenantId));
    }

    public AlertSummary refresh(String tenantId, String source) {
        AlertSummary summary = aggregationService.aggregate(tenantId, source);
        cache.put(tenantId, summary);
        return summary;
    }

    public void evict(String tenantId) {
        cache.evict(tenantId);
    }

    private AlertSummary aggregateAndBackfill(String tenantId) {
        ReentrantLock lock = localLocks.computeIfAbsent(tenantId, ignored -> new ReentrantLock());
        lock.lock();
        try {
            return cache.get(tenantId)
                .map(summary -> summary.withDeliverySource("CACHE_AFTER_WAIT"))
                .orElseGet(() -> {
                    log.info("Cache miss for tenant {}; running real-time aggregation", tenantId);
                    return refresh(tenantId, "REAL_TIME_BACKFILL");
                });
        } finally {
            lock.unlock();
            if (!lock.hasQueuedThreads()) {
                localLocks.remove(tenantId, lock);
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

The service first reads Redis. On a miss, it obtains a per-tenant local lock, checks Redis again, performs real-time aggregation only if the key is still absent, and then backfills Redis.

The second cache check is important. Multiple requests may observe the same initial miss. The first request calculates and writes the value. Requests waiting for the lock then check Redis again and reuse that result instead of repeating the database aggregation.

The local lock is appropriate for a single local application process. If several application instances are started, each process has its own lock. A distributed lock or a Redis-based single-flight mechanism would be required to coordinate misses across instances. That extension is intentionally excluded from this local setup.

Step 8: Implement Scheduled Cache Refresh

Create src/main/java/com/example/security/service/AlertSummaryScheduler.java:

package com.example.security.service;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

import java.util.List;

@Component
public class AlertSummaryScheduler {

    private static final Logger log = LoggerFactory.getLogger(AlertSummaryScheduler.class);

    private final AlertSummaryService summaryService;
    private final List<String> tenants;

    public AlertSummaryScheduler(
        AlertSummaryService summaryService,
        @Value("${app.summary.tenants}") List<String> tenants
    ) {
        this.summaryService = summaryService;
        this.tenants = List.copyOf(tenants);
    }

    @Scheduled(
        initialDelayString = "10000",
        fixedDelayString = "${app.summary.schedule-delay-ms}"
    )
    public void refreshConfiguredTenants() {
        for (String tenantId : tenants) {
            try {
                summaryService.refresh(tenantId, "SCHEDULED_REFRESH");
                log.info("Scheduled summary refresh completed for tenant {}", tenantId);
            } catch (Exception ex) {
                log.error("Scheduled summary refresh failed for tenant {}", tenantId, ex);
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

The scheduler waits ten seconds after application startup and then refreshes each configured tenant. It runs again five minutes after the previous execution finishes. fixedDelay avoids starting a new local run while the previous method is still executing.

A failure for one tenant is caught inside the loop so that the remaining tenants can still be refreshed. Scheduled execution and real-time miss recovery use the same refresh method and therefore the same aggregation definition and cache format.

Step 9: Expose Local Development Endpoints

Create src/main/java/com/example/security/controller/AlertSummaryController.java:

package com.example.security.controller;

import com.example.security.dto.AlertSummary;
import com.example.security.service.AlertSummaryService;
import jakarta.validation.constraints.Pattern;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@Validated
@RestController
@RequestMapping("/api/v1/tenants/{tenantId}/alert-summary")
public class AlertSummaryController {

    private final AlertSummaryService summaryService;

    public AlertSummaryController(AlertSummaryService summaryService) {
        this.summaryService = summaryService;
    }

    @GetMapping
    public AlertSummary getSummary(
        @PathVariable
        @Pattern(regexp = "[a-zA-Z0-9-]{1,64}")
        String tenantId
    ) {
        return summaryService.getSummary(tenantId);
    }

    @PostMapping("/refresh")
    public AlertSummary refresh(
        @PathVariable
        @Pattern(regexp = "[a-zA-Z0-9-]{1,64}")
        String tenantId
    ) {
        return summaryService.refresh(tenantId, "MANUAL_REFRESH");
    }

    @DeleteMapping("/cache")
    public ResponseEntity<Void> evict(
        @PathVariable
        @Pattern(regexp = "[a-zA-Z0-9-]{1,64}")
        String tenantId
    ) {
        summaryService.evict(tenantId);
        return ResponseEntity.noContent().build();
    }
}
Enter fullscreen mode Exit fullscreen mode

The GET route demonstrates the normal three-tier delivery path. The POST route forces immediate aggregation and overwrites the cached summary. The DELETE route removes one tenant's key so that the next GET demonstrates real-time aggregation and backfill.

These refresh and eviction routes are local development utilities. They should not be exposed without authentication and authorization in a shared or production environment.

Step 10: Build and Run the Application

Compile and test the project:

mvn clean test
Enter fullscreen mode Exit fullscreen mode

Start the application:

mvn spring-boot:run
Enter fullscreen mode Exit fullscreen mode

After startup, check application health:

curl -s http://localhost:8080/actuator/health
Enter fullscreen mode Exit fullscreen mode

The response should indicate that the application, PostgreSQL connection, and Redis connection are available.

Wait at least ten seconds for the first scheduled refresh. The application log should contain messages similar to:

Scheduled summary refresh completed for tenant tenant-a
Scheduled summary refresh completed for tenant tenant-b
Enter fullscreen mode Exit fullscreen mode

Call the summary endpoint:

curl -s http://localhost:8080/api/v1/tenants/tenant-a/alert-summary
Enter fullscreen mode Exit fullscreen mode

Because the scheduler has already populated Redis, deliverySource should be CACHE. The result should contain the total count, open count, severity counts, window start, and generation timestamp.

Inspect the Redis key directly:

docker exec -it security-redis redis-cli \
  GET security:alert-summary:v1:tenant:tenant-a:window:24h
Enter fullscreen mode Exit fullscreen mode

Check its remaining time-to-live:

docker exec -it security-redis redis-cli \
  TTL security:alert-summary:v1:tenant:tenant-a:window:24h
Enter fullscreen mode Exit fullscreen mode

The TTL should be positive and no greater than 600 seconds.

Step 11: Verify Cache Miss and Backfill Locally

Evict the cached summary through the local endpoint:

curl -i -X DELETE \
  http://localhost:8080/api/v1/tenants/tenant-a/alert-summary/cache
Enter fullscreen mode Exit fullscreen mode

Confirm that the key is absent:

docker exec -it security-redis redis-cli \
  EXISTS security:alert-summary:v1:tenant:tenant-a:window:24h
Enter fullscreen mode Exit fullscreen mode

The result should be 0.

Request the summary again:

curl -s http://localhost:8080/api/v1/tenants/tenant-a/alert-summary
Enter fullscreen mode Exit fullscreen mode

The first response after eviction should report REAL_TIME_BACKFILL. The application has queried PostgreSQL, built the summary, returned it, and written it to Redis.

Call the same endpoint again:

curl -s http://localhost:8080/api/v1/tenants/tenant-a/alert-summary
Enter fullscreen mode Exit fullscreen mode

The next response should report CACHE, demonstrating that the real-time result was backfilled successfully.

Inspect the Redis key again:

docker exec -it security-redis redis-cli \
  GET security:alert-summary:v1:tenant:tenant-a:window:24h
Enter fullscreen mode Exit fullscreen mode

The value should now exist and contain the result generated by the cache-miss path.

Step 12: Verify Data Changes and Scheduled Refresh

Insert a new critical alert into PostgreSQL:

docker exec -it security-postgres \
  psql -U security_user -d security_ops \
  -c "INSERT INTO security_alert (tenant_id, severity, status, source_ip, asset_id, detected_at) VALUES ('tenant-a', 'CRITICAL', 'OPEN', '203.0.113.99', 'server-099', CURRENT_TIMESTAMP);"
Enter fullscreen mode Exit fullscreen mode

Call the normal summary endpoint immediately:

curl -s http://localhost:8080/api/v1/tenants/tenant-a/alert-summary
Enter fullscreen mode Exit fullscreen mode

The response may still contain the previous cached count. This is expected because the cache entry has not yet been refreshed. The example deliberately demonstrates the freshness trade-off introduced by caching.

Force a refresh without waiting five minutes:

curl -s -X POST \
  http://localhost:8080/api/v1/tenants/tenant-a/alert-summary/refresh
Enter fullscreen mode Exit fullscreen mode

The response should report MANUAL_REFRESH and include the additional critical alert. A subsequent GET should report CACHE and return the updated count.

To observe scheduled behavior instead, insert another alert and wait for the configured five-minute fixed delay. After the scheduler completes, the next GET will return the refreshed cached result.

For faster local experimentation, temporarily change the schedule in application.yml:

app:
  summary:
    schedule-delay-ms: 30000
Enter fullscreen mode Exit fullscreen mode

This changes the local refresh delay to 30 seconds. Restart the application after changing the configuration. Restore five minutes after completing the experiment so that local behavior matches the intended design.

Step 13: Verify Redis Failure Fallback

Stop only the Redis container:

docker compose stop redis
Enter fullscreen mode Exit fullscreen mode

Call the summary endpoint:

curl -s http://localhost:8080/api/v1/tenants/tenant-a/alert-summary
Enter fullscreen mode Exit fullscreen mode

The cache wrapper should log a Redis read warning and continue to PostgreSQL aggregation. The response should report REAL_TIME_BACKFILL. The attempted backfill will also log a warning because Redis is unavailable, but the current API response should still be returned.

Call the endpoint several times while Redis remains stopped. Each request will aggregate from PostgreSQL because no shared cached value is available. This demonstrates why the fallback must remain bounded in a larger system. A Redis outage transfers read pressure to PostgreSQL.

Restart Redis:

docker compose start redis
Enter fullscreen mode Exit fullscreen mode

Wait until it is healthy:

docker compose ps
Enter fullscreen mode Exit fullscreen mode

Call the endpoint once. If the key was not preserved or has expired, the application performs real-time aggregation and backfills Redis. The following call should use CACHE.

If the application reports Redis connection errors for a short period after restart, wait a few seconds for the connection factory to reconnect and retry the request.

Step 14: Add Focused Local Tests

Create src/test/java/com/example/security/AlertAggregationServiceTest.java:

package com.example.security;

import com.example.security.dto.AlertSummary;
import com.example.security.repository.SecurityAlertRepository;
import com.example.security.repository.SeverityCount;
import com.example.security.service.AlertAggregationService;
import org.junit.jupiter.api.Test;

import java.time.Clock;
import java.time.Instant;
import java.time.ZoneOffset;
import java.util.List;

import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

class AlertAggregationServiceTest {

    @Test
    void aggregatesSeverityAndOpenAlertCounts() {
        SecurityAlertRepository repository = mock(SecurityAlertRepository.class);
        Clock clock = Clock.fixed(
            Instant.parse("2026-11-19T10:00:00Z"),
            ZoneOffset.UTC
        );

        SeverityCount critical = severityCount("CRITICAL", 2L);
        SeverityCount high = severityCount("HIGH", 3L);

        Instant expectedStart = Instant.parse("2026-11-18T10:00:00Z");
        when(repository.countBySeverity("tenant-a", expectedStart))
            .thenReturn(List.of(critical, high));
        when(repository.countOpenAlerts("tenant-a", expectedStart))
            .thenReturn(4L);

        AlertAggregationService service =
            new AlertAggregationService(repository, clock, 24);

        AlertSummary result = service.aggregate("tenant-a", "TEST");

        assertThat(result.totalAlerts()).isEqualTo(5L);
        assertThat(result.openAlerts()).isEqualTo(4L);
        assertThat(result.bySeverity().get("CRITICAL")).isEqualTo(2L);
        assertThat(result.bySeverity().get("HIGH")).isEqualTo(3L);
        assertThat(result.bySeverity().get("MEDIUM")).isZero();
        assertThat(result.deliverySource()).isEqualTo("TEST");
    }

    private SeverityCount severityCount(String severity, Long total) {
        return new SeverityCount() {
            @Override
            public String getSeverity() {
                return severity;
            }

            @Override
            public Long getTotal() {
                return total;
            }
        };
    }
}
Enter fullscreen mode Exit fullscreen mode

The package-private test constructor used above is located in com.example.security.service, while the test is in com.example.security. To compile this exact test, either place the test in the service package or make the constructor public. The cleaner option is to change the first line of the test to:

package com.example.security.service;
Enter fullscreen mode Exit fullscreen mode

Then add imports for the DTO and repository interfaces as shown.

Run the tests:

mvn test
Enter fullscreen mode Exit fullscreen mode

For the local implementation, manual verification of Redis miss, backfill, expiration, scheduled refresh, and Redis outage behavior remains useful because those flows cross application and container boundaries.

Step 15: Tune the Local Development Configuration

Use short cache and schedule intervals only while debugging. A practical local configuration is a two-minute cache TTL and a 30-second scheduler delay:

app:
  summary:
    cache-ttl: 2m
    schedule-delay-ms: 30000
Enter fullscreen mode Exit fullscreen mode

This makes expiration and refresh behavior visible without long waits. It also means that scheduled refresh will normally replace the key before it expires.

Use a cache TTL longer than the scheduled refresh interval. If the scheduler runs every five minutes and the TTL is only four minutes, the key normally disappears before the next scheduled refresh. That creates unnecessary cache misses. A ten-minute TTL with a five-minute refresh interval provides a simple local buffer.

Keep the Redis timeout short. Redis is the fast path. A one-second timeout is already generous for a local machine. If Redis is unavailable, the application should move to its fallback rather than wait through repeated long retries.

Keep the database connection pool small for local development. Ten connections are enough for this sample and make accidental fallback pressure easier to observe. A very large local pool can hide concurrency problems until deployment.

Enable SQL logging temporarily when investigating aggregation behavior:

logging:
  level:
    org.hibernate.SQL: DEBUG
    org.hibernate.orm.jdbc.bind: TRACE
Enter fullscreen mode Exit fullscreen mode

Disable detailed bind logging after troubleshooting because it can produce large logs and may expose sensitive values.

Inspect current Redis keys during development:

docker exec -it security-redis redis-cli --scan \
  --pattern 'security:alert-summary:v1:*'
Enter fullscreen mode Exit fullscreen mode

Delete only sample summary keys when resetting cache behavior:

docker exec -it security-redis redis-cli --scan \
  --pattern 'security:alert-summary:v1:*' | \
  xargs -r docker exec -i security-redis redis-cli DEL
Enter fullscreen mode Exit fullscreen mode

Avoid FLUSHALL when Redis is shared with unrelated local projects.

Inspect PostgreSQL query behavior:

docker exec -it security-postgres \
  psql -U security_user -d security_ops \
  -c "EXPLAIN ANALYZE SELECT severity, COUNT(*) FROM security_alert WHERE tenant_id = 'tenant-a' AND detected_at >= CURRENT_TIMESTAMP - INTERVAL '24 hours' GROUP BY severity;"
Enter fullscreen mode Exit fullscreen mode

The query should use the tenant and time index as the dataset grows. With only a few seed rows, PostgreSQL may choose a sequential scan because it is cheaper. That is normal for a tiny local dataset.

Step 16: Reset or Stop the Local Environment

Stop the application with Ctrl+C in the terminal where Maven is running.

Stop PostgreSQL and Redis while preserving their data:

docker compose stop
Enter fullscreen mode Exit fullscreen mode

Start them again later:

docker compose start
Enter fullscreen mode Exit fullscreen mode

Remove the containers while preserving named volumes:

docker compose down
Enter fullscreen mode Exit fullscreen mode

Remove the containers and all local PostgreSQL and Redis data:

docker compose down -v
Enter fullscreen mode Exit fullscreen mode

After removing volumes, the next docker compose up -d initializes PostgreSQL again and reruns schema.sql and data.sql.

The completed local flow is straightforward. Scheduled processing computes shared summaries before demand arrives. Redis serves the normal request path. A missing key activates real-time PostgreSQL aggregation. The successful result is returned and backfilled into Redis. Later requests reuse that cached result until the next scheduled refresh, manual refresh, eviction, or expiration.

Top comments (0)