DEV Community: Pavel Zeger

Optimizing Spring Boot dynamic configuration using Kubernetes ConfigMap (part 2)

Pavel Zeger — Wed, 02 Oct 2024 15:07:41 +0000

In my previous article, Optimizing Spring Boot dynamic configuration using Kubernetes ConfigMap (part 1), I discussed a ConfigMap integration approach different from the one suggested by the Spring Cloud team, focusing on the Kubernetes implementation. Now, let’s dive into the Spring Boot side and explore how seamlessly we can integrate ConfigMap key-value pairs into a Spring Boot application.

Defining Properties

We'll start by defining the necessary properties in application.properties to integrate with the ConfigMap. This allows the service to use default values in case Kubernetes does not provide them:

kubernetes.config.map.external-api-enabled=true
kubernetes.config.map.internal-api-enabled=true

Next, we’ll configure a reactive WebClient that fetches the ConfigMap via the Kubernetes API:

kubernetes.config.map.client.uri=https://<Kubernetes host>:443/api/v1/namespaces/production/configmaps/${spring.application.name}
kubernetes.config.map.client.token=${token}
kubernetes.config.map.client.timeout=500
kubernetes.config.map.client.max-attempts=5
kubernetes.config.map.client.min-backoff=100
kubernetes.config.map.client.delay-millis=${random.int(5000)}

Note: Ensure that the spring.application.name matches the name configured in Kubernetes. You will also need to create an API token for secure access to the Kubernetes API.

Configuring the WebClient

To configure the WebClient, you can refer to any Spring WebClient setup tutorial, as I won’t cover the WebClient creation in detail here. Instead, we’ll focus on creating a configuration properties bean that simplifies access to these properties as an injectable singleton.

@Validated
@ConfigurationProperties(prefix = "kubernetes.config.map.client", ignoreInvalidFields = true)
public record ConfigMapClientProperties(

        @NotNull @NotEmpty String uri,
        @NotNull @NotEmpty String token,
        @Positive int timeout,
        @Positive int maxAttempts,
        @Positive int minBackoff,
        @Positive int delayMillis

) {
}

Working with ConfigMap Data

To correctly use the ConfigMap file, we define a POJO to map the data:

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class ConfigMap {

    private ConfigMapData data;
    private boolean isLoadCompleted;

    public static boolean isNotEmpty(ConfigMap configMap) {
        return nonNull(configMap) && nonNull(configMap.getData());
    }

    @Data
    @Builder
    @NoArgsConstructor
    @AllArgsConstructor
    @JsonIgnoreProperties(ignoreUnknown = true)
    public static class ConfigMapData {

        @Builder.Default
        @JsonProperty("kubernetes.config.map.external-api-enabled")
        boolean externalApiEnabled = true;

        @Builder.Default
        @JsonProperty("kubernetes.config.map.internal-api-enabled")
        boolean internalApiEnabled = true;

    }

}

Main Components

Now that we’ve created our model, we can move on to the key components of the solution:

Custom actuator endpoint: Receives POST requests from the Spring Cloud Kubernetes Configuration Watcher.
ConfigMap loader: Fetches the ConfigMap using a reactive WebClient.
Service: Updates the values of the ConfigMap properties across the application’s services.
Initializer: Connects to the Kubernetes API at application startup to load the ConfigMap values, ensuring consistency across pods after restarts or new deployments.

Let’s dive into each class.

ConfigMap actuator endpoint

This component will receive POST request from Spring Cloud Kubernetes Configuration Watcher:

@Slf4j
@Component
@RestControllerEndpoint(id = "configmap")
public class ConfigMapEndpoint {

    private final ConfigMapService configMapService;

    public ConfigMapEndpoint(ConfigMapService configMapService) {
        this.configMapService = configMapService;
    }

    @PostMapping(path = {"/refresh"})
    public Mono<Void> configMapRefresh() {
        return configMapService.refreshProperties();
    }

    @GetMapping(path = "/")
    public Mono<ConfigMap> getConfigMap() {
        return configMapService.loadConfigMap();
    }

}

ConfigMap Loader

This component uses the WebClient to load the ConfigMap from the Kubernetes API:

public interface ConfigMapRefreshable {

    void refreshProperty(ConfigMap.ConfigMapData configMapData);

}

@Slf4j
@Component
@EnableConfigurationProperties(ConfigMapClientProperties.class)
public class ConfigMapLoader {

    private final WebClient webClient;

    @Getter
    private final ConfigMapClientProperties properties;

    public ConfigMapLoader(WebClient webClient, ConfigMapClientProperties properties) {
        this.webClient = webClient;
        this.properties = properties;
    }

    public Mono<ConfigMap> load() {
        return webClient.get()
                .uri(properties.uri())
                .headers(this::populateHeaders)
                .accept(APPLICATION_JSON)
                .retrieve()
                .bodyToMono(ConfigMap.class)
                .timeout(Duration.ofMillis(properties.timeout()))
                .retryWhen(getRetryBackoffSpec(properties))
                .doOnSuccess(this::logSuccess)
                .doOnError(this::logError)
                .map(this::updateLoadStatus)
                .delaySubscription(Duration.ofMillis(properties.delayMillis()));
    }

    private void populateHeaders(HttpHeaders headers) {
        headers.setBearerAuth(properties.token());
    }

    private RetryBackoffSpec getRetryBackoffSpec(ConfigMapClientProperties properties) {
        return Retry.backoff(
                properties.maxAttempts(),
                Duration.ofMillis(properties.minBackoff())
        );
    }

    private void logSuccess(ConfigMap configMap) {
        log.info("Retrieving ConfigMap succeeded, configMap: [{}]", configMap);
    }

    private void logError(Throwable exception) {
        String message = getMessage(exception);
        log.error("Retrieving ConfigMap from Kubernetes API failed, exception: [{}]", message);
    }

    private ConfigMap updateLoadStatus(ConfigMap configMap) {
        if (isNotEmpty(configMap)) {
            configMap.setLoadCompleted(true);
            return configMap;
        } else {
            return ConfigMap.builder().build();
        }
    }

}

ConfigMap Service

This service refreshes the application’s properties with the newly loaded ConfigMap values:

@Slf4j
@Service
public class ConfigMapService {

    private final ConfigMapLoader loader;
    private final Set<ConfigMapRefreshable> services;

    public ConfigMapService(ConfigMapLoader loader, Set<ConfigMapRefreshable> services) {
        this.loader = loader;
        this.services = services;
    }

    public Mono<Void> refreshProperties() {
        final ConfigMapClientProperties properties = loader.getProperties();
        return loadConfigMap()
                .filter(ConfigMap::isLoadCompleted)
                .repeatWhenEmpty(
                        properties.maxAttempts(),
                        flux -> flux.delayElements(Duration.ofMillis(properties.minBackoff()))
                )
                .doOnNext(this::refreshProperties)
                .then();
    }

    public Mono<ConfigMap> loadConfigMap() {
        return loader.load();
    }

    private void refreshProperties(ConfigMap configMap) {
        if (configMap.isLoadCompleted()) {
            ConfigMap.ConfigMapData data = configMap.getData();
            services.forEach(service -> service.refreshProperty(data));
            log.warn("Refreshing properties with new ConfigMap values succeeded, data: [{}], isLoadCompleted: [true]", data);
        } else {
            log.error("Refreshing properties with new ConfigMap values failed, isLoadCompleted: [false]");
        }
    }

}

ConfigMap initializer

This component connects to the Kubernetes API at the application startup to load the ConfigMap values, ensuring consistency across pods after restarts or new deployments.

@Slf4j
@Component
public class ConfigMapInitializer {

    private final ConfigMapService configMapService;

    public ConfigMapInitializer(ConfigMapService configMapService) {
        this.configMapService = configMapService;
    }

    @EventListener(classes = {ContextRefreshedEvent.class})
    public void handleConfigMapInitializing() {
        configMapService.refreshProperties()
                .publishOn(Schedulers.boundedElastic())
                .subscribe();
    }

}

Conclusion

With this setup, your Spring Boot application can dynamically load configuration values from a Kubernetes ConfigMap, ensuring that your application is always up-to-date with the latest configuration changes. This approach also allows seamless integration between Kubernetes and Spring Boot while leveraging reactive programming for efficient data retrieval.

References

Finding my articles helpful? You could give me a caffeine boost to keep them coming! Your coffee donation will keep my keyboard clacking and my ideas brewing. But remember, it's completely optional. Stay tuned, stay informed, and perhaps, keep the coffee flowing!

Optimizing Spring Boot dynamic configuration using Kubernetes ConfigMap (part 1)

Pavel Zeger — Tue, 02 Jan 2024 11:34:30 +0000

Disclaimer

This article represents my perspective on this solution, thus any suggestions, fixes, or discussions will be highly appreciated.

The short story

In the ever-evolving landscape of container orchestration, Kubernetes stands as a beacon of efficiency and scalability. However, for those deeply entrenched in the realm of managing Kubernetes Pods, the recurrent need to update properties across multiple instances can become a daunting task, consuming valuable time and resources. Imagine a scenario where a critical change needs to be applied uniformly across a myriad of Pods, demanding a repetitive and error-prone process. It is in this challenge that the quest for a seamless solution arises — a single update that ripples through the entire Kubernetes environment, ensuring consistency without the hassle of individually addressing each Pod.
ConfigMaps offer a method that allows for a one-time update, streamlining operations and bringing efficiency to the forefront of containerized environments.

The Spring framework currently offers two distinct approaches, both integrated within the Spring Cloud project:

Direct access from pods to Kubernetes resources.
This method involves accessing Kubernetes resources directly from each pod through a scheduled process. However, this approach was deprecated in 2020.
Spring Cloud Kubernetes Configuration Watcher Service.
Alternatively, the Spring Cloud team has introduced a specialized service, the Spring Cloud Kubernetes Configuration Watcher. This service is designed to recognize changes in ConfigMaps and trigger a reload of the IoC Spring container by reading a locally mounted ConfigMap file on each pod.

While both approaches have their pros and cons, it's important to highlight concerns with the first approach. This method is considered suboptimal as it necessitates resource allocation for a scheduled process. Additionally, the service's pod needs to trigger the process, which is architecturally problematic when a service attempts to trigger an external resource that has changed when each pods actually is not aware of such changes.

On the other hand, the second approach initiates a series of flows when an external resource changes, monitoring and responding to these changes. Both methods involve restarting beans or the entire context, potentially disrupting service. To address this, a flow can be created, triggered by the Spring Cloud Kubernetes Configuration Watcher. This flow would utilize a service's HTTP client to connect to the Kubernetes API, pulling ConfigMap key-value pairs to update properties for each serving class holding these properties.

The Spring Cloud Kubernetes Configuration Watcher Helm chart deploys an instance of this service. This service detects changes in a specific ConfigMap and sends empty HTTP POST requests to a custom actuator endpoint (http:<hostname>:<port>/actuator/configmap/refresh). This invocation prompts the service's HTTP client to send a GET request to the Kubernetes API, obtaining the updated ConfigMap file with key-value pairs that override the current values within the service.

Below the first part that will help to implement my approach with changes relates mostly to Kubernetes. The second part will describe service's changes that will support the chosen approach.

What is Kubernetes ConfigMap?

In Kubernetes, ConfigMap is a resource designed to handle the configuration needs of applications separately from their codebase. It serves as a key-value store or a provider of configuration data in the form of plain text or configuration files. ConfigMap offers a versatile and efficient solution for handling configuration data in containerized applications.

Key characteristics of ConfigMap

Decoupling configuration from an application code.

ConfigMap promotes the separation of configuration concerns from the application code. By externalizing configuration data, changes can be made to application settings without modifying or redeploying the application itself. This decoupling aligns with best practices, fostering modular and adaptable application architectures.

Key-Value pairs and configuration files.

ConfigMaps support two primary modes of storing configuration data. The first is through key-value pairs, allowing for simple configurations. The second involves mounting entire configuration files, which is particularly useful for complex configurations or scenarios where maintaining the structure of a configuration file is essential.

Environment specific configurations.

Microservices and containerized applications often traverse various environments, from development and testing to production. ConfigMap enables the creation of environment-specific configurations, ensuring that applications seamlessly adapt to different runtime conditions.

Live updates and dynamic configuration.

ConfigMaps support live updates, allowing for dynamic configuration changes at runtime. This feature is especially valuable in scenarios where applications need to adapt quickly to changing requirements without the need for restarts or redeployment.

Creation and usage examples of ConfigMaps

Declarative configuration in YAML or JSON: ConfigMaps are typically defined declaratively using YAML or JSON files. These files specify the configuration data, either as key-value pairs or by referencing external configuration files.

apiVersion: v1
kind: ConfigMap
metadata:
    name: example-config
data:
    key1: value1
    key2: value2

Imperative creation: ConfigMaps can also be created imperatively using the kubectl command line:

kubectl create configmap \
    example-config \
    --from-literal=key1=value1 \
    --from-literal=key2=value2

Mounting ConfigMap in pods: Once a ConfigMap is created, it can be mounted into the file system of pods as a volume or used as environment variables.

apiVersion: v1
kind: Pod
metadata:
    name: example-pod
spec:
    containers:
    - name: example-container
      image: example-image
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
volumes:
    - name: config-volume
configMap:
    name: example-config

Remember, mounting to each pod will increase amount of time for updating ConfigMap on the mounted volume. The maximum time Kubernetes will perform it can take more than 120,000 milliseconds.

Use cases and considerations

Configuration of application.properties.

ConfigMap is particularly useful for managing application properties, environment variables, and other configuration details that might vary across different deployment environments. The foundation of Spring Boot's configuration lies in properties and YAML files. These files provide a simple and effective means of configuring applications. However, as microservices environments grow in complexity, the need for externalized configuration management becomes more pronounced.

Configuration for multiple microservices.

In a microservices architecture, ConfigMaps can be employed to manage configurations for multiple services, providing a centralized and consistent approach to configuration management.

Sensitive information and security.

While ConfigMap is suitable for non-sensitive configuration data, Kubernetes Secrets are recommended for managing sensitive information such as passwords and API keys.

Integration with Spring Cloud Kubernetes Watcher.

ConfigMaps seamlessly integrate with tools like Spring Cloud Kubernetes Watcher enabling dynamic updates to configurations without requiring application restarts.

Comparison with Consul, etcd, or Spring Cloud Config Server.

Alternatively, some organizations adopt tools like Consul, etcd, or Spring Cloud Config Server for centralized configuration management. While these tools offer robust solutions, the simplicity and native integration of ConfigMap make it an attractive choice, especially in Kubernetes-centric environments.

Implementation of ConfigMaps

Let's assume our services have 2 ports: one for a main flow (8888) and another one for service's management (9999).

ConfigMap

Each service will have its own ConfigMap within its Helm chart. ConfigMap file will include key-value pairs of application properties to be updated dynamically. These pairs can be added into data key only:

{{- if .Values.configmap.enabled -}}
apiVersion: v1
kind: ConfigMap
metadata:
  name: "{{ .Release.Name }}"
  namespace: production
  labels:
    spring.cloud.kubernetes.config: "true"
    spring.cloud.kubernetes.secret: "false"
  annotations:
    spring.cloud.kubernetes.configmap.apps: "{{ .Release.Name }}"
data:
  kubernetes.config.map.external-api-enabled: "true"
  kubernetes.config.map.internal-api-enabled: "true"
{{- end }}

{{ .Release.Name }} will be you actual spring.application.name property you've already defined within your application. Otherwise it can be a default application, but I suppose you will not do it ad define for each service the dedicated name for future convenience.

The key has the following naming convention: <source name>.<resource name>.<key/property name> where:

<source name> will be kubernetes
<resource name> will be config.map (splitter by a dot for a better readability)
<key name> will be external-api-enabled property name (splitter by a hyphen to allow for Spring Boot parse a key-value into a camel case class field, e.g.: externalApiEnabled)

Implementation

The following steps need to be performed in order to add ConfigMap support for services:

Add the following configuration into values.yaml file within the Helm charts of a service:

configmap:
  enabled: true

It will allow you to trigger the first clause within a ConfigMap, that actually enables it: {{- if .Values.configmap.enabled -}}.

Add ConfigMap.yaml file into Helm charts templates directory.
Rename service.yaml file into service-one-main.yaml file and update the metadata.name value by adding the suffix -main:

apiVersion: v1
kind: Service
metadata:
  name: "{{ .Release.Name }}-main"
spec:
  ports:
    - port: {{ .Values.port }}
      targetPort: {{ .Values.port }}
      protocol: TCP
      name: "http-{{ .Values.port }}"
  selector:
    app: {{ .Release.Name }}

For each application's port there is a dedicated service.yaml file defines specifications related to this port: main or maintenance port. The value {{ .Values.port }} allows you to add a port's variable into the main values.yaml file for a convenient management of charts.

Update service-act.yaml file content by adding annotations key-value pair:

apiVersion: v1
kind: Service
metadata:
  name: {{ .Release.Name }}
  labels:
    app: {{ .Release.Name }}
  annotations:
    boot.spring.io/actuator: "http://:{{ .Values.act_port }}/actuator/configmap"
spec:
  ports:
    - port: {{ .Values.act_port }}
      targetPort: {{ .Values.act_port }}
      protocol: TCP
      name: "http-{{ .Values.act_port }}"
  selector:
    app: {{ .Release.Name }}

This file defines a Kubernetes service for a management port I've added (9999).
boot.spring.io/actuator key represents the custom actuator endpoint /configmap I've added to our service. It can any path you want because Spring Boot allows to you programmatically define any new actuator path. The reason I've added it is to control the behavior of this endpoint by adding my own logic and metrics if needed.

Update a service.name value by adding the suffix -main within ingress.yaml file to allow a traffic split (if you have such implementation) to the reconfigured main Service resource:

  - host: service-one.com
    http:
      paths:
      - backend:
          service:
            name: "{{ .Release.Name }}-main"
            port:
              number: {{ .Values.port }}
        path: /readiness
        pathType: Prefix
  - host: service-one.com
    http:
      paths:
      - backend:
          service:
            name: "{{ .Release.Name }}-main"
            port:
              number: {{ .Values.port }}

Update region/deployment related *-values.yaml files to include a new reconfigured Service name within the traffic split configuration if you have some, e.g. us-east-1-service-values.yaml:

ts:
  backends:
  - service: service-one
    weight: 99
  - service: service-one-main
    weight: 1
  service: service-one

Add new Service name and namespace into role-binding.yaml file within Helm charts of Spring Cloud Kubernetes Configuration Watcher:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-configmap-binding
subjects:
- kind: ServiceAccount
  name: username
  namespace: production
- kind: ServiceAccount
  name: service-one
  namespace: production
roleRef:
  kind: Role
  name: read-configmap
  apiGroup: rbac.authorization.k8s.io

username represents a role binding for each service that will connect to Kubernetes API.

Add Helm chart project name into read-configmap-role.yaml file within Helm charts of Spring Cloud Kubernetes Configuration Watcher:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: read-configmap
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["service-one", "service-two"]
  verbs: ["get", "watch", "list"]

Each name will be added into the resourceName array.

Update spring.application.name property within application.properties file of the service in order to allow to Kubernetes recognize the pod of this service.

This application name must be the same as a Helm chart Service name (e.g: service-one, service-two). The same name will be provided into the property kubernetes.config.map.client.uri in order to get the ConfigMap file by Kubernetes API from the dedicated resource defines the service deployment.

info.app.name=@project.name@
spring.application.name=${info.app.name}
kubernetes.config.map.client.uri=https://<Kubernetes API address>:<Kubernetes API port>/api/v1/namespaces/production/configmaps/${spring.application.name}

Enable /configmap actuator endpoint by updating the following application properties:

management.endpoint.configmap.enabled=true
management.endpoints.web.exposure.include=configmap,...

Implementation validation

For a convenient monitoring the Spring Cloud Watcher runs with the DEBUG logging level.
The following log clearly describes steps during updating ConfigMap key-value pairs:

2023-11-22 11:44:24.150  INFO 1 --- [           main] .c.w.HttpBasedSecretsWatchChangeDetector : Kubernetes event-based secrets change detector activated
2023-11-22 11:44:24.153 DEBUG 1 --- [//10.32.0.1/...] .w.HttpBasedConfigMapWatchChangeDetector : Scheduling remote refresh event to be published for ConfigMap service-one to be published in 120000 milliseconds
2023-11-22 11:44:24.165 DEBUG 1 --- [//10.32.0.1/...] .w.HttpBasedConfigMapWatchChangeDetector : Scheduling remote refresh event to be published for ConfigMap service-two to be published in 120000 milliseconds
2023-11-22 11:46:24.188 DEBUG 1 --- [oundedElastic-4] .w.HttpBasedConfigMapWatchChangeDetector : Metadata actuator uri is: http://:<pod's port>/actuator/configmap
2023-11-22 11:46:24.188 DEBUG 1 --- [oundedElastic-1] .w.HttpBasedConfigMapWatchChangeDetector : Metadata actuator uri is: http://:<pod's port>/actuator/configmap
2023-11-22 11:46:24.254 DEBUG 1 --- [oundedElastic-2] .w.HttpBasedConfigMapWatchChangeDetector : Found actuator URI in service instance metadata
2023-11-22 11:46:24.254 DEBUG 1 --- [oundedElastic-1] .w.HttpBasedConfigMapWatchChangeDetector : Found actuator URI in service instance metadata
2023-11-22 11:46:24.254 DEBUG 1 --- [oundedElastic-2] .w.HttpBasedConfigMapWatchChangeDetector : Sending refresh request for service-one to URI http://<pod's IP address>:<pod's port>/actuator/configmap/refresh
2023-11-22 11:46:24.254 DEBUG 1 --- [oundedElastic-1] .w.HttpBasedConfigMapWatchChangeDetector : Sending refresh request for service-two to URI http://<pod's IP address>:<pod's port>/actuator/configmap/refresh
2023-11-22 11:46:24.337 DEBUG 1 --- [or-http-epoll-6] .w.HttpBasedConfigMapWatchChangeDetector : Refresh sent to service-one at URI address http://<pod's IP address>:<pod's port>/actuator/configmap/refresh returned a 200 OK
2023-11-22 11:46:24.345 DEBUG 1 --- [r-http-epoll-10] .w.HttpBasedConfigMapWatchChangeDetector : Refresh sent to service-two at URI address http://<pod's IP address>:<pod's port>/actuator/configmap/refresh returned a 200 OK

If by any reason the changes in ConfigMap weren’t recognized by a watcher, the deleting of its pod or restarting it will trigger sending HTTP POST requests to all defined services within a watcher. Therefore you will update all services with their ConfigMaps.

Possible issues

This complex implementation mainly depends on Kubernetes resources configurations. Spring Cloud Kubernetes Configuration Watcher uses regular reactive Spring Boot. You can check its code here.

To prevent Kubernetes API failures there is a delay configuration (a service WebClient flow) in sending HTTP GET requests from each pod defined by a random value of time but less than 5 seconds.
It’s not recommended to change the watcher refresh delay configuration (2 minutes) because this value represents a possible maximum time for Kubernetes to update resources (it relates mainly for mounted ConfigMap).

References

Enhancing team collaboration in IntelliJ IDEA with the .run directory

Pavel Zeger — Fri, 27 Oct 2023 08:59:24 +0000

Effective collaboration is a key driver of success in software development. When working in a team, it's crucial to streamline processes and minimize friction. IntelliJ IDEA, one of the most popular Integrated Development Environments (IDEs), provides various features to facilitate teamwork. Among these features is the .run directory.

What is the `.run` directory?

The .run directory is a hidden directory that IntelliJ IDEA automatically creates within your project's root directory. It's used to store run configurations, which are settings that define how your application should be executed locally or debugged. Run configurations can include information about command-line arguments, environment variables, working directories, and more.

The utilization of this directory offers several advantages:

Standardization of run configurations

In a team environment, consistency is crucial. Different team members may have their own run configurations for running the same application, which can lead to confusion and errors. The .run directory allows you to define and share standardized run configurations. Team members can access these configurations easily, ensuring that everyone runs the application with the same settings.

Version control integration

The .run directory and its contents are meant to be committed to version control (e.g., Git). This means that run configurations become part of the project's source code. By sharing run configurations in the project repository, you make it easy for team members to sync their development environments with a simple Git pull operation. No more struggling with setting up run configurations manually.

Portability

Sharing run configurations through the .run directory makes your project more portable. Team members can clone the project, import run configurations, and start working without having to reconfigure their local IDE settings. This is especially useful when onboarding new team members or when working on different development machines.

Reduced maintenance effort

With run configurations stored in the .run directory, there's less need for constant maintenance. As project requirements change, updates can be made to the shared run configurations, ensuring that everyone remains on the same page. This minimizes the risk of configuration drift over time.

How to use the `.run` directory

Create standardized run configurations: create run configurations for your project based on the application's requirements. Ensure that these configurations are tailored for your team's needs and follow best practices.
Save run configurations to the .run directory: after creating a run configuration, IntelliJ IDEA allows you to save it to the .run directory. This can be done by selecting "Store as project file" when configuring the run configuration.
Commit to your Version Control System: commit the .run directory, including run configurations, to your project's version control system.
Team usage: instruct team members to import the run configurations from the .run directory when setting up their development environments. This can be done using the "Import Project" or "Import Settings" options in IntelliJ IDEA.

Conclusion

The .run directory in IntelliJ IDEA is a powerful tool for enhancing team collaboration in software development. By standardizing run configurations, integrating them with version control, and making them easily portable, you can significantly reduce configuration-related issues, streamline onboarding, and create a more efficient and consistent development workflow. Embrace the .run directory as part of your team's development practices to ensure that everyone is on the same page and working efficiently.

Resources

Share run/debug configurations

Organizing custom metrics for effective application monitoring

Pavel Zeger — Fri, 13 Oct 2023 08:37:44 +0000

Disclaimer

This article represents my perspective on this solution, thus any suggestions, fixes, or discussions will be highly appreciated.

The short story

In many projects I've been a part of, I've witnessed the challenges caused by disorganized metrics. Team members often struggled to discern which metrics were actively in use, especially in projects lacking a well-architected structure. In such cases, it wasn't uncommon to encounter classes like 'MetricService,' which aggregated static methods for various counters without any logical organization. To address these issues, I propose the approach described below to effectively organize this crucial aspect of application monitoring.

The intro

In the world of backend development, metrics are the eyes and ears of your application. They provide critical insights into its performance, allowing you to spot issues, optimize code, and ensure a seamless user experience. However, when metrics are scattered and disorganized, they become more of a headache than a helpful tool. In this article, we'll explore the challenges of disorganized metrics and present a practical solution: organizing all your metrics within a dedicated package and using a dedicated enum.

The importance of well-organized metrics

Metrics are essential in understanding the heartbeat of your backend application. They serve as early warning systems, helping you detect and diagnose issues promptly. Well-organized metrics offer several key advantages:

Clarity: metrics should be self-explanatory. Organized metrics make it clear what is being monitored, reducing confusion during troubleshooting.
Efficiency: accessing and analyzing metrics is significantly faster when they are neatly organized. This streamlines the debugging process.
Scalability: as your application grows, so does the number of metrics. An organized system makes it easier to manage a large number of metrics.

Dealing with disorganized metrics can lead to a range of problems:

Confusion: it's challenging to make sense of metrics when they have vague or inconsistent names. This can lead to mistakes and delays in issue resolution.
Time-consuming analysis: scouring through countless metrics to find the relevant ones wastes valuable time, especially during critical incidents.
Maintenance nightmares: as your application evolves, so do the metrics. Without proper organization, maintenance becomes a nightmare.

The solution: dedicated package and enums

To overcome these challenges, I propose a two-fold solution:

Dedicated packages for metrics: create a dedicated package or namespace for metrics. This isolates them from other parts of your codebase, enhancing clarity.
Dedicated enums for categories: use an enum to categorize and label your metrics. This adds a layer of organization and makes metrics more self-explanatory.

Organizing metrics in this way offers numerous advantages:

Clear hierarchy: metrics grouped within packages and categorized by enums create a clear hierarchy that simplifies navigation and comprehension.
Easy maintenance: when new metrics are added, or existing ones are modified, it's much easier to manage and update them when they're organized.
Enhanced collaboration: team members can quickly understand and work with metrics, which fosters better collaboration.

The implementation

Create dedicated package: designate a package, directory, or folder in your codebase for metrics. For example, com.project.util.metric.
Utilize enums: create an enum for metric categories: Metric. This can include also categories like BusinessMetric, ApiMetric and so on.

@Getter
@RequiredArgsConstructor
public enum Metric {

    METRIC_ONE("metric.one"),
    METRIC_TWO("metric.two");

    private final String name;

}

Consistent naming: ensure that all metrics follow a consistent naming convention, and use the enum values as labels:

@Getter
@RequiredArgsConstructor
public enum Label {

    STATUS("status"),
    TYPE("type");

    private final String name;

}

@Getter
@RequiredArgsConstructor
public enum Value {

    SUCCESS("success"),
    FAILURE("failure");

    private final String name;

}

Documentation: document your metrics and their categories. This provides a reference for your team and future developers. If you use Maven use it: Apache Maven Javadoc Plugin.
Clear architecture: avoid the temptation to lump all your counters into a monolithic 'god' class. This can lead to uncontrolled growth and make code reading and maintenance a cumbersome task. Instead, if a specific class uses certain metrics, place those counters within that class for better organization and maintainability.

Conclusion

In the world of backend development, organized metrics are not just a luxury but a necessity. Well-structured metrics offer clarity, efficiency, and scalability, making your life as a developer much easier. By using dedicated packages and enums, you can transform metrics from a headache into a valuable asset in your toolbox. Implement this strategy, and you'll be well on your way to mastering the art of organized metrics.

Reactive MongoDB with multi-database within a single Spring Boot application

Pavel Zeger — Sun, 24 Sep 2023 08:22:45 +0000

Disclaimer

This article represents my perspective on this solution, thus any suggestions, fixes, or discussions will be highly appreciated.

The short story

Let's assume that in any circumstance you have a few databases you need to use to catch/write data within a single Spring Boot application. It can be a result of legacy approach or a result of DBA works etc.
Spring Boot will do it easily for you. Let's start.

The implementation

In many real-world scenarios, Spring Boot applications need to interact with multiple databases. Whether it's due to legacy systems, the need for polyglot persistence, or other reasons, Spring Boot simplifies the task of managing multiple MongoDB databases, even in a reactive environment. In this article, we'll walk you through the implementation steps to seamlessly support multiple MongoDB databases within a single Spring Boot application using reactive programming.

Step 1: Configuration setup

Begin by configuring your Spring Boot application to connect to the MongoDB databases. In your application.properties (or application.yml) file, define the connection details for each database, specifying database names, connection URIs, and other necessary properties. To ensure clarity, focus on the essential MongoDB configuration properties.



spring.data.mongodb.databaseOne.database=databaseOne
spring.data.mongodb.databaseTwo.database=databaseTwo
spring.data.mongodb.url=mongodb://localhost:27017/test
spring.autoconfigure.exclude=\
org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration,\
org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration,\
org.springframework.boot.autoconfigure.data.mongo.MongoReactiveDataAutoConfiguration,\
org.springframework.boot.autoconfigure.data.mongo.MongoReactiveRepositoriesAutoConfiguration,\
org.springframework.boot.autoconfigure.data.mongo.MongoRepositoriesAutoConfiguration

In order to use multiple database approach we need to disable MongoDB autoconfigurations by adding the property spring.autoconfigure.exclude.

Step 2: Creating reactive MongoDB Repositories

To interact with each MongoDB collection, create Spring Data MongoDB repositories for each one. In your project structure, organize these repositories according to the respective database names, enhancing code readability.



package com.demo.repository.database_one;

import org.springframework.data.mongodb.repository.ReactiveMongoRepository;

public interface CollectionOneRepository extends ReactiveMongoRepository<CollectionOne, String> {

}



package com.demo.repository.database_two;

import org.springframework.data.mongodb.repository.ReactiveMongoRepository;

public interface CollectionTwoRepository extends ReactiveMongoRepository<CollectionTwo, String> {

}

Step 3: Configuration Properties class

Simplify your code and improve maintainability by creating a configuration properties class that stores configurations per prefix. This step ensures better code organization and allows for additional validations.



package com.demo.config.mongo;

import lombok.Data;
import org.springframework.boot.autoconfigure.mongo.MongoProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Primary;
import org.springframework.validation.annotation.Validated;

import jakarta.validation.constraints.NotEmpty;
import jakarta.validation.constraints.NotNull;

@Data
@Validated
@Primary
@ConfigurationProperties(prefix = "spring.data.mongodb", ignoreUnknownFields = false)
public class BaseMongoProperties {

    @NotNull private MongoProperties databaseOne;
    @NotNull private MongoProperties databaseTwo;
    @NotNull @NotEmpty private String uri;

}

Pay attention it need to be marked as a @Primary configuration, otherwise Spring Boot will throws exception about 2 existing properties. We would like to override these properties by our custom properties.

Step 4: MongoDB Template configuration

Create a main configuration class for MongoDB templates. This class defines two MongoDB templates, one for each database, using the ReactiveMongoTemplate. With this approach we can create one MongoDb client only and reuse it for different MongoTemplates, each for a certain database.



package com.demo.config.mongo;

import com.mongodb.*;
import com.mongodb.reactivestreams.client.MongoClient;
import com.mongodb.reactivestreams.client.MongoClients;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.core.ReactiveMongoTemplate;

@Configuration
@EnableConfigurationProperties(BaseMongoProperties.class)
public class BaseMongoConfig {

    protected static final String DATABASE_ONE_MONGO_TEMPLATE = "databaseOneMongoTemplate";
    protected static final String DATABASE_TWO_MONGO_TEMPLATE = "databaseTwoMongoTemplate";

    @Bean(name = {DATABASE_ONE_MONGO_TEMPLATE})
    public ReactiveMongoTemplate databaseOneMongoTemplate(MongoClient mongoClient, BaseMongoProperties mongoProperties) {
        return new ReactiveMongoTemplate(mongoClient, mongoProperties.getDatabaseOne().getDatabase());
    }

    @Bean(name = {DATABASE_TWO_MONGO_TEMPLATE})
    public ReactiveMongoTemplate databaseTwoMongoTemplate(MongoClient mongoClient, BaseMongoProperties mongoProperties) {
        return new ReactiveMongoTemplate(mongoClient, mongoProperties.getDatabaseTwo().getDatabase());
    }

    @Bean
    public MongoClient mongoClient(BaseMongoProperties mongoProperties) {
        return MongoClients.create(getMongoClientSettings(mongoProperties));
    }

    private MongoClientSettings getMongoClientSettings(BaseMongoProperties mongoProperties) {
        return MongoClientSettings.builder()
                .applyConnectionString(new ConnectionString(mongoProperties.getUri()))
                .build();
    }

}

Step 5: Repository Configuration

We inform Spring Boot about the repositories associated with each MongoDB template by importing ReactiveMongoRepositoriesRegistrar will register our per database repositories and associate them with the dedicated MongoTemplate. Create configuration classes for each database, specifying the base package and referencing the correct MongoDB template.



package com.demo.config.mongo;

import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.repository.config.EnableReactiveMongoRepositories;

import static com.demo.config.mongo. BaseMongoConfig.DATABASE_ONE_MONGO_TEMPLATE;

@Configuration
@EnableReactiveMongoRepositories(
        basePackages = {"com.demo.repository.database_one"},
        reactiveMongoTemplateRef = DATABASE_ONE_MONGO_TEMPLATE
)
public class DatabaseOneMongoConfig {

}



package com.demo.config.mongo;

import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.repository.config.EnableReactiveMongoRepositories;

import static com.demo.config.mongo.BaseMongoConfig. DATABASE_TWO_MONGO_TEMPLATE;

@Configuration
@EnableReactiveMongoRepositories(
        basePackages = {"com.demo.repository.database_two"},
        reactiveMongoTemplateRef = DATABASE_TWO_MONGO_TEMPLATE
)
public class DatabaseTwoMongoConfig {

}

Step 6: Integration testing

Lastly, add integration testing with Testcontainers to validate the behavior of your application.

Summary

In this article, we explored the process of seamlessly integrating and managing multiple MongoDB databases within a single Spring Boot application using reactive programming. We began by configuring the application properties, specifying essential MongoDB connection details for each database.

Next, we created Spring Data MongoDB repositories for each MongoDB collection, ensuring an organized project structure that enhances code readability.

To simplify configuration and improve maintainability, we introduced a configuration properties class to store database settings per prefix. This step not only streamlined the code but also allowed for additional validations.

Our main configuration class, BaseMongoConfig, defined two MongoDB templates—one for each database—using the ReactiveMongoTemplate. This setup enabled us to interact with MongoDB collections in a reactive and non-blocking manner.

To inform Spring Boot about the repositories associated with each MongoDB template, we created separate configuration classes for each database. These classes specified the base package and referenced the correct MongoDB template, ensuring that Spring Boot could scan the repositories accurately.

Lastly, we highlighted the importance of integration testing with Testcontainers to verify the behavior of our application, offering readers a practical way to validate their implementations.

By following this comprehensive guide, developers can effectively manage multiple MongoDB databases within their Spring Boot applications while harnessing the power of reactive programming for high concurrency and responsiveness. This approach is invaluable in scenarios where diverse data requirements meet the demands of modern, data-intensive applications.

Resources

Harnessing custom Kubernetes probes in Spring Boot applications

Pavel Zeger — Fri, 25 Aug 2023 15:13:49 +0000

Disclaimer

This article represents my perspective on this solution, thus any suggestions, fixes, or discussions will be highly appreciated.

The short story

In my previous article I've explained how to create customized health indicators within a Spring Boot application: Monitoring the health of Spring Boot applications with custom health indicators. Now I'll show how to use these custom indicators for Kubernetes readiness and liveness probes.

In Kubernetes, readiness and liveness probes are mechanisms used to ensure the availability and health of containerized applications running within pods. They play a crucial role in maintaining the stability and reliability of applications in a Kubernetes cluster.

A liveness probe is used to determine if a container is still functioning properly. It periodically checks the health of the application within the container. If the liveness probe fails (i.e., the application within the container becomes unresponsive or crashes), Kubernetes takes action based on the probe's configuration. This action could include restarting the container or even killing the pod and allowing it to be rescheduled on a different node. Liveness probes are essential to prevent applications from hanging in a broken or unresponsive state.

A readiness probe is used to determine if a container is ready to accept incoming traffic. It indicates whether the application within the container is fully up and running and capable of serving requests. Readiness probes are particularly useful during deployment scenarios when new versions of an application are being rolled out. By configuring a readiness probe, Kubernetes can delay sending traffic to a newly started container until it's fully prepared to handle requests. This helps prevent instances where users are directed to containers that might still be initializing or not fully functional.
Both types of probes can be configured in Kubernetes using HTTP endpoints, TCP sockets, or command execution within the container. Probes can be customized with parameters such as probe intervals, timeouts, and failure thresholds. Properly configuring these probes contributes to the overall stability and resilience of applications in a Kubernetes environment.

A startup probe is another type of probe introduced in Kubernetes to enhance the reliability of applications, especially during their startup phase. While liveness and readiness probes focus on the ongoing health and readiness of a container, a startup probe specifically targets the early stages of container initialization.

A startup probe is used to determine if a container has successfully started and become ready to handle requests. Unlike a liveness probe, which monitors the ongoing health of a container, a startup probe is concerned with the initial state of the container. It helps delay marking the container as ready until a specific condition is met during startup.

Startup probes are useful when containers take longer than usual to initialize or when certain critical services need to be up and running before the application is considered fully operational. For example, an application might need to establish a database connection or load configuration files during startup. By configuring a startup probe, Kubernetes can wait for this initialization process to complete before allowing the container to receive traffic from services or users.

In essence, a startup probe complements liveness and readiness probes by focusing on the specific requirements of a container's startup process, ensuring that the container is not considered ready until it has reached a desired level of initialization.

Overall, by using liveness, readiness, and startup probes effectively, you can create a more robust and fault-tolerant application deployment in a Kubernetes environment.

The implementation

When deploying a Spring Boot application within a Kubernetes environment, one can leverage the capabilities of the Actuator module to seamlessly integrate "Liveness" and "Readiness" information. This information is sourced from the ApplicationAvailability interface and subsequently utilized in dedicated health indicators, namely LivenessStateHealthIndicator and ReadinessStateHealthIndicator. These indicators are visible both on the global health endpoint (/actuator/health) and as distinct HTTP Probes accessible through health groups: /actuator/health/liveness and /actuator/health/readiness.

The initial step involves enabling liveness and readiness probes within the Spring Boot application for the Kubernetes environment:



management.health.kubernetes.enabled=true
management.health.livenessstate.enabled=true
management.health.readinessstate.enabled=true
management.endpoint.health.probes.enabled=true

Subsequently, the aim is to furnish comprehensive health information to each Kubernetes request pertaining to our application:



management.endpoint.health.validate-group-membership=true
management.endpoint.health.group.liveness.show-details=always
management.endpoint.health.group.liveness.show-components=always
management.endpoint.health.group.readiness.show-details=always
management.endpoint.health.group.readiness.show-components=always

The more intriguing facet lies in the integration of our custom health indicators into the liveness and readiness probes. Achieving this merely requires the reconfiguration of the Health Groups within the Spring Boot application.

Assume we possess the subsequent custom health indicators:

serviceState (derived from the ServiceStateHealthIndicator class)
cacheState (derived from the CacheStateHealthIndicator class)

Spring Boot conveniently permits customization of these health checks by employing inclusion and exclusion of diverse health check indicators:



management.endpoint.health.group.liveness.exclude=readinessState,serviceState
management.endpoint.health.group.liveness.include=cacheState
management.endpoint.health.group.readiness.include=cacheState,serviceState

The final accord pertains to the utilization of an alternative or supplementary path for liveness or readiness probes, accomplished through the following configuration:



management.endpoint.health.probes.add-additional-paths=true

management.endpoint.health.group.liveness.additional-path=server:liveness

management.endpoint.health.group.readiness.additional-path=server:readiness

Summary

When deploying a Spring Boot application in Kubernetes, you can leverage the following probes to ensure the application's reliability and responsiveness: liveness and readiness, and add additional layers of your application into these probes in a very convenient way. This approach helps maintain the stability and availability of your Spring Boot application, even in dynamic containerized environments.

Resources

Monitoring the health of Spring Boot applications with custom health indicators

Pavel Zeger — Fri, 21 Jul 2023 15:34:47 +0000

Disclaimer

This article represents my perspective on this solution, thus any suggestions, fixes, or discussions will be highly appreciated.

The short story

In today's complex software landscape, understanding the health status of an application has never been more crucial. An application's "health" extends beyond just uptime. It involves the ability of the application to interact effectively with the external sources it relies on, such as databases, APIs, or caches.

We use Spring Boot applications extensively, and these depend significantly on various external data sources such as bespoke APIs, third-party APIs, and internal caches. The primary objective is to ensure the robust health of these applications within a Kubernetes deployment.

Each application has its own unique health state, determined by the status of the external sources it uses. For instance, if your application communicates with a third-party service or a database, its health status is not only reliant on its internal functionality but also on the availability and responsiveness of these external systems.

If an essential external source experiences downtime, it could significantly affect our application's performance and business logic, even if the application itself is running perfectly. Hence, it's critical to consider these external factors when determining the health status of your application.

Health checks offer a solution to this challenge. They allow you to monitor and assess the state of the systems your application relies on. If an external source is important for your application, you need to include it in your health checks.

For example, if your application relies heavily on a particular API, you should implement a health check that ensures the API is up and running. This way, you'll be alerted if there's a problem, and you can address it promptly to prevent any severe consequences for your application.

Spring Boot's Health Indicator

Spring Boot provides a great feature that helps with this – custom health indicators. The HealthIndicator interface provides a method, health(), that returns a Health response. You can define the health rules according to your application's requirements, which can range from checking database connectivity to validating the status of an external service.

With Spring Boot, you can customize these health indicators according to your specific needs, ensuring the accuracy of your application's health status. For example, if your application uses an external API and a cache system, you could create two separate health indicators – one for the API and one for the cache. If either system encounters a problem, the relevant health indicator will report it, enabling you to take prompt action.

The implementation

Let's configure two kinds of health indicators:

for an external API
for an internal cache

Health indicator for an external API

Our custom ApiHealthIndicator interface creates a contract for all external APIs we would like to monitor by providing a doHealthCheck() method. The health status will we be created by default methods of this interface: a healthy state and a non-healthy state.

You can add additional details to the down health state like custom runtime exception, a detailed message etc.

import org.springframework.boot.actuate.health.Health;
import reactor.core.publisher.Mono;

import static org.apache.commons.lang3.BooleanUtils.isFalse;
import static org.apache.commons.lang3.BooleanUtils.isTrue;

public interface ApiHealthIndicator {

    Mono<Health> doHealthCheck();

    default Health createHealth(Boolean status, String serviceName) {
        return isTrue(status)
                ? new Health.Builder().up().build()
                : createHealthDownStatus(status, serviceName);
    }

    default Health createHealthDownStatus(String serviceName) {
        var exception = new ServiceUnavailableException(String.format("The %s isn't available", serviceName));
        return new Health.Builder()
                .down(exception)
                .build();
    }

}

The component ServiceStateHealthIndicator effectively performs the required logic: it integrates a service bean that retrieves the current status from an external API, indicating whether the API is available or unavailable (true/false). When a monitoring resource, such as Kubernetes, sends a health check request to the application's HealthEndpoint through the actuator, the getHealth() method gets triggered. This method furnishes statuses from all HealthContributor instances, which includes our custom ReactiveHealthIndicator.

import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.ReactiveHealthIndicator;
import org.springframework.stereotype.Component;
import reactor.core.publisher.Mono;

@Component
public class ServiceStateHealthIndicator implements ReactiveHealthIndicator, ApiHealthIndicator {

    private static final String SERVICE_NAME = "external-api";

    private final ApiService service;

    public ServiceStateHealthIndicator(ApiService service) {
        this.service = service;
    }

    @Override
    public Mono<Health> health() {
        return doHealthCheck()
                .onErrorResume(exception -> Mono.just(createHealthDownStatus(false, SERVICE_NAME)));
    }

    @Override
    public Mono<Health> doHealthCheck() {
        return service.getStatus()
                .map(status -> createHealth(status, SERVICE_NAME));
    }

}

And of course our custom exception to explain which status it reflects:

public class ServiceUnavailableException extends RuntimeException {

    public ServiceUnavailableException(String message) {
        super(message);
    }

}

Health indicator for an internal cache

Suppose one of our application's health prerequisites is the presence of an internal cache. If the cache is empty, we'd prefer to prevent the application from starting up altogether. To facilitate this, we can tailor the readiness state using Spring Boot's ReadinessState.class to control the availability state.

Let's assume we have an interface, CacheService, that provides an isNotEmpty() method implemented in each class related to our internal cache. This method delivers a straightforward boolean result, enabling us to verify if each of our internal caches is in a valid state.

By injecting a map containing all cache service beans, we can cycle through each of our caches. If even a single cache is empty, the readiness state prevents any traffic from reaching the application.

We can extend the AvailabilityStateHealthIndicator in Spring Boot to create a mapping of statuses and adjust the getState() method. This method yields an instance of ReadinessState based on our caches' state.

The ReadinessStateHealthIndicator is charged with providing this status when an external monitoring tool, such as Kubernetes, dispatches an HTTP request to our application's readiness endpoint.

import org.springframework.boot.actuate.availability.AvailabilityStateHealthIndicator;
import org.springframework.boot.actuate.health.Status;
import org.springframework.boot.availability.ApplicationAvailability;
import org.springframework.boot.availability.AvailabilityState;
import org.springframework.boot.availability.ReadinessState;
import org.springframework.stereotype.Component;

import java.util.Map;

@Component
public class CacheStateHealthIndicator extends AvailabilityStateHealthIndicator {

    private final Map<String, CacheService> cacheServices;

    public CacheStateHealthIndicator(Map<String, CacheService> cacheServices, ApplicationAvailability availability) {
        super(availability, ReadinessState.class, statusMappings -> {
            statusMappings.add(ReadinessState.ACCEPTING_TRAFFIC, Status.UP);
            statusMappings.add(ReadinessState.REFUSING_TRAFFIC, Status.OUT_OF_SERVICE);
        });
        this.cacheServices = cacheServices;
    }

    @Override
    protected AvailabilityState getState(ApplicationAvailability applicationAvailability) {
        boolean isEmpty = cacheServices.values()
                .stream()
                .map(CacheService::isNotEmpty)
                .toList()
                .contains(false);
        if (isEmpty) return ReadinessState.REFUSING_TRAFFIC;
        return ReadinessState.ACCEPTING_TRAFFIC;
    }

}

Summary

In conclusion, Spring Boot offers a comprehensive and customizable system for monitoring the health of your applications. Its HealthIndicator interface allows you to consider all critical factors, including essential external sources, in determining your application's health status. This functionality helps to prevent minor issues from escalating into significant problems, enhancing the reliability of your applications.

In the next article I'll show how can we use these customized health indicators for liveness and readiness Kubernetes probes within application.properties file.

Resources

How to implement GZIP decompression for incoming HTTP requests on the Netty server

Pavel Zeger — Fri, 16 Jun 2023 15:54:30 +0000

Disclaimer

This article represents my perspective on this solution, thus any suggestions, fixes, or discussions will be highly appreciated.

The short story

We use Spring Boot and Spring Webflux to run our platform with Netty HTTP server and client. As you may know, Netty doesn't have an out-of-the-box solution to decompress incoming HTTP requests to an HTTP server, hence I needed to "reinvent the wheel" using Netty capabilities.

The implementation

There are a few ways to implement a decompression on the incoming request in Netty:

via WebFilter
via custom handler added to the Netty event loop
via controller itself

The most convenient and flexible option is to use the WebFilter interface with a high precedence configuration, as this approach allows us to filter requests not only by their metadata but also by the request's data we receive from our clients.

According to the MSDN documentation (Compression in HTTP) in order to start a decompression of the payload, the server have to get an HTTP header Content-Encoding: gzip from a client. Thus, our first goal is to recognize the presence of such a header within a request. Let's create a utility class, CompressionUtils, that will hold a static method to verify the presence of this header and another static method that will convert a GZIP input stream into a byte array to transfer this array further into the Netty pipeline:



import org.springframework.http.HttpHeaders;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.stereotype.Component;

import lombok.AccessLevel;
import java.io.IOException;
import java.io.InputStream;
import java.io.StringWriter;
import java.nio.charset.StandardCharsets;

import static java.util.Objects.nonNull;
import static org.apache.commons.io.IOUtils.copy;
import static org.springframework.http.HttpHeaders.CONTENT_ENCODING;
import static org.springframework.util.CollectionUtils.isEmpty;

@Component
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public class CompressionUtils {

    public static final String GZIP = "gzip";
    public static final String UNKNOWN = "unknown";

    public static byte[] getDeflatedBytes(InputStream inputStream) throws IOException {
        String string = IOUtils.toString(inputStream, UTF_8);
        return string.getBytes();
    }

    public static boolean isGzipRequest(ServerHttpRequest serverHttpRequest) {
        return containsGzip(serverHttpRequest, CONTENT_ENCODING);
    }

    public static boolean isGzipResponseRequired(ServerHttpRequest serverHttpRequest) {
        return containsGzip(serverHttpRequest, ACCEPT_ENCODING);
    }

    private static boolean containsGzip(ServerHttpRequest serverHttpRequest, String headerName) {
        HttpHeaders headers = serverHttpRequest.getHeaders();
        if (!isEmpty(headers)) {
            String header = headers.getFirst(headerName);
            return nonNull(header) && header.contains(GZIP);
        }

        return false;
    }

}

To prevent blocking conversion of the incoming input stream into a byte array, I've used the Apache Commons IO project and the static toString method uses the StringBuilder class.

The second stage of our implementation is to wrap an incoming ServerHttpRequest instance into our implementation. In this way, we can retain the source data of the incoming request and override the getBody method. The Netty pipeline invokes this method to convert incoming data into a Netty's DataBuffer wrapper. Therefore, it's a convenient way to swap this part of the flow instead of integrating it into the Netty flow with a custom handler. Customizing it via WebFilter is a cleaner approach that does not affect all incoming requests and allows us to control the decompression process (with error handling and metrics).

So, our wrapper class GzipServerHttpRequest has the main method getBody with the following logic:

Gets a body of the source server HTTP request.
Transform it into an InputStream.
Transform a reactive stream of input streams into a SequenceInputStream to get the whole POST request body.
Decompress the GZIP body with GZIPInputStream.
Wrap the final byte array into a Netty DataBuffer again.



import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.core.io.buffer.DefaultDataBufferFactory;
import org.springframework.http.HttpCookie;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.server.RequestPath;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.http.server.reactive.SslInfo;
import org.springframework.util.MultiValueMap;
import reactor.core.publisher.Flux;
import reactor.core.publisher.SynchronousSink;

import lombok.RequiredArgsConstructor;
import java.io.InputStream;
import java.io.SequenceInputStream;
import java.net.InetSocketAddress;
import java.net.URI;
import java.util.zip.GZIPInputStream;

@SuppressWarnings("NullableProblems")
@RequiredArgsConstructor
public class GzipServerHttpRequest implements ServerHttpRequest {

    private final ServerHttpRequest serverHttpRequest;

    @Override
    public String getId() {
        return serverHttpRequest.getId();
    }

    @Override
    public RequestPath getPath() {
        return serverHttpRequest.getPath();
    }

    @Override
    public MultiValueMap<String, String> getQueryParams() {
        return serverHttpRequest.getQueryParams();
    }

    @Override
    public MultiValueMap<String, HttpCookie> getCookies() {
        return serverHttpRequest.getCookies();
    }

    @Override
    public String getMethodValue() {
        return serverHttpRequest.getMethodValue();
    }

    @Override
    public URI getURI() {
        return serverHttpRequest.getURI();
    }

    @Override
    public HttpHeaders getHeaders() {
        return serverHttpRequest.getHeaders();
    }

    @Override
    public InetSocketAddress getRemoteAddress() {
        return serverHttpRequest.getRemoteAddress();
    }

    @Override
    public SslInfo getSslInfo() {
        return serverHttpRequest.getSslInfo();
    }

    @Override
    public HttpMethod getMethod() {
        return serverHttpRequest.getMethod();
    }

    @Override
    public Flux<DataBuffer> getBody() {
        return serverHttpRequest.getBody()
                .map(dataBuffer -> dataBuffer.asInputStream(true))
                .reduce(SequenceInputStream::new)
                .handle(this::decompress)
                .flux();
    }

    private void decompress(InputStream inputStream, SynchronousSink<DataBuffer> sink) {
        try (var gzipInputStream = new GZIPInputStream(inputStream)) {
            byte[] deflatedBytes = getDeflatedBytes(gzipInputStream);
            sink.next(new DefaultDataBufferFactory().wrap(deflatedBytes));
        } catch (Exception exception) {
            sink.error(getException());
        }
    }

    private IllegalGzipRequestException getException() {
        String exceptionMessage = String.format("Decompression of a gzip content failed, URI: [%s]", serverHttpRequest.getURI());
        return new IllegalGzipRequestException(exceptionMessage);
    }

}

Once we have the main implementation, we can now add the GzipDecompressionFilter to mutate our ServerWebExchange with the new instance of a wrapped ServerHttpRequest:



import lombok.extern.slf4j.Slf4j;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.http.HttpHeaders;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import org.springframework.web.server.WebFilter;
import org.springframework.web.server.WebFilterChain;
import reactor.core.publisher.Mono;

import static java.util.Objects.isNull;
import static org.apache.commons.lang3.exception.ExceptionUtils.getMessage;
import static org.springframework.http.HttpHeaders.CONTENT_ENCODING;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.util.CollectionUtils.isEmpty;

@Slf4j
@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class GzipDecompressionFilter implements WebFilter {

    @SuppressWarnings("NullableProblems")
    @Override
    public Mono<Void> filter(ServerWebExchange serverWebExchange, WebFilterChain webFilterChain) {
        if (!isGzipRequest(serverWebExchange.getRequest()))
            return webFilterChain.filter(serverWebExchange);

        ServerWebExchange mutatedWebExchange = getMutatedWebExchange(serverWebExchange);
        return webFilterChain
                .filter(mutatedWebExchange)
                .onErrorResume(this::logError);
    }

    private ServerWebExchange getMutatedWebExchange(ServerWebExchange serverWebExchange) {
        ServerHttpRequest mutatedHttpRequest = new GzipServerHttpRequest(serverWebExchange.getRequest());
        return serverWebExchange
                .mutate()
                .request(mutatedHttpRequest)
                .build();
    }

    private Mono<Void> logError(Throwable exception) {
        log.error("Gzip decompressed HTTP request failed, exception: [{}]", getMessage(exception));
        return Mono.empty();
    }

}

The last class we'll add is our custom RuntimeException, IllegalGzipRequestException, to recognize errors of this type in the third-party system we use for intercepting exceptions in our services:



public class IllegalGzipRequestException extends RuntimeException {

    public IllegalGzipRequestException(String message) {
        super(message);
    }

}

In case our client wants to get a compressed response, we can easily add this implementation via the application.properties file:



server.compression.enabled=true

server.compression.min-response-size=1KB

DEV Community: Pavel Zeger

Optimizing Spring Boot dynamic configuration using Kubernetes ConfigMap (part 2)

Defining Properties

Configuring the WebClient

Working with ConfigMap Data

Main Components

ConfigMap actuator endpoint

ConfigMap Loader

ConfigMap Service

ConfigMap initializer

Conclusion

References

Optimizing Spring Boot dynamic configuration using Kubernetes ConfigMap (part 1)

Disclaimer

The short story

What is Kubernetes ConfigMap?

Key characteristics of ConfigMap

Creation and usage examples of ConfigMaps

Use cases and considerations

Implementation of ConfigMaps

ConfigMap

Implementation

Implementation validation

Possible issues

References

Enhancing team collaboration in IntelliJ IDEA with the .run directory

What is the .run directory?

How to use the .run directory

Conclusion

Resources

Organizing custom metrics for effective application monitoring

Disclaimer

The short story

The intro

The importance of well-organized metrics

The solution: dedicated package and enums

The implementation

Conclusion

Reactive MongoDB with multi-database within a single Spring Boot application

Disclaimer

The short story

The implementation

Summary

Resources

Harnessing custom Kubernetes probes in Spring Boot applications

Disclaimer

The short story

The implementation

Summary

Resources

Monitoring the health of Spring Boot applications with custom health indicators

Disclaimer

The short story

Spring Boot's Health Indicator

The implementation

Health indicator for an external API

Health indicator for an internal cache

Summary

Resources

How to implement GZIP decompression for incoming HTTP requests on the Netty server

Disclaimer

The short story

The implementation

Resources

What is the `.run` directory?

How to use the `.run` directory