DEV Community: Martel Richard

Correlation in logging

Martel Richard — Thu, 22 May 2025 13:34:07 +0000

What is Correlation in logging?

It refers to the practice of linking together log entries that are part of the same transaction, user session, request, or workflow, even if those entries are generated by different components or services in a distributed system.

Why does it matter?

In modern applications—especially microservices or cloud-based systems—a single user action might trigger a cascade of events across multiple services. Without correlation, it's hard to trace the full path of that action through the system.

How Correlation Works

A correlation ID (or trace ID) is a unique identifier that is:

Generated at the start of a request or transaction.
Passed along to all components or services involved.
Logged with every log entry related to that request.

This way, you can filter or search logs by the correlation ID to see the full picture.

2025-05-19 10:15:21.370 INFO [correlation-id=3b9c39ab-ad6e-47c4-b6ee-dc0ee34ff807] [thread-id=boundedElastic-1] com.app.service.LoginService:10 - User logged in

Log4j support for correlation

Log4j (Logging for Java) is a standard logging library in Java. It has MDC that allows you to enrich your log messages with contextual information that can be very helpful for debugging and tracing execution, especially in multi-threaded or distributed applications.

What is MDC?

Mapped Diagnostic Context is essentially a thread-local map where you can store key-value pairs. These values can then be included in your log output using a pattern layout. Since it's thread-local, each thread has its own copy of the MDC data, which avoids interference between thread.

For example, imagine you're handling multiple user requests in a web application. Each request is processed in a separate thread. You might want to log the user ID, session ID, or transaction ID with every log message related to that request. MDC makes this easy.

How to use MDC?

com.myapp.controller.MyController.java

try {
  MDC.put("correlation-id", UUID.randomUUID().toString());
  log.info("Testing contextual logging");
}
finally {
  MDC.remove("correlation-id");
}

log4j2.xml

<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %5p [correlation-id=%X{correlation-id}] [thread-id=%t] %logger{36} - %m%n%xwEx"/>

NOTE: When you use %X followed by a key in curly braces, like %X{key}, it will output the value associated with that key from the MDC. If you use %X without a key, it will output all entries in the MDC, in the format key1=value1, key2=value2, ...

Log Output

2025-05-22 12:56:14.216  INFO [correlation-id=1b4a4f98-54b1-4951-b3e5-7c7ca742564b] [thread-id=http-nio-9052-exec-1] com.myapp.controller.MyController - Testing contextual logging

Caveats with MDC

MDC uses thread-local storage, meaning the context is tied to the current thread. So it has the caveats same as the ThreadLocal.

Thread pools If a thread is reused and MDC is not cleared, old context data may leak into unrelated requests and lead to inconsistent logging. So always clear MDC after use.

Async processing MDC data may not propagate to new threads unless explicitly passed.

Context Propogation

In asynchronous or reactive programming like Project Reactor, context (like ThreadLocal values used for logging or tracing) can be lost when execution hops between threads. This breaks features like MDC logging or distributed tracing. In such cases, the context has to be propogated across threads. Take a look at this spring blog to understand more on this!

In a nutshell, context propogation bridges ThreadLocal and Reactor Context and allows MDC to work to work seamlessly in reactive flows. This also helps tools like Micrometer and other tracing libraries and enables distributed tracing in the reactive world.

Check out this Medium article. The author explains context propagation clearly with practical examples.

References

Collation in Database - What you need to know

Martel Richard — Thu, 27 Mar 2025 11:29:14 +0000

What is Collation?

Collation refers to a set of rules that determine how data is sorted and compared. Collation settings specify the correct character sequence, case sensitivity, accent marks, character width and the characters used in comparison operations, such as equality and inequality.

How does this affect the data?

1. Character data sort order

This specifies how string data is sequenced. For example, whether 'a' comes before 'b'.

2. Case sensitivity

This determines whether 'A' and 'a' are considered the same.

3. Accent sensitivity

This determines whether 'a' and 'á' are considered the same.

4. Kana Sensitivity

This determines whether two characters that look the same but are represented differently in Kana (a Japanese script) are treated as equal.

5. Width sensitivity

This determines whether a single-byte character (half-width) and the same character represented as a double-byte character (full-width) are treated as equal.

Where do you specify the collation?

Collation can be specified at different levels depending on the specific database system. For example, in SQL Server, it can be specified at:

Server level
Database level
Column level
Expression (SQL Query) level

The above levels are mentioned in the descending order of hierarchy i.e. when you create a database, you specify the default collation for the database. Every column in the database that uses character data types has a collation. If not specified, the column uses the default collation of the database.

How collation affects the query results?

For example, the default collation in SQL Server is

SQL_Latin1_General_CP1_CI_AS

The elements of this collation name represent the following.

SQL - This prefix means that the collation is a SQL Server collation.

Latin1_General - This is the base collation, which defines the rules that are used for sorting and comparing characters within the Latin1 character set.

CP1 - This stands for Code Page 1252, which is a character encoding of the Latin alphabet, used by default in the legacy components of Microsoft Windows in English and some other Western languages.

CI - This stands for Case Insensitive. This means that the SQL Server does not consider case during comparison. For example, 'A' and 'a' are considered to be the same.

AS - This stands for Accent Sensitive. This means that accented characters and unaccented characters are considered to be different. For example, 'a' and 'á' are considered to be different.

Overall, SQL_Latin1_General_CP1_CI_AS is a SQL Server collation that is case insensitive and accent sensitive, using the Latin1 General rules and the 1252 Code Page. This is a commonly used collation in the English language for SQL Server.

How to check your collation settings in SQL Server

You can determine the server's character set in SQL Server by using the following SQL query.

SELECT SERVERPROPERTY('Collation')

If you need more information about the collation, you can use fn_helpcollations function which returns all the collations supported by your server.

SELECT * FROM fn_helpcollations()

When characters cannot be represented in a given collation

In the context of the SQL_Latin1_General_CP1_CI_AS collation in SQL Server, which is based on the Windows-1252 code page, certain characters from other scripts or languages that are not covered by Windows-1252 may not be saved correctly.

For example, characters from scripts such as Arabic, Hebrew, Cyrillic, Chinese, Japanese, Korean, and many others, cannot be properly represented in this collation. So, if you try to save a character like the Arabic letter "ج" or the Chinese character "汉" using this collation, such characters will be replaced with "?".

Refer Collation in SQL Server to know more on various collation rules and levels.