DEV Community: Ankit Sood

Why I Stopped Using Lombok’s `@Data` on JPA Entities

Ankit Sood — Tue, 14 Apr 2026 18:20:21 +0000

A few days ago, I ran into a bug that didn’t look like a bug at all.

I was working on a fairly standard Spring Boot service, nothing unusual. A couple of entities, some relationships, and a small feature change. To keep things concise, I did what most of us do:

@Data
@Entity
public class User {
    ...
}

It felt natural, clean and efficient.

The application started fine. APIs were working. Everything looked normal.

Until I added a simple log statement.

The Bug That Didn’t Make Sense

log.info("User details: {}", user);

The moment this line executed, the application crashed with a StackOverflowError.

At first glance, it didn’t add up.

There was no recursion in my code. No complex logic. Just a log statement.

But as I traced through the stack, the pattern became clear.

When `toString()` Becomes a Trap

The User entity had a bidirectional relationship:

A User had a collection of Orders
Each Order referenced back to the User

This is a very common JPA pattern.

What I hadn’t paid attention to was what Lombok’s @Data had generated for me.

It includes a toString() method that prints all fields, including relationships.

So when I logged the User:

User.toString() tried to print orders
Each Order tried to print its user
Which again tried to print orders

And so on.

An infinite loop—triggered by logging.

The Subtle Performance Issue

While investigating, I noticed something else that was even more concerning.

In another part of the code, I had something like:

Set<User> users = new HashSet<>();
users.add(user);

Nothing unusual.

But it was unexpectedly slow.

The reason, again, traced back to @Data.

It generates equals() and hashCode() using all fields. In a JPA entity, that can be problematic especially when those fields include lazily loaded relationships.

To compute the hash, Hibernate ended up initializing those lazy fields, which meant:

Additional database queries
More data loaded than expected
All happening implicitly

No explicit query. No clear signal in the code.

The Core Issue

The root of the problem is simple:

JPA entities are not plain Java objects.

They are managed by a persistence context, often proxied, and frequently only partially loaded.

Lombok, however, generates code as if everything is a simple POJO. That disconnect is where these issues originate.

What I Changed

I didn’t stop using Lombok. But I stopped using @Data on entities.

Instead, I now prefer:

@Getter
@Setter
@Entity
public class User {
    ...
}

For methods like toString(), equals(), and hashCode(), I either:

Write them explicitly, or
Use Lombok annotations selectively (e.g., excluding relationships)

This keeps behavior predictable and avoids unintended side effects.

Takeaway

@Data is incredibly useful in the right places.

But JPA entities are not one of them.

What looks like a small convenience can introduce:

Infinite recursion
Hidden database queries
Hard-to-debug behavior

All without any obvious warning.

Final Thought

This wasn’t a complex failure. It didn’t require scale or load to surface.

It came from a perfectly normal development flow and that’s exactly why it’s worth paying attention to.

Sometimes the most dangerous problems aren’t the ones we struggle to write. They’re the ones we don’t realize we’ve already written for us.

📚 Further Reading

Vlad Mihalcea — How to implement equals and hashCode using the JPA entity identifier

https://vladmihalcea.com/hibernate-facts-equals-and-hashcode/

A must-read on why JPA entity equality should be based on identifiers rather than all fields, and how improper implementations can lead to subtle bugs.
Project Lombok — @data Annotation Documentation

https://projectlombok.org/features/Data

Explains what @Data actually generates under the hood, which helps understand why its default behavior can conflict with JPA entities.

Performance Testing Databases the Right Way: A Real-World Engineering Journey

Ankit Sood — Thu, 20 Nov 2025 11:49:04 +0000

A few months ago, I found myself facing one of those architectural decisions you can’t take lightly: Which database should power our application?

On paper, several options looked great. Azure SQL had strong consistency and a familiar relational model. Google Spanner promised global scale and near-infinite horizontal growth. But the same problem kept bothering me:

None of the available benchmarks reflected my application’s workload.

Every benchmark I found was synthetic. Every article was based on someone else’s traffic patterns. Every vendor claim assumed a workload that looked nothing like ours.

And that’s when it clicked.

If I wanted real answers, I needed to simulate our actual read/write behavior, our transaction mix, our concurrency pattern, our connection pooling.

I needed a test setup that behaved like our Java application—not a generic stress tester.This article is the story of how I modeled the workloads before I even touched Azure SQL or Google Spanner.

In Part 2, I’ll share what happened when I finally put those databases under load.

Why I Couldn’t Trust Generic Database Benchmarks

At first, I thought I’d simply compare p95 latencies or look up TPS numbers for each database. But it didn’t take long to realize the flaws:

These numbers came from engineered environments.
They assumed ideal network conditions.
And they definitely didn’t match our schema or access patterns.

Most importantly?

Our system was not a synthetic benchmark.

It had:

specific read/write ratios
certain frequently accessed tables
particular update patterns
a real connection pool
and lock-sensitive flows

So I stopped searching for benchmarks and started designing tests that mirrored our workload.

Step 1: Defining What “Good Performance” Actually Meant

Before modeling anything in JMeter, I forced myself to answer a simple question:

What does good database performance look like for this application?

This gave me the baseline metrics and below is the "Performance criteria" I defined:

p95 and p99 response time targets
Expected throughput (operations per second)
Resource utilization boundaries (CPU, memory, IOPS)
Acceptable error rate
Scalability expectations under increased concurrency

These weren’t arbitrary numbers—they were tied to real SLAs and user expectations. With this foundation, I could work backwards and ensure the workloads I modeled aligned with the actual business needs.

Step 2: Understanding the Application’s Actual Database Behaviour

Next, I analyzed our application’s schema and data-access patterns.

I listed every operation the application performed frequently:

Reads (SELECT)
Writes (INSERT)
Updates (UPDATE)
Deletes (DELETE)

But I didn’t stop there. I also identified:

which tables were read-heavy
which operations were latency-sensitive
which queries created row-level locks
which flows needed strict consistency

This step was crucial because performance testing is not just about “running queries fast.” It’s about understanding how those queries behave under real concurrency.

Step 3: Classifying Operations by Frequency and Importance

One of the biggest mistakes in performance testing is treating all operations equally. Real applications never have a perfectly even CRUD distribution.

Some flows run thousands of times per minute and others just a few.
So, I categorized operations based on:

frequency
business criticality
concurrency sensitivity

This gave me clarity on how to weight each operation in the workload.

Step 4: Designing Realistic Workload Scenarios

Now came the core of the modeling process.

I created workload mixes that reflected how our application behaves in production.

Scenario 1: Read-Heavy Workload
Most user-facing apps are read-dominant. For ours, this looked like:

70% reads  
20% writes  
10% updates

Scenario 2: Balanced Workload

For internal workflows and batch processes:

50% reads  
30% writes  
10% updates  
10% deletes

Each scenario represented a realistic slice of our system’s behaviour not a synthetic stress test.

Step 5: Designing the JMeter Setup to Behave Like a Java Application

This part was non-negotiable for me. I didn’t just want JMeter to hit the database, I wanted it to behave like our Java service.

So I built the test plan with care:

Separate thread groups for each CRUD operation

This allowed me to configure:

individual concurrency control
precise latency measurement
different ramp-up patterns

Parameterized SQL queries

Using CSV Data Set Config, every iteration pulled dynamic values.
and this avoided caching effects and mimicked real traffic variation.

Shared JDBC connection pool (1:1 ratio)

All thread groups used the same JDBC pool, just like production. This created:

connection contention
realistic queueing
lock waits

Overall, true concurrency behaviour.

At this point, JMeter wasn’t just “sending traffic.” It was simulating the exact way our application interacted with the database.

The Workload Modeling Was Complete — Now the Real Testing Could Begin

After several iterations, reviews, and dry runs, I finally had a performance test suite that felt authentic. It matched:

our query patterns
our connection pool settings
our concurrency behaviour
our read/write ratios

And most importantly, it reflected the pressure our application would put on any database—whether Azure SQL, Google Spanner, or something else entirely.

Now I was ready for the real showdown.

Coming Up Next: Azure SQL vs Google Spanner — The Actual Results

In Part 2, I’ll share how both databases performed under the exact workloads modeled in this article. And trust me the results were nothing like the vendor benchmarks.

I’ll cover:

p95/p99 latencies
read/write throughput
locking and contention behaviour
resource consumption

and which database ultimately won for our use case

Stay tuned for Part 2.

Building Custom HTTP Request Metrics in Spring Boot: A Deep Dive into Observability

Ankit Sood — Thu, 18 Sep 2025 12:15:48 +0000

Modern APIs often serve multiple clients i.e mobile apps, partner services, internal tools and Knowing who is calling you and how often is critical for:

Capacity planning: scale before peak traffic hits.
Fair usage & billing: detect high-volume consumers.
Alerting & SLOs: raise alarms when specific clients misbehave.

Let’s build a zero-dependency filter that tracks request counts by HTTP method and a custom consumer header and exports those metrics through Micrometer.

Architecture at a Glance

┌──────────┐      ┌────────────────────┐      ┌────────────┐
│  Client  │ ───▶ │Custom MetricsFilter│ ───▶ │ Controller │
└──────────┘      └────────────────────┘      └────────────┘
                      │
                      ▼
               Micrometer Registry
                      │
                      ▼
         Prometheus / Datadog / CloudWatch …

Our solution uses a Spring Boot servlet filter that intercepts every HTTP request and records metrics using Micrometer.

Flow looks like:

Request in → Filter → Controller
Filter → Micrometer Counter with tags for method and consumer.
Metrics scraped or pushed to your observability backend.

Now, let's go through the building process step by step:

Step 1: Add Micrometer

Spring Boot 3+ includes Micrometer by default when you use the spring-boot-starter-actuator.
For Prometheus:

<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Enable endpoints:

management.endpoints.web.exposure.include=health,metrics,prometheus

Step 2: The Filter Code

@Component
@Order(1)
public class CustomHttpMetricsFilter extends OncePerRequestFilter {

    private static final ConcurrentHashMap<String, Counter> COUNTERS = new ConcurrentHashMap<>();
    private static final String METRIC = "http_requests";

    @Autowired
    private MeterRegistry registry;

    @Override
    protected void doFilterInternal(HttpServletRequest req,
                                    HttpServletResponse res,
                                    FilterChain chain)
            throws ServletException, IOException {

        incrementCounter(req);
        chain.doFilter(req, res);
    }

    private void incrementCounter(HttpServletRequest req) {
        String consumer = req.getHeader("CONSUMER.ID");
        if (consumer == null) consumer = "null";

        String key = req.getMethod() + ":" + consumer;

        COUNTERS.compute(key, (k, existing) -> {
            Counter c = existing;
            if (c == null) {
                c = Counter.builder(METRIC)
                           .tag("method", req.getMethod())
                           .tag("consumer", consumer)
                           .description("Per-consumer HTTP request count")
                           .register(registry);
            }
            c.increment();
            return c;
        });
    }
}

Key points

OncePerRequestFilter guarantees one execution per HTTP request.
ConcurrentHashMap and compute makes thread-safe cache of Counter objects.
Tags method and consumer make each counter unique and queryable.

Step 3: Observe the Metrics

Run the app and hit it with a few GET/POST requests:

curl -H "CONSUMER.ID: mobile-app" http://localhost:8080/api/...

and now visit:

http://localhost:8080/actuator/prometheus

We should see the following in the response:

http_requests_total{method="GET", consumer="mobile-app"} 42.0
http_requests_total{method="POST", consumer="partner-service"} 5.0

Why Not Use @Timed?

Micrometer provides @Timed for controller methods, but that’s per-endpoint and doesn’t automatically group by a custom header.
Our filter gives you:

Cross-cutting coverage: every request, even for static resources or error pages.
Dynamic tags: any header or attribute can become a label.

Some Production Tips & Enhancements

Label Cardinality
Too many unique consumer IDs can lead to huge time series.
Solution: restrict IDs to a known set or hash/anonymize them.
Metrics Cleanup is required as counter objects never shrink.
Solution: implement TTL eviction or register only for known consumers.
Latency & Status Codes
To measure request duration and tags for HTTP status add a timer .
Security
Expose /actuator/prometheus only to your metrics scraper.
Testing
Use MockMvc and assert

Final Thoughts

With fewer than 50 lines of code, you get real-time, per-consumer traffic visibility. Whether you’re debugging a sudden spike or preparing for a product launch, these metrics become invaluable.

“You can’t improve what you don’t measure.”
Start measuring today—your future self will thank you.

🚀 Slow queries on Cosmos DB ?Here’s How I Fixed Pain with Deep Pagination & Cross-Partition Queries

Ankit Sood — Sat, 09 Aug 2025 13:22:50 +0000

If you have ever paginated through thousands of records in Azure Cosmos DB (MongoDB API) and felt your queries are slow, painfully slow, you’re not alone.

This happened to me recently.

I had a collection with a partitioned on invoiceId, but my queries were filtering on purchase Order Number and combination of vendorNumber & createDate.

The result? The first few pages were fine. But by page 10+, response time exploded.

Before jumping on to what was going on and how I fixed it, quick starter on what Cosmos DB is and how it works.

Cosmos DB Introduction

Azure Cosmos DB is Microsoft’s globally distributed, multi-model database service.

The MongoDB API mode means Cosmos DB will present itself as if you were connecting to a MongoDB server. Hence, your MongoDB drivers, tools, and queries mostly work as-is, but data is stored in Cosmos DB’s underlying storage engine.

Think of it as:

A MongoDB-compatible skin over Cosmos DB’s distributed, serverless, elastic infrastructure.

Now, let's discuss how Cosmos DB stores the data.

When you create a collection/container in Cosmos DB, you must pick a partition key (e.g., /invoiceId).
Cosmos DB then:

Hashes the value of the partition key for each document
Stores the document in a physical and logical partition based on that hash
Each physical partition has its own storage and throughput budget

So:

All docs with the same invoiceId go to the same partition
Different invoiceId values may go to different partitions

Why It Matters ?

Query by Partition Key = Laser Target 🎯

If you query using the partition key, Cosmos DB knows exactly which single physical partition has your data.

Example:
Partition key = /invoiceId

db.invoices.find({ invoiceId: "u123" })

Here’s what happens internally:

Cosmos DB hashes "u123" → finds Partition #7.
It queries only Partition #7.
You get the result quickly and cheaply.

Advantages:

Low RU cost (minimal data scanned)
Fast (single partition)
No fan-out (doesn’t hit every partition)

Cross-Partition Query = Scatter & Gather 🍂

If you don’t provide the partition key in your filter — or you filter by a field that isn’t the partition key — Cosmos DB must check all partitions.

Example:
Partition key = /invoiceId

db.invoices.find({ status: "Paid" })

What happens internally:

Cosmos DB doesn’t know which partition contains matching docs.
Sends the query to all partitions in parallel (scatter).
Collects results from each (gather).
Merges them, applies sorting, etc.
Returns the final set.

Drawbacks:

Higher RU cost (every partition processes the query)
Slower (parallel calls + merge step)
Possible page-by-page retrieval if result is large

Real-World Scenario

Suppose you have documents like this, with a partition key on invoiceId:

{
    "invoiceId": 890,
    "source": "POS",
    "vendorNumber": 10,
    "invoiceNumber": "INV-123",
    "destStoreNbr": 11,
    "country": "IND",
    "totalAmt": 21.76,
    "createDate": "1970-01-01T00:14:05.691Z",
    "updateDate": "1970-01-01T00:14:05.691Z",
}

❌ The Problem

Our clients wanted to search invoices by:
1. Invoice Number
2. Vendor Number + Date
The primary collection was partitioned for write scalability, not for search queries.
Even worse, a vendor number + date could have over 10,000 invoice IDs in a single day in some cases.

Why were these search patterns a problem ?

All these searches were resulting in the usecases mentioned below:

Cross-partition queries forcing database to scan across multiple partitions (slower by design)
Deep pagination using skip and limit forcing database to skip thousands of docs before returning required page (skip + limit problem). Potentially resulting in RU throttling if too much data is requested.

It’s like flipping to page 10,000 in a book by reading every page before it. Inefficient, right?

Hence, we needed a way to:

Make these queries fast and cheap.
Avoid schema duplication or extra writes in the request path.

✅ Solution

Instead of querying the primary collection directly, we created a Lookup Collection with ahead of time processing. You can think of it as a small, precomputed “routing table” for searches.

What’s in the Lookup Collection?

Type = inv → maps invoiceNumber → invoiceId (with partition key)
Type = vendorNumber → maps vendorNumber + date → multiple invoiceIds
A single lookup doc contains at most 1000 invoice IDs.
- If more exist, we create multiple docs for the same vendor/date with a count field to track pagination.
TTL = time to live for each lookup document.

⚙️ The Architecture

The solution has 4 different components:

Primary Collection:
- It holds all invoice documents.
- It is partitioned for writes, not for queries.
- Acts as the source of truth.
Change Data Capture (CDC):
- It listens to inserts only in the primary collection.
- Publishes those inserts to Kafka.
Lookup Consumers
- Two independent java deployments running with their own Kafka consumer groups one for each type of search pattern.
  - Invoice Lookup Consumer → creates type="inv" docs.
  - Vendor Lookup Consumer → creates type="vendorNumber" docs, with pagination logic.
- Both writing to same lookup collection.
API Search Flow
- Search by invoice number:
  - Query lookup collection for type="inv" and prtnKey=invoiceNumber.
  - If found → query primary collection with the exact partition key.
- Search by vendor number + date:
  - Query lookup collection for type="vendorNumber" with pagination.
  - Gather invoice IDs → query primary collection with partition keys.
- This ensures every primary collection query has the partition key → no cross-partition scans.

Sample Invoice mapping document:

{
    "_id" : "6370cfb35934c7afdcffbb6f",
    "prtnKey" : "INV-123",
    "type" : "inv",
    "invIds" : [
        890
    ],
    "ttl" : 252288000,
}

Sample Vendor mapping document:

{
    "_id" : "6370cfb35934c7afdcffbb6a",
    "prtnKey" : "10",
    "createdDate" : 1754739742,
    "type" : "vendorNumber",
    "invIds" : [
        890,
        891,
        892,
        893,
    ],
    "count" : 4
    "ttl" : 252288000,
}

In both the documents shared above, prtnKey is the partition key for the lookup collection and type signified which value is stored as part of the prtnKey.

📃 Why this works ?

Criteria	Results
Speed	Lookup docs are tiny and indexed for lightning-fast reads.
Scalability	CDC keeps lookup data fresh without affecting write throughput.
Flexibility	Adding a new search type? Just add a new lookup doc type.
Pagination	The 1000-ID limit per doc avoids monster documents and RU spikes.

Final Thoughts

Deep pagination and cross-partition queries can cripple performance in Cosmos DB. The good news? With smarter strategies and thoughtful data modeling, you can keep queries fast—even at scale.

Have you faced similar issues? Share your experience or tips in the comments!

Demystifying Java Records: A Developer's Guide

Ankit Sood — Sun, 03 Aug 2025 06:53:29 +0000

Oracle has released the Java version 24 this year, but still I see the usage of the Java Records (release in Java 16) is limited. This article will be focussed on Java Records what it does and where we can use it. Lets dive into it.

Introduction

Based on JEP:359, Records are kind of type declarations and is a restricted form of class. It gives up the freedom that classes usually enjoy and in return, gain a significant degree of concision. Below code sample tells you, how you can create a record.

public record Person(String name, int age) {}

A record acquires many standard members automatically, like:

A private final field for each component of the state description.
A public read accessor method for each component of the state description, with the same name and type as the component.
A public constructor, whose signature is the same as the state description, which initialises each field from the corresponding argument.
Implementations of equals and hashCode that say two records are equal if they are of the same type and contain the same state.
An implementation of toString that includes the string representation of all the record components, with their names.

Characteristics

Let's go through some of these characteristics one by one.

🔒 Fields of Records are private and final by Default
In Java Records, all fields are implicitly declared as private and final. This means that once the values are set via the canonical constructor, they cannot be changed. Also, since the fields are private, they cannot be accessed directly from outside the record. Instead, you access them via the automatically generated accessor methods.

This immutability is one of the key traits of records, making them a perfect choice for modeling data carriers or value objects.

public record Person(String name, int age) {
    // No need to explicitly declare fields, getters, or constructor
}

💡 Internally, this is roughly equivalent to:

public final class Person {
    private final String name;
    private final int age;

    public Person(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public String name() {
        return name;
    }

    public int age() {
        return age;
    }
}

As you can see, the name and age fields are private final, and you get read-only access through the generated methods name() and age().

🔁 equals(), hashCode(), and toString() are Generated by Default
One of the most convenient features of Java Records is that they automatically generate implementations for the equals(), hashCode(), and toString() methods based on all the record fields.

This makes records ideal for use cases like data transfer objects or value types, where equality is based on content rather than identity — and saves you from writing a lot of boilerplate code manually.

🧠 Behind the Scenes, compiler automatically provides:

equals() — Compares all fields for equality
hashCode() — Computes hash based on all fields
toString() — Generates a string like Person[name=John, age=30]

No need to override any of these unless you want to customize the behaviour, records handle it for you out of the box.

🚫 Records Can't Participate in Data Hierarchies
Java Records cannot extend other classes, including other records. This is because records implicitly extend java.lang.Record, and Java does not support multiple inheritance for classes.

This restriction ensures that records remain simple, immutable data carriers without the complexities of inheritance hierarchies.

✅ You can implement interfaces:

public interface Identifiable {
    String id();
}

public record User(String id, String name) implements Identifiable {
}

❌ You cannot extend another class or record:

public class Base {
    // ...
}

// ❌ This will not compile
public record Derived(String id) extends Base {
}

💡 Why this restriction?
Allowing records to participate in inheritance hierarchies would contradict their design goals of simplicity, transparency, and immutability. They're intended to be final, shallow, and self-contained data holders, not flexible object-oriented components.

📦 Records Can Be Declared Inline
Since Java 16, records can also be declared inline (i.e., as local or nested classes) within methods, constructors, or other blocks of code. This is useful when you need a simple data carrier only within a limited scope — for example, as a helper during data transformation or aggregation.

This feature promotes cleaner code without polluting the top-level class or package namespace.

Example:
✅ Declaring a record inside a method

public class OrderService {
    public void processOrder() {
        record OrderItem(String name, int quantity) {}

        OrderItem item = new OrderItem("Book", 3);
        System.out.println("Processing item: " + item);
    }
}

⛔ Scope Limitation
Inline/local records are only visible within the block/method they’re defined in. This is very useful for temporary data structures that are not reused elsewhere.

💡 When to Use

Grouping temporary fields in data processing
Returning lightweight objects from helper methods
Avoiding boilerplate for short-lived data carriers

This feature helps to use the full power of records even for short-lived and context-specific use cases.

✨ Records Can Use Compact Constructors
Java Records support a special form of constructor called a compact constructor, which allows you to add validation or custom logic without repeating the assignment of fields.

In compact constructors, you don’t need to explicitly assign parameters to fields — the compiler does it for you automatically. You just focus on preconditions or side effects.

✅ Example: Validating Fields in a Compact Constructor

public record Product(String name, double price) {
    public Product {
        if (price < 0) {
            throw new IllegalArgumentException("Price cannot be negative");
        }
        name = name.strip(); // You can also transform fields
    }
}

🔍 What’s Happening Here?
The compiler automatically generates:

this.name = name;
this.price = price;

after the custom code block runs.

This makes the code cleaner compared to explicitly writing a full canonical constructor.

❗ Restrictions

You can’t change the fields directly (they’re final)
You can modify or validate parameters before the assignment happens
You must use the parameter names exactly as declared in the record header

This makes the java records much more than just “dumb data holders” and allows to enforce invariants and transformations cleanly and concisely.

🟰 Records Use Data Equality, Not Reference Equality
Unlike regular Java classes (where equals() by default compares references), records are designed to represent data, so they use data (value-based) equality out of the box.

This means two record instances are considered equal if all their fields are equal, even if they're different objects in memory.

✅ Example:

public record Point(int x, int y) {}

public class Demo {
    public static void main(String[] args) {
        Point p1 = new Point(3, 4);
        Point p2 = new Point(3, 4);

        System.out.println(p1 == p2);        // false (reference equality)
        System.out.println(p1.equals(p2));   // true  (data equality)
    }
}

💡 What's Going On?

"==" checks if p1 and p2 are the same object → returns false
.equals() (auto-generated by the record) checks if all fields match → returns true

🚀 Why This Matters ?
This behaviour makes records ideal for use cases where:

You care about what data an object holds, not which instance it is.
You need reliable comparisons in collections like Set, Map, or during testing

So with records, you get true value semantics just like data classes in Kotlin or case classes in Scala.

🔐 Records Are Strongly Immutable, Even Reflection Can't Modify Them

Unlike regular classes where immutability is often conventional (you just don’t provide setters), with records it is enforced by the JVM.

All fields in a record are private, final and assigned once via the constructor. Even the reflection which can typically bypass access modifiers can't modify record fields, unless you use JVM-breaking hacks like Unsafe or deep instrumentation.

✅ Example:

public record User(String username, int age) {
}

🧪 Trying to Break with Reflection:

import java.lang.reflect.Field;

public class Demo {
    public static void main(String[] args) throws Exception {
        User user = new User("Ankit", 28);
        Field field = User.class.getDeclaredField("age");
        field.setAccessible(true);
        field.set(user, 35);  // Throws IllegalAccessException or InaccessibleObjectException
    }
}

💥 Result is an exception like:

java.lang.IllegalAccessException: Cannot modify a final field

Or, on newer JVMs:

java.lang.reflect.InaccessibleObjectException

🚫 Why This Matters

Ensures data integrity
Safer to use in multi-threaded environments
Makes records ideal for cache keys, messages, DTOs, and functional programming patterns
With records, immutability isn’t just a best practice. It’s a guarantee baked into the language and runtime.

👉 "Hope this article helps other developers get started with Java Records. Let me know what you think. Have you used records in production yet?"

References

JEP-359

🚀 How We Preserved Event Order in Kafka While Streaming From Cosmos DB: A Real-World Journey

Ankit Sood — Sat, 02 Aug 2025 16:57:25 +0000

Just publish the event after saving to the database. How hard can it be?

Famous last words. 😅

What started as a simple requirement in one of my projects quickly turned into a deep dive on ordering guarantees, idempotence, and the quirks of using Cosmos DB’s Mongo API.

This is the story of how we used the Outbox Pattern, Spring Boot, and Apache Kafka to build a resilient system that preserved strict ordering of events—even in the face of failures.

🎯 Goal

We were building an accounts payable backend on Azure:

📦 Cosmos DB (Mongo API) for storing invoices.
🔥 Apache Kafka as the backbone for event-driven workflows.
🌱 Spring Boot for application logic.

🎯 Requirements

✅ When a user places an order, save it to the database
✅ Immediately publish an OrderConfirmed event to Kafka
✅ Ensure downstream consumers always process events in order

Sounds simple enough… right?

🎯 Approaches

😅 The First Naive Attempt and Here’s how we started:

// 1. Save order
mongoTemplate.insert(order);

// 2. Publish to Kafka
kafkaTemplate.send("order-events", orderId, payload);

This worked for some time But pretty soon, things blew up.

🔥 Problem #1: Lost Events
If Kafka was down after the DB write, the event was lost forever. Downstream services never found out about the order.

🔥 Problem #2: Out-of-Order Delivery
Even with retries, we saw scenarios like:

OrderShipped(order-123) arrives BEFORE OrderConfirmed(order-123)

This broke downstream systems relying on proper sequencing.

🧩 Enter the Outbox Pattern to save the day

Instead of trying to send events directly to Kafka, we:

Extended our DB schema with an outbox collection
When saving an order, also saved a pending outbox event in the same transaction
Built a background publisher to read outbox events and send them to Kafka in order
Marked events as "SENT" only after Kafka acknowledged

Here’s the flow:

[App writes Order + Outbox event] → [Outbox Processor] → [Kafka]

📦 Outbox pattern acts as a reliable buffer between your DB and Kafka. This pattern works well with any SQL Database but doesn't work with Cosmos DB mongo API.

⚡ Challenges With Cosmos DB Mongo API

We knew Cosmos DB’s Mongo API isn’t MongoDB and Transactions only work if:
✅ Both writes are in the same collection
✅ Documents share the same partitionKey

So we designed a shared collection:

🗄 Structure of Documents

Order Document

{
  "_id": "order-123",
  "type": "order",
  "userId": "user-456",
  "status": "confirmed",
  "partitionKey": "user-456"
}

Outbox Event Document

{
  "_id": "event-789",
  "type": "event",
  "eventType": "OrderConfirmed",
  "payload": { "orderId": "order-123" },
  "createdAt": "2025-06-27T12:00:00Z",
  "status": "PENDING",
  "partitionKey": "user-456"
}

Both uses the same partitionKey to enable transactions.

💻 The Implementation:

We have used Java and Spring boot to implement this. Below is how we put it all together:

🏗 Atomic Write Service
We ensured order + outbox event writes were atomic:

@Service
public class OrderService {
    @Autowired private MongoTemplate mongoTemplate;

    @Transactional
    public void createOrderWithEvent(Order order, OutboxEvent event) {
        mongoTemplate.insert(order, "sharedCollection");
        mongoTemplate.insert(event, "sharedCollection");
    }
}

🚀 Kafka Outbox Publisher
A scheduled Spring component polls pending outbox events and sends them to Kafka in creation order.

@Component
public class OutboxEventPublisher {
    @Autowired private MongoTemplate mongoTemplate;
    @Autowired private KafkaTemplate<String, String> kafkaTemplate;

    @Scheduled(fixedDelay = 1000)
    public void publishEvents() {
        Query query = Query.query(Criteria.where("type").is("event").and("status").is("PENDING"))
                .with(Sort.by("createdAt")).limit(10);

        List<OutboxEvent> events = mongoTemplate.find(query, OutboxEvent.class, "sharedCollection");

        for (OutboxEvent event : events) {
            try {
                kafkaTemplate.send("order-events", event.getPartitionKey(), event.getPayload()).get();

                // Mark as SENT
                mongoTemplate.updateFirst(
                    Query.query(Criteria.where("_id").is(event.getId())),
                    Update.update("status", "SENT"),
                    "sharedCollection"
                );

                System.out.println("Published event: " + event.getId());

            } catch (Exception e) {
                System.err.println("Kafka send failed for event: " + event.getId());
                break; // Stop to avoid out-of-order sends
            }
        }
    }
}

🔐 Kafka Producer Configuration
We enabled idempotence and sync sends to preserve order:

props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, 1);

🏗 Transaction Manager Configuration
Spring’s @Transactional is not same as Transaction for Cosmos or Mongo.

It's designed for relational databases (JPA, JDBC, etc.)
It Relies on a transaction manager like PlatformTransactionManager
For MongoDB, it needs MongoDB 4.x+ with replica sets and Spring Data MongoDB transaction support.

@Bean
    MongoTransactionManager transactionManager(MongoDatabaseFactory dbFactory) {
        return new MongoTransactionManager(dbFactory);
    }

To use MongoDB transactions via Spring Boot, all the below things must be true:
✅ MongoDB server version ≥ 4.0
✅ Connected to a replica set (Cosmos DB Mongo API v4.0+ does support transactions but only under certain conditions)
✅ Use MongoTransactionManager explicitly in Spring

But Cosmos DB Mongo API:
⚠ Only supports transactions within the same partition (documents must share partitionKey)
⚠ Has subtle differences from native MongoDB transactions

In case, a transaction should be started explicitly, we can use

try (ClientSession session = mongoTemplate.getMongoDbFactory().getMongoClient().startSession()) {
    session.startTransaction();

    try {
        mongoTemplate.withSession(() -> session).insert(order, "sharedCollection");
        mongoTemplate.withSession(() -> session).insert(event, "sharedCollection");
        session.commitTransaction();
    } catch (Exception e) {
        session.abortTransaction();
        throw e;
    }
}

🎯 Final Thoughts

The Outbox Pattern helped us bridge the gap between Cosmos DB and Kafka. It’s a robust, production-ready solution for ordered, reliable event streaming—even when your DB doesn’t support native CDC.

CDC was one more approach that we could have used. In the past, we have used cosmos change stream which had its own issues and hence we decided to go ahead with this.

👉 Have you faced ordering challenges in Kafka pipelines? How did you solve them? Share your experiences below!

References

Source Code

Getting Started with Kafka Connect: A Comprehensive Guide

Ankit Sood — Sun, 16 Feb 2025 17:53:49 +0000

This article will be focussed on understanding what Kafka Connect is, what it does and how to run this in any environment. Inspiration for this article was my struggle to identify how to run the Kafka Connect with out using the Confluent Cloud as most of the articles on the web either uses Confluent Cloud or a shell script to start the connector. I wanted to create a Java application which can use the open source Kafka Connect jar and can be hosted like any other FAT jar.

Source Code

If you would like to try it by yourself or you want to go through my source code, you can clone my GitHub repository and go through the readme file. So, Let's begin.

Prerequisites

We will be using Java as primary Language and Maven as the dependency management solution. So, to run this you'll need atleast Java 17 and Maven 3.9 installed in your system. In case you don't have Java or maven installed, you can download and install it by using below links.

JDK
Maven

I'll running Kafka locally in my system. If you want to know how to setup Kafka Locally, please use this link to set it up. Also, I am using Kafdrop to view Kafka brokers, topics, messages, consumer groups, and ACLs.

Introduction

Kafka Connect is a tool that helps move data between Apache Kafka and other systems in a reliable and scalable way. It makes it easy to set up connectors that transfer large amounts of data in and out of Kafka. You can use it to bring in data from entire databases or collect metrics from servers into Kafka topics for real-time processing. It also allows exporting data from Kafka to storage, query systems, or batch processing for offline analysis. Kafka Connect stores different types of information regarding the connector i.e. configuration of the connector, working status of the connector and last committed offset of the connector.

Kafka Connect currently supports two modes of execution:

Standalone(single process)
Distributed

In Standalone mode, all work is performed in a single process and the details related to configuration, offset and status is maintained in a file. This configuration is simpler to setup and get started with and may be useful in situations where only one worker makes sense (e.g. collecting log files), but it does not benefit from some of the features of Kafka Connect such as fault tolerance.

In Distributed mode, Kafka Connect stores the offsets, configs and task statuses in Kafka topics. It is recommended to manually create the topics for offset, configs and statuses in order to achieve the desired the number of partitions and replication factors.

Implementation

For the purpose of this exercise, we will just be running our Kafka Connect in distributed mode. Hence, we will create our offset, config and status topics in advance. We will use script given below to create these.

connect-offsets: Topic is used for storing offsets. This topic should have many partitions and be replicated and compacted.

# offset.storage.topic
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --topic connect-offsets --replication-factor 3 --partitions 1 --config cleanup.policy=compact

connect-configs: Topic is used for storing connector and task configurations; note that this should be a single partition, highly replicated, and compacted topic.

# config.storage.topic
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --topic connect-configs --replication-factor 1 --partitions 1 --config cleanup.policy=compact

connect-status: Topic to use for storing statuses. This topic can have multiple partitions and should be replicated and compacted.

# status.storage.topic
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --topic connect-status --replication-factor 1 --partitions 10 --config cleanup.policy=compact

We can see in the above screenshot that the required topics are created. Next step would be to create a sample java project. I'll be using my InteliJ to create it but you can use any other IDE or CLI as well.

After the project is successfully, created we need to add the following dependencies into our pom.xml.

<!-- Logging related dependencies starts -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>${log4j2.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>${log4j2.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>${log4j2.version}</version>
        </dependency>
        <!-- Logging related dependencies ends -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <scope>provided</scope>
            <version>1.18.30</version>
        </dependency>
        <!-- Apache Kafka Connect dependencies starts -->
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-runtime</artifactId>
            <version>3.7.2</version>
        </dependency>
        <!-- Apache Kafka Connect dependencies ends -->

The dependencies "log4j-api", "log4j-core" and "log4j-slf4j-impl" are used for logging purposes. We also need to add log4j2.xml to our classpath in order to specify the format of our logs.

One more thing to note here is "connect-runtime" dependency. We only need to add only this dependency and it brings libraries like "connect-api", "connect-json" & "connect-transforms" transitively. Since, all these libraries are also part of our classpath, our Kafka Connect Application will work as a server and we should be able to interact with it using the REST endpoints.

We also need to add the "spring-boot-maven-plugin" to our pom.xml under build section. This will allow us to generate the fat jar and provide the class which has the main method. This method is called on the startup of the jar.

<plugins>
    <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <executions>
            <execution>
                <configuration>
                    <mainClass>dev.ankis.KafkaConnectApplication</mainClass>
                </configuration>
                <goals>
                    <goal>repackage</goal>
                </goals>
            </execution>
        </executions>
        <version>3.4.2</version>
    </plugin>
</plugins>

As a last step, we need to add a configuration file with the name "distributed-config.properties" under the src/main/resources.This will store all the configuration properties.

bootstrap.servers=localhost:9092
group.id=connect-cluster
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.topic=connect-offsets
offset.storage.replication.factor=1
status.storage.topic=connect-status
status.storage.replication.factor=1
offset.flush.interval.ms=10000
listeners=HTTP://:8083

Post these configurations, we can go ahead and run the application.

To see if our kafka connector server is up and running, we can run the following cURL

curl --request GET \
  --url http://localhost:8083/connector-plugins

DEV Community: Ankit Sood

Why I Stopped Using Lombok’s `@Data` on JPA Entities

The Bug That Didn’t Make Sense

When toString() Becomes a Trap

The Subtle Performance Issue

The Core Issue

What I Changed

Takeaway

Final Thought

📚 Further Reading

Performance Testing Databases the Right Way: A Real-World Engineering Journey

Why I Couldn’t Trust Generic Database Benchmarks

Step 1: Defining What “Good Performance” Actually Meant

Step 2: Understanding the Application’s Actual Database Behaviour

Step 3: Classifying Operations by Frequency and Importance

Step 4: Designing Realistic Workload Scenarios

Step 5: Designing the JMeter Setup to Behave Like a Java Application

The Workload Modeling Was Complete — Now the Real Testing Could Begin

Coming Up Next: Azure SQL vs Google Spanner — The Actual Results

Building Custom HTTP Request Metrics in Spring Boot: A Deep Dive into Observability

Architecture at a Glance

Step 1: Add Micrometer

Step 2: The Filter Code

Step 3: Observe the Metrics

Final Thoughts

🚀 Slow queries on Cosmos DB ?Here’s How I Fixed Pain with Deep Pagination & Cross-Partition Queries

Cosmos DB Introduction

Why It Matters ?

Query by Partition Key = Laser Target 🎯

Cross-Partition Query = Scatter & Gather 🍂

Real-World Scenario

❌ The Problem

Why were these search patterns a problem ?

✅ Solution

⚙️ The Architecture

📃 Why this works ?

Final Thoughts

Demystifying Java Records: A Developer's Guide

Introduction

Characteristics

References

🚀 How We Preserved Event Order in Kafka While Streaming From Cosmos DB: A Real-World Journey

🎯 Goal

🎯 Requirements

🎯 Approaches

⚡ Challenges With Cosmos DB Mongo API

💻 The Implementation:

🎯 Final Thoughts

References

Getting Started with Kafka Connect: A Comprehensive Guide

Source Code

Prerequisites

Introduction

Implementation

References

When `toString()` Becomes a Trap