DEV Community: Ed Legaspi

Logs Aren't an Audit Trail: Debugging Business Data in Spring Boot

Ed Legaspi — Thu, 29 Oct 2026 10:35:02 +0000

A production support ticket comes in:

"Why was this account deactivated?"

You check the database:

id     = 1042
status = DEACTIVATED

Then you search the application logs:

2026-10-12T09:42:18Z INFO AccountService
Account 1042 updated successfully

Great. The application updated the account.

But that doesn't answer the question.

What was the previous status? Who initiated the change? Did the update actually commit? Was this the first transition, or had the account changed several times?

This is a distinction that becomes increasingly important when supporting production applications:

Application logs describe execution. Audit trails describe changes to business data.

They are related, but they aren't interchangeable.

1. A Log Message Isn't Proof of a Committed Change

Consider a typical Spring Boot service:

@Service
@RequiredArgsConstructor
public class AccountService {

    private final AccountRepository accountRepository;

    @Transactional
    public void deactivate(Long accountId) {
        Account account = accountRepository.findById(accountId)
            .orElseThrow(() -> new AccountNotFoundException(accountId));

        account.setStatus(AccountStatus.DEACTIVATED);

        log.info("Account {} deactivated", accountId);

        validateAdditionalRules(account);
    }
}

Suppose validateAdditionalRules() throws an unchecked exception.

By default, Spring rolls back the transaction.

But the log message has already been emitted.

You might end up with:

APPLICATION LOG
09:42:18 Account 1042 deactivated

DATABASE
Account 1042
status = ACTIVE

The log is not necessarily incorrect. It tells us that execution reached the logging statement.

But it does not prove the database change was committed.

This is one of the most important reasons not to treat application logs as authoritative business history.

What about logging after commit?

You could use Spring transaction synchronization to log only after a successful commit.

That addresses the premature-success-message problem, but it introduces another limitation.

What happens if the database commits successfully and the process crashes before the log is delivered?

The business change exists, but the corresponding log message might not.

Moving logging after commit does not make the database update and log delivery atomic.

For important business-data history, we need a different persistence guarantee.

2. Hibernate Envers Provides Transactional Revision History

Hibernate Envers is a natural fit for applications using Hibernate ORM.

Instead of manually logging every entity modification, we can configure entities for auditing.

For example:

@Entity
@Audited
@Table(name = "accounts")
public class Account {

    @Id
    private Long id;

    @Enumerated(EnumType.STRING)
    private AccountStatus status;

    private String reasonCode;
}

With Envers configured, changes made through Hibernate's audited persistence lifecycle can produce corresponding revision records.

Those records are normally persisted within the same database transaction as the entity modifications.

Conceptually:

BEGIN TRANSACTION
    |
    +-- Update Account
    |
    +-- Persist Audit Revision
    |
COMMIT

If the transaction rolls back, the entity modification and associated audit revision roll back together.

That gives us an important property:

The audit revision is tied to the committed business-data change.

There are still limitations.

Direct SQL modifications outside Hibernate, unsupported write paths, incorrect mappings, and incomplete revision metadata can compromise audit completeness.

Envers is not a universal database change-data-capture mechanism.

But for audited Hibernate-managed entities, it provides a solid foundation.

3. Logs and Audit Trails Have Different Data Models

Consider these application logs:

09:41:12 Account update requested
09:41:12 Validating account 1042
09:41:12 Validation successful
09:41:13 Updating account 1042
09:41:13 Account updated successfully

They help us reconstruct the execution.

Now compare that with an audit history:

Account #1042

Revision 18
2026-10-12T09:41:13Z

status:
ACTIVE -> SUSPENDED

reasonCode:
null -> CUSTOMER_REQUEST

changedBy:
admin-17

The second representation is organized around the business entity.

It provides a history of what changed, rather than a sequence of messages describing application execution.

Over time, the audit history might look like:

Oct 12
ACTIVE -> SUSPENDED

Oct 15
SUSPENDED -> ACTIVE

Oct 21
ACTIVE -> DEACTIVATED

That is much easier to consume when someone asks:

How did this account reach its current state?

It also opens the door to building administrative interfaces and support tools around revision history.

4. Why Not Just Log Every Field Change?

It's possible to write structured logs containing old and new values.

For some applications, that may be sufficient.

But there are several challenges.

Consistency: Every write path must follow the same logging convention, including scheduled jobs, message consumers, and administrative operations.

Transaction boundaries: Logging a change does not necessarily prove the corresponding transaction committed.

Retention: Application logs may be deleted long before business history is no longer needed.

Queryability: Searching logs is not the same as querying entity revisions, changed properties, and historical values.

Security: Logging complete entity snapshots can expose sensitive information.

These concerns deserve explicit architectural decisions.

If historical business-data changes are important, they should not depend entirely on individual developers remembering to add the correct log statements.

5. Audit History Still Doesn't Explain Everything

Suppose we have the following revision:

Account #1042

status:
ACTIVE -> SUSPENDED

changedBy:
account-service

changedAt:
2026-10-12T09:41:13Z

We know what changed.

But do we know why?

Not necessarily.

Hibernate Envers cannot automatically infer the business intent behind a state transition.

To make that possible, the application needs to capture relevant context.

For example:

reasonCode:
CUSTOMER_REQUEST

correlationId:
c5d8f1e2-7a40-4d9c-8b1e-9f1c02d5a611

The reason code provides explicitly recorded business context.

The correlation ID may allow us to connect the revision to related application logs and distributed traces, provided the identifier was propagated consistently.

This is where audit history and observability complement each other.

Audit history tells us what changed. Logs and traces help investigate the execution that produced the change.

Neither should invent information that was never captured.

6. Distributed Systems Need Both Event History and Audit History

Consider a simple asynchronous workflow:

Account API
    |
    v
Account Service
    |
    v
Database
    |
    v
Transactional Outbox
    |
    v
Kafka
    |
    v
Notification Service

Suppose an account is suspended.

Several things may happen:

The account status changes from ACTIVE to SUSPENDED.
An audit revision records the state change.
An outbox event is persisted.
The transaction commits.
The event is published asynchronously.
A downstream consumer processes the event.

Each step raises different operational questions.

Was the account state committed?

Was the event persisted?

Was the event published?

Did the consumer process it?

Was the notification delivered?

A single logging mechanism should not be expected to answer all these questions.

This is also where two of my open-source libraries complement each other.

NERV Event: What Happened to the Message?

NERV Event focuses on reliable event-driven processing through patterns such as:

Transactional outbox
Inbox processing
Retries
Idempotency

These capabilities help applications reason about asynchronous message delivery and processing.

NERV Audit: What Happened to the Data?

NERV Audit focuses on making historical changes to business entities easier to inspect and consume.

It builds on Hibernate Envers rather than replacing it.

The goal is to make revision history usable by applications, support tools, and administrative interfaces.

For example:

Account #1042

Revision 18
ACTIVE -> SUSPENDED

Revision 19
SUSPENDED -> ACTIVE

Revision 20
ACTIVE -> DEACTIVATED

The audit trail helps reconstruct committed entity changes.

The event infrastructure helps investigate the asynchronous workflow.

Both are useful, but they provide different evidence.

7. What Should a Production System Capture?

I find it useful to think about production diagnostics in terms of questions.

Question	Useful source
Why did this request fail?	Logs, exceptions, traces
Was this event published?	Outbox and messaging records
Was the message processed?	Consumer and inbox records
What was the previous entity state?	Audit history
Who changed this entity?	Audit actor metadata
Why was the change requested?	Explicit business context
How was the system performing?	Metrics

These mechanisms work best when their responsibilities are clear.

We don't need to audit every field or retain every log indefinitely.

We do need to identify which business changes matter and design reliable ways to investigate them.

8. The Engineering Principle

After years of working on enterprise Java systems, one lesson has become increasingly important to me:

A production system should make important business changes explainable through evidence, not guesswork.

That doesn't mean a single tool can explain everything.

It means having the right evidence available when an investigation begins.

Application logs help reconstruct execution.

Audit history helps reconstruct business-data changes.

Event-processing records help reconstruct asynchronous workflows.

Distributed tracing connects execution across services.

And explicit business context helps explain why particular decisions were made.

That's the philosophy behind NERV Audit and the broader NERV ecosystem.

Final Thoughts

Logs are essential for production support.

But they are not an audit trail.

A log statement may describe an operation that eventually rolls back.

A structured log message may not survive as long as the business needs its history.

And a sequence of application events may still be insufficient to reconstruct an entity's committed state transitions.

For Spring Boot applications using Hibernate, Envers provides a strong starting point for transactional revision history.

NERV Audit builds on that foundation to make historical changes easier to query, inspect, and consume.

Because when production support asks:

"Why is this account in this state?"

We should have better tools than searching through thousands of unrelated log messages.

Logs tell us what the application was doing. An audit trail tells us what happened to the data.

This is Article #2 in my NERV Audit series, exploring practical approaches to production-ready auditing in Java and Spring Boot.

Explore the open-source NERV projects: GitHub — CzetsuyaTech

Read the canonical article: CzetsuyaTech

Dynamic Queries in Spring Data JPA Without Repository Method Explosion

Ed Legaspi — Tue, 27 Oct 2026 10:07:27 +0000

Spring Data JPA makes repository development incredibly convenient.

You define an entity, extend JpaRepository, and start writing methods such as:

List<Customer> findByStatus(CustomerStatus status);

List<Customer> findByCountry(String country);

List<Customer> findByStatusAndCountry(
    CustomerStatus status,
    String country
);

Nothing wrong with that.

Until your application needs a search endpoint with eight optional filters.

Suddenly, the repository starts accumulating methods that differ only in which filters are applied.

The problem isn't Spring Data JPA.

The problem is using fixed repository methods to represent a dynamic query.

Let's build a cleaner approach using Spring Data JPA Specifications.

1. Why Derived Query Methods Stop Scaling

Imagine a customer search API with the following filters:

Status
Country
Customer type
Registration date
Email
Account manager

All filters are optional.

A user might search by status alone, country and type together, or status, country, registration date, and account manager.

With six independent optional filters, there are potentially:

2⁶ = 64 combinations.

With eight filters, that becomes 256.

Of course, you don't necessarily need to implement every combination.

But the design problem is obvious.

Repository methods describe specific queries, while the API needs composable conditions.

We need to model the individual predicates instead of their combinations.

2. The Tempting Solution: Nullable JPQL Parameters

A common approach is to create one query with optional parameters.

@Query("""
    SELECT c
    FROM Customer c
    WHERE (:status IS NULL OR c.status = :status)
      AND (:country IS NULL OR c.country = :country)
      AND (:type IS NULL OR c.type = :type)
    """)
List<Customer> search(
    @Param("status") CustomerStatus status,
    @Param("country") String country,
    @Param("type") CustomerType type
);

This works for many applications.

However, it becomes harder to maintain as more filters are introduced.

You also need to consider how the database optimizer handles the optional OR predicates.

Depending on the database, parameter values, indexes, and execution plan, the query may perform differently from a query containing only the conditions actually needed.

This doesn't mean nullable JPQL parameters are always wrong.

It means they're not always the best abstraction for an evolving search API.

3. Enable Specification Execution

Spring Data JPA provides JpaSpecificationExecutor, which allows repositories to execute composable Specification<T> predicates.

First, consider a simple entity.

@Entity
@Table(name = "customer")
public class Customer {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Enumerated(EnumType.STRING)
    private CustomerStatus status;

    private String country;

    @Enumerated(EnumType.STRING)
    private CustomerType type;

    private Instant registeredAt;

    // Getters and setters
}

Now extend JpaSpecificationExecutor.

public interface CustomerRepository
        extends JpaRepository<Customer, Long>,
                JpaSpecificationExecutor<Customer> {
}

That's it.

We can now execute Specifications without creating a separate repository method for each filter combination.

4. Create Reusable Specifications

Let's define one Specification per condition.

public final class CustomerSpecifications {

    private CustomerSpecifications() {
    }

    public static Specification<Customer> hasStatus(
            CustomerStatus status) {

        return (root, query, cb) ->
            cb.equal(root.get("status"), status);
    }

    public static Specification<Customer> hasCountry(
            String country) {

        return (root, query, cb) ->
            cb.equal(root.get("country"), country);
    }

    public static Specification<Customer> hasType(
            CustomerType type) {

        return (root, query, cb) ->
            cb.equal(root.get("type"), type);
    }
}

Each method describes one predicate.

It doesn't know which other filters will be applied.

That's the key advantage.

We can combine these conditions without creating additional repository methods.

Specification<Customer> specification =
    Specification.allOf(
        CustomerSpecifications.hasStatus(
            CustomerStatus.ACTIVE
        ),
        CustomerSpecifications.hasCountry("PH")
    );

List<Customer> customers =
    customerRepository.findAll(specification);

Conceptually, the resulting SQL resembles:

SELECT *
FROM customer
WHERE status = 'ACTIVE'
  AND country = 'PH';

The exact SQL depends on the JPA provider and dialect.

5. Build a Dynamic Search Criteria Object

Real search endpoints receive optional values.

Let's represent them explicitly.

public record CustomerSearchCriteria(
    CustomerStatus status,
    String country,
    CustomerType type,
    LocalDate registeredFrom,
    LocalDate registeredTo
) {
}

Now create a method that builds a Specification using only the supplied filters.

Add this method to CustomerSpecifications:

public static Specification<Customer> from(
        CustomerSearchCriteria criteria) {

    List<Specification<Customer>> filters =
        new ArrayList<>();

    if (criteria.status() != null) {
        filters.add(hasStatus(criteria.status()));
    }

    if (criteria.country() != null
            && !criteria.country().isBlank()) {

        filters.add(hasCountry(criteria.country()));
    }

    if (criteria.type() != null) {
        filters.add(hasType(criteria.type()));
    }

    if (criteria.registeredFrom() != null) {
        filters.add(
            registeredOnOrAfter(criteria.registeredFrom())
        );
    }

    if (criteria.registeredTo() != null) {
        filters.add(
            registeredBefore(criteria.registeredTo().plusDays(1))
        );
    }

    return Specification.allOf(filters);
}

Notice what we're not doing.

We're not writing separate methods for:

findByStatusAndCountry()
findByStatusAndType()
findByCountryAndType()
findByStatusAndCountryAndType()

Instead, we're assembling predicates dynamically.

Adding another filter becomes a localized change.

A note about empty filters

Specification.allOf(...) supports an empty collection.

An empty set of filters results in an unrestricted query.

That might be appropriate for an administrative search endpoint, but not every API should permit unrestricted searches.

You may need additional authorization, tenant, or visibility restrictions that apply regardless of user-supplied filters.

6. Handle Date Ranges Correctly

Date filtering is one of those details that looks simple until you encounter timezone and timestamp precision problems.

Suppose the entity stores registration timestamps as Instant.

We want users to search between two calendar dates.

One tempting approach is to create an end-of-day timestamp such as:

2026-10-05T23:59:59.999999999Z

But timestamp precision varies between databases and column types.

A better approach is to use a half-open interval.

registeredAt >= start
registeredAt <  endExclusive

For example, October 1 through October 5 becomes:

registeredAt >= 2026-10-01T00:00:00Z
registeredAt <  2026-10-06T00:00:00Z

Implement the predicates:

public static Specification<Customer> registeredOnOrAfter(
        LocalDate date) {

    Instant start = date
        .atStartOfDay(ZoneOffset.UTC)
        .toInstant();

    return (root, query, cb) ->
        cb.greaterThanOrEqualTo(
            root.get("registeredAt"),
            start
        );
}

public static Specification<Customer> registeredBefore(
        LocalDate exclusiveDate) {

    Instant end = exclusiveDate
        .atStartOfDay(ZoneOffset.UTC)
        .toInstant();

    return (root, query, cb) ->
        cb.lessThan(
            root.get("registeredAt"),
            end
        );
}

These examples use UTC calendar boundaries.

If your business defines dates in a specific local timezone, convert those dates using the appropriate ZoneId.

Don't assume UTC calendar days and business-local calendar days are interchangeable.

7. Add Pagination and Sorting

Dynamic filtering is useful, but a production search API also needs pagination.

Fortunately, JpaSpecificationExecutor already supports Pageable.

@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
public class CustomerSearchService {

    private final CustomerRepository customerRepository;

    public Page<Customer> search(
            CustomerSearchCriteria criteria,
            Pageable pageable) {

        Specification<Customer> specification =
            CustomerSpecifications.from(criteria);

        return customerRepository.findAll(
            specification,
            pageable
        );
    }
}

The controller can expose the search endpoint:

@RestController
@RequestMapping("/api/customers")
@RequiredArgsConstructor
public class CustomerController {

    private final CustomerSearchService searchService;

    @GetMapping
    public Page<CustomerResponse> search(
            @ModelAttribute CustomerSearchCriteria criteria,
            Pageable pageable) {

        return searchService
            .search(criteria, pageable)
            .map(CustomerResponse::from);
    }
}

Here, CustomerResponse.from(...) represents an application-defined mapping method, not a Spring Data JPA API.

A request might look like:

GET /api/customers?status=ACTIVE&country=PH&page=0&size=20&sort=registeredAt,desc

The repository doesn't need to know which filter combination was selected.

It receives a Specification and a Pageable.

For production APIs, remember to enforce maximum page sizes and restrict which properties can be used for sorting.

8. What About Joins?

Specifications can also filter through entity relationships.

Suppose a customer has many orders.

We want customers with completed orders.

A straightforward implementation might be:

public static Specification<Customer> hasCompletedOrder() {

    return (root, query, cb) -> {
        Join<Customer, Order> orders =
            root.join("orders");

        return cb.equal(
            orders.get("status"),
            OrderStatus.COMPLETED
        );
    };
}

But there's an important catch.

If a customer has multiple completed orders, the SQL join can produce multiple matching rows.

This can complicate pagination and counts.

For an existence condition, an EXISTS subquery is often a better fit.

public static Specification<Customer> hasCompletedOrder() {

    return (root, query, cb) -> {

        Subquery<Long> subquery =
            query.subquery(Long.class);

        Root<Order> order = subquery.from(Order.class);

        subquery.select(cb.literal(1L));

        subquery.where(
            cb.equal(order.get("customer"), root),
            cb.equal(
                order.get("status"),
                OrderStatus.COMPLETED
            )
        );

        return cb.exists(subquery);
    };
}

This example assumes Order has a customer relationship.

The important lesson is that query composition does not remove the need to understand relational behavior.

A Specification is still a database query.

9. Watch Out for Count Queries

When you call:

Page<Customer> page =
    customerRepository.findAll(specification, pageable);

Spring Data JPA generally needs to determine the total number of matching records to populate the page metadata.

For simple queries, that's usually fine.

For queries involving expensive joins or large datasets, the count query can become a significant part of the request cost.

Some useful options include:

Use a Slice-oriented retrieval strategy when total counts aren't required.
Keep expensive fetch behavior out of count queries.
Consider separate content and count Specifications where supported.
Inspect generated SQL and execution plans.

Don't assume that a dynamic query is efficient just because the Java code is concise.

10. Where NERV Persistence Fits

So far, everything we've implemented uses native Spring Data JPA functionality.

That's intentional.

Spring Data JPA already gives us the core query composition primitives.

But when multiple services implement dynamic search, developers may still repeatedly write infrastructure for:

Optional filter handling
Operator translation
Search request validation
Pagination conventions
Sorting conventions
Common query construction

This is the kind of repetition that motivates reusable persistence infrastructure.

NERV Persistence is built around the idea of standardizing common persistence mechanics while allowing each application to retain its own domain-specific behavior.

The intended architectural separation is:

REST API
    |
    v
Search Criteria
    |
    v
Domain Specifications
    |
    v
Reusable Persistence Infrastructure
    |
    v
Spring Data JPA
    |
    v
Hibernate / Database

This diagram is conceptual; it doesn't imply that every layer corresponds to a specific NERV Persistence class.

The goal isn't to hide Spring Data JPA.

It's to reduce repeated infrastructure work without losing the flexibility of the underlying framework.

You can explore the project here:

NERV Persistence on GitHub

11. When Not to Use Specifications

Specifications are powerful, but they're not always the right abstraction.

For a fixed lookup, a derived method is often clearer:

Optional<Customer> findByExternalId(String externalId);

For complex reporting queries involving aggregations, window functions, or database-specific features, explicit SQL may be easier to maintain and optimize.

For high-volume read models, specialized projections or query implementations may be more appropriate.

The objective isn't to eliminate repository methods.

It's to stop representing dynamic search requirements as a growing collection of fixed repository methods.

Final Thoughts

Derived repository methods are one of Spring Data JPA's best features.

But they work best when the query is fixed.

Once search filters become optional and combinatorial, query composition becomes a better abstraction.

Specifications allow us to:

Define predicates independently.
Combine only the filters supplied by a request.
Integrate naturally with pagination and sorting.
Keep repository interfaces small.
Evolve search behavior without multiplying repository methods.

And when these patterns appear across multiple applications, reusable persistence infrastructure can help standardize the mechanics without hiding domain intent.

The goal isn't to write the least amount of code.

It's to make the next search requirement easier to implement without making the existing persistence layer harder to understand.

This is Article #2 in the NERV Persistence series, exploring practical persistence architecture for Java and Spring Boot applications.

Explore NERV Persistence:

https://github.com/czetsuyatech/nerv-persistence

Read the canonical article on CzetsuyaTech:

https://www.czetsuyatech.com/2026/10/dynamic-queries-spring-data-jpa.html

If you work with dynamic search APIs in Spring Data JPA, what's your preferred approach: Specifications, QueryDSL, Criteria API, or explicit JPQL?

Designing a Consistent Error Contract for Spring Boot APIs

Ed Legaspi — Sat, 24 Oct 2026 07:06:06 +0000

Most Spring Boot applications have a global exception handler.

Far fewer have a well-defined error contract.

There's an important difference.

A global exception handler decides how an exception becomes an HTTP response. An error contract defines what that response means to the application consuming it.

Once services begin communicating over HTTP, that distinction becomes critical.

A frontend needs stable identifiers to display meaningful messages. A backend service needs to distinguish permanent business failures from potentially recoverable dependency failures. Operators need enough diagnostic context to investigate incidents.

These requirements cannot be satisfied reliably by returning an exception message and an HTTP status alone.

While building NERV Exception, an open-source Java and Spring Boot exception-handling library, I explored how to make API failures consistent without tightly coupling independently deployed microservices.

This article walks through the design principles, response structure, Spring Boot implementation patterns, and service-boundary decisions behind that approach.

1. Why HTTP Status Codes Are Not Enough

Consider an order-processing API.

Three different requests fail with HTTP 409 Conflict:

ORDER_ALREADY_PROCESSED
PAYMENT_ALREADY_CAPTURED
INVENTORY_RESERVATION_CONFLICT

All three represent conflicts, but they do not mean the same thing.

A client receiving only:

{
  "status": 409,
  "message": "Conflict"
}

cannot reliably determine what action to take.

It might need to show a warning, initiate reconciliation, or ask the user to try a different operation.

Parsing the message isn't a good solution because human-readable messages can change.

Instead, introduce stable application error codes:

{
  "code": "ORDER_ALREADY_PROCESSED",
  "message": "The order has already been processed",
  "status": 409
}

The HTTP status communicates the broad outcome.

The application error code communicates the specific failure.

An HTTP status is a classification mechanism, not a complete business error vocabulary.

2. Define a Consistent Error Response

A useful error response should support three audiences:

API consumers, which need stable identifiers and actionable information.
Developers, who need predictable behavior across applications.
Operators, who need enough context to investigate failures.

NERV Exception uses a flat response structure represented by NervErrorResponse.

An illustrative response looks like this:

{
  "code": "ORDER_ALREADY_PROCESSED",
  "message": "The order has already been processed",
  "status": 409,
  "retryable": false,
  "category": "BUSINESS",
  "traceId": "abc123",
  "spanId": "def456",
  "path": "/api/orders/123/process",
  "timestamp": "2026-10-24T08:30:00Z",
  "origin": {
    "service": "order-service",
    "instance": "order-service-1",
    "version": "1.0.0",
    "environment": "production"
  },
  "details": {}
}

The application-specific code and category above are illustrative. Not every deployment will populate every diagnostic field.

Each field serves a different purpose.

Field	Responsibility
`code`	Stable application error identifier
`message`	Human-readable explanation
`status`	HTTP response status
`retryable`	Recovery guidance
`category`	General failure classification
`traceId`	Distributed tracing correlation
`spanId`	Span correlation
`path`	Request path
`timestamp`	Response timestamp
`origin`	Service-origin information
`details`	Structured, error-specific context

The important design choice is to separate these responsibilities rather than overload a single message or status field.

3. Model Error Codes as Stable Contracts

Suppose your API returns:

CUSTOMER_DISABLED

A frontend recognizes the code and displays an account-specific message.

Later, someone improves the wording of the response:

Old: Customer is disabled.
New: This customer account is currently inactive.

Nothing should break.

The code remains stable even though the message changes.

Now imagine changing the code itself:

CUSTOMER_DISABLED
        ↓
CUSTOMER_INACTIVE

That may break consumers relying on the original identifier.

This is why error codes should be treated like other published API contracts.

A simple Java model

For an application-specific implementation, you might define:

public record ApiError(
        String code,
        String message,
        int status,
        String category,
        boolean retryable
) {}

This is a simplified educational example, not a replacement for NERV Exception's full response model.

You can then represent application errors through definitions such as:

public enum OrderError {

    ORDER_ALREADY_PROCESSED(
            "ORDER_ALREADY_PROCESSED",
            409,
            "BUSINESS",
            false
    ),

    ORDER_NOT_FOUND(
            "ORDER_NOT_FOUND",
            404,
            "CLIENT",
            false
    );

    private final String code;
    private final int status;
    private final String category;
    private final boolean retryable;

    OrderError(
            String code,
            int status,
            String category,
            boolean retryable
    ) {
        this.code = code;
        this.status = status;
        this.category = category;
        this.retryable = retryable;
    }

    public String code() {
        return code;
    }

    public int status() {
        return status;
    }

    public String category() {
        return category;
    }

    public boolean retryable() {
        return retryable;
    }
}

The purpose isn't to recommend that every team build its own enum framework.

It's to demonstrate that an error definition contains more than a message.

It represents an agreement about how a particular failure should be interpreted.

4. Keep Status, Category, and Retryability Independent

It's tempting to derive everything from HTTP status.

For example:

boolean retryable = status >= 500;

But that rule is too simplistic.

A 500 response doesn't necessarily mean the operation is safe to repeat.

A 409 conflict may describe a permanent business condition or a concurrency conflict that can be resolved through a controlled retry.

Likewise, categories should describe the nature of a failure rather than merely repeat the HTTP status family.

Consider:

Failure	Status	Retryable
Invalid request	400	No
Resource not found	404	No
Already processed	409	No
Rate limit exceeded	429	Policy-dependent
Dependency unavailable	503	Potentially

These are illustrative classifications, not universal rules.

In NERV Exception, categories are represented as strings rather than a closed enum.

That allows applications to define categories appropriate to their domains.

However, flexibility also creates responsibility: teams should establish consistent category naming conventions.

Error semantics should be explicit, not inferred from HTTP status alone.

5. Centralize Response Conversion in Spring Boot

A consistent error contract is useful only if the application produces it consistently.

Spring Boot provides @RestControllerAdvice for centralized exception handling.

A minimal educational example might look like this:

@RestControllerAdvice
public class ApiExceptionHandler {

    @ExceptionHandler(OrderAlreadyProcessedException.class)
    public ResponseEntity<ApiError> handle(
            OrderAlreadyProcessedException exception
    ) {
        ApiError response = new ApiError(
                "ORDER_ALREADY_PROCESSED",
                "The order has already been processed",
                409,
                "BUSINESS",
                false
        );

        return ResponseEntity
                .status(HttpStatus.CONFLICT)
                .body(response);
    }
}

This is a simplified Spring MVC pattern. The custom exception and DTO are illustrative.

The problem with implementing this independently in every service is that teams gradually introduce differences.

One service includes error codes.

Another includes only messages.

Another returns validation details in an incompatible format.

Another exposes internal exception causes.

A reusable library can centralize common response conversion behavior while allowing each application to define its own errors.

That is one of the reasons NERV Exception separates shared error contracts, core behavior, and Spring Web integration into different modules.

6. Represent Validation Failures as Structured Data

Validation errors require special consideration.

A single request can contain multiple invalid fields.

For example:

{
  "name": "",
  "quantity": -5
}

Returning only:

{
  "message": "Validation failed"
}

forces consumers to guess which fields need attention.

A more useful response is:

{
  "code": "VALIDATION_ERROR",
  "message": "Request validation failed",
  "status": 400,
  "category": "CLIENT",
  "retryable": false,
  "details": {
    "errors": {
      "name": "must not be blank",
      "quantity": "must be greater than or equal to 1"
    }
  }
}

This illustrates the details.errors object-map structure used by NERV Exception's validation handling.

The exact error code and messages depend on the application and configuration.

The main benefit is that clients can map validation messages to individual form fields without parsing arbitrary strings.

Structured validation details are especially useful for frontend applications, mobile clients, and API integrations.

7. Don't Require Every Microservice to Know Every Error Code

Now consider a distributed architecture:

API Gateway
     |
     v
Order Service
     |
     v
Payment Service

Payment Service returns:

{
  "code": "PAYMENT_ALREADY_PROCESSED",
  "message": "Payment has already been processed",
  "status": 409,
  "category": "BUSINESS",
  "retryable": false
}

Order Service has never encountered this code before.

There are several possible approaches.

Option A: Replace the error with a generic integration failure.

This is simple but discards potentially useful information.

Option B: Import every Payment Service error definition.

This makes the error recognizable but creates tighter coupling between services.

Option C: Preserve unknown error codes using the shared contract.

This allows services to evolve their business error catalogs independently.

NERV Exception's Feign integration supports the third approach through RemoteNervErrorCode.

Unknown versus known codes

The distinction is important.

When a downstream error code is unknown locally, the Feign decoder can construct a remote error definition that preserves the code, message, response-body status, retryability, and category.

When the code is registered locally, the local definition determines status, category, and retryability.

The downstream message, origin, and supported details remain available through the downstream exception.

This means a local registry can influence how a remote error is interpreted.

Registration isn't merely a lookup convenience; it can establish local semantic authority.

What isn't preserved automatically?

Not every field travels unchanged.

The original downstream traceId, spanId, path, and timestamp are not retained by the downstream exception.

When a receiving service generates a new HTTP response, those fields are populated from its current context.

Also, malformed or non-NERV responses are delegated to the configured fallback Feign decoder.

Therefore, a consistent error contract improves propagation but does not guarantee lossless forwarding of every field.

8. Retryable Errors Still Need a Retry Policy

Suppose Payment Service returns:

{
  "code": "PAYMENT_PROVIDER_UNAVAILABLE",
  "status": 503,
  "category": "DEPENDENCY",
  "retryable": true
}

Should Order Service automatically retry?

Not necessarily.

Consider a payment that succeeded but whose response was lost.

Repeating the operation without idempotency protection could create a duplicate charge.

Retry decisions must consider:

Idempotency and operation safety
Unknown outcomes
Retry limits
Backoff and jitter
Dependency health
Retry amplification across multiple services

NERV Exception's Feign decoder can wrap a retryable downstream failure in Feign's RetryableException.

But it does not automatically configure a retry policy.

The application and its Feign configuration determine whether retries occur.

Another consideration is what happens after retry exhaustion.

If the wrapper reaches a generic exception handler without application-specific handling, the final response may be a generic internal server error rather than the original downstream error contract.

Retry classification, retry execution, and final error presentation must be designed together.

9. Make Errors Observable Without Exposing Internals

A production error response should help engineers investigate incidents.

But it shouldn't expose sensitive implementation details to public clients.

For example, avoid returning raw database errors, internal hostnames, stack traces, or confidential request values.

Instead, return safe application-level information and use logs and tracing for deeper diagnostics.

A response might include:

{
  "code": "PAYMENT_PROVIDER_UNAVAILABLE",
  "message": "Payment processing is temporarily unavailable",
  "status": 503,
  "traceId": "abc123",
  "spanId": "def456"
}

NERV Exception can read Micrometer tracing context when available.

However, adding tracing fields to a response does not automatically create distributed traces.

The application must configure the necessary tracing infrastructure.

Review production exposure settings

There is also a practical configuration lesson.

In the reviewed NERV Exception 1.4.0 implementation, the Spring Boot starter defaults enable internal message and cause exposure, although stack-trace exposure is disabled.

For public APIs, review and explicitly restrict these settings.

The effective deployed configuration matters more than assumptions about what a library should hide.

10. Error Contracts Need Compatibility Rules

An error contract is a public API contract.

That means changes to its semantics can affect consumers.

Consider:

{
  "code": "PAYMENT_TIMEOUT",
  "retryable": false
}

Later, the service changes the response to:

{
  "code": "PAYMENT_TIMEOUT",
  "retryable": true
}

The JSON schema is unchanged.

But the behavior expected from consumers may change significantly.

A client might begin retrying an operation it previously treated as final.

For this reason, compatibility reviews should include more than field names and data types.

They should consider:

Error code stability
Status changes
Category changes
Retryability changes
Unknown-code behavior
Diagnostic-field exposure

A stable schema doesn't necessarily mean stable semantics.

11. Where NERV Exception Fits

NERV Exception provides reusable infrastructure for consistent exception handling in Java and Spring Boot.

Its modules include:

Module	Responsibility
`nerv-exception-api`	Shared error contracts
`nerv-exception-core`	Core exception and error-code behavior
`nerv-exception-spring-web`	Spring Web integration
`nerv-exception-spring-feign`	OpenFeign error decoding
`nerv-exception-spring-boot-starter`	Spring Boot integration and configuration
`nerv-exception-jpa`	Persistence-related integration
`nerv-exception-event`	Event-related support
`nerv-exception-spring-kafka`	Kafka-related exception support

The library doesn't eliminate the need for application-specific decisions.

Each service still needs to own its business error codes, decide which details are safe to expose, and define appropriate retry and recovery policies.

The objective is to standardize the infrastructure without centralizing every business decision.

Final Thoughts

A consistent error contract is more than a JSON format.

It is an agreement about how services communicate failures.

A good contract gives clients stable identifiers, separates error classification from recovery decisions, provides safe diagnostic context, and tolerates independently evolving services.

The difficult part isn't serializing an exception.

It's preserving meaningful failure semantics as applications and microservices grow.

Share the language of failure without forcing every service to share the same business error catalog.

Explore NERV Exception

NERV Exception is part of the open-source NERV ecosystem of Java and Spring Boot libraries.

Explore the projects:

https://github.com/czetsuyatech

For the canonical version of this article and the rest of the NERV Exception series, visit:

https://www.czetsuyatech.com

Production-Ready Exception Handling in Spring Boot

Ed Legaspi — Mon, 19 Oct 2026 23:01:08 +0000

Exception handling is one of those concerns that seems simple until an application grows.

In a small Spring Boot application, a @RestControllerAdvice and a few exception handlers may be enough.

But as the system evolves, new questions emerge:

How do clients distinguish business failures from infrastructure failures?
Should consumers depend on exception messages or stable error codes?
How should validation errors be represented?
What happens when a downstream microservice returns an error your service doesn't recognize?
How do you expose useful diagnostics without leaking internal implementation details?

These questions turn exception handling from a controller-level concern into an API design problem.

While building NERV Exception, an open-source exception-handling library for Java and Spring Boot, I explored how to address these problems through reusable contracts and integration modules.

This article covers the design principles behind that approach, along with the trade-offs that matter in production.

1. Why Basic Exception Handling Is Not Enough

Consider a typical Spring Boot exception handler:

@RestControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(Exception.class)
    public ResponseEntity<String> handle(Exception exception) {
        return ResponseEntity
                .status(HttpStatus.INTERNAL_SERVER_ERROR)
                .body(exception.getMessage());
    }
}

This is convenient, but it creates several problems.

First, an exception message isn't a reliable API contract. Developers may change it without realizing that consumers depend on the exact text.

Second, not every exception represents an internal server error. Business conflicts, validation failures, missing resources, and dependency failures have different meanings.

Third, returning raw exception messages may expose internal details.

Finally, there is no structured information for clients to make decisions.

A frontend may need to display a validation message. A backend consumer may need to distinguish a permanent rejection from a transient dependency failure. An operator may need a trace identifier to investigate the incident.

A plain string cannot reliably support all three.

2. Treat Error Responses as API Contracts

A production-oriented error response should communicate enough information for consumers to understand and handle the failure without exposing unnecessary implementation details.

For example:

{
  "code": "ORDER_ALREADY_PROCESSED",
  "message": "The order has already been processed",
  "status": 409,
  "retryable": false,
  "category": "BUSINESS",
  "traceId": "abc123",
  "spanId": "def456",
  "path": "/api/orders/123/process",
  "timestamp": "2026-10-20T08:30:00Z",
  "origin": {
    "service": "order-service",
    "instance": "order-service-1",
    "version": "1.0.0",
    "environment": "production"
  },
  "details": {}
}

This is an illustrative payload using the fields supported by NERV Exception's NervErrorResponse. The example business code and values are application-defined.

Each field has a purpose.

code provides a stable identifier that clients can interpret without parsing message text.

message provides a readable explanation appropriate for the intended audience.

status identifies the HTTP status associated with the response.

category describes the general class of failure.

retryable communicates whether retrying may be appropriate, although the caller must still determine whether the operation is safe to repeat.

traceId and spanId help correlate the response with distributed tracing when tracing context is available.

origin identifies the service associated with the failure.

details allows structured, error-specific information to be included when appropriate.

The important design principle is that consumers should depend on stable error semantics rather than exception class names or message text.

3. Separate Application Error Codes from HTTP Status Codes

HTTP statuses are necessary, but they aren't sufficient to describe application failures.

For example, several distinct problems may legitimately return HTTP 409:

409 Conflict
  ORDER_ALREADY_PROCESSED
  PAYMENT_ALREADY_CAPTURED
  INVENTORY_RESERVATION_CONFLICT

A client that receives only 409 Conflict knows the request could not be completed because of a conflict.

It doesn't necessarily know which conflict occurred.

Application error codes add the missing distinction.

However, it's important not to confuse error codes with exception messages.

An error code should remain stable even if the wording of the message changes.

For example:

Code: ORDER_ALREADY_PROCESSED

Old message:
The order was already processed.

New message:
This order has already been completed.

Clients should continue to recognize the same failure.

That is why a stable application error code is a more reliable integration point than a message string.

4. Keep Error Classification Independent

A common mistake is to assume that HTTP status, error category, and retryability are interchangeable.

They are not.

Consider these illustrative examples:

Failure	HTTP status	Retryable
Invalid request	400	No
Missing resource	404	No
Business conflict	409	Usually no
Rate limit exceeded	429	Depends on policy
Temporary dependency failure	503	Potentially

A 500-series status doesn't automatically mean an operation is safe to retry.

Likewise, a 409 response doesn't necessarily tell the consumer whether the conflict is a business rejection or a concurrency condition.

The error category and retryability flag should express their own meanings.

In NERV Exception, categories are represented as strings, allowing applications to define categories appropriate to their domains.

This provides flexibility, but it also means teams should establish conventions for naming and interpreting those categories.

5. Centralize HTTP Error Mapping

A consistent contract is useful only if applications produce it consistently.

Without centralized handling, individual controllers may return different shapes:

{
  "error": "Invalid order"
}

{
  "message": "Customer not found",
  "status": 404
}

{
  "success": false,
  "reason": "Payment rejected"
}

These variations force consumers to implement endpoint-specific error parsing.

Spring's @RestControllerAdvice provides a central place to translate exceptions into HTTP responses.

A reusable exception-handling library can take this further by standardizing:

Application error definitions
Exception-to-response conversion
Error response serialization
Structured diagnostic information
Spring Web integration

NERV Exception separates these responsibilities across modules rather than requiring every application to copy the same exception handler implementation.

The architectural benefit is consistency without duplicating infrastructure code across services.

6. Validation Errors Need Structured Details

Validation failures are different from most application exceptions.

A single request may violate multiple constraints.

For example:

{
  "name": "",
  "quantity": -5
}

A consumer benefits from knowing which fields failed and why.

A structured response might contain:

{
  "code": "VALIDATION_ERROR",
  "message": "Request validation failed",
  "status": 400,
  "retryable": false,
  "category": "CLIENT",
  "details": {
    "errors": {
      "name": "must not be blank",
      "quantity": "must be greater than or equal to 1"
    }
  }
}

This example illustrates the details.errors object-map structure used by NERV Exception's validation handling. The exact error code and messages depend on the application and configuration.

The important point is that validation information should be machine-readable.

Clients should not need to extract field names from a sentence.

For public APIs, validation details should also avoid exposing sensitive values or internal implementation information.

7. Microservice Error Propagation Is Not Just Serialization

Exception handling becomes more complicated when services call each other.

Consider:

API Gateway
    |
    v
Order Service
    |
    v
Payment Service

Payment Service returns:

{
  "code": "PAYMENT_ALREADY_PROCESSED",
  "message": "Payment has already been processed",
  "status": 409,
  "category": "BUSINESS",
  "retryable": false
}

What should Order Service do?

It could replace the response with a generic downstream error.

But that would discard useful business information.

Alternatively, Order Service could maintain a copy of every Payment Service error definition.

That creates tighter coupling between services.

A third approach is to use a shared response structure while allowing services to own their business error codes.

This is the approach explored by NERV Exception's Feign integration.

Known and unknown downstream errors

The Feign decoder distinguishes between error codes that are registered locally and those that are not.

For an unknown code, NERV Exception can represent the downstream definition through RemoteNervErrorCode.

That allows the application to retain meaningful information without requiring every remote business code to exist in a shared local catalog.

For a locally registered code, the local definition determines the resolved status, category, and retryability.

This distinction is important because propagation is not always transparent.

A receiving service can reinterpret an error when it has a matching local definition.

Also, the original downstream trace identifiers, request path, and timestamp are not automatically preserved in the regenerated HTTP response.

Those fields are populated from the receiving application's current context.

Therefore, a shared error contract improves interoperability, but it does not guarantee byte-for-byte preservation of downstream responses.

8. Retryable Does Not Mean Automatically Retried

A retryability flag is useful, but it can be misunderstood.

Consider a payment request that times out after the payment provider has already accepted the operation.

The caller cannot determine from the timeout alone whether the payment succeeded.

Retrying without an idempotency mechanism could produce a duplicate operation.

That means retry decisions must account for more than an error flag.

They must consider:

Whether the operation is idempotent
Whether the outcome is known
Retry limits and backoff
Dependency health
The risk of retry amplification

NERV Exception's Feign decoder can represent retryable downstream failures using Feign's RetryableException.

But that does not automatically install a retry policy.

Actual retry execution depends on the configured Feign retry mechanism.

Applications must also decide how retry exhaustion should be handled and how the final failure should be exposed.

Retry classification, retry execution, and business recovery are separate responsibilities.

Keeping them separate makes distributed systems easier to reason about.

9. Observability Needs More Than an Error Message

In production, a useful error response should help engineers locate the failure without revealing internal implementation details to API consumers.

For example:

{
  "code": "PAYMENT_PROVIDER_UNAVAILABLE",
  "message": "Payment processing is temporarily unavailable",
  "status": 503,
  "traceId": "abc123",
  "spanId": "def456"
}

The trace and span identifiers can help connect the client-visible failure to internal telemetry.

However, including tracing fields in a response does not automatically create distributed traces.

The application must have tracing infrastructure configured, and the relevant context must be available.

NERV Exception can read Micrometer tracing context when present.

It does not independently create tracing spans for every exception.

This distinction matters when integrating error handling with observability tools.

A response contract provides correlation information.

Tracing infrastructure provides the actual execution history.

10. Protect Internal Error Information

A production error handler must balance diagnostics with security.

Developers may want to expose exception causes, stack traces, SQL errors, or dependency details during development.

Public consumers generally should not receive that information.

For example, avoid exposing messages such as:

Connection refused to internal-db.company.local:5432

or internal exception stack traces.

Instead, return a safe message and retain detailed diagnostics in controlled logs and tracing systems.

One practical lesson from reviewing NERV Exception's Spring Boot integration is that effective configuration matters.

The starter's default properties in the reviewed 1.4.0 implementation enable internal message and cause exposure, although stack-trace exposure is disabled by default.

For public APIs, explicitly configure these settings to prevent unintended disclosure.

A reusable library can provide controls, but applications are responsible for selecting appropriate production settings.

11. What NERV Exception Provides

NERV Exception organizes exception-handling concerns into reusable modules.

The reviewed project includes:

Module	Responsibility
`nerv-exception-api`	Shared error contracts
`nerv-exception-core`	Core exception and error-code behavior
`nerv-exception-spring-web`	Spring Web exception handling
`nerv-exception-spring-feign`	Feign error decoding
`nerv-exception-spring-boot-starter`	Spring Boot integration and configuration
`nerv-exception-jpa`	Persistence-related integration
`nerv-exception-event`	Event-related error support
`nerv-exception-spring-kafka`	Kafka-related exception support

The objective is not to eliminate application-specific exception handling.

Applications still need to define their own error codes, business recovery behavior, security boundaries, and retry policies.

The library provides common infrastructure so teams can focus on those decisions rather than repeatedly implementing the same response contract.

12. Lessons for Production Systems

After exploring these concerns, several principles stand out.

Treat errors as part of your public API. Error codes and their meanings should be stable and documented.

Share contracts without unnecessarily sharing business ownership. Independent services should be able to introduce their own error codes.

Separate classification from recovery. A retryable error does not guarantee a safe retry.

Preserve useful context deliberately. Error propagation may transform metadata, status, and tracing fields.

Design for unknown codes. Consumers should remain functional when downstream services introduce new failures.

Review effective production configuration. Internal diagnostic information should not leak through public error responses.

Most importantly, recognize that consistent exception handling is not simply about formatting JSON.

It's about giving consumers reliable information while allowing applications and services to evolve independently.

Final Thoughts

A global exception handler is a useful starting point.

But production systems need more than centralized try/catch behavior.

They need stable error identifiers, meaningful classifications, structured validation responses, controlled diagnostics, and deliberate error propagation across service boundaries.

These are the concerns that shaped NERV Exception.

The goal isn't to hide every failure behind a uniform response.

It's to make failures understandable, actionable, and consistent without taking away each application's responsibility for its own behavior.

A good error contract standardizes communication without pretending every failure has the same meaning.

Explore NERV Exception

NERV Exception is part of NERV — Next-Generation Engineering for Runtime Velocity, a collection of open-source Java and Spring Boot libraries focused on production-oriented engineering.

Explore the projects:

https://github.com/czetsuyatech

For the canonical version of this article and other engineering articles, visit:

https://www.czetsuyatech.com

This article is part of the NERV Exception technical series. Future articles explore error contracts, service boundaries, error taxonomies, Feign propagation, retries, and observability in more depth.

A Trading Bot Is Easy. A Trading Engine Is Not

Ed Legaspi — Sat, 17 Oct 2026 04:34:15 +0000

Building a trading bot is surprisingly straightforward.

Get some market data. Calculate a few indicators. Generate a BUY or SELL signal.

Maybe connect it to an exchange.

Congratulations. You have a trading bot.

But here's the question that changed how I approached NERV Trade, my Java-based trading engine:

What happens after the signal?

What if the broker accepts your order but never responds?

What if an order is partially filled?

What if the same execution arrives twice?

What if the application crashes while updating a position?

And what happens when the application restarts while orders are still active?

These aren't hypothetical edge cases you can safely ignore when building an automated trading system.

They're architectural problems.

And solving them is what separates a simple trading bot from a reliable trading engine.

1. The Trading Bot Everyone Starts With

Most trading projects begin with an architecture similar to this:

Market Data
     |
     v
Technical Indicators
     |
     v
Trading Strategy
     |
     v
BUY / SELL / HOLD

For example, imagine a simple Java strategy:

public enum Signal {
    BUY,
    SELL,
    HOLD
}

public Signal evaluate(
        BigDecimal fastMovingAverage,
        BigDecimal slowMovingAverage) {

    int comparison =
            fastMovingAverage.compareTo(slowMovingAverage);

    if (comparison > 0) {
        return Signal.BUY;
    }

    if (comparison < 0) {
        return Signal.SELL;
    }

    return Signal.HOLD;
}

This is a simplified illustration, not NERV Trade's actual strategy implementation.

It demonstrates the basic idea.

Given market observations, produce a trading signal.

You can improve the strategy by introducing:

Multiple indicators
Multi-timeframe confirmation
Volatility analysis
Market regime detection
Risk-adjusted signals

But regardless of how sophisticated the analysis becomes, the output is still fundamentally a decision.

BUY
SELL
HOLD

And a decision isn't an execution.

That's where the interesting engineering begins.

2. A BUY Signal Is Not an Order

Suppose our strategy produces:

BUY BTCUSDT

What does that mean?

Does the strategy want to buy Bitcoin?

Has an order been created?

Has the exchange accepted the order?

Has the order been filled?

These are different states.

A more realistic architecture separates them:

Market Data
     |
     v
Analysis
     |
     v
Strategy
     |
     v
Risk Evaluation
     |
     v
Trade Intent
     |
     v
Local Order
     |
     v
Broker Submission
     |
     v
Execution Reports

Each component has a specific responsibility.

Component	Responsibility
Analysis	Interpret market conditions
Strategy	Decide what action to propose
Risk	Determine whether the action is permitted
Trade Intent	Own the trading lifecycle
Local Order	Represent the requested broker action
Broker Adapter	Communicate with the external venue
Execution Processor	Accept and account for actual fills

This separation is important because a trading decision can lead to multiple orders.

And one order can produce multiple executions.

A strategy should not need to understand those execution details.

Its responsibility is to express an intention.

The execution engine is responsible for determining what actually happened.

3. An Accepted Order Is Not an Executed Order

Consider an order to buy 10 units.

The simplest implementation assumes:

BUY 10
   |
   v
FILLED 10

But a broker might produce:

BUY 10
   |
   v
ACKNOWLEDGED
   |
   v
PARTIALLY FILLED: 3
   |
   v
PARTIALLY FILLED: 7
   |
   v
FILLED: 10

The order has a lifecycle.

Its state changes over time.

A simplified representation might look like:

public enum OrderStatus {
    CREATED,
    SUBMITTED,
    ACKNOWLEDGED,
    PARTIALLY_FILLED,
    FILLED,
    CANCELLED,
    REJECTED
}

Again, this is an illustrative model rather than NERV Trade's exact domain API.

The important distinction is that an order's status is not the same thing as its economic effect.

An order might be acknowledged without any execution.

It might be partially filled and later cancelled.

It might be fully filled through several independent executions.

The engine must track both:

What the broker says about the order

and

What executions have actually occurred.

Those are related but different responsibilities.

4. Partial Fills Introduce Accounting

Let's take the previous example.

Order: BUY 10

Execution 1: BUY 3 @ 100
Execution 2: BUY 4 @ 101
Execution 3: BUY 3 @ 102

The engine cannot simply record:

BUY 10 @ 100

That would misrepresent the actual execution.

Instead, it needs to account for each fill.

For this example:

Total quantity = 10

Total cost =
    (3 × 100)
  + (4 × 101)
  + (3 × 102)

Total cost = 1,010

Average price = 101

And that is before considering fees.

In Java, the arithmetic might look like:

BigDecimal totalCost = executions.stream()
        .map(execution ->
                execution.price()
                        .multiply(execution.quantity()))
        .reduce(BigDecimal.ZERO, BigDecimal::add);

The arithmetic is easy.

The lifecycle surrounding it is not.

What happens if an execution is delivered twice?

What if the order is cancelled after a partial fill?

What if the application crashes before the accounting is committed?

What if reconciliation discovers an execution that was never received through the live stream?

These questions introduce a requirement that extends far beyond calculating an average execution price.

Execution accounting must remain correct under duplicate delivery, failure, and recovery.

5. The Same Execution Can Arrive More Than Once

This is where trading systems begin to resemble other distributed systems.

Imagine the broker reports:

Execution ID: 12345
BUY 0.001 BTC

The application processes the fill.

Its exposure increases by 0.001 BTC.

Then the connection drops.

After reconnecting, the application receives the same execution again.

Or perhaps it discovers that execution through a reconciliation request.

If the engine processes it again:

Actual purchased: 0.001 BTC

Recorded exposure: 0.002 BTC

The internal state is now wrong.

This is why execution identity matters.

A simplified execution-processing contract might be:

public interface ExecutionProcessor {

    ExecutionResult accept(ExecutionReport report);
}

The implementation must determine whether the execution has already been accepted.

Conceptually:

Receive Execution
        |
        v
Check Durable Identity
        |
   +----+----+
   |         |
   v         v
Already     New
Accepted    Execution
   |         |
   v         v
No New      Apply Accounting
Effect      Atomically

The word durable is important.

An in-memory Set of execution IDs might prevent duplicates while the application is running.

But it doesn't survive a restart.

And a restart is precisely when previously observed executions may be encountered again.

In NERV Trade, accepted execution identities are persisted so duplicate delivery does not create duplicate economic effects.

The principle is straightforward:

The same execution can be observed multiple times, but its accounting effects must be applied only once.

We'll explore this problem in much greater detail later in the series.

6. The Most Dangerous Failure Is an Unknown Outcome

Consider a common backend pattern:

try {
    broker.submit(order);
} catch (Exception exception) {
    retry(order);
}

For ordinary read operations, retrying after a transient failure can be reasonable.

For order submission, this pattern can be dangerous.

Imagine the following sequence:

Trading Engine                 Broker
      |                           |
      |------ Submit BUY -------->|
      |                           |
      |                     Order Accepted
      |                           |
      X                           |
 Connection Lost                 |

The broker accepted the order.

But the engine never received the response.

From the application's perspective, the request failed.

From the broker's perspective, the order exists.

If the engine blindly retries with a new identity, it may create another order.

Now the account has more exposure than intended.

This is an example of an uncertain external side effect.

The system knows that a request was attempted.

It does not know whether the external operation succeeded.

A safer conceptual flow is:

Submit Order
      |
      v
Response Received?
      |
  +---+---+
  |       |
 YES      NO
  |       |
  v       v
Record   Preserve Uncertain
Result   Submission State
              |
              v
       Reconcile With Broker
              |
              v
       Establish Actual Outcome

The engine needs stable order correlation and a recovery mechanism.

It must not assume that a missing response means the broker rejected the request.

This is one of the architectural challenges I found particularly interesting because it closely resembles problems in payment gateways.

A payment request might succeed even when the client times out.

A trading order might be accepted even when the application loses its connection.

In both cases, blindly repeating the operation can be worse than temporarily preserving uncertainty.

7. The Database Becomes Part of the Execution Engine

At some point, the application must survive a restart.

That means persisting state.

But saving an order isn't enough.

When an execution is accepted, several related records may need to change:

Execution Identity
       +
Execution Details
       +
Order Accounting
       +
Trade Exposure
       +
Cost Basis
       +
Realized P&L
       +
Lifecycle State
       +
Journal Outcome

These changes represent one logical economic operation.

Consider what happens if the application crashes halfway through processing.

Execution recorded: YES

Exposure updated: NO

Now the database contains an execution that isn't reflected in the position.

The opposite failure is equally problematic:

Execution recorded: NO

Exposure updated: YES

After restarting, the engine may accept the same execution again.

NERV Trade eventually moved toward atomic execution processing using PostgreSQL transactions.

The requirement is:

Either all accounting effects of an accepted execution commit, or none of them do.

This is more than a database implementation detail.

It affects how the domain model and execution-processing boundaries are designed.

A transaction must encompass the complete state transition.

Otherwise, persistence can preserve an inconsistent result perfectly.

8. Restart Recovery Changes the Definition of State

Suppose the application has been running for several hours.

It has:

Active trade intents
Submitted orders
Partially filled orders
Accepted executions
Open exposure
Completed lifecycles

Then the process stops.

When it restarts, it cannot simply create a new empty state.

The broker still knows about its orders.

The account still has its assets.

And the market may have continued executing orders while the application was offline.

The engine needs to reconstruct its durable state and reconcile unresolved activity with the broker.

This means recovering more than the latest aggregate position.

It needs to understand which trade intents own the exposure, which executions have already been accepted, and which orders remain unresolved.

A restart must not turn an existing trade into a new trade.

And recovery must not silently invent an outcome that hasn't been confirmed.

This is why recovery became an architectural responsibility in NERV Trade rather than just a startup routine.

9. Market Data Has Similar Problems

Execution isn't the only place where a simple design becomes complicated.

Market data introduces its own consistency requirements.

Suppose a strategy analyzes five-minute candles.

M5

Then we introduce higher-timeframe context:

M5
M15
H1
H4

Now the engine must answer questions such as:

Which higher-timeframe candles were available at a particular decision time?
Has the current candle closed?
Can a partially formed candle be used?
What happens when market-data events arrive late?
How are missing intervals detected?
Can the same historical input reproduce the same analysis?

The most dangerous mistake is accidentally using information that wasn't available when a historical decision was made.

That creates look-ahead bias.

A backtest might look excellent while relying on future information.

For NERV Trade, deterministic analysis and multi-timeframe consistency became important foundations.

We'll cover both in dedicated articles.

10. What the Architecture Eventually Became

The original idea was simple:

Market Data
     |
     v
Analysis
     |
     v
BUY / SELL

But as the project evolved, each newly discovered problem introduced another responsibility.

The resulting architecture became closer to:

                    Market Data
                         |
                         v
                Multi-Timeframe State
                         |
                         v
                      Analysis
                         |
                         v
                      Strategy
                         |
                         v
                        Risk
                         |
                         v
                    Trade Intent
                         |
                         v
                   Order Lifecycle
                         |
                         v
                      BrokerPort
                         |
              +----------+----------+
              |                     |
              v                     v
         Simulator             Binance Spot
              |                     |
              +----------+----------+
                         |
                         v
                  Execution Reports
                         |
                         v
                 Execution Accounting
                         |
                         v
                     PostgreSQL
                         |
                         v
                Recovery / Reconciliation

These components weren't introduced simply to make the system more sophisticated.

They appeared because the earlier design couldn't reliably answer increasingly difficult questions.

That distinction matters.

Complexity is not an achievement by itself.

Every architectural component should exist because it protects an important responsibility or invariant.

11. A Trading Bot Makes Decisions. A Trading Engine Owns Their Consequences.

I don't think there's a universally useful definition that separates every trading bot from every trading engine.

But for NERV Trade, the distinction became clear.

A trading bot primarily answers:

What should I trade?

A trading engine must also answer:

What happened?

What do I currently own?

Which strategy owns that exposure?

What has already been processed?

What remains uncertain?

Can I recover after a failure?

Does my internal state still agree with the broker?

These questions shaped the engineering direction of NERV Trade.

And they changed how I measured progress.

Early progress meant generating signals and improving analysis.

Later progress meant proving that:

Duplicate executions don't duplicate exposure.
Partial fills produce correct accounting.
Failed transactions don't leave half-committed state.
Uncertain submissions aren't blindly repeated.
Restart recovery preserves lifecycle identity.
Broker reconciliation doesn't invent new trades.

Those improvements aren't always visible on a trading chart.

But they're fundamental to building a system that can operate reliably.

12. Before Everything Else, We Needed Determinism

There's one more important part of this story.

Before implementing broker adapters, persistence, or restart recovery, NERV Trade needed to answer a more fundamental question.

Suppose we provide the same sequence of market data twice.

Should the engine produce the same analysis and trading decision?

It sounds obvious.

But timestamps, mutable state, event ordering, candle boundaries, and partially formed market data can introduce subtle differences.

If analysis isn't reproducible, debugging everything downstream becomes much harder.

A strategy might produce different decisions during replay.

A backtest might accidentally use information from the future.

And a failure might be impossible to reproduce.

That led to one of the earliest design principles behind NERV Trade:

Determinism.

Before worrying about whether the engine could reliably execute a decision, we needed confidence that it could reliably reproduce one.

And that's where the engineering journey really begins.

What's Next?

This article is Part 2 of the NERV Trade engineering series.

The series documents the real development journey behind building a Java trading engine, including architectural decisions, failures, testing, persistence, broker integration, and recovery.

Previous article:

I Built a Trading Engine in Java — Here's What It Took

Next article:

Deterministic Trading: Why the Same Market Data Must Produce the Same Decision

We'll explore why deterministic analysis matters, how reproducibility affects trading architecture, and why reliable market-data processing must come before execution.

About NERV Trade

NERV Trade is part of NERV — Next-Generation Engineering for Runtime Velocity, an open-source engineering initiative focused on building reusable Java and Spring-based systems.

The project explores:

Deterministic market analysis
Multi-timeframe trading
Strategy and risk boundaries
Trade lifecycle management
Execution accounting
PostgreSQL persistence
Broker integration
Recovery and reconciliation

This article is adapted from the original CzetsuyaTech publication:

A Trading Bot Is Easy. A Trading Engine Is Not.

This series focuses on software engineering and trading infrastructure, not financial advice or claims of trading profitability.

Building a Trading Engine in Java: Beyond BUY and SELL Signals

Ed Legaspi — Sat, 17 Oct 2026 04:11:34 +0000

Generating a BUY or SELL signal is relatively straightforward.

Building a system that can reliably execute that signal is a different engineering problem.

When I started developing NERV Trade, I was primarily interested in building a trading system capable of analyzing market conditions across multiple timeframes.

The initial idea was simple:

Market Data
    |
    v
Technical Analysis
    |
    v
Trading Strategy
    |
    v
BUY / SELL / HOLD

But as development progressed, the questions became increasingly difficult.

What happens when an order is only partially filled?

What if the broker sends the same execution twice?

What if the application crashes after processing an execution but before persisting the result?

What if an order reaches the broker, but the connection fails before the application receives an acknowledgment?

And what happens when the application restarts while orders are still active?

These questions pushed NERV Trade beyond signal generation.

It became an exercise in building a durable, broker-neutral trading engine using Java, Spring Boot, and PostgreSQL.

This article introduces the architecture, the engineering problems we encountered, and the decisions that shaped the system.

1. The Architecture We Ended Up Building

NERV Trade now has several distinct responsibilities.

                     MARKET DATA
                          |
                          v
                Multi-Timeframe Analysis
                  M5 / M15 / H1 / H4
                          |
                          v
                       Analysis
                          |
                          v
                       Strategy
                          |
                          v
                         Risk
                          |
                          v
                     Trade Intent
                          |
                          v
                    Order Lifecycle
                          |
                          v
                      BrokerPort
                          |
                +---------+---------+
                |                   |
                v                   v
           Simulator          Binance Spot
                |                   |
                +---------+---------+
                          |
                          v
                  Execution Reports
                          |
                          v
                 Execution Accounting
                          |
                          v
                    PostgreSQL
                          |
               +----------+----------+
               |          |          |
               v          v          v
            Orders     Positions    Journal
                          |
                          v
                 Recovery / Reconciliation

The important design decision is that market analysis, trading decisions, execution, and persistence are separate responsibilities.

A strategy should not need to understand Binance REST endpoints.

A broker adapter should not calculate technical indicators.

And an execution report should not directly mutate arbitrary trading state without passing through the engine's lifecycle rules.

These boundaries became increasingly important as the system evolved.

2. A Trading Signal Is Not a Trade

One of the earliest architectural realizations was that a trading signal and a trade are fundamentally different concepts.

Consider:

BUY BTCUSDT

This represents a trading decision.

It does not mean an order was submitted.

It does not mean the broker accepted the order.

And it certainly does not mean an execution occurred.

In NERV Trade, the lifecycle is closer to:

Strategy Decision
       |
       v
Trading Signal
       |
       v
Trade Intent
       |
       v
Local Order
       |
       v
Broker Order
       |
       v
Execution / Fill
       |
       v
Exposure and Accounting
       |
       v
Lifecycle Completion

Each stage answers a different question.

Concept	Responsibility
Strategy decision	What does the strategy want to do?
Trading signal	What action is being proposed?
Trade intent	Which trading lifecycle owns this decision?
Local order	What order did NERV create?
Broker order	What order does the broker recognize?
Execution	What quantity actually traded?
Exposure	What quantity remains owned by the intent?
Lifecycle	Is the trading operation finished?

Why introduce so many concepts?

Because once execution becomes asynchronous, a single BUY signal can produce multiple orders and multiple executions.

Without clear ownership, accounting and recovery quickly become difficult.

3. A Position Is Not a Trade Either

Another important distinction emerged when implementing trade lifecycle management.

Suppose two strategies independently generate long exposure in the same instrument.

Strategy A
    |
    v
Intent A
    |
    v
BUY 5

Strategy B
    |
    v
Intent B
    |
    v
BUY 3

The aggregate position is:

LONG 8

But there are still two independent trading lifecycles.

If Strategy A exits, it should not automatically close Strategy B's exposure.

This led to an important design principle:

A trade intent owns its exposure. The aggregate position is a derived view.

That distinction also matters when handling reversals.

For example:

Intent A
    |
    +-- BUY 5
    |
    +-- SELL 3
    |
    +-- Remaining exposure: LONG 2

The lifecycle cannot simply be considered complete because an order reached a terminal state.

In NERV Trade, lifecycle completion requires:

The intent's exposure is flat.
All associated orders are terminal.

Both conditions matter.

An intent with zero exposure but an outstanding order may still produce another execution.

4. Partial Fills Changed Everything

A common simplification in trading examples is to assume that an order executes completely.

BUY 10
    |
    v
FILLED 10

Real execution is more complicated.

An order for 10 units might generate:

Order: BUY 10

Execution 1: BUY 3
Execution 2: BUY 4
Execution 3: BUY 3

Total filled: 10

Now the engine must account for each execution independently.

For every accepted execution, the system needs to update the appropriate lifecycle state.

That can include:

Order filled quantity
Remaining order quantity
Intent-owned exposure
Cost basis
Realized profit and loss
Lifecycle completion
Trading journal outcome

And those updates must remain consistent.

This becomes particularly interesting when executions arrive asynchronously or are discovered later through reconciliation.

The engine cannot assume that receiving an order acknowledgment means the order has been completely executed.

Acknowledgment and execution are different events.

5. What Happens When the Same Fill Arrives Twice?

Distributed systems frequently deliver information more than once.

Trading infrastructure is no exception.

An execution might arrive through a live stream.

Later, the application might discover the same execution while reconciling with the broker.

After a restart, the same execution might be encountered again.

Consider:

Execution ID: 98765
Symbol: BTCUSDT
Side: BUY
Quantity: 0.001

The engine processes the execution.

Its exposure increases.

Then the same execution arrives again.

If the engine processes it a second time, its internal position becomes incorrect.

Actual execution:

BUY 0.001 BTC

Incorrect internal accounting:

BUY 0.002 BTC

This is not merely a duplicate notification problem.

It is an accounting correctness problem.

NERV Trade therefore uses durable execution identities to prevent duplicate economic effects.

The execution identity must remain stable across:

Live WebSocket Delivery
          |
          v
Application Reconnect
          |
          v
Broker Reconciliation
          |
          v
Application Restart

A reconnect should not transform an existing execution into a new one.

The broader principle is:

Delivery may happen more than once. Accepted economic effects must not.

This is closely related to idempotency problems found in payment processing, messaging systems, and distributed transaction workflows.

6. The Most Dangerous Order Is the One You're Unsure About

Consider this sequence:

NERV Trade                  Broker
     |                         |
     |------ BUY ORDER ------->|
     |                         |
     |                    Order Accepted
     |                         |
     X                         |
 Connection Lost               |

The broker accepted the order.

But the application never received the acknowledgment.

What should happen next?

A naive implementation might retry the request.

retryOrderSubmission();

That is potentially dangerous.

If the first order succeeded, a second submission could create additional exposure.

The application cannot safely assume:

No response = No order

Instead, the system needs to represent uncertainty explicitly.

NERV Trade uses stable order correlation and conservative recovery rules.

Conceptually:

Submit Order
      |
      v
Response Received?
      |
   +--+--+
   |     |
  YES    NO
   |     |
   v     v
Record  Mark Submission
Result  Uncertain
           |
           v
     Reconcile With Broker
           |
           v
     Establish Actual State

The important rule is:

Do not blindly resubmit an order whose outcome is unknown.

First establish what happened.

This is one of the places where trading architecture starts resembling reliable payment processing.

Both involve external side effects that cannot necessarily be rolled back.

7. Persistence Is More Than Saving Orders

Eventually, NERV Trade needed to survive application restarts.

That meant moving lifecycle state into durable persistence.

But simply storing orders was insufficient.

Consider what happens when an execution is accepted.

The system may need to persist:

Execution Identity
       +
Execution Payload
       +
Order Accounting
       +
Intent Exposure
       +
Cost Basis
       +
Realized P&L
       +
Lifecycle Completion
       +
Journal Outcome

These changes represent one logical economic operation.

If only some of them commit, the database can contain contradictory information.

For example:

Execution recorded: YES

Position updated: NO

Now the engine knows the execution happened but its exposure does not reflect it.

The opposite is equally dangerous.

Execution recorded: NO

Position updated: YES

After a restart, the same execution might be accepted again.

NERV Trade addresses this by committing accepted execution accounting atomically using PostgreSQL transactions.

The persistence layer uses:

Spring JDBC
PostgreSQL
Flyway migrations
Database uniqueness constraints
Intent-level locking
Durable execution identities

The domain model remains independent of database annotations.

This allows the core lifecycle logic to remain separate from its persistence implementation.

The goal is not simply to store state.

It is to ensure that the database never commits half of an accepted execution.

8. Restart Recovery Is an Architectural Feature

Imagine an application running with active orders and nonzero exposure.

Then the process stops.

When it starts again, it must recover more than the latest aggregate position.

It needs to understand:

Which trade intents remain active
Which orders belong to those intents
Which executions have already been accepted
Which orders are terminal
Which submissions remain uncertain
What exposure belongs to each intent
Whether additional broker reconciliation is necessary

The engine cannot treat a restart as the beginning of a new trading session with no history.

The broker still has its orders.

The account still has its assets.

And previously accepted executions still matter.

Recovery must therefore reconstruct the engine's durable state and reconcile unresolved external activity.

This is why persistence and recovery became fundamental parts of the architecture rather than optional infrastructure.

9. Why We Built the Simulator First

Before integrating a real exchange, NERV Trade used a deterministic simulator.

Initially, a simulator might seem like a convenient way to test trading strategies.

But it became something more important.

It allowed us to validate the execution engine itself.

For example:

Strategy
    |
    v
Trade Intent
    |
    v
BrokerPort
    |
    v
Deterministic Simulator
    |
    v
ExecutionReportPort
    |
    v
Durable Lifecycle

The simulator exercises the same conceptual execution boundaries that an external broker uses.

That makes it possible to test:

Partial fills
Multiple executions
Duplicate execution delivery
Order lifecycle transitions
Reversals
Intent-level exposure
Realized P&L
Failure handling
Restart recovery

The simulator also provides something a live exchange cannot easily guarantee:

Reproducible execution scenarios.

When a test fails, we can replay the same conditions.

When an invariant is violated, we can isolate the sequence that caused it.

This became especially valuable while hardening the lifecycle.

The simulator was no longer just a trading-strategy testing tool.

It became a controlled environment for validating execution correctness.

10. Then We Connected It to Binance

Once the lifecycle and persistence architecture had matured, we introduced a Binance Spot Testnet adapter.

The important requirement was that Binance should remain an adapter.

It should not become the owner of NERV Trade's business logic.

                  NERV Trade Core
                        |
                        v
                    BrokerPort
                        |
             +----------+----------+
             |                     |
             v                     v
        Simulator             Binance Spot
                                   |
                                   v
                              REST / WS

The adapter handles exchange-specific concerns such as:

Authentication and request signing
Exchange symbol filters
Order submission
Broker order correlation
Execution event mapping
Historical candle retrieval
Live market-data streams
Reconnection and gap recovery
Broker reconciliation

The engine continues to own:

Analysis
Strategy decisions
Risk decisions
Trade intents
Execution accounting
Lifecycle state
Persistence
Recovery rules

This separation is particularly important because Binance Spot has different trading semantics from a general-purpose simulator.

For example, ordinary Spot trading cannot be treated as if it supports unrestricted short exposure.

The adapter must respect the capabilities of the venue without redefining the engine's core domain model.

11. What Did We Actually Verify?

At the initial Binance Spot Testnet adapter milestone, NERV Trade reached:

601 passing tests, with zero failures, errors, or skips.

The verification included the existing deterministic simulator, lifecycle and persistence tests, along with the new adapter test coverage.

The packaged simulator also continued to pass its deterministic execution scenario.

However, there is an important distinction.

Those 601 passing tests did not mean we had already proven every assumption against the real Binance Testnet.

At that milestone, live exchange verification had not yet been executed.

The adapter was implemented and locally verified.

Actual exchange behavior still needed to be validated separately.

That distinction matters.

A mocked broker can prove that your application handles the behavior you modeled.

It cannot prove that you modeled every relevant behavior of the real broker correctly.

12. What Building NERV Trade Taught Me

The biggest lesson wasn't about finding better trading signals.

It was about separating different kinds of correctness.

Analysis correctness

Does the engine interpret market data consistently and without using information from the future?

Decision correctness

Does the strategy produce decisions based on the intended analysis and risk constraints?

Execution correctness

Does the system accurately represent orders, acknowledgments, fills, and lifecycle transitions?

Accounting correctness

Does every accepted execution produce exactly one set of economic effects?

Recovery correctness

Can the application restart without forgetting what it previously owned or processed?

Integration correctness

Does the broker adapter preserve the semantics of the external venue without contaminating the core architecture?

These are different problems.

And each deserves its own design and verification strategy.

A profitable backtest cannot establish execution correctness.

A successful API request cannot establish accounting correctness.

And hundreds of passing unit tests cannot, by themselves, establish real-exchange correctness.

13. Where the Engineering Journey Goes Next

NERV Trade didn't begin with this architecture.

It evolved as increasingly difficult problems exposed weaknesses in the previous design.

Each one forced another architectural decision:

Deterministic Analysis
          |
          v
Multi-Timeframe Context
          |
          v
Strategy and Risk Boundaries
          |
          v
Trade Intent Ownership
          |
          v
Order and Execution Lifecycle
          |
          v
Execution Idempotency
          |
          v
Atomic Persistence
          |
          v
Restart Recovery
          |
          v
Broker Integration
          |
          v
Reconciliation and Safety

And that is what this series will document.

Not a collection of trading indicators.

Not a promise of profitable strategies.

And not another tutorial that stops after printing a BUY signal.

Instead, we'll explore the engineering decisions involved in building a trading engine that can reason about its own state, execute through different brokers, and recover when things go wrong.

We'll start at the beginning.

With a deceptively simple question:

How difficult can it really be to build a trading bot?

About NERV Trade

NERV Trade is part of NERV — Next-Generation Engineering for Runtime Velocity, an engineering initiative focused on building reusable Java and Spring-based systems.

The project explores trading architecture, deterministic analysis, execution lifecycle management, durable persistence, broker integration, and operational safety.

This article is adapted from the original CzetsuyaTech publication:

I Built a Trading Engine in Java — Here's What It Took

Continue the Series

Next: A Trading Bot Is Easy. A Trading Engine Is Not.

The next article rewinds to the beginning and explores how a simple signal-generation idea evolved into a much larger engineering problem.

This series focuses on software engineering and trading infrastructure, not financial advice or claims of trading profitability.

Building Long-Running Payment Workflows with Spring Boot and NERV Event

Ed Legaspi — Sun, 04 Oct 2026 03:23:31 +0000

Building Long-Running Payment Workflows with Spring Boot and NERV Event

Some payment operations finish in milliseconds.

Others don't.

Imagine a payment service that sends a request to an external payment provider.

Normally, the provider responds quickly.

But sometimes that operation can take several minutes — perhaps even 30 minutes.

The obvious implementation might look something like this:

@Transactional
public Payment processPayment(PaymentRequest request) {

    Payment payment = createPayment(request);

    GatewayResult result = paymentGateway.validate(payment);

    payment.complete(result);

    return payment;
}

It looks safe.

The method is transactional.

If something fails, Spring rolls everything back.

But there is a problem hiding inside that transaction.

What happens if paymentGateway.validate() takes 30 minutes?

Your business workflow is now long-running.

And potentially, so is your database transaction.

That is usually not what we want.

The solution isn't simply increasing the timeout.

The better question is:

Should the lifetime of a database transaction really be the same as the lifetime of the business workflow?

For distributed payment workflows, the answer is usually no.

A 30-Minute Workflow Shouldn't Mean a 30-Minute Transaction

Consider what our original implementation is effectively doing:

BEGIN TRANSACTION

    Create Payment

    Call External Payment Gateway

    Wait...

    Wait...

    Wait...

    30 minutes later...

    Update Payment

COMMIT

During those 30 minutes, the application may be holding database resources while doing no database work at all.

Depending on the transaction, this can contribute to:

long-held database connections
connection-pool pressure
locks being held longer than necessary
long-running MVCC snapshots
increased contention
harder failure recovery

But there is an even more fundamental problem.

The external payment provider does not participate in your database transaction anyway.

Rolling back PostgreSQL cannot undo an external payment request that has already been accepted.

So what exactly are we protecting by keeping the transaction open?

This is where the architecture needs to change.

Separate the Workflow from the Transaction

A long-running business workflow does not need to be implemented as one long-running database transaction.

Instead, we can divide it into short, durable steps.

The first transaction creates the payment and records what needs to happen next.

┌───────────────────────────────┐
│       Transaction #1          │
│                               │
│   Create Payment              │
│          +                    │
│   Create Outbox Event         │
│                               │
└──────────────┬────────────────┘
               │
             COMMIT
               │
               ▼
         Payment exists

The payment and the event are stored atomically.

If the transaction rolls back:

Payment:       NOT CREATED
Outbox Event:  NOT CREATED

If it commits:

Payment:       CREATED
Outbox Event:  CREATED

We now have something extremely useful:

a durable record of what needs to happen next.

Commit Before Crossing the Network

This leads to a useful rule for long-running workflows:

Commit local state before performing slow remote work whenever the business semantics allow it.

Instead of:

BEGIN TX
   │
   ├── Save Payment
   │
   ├── Call External System
   │       │
   │       └── wait 30 minutes
   │
   └── Update Payment
COMMIT

we move toward:

TX #1
   │
   ├── Save Payment
   ├── Save Outbox Event
   │
COMMIT
   │
   ▼
External Work
   │
   │ possibly 30 minutes
   ▼
TX #2
   │
   ├── Update Payment
   ├── Save Next Event
   │
COMMIT

The business workflow can still take 30 minutes.

The individual database transactions do not.

That distinction is critical.

Let the Event Move the Workflow Forward

After the first transaction commits, NERV Event can dispatch the Outbox event.

The architecture becomes:

Client
  │
  ▼
Payment Service
  │
  │ Transaction #1
  ▼
Payment + Outbox
  │
  │ COMMIT
  ▼
Outbox Dispatcher
  │
  ▼
Message Broker
  │
  ▼
Payment Gateway Service
  │
  ▼
External Payment Provider

The payment might now have an explicit state such as:

PENDING_VALIDATION

That state tells us:

The payment exists, but the workflow has not finished.

This is an important architectural shift.

We stop using an open transaction to represent progress.

Instead, we use durable business state.

Long-Running External Work Does Not Need a Database Transaction

Suppose the Payment Gateway Service now needs to call the external provider.

If this operation can take a long time, we don't want an unnecessary database transaction surrounding it.

In Spring, one option is to make that boundary explicit:

@Transactional(propagation = Propagation.NOT_SUPPORTED)
public GatewayResult validate(PaymentRequest payment) {
    return externalGateway.validate(payment);
}

NOT_SUPPORTED means the method should execute without an active transaction.

If a transaction exists, Spring suspends it while this method executes.

The distinction is important:

Short Database Work
        ≠
Long Network Operation

The application may still have an HTTP request waiting.

A thread may still be occupied depending on the implementation.

The external operation may still take 30 minutes.

But we do not need to keep a database transaction open simply because the workflow is still running.

When the Provider Responds, Start Another Short Transaction

Eventually, the external provider returns a result.

Now we need another local transaction.

For example:

┌───────────────────────────────┐
│       Transaction #2          │
│                               │
│   Update Gateway State        │
│          +                    │
│   Create Result Event         │
│                               │
└──────────────┬────────────────┘
               │
             COMMIT

The complete flow now looks like this:

TX #1
Create Payment
Create Outbox Event
      │
      ▼
    COMMIT
      │
      ▼
External Gateway Call
NO DATABASE TRANSACTION
      │
      │ possibly 30 minutes
      ▼
Gateway Response
      │
      ▼
TX #2
Update State
Create Outbox Event
      │
      ▼
    COMMIT

The transaction lifetime now reflects the database work.

The workflow lifetime reflects the business process.

They no longer need to be the same.

But Now We Can't Roll Everything Back

Correct.

And that's one of the most important realizations when designing distributed workflows.

Suppose this happens:

Payment created
      │
      ▼
Transaction committed
      │
      ▼
External provider unavailable

We cannot roll back the original payment transaction.

It already happened.

But even if we had kept a local database transaction open, rolling it back would not necessarily undo something that had already happened in another system.

The idea of one giant rollback is often an illusion once a workflow crosses service boundaries.

Instead, we need to ask different questions:

What has already happened?
What is the current business state?
What still needs to happen?
Can the failed step be retried?
Is retrying safe?
Does compensation need to occur?

Those are workflow questions.

Not database transaction questions.

Make Partial Progress Explicit

A payment workflow might use states such as:

CREATED
   │
   ▼
PENDING_VALIDATION
   │
   ▼
VALIDATING
   │
   ├───────────────► COMPLETED
   │
   └───────────────► FAILED

The exact state machine depends on the payment domain.

The important part is that partial progress becomes explicit and durable.

Instead of the workflow existing primarily inside a Java call stack:

methodA()
   │
   ▼
methodB()
   │
   ▼
methodC()

it exists in persisted state:

Payment State
     +
Outbox State
     +
Inbox State

A JVM restart no longer erases our understanding of where the workflow is.

The Inbox Protects the Receiving Side

So far, we've concentrated mostly on publishing.

But reliable delivery is only half the problem.

Suppose the Payment Gateway Service receives:

PaymentValidationRequested

The broker may deliver that message more than once.

This is normal in an at-least-once delivery architecture.

Without protection, the consumer might execute the same operation multiple times.

NERV Event uses the Inbox Pattern on the receiving side.

Conceptually:

Broker
   │
   ▼
Inbox
   │
   ▼
Handler
   │
   ▼
Business Operation

The Inbox gives us a durable record of received messages.

It can support:

deduplication
idempotency
processing status
retry tracking
failure inspection

But there is an important detail.

The Inbox is not merely a table of message IDs.

Its transaction boundary matters.

Inbox + Business State + Outbox

Consider a consumer that receives one event and produces another.

For example:

PaymentValidationRequested
           │
           ▼
   Validate Payment
           │
           ▼
PaymentValidationCompleted

The local transaction may need to include three things:

┌──────────────────────────────────┐
│        Database Transaction      │
│                                  │
│  Inbox Processing State          │
│           +                      │
│  Business State                  │
│           +                      │
│  Outbox Event                    │
│                                  │
└────────────────┬─────────────────┘
                 │
               COMMIT

This creates a powerful reliability boundary.

The Inbox protects the incoming side.

The business transaction protects local state.

The Outbox protects the outgoing side.

Or more simply:

Inbox
  │
  ▼
Business State
  │
  ▼
Outbox

All inside one local database transaction.

This is one of the cleanest patterns I've found for building durable event-driven workflows.

Why the Inbox Transaction Boundary Matters

Imagine this sequence:

Receive Event
    │
    ▼
Perform Business Operation
    │
    ▼
Mark Inbox PROCESSED

Now suppose the business operation commits but updating the Inbox fails.

The broker delivers the message again.

The system may execute the business operation twice.

Now reverse the failure:

Inbox marked PROCESSED
        │
        ▼
Business transaction fails

The infrastructure thinks the message has been processed.

But the business state says otherwise.

Both situations are dangerous.

Where possible, the business state and Inbox completion should participate in the same local transaction.

And if processing creates another event, the Outbox write belongs there too.

BEGIN

    Update Business State

    Mark Inbox Completed

    Insert Outbox Event

COMMIT

Now either the whole processing step commits or it does not.

Durable Handoffs Are the Real Workflow

This led me to a useful way of thinking about long-running event-driven workflows.

Each service performs three responsibilities.

1. Receive responsibility

Usually through an incoming event recorded in an Inbox.

2. Perform a local unit of work

Using a short local transaction where appropriate.

3. Hand responsibility to the next step

By creating an Outbox event.

Conceptually:

Inbox
  │
  ▼
Local Work
  │
  ▼
Outbox
  │
  ▼
Next Service

The event isn't merely a notification.

It represents a durable handoff of responsibility.

This model becomes particularly useful when a workflow spans multiple services.

This Is Not a Distributed Transaction

The architecture is deliberately not trying to do this:

BEGIN GLOBAL TRANSACTION

    Payment Database

    Gateway Database

    Kafka

    External Payment Provider

COMMIT EVERYTHING

Instead:

Payment Service
      │
      └── Local Transaction

            │
            ▼

       Durable Event

            │
            ▼

Gateway Service
      │
      └── Local Transaction

            │
            ▼

    External Provider
      │
      └── Independent System

Each component owns its local consistency boundary.

Durable events connect those boundaries.

That means the system accepts an unavoidable reality:

A distributed workflow can be partially complete.

Once we accept that, we can design explicitly for recovery.

What Happens If the Application Crashes?

Let's look at several failure windows.

Crash after the payment commits but before Kafka receives the event

The Outbox record still exists.

The dispatcher can publish it later.

Crash after the Gateway Service receives the event

The Inbox provides durable processing state.

External gateway temporarily fails

The retry policy determines whether and when processing should continue.

Crash after creating the result event but before publishing it

The result remains in the Outbox.

The dispatcher can recover it after restart.

The architecture does not prevent crashes.

It does something more practical:

It reduces the amount of important information that disappears when a crash happens.

Retries Require Idempotency

Reliable systems retry things.

And once retries exist, duplicate execution becomes possible.

Imagine this scenario:

Application
    │
    ├── Send payment request
    │
    ▼
External Provider
    │
    ├── Accept request
    │
    ▼
Network timeout

From the application's perspective, the request failed.

From the provider's perspective, it may have succeeded.

Blindly retrying could now perform the operation twice.

This is why retries and idempotency must be designed together.

For events, we might use:

stable event IDs
Inbox deduplication

For external payment providers:

idempotency keys

For local business state:

valid state-transition checks
unique business constraints
processed-operation records

The important rule is:

Retries without idempotency can turn a reliability feature into a correctness bug.

Observability Becomes Part of the Workflow

Once the process becomes asynchronous, one question becomes much harder:

Where is my payment?

A synchronous request may be relatively easy to follow through a stack trace.

A long-running workflow can span:

Multiple transactions
        +
Multiple services
        +
Multiple events
        +
Retries
        +
Broker delivery
        +
External providers
        +
30 minutes of elapsed time

We therefore need to answer operational questions such as:

What is the current payment state?
Which event started this step?
Was the Outbox event dispatched?
Did the consumer receive it?
What is the Inbox processing status?
How many times has processing been attempted?
Why did the previous attempt fail?
Is another retry scheduled?
Which downstream event was produced?

This is one reason I prefer durable Outbox and Inbox state over hiding all reliability behavior inside broker configuration.

Persistent state gives us an operational window into the workflow.

Putting Everything Together

The architecture eventually looks something like this:

┌────────────────────┐
│   Payment Service  │
│                    │
│ Payment + Outbox   │
└─────────┬──────────┘
          │
        COMMIT
          │
          ▼
       Broker
          │
          ▼
┌─────────────────────────┐
│ Payment Gateway Service │
│                         │
│ Inbox                   │
└────────────┬────────────┘
             │
             ▼
    External Gateway Call
    (No DB Transaction)
             │
             │ potentially long-running
             ▼
      Gateway Response
             │
             ▼
┌─────────────────────────┐
│ Short Local Transaction │
│                         │
│ Business State          │
│ Inbox Completion        │
│ Outbox Result Event     │
└────────────┬────────────┘
             │
           COMMIT
             │
             ▼
          Broker
             │
             ▼
      Payment Service
             │
             ▼
       Payment Updated

There is no 30-minute database transaction.

There is a 30-minute business workflow composed of short transactions and durable handoffs.

That distinction is the heart of the architecture.

The Lesson Isn't Really About Payments

Payments make the problem easy to see because correctness matters so much.

But the same architecture applies to many long-running workflows:

order fulfillment
identity verification
document processing
external approval workflows
provisioning
subscription activation
fraud checks
shipping
asynchronous reporting

Whenever a business process crosses service boundaries and may take significant time, it is worth asking:

Are we modeling a workflow, or are we accidentally trying to stretch a database transaction across it?

Those are very different things.

What Building This Changed for Me

Spring makes @Transactional so convenient that it is easy to think about transaction boundaries in terms of methods.

Distributed workflows force us to think differently.

A transaction boundary should follow the local consistency requirement, not necessarily the entire business operation.

A payment may take 30 minutes.

That doesn't mean its transaction should take 30 minutes.

A workflow may cross five services.

That doesn't mean those services should share one transaction.

An external provider may fail after local state has already committed.

That doesn't mean the architecture is broken.

It means failure and partial progress need to become first-class parts of the design.

This is where Outbox and Inbox become much more interesting than simple messaging patterns.

Together, they let us transform:

One fragile long-running transaction

into:

A sequence of short,
durable,
observable,
recoverable steps

The Bigger Lesson

The goal of a reliable distributed workflow is not to make the entire system atomic.

In most real-world architectures, it can't be.

The goal is to make every transition understandable and recoverable.

Commit local state.

Record what needs to happen next.

Release the transaction.

Perform slow external work outside the transaction.

Persist the result.

Reliably hand responsibility to the next component.

Make every step observable.

For a 30-minute payment workflow, that architecture is much more valuable than simply increasing a timeout.

Because the real problem was never the 30 minutes.

The real problem was asking one transaction to represent a process that was never truly transactional in the first place.

About NERV Event

NERV Event is an open-source event-driven infrastructure library for Spring Boot focused on reliable event delivery and processing.

It provides infrastructure for:

Transactional Outbox
Inbox processing
retries
idempotency
ordering
Kafka and SQS integration
multi-instance processing
scheduler resilience
operational visibility

NERV Event is part of NERV — Next-Generation Engineering for Runtime Velocity.

If you're interested in the implementation, architecture, or want to contribute:

👉 NERV Event on GitHub

This article is part of my ongoing series about building production-ready event-driven systems with Spring Boot and NERV Event.

The Database Tells You What Is. An Audit Trail Tells You What Happened

Ed Legaspi — Sat, 03 Oct 2026 22:23:52 +0000

Something is wrong with a customer's account.

You open the database and run:

SELECT *
FROM customer_account
WHERE id = 12345;

The row looks perfectly valid.

id       = 12345
status   = ACTIVE
plan     = PREMIUM
limit    = 10000

The database has answered your question:

What does the account look like right now?

Unfortunately, that wasn't really the question you needed answered.

The support ticket says:

"My account was disabled yesterday. Why?"

Now you need completely different information.

Was the account actually disabled?
When did it happen?
What was the previous status?
Who changed it?
Was it a user action?
Was it an automated process?
Did another update happen afterward?
What else changed in the same operation?

The current row can't tell you.

Because:

The database tells you what is. An audit trail tells you what happened.

That distinction becomes increasingly important once an application reaches production.

Current State Is Not History

Most relational databases are designed around current state.

Consider:

UPDATE customer_account
SET status = 'ACTIVE'
WHERE id = 12345;

Before the update:

status = SUSPENDED

After the update:

status = ACTIVE

Query the table afterward and you see:

ACTIVE

That's correct.

But the previous state has disappeared.

Unless something else recorded it, you no longer know that the account was ever SUSPENDED.

The database has faithfully persisted the new truth.

It hasn't preserved the story that produced it.

`updated_at` Is Not an Audit Trail

A common first step is adding timestamps:

@CreatedDate
private Instant createdAt;

@LastModifiedDate
private Instant updatedAt;

These fields are useful.

They tell us:

Something changed at 10:42.

They don't tell us:

What changed at 10:42?

Suppose we have:

status       = ACTIVE
creditLimit  = 10000
country      = PH
updatedAt    = 2026-10-01T10:42:15Z

What happened at 10:42:15Z?

Did the status change?

Did the credit limit change?

Did someone correct the country?

Did all three change?

What were their previous values?

We can't know from updatedAt.

Even adding:

private String updatedBy;

only gives us:

someone changed something at this time

That's valuable metadata.

But it still isn't history.

Logs Aren't the Same Thing Either

The next place we often look is application logs.

Perhaps somewhere we have:

Updating customer 12345

Or, if we're lucky:

Changing customer 12345 status from SUSPENDED to ACTIVE

Logs are extremely useful for observability.

But using them as the authoritative history of entity changes creates problems.

Logs may be:

rotated
sampled
retained for limited periods
distributed across services
formatted differently between code paths
missing for bulk/database operations
difficult to correlate with entity revisions

More importantly, logs describe application execution.

An audit trail describes business data evolution.

Those concerns overlap, but they aren't identical.

What We Actually Want

For an audited entity, we want to reconstruct something like:

Revision 1
2026-09-01 08:30 UTC
status = PENDING
plan   = BASIC

        ↓

Revision 2
2026-09-03 14:12 UTC
status = ACTIVE
plan   = BASIC

        ↓

Revision 3
2026-09-20 09:41 UTC
status = ACTIVE
plan   = PREMIUM

        ↓

Revision 4
2026-10-01 10:42 UTC
status = SUSPENDED
plan   = PREMIUM

        ↓

Revision 5
2026-10-01 11:07 UTC
status = ACTIVE
plan   = PREMIUM

Now the production investigation becomes much easier.

The database still gives us:

ACTIVE

But the audit trail explains how we got there.

Hibernate Envers Gives Us Revision History

If you're already using Hibernate, one option for entity auditing is Hibernate Envers.

At its simplest, an entity can be audited with:

@Entity
@Audited
public class CustomerAccount {

    @Id
    private Long id;

    private String status;

    private String plan;

    private BigDecimal creditLimit;
}

When the entity changes, Envers stores revisions in audit tables.

Conceptually, instead of only having:

customer_account

you also get historical data representing:

customer_account_AUD

Each revision captures the entity state associated with a particular change.

Now we have something much more powerful than:

updatedAt = ...

We have actual historical versions.

Revision History Changes How You Debug Production

Consider the original support ticket:

"My account was disabled yesterday. Why?"

Without history, you might inspect the current database:

SELECT status
FROM customer_account
WHERE id = 12345;

Result:

ACTIVE

Now you're searching logs and trying to reconstruct what happened.

With revision history, you can inspect the entity's evolution:

Revision 41
status = ACTIVE

Revision 42
status = SUSPENDED

Revision 43
status = ACTIVE

Now you know the customer was correct.

The account really was suspended.

And you know exactly which revision introduced the change.

That gives you a much better starting point for investigating why.

But Production Auditing Needs More Than `@Audited`

This is where things become more interesting.

Adding @Audited is easy.

Building a useful production audit system around it is harder.

Soon you need answers to questions such as:

Who performed the change?
Which application or service produced it?
When exactly did it happen?
Which fields changed?
What were the old and new values?
Can the history be queried efficiently?
Can it be exposed through an API?
How should timestamps behave across time zones?
How should audit schemas evolve?
How do we validate the audit schema during deployments?

These are the kinds of concerns that tend to appear after the first implementation works.

Time Is More Subtle Than It Looks

Audit timestamps are particularly important.

If an audit trail is supposed to explain exactly when something happened, time needs a clear semantic meaning.

For distributed systems, that usually means UTC.

Application servers may run in different time zones.

Developers may run applications locally in different regions.

Databases may have different timezone configurations.

Containers may inherit different environment settings.

An audit record shouldn't change meaning because one JVM happened to run in Manila and another in Frankfurt.

A useful pattern is to make the clock explicit:

@Bean
public Clock nervAuditClock() {
    return Clock.systemUTC();
}

Then generated audit timestamps can use:

Instant.now(clock);

This also makes time deterministic during testing:

Clock fixedClock =
    Clock.fixed(
        Instant.parse("2026-10-01T10:42:15Z"),
        ZoneOffset.UTC);

Now tests don't depend on:

whatever time the machine happens to think it is

That sounds like a small implementation detail.

For an audit system, it isn't.

Time is part of the evidence.

Audit Data Is Part of Your Database Schema

There's another mistake that's easy to make.

We often think of audit tables as an implementation detail created by the ORM.

But once audit history matters operationally, those tables are part of your production data model.

That means schema changes matter.

Imagine adding:

private String riskCategory;

to an audited entity.

Your primary table changes.

The corresponding audit representation may need to change too.

If your migration updates one but not the other, application startup might succeed while audit behavior breaks later.

Production audit infrastructure therefore needs the same discipline as normal persistence infrastructure:

schema migration
        +
audit schema migration
        +
schema validation
        +
integration testing

Audit history shouldn't be treated as an accidental side effect of Hibernate.

It's production data.

Audit History Should Be Queryable

Recording history is only useful if developers and applications can actually retrieve it.

A useful audit API might expose something conceptually like:

GET /customers/12345/audit

and return:

[
  {
    "revision": 41,
    "timestamp": "2026-10-01T09:00:00Z",
    "status": "ACTIVE"
  },
  {
    "revision": 42,
    "timestamp": "2026-10-01T10:42:15Z",
    "status": "SUSPENDED"
  },
  {
    "revision": 43,
    "timestamp": "2026-10-01T11:07:33Z",
    "status": "ACTIVE"
  }
]

But we can go further.

For many support and operational use cases, developers don't really want complete snapshots.

They want the difference.

Something closer to:

Revision 42

status:
    ACTIVE → SUSPENDED

changedBy:
    account-service

changedAt:
    2026-10-01T10:42:15Z

That turns raw revision storage into something humans can actually use.

Vertical History Can Be More Useful Than Full Revisions

Sometimes the question isn't:

What did this entity look like at revision 42?

The question is:

How has this particular field changed over time?

For example:

CustomerAccount.status

PENDING
    ↓
ACTIVE
    ↓
SUSPENDED
    ↓
ACTIVE

Or:

creditLimit

5,000
    ↓
7,500
    ↓
10,000

This kind of vertical audit history is particularly useful for:

support investigations
administrative interfaces
compliance reviews
debugging state transitions
explaining automated decisions

The database row tells you the current value.

Vertical history tells you the life of that value.

This Is Why I Built NERV Audit

These production concerns are what led me to build NERV Audit.

NERV Audit is an open-source auditing foundation built around Hibernate Envers for Spring applications.

The goal isn't to replace Envers.

Envers already solves the difficult problem of storing entity revisions.

The goal is to build reusable infrastructure around the things production systems repeatedly need on top of it:

Application
      |
      v
   NERV Audit
      |
      +-- Audit Querying
      +-- Revision Metadata
      +-- Change Detection
      +-- Vertical History
      +-- UTC Time Handling
      +-- Schema Support
      |
      v
Hibernate Envers
      |
      v
   Database

The same principle I use across the NERV libraries applies here:

Build on the framework. Don't hide it.

If you already understand Hibernate and Envers, that knowledge should remain useful.

NERV Audit provides infrastructure around them rather than inventing an entirely different auditing model.

Auditing Is About Explainability

It's easy to think of auditing as a compliance feature.

Sometimes it is.

But one of the most practical reasons for maintaining audit history is much simpler:

Production systems need to explain themselves.

A user says:

"My account changed."

Support needs to know whether they're correct.

An engineer sees an unexpected state.

They need to know how it got there.

A business rule produces an unexpected result.

Someone needs to reconstruct the sequence of changes that led to it.

Without history, you're often left combining:

current database state
+
application logs
+
distributed traces
+
educated guesses

With a proper audit trail, the entity itself has a history.

And that can dramatically change how production incidents are investigated.

The Database Is Still Doing Its Job

None of this means the database is missing something.

The database is doing exactly what we asked it to do.

When we execute:

UPDATE customer_account
SET status = 'ACTIVE'
WHERE id = 12345;

we are explicitly asking it to make ACTIVE the current state.

The old value disappearing is expected.

If historical state matters to the application, preserving that history is an application architecture decision.

That's why I like the distinction:

The database tells you what is.

An audit trail tells you what happened.

For many production systems, you eventually need both.

What's Next?

This article introduces the problem behind NERV Audit.

The broader series will go deeper into the engineering challenges involved in building production audit infrastructure around Hibernate Envers, including:

revision metadata
identifying changed properties
vertical audit history
UTC timestamp semantics
deterministic audit testing
PostgreSQL integration
audit schema migrations
schema validation
exposing audit history safely through APIs
designing audit infrastructure that remains reusable across services

The interesting part isn't adding @Audited.

It's everything that happens after that.

Try NERV Audit

NERV Audit is open source and part of NERV — Next-Generation Engineering for Runtime Velocity.

GitHub:

https://github.com/czetsuyatech/nerv-audit

The canonical version of this article and my other engineering articles are available at:

https://www.czetsuyatech.com/

If you've had to investigate a production issue where the database showed the correct current state but couldn't explain how it got there, you've already encountered the problem an audit trail is designed to solve.

This article is part of the NERV Audit series. NERV is an open-source collection of Java and Spring libraries focused on reusable infrastructure for production applications.

Building a Production-Ready Persistence Layer in Spring Boot

Ed Legaspi — Mon, 28 Sep 2026 05:27:17 +0000

Spring Data JPA makes persistence remarkably easy to start with.

Create an entity.

Extend JpaRepository.

Start writing business logic.

public interface CustomerRepository
        extends JpaRepository<Customer, Long> {
}

For many Spring Boot applications, that's exactly how the persistence layer begins.

And there's nothing wrong with it.

The problems usually appear later.

As the application grows:

repositories accumulate query methods
entities repeat auditing fields
search APIs require increasingly complex filters
specifications appear throughout the codebase
pagination and sorting are implemented differently
entire entities are loaded when only a few fields are needed

Eventually, persistence stops being simply about storing entities.

The more interesting question becomes:

How do you keep a Spring Boot persistence layer consistent, reusable, and understandable as the application grows?

That's the problem I wanted to solve with NERV Persistence.

It Usually Starts With `JpaRepository`

Imagine we're building a customer service.

Initially, we need to find customers by status:

List<Customer> findByStatus(CustomerStatus status);

Simple.

Then we need country:

List<Customer> findByStatusAndCountry(
        CustomerStatus status,
        String country);

Then customer type:

List<Customer> findByStatusAndCountryAndType(
        CustomerStatus status,
        String country,
        CustomerType type);

Eventually, the API becomes something like:

GET /customers
    ?status=ACTIVE
    &country=PH
    &type=PREMIUM

Except every parameter is optional.

Now our repository starts heading toward this:

findByStatus(...)
findByCountry(...)
findByType(...)

findByStatusAndCountry(...)
findByStatusAndType(...)
findByCountryAndType(...)

findByStatusAndCountryAndType(...)

Add creation dates, account types, or other search criteria and the number of combinations keeps growing.

This is where repository method derivation stops being the right abstraction.

Dynamic Queries Should Be Composable

Spring Data JPA already gives us a powerful solution:

Specification<T>

Instead of defining every possible combination, we define individual conditions.

For example:

public static Specification<Customer> hasStatus(
        CustomerStatus status) {

    return (root, query, cb) ->
            status == null
                    ? cb.conjunction()
                    : cb.equal(root.get("status"), status);
}

Country becomes another specification:

public static Specification<Customer> hasCountry(
        String country) {

    return (root, query, cb) ->
            country == null
                    ? cb.conjunction()
                    : cb.equal(root.get("country"), country);
}

And customer type another:

public static Specification<Customer> hasType(
        CustomerType type) {

    return (root, query, cb) ->
            type == null
                    ? cb.conjunction()
                    : cb.equal(root.get("type"), type);
}

Now we can compose them:

Specification<Customer> specification =
        Specification.where(hasStatus(status))
                .and(hasCountry(country))
                .and(hasType(type));

The repository only needs to support specifications:

public interface CustomerRepository
        extends JpaRepository<Customer, Long>,
                JpaSpecificationExecutor<Customer> {
}

And querying becomes:

Page<Customer> customers =
        customerRepository.findAll(specification, pageable);

Adding another optional filter no longer requires another collection of repository methods.

We add another composable condition.

That's a much better match for dynamic search APIs.

But Specifications Can Become Boilerplate Too

Specifications solve the combination problem.

But after using them across a large application, another pattern emerges.

You repeatedly build predicates for:

equals
not equals
IN
ranges
dates
strings
null checks
relationships

Eventually, the codebase contains many slightly different versions of the same Criteria API logic.

For example:

(root, query, cb) ->
        cb.equal(root.get("status"), status)

and elsewhere:

(root, query, cb) ->
        cb.equal(root.get("country"), country)

The business meaning is different.

The persistence mechanics are almost identical.

This is a good candidate for reusable infrastructure.

The application should express what it wants to query.

The persistence foundation can handle the repetitive mechanics of constructing those predicates.

Ideally, application code starts becoming more expressive:

CustomerSpecifications.activeCustomers()

or:

CustomerSpecifications.createdBetween(from, to)

The goal isn't to hide JPA.

It's to make the application-specific part of the query obvious.

The Same Problem Exists With Entities

Query infrastructure isn't the only thing we repeat.

Look at several entities in a typical production application and you'll often find:

private Instant createdAt;
private Instant updatedAt;
private String createdBy;
private String updatedBy;

Then the same auditing annotations.

Then similar lifecycle behavior.

A reusable base model can establish a consistent convention:

@MappedSuperclass
public abstract class AuditableModel {

    @CreatedDate
    private Instant createdAt;

    @LastModifiedDate
    private Instant updatedAt;

    @CreatedBy
    private String createdBy;

    @LastModifiedBy
    private String updatedBy;
}

Business entities can then concentrate on business data:

@Entity
public class Customer extends AuditableModel {

    @Id
    @GeneratedValue
    private Long id;

    private String name;

    private String country;

    @Enumerated(EnumType.STRING)
    private CustomerStatus status;
}

Saving four fields isn't the interesting part.

Consistency is.

When persisted entities follow predictable conventions:

developers know what to expect
infrastructure becomes easier to build
operational investigation becomes easier
new services don't need to recreate the same foundation

Don't Automatically Return Entities

There's another habit that's easy to develop with JPA:

I have an entity, therefore my query should return the entity.

That isn't always true.

Suppose Customer eventually contains 30 fields and several relationships.

Our search endpoint might only need:

id
name
status
country
createdAt

Loading the complete entity can be unnecessary.

It can also introduce:

accidental lazy loading
larger persistence contexts
unnecessary data retrieval
unwanted entity serialization
coupling between the API and persistence model

Instead, we can define a projection:

public interface CustomerSummary {

    Long getId();

    String getName();

    CustomerStatus getStatus();

    String getCountry();

    Instant getCreatedAt();
}

This gives us an important persistence rule:

A persisted entity is not automatically the correct model for every read operation.

Entities, DTOs, and projections solve different problems.

Pagination Is Easy Until Every API Does It Differently

Spring makes pagination straightforward:

PageRequest.of(page, size)

But real APIs eventually need rules around:

default page size
maximum page size
page numbering
allowed sorting properties
sort direction
multiple sort fields
invalid parameters

If every controller independently handles these concerns, subtle inconsistencies start appearing.

One endpoint uses zero-based pages.

Another exposes one-based pages.

One endpoint allows arbitrary sorting.

Another validates fields.

Again, this isn't complicated code.

It's repetitive infrastructure code.

And repetitive infrastructure is where shared conventions can provide value.

Don't Build a Framework on Top of a Framework

There's an important danger when creating reusable persistence infrastructure.

It's very easy to overdo it.

We remove one piece of boilerplate.

Then add an abstraction.

Then another.

Eventually developers are using a proprietary persistence framework and barely recognize Spring Data JPA underneath it.

That's not what I want from a persistence library.

If you already understand:

JpaRepository
JpaSpecificationExecutor
Specification
Pageable

that knowledge should remain useful.

Reusable infrastructure should extend those concepts rather than replace them.

This is particularly important when debugging production problems.

When a query behaves unexpectedly, the execution path should still be understandable:

Application
     |
     v
Persistence Infrastructure
     |
     v
Spring Data JPA
     |
     v
Hibernate
     |
     v
Database

Removing boilerplate is useful.

Hiding behavior isn't.

This Is Why I Built NERV Persistence

These recurring problems led me to build NERV Persistence.

NERV Persistence is an open-source persistence foundation for Spring Boot applications built on Spring Data JPA.

The goal is straightforward:

Provide reusable persistence building blocks without replacing the framework developers already know.

It focuses on recurring concerns such as:

common persistence models
auditable entities
reusable specifications
dynamic querying
projections
repository conventions
consistent persistence patterns

Conceptually:

Application
    |
    +-- REST APIs
    +-- Application Services
    +-- DTOs / Mappers
    +-- Domain Logic
            |
            v
      NERV Persistence
            |
            +-- Persistence Models
            +-- Specifications
            +-- Query Infrastructure
            +-- Projections
            +-- Repository Foundation
                    |
                    v
            Spring Data JPA
                    |
                    v
                Hibernate
                    |
                    v
                Database

NERV Persistence doesn't replace Spring Data JPA.

It doesn't replace Hibernate.

And it shouldn't contain your application's business rules.

It provides reusable infrastructure around them.

What About Microservices?

This distinction becomes even more important with microservices.

Imagine:

customer-service
payment-service
order-service
subscription-service
notification-service

Each service should own its domain model and its database.

That independence is important.

But independence doesn't mean every service needs its own implementation of:

auditing
pagination
specifications
query helpers
repository conventions

Those are infrastructure concerns.

This leads to one of the principles behind NERV Persistence:

Share infrastructure conventions, not domain models.

Putting a Customer entity into a common library because several services understand customers creates domain coupling.

Sharing infrastructure used to audit, query, paginate, and persist entities is different.

One shares business ownership.

The other shares engineering plumbing.

Production-Ready Doesn't Have to Mean Complicated

A production-ready persistence layer doesn't need hundreds of abstractions.

I prefer the opposite.

The common path should be boring:

define entity
    |
    v
define repository
    |
    v
compose query
    |
    v
retrieve required model
    |
    v
map to application response

Complexity should appear only when the actual problem requires it.

That's the philosophy behind NERV Persistence and the broader NERV project:

Build reusable infrastructure around problems that repeatedly appear in production systems while keeping the underlying technology visible, understandable, and debuggable.

What's Next?

This is Article #1 of my NERV Persistence series.

Rather than turning the series into library documentation, I'm going to use it to explore the engineering problems that led to the library.

Next up:

Dynamic Queries in Spring Data JPA Without Repository Method Explosion

After that, we'll dig into:

composable Spring Data JPA specifications
auditable base entities
JPA projections and efficient read models
reusable pagination, sorting, and filtering
entity vs DTO vs projection
what belongs in a base JPA entity
reusable repository infrastructure
consistent persistence foundations across microservices

Try NERV Persistence

NERV Persistence is open source and part of NERV — Next-Generation Engineering for Runtime Velocity.

GitHub:

https://github.com/czetsuyatech/nerv-persistence

The original version of this article and the rest of my engineering articles are available at:

https://www.czetsuyatech.com/

If you've worked on a large Spring Data JPA application, I'd also be interested in hearing which persistence problem you find yourself solving repeatedly.

This article is part of the NERV Persistence series. NERV is an open-source collection of Java and Spring libraries focused on reusable infrastructure for production applications.

The Inbox Transaction Boundary: Getting Event Processing Right in Spring Boot

Ed Legaspi — Wed, 16 Sep 2026 00:52:36 +0000

The Inbox Transaction Boundary: Getting Event Processing Right in Spring Boot

One of the hardest parts of building an event-driven system isn't getting messages from one service to another.

It's making sure the system behaves correctly when something fails halfway through processing.

While building NERV Event, I ran into an important reliability problem:

Where should the transaction boundary be when processing an event?

At first, an Inbox can look deceptively simple.

Receive a message.

Check whether it was already processed.

Execute the business operation.

Mark the message as processed.

Acknowledge the message.

But these operations don't automatically share the same transactional boundary.

And that creates some uncomfortable failure scenarios.

The Inbox Pattern

The Inbox pattern is commonly used to make event consumers idempotent.

A simplified flow looks like this:

                    ┌─────────────────┐
                    │     Broker      │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │     Consumer    │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │      Inbox      │
                    │                 │
                    │ event_id        │
                    │ status          │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │ Business Logic  │
                    └─────────────────┘

The Inbox records that an event has been received and tracks its processing state.

For example:

RECEIVED
   │
   ▼
PROCESSING
   │
   ▼
PROCESSED

If the broker delivers the same event again, the consumer can inspect the Inbox and determine whether the event has already been processed.

This gives us a foundation for idempotent event processing.

But there's an important question:

How does the Inbox transaction relate to the business transaction?

The Transaction Boundary Problem

Consider a consumer processing an OrderCreated event.

The consumer needs to:

Receive the event.
Create an Inbox record.
Execute the business operation.
Mark the Inbox record as processed.
Acknowledge the broker message.

It is tempting to think of this as one atomic operation.

It isn't necessarily one transaction.

The database transaction and the broker acknowledgment belong to different systems.

Even within the database, the Inbox update and business operation can accidentally end up in different transactional boundaries.

That matters when something fails.

When the Business Operation Succeeds but the Inbox Doesn't

Consider this sequence:

Business operation
       │
       ▼
    SUCCESS
       │
       ▼
Inbox update
       │
       ▼
    FAILURE

If the business operation and Inbox update participate in the same database transaction, the failure can cause the entire transaction to roll back.

The event can then be retried.

But consider a design where they use separate transactions:

Transaction A
    │
    └── Business operation COMMITTED

Transaction B
    │
    └── Inbox update FAILED

Now the business operation has already committed.

The Inbox doesn't indicate that processing completed.

The broker may redeliver the event.

The consumer sees an event that appears unprocessed and executes the business operation again.

Now idempotency becomes critical.

When the Inbox Says Processed but the Business Operation Rolls Back

The reverse situation can be even more problematic.

Suppose the Inbox is marked as processed before the business operation is committed:

Inbox
  │
  └── PROCESSED

Then the business transaction fails:

Business transaction
  │
  └── ROLLBACK

If those changes aren't part of the same appropriate transactional boundary, the system can end up with contradictory state.

The Inbox says:

PROCESSED

But the business data says:

NOT PROCESSED

A retry may then be suppressed because the Inbox says the event has already been processed.

The system has effectively lost the opportunity to recover.

What Should Share the Transaction?

For a database-backed Inbox, the important relationship is between the Inbox state and the business state.

Conceptually:

┌──────────────────────────────────────────┐
│          Database Transaction            │
│                                          │
│  Inbox state                             │
│       +                                  │
│  Business state                          │
│                                          │
│       └── Atomic commit / rollback       │
└──────────────────────────────────────────┘

The goal is that the Inbox processing state and the business changes have a consistent relationship.

A simplified Spring Boot example might look like:

@Transactional
public void process(Event event) {
    Inbox inbox = inboxRepository.findByEventId(event.id());

    if (inbox.isProcessed()) {
        return;
    }

    businessService.apply(event);

    inbox.markProcessed();
}

The important part isn't the @Transactional annotation itself.

The important part is understanding which operations actually participate in that transaction.

If businessService.apply(event) and the Inbox update use the same database transaction, the database can commit or roll back those changes together.

But What About the Broker?

This is where things become more interesting.

A database transaction normally cannot make a Kafka acknowledgment atomic with the database commit.

You effectively have two systems:

        Database                    Broker
           │                          │
           │                          │
     DB transaction              Message ack
           │                          │
           └──────────┬───────────────┘
                      │
                Different systems

Even with a correctly designed Inbox transaction, there is still a failure window around the broker acknowledgment.

For example:

Database transaction
        │
        ├── Business change
        └── Inbox = PROCESSED
        │
        ▼
     COMMIT
        │
        ▼
Broker acknowledgment
        │
        └── FAILURE

The broker may deliver the message again.

But this is exactly where the Inbox pattern provides its value.

On redelivery:

Event arrives
     │
     ▼
Check Inbox
     │
     ▼
Already PROCESSED
     │
     ▼
Skip business operation

The consumer can safely tolerate duplicate delivery.

This is the fundamental relationship between the Inbox pattern and at-least-once delivery.

The Goal Isn't Exactly-Once Processing

A common misconception is that the Inbox pattern makes event processing exactly-once.

It doesn't.

The broker may still deliver the same message more than once.

Instead, the goal is closer to:

At-least-once delivery
          +
Idempotent processing
          =
Reliable consumer behavior

The Inbox gives the consumer persistent knowledge about an event's processing state.

That state can then be used to make retries deterministic.

Processing State Matters

A simple boolean such as:

processed = true

often isn't enough for a production event-processing system.

A useful Inbox model can distinguish states such as:

RECEIVED
PROCESSING
PROCESSED
FAILED

This makes failures inspectable.

For example:

RECEIVED
   │
   ▼
PROCESSING
   │
   ├──────────────► PROCESSED
   │
   └──────────────► FAILED

Now the system has an explicit record of what happened.

That becomes particularly valuable when retries are involved.

What Happens During a Retry?

Suppose processing fails:

Event
  │
  ▼
PROCESSING
  │
  ▼
Business operation fails
  │
  ▼
FAILED

A retry can then use the Inbox state to determine what should happen next.

Depending on the architecture, the consumer can:

retry immediately
retry after a delay
increment an attempt counter
record the failure
eventually move the event to a failure or DLQ mechanism

The important thing is that the failure becomes persistent state rather than disappearing with the consumer process.

The Inbox Is More Than Deduplication

This is the architectural distinction I find most important.

An Inbox is often introduced as:

A table that prevents duplicate messages.

That's only part of the story.

A production Inbox can provide:

idempotency
processing state
retry state
failure information
inspection
operational visibility
deterministic recovery

That makes the Inbox part of the consumer's reliability boundary.

The question therefore isn't simply:

"Have we seen this event before?"

It becomes:

"What do we know about this event's processing lifecycle?"

Designing the Boundary Explicitly

When designing an event consumer, it helps to make the boundaries explicit.

Think about three separate concerns:

             Broker
                │
                ▼
        Message Delivery
                │
                ▼
        ┌───────────────┐
        │    Inbox      │
        │               │
        │ Receipt/state │
        └───────┬───────┘
                │
                ▼
        ┌───────────────┐
        │   Database    │
        │  Transaction  │
        │               │
        │ Inbox +       │
        │ Business Data │
        └───────────────┘
                │
                ▼
          Broker Ack

The database transaction provides atomicity between the Inbox state and the business changes.

The Inbox provides persistent processing state.

The broker acknowledgment controls message delivery semantics.

These are related, but they are not the same thing.

Spring Boot Makes the Boundary Easy to Hide

Spring Boot makes transaction management straightforward.

That is useful, but it can also make the boundary easy to overlook.

For example:

@Transactional
public void handle(OrderCreatedEvent event) {
    Inbox inbox = inboxService.startProcessing(event);

    if (inbox.isProcessed()) {
        return;
    }

    orderService.createOrder(event);

    inboxService.markProcessed(inbox);
}

The code looks simple.

But several architectural questions are hiding behind it:

Which database transaction is active?
Which operations participate in it?
Does orderService start another transaction?
Can the Inbox update commit independently?
When is the Inbox state persisted?
When is the broker message acknowledged?
What happens if the application crashes after the database commit?
What happens if the broker acknowledgment fails?

The annotation doesn't answer those questions.

The architecture does.

Failure Windows Are Part of the Design

One of the most useful lessons from event-driven architecture is that failure isn't an exceptional edge case.

It is part of the normal operating model.

A consumer can crash after committing its database transaction:

Business operation
       │
       ▼
Inbox + business data
       │
       ▼
    COMMIT
       │
       X
  Application
    crashes
       │
       ▼
Broker redelivers

Or:

Inbox + business transaction
       │
       ▼
    COMMIT
       │
       X
Broker acknowledgment fails
       │
       ▼
Redelivery

A reliable system doesn't try to pretend these scenarios cannot happen.

It defines what should happen when they do.

That's the real purpose of the Inbox.

Building NERV Event

These are the kinds of engineering decisions that shaped NERV Event.

The goal isn't simply to provide an Inbox implementation.

It's to provide explicit persistence and processing semantics that make event-driven behavior easier to understand, operate, and debug.

The Inbox is treated as part of the consumer reliability model rather than just a mechanism for detecting duplicate event IDs.

That distinction becomes increasingly important as systems move from simple message consumption toward production workloads involving:

multiple application instances
retries
failures
duplicate delivery
long-running processing
operational recovery

Final Thoughts

The Inbox transaction boundary is easy to overlook.

Most event-driven examples focus on the happy path:

Receive → Process → Acknowledge

Production systems need to think differently:

Receive
   │
   ▼
Persist state
   │
   ▼
Process
   │
   ├── success ──► Commit
   │
   └── failure ──► Retry / Failure state
                         │
                         ▼
                    Redelivery

The important question isn't whether failures will happen.

They will.

The important question is whether the system's state still makes sense after they do.

That's why the transaction boundary around the Inbox and business operation deserves deliberate architectural attention.

NERV Event

NERV Event is an open-source Spring Boot library for building reliable event-driven systems with patterns such as:

Transactional Outbox
Inbox
Idempotent consumers
Retries
Failure handling
Broker integrations

If you're building event-driven systems with Spring Boot, the transaction boundary is one of those details worth getting right before production forces you to think about it.

Transactional Outbox Pattern with Spring Boot: Reliable Event

Ed Legaspi — Thu, 10 Sep 2026 10:58:02 +0000

A service saves data to the database.

Then it publishes an event to Kafka.

Simple enough.

Until the database transaction succeeds and the Kafka publish fails.

Now your application says something happened, but the rest of the system never hears about it.

This is the dual-write problem, and it's one of the fundamental reliability problems in event-driven systems.

The Transactional Outbox Pattern gives us a practical way to solve it without trying to coordinate a distributed transaction between the database and the message broker.

In this article, we'll look at how it works, what guarantees it actually provides, and some of the production concerns that appear once you start running it across multiple application instances.

The Dual-Write Problem

Consider a typical Spring Boot service:

@Transactional
public void completePayment(Payment payment) {
    paymentRepository.save(payment);

    kafkaTemplate.send(
        "payments",
        new PaymentCompletedEvent(payment.getId())
    );
}

At first glance, this looks reasonable.

But two independent systems participate in this operation:

The application database
Kafka

Your database transaction does not automatically include Kafka.

That creates failure scenarios.

Database succeeds, Kafka fails

Database
Payment = COMPLETED  ✓

Kafka
PaymentCompleted     ✗

The payment exists, but downstream services never receive the event.

Kafka succeeds, database fails

The opposite can also happen depending on when publishing occurs:

Kafka
PaymentCompleted     ✓

Database
Payment = COMPLETED  ✗

Consumers may now process an event representing state that was never committed.

What we really want is this:

If the business transaction commits, the intention to publish its event must commit with it.

That's the problem the Transactional Outbox Pattern addresses.

Introducing the Transactional Outbox

Instead of publishing directly to Kafka inside the business transaction, we persist the event in the same database transaction as the business data.

          DATABASE TRANSACTION
     ┌─────────────────────────────┐
     │                             │
     │  Update Payment             │
     │         +                   │
     │  Insert Outbox Event        │
     │                             │
     └──────────────┬──────────────┘
                    │
                  COMMIT
                    │
                    ▼
             Outbox Dispatcher
                    │
                    ▼
                  Kafka

Now the database provides the atomic boundary.

Either both records commit:

Payment       ✓
Outbox Event  ✓

or neither does:

Payment       ✗
Outbox Event  ✗

Kafka no longer has to participate in the business transaction.

That's the key idea.

We're not making the database and Kafka transactional together.

We're making the business state and the intention to publish transactional together.

What Goes Into the Outbox?

An outbox record contains enough information to publish the event later.

Conceptually:

event_id       = 01J...
event_type     = PaymentCompleted
source         = payment-service
payload        = {...}
content_type   = application/json
status         = PENDING
created_at     = ...

The important part is that both the event payload and its delivery state are persisted.

I prefer keeping the payload readable rather than hiding it behind opaque serialization.

Why?

Because eventually something will fail in production.

When that happens, being able to inspect exactly what your service intended to publish makes debugging considerably easier.

The outbox isn't only a delivery mechanism.

It also becomes useful operational history.

Dispatching the Event

Once the business transaction has committed, a separate dispatcher processes the outbox.

A simplified lifecycle might look like this:

PENDING
   │
   ▼
PROCESSING
   │
   ├───────────────► PUBLISHED
   │
   └── failure
          │
          ▼
       retry later

If an event repeatedly fails and exhausts its retry policy, it can eventually transition to:

FAILED

This separation is important.

The business transaction is responsible for recording what happened.

The dispatcher is responsible for delivering that information.

Those responsibilities should be able to fail independently.

Why Not Just Retry Kafka Inside the Transaction?

A common alternative is to retry the Kafka operation:

@Transactional
public void completePayment(Payment payment) {
    paymentRepository.save(payment);

    retryTemplate.execute(context ->
        kafkaTemplate.send("payments", event)
    );
}

Retries help with temporary broker failures.

But they don't solve the underlying coupling.

Imagine Kafka is unavailable for several minutes.

Should your payment transaction stay open while your application waits for Kafka to recover?

Usually, no.

Long-running transactions can:

hold database resources longer
increase lock duration
increase contention
couple business availability to broker availability

With an outbox, the responsibilities are separated:

Business Transaction
        │
        ├── commits quickly
        │
        ▼
      Outbox
        │
        │ Kafka unavailable?
        │
        └── retry asynchronously

Your business operation doesn't need to wait for the broker.

The Important Catch: At-Least-Once Delivery

The outbox solves message loss, but it introduces another architectural consideration.

Imagine this:

1. Dispatcher publishes event to Kafka
2. Kafka accepts the event
3. Application crashes
4. Outbox wasn't marked PUBLISHED
5. Application restarts
6. Dispatcher publishes the event again

The same event may be delivered twice.

This isn't necessarily a bug.

It's a consequence of favoring reliable delivery over silently losing events.

So a reliable event architecture usually becomes:

Transactional Outbox
        +
At-Least-Once Delivery
        +
Idempotent Consumer

The producer cannot simply assume that every event will be processed exactly once.

Consumers must be designed to tolerate duplicates.

That naturally leads us to the Inbox Pattern, which I'll cover in the next article in this series.

What Happens With Multiple Application Instances?

Production applications rarely run as a single instance.

Suppose we have three pods:

                 OUTBOX
                   │
          ┌────────┼────────┐
          ▼        ▼        ▼
        Pod A    Pod B    Pod C

If every instance polls the same outbox table without coordination, multiple workers could select the same events.

Database-level locking can provide one solution.

For databases that support it, a common approach is conceptually similar to:

SELECT ...
FROM outbox
WHERE status = 'PENDING'
FOR UPDATE SKIP LOCKED;

Suppose Pod A locks:

1  2  3

Pod B doesn't wait for those rows.

It skips them and can claim:

4  5  6

This makes it possible for multiple workers to process the outbox concurrently without all competing for the same records.

But locking strategy matters.

A naive SELECT ... FOR UPDATE implementation can create unnecessary contention as throughput and worker count increase.

The important goal is:

Multiple workers should be able to safely claim work without serializing the entire outbox.

Polling vs Change Data Capture

Polling isn't the only way to implement the outbox pattern.

Another common architecture uses Change Data Capture (CDC):

Application
     │
     ▼
Database Outbox
     │
     ▼
    CDC
     │
     ▼
   Kafka

Tools such as Debezium can stream database changes into Kafka instead of having application workers poll the outbox.

CDC can be a great choice, particularly when the required infrastructure already exists.

But it comes with different operational trade-offs.

Application-Level Polling

Advantages include:

simpler infrastructure
straightforward local development
application-controlled retries
easier application-level debugging
potential broker independence

Change Data Capture

Advantages can include:

reduced application polling
high-throughput change streaming
natural Kafka integration
separation of event extraction from application workers

Neither approach is universally better.

More importantly, polling vs CDC isn't the core of the pattern.

The architectural guarantee comes from this:

Business State + Outbox Event
              │
       SAME TRANSACTION

How you transport that committed event afterward is a separate design decision.

Why I Built NERV Event

Once you've implemented this architecture several times, you start seeing the same infrastructure repeatedly:

Outbox persistence
       │
       ├── Serialization
       ├── Dispatching
       ├── Retries
       ├── Concurrency
       ├── Failure states
       ├── Metrics
       ├── Kafka integration
       ├── SQS integration
       └── Operational tooling

That's infrastructure every business application shouldn't have to rebuild.

It's one of the reasons I created NERV Event.

NERV Event is an open-source Spring Boot library focused on reliable event processing using transactional Outbox and Inbox patterns.

The architecture is intentionally straightforward:

Business Service
       │
       ▼
   NERV Event
       │
       ▼
Transactional Outbox
       │
       ▼
   Dispatcher
       │
     ┌─┴─┐
     ▼   ▼
  Kafka  SQS

The business application owns the event.

The infrastructure owns its reliable delivery.

And importantly, the state remains inspectable when something goes wrong.

Reliability Doesn't End at the Producer

The Transactional Outbox Pattern answers an important question:

How do I prevent an event from being lost after my business transaction commits?

But successfully publishing the event isn't the end of the reliability story.

Now we have another service receiving it.

What happens if:

the same event arrives twice?
processing succeeds but acknowledgement fails?
the consumer crashes halfway through processing?
processing needs to be retried?
we need to determine whether an event has already been handled?

These are consumer-side reliability problems.

And that's where the Inbox Pattern comes in.

In Article #2, we'll look at:

Inbox Pattern and Idempotent Consumers

and build the other half of reliable event processing.

NERV Event

NERV Event is open source and available on GitHub:

👉 github.com/czetsuyatech/nerv-event

If you're building event-driven Spring Boot systems and have dealt with outbox/inbox implementations in production, I'm interested in hearing how you're handling these problems — particularly polling vs CDC and multi-instance event claiming.

This article is part of my series on building reliable event-driven systems with Spring Boot.

Reliable Event-Driven Architecture in Spring Boot: Outbox, Inbox, Retries, and Idempotency

Ed Legaspi — Tue, 01 Sep 2026 14:56:23 +0000

Event-driven architecture looks simple at first.

Your application performs a business operation, publishes an event to Kafka or SQS, and another service consumes it.

@Transactional
public void createOrder(CreateOrderCommand command) {
  Order order = orderRepository.save(...);

  kafkaTemplate.send(
      "orders",
      new OrderCreatedEvent(order.getId())
  );
}

Looks reasonable.

The order is saved, an OrderCreatedEvent is published, and other services can react to it.

But there is a problem hiding in those few lines:

What happens if the database transaction succeeds, but publishing the event fails?

And on the consumer side:

What happens if the same event is delivered twice?

These questions lead to several patterns that become essential once event-driven systems move beyond simple demos:

Transactional Outbox
Inbox Pattern
Idempotent Consumers
Durable Retries
Multi-instance-safe processing
Observable failure handling

Let's build the architecture step by step.

The Dual-Write Problem

Imagine an order service that needs to:

Save an order to PostgreSQL.
Publish OrderCreated to Kafka.

Conceptually:

Database ──────> COMMIT ✓
                   |
Kafka ─────────> PUBLISH ✗

The database and Kafka are independent systems.

A successful database commit does not guarantee a successful Kafka publish.

Consider:

1. INSERT order
2. COMMIT
3. Publish OrderCreated
4. Application crashes

If the application crashes between steps 2 and 3, the order exists but the event doesn't.

Other services may never know that the order was created.

Reversing the operations doesn't fix it:

1. Publish OrderCreated
2. INSERT order
3. Database transaction fails

Now consumers may receive an event for an order that doesn't exist.

This is the classic dual-write problem.

The Transactional Outbox Pattern

Instead of trying to atomically update the database and message broker, persist the event as part of the same database transaction as the business operation.

             Database Transaction
        ┌─────────────────────────────┐
        │                             │
Request ──> Business Data             │
        │       +                     │
        │   Outbox Event              │
        │                             │
        └────────── COMMIT ───────────┘
                       |
                       v
                Outbox Dispatcher
                       |
                       v
                  Kafka / SQS

When an order is created, we persist:

ORDER
  +
OUTBOX EVENT

inside the same transaction.

Either both commit or neither commits.

A separate dispatcher finds pending outbox records and publishes them to the broker.

A simplified lifecycle could look like:

PENDING
   |
   v
PROCESSING
   |
   +──── success ────> PUBLISHED
   |
   └──── failure ────> RETRY / FAILED

This removes the dangerous database-and-broker dual write from the business transaction.

It also gives us something extremely useful:

persistent delivery state.

Make the Outbox Debuggable

Reliability isn't only about retrying failed operations.

Eventually, someone will need to answer:

What happened to this event?

A useful outbox should contain enough information to answer that question:

eventId
eventType
source
correlationId
payload
status
attempts
createdAt
availableAt
publishedAt
lastError

Instead of reconstructing everything from distributed logs, an engineer can inspect the actual state:

SELECT *
FROM event_outbox
WHERE status = 'FAILED';

This leads to an important principle:

Reliability mechanisms should also improve debuggability.

If an event cannot be delivered, that failure should be visible and inspectable.

Reliable Publishing Is Only Half the Problem

Suppose our outbox works perfectly.

Every event eventually reaches Kafka.

We're still not finished.

Most event-driven systems operate with at-least-once delivery.

That means the same event may arrive more than once.

Producer
   |
   v
Kafka
   |
   +──── OrderCreated #123 ────> Consumer
   |
   +──── OrderCreated #123 ────> Consumer

A consumer could successfully process an event and then crash before acknowledging it.

The broker delivers the event again.

If the handler sends an email, perhaps the customer receives two emails.

If it performs a payment operation, the consequences can be considerably worse.

Consumers therefore need to assume:

Every event can arrive more than once.

The Inbox Pattern

The Inbox Pattern provides a durable record of received events.

Before processing an event, the consumer registers its unique event ID.

Broker
   |
   v
Inbox Registration
   |
   +── event already exists ──> DUPLICATE
   |
   └── new event
          |
          v
       RECEIVED
          |
          v
      PROCESSING
        /     \
       v       v
 PROCESSED   FAILED

If the same eventId arrives again, the consumer knows that it has already seen it.

The event ID becomes an idempotency boundary.

Instead of depending on exactly-once delivery, we make duplicate delivery safe.

The Inbox Is More Than a Deduplication Table

A minimal inbox could contain nothing more than processed event IDs.

In a production system, however, it can become a durable history of event processing.

Consider storing:

eventId
eventType
source
correlationId
payload
status
attempts
receivedAt
processedAt
availableAt
lastError

Now imagine an incident:

Order 123 was created, but the downstream action never happened.

You can inspect the inbox.

Was the event received?

Did processing start?

Did the handler fail?

How many attempts were made?

When is the next retry?

What was the last error?

Those questions become much easier to answer when processing state is explicit.

Retries Should Survive Application Restarts

Spring provides excellent retry mechanisms.

For example:

@Retryable
public void handle(OrderCreatedEvent event) {
  ...
}

This can be perfectly appropriate for short-lived transient failures.

But durable event processing introduces another question:

What happens if the JVM dies?

Event processing fails
        |
        v
Retry scheduled in memory
        |
        v
Application restarts

If retry state exists only in memory, it disappears with the process.

For critical event processing, retry state can instead be persisted:

FAILED
   |
   | availableAt <= now
   v
PROCESSING
   |
   +──── success ────> PROCESSED
   |
   └──── failure ────> FAILED
                         |
                         + attempts++
                         + availableAt = next retry

A scheduler periodically finds events whose retry time has arrived.

Because the state lives in the database, restarting the application doesn't destroy the retry information.

Back Off Instead of Hammering a Failing Dependency

Retrying continuously can make an outage worse.

If a downstream service is unavailable, thousands of failed events retrying as quickly as possible only add pressure.

A better strategy is exponential backoff:

Attempt 1 → immediate
Attempt 2 → +1 second
Attempt 3 → +2 seconds
Attempt 4 → +4 seconds
Attempt 5 → +8 seconds

Eventually, the configured retry limit is exhausted.

At that point, the event can remain explicitly failed:

status = FAILED
availableAt = null

Automatic retries stop.

But importantly, the failure doesn't disappear.

The event remains available for investigation and operational recovery.

What About Dead-Letter Queues?

Dead-letter queues are useful, particularly for broker-level failures.

But the broker's DLQ doesn't necessarily need to become the application's primary record of processing failure.

There is a useful distinction:

Broker concern               Application concern

Delivery failure             Processing failure
Malformed message            Business handler failure
Transport problem            Retry exhaustion
        |                            |
        v                            v
       DLQ                         INBOX

The two mechanisms can coexist.

A database-backed inbox gives the application direct visibility into its own processing state, while a DLQ remains available for appropriate broker- and transport-level failures.

Then You Deploy Multiple Pods

Everything becomes more interesting once the application runs more than one instance.

                  OUTBOX
                     |
               pending event
                     |
          ┌──────────┴──────────┐
          v                     v
        Pod A                 Pod B
     Dispatcher             Dispatcher

Both instances may discover the same pending event.

Without concurrency control, both may attempt to process it.

Production implementations therefore need a concept of claiming or locking.

For example:

status
lockOwner
lockedAt

An instance claims records transactionally before processing them.

Other instances can then determine that those records are already being handled.

But this creates another question:

What happens if a pod claims an event and then dies?

The system needs a deterministic mechanism for recovering stale claims after an appropriate timeout.

At this point, the outbox is no longer just a database table plus a scheduled query.

It has become infrastructure.

Even the Scheduler Can Fail

There is another failure mode that is surprisingly easy to overlook.

Suppose the dispatcher completes successfully and schedules its next execution:

Dispatcher completes
        |
        v
schedule(nextRun)
        |
        X
TaskScheduler rejects the task

If the scheduler continues reporting itself as running, the application has entered a dangerous state.

Everything appears healthy.

But no future dispatch will happen.

Events can quietly accumulate in the outbox.

A reliable scheduler therefore benefits from an explicit lifecycle:

STOPPED
   |
   v
STARTING
   |
   v
RUNNING
   |
   +──── scheduling failure ────> FAILED

A useful invariant is:

A scheduler must not report itself as running if no task is scheduled and no work is currently executing.

Scheduling infrastructure itself needs observable failure semantics.

Observability Is Part of Reliability

Imagine receiving a production incident at 2 AM:

We created the order, but the downstream system didn't process it.

Ideally, you should be able to follow the event:

Order
  |
  v
Outbox Event
  |
  +── created
  +── claimed
  +── publish attempts
  +── published
  |
  v
Broker
  |
  v
Inbox Event
  |
  +── received
  +── processing attempts
  +── failure reason
  +── retry schedule
  +── processed

Correlation metadata should connect the pieces.

Payloads should be readable.

State transitions should be explicit.

Failures should remain inspectable.

Logs should explain what the infrastructure is doing without becoming the only source of truth.

A reliable system isn't only one that recovers from failures. It is one that helps engineers understand those failures.

Putting It All Together

Once these pieces are combined, the architecture starts looking like this:

Business Transaction
        |
        v
Transactional Outbox
        |
        v
Outbox Dispatcher
        |
        v
   Kafka / SQS
        |
        v
      Inbox
        |
        v
Idempotency Check
        |
        v
 Event Handler
        |
        +── Success
        |
        └── Durable Retry

Underneath it all is persistent state:

Persistence
├── Outbox delivery state
├── Inbox processing state
├── Attempts
├── Retry scheduling
├── Lock ownership
└── Failure information

That's quite a bit of infrastructure around what originally looked like:

kafkaTemplate.send(...);

From Architecture to Implementation: NERV Event

These are the problems I wanted to solve consistently across Spring Boot applications.

None of the individual patterns are new.

Transactional outbox is well understood.

Idempotent consumers are well understood.

Retries, locking, and message brokers are well understood.

The difficult part is making them work together consistently in a production application.

That's what led me to build NERV Event.

NERV Event is an open-source event infrastructure library for Spring Boot that implements the architecture described in this article.

It brings together:

Transactional outbox persistence
Durable inbox processing
Idempotent event consumption
Persistent retries
Exponential retry policies
Multi-instance-safe processing
Kafka integration
AWS SQS integration
Scheduler lifecycle and failure visibility
Event retention
Operational inspection
Correlation metadata
Human-readable persisted payloads

The goal isn't to hide event-driven architecture behind magic.

The goal is to make its behavior predictable, observable, and easy to debug.

NERV Event

The project is open source:

👉 NERV Event on GitHub

The repository contains the source code, documentation, configuration, and examples.

Even if you don't use the library directly, I hope the architecture and implementation can be useful as a reference when designing reliable event-driven Spring Boot applications.

I'll cover individual parts of NERV Event in future articles, including transactional publishing, inbox processing, retries, Kafka and SQS integration, multi-pod deployments, and operational tooling.

Final Thoughts

Adding Kafka or SQS to a Spring Boot application doesn't automatically make the application reliably event-driven.

The difficult parts exist around the broker:

Business Transaction
        |
        v
Transactional Outbox
        |
        v
Reliable Delivery
        |
        v
At-Least-Once Messaging
        |
        v
Inbox + Idempotency
        |
        v
Durable Processing
        |
        v
Retries + Recovery
        |
        v
Observability

Each layer addresses a different failure mode.

And in distributed systems, those failures aren't theoretical.

Processes restart. Networks fail. Messages are redelivered. Dependencies become unavailable. Schedulers fail. Multiple instances compete for the same work.

The objective isn't to pretend these failures won't happen.

It's to design the system so that when they do:

State is preserved. Recovery is predictable. And engineers can understand exactly what happened.

That's the philosophy behind NERV Event.