DEV Community: Shitanshu Jha

Beyond Backend Development: Exploring System Design with ShopEase

Shitanshu Jha — Sat, 14 Nov 2026 05:39:16 +0000

Introduction

Building a backend application is not just about writing APIs and connecting databases. As an application grows, important questions start appearing:

How can the system handle increasing traffic?

What happens when a service becomes unavailable?

How should databases and caches be used?

How do different services communicate?

How can we maintain data consistency?

In Module 41 — System Design, I focused on understanding these architectural decisions through the ShopEase e-commerce backend.

Scalability: Handling Application Growth

Scalability is the ability of a system to handle increasing workloads.

Two common approaches are:

Vertical Scaling: Increasing the resources of an existing server, such as CPU and RAM.

Horizontal Scaling: Adding more instances of an application to distribute incoming requests.

Understanding the difference helps in choosing an appropriate scaling strategy as application traffic grows.

Availability and Reliability

Availability refers to whether a system is accessible when users need it.

Reliability focuses on whether the system continues to perform its intended operations correctly.

These are important considerations for e-commerce applications, where failures in order processing, payments, or inventory operations can affect users.

CAP Theorem and Data Consistency

The CAP theorem describes the trade-offs between Consistency, Availability, and Partition Tolerance in distributed data systems.

I explored how these concepts influence architectural decisions, particularly when multiple services and databases are involved.

Data consistency is especially important in operations such as order placement, inventory updates, and payment processing.

Load Balancing and Database Scaling

Load balancing distributes incoming requests across multiple application instances.

Database scaling focuses on handling growing data volumes and query workloads.

These concepts help explain how backend systems can manage increasing traffic without placing all the workload on a single application instance or database.

Caching Architecture

Caching helps reduce repeated database queries by temporarily storing frequently accessed data.

In ShopEase, Redis is part of the backend architecture. Understanding caching at the system-design level helped connect cache usage with response time, database workload, and cache consistency.

Message Queues and Microservices Architecture

Microservices divide an application into smaller services with specific responsibilities.

Message queues support asynchronous communication between services.

In ShopEase, Apache Kafka is used as part of the event-driven architecture. This provides a practical context for understanding asynchronous communication and service coordination.

Fault Tolerance and High Availability

Fault tolerance focuses on how a system handles component failures.

High availability aims to minimize service interruptions.

I studied these concepts to understand the architectural approaches used to reduce the impact of failures in distributed applications.

Key Learnings

Module 41 helped connect backend development with broader system-design concepts:

Scalability: Vertical and horizontal scaling.

Availability and reliability.

CAP theorem and distributed data consistency.

Load balancing and database scaling.

Redis-based caching architecture.

Kafka and asynchronous messaging.

Microservices communication.

Fault tolerance and high availability.

Conclusion

System design is about understanding the decisions behind an application's architecture, not simply adding more technologies.

Through ShopEase, I am connecting concepts such as caching, microservices, databases, and messaging with the architectural challenges that appear as an application grows.

Module 41 marks an important step in understanding not just how to build backend systems, but also how to think about their architecture.

Project: ShopEase — Full-Stack E-Commerce Application

GitHub Repositories:

Backend: https://github.com/Shitanshu686/shopping-app-backend

Frontend: https://github.com/Shitanshu686/shopping-app

Java #SpringBoot #SystemDesign #Microservices #BackendDevelopment

Building Enterprise-Grade Distributed Tracing in a Spring Boot Event-Driven Architecture (ShopEase)

Shitanshu Jha — Fri, 13 Nov 2026 06:59:08 +0000

Modern distributed microservices architectures offer incredible scalability, but they introduce a fundamental observability challenge: The Black Box Problem of Asynchronous Messaging.

When a client makes an HTTP request to an e-commerce platform, tracing the request lifecycle via a unique Correlation ID across REST and gRPC endpoints is relatively straightforward using thread-local storage like SLF4J's MDC (Mapped Diagnostic Context). However, the moment that request triggers a state change—such as placing an order—and pushes an event into an asynchronous message broker like Apache Kafka, that execution jumps across thread boundaries.

The background consumer thread picking up the message from the queue has no native awareness of the incoming HTTP thread's context. Without explicit propagation, your correlation ID is lost, turning distributed log aggregation and debugging in production into a guessing game.

In this article, we look at how we solved this architectural hurdle in ShopEase—a production-grade Spring Boot e-commerce backend—by engineering a seamless, non-intrusive Distributed Tracing and Correlation ID Propagation Pipeline across Kafka thread boundaries.

The Problem: Context Loss Across Asynchronous Boundaries
In ShopEase, our order orchestration relies on an event-driven choreography using Apache Kafka. The lifecycle flows roughly as follows:

HTTP/REST Request: A client initiates a checkout request. Our custom CorrelationIdInterceptor intercepts the incoming request header (X-Correlation-ID), generates one if missing, and injects it into the SLF4J MDC context.

Synchronous/Service Execution: Services talk via gRPC and REST, logging actions tied directly to this correlationId.

The Asynchronous Leap (OrderSagaOrchestrator): An order is placed, and an OrderPlacedEvent is published to Kafka.

The Break: The publishing thread finishes. A completely separate consumer thread pool picks up the record from the Kafka broker to update inventory or trigger payment processing. Because standard Kafka template sends and standard listener containers do not automatically map MDC contexts across thread handoffs, the correlationId drops to null.

The Solution: Bridging Kafka Producers and Consumers with MDC
To solve this without cluttering our core business logic, we leveraged Kafka’s built-in Headers mechanism combined with Spring Kafka’s interception hooks. We split the solution into two symmetrical pillars: Producer Injection and Consumer Interception.

Injecting the Correlation ID at the Producer Level When publishing events via KafkaTemplate, instead of relying on a bare record send, we wrap our payload inside a Kafka ProducerRecord. We read the active correlationId from the thread's MDC context and append it as a Kafka record header:

Java
public void publishOrderPlacedEvent(OrderPlacedEvent event) {
log.info("Publishing OrderPlacedEvent to Kafka topic '{}': {}", KafkaTopicConfig.ORDER_PLACED_TOPIC, event);

// 1. Grab current correlation ID from MDC
String correlationId = MDC.get("correlationId");

ProducerRecord<String, Object> record = new ProducerRecord<>(
    KafkaTopicConfig.ORDER_PLACED_TOPIC, 
    String.valueOf(event.getOrderId()), 
    event
);

// 2. Inject into Kafka headers if present
if (correlationId != null && !correlationId.isBlank()) {
    record.headers().add("correlationId", correlationId.getBytes(StandardCharsets.UTF_8));
}

this.kafkaTemplate.send(record);

}

Restoring Context via Consumer Record Interceptors On the consuming side, when a worker thread picks up the message from the queue, we intercept the incoming record before it reaches any @KafkaListener method. Using ConcurrentKafkaListenerContainerFactory#setRecordInterceptor, we extract the correlationId header and populate the worker thread's MDC context:

Java
@bean
public ConcurrentKafkaListenerContainerFactory kafkaListenerContainerFactory(
ConsumerFactory consumerFactory,
KafkaTemplate kafkaTemplate) {

ConcurrentKafkaListenerContainerFactory<String, Object> factory =
        new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory);

factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL_IMMEDIATE);

// Kafka Record Interceptor for Correlation ID Propagation in MDC
factory.setRecordInterceptor(record -> {
    Header header = record.headers().lastHeader("correlationId");
    if (header != null) {
        String correlationId = new String(header.value(), StandardCharsets.UTF_8);
        MDC.put("correlationId", correlationId);
    } else {
        MDC.put("correlationId", UUID.randomUUID().toString());
    }
    return record;
});

// Error handler & DLQ recoverer setup...
return factory;

}
Crucial Production Note: Because consumer threads are pooled and reused across messages, failing to clear or overwrite MDC can cause context leaks (where Message B inherits Message A's trace ID). Always ensure your consumers clear or wrap logging execution contexts cleanly.

Architectural Impact & Benefits
By implementing this pattern in ShopEase, we achieved:

End-to-End Observability: A single correlationId now flows seamlessly from the initial HTTP entry point, through gRPC service calls, into Kafka topics, and across downstream consumer workers.

Zero Business Logic Pollution: Developers writing event listeners don't need to manually pass or unpack tracing headers; logging frameworks pick up the trace context automatically.

Resilient Debugging: When a transaction fails and hits our Dead Letter Queue (DLQ), we can query our centralized log aggregation tool using the correlation ID and inspect the entire distributed timeline instantly.

Conclusion
Distributed tracing isn't just an afterthought or an APM tool feature—it's an architectural requirement. By leveraging Kafka headers and Spring interceptors, we bridged the gap between synchronous web requests and asynchronous event-driven pipelines, making ShopEase truly enterprise-ready.

Engineering a High-Concurrency E-Commerce Backend: Testing, Concurrency & Resilience at Scale

Shitanshu Jha — Thu, 12 Nov 2026 10:44:06 +0000

Introduction
Modern e-commerce platforms operate under extreme constraints. During flash sales or peak traffic events, backends face massive spikes in concurrent requests, threatening data integrity, inventory accuracy, and system availability.

In building ShopEase (shopping-app-backend), a Spring Boot 3.x Java 17 e-commerce backend, achieving functional correctness was only half the battle. The true challenge lay in engineering a robust Testing & Quality Engineering pipeline to guarantee thread safety, zero race conditions, and predictable performance under heavy load.

The Testing & Quality Engineering Strategy
To ensure zero regressions and high availability, our testing architecture covers every critical tier of the application:

Unit & Slice Testing: Isolating controllers and services using Mockito (OrderControllerTest, ProductControllerTest) to validate HTTP statuses and JSON serialization instantly.

Integration Testing: Verifying cross-component flows and database interactions (OrderServiceIntegrationTest).

Concurrency & Race Condition Testing: Simulating multi-threaded contention to prevent inventory overselling and lost updates (InventoryConcurrencyTest).

Load & Stress Testing: Automating high-traffic simulations via custom Java concurrency harnesses to measure throughput and failure rates under pressure (LoadAndStressTest).

Key Technical Highlights

Zero-Mock Concurrency Testing
To prevent race conditions during high-demand operations, we validated thread safety using multi-threaded execution services (ExecutorService), ensuring that simultaneous updates maintain strict data consistency.
Lightweight Load & Stress Harness
Instead of relying on heavy external CLI tools, we built an integrated Java-based load testing harness using standard HTTP clients to bombard endpoints with concurrent virtual users, measuring real-time latency, success ratios, and system stability under load.

Conclusion
By implementing this comprehensive testing suite, we achieved 100% build success across all modules, robust thread safety for inventory management, and high test maintainability. Building resilient systems isn't just about clean code—it's about proving that your architecture holds up under real-world pressure.

Is Your E-Commerce Backend Secure? ShopEase's JWT, Token Rotation & Checkout Upgrades

Shitanshu Jha — Wed, 11 Nov 2026 02:19:25 +0000

Recent Backend Engineering Milestones | November 6–11, 2026
Building an e-commerce backend is not just about implementing product listings, shopping carts, and checkout APIs.

The real engineering challenges begin when security, performance, and data consistency become critical.

Over the past few days, I focused on strengthening the ShopEase backend across three major areas:

Security and authentication
Database performance and transaction management
Reliable order processing under concurrent requests Here's what changed.
Advanced Database Transactions — November 11 One of the most important challenges in e-commerce is maintaining data consistency during checkout.

Multiple users may place orders simultaneously, and transaction isolation becomes an important part of handling concurrent database operations.

What I implemented:

Configured REPEATABLE_READ transaction isolation.
Applied the configuration to OrderService.createOrder.
Strengthened transaction consistency during order processing.
Addressed data anomalies associated with concurrent transactions. The objective was to make checkout operations more reliable when multiple transactions interact with the database simultaneously.
Security Hardening — November 10 Security cannot be treated as an optional feature in an application that handles user accounts, orders, and payments.

I strengthened the application's security configuration.

What I implemented:

Hardened CORS configuration.
Added startup environment validation for sensitive secrets.
Integrated HTTP security headers.
Configured HSTS and FrameOptions in SecurityConfig. These changes help establish stricter security controls and reduce configuration-related risks.
JWT Authentication Across Sensitive Endpoints — November 9 Previously, securing individual APIs was not enough. Sensitive operations needed consistent authentication enforcement.

What I implemented:

Enforced JWT authentication across cart endpoints.
Secured order endpoints.
Protected payment endpoints.
Strengthened access control for sensitive operations. The objective was to ensure that these operations require valid authentication.
Cart Performance Optimization — November 8 Performance optimization is not always about adding more infrastructure. Sometimes, the biggest improvement comes from fixing inefficient database access patterns.

What I optimized:

Refactored buildCartResponse.
Replaced table-scan-style retrieval with direct repository queries.
Improved how cart response data is retrieved. I also finalized refresh token rotation and completed endpoint wiring with a stable clean build.
Refresh Token Rotation — November 6–8 Authentication should not depend on a single long-lived access token.

I implemented the refresh token rotation flow to strengthen session management.

Key changes:

Implemented token rotation logic.
Added TokenRefresh request and response DTOs.
Wired refresh token endpoints.
Finalized the authentication flow. This provides a foundation for safer token lifecycle management.

Development Timeline
Date Milestone
November 6 Refresh token rotation and DTOs
November 8 Cart optimization and endpoint wiring
November 9 JWT authentication enforcement
November 10 CORS, environment validation, HSTS and security headers
November 11 REPEATABLE_READ transaction isolationWhat's Next?
The focus is on continuing to improve ShopEase's backend reliability, security, and performance while validating the implementation through practical testing.
Final Thoughts
These updates represent an important step in evolving ShopEase beyond basic e-commerce functionality.

The goal is not simply to build APIs that work. It is to build backend systems that handle authentication securely, retrieve data efficiently, and maintain consistency during critical operations.

Secure authentication. Optimized queries. Reliable transactions.

That's the direction I'm taking with ShopEase.

About ShopEase

ShopEase is a full-stack e-commerce web application designed to provide a complete online shopping experience, from product discovery to cart management, order placement, and payment processing.

The project is built around a Java Spring Boot backend and integrates multiple technologies to handle different application requirements.

Key technologies and capabilities:

Java and Spring Boot for backend development.
Spring Security and JWT for authentication and authorization.
MySQL for relational business data.
MongoDB for dynamic product information.
Redis for caching and low-latency operations.
PostgreSQL for audit logging.
REST APIs for frontend-backend communication.
Docker and other backend technologies for application development and deployment.

ShopEase is an ongoing engineering project through which I am exploring backend architecture, database management, application security, performance optimization, and scalable system design.

GitHub:

Backend: https://github.com/Shitanshu686/shopping-app-backend

Frontend: https://github.com/Shitanshu686/shopping-app

Java #SpringBoot #BackendDevelopment #JWT #Redis #MySQL #PostgreSQL #DatabaseTransactions #SystemDesign #Ecommerce

My First Contribution to Zerocode: Building a Configurable Delay Executor,I Fixed the Code, But Who's Going to Fix My Sleep Schedule? 💀

Shitanshu Jha — Sun, 08 Nov 2026 05:05:35 +0000

Writing

My First Contribution to Zerocode: Building a Configurable Delay Executor
Introduction
My open-source journey with Zerocode became a practical learning experience when I started working on Issue #810 — Configurable Delay Executor.

Instead of simply writing a delay using Thread.sleep(), the challenge was to understand how Zerocode handles API execution internally and integrate a dedicated delay executor into its existing architecture.

I spent approximately 6–7 hours understanding the codebase, implementing the feature, debugging integration points, writing tests, and verifying the final implementation.
The Problem
The requirement was to introduce a configurable delay mechanism that could be used directly inside Zerocode test scenarios.

A test scenario should be able to request a delay in milliseconds without requiring an external API.

For example:
{
"name": "wait_for_100_ms",
"url": "util://DelayExecutor",
"operation": "DELAY",
"request": 100,
"assertions": {
"status": "Success",
"delayMillis": 100
}
}
The challenge was not just waiting for a specified duration. The delay operation needed to work with Zerocode's existing execution flow, dependency injection, API routing, and scenario runner.
What I Implemented

Dedicated Delay Executor I created a DelayExecutor interface and its implementation, DelayExecutorImpl.

The executor accepts a delay value in milliseconds, validates it, executes the delay, and returns a structured JSON response.

API Routing Integration I introduced a dedicated DELAY_CALL API type and configured the URL:

util://DelayExecutor

This ensures that delay requests are recognized as delay operations instead of being incorrectly routed through the generic Java API execution path.

Dependency Injection with Guice I integrated the new executor into Zerocode's Guice dependency injection configuration.

This allowed the scenario execution layer to access the delay executor through the framework's existing dependency injection mechanism.

Scenario Runner Integration I updated ZeroCodeMultiStepsScenarioRunnerImpl to recognize DELAY_CALL and delegate execution to the API executor.

This made the new functionality available directly inside test scenarios.

Validation and Interruption Handling I implemented handling for:
Null and empty delay values.
Invalid numeric values.
Negative delay values.
Thread interruption during execution. When interrupted, the implementation restores the thread's interrupt status before reporting the failure.
Testing and Documentation I added unit tests, API routing tests, and an integration scenario.

I also updated the README with usage instructions and an example of the new executor.
Testing Results
The targeted Maven test execution completed successfully:

19 tests executed.
0 failures.
0 errors.
1 skipped.
Build: SUCCESS. The tests covered delay execution, zero delay, invalid inputs, negative values, interruption handling, API routing, and scenario-level integration. What I Learned This contribution helped me understand several important concepts in real-world Java development:
Understanding an existing codebase: How to trace execution across interfaces, implementations, API routing, and scenario runners.
Dependency Injection: How Guice connects components without tightly coupling their implementations.
Framework Architecture: Why adding a feature requires integration across multiple layers rather than writing an isolated method.
Exception Handling: How to handle invalid inputs and thread interruption correctly.
Unit and Integration Testing: Why testing individual methods is not enough when a feature must work across an entire execution pipeline.
Open-Source Workflow: How to investigate an issue, work on a fork and feature branch, create commits, push changes, and submit a pull request. Contribution Status I submitted my implementation through:

Pull Request #814 — [Issue #810] Add configurable delay executor

Repository: https://github.com/authorjapps/zerocode

Pull Request: https://github.com/authorjapps/zerocode/pull/814

The pull request is currently open and awaiting review.
Final Thoughts
This contribution was a reminder that real software development involves much more than writing code.

Understanding an unfamiliar codebase, identifying the correct integration points, handling edge cases, writing meaningful tests, and verifying the implementation are all part of the engineering process.

My goal with open source is to keep learning how production-level projects are structured while contributing useful, tested improvements to the developer community.

MySQL Wasn't Enough! Why I Added MongoDB, Redis & PostgreSQL to ShopEase

Shitanshu Jha — Thu, 05 Nov 2026 10:02:02 +0000

Project: ShopEase — E-Commerce Web Application

What happens when you try to manage orders, payments, dynamic product specifications, caching, and audit logs using just one database?

You might end up forcing different types of workloads into a system that wasn't designed to handle all of them equally well.

While developing ShopEase, I chose a different approach: Polyglot Persistence.

Instead of depending on a single database, I integrated four different data technologies — MySQL, MongoDB, Redis, and PostgreSQL — and assigned each one a specific responsibility.

Here's why.

MySQL — The Transactional Backbone

E-commerce applications deal with critical business data: users, orders, payments, and cart records.

These operations require reliable transactions and strong data consistency.

That's why I selected MySQL as the primary relational database.

Why MySQL?

ACID transaction support.

Foreign key constraints and relational integrity.

Efficient joins and structured queries.

Reliable management of transactional business data.

MySQL handles the core relational data where consistency matters most.

MongoDB — Because Products Don't Follow One Fixed Structure

Consider two products:

A smartphone has RAM, storage, processor, and battery specifications.

A T-shirt has size, color, fabric, and fit.

Trying to represent every product using the same rigid structure can become difficult as the catalog grows.

That's where MongoDB comes in.

Why MongoDB?

Flexible document-based structure.

Different products can have different attributes.

Easier handling of dynamic product specifications.

Suitable for evolving product catalog requirements.

MongoDB gives ShopEase the flexibility to manage product-specific information without forcing every category into an identical schema.

Redis — Speed Matters

Repeatedly querying a relational database for frequently accessed information can increase database load and response time.

For performance-sensitive operations, I integrated Redis.

Why Redis?

In-memory data access.

Low-latency caching.

Session and frequently accessed data management.

Support for real-time cart-related operations and rate limiting.

Redis helps reduce unnecessary database queries and improves responsiveness for frequently accessed data.

PostgreSQL — Keeping an Independent Audit Trail

A production-oriented application also needs to track important system activities.

Questions such as these matter:

Who performed an action?

What action was performed?

Which entity was affected?

When did the action happen?

For this responsibility, I introduced PostgreSQL as a dedicated audit-log database.

Why PostgreSQL?

Independent storage for audit records.

Structured querying of audit history.

Separation of audit data from core business data.

Better organization of security and activity tracking.

This separation helps keep audit data management independent of the primary transactional database.

The Architecture at a Glance

Technology

Responsibility

MySQL

Users, orders, payments, and relational data

MongoDB

Dynamic product specifications

Redis

Caching, sessions, and low-latency operations

PostgreSQL

Audit logs and activity tracking

The Bigger Lesson: Polyglot Persistence

Using four databases wasn't about adding technologies just to make the architecture look advanced.

Each database solves a different problem.

MySQL provides transactional reliability.

MongoDB provides schema flexibility.

Redis provides low-latency data access.

PostgreSQL provides a dedicated audit data store.

The important engineering decision wasn't choosing four databases. It was understanding which data belongs where and why.

ShopEase gave me practical experience with designing a multi-database architecture and understanding the trade-offs involved in using different storage technologies within one application.

One application. Four databases. Four distinct responsibilities.

That's the idea behind Polyglot Persistence.

Java #SpringBoot #MySQL #MongoDB #PostgreSQL #Redis #PolyglotPersistence #BackendDevelopment #SystemDesign #Ecommerce #ShitanshuJha

# Beyond a Single Database: Engineering ShopEase with Polyglot Persistence & Event-Driven Microservices

Shitanshu Jha — Thu, 29 Oct 2026 06:42:43 +0000

What happens when an e-commerce backend needs transactional consistency, flexible product data, asynchronous processing, and reliable audit logging—all at the same time?

A single database and tightly coupled services aren't always the best fit for every workload. While working on ShopEase, my full-stack e-commerce project, I worked on a backend architecture that separates storage responsibilities and decouples downstream processing using Apache Kafka.

This engineering work focuses on three areas:

Polyglot Persistence Architecture
CQRS-inspired, event-driven processing
An isolated asynchronous compliance and audit pipeline

Let's explore the architecture and the problems these patterns are designed to address.

1. Polyglot Persistence: The Right Database for the Right Job

Different types of application data have different requirements. Transactional records, dynamic product attributes, and compliance logs don't necessarily benefit from the same storage model.

For ShopEase, I designed a hybrid storage architecture using three database technologies.

MySQL 8.4 — Core Transactional Data

Orders, payments, and inventory require reliable transactional operations and consistency.

MySQL handles these core transactional workloads, where ACID transactions are important for maintaining data integrity.

MongoDB — Dynamic Product Catalog

Product catalogs can contain different attributes depending on the product category. A laptop, a T-shirt, and a smartphone don't necessarily share the same fields.

MongoDB's BSON document model supports flexible, polymorphic product structures without forcing every product into an identical document shape.

PostgreSQL — Compliance and Audit Logs

Audit records have different access and retention requirements from transactional application data.

For this pipeline, PostgreSQL stores append-only compliance logs using JSONB, with GIN indexing to support queries over relevant JSON fields.

The architectural principle: choose storage based on the workload, data structure, and consistency requirements—not simply because a particular database is popular.

2. CQRS-Inspired Processing with Apache Kafka

Checkout workflows can involve several downstream operations: payment processing, inventory updates, notifications, and audit events.

Coupling all of these operations directly to the checkout request can increase dependencies and make failures harder to isolate.

In ShopEase, I worked on an event-driven design using Apache Kafka to decouple primary checkout processing from downstream services.

The high-level flow is:

A checkout request enters the order-processing workflow.
The relevant event is published to Kafka.
Downstream consumers process events for their respective responsibilities.
Payment, inventory, and notification processing can proceed independently of the synchronous request path where the workflow permits.
Failed event processing can be routed through retry or Dead Letter Queue (DLQ) handling.

The implementation also uses manual Kafka offset acknowledgements, allowing message acknowledgement to be coordinated with processing outcomes.

This requires careful handling of retries and duplicate deliveries. Kafka delivery guarantees alone do not automatically make database updates or external payment operations idempotent.

Where CQRS fits

CQRS separates command responsibilities from query responsibilities. It is not synonymous with Kafka or microservices.

In ShopEase, the event-driven decoupling supports a CQRS-inspired architectural approach. The exact command/query separation depends on how individual read and write models are implemented.

3. An Isolated Asynchronous Audit Pipeline

Audit logging can introduce additional database writes into application workflows. If those writes share critical request-processing resources, they can contribute to contention and additional latency.

To address this architectural concern, I worked on an asynchronous audit ingestion pipeline using Spring's @Async and dedicated secondary data sources.

The intended processing flow is:

Application events → Audit listener → Asynchronous processing → PostgreSQL audit storage

This separates audit-write responsibilities from the core transaction-processing path.

The design aims to:

Reduce coupling between core business operations and audit persistence.
Keep audit ingestion off the main request-processing thread where configured.
Store structured audit data using PostgreSQL JSONB.
Support JSON-field queries using appropriate GIN indexes.
Improve traceability and operational separation.

One important engineering consideration is reliability: asynchronous execution alone does not guarantee that every audit event will be persisted. Durable event delivery, retry policies, monitoring, and failure recovery must be considered according to the compliance requirements.

4. Putting the Architecture Together

The main components of this ShopEase backend architecture are:

Spring Boot: application services and backend APIs.
MySQL 8.4: core transactional data.
MongoDB: flexible product catalog documents.
PostgreSQL: compliance and audit records.
Apache Kafka: asynchronous event-driven communication.
Spring @Async: asynchronous audit processing.
JSONB and GIN indexes: structured audit data and JSON-oriented query support.

Each component has a distinct responsibility, helping the architecture address different data and processing requirements.

5. Key Engineering Lessons

Working on these architectural patterns reinforced several important backend engineering principles:

Database selection is a design decision. Multiple databases can be useful when workloads have genuinely different requirements, but they also introduce operational complexity.

Asynchronous processing changes failure handling. Retries, duplicate events, offset acknowledgement, and dead-letter handling need deliberate design.

Audit logging deserves its own reliability strategy. Separating audit ingestion can reduce coupling, but durability and recovery must be designed explicitly.

Distributed systems require observability. Logs, metrics, traces, and consumer-lag monitoring help identify problems across service boundaries.

What's Next?

The next step is to strengthen this architecture through integration testing, failure-injection tests, end-to-end tracing, and reproducible performance benchmarks. These help validate not just whether individual components work, but whether the complete system behaves correctly under failures and concurrent workloads.

About ShopEase

ShopEase is a full-stack e-commerce application developed by Shitanshu Jha, focused on Java backend engineering, database design, secure transactions, performance optimization, and scalable application architecture.

GitHub repositories:

Building ShopEase has been a practical journey in understanding the trade-offs behind modern backend systems—and turning architectural concepts into implementation work.

94% Faster Reads! How I Optimized the ShopEase Backend

Shitanshu Jha — Sat, 17 Oct 2026 05:45:18 +0000

ShopEase: Optimizing Backend Performance with Redis, Hibernate & MySQL

Overview

As part of my ShopEase e-commerce project, I worked on improving backend performance by optimizing the catalog and order data pipelines. The goal was to reduce read latency, eliminate inefficient database queries, improve connection management, and make data retrieval more efficient.

Key Optimizations

1. Redis Cache-Aside

Implemented a Redis Cache-Aside layer for catalog and order pipelines, along with automated cache invalidation to keep cached data consistent with updates.

Reported result: Read latency reduced from 1.88 seconds to 110 milliseconds — a 94% reduction.

2. Hibernate N+1 Query Optimization

Resolved Hibernate N+1 query bottlenecks using JOIN FETCH, allowing related data to be retrieved more efficiently and reducing unnecessary database round trips.

3. HikariCP Connection Pool Tuning

Optimized HikariCP connection pool settings to improve database connection management and resource utilization.

4. MySQL Composite B-Tree Indexes

Introduced composite B-Tree indexes to improve query execution efficiency and reduce unnecessary row scans.

Reported result: MySQL query row-scans reduced by 95%.

Technology Stack

Language: Java
Backend: Spring Boot, Hibernate, Spring Data JPA
Database: MySQL
Caching: Redis
Connection Pooling: HikariCP
Query Optimization: JOIN FETCH and composite B-Tree indexes

What I Learned

This optimization work reinforced that backend performance depends on multiple layers working together. Caching reduces repeated database reads, efficient ORM queries help avoid unnecessary round trips, connection pooling improves resource management, and indexing makes data retrieval more efficient.

Conclusion

Improving ShopEase was not just about adding features; it was also about making existing functionality faster and more efficient. These optimizations strengthened my practical understanding of Java backend development, database performance tuning, caching strategies, and scalable application design.

Engineered a multi-tier performance optimization for catalog and order pipelines: implemented a Redis Cache-Aside layer cutting read latency by 94% (1.88s → 110ms) with automated cache invalidation, resolved Hibernate N+1 query bottlenecks via JOIN FETCH, tuned HikariCP connection pooling, and introduced composite B-Tree indexes reducing MySQL query row-scans by 95%.

About ShopEase
ShopEase is a full-stack e-commerce application built with Java, Spring Boot, MySQL, and Redis. It focuses on secure shopping, efficient data management, and high-performance backend architecture.

Developed by Shitanshu Jha 💻

Project: ShopEase — Full-Stack E-Commerce Application

GitHub — Backend: https://github.com/Shitanshu686/shopping-app-backend

GitHub — Frontend: https://github.com/Shitanshu686/shopping-app

Java #SpringBoot #Redis #MySQL #Hibernate #BackendDevelopment #PerformanceOptimization #E-Commerce

I Fixed an Idempotency Bug in My Spring Boot E-Commerce Backend

Shitanshu Jha — Mon, 12 Oct 2026 07:58:30 +0000

Sometimes, the most useful improvements to a project come from someone asking a question you had not fully considered.

While working on ShopEase, my Spring Boot e-commerce backend, I implemented idempotency protection to prevent duplicate checkout requests.

My initial approach returned HTTP 409 Conflict whenever an Idempotency-Key was reused.

Then Andrei Ovcharenko commented on my Dev.to article with an important observation:

What if the order commits successfully, but the client never receives the HTTP response?

That question exposed an edge case in my implementation, so I worked on fixing it.

What Was Wrong With My Original Approach?

Imagine a checkout request:

The customer submits an order.

The backend creates the order and commits the database transaction.

A network issue prevents the response from reaching the customer.

The customer retries with the same idempotency key.

The backend returns 409 Conflict.

The duplicate order is prevented, but the customer still cannot recover the result of the successful checkout.

The server knows the order succeeded. The client does not.

A retry should be able to recover the original result, not just discover that the key has already been used.

Why Does Multithreading Make This Harder?

Concurrent requests can arrive using the same idempotency key before the first request finishes.

If both threads check the key before either one claims it, they may both attempt to process the checkout.

A reliable implementation needs to distinguish between a completed request and one that is still processing.

Completed request with an identical payload: replay the stored result.

Same key with a different payload: reject the request.

Request already in progress: prevent a second execution from processing it independently.

This requires atomic state management, not just an if statement checking whether a key exists.

What Did I Change?

I updated IdempotencyService.java and ConcurrencyController.java in ShopEase.

Completed-response replay

For a repeated key with the same payload, the backend can now return the previously completed result.

The original order ID is preserved, so a retry does not create a new order merely because the first HTTP response was lost.

SHA-256 payload fingerprinting

I added SHA-256 fingerprinting to detect when the same key is reused with a different checkout payload.

A mismatched payload is rejected with HTTP 422 Unprocessable Content.

This prevents a key originally associated with one checkout from silently being reused for another.

An explicit replay header

Replayed responses include:

Idempotency-Replayed: true

This makes it possible for the client to distinguish a replayed result from a newly executed request.

In-flight request protection

I added protection around the PROCESSING state to stop concurrent duplicate requests from independently starting the same checkout.

For a multi-instance deployment, the key claim must be coordinated through shared, atomic storage. A local in-memory lock alone is not enough when requests can reach different application instances.

How Should the Fix Be Tested?

The most important acceptance test is:

Submit a checkout request.

Allow the order transaction to commit.

Drop the HTTP response.

Retry using the original key and identical payload.

Confirm that the original order ID is returned.

Confirm that only one order was created and stock was decremented only once.

I would also test concurrent retries and reuse of the same key with a different payload.

The key lesson is that rejecting a duplicate request does not necessarily make the overall checkout experience reliable.

What Did I Learn?

This improvement reminded me that real backend development involves handling failure scenarios, not only successful API calls.

Idempotency needs to account for network failures, transaction commits, concurrent execution, and retries. The original request may have succeeded even when the client never received its response.

I'm glad I received this feedback because it helped me identify and improve a real edge case in ShopEase.

Tech stack: Java, Spring Boot, Spring Data JPA, MySQL, SQL, multithreading.

What strategies have you used to handle idempotency in concurrent checkout or payment systems?

Java #SpringBoot #BackendDevelopment #WebDev #ShopEase #ShitanshuJha

WalkQuest AI: Turning Screen Time into Outdoor Adventures 🌿

Shitanshu Jha — Sun, 11 Oct 2026 08:04:23 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
Hacktoberfest Open-Source AI Challenge — Week 1: Touch Grass

What if AI didn't try to keep you on your screen, but actually encouraged you to step away from it?

That question inspired WalkQuest AI, a small project built around a simple idea: turn a user's available time, interests, and mood into a short outdoor quest.

🌱 What is WalkQuest?

WalkQuest AI helps people take small breaks from screens and reconnect with the world around them. Instead of scrolling through another feed, users can generate a simple outdoor activity plan for walking, gardening, mindfulness, nature observation, or exploration.

The goal isn't to replace the outdoors with another app. It's to make the time spent using the app short, useful, and purposeful.

✨ Features

AI-generated quests: Describe what you feel like doing and how much time you have.
Multiple activity categories: Walking, gardening, mindfulness, nature, exploration, and observation.
Three-step quests: Break an outdoor break into three manageable activities.
Phone Pocket: A reminder to put the phone away and focus on the real world.
Quest timer and completion flow: Start an outdoor session, return, and reflect on the experience.
Progress tracking: Keep track of completed quests and related activity stats.

🧠 How I built it

WalkQuest uses Gemma 3 1B through Ollama for local AI inference. The application is built with Java and Spring Boot, with a browser-based frontend.

The backend sends a structured prompt to the local model and processes the generated quest into a title, duration, category, difficulty, and three activities.

I also worked on fallback behavior so that Hinglish requests can receive predictable, understandable activities when generated language quality is unreliable.

Tech stack

Java and Spring Boot
Gemma 3 1B
Ollama for local inference
HTML, CSS, and JavaScript
Maven

🎬 Demo

Watch the WalkQuest AI demo: Upload/embed the attached
Demo Video: https://youtu.be/PcGwOobycqA video here.

The demo shows the quest-generation flow and the app's outdoor-session experience.

🌍 Why open innovation matters

An app designed to help people spend less time on screens should not depend entirely on a remote AI service.

Using an open-weight model with local inference gives developers more control over how AI is run, how prompts are designed, and how the model fits into the application. Once the model has been downloaded, inference can run locally through Ollama, without sending each generation request to a hosted model API.

It also makes experimentation more accessible: developers can inspect the code, change the prompts, improve fallback behavior, and explore other compatible models instead of being locked into one closed service.

Local inference does not automatically make every part of an application offline, but it gives this project a foundation for more private and flexible AI experiences.

🛠️ Run it locally

Repository: https://github.com/Shitanshu686/walkquest

Requirements:

Java 17 or later
Git
Ollama installed
The gemma3:1b model downloaded

Clone the repository:

git clone https://github.com/Shitanshu686/walkquest.git
cd walkquest

Download the model and start Ollama:

ollama pull gemma3:1b
ollama serve

In another terminal, launch the application using the repository's run-walkquest.bat script on Windows, or use the Maven wrapper:

./mvnw spring-boot:run

Open the local address and port printed by the application.

🚀 What's next?

I'd like to improve multilingual generation quality, expand the available quest types, and make outdoor progress tracking more useful without adding unnecessary screen time.

WalkQuest is an experiment in building AI that helps people do something beyond interacting with AI itself.

Build with open AI. Step outside. Touch grass. 🌿

Prize categories: Best Use of Gemma

Tags: #devchallenge #hf26challenge #opensource #ai #java

🎬 WalkQuest AI — Live Demo

Watch the demo to see WalkQuest AI in action, including outdoor quest generation and the screen-free activity flow.
Demo Video: https://youtu.be/PcGwOobycqA

▶️ Watch WalkQuest AI Demo on YouTube

The demo showcases how WalkQuest turns a simple request into an outdoor activity experience.

Source Code: https://github.com/Shitanshu686/walkquest

Handling Concurrent Requests and Duplicate Checkouts in Spring Boot

Shitanshu Jha — Sat, 10 Oct 2026 05:52:34 +0000

While building ShopEase, my e-commerce backend project, I reached a stage where implementing CRUD endpoints was no longer enough.

I needed to think about what happens when multiple customers try to update the same inventory or submit the same checkout request simultaneously.

That led me to work on a new backend milestone focused on high-concurrency handling and idempotency protection.

Here are the key implementations and what I learned from them.

Atomic SQL Updates for Inventory

Consider a product with five units in stock. Multiple purchase requests may arrive at almost the same time.

Reading the stock in Java, subtracting a quantity, and saving the entity can introduce race conditions if competing requests read the same initial value.

I added a conditional atomic SQL update:

UPDATE products
SET stock = stock - :qty
WHERE id = :id
AND stock >= :qty;

The database performs the stock check and deduction as one statement.

The affected-row count indicates whether the update succeeded, allowing the application to handle insufficient stock.

This approach reduces unnecessary application-side read-modify-write operations. It does not mean the database performs the operation without any internal locking.

Pessimistic vs. Optimistic Locking

I worked with both locking strategies in DatabaseConcurrencyService.

Pessimistic locking

Pessimistic locking requests a write lock on a database record while the transaction performs its work.

With JPA, this can be implemented using LockModeType.PESSIMISTIC_WRITE.

It is useful when conflicting updates are likely, although excessive lock contention can reduce throughput.

Optimistic locking

Optimistic locking detects conflicting changes rather than preventing every competing transaction from proceeding.

I added a JPA version field:

@version
private Long version;

Hibernate uses the version to detect stale updates. A conflicting update can fail and require the application to handle or retry the operation.

My takeaway: Both strategies are useful, but they address concurrency differently. The choice depends on the workload and conflict rate.

Spring Transaction Proxies and Executor Threads

One important detail I worked on was self-proxy injection.

Spring typically applies @Transactional behavior through a proxy. A direct call such as this.someTransactionalMethod() can bypass the proxy and therefore bypass the expected transaction interception.

Calling the method through the injected Spring proxy allows the transaction interceptor to run.

There is another important detail: an executor thread does not automatically inherit the caller's transaction. A transactional method invoked through the proxy needs to establish its own transaction as required.

This was a useful reminder that asynchronous execution and transaction boundaries must be considered together.

Idempotency Protection for Checkout

Checkout requests can be repeated because of network issues, client retries, or accidental double-clicks.

I implemented IdempotencyKey validation to detect duplicate checkout requests and return 409 Conflict for duplicates.

The basic flow is:

Receive a checkout request with an idempotency key.

Validate whether the key has already been used.

Process a new request.

Reject a duplicate request.

For reliable production behavior, checking and recording the key must be atomic. A database uniqueness constraint is one common way to enforce this. The design also needs clear rules for retries, failed requests, and payment side effects.

The goal is to avoid processing the same logical checkout operation more than once.

JPA Versioning and Application Configuration

I added the version column required for optimistic locking and resolved a Spring bean collision.

These changes help detect stale database updates and allow the application context to initialize with the intended bean configuration.

Key Takeaways

This milestone helped me understand that reliable backend development involves more than writing endpoints.

Use atomic database operations for conditional inventory updates.

Choose pessimistic or optimistic locking according to the workload.

Understand how Spring proxies affect transaction management.

Design executor-thread work with explicit transaction boundaries.

Protect checkout side effects against duplicate requests.

Use JPA versioning to detect conflicting updates.

Final Thoughts

Working on ShopEase is helping me connect Java, Spring Boot, database concurrency, and real e-commerce requirements.

This milestone is a step toward making inventory management and checkout processing more reliable under concurrent load.

I'm continuing to build and improve ShopEase one backend milestone at a time.

Tech stack: Java, Spring Boot, Spring Data JPA, Hibernate, MySQL

Java #SpringBoot #BackendDevelopment #MySQL

ShopEase — Module 32: Solving Inventory Overselling with Concurrency Control

Shitanshu Jha — Mon, 05 Oct 2026 04:21:33 +0000

In an e-commerce application, multiple users can try to purchase the same product at exactly the same time.
This creates a common backend problem: Race Conditions.
The Problem
Suppose a product has only 10 items in stock.
Now imagine 50 users place an order simultaneously.
Without proper concurrency handling, multiple threads may read the same stock value before any of them updates it:
Stock = 10
→ Thread 1 reads 10
→ Thread 2 reads 10
→ Thread 3 reads 10
→ ...
→ Multiple threads try to purchase
The result can be incorrect stock updates and overselling, where the system accepts more orders than the available inventory.
This is a serious problem for an e-commerce backend.
The Solution
In ShopEase, I implemented Concurrency, Multithreading & Database Locking to handle these situations safely.
The inventory update process now uses:

Multithreading for concurrent request testing
Optimistic Locking using @version
Pessimistic Locking using @lock(LockModeType.PESSIMISTIC_WRITE)
Database row locking with FOR UPDATE
Atomic stock deduction
Proper transaction handling With pessimistic locking, when one transaction is updating a product's stock, other transactions have to wait instead of modifying the same row simultaneously. 50-Thread Stress Testing To verify the implementation, I performed concurrent testing with 50 threads trying to purchase the same product. The test focused on: 50 concurrent requests → Stock validation → Locking → Atomic deduction → Order result The important result was: No negative stock. No overselling. Concurrent requests handled safely. What This Module Solved The main problem solved in this module was protecting inventory from race conditions during simultaneous purchases. This makes the ShopEase backend more reliable when multiple users interact with the same product at the same time. Key Takeaway Concurrency isn't just about running multiple threads. In a real e-commerce system, it is about making sure that multiple operations happening simultaneously still produce a correct and consistent result. Tech Stack: Java • Spring Boot • Spring Data JPA • Hibernate • MariaDB • Multithreading • Optimistic Locking • Pessimistic Locking #Java #SpringBoot #Concurrency #Multithreading #DatabaseLocking #Hibernate #MariaDB #BackendDevelopment #ShopEase