Architecting a High-Concurrency Railway Booking Platform with Node.js, Apache Kafka, and Redis Distributed Locks

By Ramanand Tomar — Software Engineer & Backend Systems Specialist

---

Handling massive bursts of concurrent requests during narrow reservation windows is one of the classic challenges in distributed system design. When thousands of transactions attempt to reserve the final remaining berth on a high-demand route simultaneously, standard relational locking mechanisms fail under contention.

In engineering the backend for IRCTC RailYatra, my objective was to implement an enterprise-grade, event-driven microservices architecture capable of addressing core distributed systems challenges:

1. High-concurrency seat contention and race condition mitigation.
2. Distributed data consistency across boundaries without blocking two-phase commits (2PC).
3. Fault isolation and recovery using Dead-Letter Queues (DLQ) and circuit breakers.
4. Sub-50 millisecond inverted-index search across multi-stop schedules.

Here is an analysis of the architectural patterns, trade-offs, and implementation details.

---

1. System Topology and Domain Boundaries

The platform strictly implements the Database-per-Service pattern. Each microservice encapsulates its domain models, persistence store, and business rules, communicating asynchronously over an event bus or via the edge gateway.

graph TD
    Client[Web and Mobile Clients] -->|HTTPS / REST| APIGW[API Gateway: NGINX / Express]

    subgraph Edge Layer
        APIGW -->|JWT Verification & Rate Limiting| AuthModule[JWT Auth Middleware]
        APIGW -->|Catalog Lookups| SearchService[Search Service]
        APIGW -->|User Domain| UserService[User Service]
        APIGW -->|Reservation Ingestion| BookingService[Booking Service]
        APIGW -->|Payment Processing| PaymentService[Payment Service]
        APIGW -->|Route & Train Master| AdminService[Admin Service]
    end

    subgraph Data Stores
        BookingService -->|Relational State| BookingDB[(PostgreSQL: Booking DB)]
        PaymentService -->|Ledger Records| PaymentDB[(PostgreSQL: Payment DB)]
        InventoryService -->|Berths & Availability| InventoryDB[(PostgreSQL: Inventory DB)]
        AdminService -->|Master Tables| AdminDB[(PostgreSQL: Admin DB)]
        UserService -->|Auth & Profiles| UserDB[(PostgreSQL: User DB)]
        SearchService -->|Denormalized Routes| Elastic[(Elasticsearch)]
        BookingService -->|Atomic Locks & State| RedisCache[(Redis Cluster)]
    end

    subgraph Event Broker: Apache Kafka
        BookingService -->|Event: booking.created| KafkaTopic[Apache Kafka Event Bus]
        PaymentService -->|Event: payment.success / payment.failed| KafkaTopic
        KafkaTopic -->|Consume Events| InventoryService[Inventory Service]
        KafkaTopic -->|Consume Events| NotificationService[Notification Service]
        KafkaTopic -->|Poison Pill Messages| DLQ[Kafka Dead-Letter Queue - DLQ]
    end

Architectural Principles:

• API Gateway: Single reverse proxy terminating SSL, executing JWT validation, enforcing sliding-window rate limits, and implementing circuit breakers to prevent cascading failures.
• Isolated Relational Stores: Each service maintains its own PostgreSQL instance. Cross-boundary table joins are prohibited at the schema level.
• Distributed Message Log (Apache Kafka): Decouples time-sensitive write paths from asynchronous side effects (inventory updates, PDF generation, transactional email dispatch).
• In-Memory Locking Layer (Redis): Offloads volatile contention locks from the primary transactional database.
• Search Acceleration (Elasticsearch): Houses denormalized route schedules to eliminate multi-table SQL joins during catalog traversal.

---

2. Concurrency Control: Redis Distributed Locks

Relying on database-level row locking (SELECT ... FOR UPDATE) during high-traffic bursts leads to connection pool saturation, extended lock wait times, and deadlocks.

To resolve this, seat reservation state is temporarily decoupled from the primary database using atomic locks in Redis.

sequenceDiagram
    autonumber
    actor Client as Passenger Client
    participant GW as API Gateway
    participant BS as Booking Service
    participant Redis as Redis Cluster
    participant DB as Booking PostgreSQL
    participant Kafka as Apache Kafka

    Client->>GW: POST /api/bookings/reserve (trainId, seatNo, journeyDate)
    GW->>BS: Validate Token & Forward Request
    BS->>Redis: SET lock:seat:{trainId}:{journeyDate}:{seatNo} {userId} NX EX 600
    alt Lock Contention (Seat Already Reserved)
        Redis-->>BS: nil (Key exists)
        BS-->>Client: 409 Conflict: Seat currently held by another transaction.
    else Lock Acquired (Successful Reservation)
        Redis-->>BS: OK (Exclusive 10-minute hold active)
        BS->>DB: INSERT INTO bookings (status = 'PENDING_PAYMENT')
        BS->>Kafka: Publish `booking.initiated`
        BS-->>Client: 201 Created: Reservation hold active. 10 minutes to complete checkout.
    end

Atomic Lock Implementation with Lua Scripting

To avoid race conditions where a process releases another passenger's newly acquired lock following a timeout, unlock operations must verify ownership atomically:

import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

export async function acquireSeatLock(
  trainId: string, 
  seatId: string, 
  journeyDate: string, 
  userId: string,
  holdDurationSeconds = 600 // 10 minutes TTL
): Promise<boolean> {
  const lockKey = `lock:seat:${trainId}:${journeyDate}:${seatId}`;

  // Set if Not eXists (NX) with Expiration Time (EX)
  const status = await redis.set(lockKey, userId, 'NX', 'EX', holdDurationSeconds);
  return status === 'OK';
}

export async function releaseSeatLock(
  trainId: string, 
  seatId: string, 
  journeyDate: string, 
  userId: string
): Promise<boolean> {
  const lockKey = `lock:seat:${trainId}:${journeyDate}:${seatId}`;

  // Lua script ensures atomicity: compare owner before deletion
  const luaScript = `
    if redis.call("get", KEYS[1]) == ARGV[1] then
      return redis.call("del", KEYS[1])
    else
      return 0
    end
  `;

  const result = await redis.eval(luaScript, 1, lockKey, userId);
  return result === 1;
}

---

3. Distributed Transactions: Choreographed Saga Pattern

Because microservices prohibit distributed ACID transactions across multiple databases, the system applies the Saga Pattern over Apache Kafka to maintain eventual consistency.

sequenceDiagram
    autonumber
    participant Booking as Booking Service
    participant Kafka as Kafka Event Broker
    participant Payment as Payment Service
    participant Inventory as Inventory Service
    participant Notif as Notification Service

    Note over Booking: Phase 1: Reservation Initiated
    Booking->>Kafka: Emit `booking.created`

    Kafka->>Payment: Consume `booking.created`
    Note over Payment: Phase 2: Gateway Execution
    alt Payment Successful
        Payment->>Kafka: Emit `payment.succeeded`
        Kafka->>Inventory: Consume `payment.succeeded`
        Note over Inventory: Phase 3: Final Inventory Decrement
        Inventory->>Kafka: Emit `inventory.confirmed`
        Kafka->>Booking: Transition Status: CONFIRMED
        Kafka->>Notif: Dispatch Confirmation Payload (SendGrid)
    else Payment Declined / Gateway Timeout
        Payment->>Kafka: Emit `payment.failed`
        Note over Booking,Inventory: Compensating Transactions Executed
        Kafka->>Booking: Consume `payment.failed` -> Mark CANCELLED
        Kafka->>Inventory: Consume `payment.failed` -> Release Berth Quota
        Booking->>Booking: Evict Redis Lock Key
    end

Dead-Letter Queues (DLQ) for Poison Pill Handling

When a consumer encounters an unrecoverable deserialization fault or database timeout:

1. The consumer retries up to three times with exponential backoff.
2. If failures persist, the message is routed to booking.events.dlq alongside exception headers.
3. Partition consumption continues without blocking the offset lag of healthy messages.

---

4. Search Acceleration via Inverted Indexing

Relational tables normalized across trains, intermediate stops, and schedules require expensive joins:
$$\text{Trains} \bowtie \text{Schedules} \bowtie \text{Stations} \bowtie \text{Quotas}$$

During high traffic, read queries against SQL degrade throughput. To optimize this, the Search Service denormalizes train routes into an Elasticsearch index:

{
  "trainNumber": "12002",
  "trainName": "Bhopal Shatabdi Express",
  "sourceStation": "NDLS",
  "destinationStation": "RKMP",
  "stops": [
    { "stationCode": "NDLS", "stationName": "New Delhi", "stopOrder": 1, "departure": "06:00" },
    { "stationCode": "MTJ",  "stationName": "Mathura Jn", "stopOrder": 2, "departure": "07:20" },
    { "stationCode": "AGC",  "stationName": "Agra Cantt",  "stopOrder": 3, "departure": "07:55" },
    { "stationCode": "GWL",  "stationName": "Gwalior Jn",  "stopOrder": 4, "departure": "09:23" }
  ]
}

A nested query checks that both the boarding station and debarkation station exist in stops, verifying that stopOrder(Destination) > stopOrder(Origin). This query completes in under 20 milliseconds, scaling independently of transactional database load.

---

5. Architectural Trade-Offs & Takeaways

1. Eventual Consistency vs. Immediate Consistency: Utilizing Saga choreography requires designing consumer logic around idempotency (idempotency_key stored in Redis) to safely handle at-least-once message delivery semantics.
2. Operational Complexity: Managing separate PostgreSQL instances, Kafka clusters, and Redis instances demands robust container orchestration and centralized structured logging (ELK / Prometheus).
3. Decoupled Failure Domains: An outage in the Notification Service or Payment Gateway does not compromise catalog availability or schedule lookups.

---

Author Information

Ramanand Tomar is a Software Engineer specializing in backend distributed architectures, network protocol engineering, and AI systems.

• Portfolio: ramanand-tomar.vercel.app
• GitHub: github.com/Ramanand-tomar
• LinkedIn: linkedin.com/in/ramanand-tomar-478528342