Building a Zero-Loss Inventory Engine: Java + Go Hybrid Microservices in Production

The Problem

When you're building a supply-chain SaaS that handles procurement, sales, inventory, POS, and finance across 12+ microservices, inventory consistency isn't a nice-to-have — it's the difference between a working product and a lawsuit.

The challenge: multiple concurrent requests trying to deduct the same stock item simultaneously. Traditional approaches (database pessimistic locks, simple Redis locks) either kill throughput or leave race condition gaps that cause overselling.

I needed a system that could handle high-concurrency stock operations with zero data loss — not "eventually consistent," not "mostly correct," but zero.

The Architecture: Four-Layer Concurrency Defense

I designed a hybrid Java + Go approach where each layer catches what the previous one misses:

Request → [1] Redis Lua Pre-deduct → [2] DB CAS Optimistic Lock → [3] Go Mutex → [4] Auto Compensation

Layer 1: Redis Lua Atomic Pre-deduction

The first line of defense is a Go microservice that handles stock pre-deduction via Redis Lua scripts. Why Go? Because the inventory service needs to handle thousands of concurrent stock operations per second, and Go's goroutine model is ideal for this workload.

// Simplified: Redis Lua script for atomic stock deduction
const deductScript = `
  local current = tonumber(redis.call('GET', KEYS[1]))
  if current == nil then return -1 end
  local qty = tonumber(ARGV[1])
  if current < qty then return 0 end
  redis.call('SET', KEYS[1], current - qty)
  return 1
`

The key insight: Lua scripts execute atomically in Redis. No other operation can interleave between the check and the deduction. This eliminates the race condition at the cache layer.

Why not just use Redis and call it a day? Because Redis is volatile. If Redis crashes between pre-deduction and the database commit, you've lost the deduction. You need the database as the source of truth.

Layer 2: Database CAS Optimistic Lock

After Redis pre-deducts, the Java service commits to MySQL using Compare-And-Swap:

UPDATE inventory 
SET available_qty = available_qty - #{qty}, version = version + 1
WHERE sku_id = #{skuId} AND available_qty >= #{qty} AND version = #{version}

If the version doesn't match (another transaction committed first), the update affects 0 rows and we retry. This is standard optimistic locking, but it's the critical second layer — if Redis crashes, the database still prevents overselling.

Layer 3: Go Mutex for In-Process Serialization

Within the Go service itself, a mutex ensures that concurrent requests for the same SKU are serialized within a single process. This reduces the number of CAS retries at the database layer:

type StockLock struct {
    mu sync.Map // key: skuID, value: *sync.Mutex
}

func (l *StockLock) Lock(skuID string) {
    mu, _ := l.mu.LoadOrStore(skuID, &sync.Mutex{})
    mu.(*sync.Mutex).Lock()
}

Layer 4: Automatic Compensation

If something fails after pre-deduction (e.g., order creation fails, payment fails), the system automatically compensates by reversing the stock deduction. This is powered by a Saga compensation pattern with a transactional outbox:

@Transactional
public void deductStock(Long skuId, Integer qty) {
    // 1. Redis pre-deduct (via Go service)
    redisInventoryService.preDeduct(skuId, qty);

    // 2. DB CAS update
    int updated = inventoryMapper.casDeduct(skuId, qty, currentVersion);
    if (updated == 0) throw new OptimisticLockException();

    // 3. Record for compensation
    outboxMessageRepository.save(
        StockCompensationMessage.of(skuId, qty, CompensationType.REVERSE)
    );

    // 4. afterCommit dispatch
    transactionTemplate.executeAfterCommit(() -> 
        rabbitTemplate.convertAndSend("stock.events", message)
    );
}

If the transaction rolls back, the outbox message is never dispatched — but neither is the DB deduction. If the transaction commits but downstream processing fails, the dead letter queue catches it and compensation kicks in.

Why the Java + Go Split?

You might wonder: why not do this entirely in Java or entirely in Go?

Go for the inventory service: Go's goroutines handle 10K+ concurrent stock operations with minimal memory overhead. A single Go service can saturate Redis's throughput without being the bottleneck. The custom Nacos HTTP client in Go also means the service is self-contained — no JVM dependency for a performance-critical path.

Java for business logic: The rest of the platform (12 services, 648 APIs) runs on Spring Boot 3.2 with Java 21 virtual threads. Java's ecosystem — Spring Cloud, MyBatis-Plus, RabbitMQ integration — is unmatched for complex business logic. The Java services call the Go inventory service through Resilience4j circuit breaker (50% failure threshold, 2s slow-call detection), so if the Go service is degraded, the system fails gracefully.

Feature toggles for canary switching: The architecture supports zero-downtime switching between the Go implementation and a Java fallback. In production, we can route traffic to either implementation without redeployment — invaluable for testing and rollback.

The Results

In production across African markets (Zambia), this architecture has handled:

• Zero data loss under high-concurrency flash-sale scenarios
• Automatic recovery from Redis failures (DB CAS catches it)
• Graceful degradation via circuit breaker when the Go service is slow
• Full audit trail via custom distributed tracing (AspectJ LTW)

Key Takeaways

1. Defense in depth works. No single mechanism is perfect, but four layers complement each other — Redis for speed, DB for durability, mutex for efficiency, compensation for safety.

2. Choose the right tool for the job. Go for performance-critical paths, Java for complex business logic. Don't be religious about language choice.

3. Feature toggles > redeployment. Being able to switch between implementations without downtime is worth the extra abstraction cost.

4. Transactional outbox is non-negotiable. If you're doing event-driven architecture, the outbox pattern ensures your events and your database state are always consistent.

---

The full source code is open source: github.com/moyuping-java-architect/inventory-pos-microsystem

I'm a software architect with 13+ years of experience, currently open to remote positions in Europe. Feel free to reach out at [e-mail] or connect on GitHub.

---

What's your approach to inventory consistency? Have you used a similar multi-layer defense, or do you prefer a different pattern? Let me know in the comments!